Skip to content

Conversation

@xerial
Copy link
Member

@xerial xerial commented May 23, 2014

Implemented MessageUnpacker whose performance is faster than v06. You can run benchmark tests that compare the performance of v06 and v07:

$ ./sbt test

benchmark code: https://github.com/msgpack/msgpack-java/blob/v07-unpacker/msgpack-core/src/test/scala/org/msgpack/core/MessageUnpackerTest.scala

When reading 10,000 message-packed records (including arrays, maps, strings, etc.), skipping records is 4 times faster than v6, and reading values is about 2 times faster:
1__leo_weaver_local___work_git_msgpack-java__java_

Summary of the changes:

  • To simplify the implementation MessageUnpacker.unpackXXX methods block until the next input buffer is available. Supporting event-driven I/O Supporting event-driven I/O in unpackXXX #92 is not included in this change.
  • At the boundary of MessageBuffers, the unpacker copies the contents of the current buffer and the next buffers into an extra buffer so that readXXX can read its required length. (see ensure(n) in MessageUnpacker)
  • Added README file
  • skipValue() uses switch-case statements instead of polymorphic function calls in MessageFormat because using switch statement is slightly faster. The skip length table Using skip length table in MessageUnpacker #94 is not yet implemented because this part is already fast enough.
  • Implemented packString and unpackString that use pre-allocated buffers to encode/decode UTF-8 strings.
  • Added MessagePack.Config object to specify the behaviour of packer and unpacker. For example, options for handling malformed UTF-8 strings, making RAW/STRING headers compatible, string coder buffer size, etc.
  • Added property-based tests that check the behaviour of packer/unpacker for randomly generated inputs. (See MessagePackTest.scala)
  • Added benchmark tests. It runs a code block 10 times (for warm-up), then measures the performance.
  • Fixed several pack/unpack bugs when reading array/map/extended type headers.
  • Enabled code coverage report via ./sbt jacoco:cover http://www.eclemma.org/jacoco/
  • Enabled findbugs report via ./sbt findbugs http://findbugs.sourceforge.net/

TODO

  • Returning rawString references in unpacker (requires dedicated memory management)
  • Event-driven I/O for implementing RPC service Supporting event-driven I/O in unpackXXX #92
  • Value/Cursor for traversing message-packed values.

xerial added 30 commits May 20, 2014 15:33
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain why you use the value bb.position() - bb.remaining()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's my mistake. This should be bb.limit() - bb.remaining(). Thanks.

@komamitsu
Copy link
Member

👍

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we check whether obj is null or not?

    if (other == null) {
      return false;
    } else if(obj instanceof ExtendedTypeHeader) { 

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

null instanceof xxx returns false, so we do not need null check here:
http://stackoverflow.com/questions/2950319/is-null-check-needed-before-calling-instanceof

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the explanation! 👍

xerial added a commit to xerial/msgpack-java that referenced this pull request Jun 3, 2014
xerial added a commit to xerial/msgpack-java that referenced this pull request Jun 3, 2014
xerial added a commit to xerial/msgpack-java that referenced this pull request Jun 3, 2014
xerial added a commit to xerial/msgpack-java that referenced this pull request Jun 3, 2014
xerial added a commit to xerial/msgpack-java that referenced this pull request Jun 3, 2014
xerial added a commit to xerial/msgpack-java that referenced this pull request Jun 3, 2014
xerial added a commit that referenced this pull request Jun 4, 2014
@xerial xerial merged commit 8f5f487 into v07 Jun 4, 2014
@xerial
Copy link
Member Author

xerial commented Jun 4, 2014

Now closes this ticket. Thanks for the review!

@kzk
Copy link
Member

kzk commented Jun 6, 2014

Epic 100th issue! 👍

@oza
Copy link
Member

oza commented Jun 7, 2014

Thanks for your great work, @xerial!

@muga
Copy link
Member

muga commented Jun 17, 2014

Good job 👍 @xerial

@xerial xerial deleted the v07-unpacker branch July 23, 2014 06:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants