Skip to content

Support msgpack 1.0.0#156

Closed
pcdinh wants to merge 2 commits intotarantool:masterfrom
pcdinh:patch-strict_map_key
Closed

Support msgpack 1.0.0#156
pcdinh wants to merge 2 commits intotarantool:masterfrom
pcdinh:patch-strict_map_key

Conversation

@pcdinh
Copy link

@pcdinh pcdinh commented Mar 12, 2020

In msgpack 1.0.0, the default value of strict_map_key is changed to True to avoid hashdos. However, it will break the Tarantool driver, causing an exception:

ValueError: int is not allowed for map key

As a result, the driver is unable to parse server response. The issue is reported at #155

…sgpack 1.0 makes the driver unable to parse server response
@Totktonada
Copy link
Contributor

Totktonada commented Mar 12, 2020

In CI:

TypeError: __init__() got an unexpected keyword argument 'strict_map_key'

It seems we should support both old and new implementations.

@Totktonada Totktonada linked an issue Jul 10, 2020 that may be closed by this pull request
Copy link
Contributor

@Totktonada Totktonada left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please, squash commits and explain changes in the commit message (just like you did for the PR description). Don't forget to include Fixes #155 or Closes #155 tag.

See also the comment below.

Comment on lines +57 to +70
if msgpack.version >= (1, 0, 0):
unpacker = msgpack.Unpacker(use_list=True, raw=False, strict_map_key=False)
else:
unpacker = msgpack.Unpacker(use_list=True, raw=False)
elif conn.encoding is not None:
unpacker = msgpack.Unpacker(use_list=True, encoding=conn.encoding)
if msgpack.version >= (1, 0, 0):
unpacker = msgpack.Unpacker(use_list=True, encoding=conn.encoding, strict_map_key=False)
else:
unpacker = msgpack.Unpacker(use_list=True, encoding=conn.encoding)
else:
unpacker = msgpack.Unpacker(use_list=True)
if msgpack.version >= (1, 0, 0):
unpacker = msgpack.Unpacker(use_list=True, strict_map_key=False)
else:
unpacker = msgpack.Unpacker(use_list=True)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can reduce code duplication by unpacking a dictionary to keyword parameters. A kind of this (not tested):

kwargs = dict(use_list=True)
if msgpack.version >= (1, 0, 0):
    # XXX: Explain why it is necessary.
    kwargs['strict_map_key'] = False
if msgpack.version >= (0, 5, 2) and conn.encoding == 'utf-8':
    # Get rid of the following warning.
    # > PendingDeprecationWarning: encoding is deprecated,
    # > Use raw=False instead.
    kwargs['raw'] = False
elif conn.encoding is not None:
    kwargs['encoding'] = conn.encoding

unpacker = msgpack.Unpacker(**kwargs)

@Totktonada Totktonada changed the title Set strict_map_key to False, making the driver work with msgpack 1.0 Support msgpack 1.0.0 Jul 10, 2020
@Totktonada
Copy link
Contributor

From python-msgpack 1.0.0 release notes:

  • Remove encoding option from the Packer and Unpacker.

Are'we affected by this?

@Totktonada Totktonada mentioned this pull request Aug 22, 2020
@Totktonada
Copy link
Contributor

From python-msgpack 1.0.0 release notes:

  • Remove encoding option from the Packer and Unpacker.

Are'we affected by this?

In fact, no:

I dived a bit further and it reminds me changes we did for msgpack-0.5.2.

We already use raw=False when encoding is 'utf-8' since msgpack-0.5.2 (2097884), because otherwise it emits the deprecation warning regarding encoding option usage. So we should just continue passing raw=False on msgpack-1.0.0.

@Totktonada Totktonada mentioned this pull request Aug 26, 2020
@Totktonada
Copy link
Contributor

Closed in favor of PR #173.

@Totktonada Totktonada closed this Aug 29, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

New python-msgpack version brokes connector

2 participants