Skip to content

Use PyObject_GetItem when converting mappings with from_attributes=False - #1302

Open
princejain756 wants to merge 2 commits into
msgspec:mainfrom
princejain756:fix-convert-mapping-method-names
Open

princejain756 wants to merge 2 commits into
msgspec:mainfrom
princejain756:fix-convert-mapping-method-names

Conversation

@princejain756

Copy link
Copy Markdown

Closes #1246.

When converting a non-dict mapping to a Struct, tagged Struct union, or dataclass with from_attributes=False, convert_other previously set getter = getattr_then_getitem (introduced in 71c938c). Because getattr_then_getitem tries PyObject_GetAttr first, any field sharing a name with an attribute or method on the mapping (keys, items, values, get, etc.) resolved to the bound method instead of looking up the item via __getitem__.

This sets getter = PyObject_GetItem when self->from_attributes is false, while preserving getattr_then_getitem for mappings when self->from_attributes is true.

Also removes the accidental (GetAttrOrItemObj, False, False) entry from mapcls_from_attributes_and_array_like (since GetAttrOrItemObj stores part of its fields only as attributes) and adds regression tests for Struct, tagged Struct unions, and dataclass targets.

Closes msgspec#1246.

In convert_other, when self->from_attributes is false and the input is a
mapping, field lookups previously used getattr_then_getitem instead of
PyObject_GetItem. Any Struct or dataclass field sharing a name with a
mapping method (such as keys, items, values, or get) resolved to the bound
method rather than calling __getitem__.

Use PyObject_GetItem when self->from_attributes is false, keeping
getattr_then_getitem only when self->from_attributes is true.
sobolevn
sobolevn previously approved these changes Oct 22, 2026

@sobolevn sobolevn left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, but I would like a to have second review :)

@provinzkraut

provinzkraut commented Oct 22, 2026 •

Copy link
Copy Markdown
Member

Haven't looked at it too much yet, but I don't think we're testing this at the moment so: How does this behave with a @property that shadows a mapping method? I would expect this to have no effect but.. I'm also not 100% certain.

@princejain756

princejain756 commented Oct 22, 2026 •

Copy link
Copy Markdown
Author

With PyObject_GetItem attributes are never consulted, so a @property named keys/get/items on a Mapping subclass has no effect - the field is read via obj["get"] only, and if the mapping does not store that key it errors as a missing required field rather than picking up the property value. That is the point of the change: mapping-protocol objects get read through the mapping interface.

test_dataclass_mapping_ignores_attributes covers the dict-subclass version of this; happy to add an explicit @property case if you want it pinned.

@provinzkraut

Copy link
Copy Markdown
Member

test_dataclass_mapping_ignores_attributes covers the dict-subclass version of this; happy to add an explicit @property case if you want it pinned.

I think a test case would be good. We're dealing with oddities here, so adding more tests for another oddity, even if just to prevent someone asking that question again, seems reasonable :)

@princejain756

Copy link
Copy Markdown
Author

Added the @property shadowing cases in test_dataclass_mapping_ignores_attributes and test_struct_mapping_ignores_attributes.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

convert: mapping fields named like mapping methods (keys, get, ...) resolve to the bound method

3 participants