Repository navigation
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Non-dict mappings were converted with getattr before __getitem__, so fields named keys, items, values, or get received bound methods. from_attributes=True still prefers attributes. Fixes msgspec#1246.
3850421 to
272b924
Compare
cig13zs
left a comment
There was a problem hiding this comment.
Tested this (272b924) on Python 3.11 against main 75e1341. On main, #1246 reproduces for UserDict and sqlite3.Row (a keys column gives "Expected int, got builtin_function_or_method"). With this branch those convert correctly, and so do attrs targets, tagged struct unions, and fields named pop, setdefault, update, clear, copy, fromkeys. pytest tests/unit passes (6347 passed, 324 skipped).
One regression though: anything that has __getitem__ but exposes fields only as attributes no longer converts without from_attributes=True. SQLAlchemy 2.x Row is the common case, since row["id"] raises and the columns are attributes:
import sqlalchemy as sa, msgspec
class U(msgspec.Struct):
id: int
name: str
with sa.create_engine("sqlite://").connect() as c:
row = c.execute(sa.text("select 1 as id, 'x' as name")).first()
msgspec.convert(row, U)main: U(id=1, name='x'). This branch: ValidationError: Object missing required field `id` (SQLAlchemy 2.1.4). Same for a plain class with __getitem__ plus attributes. The PR also drops the (GetAttrOrItemObj, False, False) row from the test table in test_convert.py, which is the test that covered this case.
Trying the item first and falling back to the attribute keeps both working:
static PyObject *
getitem_then_getattr(PyObject *obj, PyObject *key) {
PyObject *out = PyObject_GetItem(obj, key);
if (out == NULL) {
PyErr_Clear();
out = PyObject_GetAttr(obj, key);
}
return out;
}with getter = getitem_then_getattr; in the from_attributes=False branch. With that, the unit suite still passes (6347 passed), including your new tests, the #1246 cases above convert, and the SQLAlchemy row converts again. The one thing it doesn't cover is a mapping that lacks a keys key while the target has an optional keys field: that still picks up the bound method, but that is what main does today. Maybe worth a test with an attribute-only __getitem__ object either way.
Objects with __getitem__ that expose fields only as attributes (for example SQLAlchemy Row) converted on main but failed after the previous commit. Try the item first, then the attribute. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Thanks for testing this so thoroughly, @cig13zs. You're right, dropping the attribute fallback broke objects like SQLAlchemy's Pushed 13022db:
Agreed on the remaining case (a mapping without a |
|
Rechecked 13022db on Python 3.11 with SQLAlchemy 2.1.5. The |
Fixes #1246
What changed
msgspec.convertreads fields from a non-dict mapping with__getitem__first whenfrom_attributesis false, and falls back to the attribute if the item lookup fails. Struct, dataclass, attrs, and tagged struct union targets share that lookup. A field namedkeys,items,values, orgetreceives the value stored under that key, and objects that expose fields only as attributes (such as SQLAlchemyRow) still convert as on main.from_attributes=Truestill checks the attribute first, then the item.Tests
tests/unit/test_convert.pycovers those four targets. Withfrom_attributes=True, conversion still raises ValidationError because the field value is a method.test_getitem_raises_falls_back_to_attributescovers an object whose__getitem__raises while its fields are attributes, and the(GetAttrOrItemObj, False, False)case stays in the shared parametrize table.uv run --only-dev just rebuild=1 test-allreported6568 passed, 116 skippedfor the unit suite and7 passedfor doctests.uv run --only-dev just checkpassed.