Skip to content

Raise DecodeError for invalid UTF-8 TOML buffers - #1270

Open
xiehuanyi wants to merge 2 commits into
msgspec:mainfrom
xiehuanyi:fix/toml-invalid-utf8-decodeerror-20261013
Open

xiehuanyi wants to merge 2 commits into
msgspec:mainfrom
xiehuanyi:fix/toml-invalid-utf8-decodeerror-20261013

Conversation

@xiehuanyi

Copy link
Copy Markdown
Contributor

msgspec.toml.decode(b"name = '\xff'") currently raises a bare UnicodeDecodeError. Applications catching msgspec.DecodeError for malformed TOML do not catch it, even though syntax failures are already wrapped as DecodeError.

Keep UTF-8 conversion inside the parse-error boundary and wrap UnicodeDecodeError using the TOML wrapper's existing error convention. Buffer-type errors still propagate as TypeError. Regressions cover bytes, bytearray, and memoryview, with valid non-ASCII TOML controls.

Validation: all three malformed UTF-8 cases fail before the fix; the complete unit suite passes afterward (6553 passed, 116 existing skips). Ruff 0.17.0 lint and format checks pass for both changed files.

Copilot AI balanced review requested due to automatic review settings October 13, 2026 18:24

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants