Skip to content

Reject JSON input truncated at an embedded NUL - #245

Merged
lelit merged 1 commit into
python-rapidjson:masterfrom
00200200:codex/reject-null-truncated-json
Oct 12, 2026
Merged

lelit merged 1 commit into
python-rapidjson:masterfrom
00200200:codex/reject-null-truncated-json

Conversation

@00200200

Copy link
Copy Markdown
Contributor

loads('123\0garbage'), load() and Decoder() currently return 123, silently ignoring the NUL and everything after it. Refs #241, specifically the reported jts-8259-n_multidigit_number_then_00 compliance failure (123\0).

RapidJSON uses NUL as its end-of-input sentinel. After an otherwise successful parse, check the actual input length or the stream wrapper's EOF state. Report remaining input through the existing JSONDecodeError path, retaining the byte offset and the standard document-root error message. Escaped NULs in JSON strings remain valid.

Regression tests cover all six decoding routes, scalar/container documents, UTF-8 byte offsets, bytes/bytearray inputs, and stream chunk boundaries. This change does not alter the parse-flag dispatch addressed by #243.

Validation on macOS with CPython 3.11.5, using a locally built C++ extension:

  • Before the fix: the 92 new cases produced 88 failures and 4 passes.
  • After the fix: all 92 new cases pass. Full pytest tests -o addopts='' -q: 1029 passed, 17 skipped, 2 xfailed. The unchanged base passes its original suite: 937 passed, 17 skipped, 2 xfailed.
  • python -m mypy.stubtest rapidjson, Python compilation of the modified test files, and git diff --check pass. The repository has no configured lint command.
  • The prescribed Sphinx 8.2.3 doctest check runs 269 examples with one failure at docs/loads.rst:114, where the native-mode large integer is represented as 1.2345678901234568e+29 instead of 1.2345678901234566e+29. A separately built, unchanged checkout at 7337070f5bff88c8dc9803eded753f714c6fc1c0 reproduces the identical failure. Both runs also report the existing missing _static directory warning.

AI disclosure: OpenAI Codex generated and locally validated this patch and its tests. No separate human code review has been performed. Hosted CI, other Python versions/platforms, debug/free-threaded builds, sanitizers, and performance benchmarks have not been verified locally.

@lelit

lelit commented Oct 12, 2026

Copy link
Copy Markdown
Contributor

Thanks for the PR, I'm going to merge it.

@lelit
lelit merged commit 461fc0f into python-rapidjson:master Oct 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants