Repository navigation
Reject non-str reads in the tokenizer instead of looping forever - #1553
Closed
afonsojanu wants to merge 1 commit into
Closed
afonsojanu wants to merge 1 commit into
afonsojanu wants to merge 1 commit into
Conversation
_timelex.__init__ only checked for a `read` attribute on the input stream, which passes for anything that auto-vivifies attributes, like a MagicMock. If one of those reaches parse() (typically by accident, e.g. a mocking mistake in a caller's own test suite), get_token() never sees a real empty string to signal EOF and keeps calling read(1) forever, allocating memory the whole time until the process runs out. _timelex now checks that read() actually returned a str each time it's called, and raises a clear TypeError the first time it doesn't, rather than looping. Fixes #1428
afonsojanu
force-pushed
the
fix/timelex-reject-non-str-instream-read-1428
branch
from
August 30, 2026 14:22
2ea79d9 to
19bd55f
Compare
Author
|
Part of a general cleanup on my end, closing this along with the rest of what I had open. |
2 of 3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1428.
If something that isn't a string or a real character stream reaches
parse(),_timelex.__init__only checks for areadattribute before accepting it. That passes for anything that auto-vivifies attributes, like aMagicMock, which is exactly how this got reported: a test suite accidentally passed a mock object intoparse()instead of a real value.Once that happens,
get_token()callsself.instream.read(1)in a loop waiting for a falsy value to signal EOF. AMagicMockis never falsy and never returns a real empty string, so the loop never terminates, and it keeps allocating memory the whole time until the process runs out and the test suite just hangs with no useful error, which is what made this so hard to track down originally._timelexnow checks thatread()actually returned astron every call, and raises a clearTypeErrorthe moment it doesn't, instead of looping.Added a regression test that reproduces the exact scenario from the issue (a
MagicMockreachingparse()), verified against both the fixed and unfixed code with a bounded subprocess timeout, since the whole point of the bug is that it doesn't fail cleanly without the fix.