Repository navigation
Conversation
calliter_iternext() used borrowed references to it_callable and it_sentinel, which could be freed by __setstate__() during the call or comparison (pythongh-158031), or by another thread in the free-threaded build. Take strong references when reading the fields, and update them with an atomic exchange plus a delayed DECREF.
nascheme
added
the
type-crash
nascheme
marked this pull request as ready for review
November 11, 2026 05:16
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
calliter_iternext()calledit_callableand compared the result againstit_sentinelthrough borrowed pointers.__eq__can call__setstate__()and free the sentinel while the comparison is using it (uaf of it_sentinel in callable_iterator.__next__ #158031).__setstate__(), and two threads exhausting the iterator can both DECREF the callable (Crash when a callable iterator is shared between threads #161792).Sharing a callable iterator between threads, or calling
__setstate__()on one that is in use, is not supported: the values produced are unspecified. It must not crash, though. That lets the fix avoid a lock:__setstate__()) swap the field with an atomic exchange, so only one thread gets the old value, and release it with_PyObject_XDecRefDelayed(). In the free-threaded build that defers the DECREF until no thread can still be between its load and its incref; in the default build it is a plainPy_XDECREF().This also covers gh-158031, so it overlaps with #158046, which takes a strong reference to the sentinel only.
Performance
Release builds (Clang 20, no PGO/LTO), single thread pinned to one CPU, ns per item including consumption overhead (
deque(islice(it, N), maxlen=0)), best of 7, mean of two runs.Free-threaded build:
iter(int, 1)1object()sentinel__next__ofitertools.repeat, sentinel1Default (GIL) build:
iter(int, 1)1object()sentinel__next__ofitertools.repeat, sentinel1The cost is about 1 ns per item in the free-threaded build and 0-3 ns in the default build, from the strong references to the callable and sentinel. It is only visible when the callable is trivially cheap.