Repository navigation
Conversation
In the free-threaded build, dequeiter_next(), dequereviter_next() and dequeiter_reduce() entered a two-object critical section on the iterator and its deque. All writes to the iterator's state already happen while holding the deque's lock, so the deque's critical section alone is enough to serialize concurrent next() calls on a shared iterator. This halves the atomic read-modify-write operations per item. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
list(d), [*d], list.extend(d) and PySequence_Fast(d) -- and hence str.join(d), which hands the deque's iterator to list.extend() -- fetched one item per tp_iternext call, each entering a critical section. Add a fast path that copies all remaining items of a deque or a forward deque iterator while holding the list's and the deque's critical sections once. The _collections module registers it with listobject.c, since the deque type lives in that module. The result is identical to running the iterator to exhaustion: no user code runs, the mutation check matches next(), and the list's allocation matches the generic path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
All commit authors signed the Contributor License Agreement. |
Keep the mutable deque hook pointer in PyInterpreterState instead of a C global so check-c-globals accepts it. Preserve release/acquire publication for concurrent free-threaded registration. Validated with ordinary and free-threaded pydebug builds, the focused C globals check, and deque/list/collections/embed/interpreter tests. pythongh-161850
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Depends on #161853. This is a draft follow-up. The branch currently contains the commits from #161853, so the diff against
mainincludes that prerequisite. The PR2-only changes are deque/list bulk materialization and its tests/NEWS entry.Closes #161850 once both changes are merged.
Changes
_collections, where the deque types are defined. Store the hook pointer in per-interpreter state, using release/acquire publication for concurrent free-threaded access.The atomic bulk copy can change which outcomes are observed under concurrent mutation; this is not a claim that it reproduces every possible interleaving of repeated
next()calls. The supported built-in-slot paths avoid user-defined iteration/length callbacks.Test Plan
git diff --checkpass for the PR2-only delta and the combined branch.test_deque,test_list,test_collections,test_free_threading.test_collections,test_embedandtest_interpreters(the GIL-specific suites are skipped in the opposite configuration).b83e4241979bfbf5036008b0b3675ac50ca884c5, including ordinary/free-threaded TSan, ASan, MSan, UBSan, fuzz and platform tests.For context, the combined 3.14.6 backport reduced unmodified Mako's FT render time by approximately 33% on Kunpeng 950 and 9% on EPYC 9654 in the issue's measurements. These are single-thread workload results and do not isolate this PR from its prerequisite.
AI tools assisted preparation of the change and submission text.
list(deque)take a critical section per item #161850