Skip to content

gh-161850: Lock only the deque when advancing a deque iterator - #161853

Draft
113xiaoji wants to merge 2 commits into
python:mainfrom
113xiaoji:deque-iter-single-lock
Draft

113xiaoji wants to merge 2 commits into
python:mainfrom
113xiaoji:deque-iter-single-lock

Conversation

@113xiaoji

@113xiaoji 113xiaoji commented Nov 12, 2026 •

Copy link
Copy Markdown

Summary

Refs #161850.

Serialize forward and reverse deque iteration, and iterator reduction, using the deque's mutex alone. The iterator retains its deque throughout its lifetime, and its mutable iteration state is protected by that mutex. This removes the iterator mutex acquisition/release from the per-item path.

Changes

Test Plan

  • Source review and git diff --check pass for this main-based branch.
  • The main-based CI run passed all 48 jobs on head d0ed2da495922d2887672d3fd0c2cdeb22df4278, including ordinary/free-threaded builds and TSan jobs.
  • Performance measurements below were run on a CPython 3.14.6 backport, not this main-based (3.16 development) port. This PR stays draft for core review.

On the tested 950, the 3.14.6 backport's 1000-element deque iteration benchmark decreased from about 52.3 to 30.6 microseconds. This is supporting evidence for the approach, not a performance claim for this branch.

AI tools assisted preparation of the change and submission text.

113xiaoji and others added 2 commits November 12, 2026 14:52
In the free-threaded build, dequeiter_next(), dequereviter_next() and
dequeiter_reduce() entered a two-object critical section on the iterator
and its deque.  All writes to the iterator's state already happen while
holding the deque's lock, so the deque's critical section alone is enough
to serialize concurrent next() calls on a shared iterator.  This halves
the atomic read-modify-write operations per item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@python-cla-bot

python-cla-bot Bot commented Nov 12, 2026 •

Copy link
Copy Markdown

All commit authors signed the Contributor License Agreement.

CLA signed

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

type-feature A feature request or enhancement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant