Skip to content

gh-120321: Make gen.gi_frame.clear() thread-safe #143112

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
colesbury wants to merge 1 commit into
base: main
Choose a base branch
from
Open

gh-120321: Make gen.gi_frame.clear() thread-safe #143112

colesbury wants to merge 1 commit into from
+39 −21

Conversation

@colesbury
Copy link
Contributor

@colesbury colesbury commented Dec 23, 2025 •
edited by bedevere-app bot
Loading

This splits up the clearing of a generator frame from the tp_finalize implementation, which makes it easier to do the frame transition atomically.

Note that since gh-111792, it's only legal to call generator.gi_frame.clear() on a generator in the FRAME_CREATED state or finished state, not any of the suspended states.

@bedevere-app bedevere-app bot mentioned this pull request Dec 23, 2025
Open
@colesbury
Copy link
Contributor Author

colesbury commented Dec 23, 2025

I've seen this reported in TSan, although I'm not sure where the frame.clear() call is happening in Python:

test_imap_handle_iterable_exception (test.test_multiprocessing_forkserver.test_threads.WithThreadsTestPool.test_imap_handle_iterable_exception) ... ==================
WARNING: ThreadSanitizer: data race (pid=952942)
  Atomic write of size 1 at 0x7fffb56c24a3 by thread T79:
    #0 _Py_atomic_compare_exchange_int8 /home/sgross/cpython/./Include/cpython/pyatomic_gcc.h:75:10 (python+0x2ba69c) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
    #1 gen_send_ex /home/sgross/cpython/Objects/genobject.c:327:15 (python+0x2ba69c)
    #2 gen_iternext /home/sgross/cpython/Objects/genobject.c:721:9 (python+0x2b6ec2) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
    #3 _PyForIter_VirtualIteratorNext /home/sgross/cpython/Python/ceval.c:4115:24 (python+0x527188) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
    #4 _PyEval_EvalFrameDefault /home/sgross/cpython/Python/generated_cases.c.h:5375:36 (python+0x50aa13) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
    #5 _PyEval_EvalFrame /home/sgross/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x4ec83d) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
...

  Previous read of size 1 at 0x7fffb56c24a3 by main thread:
    #0 frame_clear_impl /home/sgross/cpython/Objects/frameobject.c:2019:18 (python+0x2d06f2) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
    #1 frame_clear /home/sgross/cpython/Objects/clinic/frameobject.c.h:407:20 (python+0x2d06f2)
    #2 _PyEval_EvalFrameDefault /home/sgross/cpython/Python/generated_cases.c.h:3556:35 (python+0x50b406) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
    #3 _PyEval_EvalFrame /home/sgross/cpython/./Include/internal/pycore_ceval.h:119:16 (python+0x4ec83d) (BuildId: 59de1379ec77c44e6bf58e50950f5e82982a1cca)
...

@colesbury colesbury marked this pull request as ready for review December 23, 2025 23:13
@colesbury colesbury requested a review from mpage December 23, 2025 23:13
@colesbury colesbury mentioned this pull request Dec 24, 2025
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markshannon markshannon Awaiting requested review from markshannon markshannon is a code owner

@mpage mpage Awaiting requested review from mpage

Assignees

No one assigned

Labels

awaiting core review skip news topic-free-threading

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@colesbury