Free-threaded GC can abort during concurrent subinterpreter teardown #161751

Crash report

OpTree CI hit this assertion on an unmodified Windows x64 free-threaded debug build of CPython 3.14.10 (b703b2c). An InterpreterPoolExecutor worker was destroying its interpreter. The native stack went from Py_EndInterpreter() through shutdown GC to mi_abandoned_visit_blocks() and validate_alive_bits().

CI backtrace: https://github.com/metaopt/optree/actions/runs/42811376193/job/129813367183#step:18:1

Python/gc_free_threading.c:1082: validate_alive_bits:
Assertion "!gc_is_alive(op)" failed: object should not be marked as alive yet

I suspect a race in the abandoned-segment scan introduced with the mimalloc 2.5.2 update. One interpreter can temporarily clear a segment’s bitmap bit, discover that the segment belongs to another interpreter, and restore the bit. The owning interpreter’s GC reads the bitmap before acquiring abandoned_visit_lock. If it reads during that interval, it can skip the segment. The same scan is present on main at 9d4a35c.

Allocator code: https://github.com/python/cpython/blob/v3.14.10/Objects/mimalloc/arena-abandon.c#L209-L269

• 3.14 backport: [3.14] gh-159111: Update vendored mimalloc to v2.5.2 (gh-159112) #160183

Standard Library Reproducer

Starting from v3.14.10, add usleep(100); inside the foreign-segment branch of mi_arena_segment_clear_abandoned_at(), immediately before it restores the bitmap bit.

   if (segment->subproc != subproc) {
+    usleep(100);
     // it is from another sub-process, re-mark it and continue searching

Build with ./configure --with-pydebug --disable-gil and make. Save this as repro.py and run ./python.exe -I -X dev repro.py on macOS.

from concurrent import interpreters
from concurrent.futures import ThreadPoolExecutor
from threading import Barrier

workers = 8
barrier = Barrier(workers)


def run(_worker_id):
    for _ in range(10):
        interp = interpreters.create()
        barrier.wait()
        interp.exec('import gc\ngc.collect()')
        interp.close()


with ThreadPoolExecutor(workers) as pool:
    list(pool.map(run, range(workers)))

With the timing change, this reduced case aborted in validate_gc_objects() on all three runs. A longer standard-library stress test also reproduced the exact validate_alive_bits() assertion from Windows. On stock macOS debug builds, that longer test completed 340 rounds each on 3.14.9 and 3.14.10.

As a control, I kept the delay and moved lock acquisition ahead of the first bitmap read for full scans (previous->visit_all). The longer workload then completed a 10-round run. The precise thread interleaving in the original Windows crash is unknown.

CPython versions tested on:

3.14

Operating systems tested on:

macOS

Output from running 'python -VV' on the command line:

No response