Short, frequent GIL releases block other threads for seconds #161868

Bug report

What happens

If one thread runs Python code and releases the GIL for a very short time every few milliseconds, a second thread that wants the GIL can wait for seconds. The switch interval (sys.getswitchinterval(), 5 ms by default) should keep that wait near 5 ms.

We found this in a service that runs asyncio and parses .xlsx files in a thread pool via loop.run_in_executor. The parser reads the file through zipfile, and each read calls zlib, which releases the GIL. These releases came every 2–5 ms and the event loop thread could not run for up to 530 ms at a time.

Reproducer

A worker thread runs 2 ms of Python code, then calls zlib.crc32() on 6,000 bytes, which releases the GIL for a few microseconds. The main thread calls time.sleep(0.001) in a loop for 2 seconds. Each sleep should take about 6 ms at most: 1 ms of sleep plus up to one 5 ms switch interval.

import sys
import threading
import time
import zlib

DATA = b"x" * 6000  # zlib.crc32() releases the GIL for inputs over 5 KiB


def worker(stop):
    while not stop.is_set():
        # About 2 ms of pure-Python work, then a short GIL release.
        end = time.perf_counter() + 0.002
        while time.perf_counter() < end:
            pass
        zlib.crc32(DATA)


stop = threading.Event()
thread = threading.Thread(target=worker, args=(stop,))
thread.start()

longest = 0.0
start = time.perf_counter()
while time.perf_counter() - start < 2.0:
    before = time.perf_counter()
    time.sleep(0.001)
    longest = max(longest, time.perf_counter() - before)
stop.set()
thread.join()

print(sys.version)
print(f"switch interval: {sys.getswitchinterval() * 1000:.0f} ms")
print(f"longest time.sleep(0.001) in the main thread: {longest * 1000:.0f} ms")

Results

Every build with a GIL waits for seconds.

$ uv run --no-project --python 3.14+gil python -I /tmp/gil_handoff_repro.py
3.14.10 (main, Oct 26 2026, 13:59:48) [Clang 23.1.3 ]
switch interval: 5 ms
longest time.sleep(0.001) in the main thread: 14335 ms

$ uv run --no-project --python 3.15+gil python -I /tmp/gil_handoff_repro.py
3.15.0 (main, Oct  9 2026, 15:11:47) [Clang 22.1.3 ]
switch interval: 5 ms
longest time.sleep(0.001) in the main thread: 15049 ms

$ ./python.exe -I /tmp/gil_handoff_repro.py
3.16.0a2+dev (main, Nov 11 2026, 16:37:20) [Clang 21.0.0 (clang-2100.4.1.1)]
switch interval: 5 ms
longest time.sleep(0.001) in the main thread: 15633 ms

I've run this on an Apple M3 Pro with 12 cores and macOS 26.7.1. The released versions are the uv builds. main (0ada998) was built from source with a plain ./configure.
On Linux (6-vCPU x86-64), 3.9 and every version from 3.11 to main waited 0.7–26 s. With both threads pinned to one CPU core, the problem does not occur.

Why it happens

The waiting thread uses a 5 ms timer, and each short release starts that timer again from zero.

1. A thread that wants the GIL waits up to 5 ms. If nobody hands the GIL over in that time, it asks the running thread to let go.
2. Each time the running thread releases the GIL, even for a few microseconds, it wakes the waiting thread.
3. On a machine with more than one core, the running thread takes the GIL back before the waiting thread gets to run.
4. The waiting thread then starts a new 5 ms wait. The time it already waited is lost.
5. If the releases come more often than every 5 ms, the timer never runs out. The waiting thread never asks for the GIL, so it can wait for seconds.

Related

We found no existing report of this problem. The nearest items:
• Daniel Stutzbach, python-dev, 26 October 2009, on the new GIL design: "if a thread's time budget is merely paused during I/O rather than reset, then a thread making frequent (but short) I/O requests cannot starve the system." Nobody replied.
• gh-52194 (convoy effect): a different problem. There, the thread that releases the GIL often is the one that waits.
• gh-91331: a proposal to let applications tune when hashlib, binascii.crc32() and zlib.crc32() release the GIL. Today each uses a fixed size check. For zlib.crc32(), the input must be more than 5 KiB.
• gh-89977 and PR gh-157981 (merged October 2026): setblocking(), getsockname(), getpeername() and gethostname() now keep the GIL, to avoid a hand-off between threads on short system calls. It does not change take_gil().

CPython versions tested on:

CPython main branch, 3.15, 3.14

Operating systems tested on:

Linux, macOS