Skip to content

Avoid stop-the-world when __abstractmethods__ does not change the abstract flag #161744

Open

Description

@lpyu001

Summary

In a free-threaded CPython build, type_set_abstractmethods() stops the worldevery time ABCMeta writes __abstractmethods__, even when the write does notchange Py_TPFLAGS_IS_ABSTRACT.

ABCMeta performs this write for every newly created class. For a concrete subclass of an ABC, the value is normally an empty set, so the abstract flag is already clear. Nevertheless, the current implementation briefly stops all other Python threads.

Benchmark

import argparse, gc, statistics, sys, threading, time, typing
from abc import ABC, abstractmethod
from collections.abc import Sized

def make(kind, n):
    if kind == "concrete":           
        for _ in range(n):
            class X(Sized):
                def __len__(self): return 0
    elif kind == "protocol":          
        for _ in range(n):
            class P(typing.Protocol):
                def m(self) -> int: ...
            class C(P):
                def m(self) -> int: return 0
    elif kind == "abstract":
        for _ in range(n):
            class Y(ABC):
                @abstractmethod
                def m(self): ...
    elif kind == "plain":
        for _ in range(n):
            class Z: pass
    else:
        raise SystemExit(f"unknown kind {kind}")

def one_round(kind, k, n):
    barrier = threading.Barrier(k + 1)
    def worker():
        barrier.wait()
        make(kind, n)
    ts = [threading.Thread(target=worker) for _ in range(k)]
    for t in ts: t.start()
    barrier.wait()
    t0 = time.perf_counter()
    for t in ts: t.join()
    return time.perf_counter() - t0

def main():
    ap = argparse.ArgumentParser()
    ap.add_argument("--kind", default="concrete")
    ap.add_argument("--threads", type=int, default=1)
    ap.add_argument("--n", type=int, default=2000)
    ap.add_argument("--repeat", type=int, default=5)
    ap.add_argument("--no-gc", action="store_true")
    a = ap.parse_args()
    if a.no_gc:
        gc.disable()
    make(a.kind, 50)                  
    per = [one_round(a.kind, a.threads, a.n) / (a.n * a.threads) * 1e6
           for _ in range(a.repeat)]
    gil = getattr(sys, "_is_gil_enabled", lambda: True)()
    print(f"{a.kind},{a.threads},{a.n},{statistics.median(per):.2f},{min(per):.2f},{gil}")

if __name__ == "__main__":
    main()

Multiple threads run make_classes(2000) concurrently. The benchmark reports the median time per created class over three repetitions. It also includes three controls:

  • protocol: a typing.Protocol and a concrete subclass;
  • abstract: an ABC subclass that remains abstract;
  • plain: a class that does not use ABCMeta.

Implementation

In type_set_abstractmethods(), compare the requested abstract state with the current Py_TPFLAGS_IS_ABSTRACT flag before entering the stop-the-world section:

if (abstract != ((type->tp_flags & Py_TPFLAGS_IS_ABSTRACT) != 0)) {
    type_lock_prevent_release(&pinned);
    types_stop_world();
    if (abstract)
        type_add_flags(type, Py_TPFLAGS_IS_ABSTRACT);
    else
        type_clear_flags(type, Py_TPFLAGS_IS_ABSTRACT);
    types_start_world();
    type_lock_allow_release(&pinned);
}

Performance results

Each result below is the median of five outer rounds. Each benchmark invocation
uses three inner repetitions and creates 2000 classes per thread.

Case Threads Baseline Patched Speedup Spread, baseline / patched
concrete 1 6.90 6.46 1.07x ±24% / ±14%
concrete 2 920.77 6.06 151.94x ±15% / ±11%
concrete 4 704.21 6.18 113.95x ±3% / ±8%
concrete 8 490.60 7.18 68.33x ±6% / ±11%
concrete 16 348.28 8.45 41.22x ±6% / ±6%
protocol 1 22.62 21.65 1.04x ±7% / ±5%
protocol 2 2801.11 1038.10 2.70x ±4% / ±2%
protocol 4 2463.42 1111.75 2.22x ±1% / ±0%
protocol 8 1963.49 1028.07 1.91x ±1% / ±1%
protocol 16 1397.80 783.15 1.78x ±3% / ±4%
abstract 1 6.62 6.93 0.96x ±16% / ±12%
abstract 2 885.09 894.81 0.99x ±10% / ±10%
abstract 4 636.98 632.91 1.01x ±1% / ±3%
abstract 8 445.71 461.46 0.97x ±6% / ±5%
abstract 16 329.88 333.72 0.99x ±9% / ±3%
plain 1 3.74 3.95 0.95x ±20% / ±39%
plain 2 3.46 3.67 0.94x ±13% / ±11%
plain 4 3.91 3.98 0.98x ±12% / ±7%
plain 8 4.16 4.32 0.96x ±7% / ±7%
plain 16 4.16 4.14 1.00x ±13% / ±3%

Linked PRs

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions