Description

Less aggressive version of #677.

Based on the summary in #658 (comment), this PR offers a performance optimization over identified hotspots to bring us to a lot closer with our reference (CuPy). The optimization strategy is to

• implement everything in pure Python (we do this today for cuda.core)
• once hotspots are identified, we lower to Cython
• most importantly, we still call cuda.bindings Python APIs in the Cython code, so as to avoid introducing CTK as a build-time dependency (and therefore having to ship two separate packages cuda-core-cu11 and cuda-core-cu12)

In other words, this PR tries to find a reasonable balance between performance, easy of development, and easy of deployment, without introducing any breaking change.

Preliminary data:

• cuda.core main branch

In [5]: %timeit e = dev.create_event()
4.65 μs ± 18.4 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

In [5]: %timeit s = dev.create_stream()
7.7 μs ± 9.98 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

• this PR

In [6]: %timeit e = dev.create_event()
1.11 μs ± 6.91 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [4]: %timeit s = dev.create_stream()
4.12 μs ± 14 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

• cupy

In [8]: %timeit e = cp.cuda.Event(disable_timing=True)
749 ns ± 5.45 ns per loop (mean ± std. dev. of 7 runs, 1,000,000 loops each)

In [14]: %timeit s = cp.cuda.Stream(non_blocking=True)
3.8 μs ± 8.54 ns per loop (mean ± std. dev. of 7 runs, 100,000 loops each)

Checklist

• New or existing tests cover these changes.
• The documentation is up to date with these changes.