Repository navigation
How to deal with C-API extensions? #158
Description
Create a prototype on how to interact with a python library with a C-API extension.
See also:
https://pythoncapi.readthedocs.io/
There are some questions regarding C extensions. Primary question: how to handle this? Maybe in a similar way to cpyext?
Activity
windelbouwman commentedon Jan 12, 2020
Just wanted to bump this issue with a link to handlepy: https://github.com/pyhandle/hpy this might be the way to go forwards.
ShadowJonathan commentedon Aug 14, 2021
hpy is definitely the way, it's getting increasing support and attention from the python community
encukou commentedon Jan 16, 2023
Hi!
I help maintain CPython's Limited API & Stable ABI. If you want to support a subset of the C API, here's one. Not perfect -- it has decade of historical baggage itself -- but it's there. A bunch of real-world extensions use it, including PyO3 (there's even a suggestion to make it the default).
To support it, you'd need to expose a
PyObjectstruct with a refcount (C int) and a type (C pointer). Roughly speaking, those are the only two instances where Stable ABI extensions reach directly into memory; everything else is a function call. (Replacing refcounting by function calls turned out to be too slow, unfortunately.)I'd love feedback on what is hard to implement in Rust. We can add better API for the bad parts, and eventually even deprecate/remove stuff.
Feel free to contribute to my shared notes, post here, or reach me in any other way :)
arihant2math commentedon Apr 5, 2025
I've started with an initial implementation with #5604. Currently I haven't found a way to properly expose
PyObject, but I don't see that being a massive issue for now.bschoenmaeckers commentedon Apr 4, 2026
Pep 803 makes
PyObjectan opaque pointer. So this might help improve things.encukou commentedon Apr 5, 2026
Rust wasn't the primary motivation for PEP 803, but I definitely had alternate implementations in mind :)
arihant2math commentedon Apr 5, 2026
This seems interesting, if I have time, I'll take another look at a C-API.
youknowone commentedon Oct 6, 2026
crates/capiis the C-API prototype: pyo3-ffi-shaped headers, a growing Stable ABI surface, and tests undercrates/capi. Tracking leftover functions continues in #8156.Closing.
— commented by Grok 4.6