Align ctypes call ownership and C error state with Python 3.15 by youknowdot · Pull Request #9218 · RustPython/RustPython

youknowdot · 2026-10-19T14:50:47Z

Summary

Extract the Python3.15 ctypes call-contract closure from #8954, with its required CAPI error-state dependency:

Retain converted argument owners, traverse CArg owners, support pointer callback arguments and preserve attachment around native calls/callbacks.
Use fallible variadic call interfaces and match the canonical optional complex-type/type-layout contracts.
Keep raised C errors separate from handled exceptions, including the existing mega CAPI reader/writer changes.
Preserve pending C errors across finalizers, weakref callbacks and Python buffer-release hooks; clear before exception constructors and match arbitrary-object _ctypes.__getattr__ behavior.
Regenerate three production ctypes modules and twelve related canonical test/helper files from CPython3.15.0 d220409a13b77de7a6c30abe16ea90db01528edc through the existing updater. Preserve existing guards except the proven Linux system-enumeration skip, narrowed through documented updater marker data. No manual Lib changes or updater special cases.

The utility/dllist dependency is included: synchronous native enumeration on Linux and Android copies loader-owned names before Python decoding. Existing macOS/Windows paths are preserved; unsupported outer targets are not newly supported. Android uses the repository API24 baseline, above the loader API21 requirement. Contributor-owned sequence-repetition changes are not duplicated.

Validation

Current integrated checkpoint `841a1bb`

Replayed the original three commits onto main 43368209c505c8807f5efdf3a39729bb30866722, preserving their authors and messages, followed by a separate C argument representation compatibility fix. The final complete source tree matches the locally validated candidate. This incorporates the already-landed benchmark harness correction without duplicating its changes in the PR-only diff.

The unchanged canonical parameter-representation test exposed two pre-existing issues also present on the pinned main: Rust numeric Display expanded 1e300 rather than using Python float repr, and long-double tag g incorrectly took the numeric representation path. The fix uses the existing Python float formatter for f/d and the existing generic-address representation for g, without changing FFI conversion, storage, or test assertions. Both sequential failures and the original candidate were retained before validating the final correction.

Final scoped validation:

Affected VM all-targets compilation with threading and strict Clippy passed; normal changed-file hooks and all four normal commit hooks passed.
A source-bound default+CAPI CLI build passed. VM, CAPI and root have final compiler invocation evidence; host_env reuse is tied to its prior compiler invocation and 74 identical source files.
Five unchanged, AST-selected canonical methods passed independently: deprecated module version warning (including filename), parameter repr, incomplete-pointer deprecation, pointer format update, and loaded-system-library enumeration. Original fixtures and decorators were preserved; whole-module discovery and the unnecessary _ctypes_test import were not executed. No foreign callback invocation or broad ctypes suite is claimed.
Each selected case ran in a fresh bounded process; no skips or resource stops. Full tracked source/lock checks and all protected binary seals passed.
The original uncorrected replay separately passed affected library/test compilation and root+CAPI integration compilation. Those historical compile-only passes are not substituted for the final corrected-source gates above.

Limits: the pinned CPython installation used for this validation lacks its standard-library _ctypes extension, so the control attempt stopped at import and no new cross-runtime success is claimed. No full suite, benchmark, crash/stress tests, or new ABI coverage was run in this integration pass.

The historical frozenset CodSpeed regression (358.4 to 413.8 microseconds on old head 607e121) has not yet been remeasured or explained by this repr fix. New exact-head CI is pending after publication. The current benchmark environment guard can skip measurement while the workflow succeeds; only actual benchmark execution/upload and comparison evidence will count as a remeasurement.

Historical combined checkpoint `607e121`

The initial hosted Linux run at 8e78504 failed with test_ctypes ENV CHANGED: the old Python/libffi dllist callback raised an unraisable ValueError: PyObject is NULL, then the test skipped its missing extension fixture. Its automatic rerun reproduced the same environmental failure. The native dllist dependency removes that obsolete callback path.

Fresh source-bound sqlite+CAPI release, normal/-E/-I provenance and all 21 focused Python stages pass.
Existing canonical ListSharedLibraries under canonical unraisable/environment checking: system test passes, genuine missing _ctypes_test fixture skips, environment_altered=false. The fixture guard is retained.
Four canonical Linux discovery cases pass on both candidate and pinned CPython. A separate verified official-libffi copy/load observation passes on both; no library function is invoked and this is not represented as canonical fixture execution.
Prior ctypes, PythonAPI, teardown and unchanged snippet selections rerun successfully on combined source.
Combined root/VM tests: 174 passed, 2 ignored; CAPI tests: 123 passed, 4 ignored. Workspace and CAPI strict Clippy passed. Source/protected audits clean. No incidental main rebase was mixed into this repair.

Historical constituent evidence

Standalone dllist on main eb2ffc6: fresh release, 1,401 workspace tests passed/18 ignored, 123 CAPI tests passed/4 ignored, both strict Clippy gates passed. Historical main58b54 fails the unchanged system-enumeration body with Unsupported argument type; native candidate and oracle pass. Local Android runtime and OpenGL-loader coverage are not claimed.

Original ctypes checkpoint 8e78504 (fresh sqlite+capi release):

Source-bound normal/-E/-I provenance checked.
Candidate31 canonical ctypes executions, including2 skips; five other selected modules are entirely skipped by their existing missing _ctypes_test guard.
Pinned oracle136 ctypes executions, including4 skips.
Eight existing finalizer/weakref/buffer selections pass on both.
Two unchanged scalar/pointer/errno and aggregate-ABI snippets pass on both.
PythonAPI observations match the oracle: successful call inside except, real conversion error, callback/constructor reentry, clear and handled identity.
_ctypes.__getattr__(None/42/tuple) produced TypeError on sealed mega baseline and now matches the oracle's exact AttributeError/messages.
Source/marker audits and68 oracle binary/extension hashes unchanged; normal hooks pass.
Full workspace baseline at 54e6f3d0: 1,402 passed, 18 ignored. The final checkpoint aligns raised-field destruction order with mega; after that change, affected root/VM tests were rerun: 174 passed, 2 ignored, including the native guard invariant and existing drop/GC tests.
Final CAPI tests: 123 passed, 4 ignored. Workspace and CAPI strict Clippy passed.

Important provenance: mega already had the separate raised slot. The initial narrowed extraction omitted that dependency; it was restored before candidate runtime. The same PYFUNCTYPE inside-except probe passed on both mega and oracle, so this PR does not claim a newly reproduced mega conflation bug.

The native guard test checks storage nesting and Rust unwind. The Python teardown selections exercise real boundaries, but do not establish every nonempty-native-error reentrancy combination. No all-platform ABI or comprehensive callback/GC parity claim. Existing foreign-thread callback entry, callback conversion-error reporting, errcheck locking and incomplete callback metadata traversal remain separate limitations.

OpenAI Codex assisted with implementation, source review, canonical regeneration, automated validation and this PR description. The stated local checks were automated; this does not claim manual human testing. Prepared for the user's maintainer-directed Python3.15 upgrade work.

coderabbitai · 2026-10-19T14:52:17Z

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Source mode: Auto
Repository: RustPython/RustPython
Branch: main
File: .coderabbit.yml
Commit: 4336820
Layers: Repository YAML
Review profile: CHILL
Plan: Advanced
Run ID: 775bd39b-3405-480c-8797-7464a9cf084f

📥 Commits

Reviewing files that changed from the base of the PR and between 4336820 and 841a1bb.

⛔ Files ignored due to path filters: 15

Lib/ctypes/__init__.py is excluded by !Lib/**
Lib/ctypes/_layout.py is excluded by !Lib/**
Lib/ctypes/util.py is excluded by !Lib/**
Lib/test/test_ctypes/__init__.py is excluded by !Lib/**
Lib/test/test_ctypes/_support.py is excluded by !Lib/**
Lib/test/test_ctypes/test_as_parameter.py is excluded by !Lib/**
Lib/test/test_ctypes/test_c_simple_type_meta.py is excluded by !Lib/**
Lib/test/test_ctypes/test_callbacks.py is excluded by !Lib/**
Lib/test/test_ctypes/test_dllist.py is excluded by !Lib/**
Lib/test/test_ctypes/test_find.py is excluded by !Lib/**
Lib/test/test_ctypes/test_generated_structs.py is excluded by !Lib/**
Lib/test/test_ctypes/test_incomplete.py is excluded by !Lib/**
Lib/test/test_ctypes/test_numbers.py is excluded by !Lib/**
Lib/test/test_ctypes/test_parameters.py is excluded by !Lib/**
Lib/test/test_ctypes/test_structures.py is excluded by !Lib/**

📒 Files selected for processing: 11

crates/capi/src/pyerrors.rs
crates/capi/src/util.rs
crates/host_env/src/ctypes.rs
crates/vm/src/builtins/memory.rs
crates/vm/src/gc_state.rs
crates/vm/src/object/core.rs
crates/vm/src/stdlib/_ctypes.rs
crates/vm/src/stdlib/_ctypes/base.rs
crates/vm/src/stdlib/_ctypes/function.rs
crates/vm/src/stdlib/_ctypes/simple.rs
crates/vm/src/vm/mod.rs

+-----------------------------------------------+
| Don't count your bugs before they're patched. |
+-----------------------------------------------+
  \
   \     /)/)
        ( '')
        /  |
      o(___)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create a new PR

Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-10-19T16:31:37Z

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] lib: cpython/Lib/ctypes
[ ] test: cpython/Lib/test/test_ctypes (TODO: 8)
[ ] test: cpython/Lib/test/test_stable_abi_ctypes.py

dependencies:

ctypes (native: _ctypes, ctypes._aix, ctypes._endian, ctypes.macholib.dyld, ctypes.macholib.dylib, ctypes.macholib.framework, importlib.machinery, itertools, nt, sys)
- sysconfig (native: _sysconfig, _winapi, importlib.machinery, importlib.util, os.path, sys)
- os, re, shutil, struct, subprocess, tempfile, types, warnings

dependent tests: (37 tests)

ctypes: test_android test_buffer test_bytes test_code test_codecs test_ctypes test_genericalias test_ntpath test_os test_ssl test_venv
- platform: test__locale test__osx_support test_asyncio test_baseexception test_builtin test_calendar test_cmath test_fcntl test_math test_mimetypes test_os test_platform test_regrtest test_shutil test_socket test_strptime test_sysconfig test_time test_tkinter test_winreg test_wsgiref
  - profiling.sampling.gecko_collector: test_profiling
  - profiling.sampling.heatmap_collector: test_profiling
  - pydoc: test_enum test_pydoc
- webbrowser: test_webbrowser

Legend:

[+] path exists in CPython
[x] up-to-date, [ ] outdated

codspeed · 2026-10-19T19:27:35Z

Merging this PR will degrade performance by 13.4%

❌ 1 regressed benchmark
✅ 61 untouched benchmarks
⏩ 4 skipped benchmarks¹

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
❌	`rustpython[frozenset.py]`	358.4 µs	413.8 µs	-13.4%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing youknowdot:split/ctypes-call-contracts-315 (607e121) with main (c08375b)}

4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

Keep native raised errors separate from handled exceptions and preserve them across teardown callbacks. Import the related canonical ctypes call and type contracts. Assisted-by: OpenAI Codex: model version unavailable

Copy loader-owned names before Python filesystem decoding and update the related canonical ctypes utility and discovery tests from CPython 3.15. Preserve existing platform paths and dllist test markers. Assisted-by: OpenAI Codex: model version unavailable

Regenerate the pinned test through update_lib patches with reviewed marker data. Preserve other platform conditions and the unavailable extension-fixture guard. Assisted-by: OpenAI Codex: model version unavailable

Use the existing Python float formatter for float and double C argument representations. Keep long-double arguments on the canonical generic-address representation path. Preserve the unchanged canonical parameter repr assertions. Assisted-by: OpenAI Codex: model version unavailable

chatgpt-codex-connector · 2026-10-22T09:16:34Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-22T09:28:49.426797Z	`841a1bb`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 841a1bbc9c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-10-22T09:28:46Z

-            let msg = format!("Exception ignored while calling ctypes callback function {repr}");
-            vm.run_unraisable(exc.clone(), Some(msg), vm.ctx.none());
-        }
+        vm.attach_for_callback(|| {


Save callback errno before reattaching the VM

When a use_errno=True callback is invoked from a normal detached CDLL call, this transition runs before with_callback_errno_preserved snapshots the real C errno. attach_for_callback can wait for stop-the-world or reacquire a critical-section lock, so an interrupted synchronization can overwrite errno and the helper then restores that value to the native caller instead of the callback-entry value; bracket the attach/detach transition with errno preservation or save errno before attaching.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-10-22T09:28:47Z

 import_helper.import_module('ctypes')

+
+class TestModule(unittest.TestCase):


Keep the canonical test bodies unchanged

This adds a new test class and assertions under Lib/test, but the repository policy permits test-file changes only for adding an expected-failure marker or removing one with its TODO after a pass. The canonical ctypes refresh also adds or rewrites other test bodies, so those changes need to be excluded rather than landed as test logic.

AGENTS.md reference: AGENTS.md:L280-L282

Useful? React with 👍 / 👎.

youknowdot mentioned this pull request Oct 19, 2026

Integrate remaining Python 3.15 final runtime and libraries #8954

Draft

youknowdot added 4 commits October 22, 2026 12:39

Align ctypes call ownership and C error state with Python 3.15

c6410d0

Keep native raised errors separate from handled exceptions and preserve them across teardown callbacks. Import the related canonical ctypes call and type contracts. Assisted-by: OpenAI Codex: model version unavailable

Enable canonical dllist system enumeration coverage on Linux

ea77020

Regenerate the pinned test through update_lib patches with reviewed marker data. Preserve other platform conditions and the unavailable extension-fixture guard. Assisted-by: OpenAI Codex: model version unavailable

youknowdot force-pushed the split/ctypes-call-contracts-315 branch from 607e121 to 841a1bb Compare October 22, 2026 03:41

youknowone approved these changes Oct 22, 2026

View reviewed changes

youknowone marked this pull request as ready for review October 22, 2026 09:16

youknowone merged commit 1ba1107 into RustPython:main Oct 22, 2026
30 checks passed

chatgpt-codex-connector Bot reviewed Oct 22, 2026

View reviewed changes

Navigation Menu

Repository navigation

Align ctypes call ownership and C error state with Python 3.15 - #9218

Align ctypes call ownership and C error state with Python 3.15#9218
youknowone merged 4 commits into
RustPython:mainfrom
youknowdot:split/ctypes-call-contracts-315

youknowdot commented Oct 19, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

github-actions Bot commented Oct 19, 2026 •

edited

Loading

Uh oh!

codspeed Bot commented Oct 19, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 22, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Oct 22, 2026

Uh oh!

chatgpt-codex-connector Bot Oct 22, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		import_helper.import_module('ctypes')


		class TestModule(unittest.TestCase):

Conversation

youknowdot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Current integrated checkpoint 841a1bb

Historical combined checkpoint 607e121

Historical constituent evidence

Uh oh!

coderabbitai Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions Bot commented Oct 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📦 Library Dependencies

Uh oh!

codspeed Bot commented Oct 19, 2026

Merging this PR will degrade performance by 13.4%

Performance Changes

Footnotes

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codex Review Summary

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Oct 22, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

youknowdot commented Oct 19, 2026 •

edited

Loading

Current integrated checkpoint `841a1bb`

Historical combined checkpoint `607e121`

coderabbitai Bot commented Oct 19, 2026 •

edited

Loading

github-actions Bot commented Oct 19, 2026 •

edited

Loading

chatgpt-codex-connector Bot commented Oct 22, 2026 •

edited

Loading