Skip to content

Avoid allocating a result vector for a single ctypes output - #9749

Merged
youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:perf/ctypes-single-output-20261108
Nov 11, 2026
Merged

youknowone merged 1 commit into
RustPython:mainfrom
youknowdot:perf/ctypes-single-output-20261108

Conversation

@youknowdot

@youknowdot youknowdot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Follow up on #9618 without changing its OutValue representation or object ownership.

For one OUT/INOUT result, convert/clone the borrowed value directly instead of collecting it into a temporary Vec and immediately extracting it. Multiple outputs still collect into a tuple. Raw-result disposal, errcheck short-circuiting, callback order, exception propagation, and the lifetime of all output owners are unchanged.

This follows the singleton-result shape of CPython 3.15 _build_result. It is not a bitmask redesign or a claim of complete ctypes compatibility.

Real native-call regression test

The existing ordinary DLL constructor discards paramflags, so using that constructor would not exercise this path. The test fixture sets only the internal parameter metadata (as the COM constructor does), then executes Python calls through the actual VM and libffi to small native C-ABI functions.

It checks single custom OUT conversion, original INOUT identity, multiple outputs, raw-result disposal order, errcheck replacement/identity, and conversion exceptions. It does not assert source tokens or macro expansion.

The separate DLL-constructor/resolved-callargs/errcheck-argument behavior is deliberately not enabled or changed here.

Original validation (before the November 11 rebase)

Passed on Linux:

  • cargo test --workspace --exclude rustpython_wasm --exclude rustpython-venvlauncher --exclude rustpython-capi
  • C-API cargo test
  • Workspace and C-API Clippy with --all-targets -- -D warnings
  • Native-call regression in both debug/workspace and release configurations
  • Release -m test test_ctypes: SUCCESS (run=323, skipped=56)
  • Existing extra_tests/snippets/stdlib_ctypes.py in debug and release
  • Normal pre-commit hooks

No throughput speedup percentage is claimed; this removes the singleton result-vector construction, not all allocations in a ctypes call. Windows/macOS CI remains to be checked.

November 11 rebase

Rebased independently onto dd8c34399621e60e703caeff89b2d5ae5c2f64f6 without changing the patch (git range-diff reports equality). Changed-file pre-commit hooks passed. Fresh CI for this head and local full validation are pending.

The previous failed jobs stopped during external tool setup: HTTP 504 downloading Python, prek/reviewdog, and CodSpeed, and a missing Wasmer executable. None reported a failure in the changed ctypes code.

AI assistance: OpenAI Codex assisted with implementation, testing, review, and the rebase.

@coderabbitai

coderabbitai Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

Pre-merge checks: Passed 1 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Title check Pending This assessment has not completed. The check will be retried on the next pre-merge check run.
Passed Passed checks: 1 passed

Check name Status Explanation
Description Check Passed Check skipped - CodeRabbit’s high-level summary is enabled.
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

Keep OutValue and all owners alive across output conversion. Exercise the real VM/libffi return path with a native test fixture, without enabling the separately incomplete DLL paramflags constructor.

Assisted-by: OpenAI Codex:model version not exposed
@youknowdot
youknowdot force-pushed the perf/ctypes-single-output-20261108 branch from bf7162b to 5aaaf67 Compare November 11, 2026 03:16
@youknowone
youknowone marked this pull request as ready for review November 11, 2026 14:07
@youknowone
youknowone merged commit b47d20f into RustPython:main Nov 11, 2026
28 checks passed
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants