Skip to content

Align memoryview scalar, cast, and indexing controls - #9692

Merged
youknowone merged 2 commits into
RustPython:mainfrom
youknowdot:fix/memoryview-index-parity-20261104
Nov 7, 2026
Merged

youknowone merged 2 commits into
RustPython:mainfrom
youknowdot:fix/memoryview-index-parity-20261104

Conversation

@youknowdot

@youknowdot youknowdot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Align memoryview scalar formats, casts, indexing, assignment and comparison with CPython 3.15. Rebased onto main 22ee0b0380 through #9563 (a5d24851a5e94c81770935e6578c6081fda10705), retaining its strict shared Clinic binding and string conversion.

  • Keep a small native memoryview scalar adapter, as CPython does, while sharing the binary format types, unpack functions, numeric converters and existing struct format cache. Text formatting for str/bytes has a different grammar and is not used for binary layouts.
  • Use allocation-free native equality and fixed-size scalar write buffers. Borrow conversion inputs and existing memoryview descriptors instead of cloning objects or allocating a Vec per scalar write. Retain temporary arguments until the operation finishes, including their observable finalizers.
  • Make the shared struct parser strict rather than maintaining a second comparison grammar. Match whitespace, repeat-count, NUL and native-only format validation. Share bit-exact half conversion and preserve half/complex signaling-NaN behavior.
  • Match cast and index conversion/error ordering, saturated slice bounds, reflected comparison, tuple indexing, iterator state and overlapping assignment. Propagate multidimensional comparison failures and permit shared immutable buffer reads without recursive mutex deadlocks.
  • Remove obsolete packing error wrappers. Add focused Rust and snippet regressions without changing existing test assertions. The included Enforce Clinic binding by default #9563 revision removes only two obsolete SQLite expected-failure decorators.

Reference: CPython 3.15 memoryobject.c, struct and float packing.

CI failure and rebase

The previous head failed the CPython suites on Linux, macOS and Windows only because two SQLite tests unexpectedly passed while still marked expectedFailure: test_set_progress_handler_keyword_args and test_func_keyword_args. The inherited binder fixed their behavior; #9563 retires those two obsolete decorators. Memoryview and struct passed on all three OSes.

The rebase preserves main's new staged CSV parser and list/tuple/slice behavior. Its shared unnamed duplicate-argument diagnostic now omits parentheses, matching CPython, without restoring parser opt-outs. Saturating index conversion is shared with the new sequence helper while preserving temporary argument lifetimes.

Validation

Verified locally on Linux x86_64 with CPython 3.15.0 as the reference:

  • Release RustPython build passed with stdlib,stdio,importlib,host_env,doc,threading,sqlite and no default features.
  • Strict workspace and separate C-API Clippy passed with -D warnings. All changed-file hooks and formatting checks passed.
  • Ten focused snippets passed, including memoryview, argument errors, CSV, list, tuple and slice. The slice snippet ran through its normal fixture-generating pytest harness.
  • All 698 behavioral differential cases matched CPython exactly.
  • Relevant stdlib suites, including SQLite, CSV, memoryview, struct, list, tuple, slice and index: 3,451 tests run, 242 skipped; all 23 test files passed.
  • CPython-only half-precision reference checks passed 255,994 assertions. Rust unit tests independently cover every half encoding and finite half rounding boundary.
  • Full extra_tests pytest: 487 passed, 19 failed, 15 subtests passed. Failures were restricted socket/chown operations (14) and optional SSL/legacy-Unicode features omitted from this profile (5). This is not a full-suite pass.
  • Full workspace/C-API test reruns are still pending on this rebased head. The prior revision passed 1,507 workspace tests and 149 C-API tests; those results are not substituted for the final reruns.

The release-after-index char assignment intentionally retains RustPython's safety check: when an index callback releases a cast('c') view, assignment raises ValueError rather than writing through the released view as CPython currently does. Fresh hosted CI remains required. No claim of complete memoryview compatibility or a measured speedup is made.

AI-assisted implementation, testing and source review using OpenAI Codex; model version is not exposed.

@coderabbitai

coderabbitai Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Explain Change →

Caution

Review failed

The pull request is closed.

  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

@youknowdot youknowdot changed the title Align memoryview indexing and slice assignment Align memoryview scalar and indexing controls Nov 4, 2026
@youknowdot youknowdot changed the title Align memoryview scalar and indexing controls Align memoryview scalar, cast, and indexing controls Nov 4, 2026
@codspeed

codspeed Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 62 untouched benchmarks
⏩ 4 skipped benchmarks1


Comparing youknowdot:fix/memoryview-index-parity-20261104 (3e5d184) with main (eda879c)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

@youknowdot
youknowdot force-pushed the fix/memoryview-index-parity-20261104 branch 2 times, most recently from 3e5d184 to af5cc4e Compare November 6, 2026 13:07
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/sqlite3
[x] test: cpython/Lib/test/test_sqlite3 (TODO: 11)

dependencies:

  • sqlite3

dependent tests: (2 tests)

  • sqlite3: test_dbm_sqlite3 test_sqlite3

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowdot
youknowdot force-pushed the fix/memoryview-index-parity-20261104 branch from af5cc4e to 7700b57 Compare November 6, 2026 14:33
codex added 2 commits November 7, 2026 15:22
Reuse the shared Clinic binder, numeric converters and struct cache while preserving memoryview-specific scalar policies. Avoid per-scalar temporary allocations and redundant object clones. Match cast, indexing, comparison and half conversion behavior, preserve callback lifetimes, and fix multidimensional comparison and shared-buffer reads.

Add focused Rust and Python regression coverage. AI assistance covered implementation, test preparation and source review.

Assisted-by: OpenAI Codex:model version not exposed
Keep retained owners outside lock scopes and preserve callback-sensitive release ordering.

Assisted-by: OpenAI Codex:model version not exposed
@youknowdot
youknowdot force-pushed the fix/memoryview-index-parity-20261104 branch from 7700b57 to 8451d4f Compare November 7, 2026 09:32
@youknowone
youknowone marked this pull request as ready for review November 7, 2026 11:31
@chatgpt-codex-connector

Copy link
Copy Markdown
Codex Review Summary: ❌ Failed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ❌ Failed 2026-11-07T11:31:48.954435Z
We couldn’t complete this review. Comment @codex review to try again.
8451d4f Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@youknowone
youknowone merged commit 2ac449b into RustPython:main Nov 7, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants