Skip to content

Optimize list.remove() lock usage - #8451

Merged
youknowone merged 1 commit into
RustPython:mainfrom
KRRT7:perf-list-remove-lock
Aug 4, 2026
Merged

Optimize list.remove() lock usage#8451
youknowone merged 1 commit into
RustPython:mainfrom
KRRT7:perf-list-remove-lock

Conversation

@KRRT7

@KRRT7 KRRT7 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Reduce list.remove() from two lock acquisitions to one.
  • Update pre-commit Python hooks to use python3.
  • Update the redundant-test-patches script shebang.

Performance

PyList::remove now checks bounds and removes the item under a single mutable lock, avoiding an extra read-lock acquisition.

Testing

  • prek run --all-files passes.
  • Full workspace tests compile successfully.
  • rustpython-capi tests currently fail with an unrelated SIGSEGV in PyO3's CPython-specific PyList_SET_ITEM.

Summary by CodeRabbit

Summary by CodeRabbit

  • Bug Fixes
    • Improved list element removal reliability by validating the target index against the current list length before removal.
    • Updated development checks and metadata generation to consistently use Python 3.

@coderabbitai

coderabbitai Bot commented Aug 4, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 833fca92-5083-42af-9dd6-05d08024fe58

📥 Commits

Reviewing files that changed from the base of the PR and between fb2941a and d917aca.

📒 Files selected for processing (3)
  • .pre-commit-config.yaml
  • crates/vm/src/builtins/list.rs
  • scripts/check_redundant_patches.py
🚧 Files skipped from review as they are similar to previous changes (3)
  • scripts/check_redundant_patches.py
  • crates/vm/src/builtins/list.rs
  • .pre-commit-config.yaml

📝 Walkthrough

Walkthrough

The change updates Python script invocation to use python3 and revises list.remove to use one mutable borrow with explicit bounds handling.

Changes

Python 3 invocation updates

Layer / File(s) Summary
Python 3 command standardization
.pre-commit-config.yaml, scripts/check_redundant_patches.py
The opcode metadata hooks and redundant-patch checker now invoke python3.

List removal handling

Layer / File(s) Summary
List removal bounds handling
crates/vm/src/builtins/list.rs
list.remove uses one mutable list guard, checks the index against the current length, and removes the element when valid.

Estimated code review effort: 2 (Simple) | ~10 minutes

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely identifies the main change: reducing lock usage in list.remove().
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@crates/vm/src/builtins/list.rs`:
- Around line 375-377: Update the removal logic around the mutable list borrow
so the PyObjectRef returned by elements.remove(index) is stored and kept alive
until after the borrow guard is released, then explicitly drop it. Add a
regression test covering an item whose __del__ accesses the list during
deallocation.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro Plus

Run ID: 4a8da830-9d9e-4a7e-88b5-2f6bf20599d2

📥 Commits

Reviewing files that changed from the base of the PR and between 331c3b1 and 8278e0b.

📒 Files selected for processing (3)
  • .pre-commit-config.yaml
  • crates/vm/src/builtins/list.rs
  • scripts/check_redundant_patches.py

Comment thread crates/vm/src/builtins/list.rs Outdated
@KRRT7
KRRT7 force-pushed the perf-list-remove-lock branch from 8278e0b to fb2941a Compare August 4, 2026 06:56

@ShaharNaveh ShaharNaveh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TYSM!
this LGTM overall:)

  1. can you please check our AI policy (I've added it to your PR)
  2. is there a reason why you've changed python -> python3 in the ci and the script?

@KRRT7

KRRT7 commented Aug 4, 2026

Copy link
Copy Markdown
Contributor Author

FTR: this PR was done by a proprietary tool of mine which does leverage AI to make decisions, in this case DeepSeek V4 Flash was used

and for the python -> python3, MacOS removed python2.7 support a while back and it ended up breaking the precommit for me since it couldn't find it .

@ShaharNaveh ShaharNaveh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tysm!

@youknowone youknowone left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for contributing!

Please also add AI disclosure in your commit message, not in PR comment.

https://github.com/RustPython/.github/blob/main/AI_POLICY.md#disclosing-ai-assistance-in-commit-messages

// defer delete out of borrow
let is_inside_range = index < self.borrow_vec().len();
Ok(is_inside_range.then(|| self.borrow_vec_mut().remove(index)))
let removed = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure how creating removed helps here. what does it hold and why do we need removed variable?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I believe the main change was to call .borrow_vec_mut() once instead of twice

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes, in the summary: Reduce list.remove() from two lock acquisitions to one

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Assisted-by: WarpForgeAgent:DeepSeek-v4-flash
@KRRT7
KRRT7 force-pushed the perf-list-remove-lock branch from fb2941a to d917aca Compare August 4, 2026 08:15
// defer delete out of borrow
let is_inside_range = index < self.borrow_vec().len();
Ok(is_inside_range.then(|| self.borrow_vec_mut().remove(index)))
let removed = {

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@youknowone
youknowone merged commit 5febd2e into RustPython:main Aug 4, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants