-
Notifications
You must be signed in to change notification settings - Fork 1.4k
Update doctest,test_generators from v3.14.2 and fix generator bugs #7067
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds REPL printing in the compiler; per-frame locals_dirty tracking and locals_to_fast syncing; extends tracing with Call/Return/Line and CCall/CReturn/CException and fires trace events around frames and C-callables; refines generator/coroutine destructor/close behavior and return types; improves OSError/WinError handling; fixes list indexing and tee reentrancy; narrows a syntax-error caret. Changes
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 1 | ❌ 2❌ Failed checks (1 warning, 1 inconclusive)
✅ Passed checks (1 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Code has been automatically formatted The code in this PR has been formatted using:
git pull origin doctest |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 3
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
crates/vm/src/builtins/int.rs (1)
123-145:⚠️ Potential issue | 🟠 MajorFix error messages to match CPython for integer division operations.
CPython 3.12 uses:
inner_mod:"integer modulo by zero"(not"division by zero")inner_floordiv:"integer division or modulo by zero"(not"division by zero")inner_divmod:"integer division or modulo by zero"(not"division by zero")crates/vm/src/coroutine.rs (1)
203-228:⚠️ Potential issue | 🟡 Minor
close()returns the generator's return value instead ofNone.CPython's
gen_close()discards the generator's return value and always returnsNone:Py_DECREF(retval); Py_RETURN_NONE;Here,
ExecutionResult::Return(value)on line 226 passes through the generator's actual return value. If a generator hasreturn 42and the caller doesresult = gen.close(), CPython yieldsNonebut this implementation yields42.This is unlikely to matter in practice (the return value of
.close()is almost never used), but it's a behavioral divergence from CPython.Fix to match CPython
- Ok(ExecutionResult::Return(value)) => Ok(value), + Ok(ExecutionResult::Return(_)) => Ok(vm.ctx.none()),
🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 2159-2166: The current interactive printing (guarded by
self.interactive && !self.ctx.in_func() && !self.ctx.in_class) also prints
expression statements nested inside compound blocks; restrict printing to true
top-level only by adding a check that there is no block nesting (e.g., require
ctx.block_depth == 0 or !self.ctx.in_block, or add/use a helper like
ctx.is_module_toplevel()) before emitting Instruction::CallIntrinsic1 { Print },
or alternatively move the echo logic out of the per-statement path into
compile_program_single so only top-level expressions are printed; update the
condition that currently references self.interactive, self.ctx.in_func(), and
self.ctx.in_class accordingly.
In `@crates/vm/src/frame.rs`:
- Around line 261-281: The locals_to_fast method clears locals_dirty before
syncing, so if mapping().subscript returns an Err and the function returns early
the dirty flag is lost; move the locals_dirty.store(false,
atomic::Ordering::Release) to after the for loop completes successfully (just
before returning Ok(())) so the flag is only cleared when all code.varnames have
been processed and fastlocals updated, ensuring a retry on error; update
references in locals_to_fast accordingly (keep use of fastlocals.lock() and the
existing error handling for key_error).
🧹 Nitpick comments (4)
crates/vm/src/vm/vm_new.rs (1)
555-560:narrow_caretlogic is unreachable whenpython_end_location()returnsNone.If the parser ever produces an
InvalidStarredExpressionUsageerror without an end location,narrow_caret = truewould have no effect since this code is gated onif let Some(...). In practice this is likely fine since ruff should always provide end locations for this error type, but worth noting.crates/vm/src/builtins/frame.rs (1)
46-49: Minor:locals_dirtyis set even whenself.locals(vm)fails.If
self.locals(vm)returns anErr, the caller never receives the locals dict, so markinglocals_dirty = trueis unnecessary. It's harmless (just triggers a no-op sync later), but for correctness:Suggested fix
fn f_locals(&self, vm: &VirtualMachine) -> PyResult { - let result = self.locals(vm).map(Into::into); - self.locals_dirty - .store(true, core::sync::atomic::Ordering::Release); - result + let result = self.locals(vm)?; + self.locals_dirty + .store(true, core::sync::atomic::Ordering::Release); + Ok(result.into()) }crates/vm/src/builtins/coroutine.rs (1)
155-169: Duplicated destructor logic betweenPyGeneratorandPyCoroutine.The
delimplementation here is identical toPyGenerator::delingenerator.rs(lines 151-168). Consider extracting the common finalization logic into a method onCoroitself (e.g.,Coro::finalize(&self, jen: &PyObject, vm: &VirtualMachine) -> PyResult<()>) and having both destructors delegate to it.Sketch
Add to
Coro:pub fn finalize(&self, jen: &PyObject, vm: &VirtualMachine) -> PyResult<()> { if self.closed() || self.running() { return Ok(()); } if self.frame.lasti() == 0 { self.closed.store(true); return Ok(()); } if let Err(e) = self.close(jen, vm) { vm.run_unraisable(e, None, jen.to_owned()); } Ok(()) }Then both destructors become:
fn del(zelf: &Py<Self>, vm: &VirtualMachine) -> PyResult<()> { zelf.inner.finalize(zelf.as_object(), vm) }As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."
crates/vm/src/protocol/callable.rs (1)
55-56:is_python_callablecheck may miss other Python-level callables.The check only covers
PyFunctionandPyBoundMethod. Other Python-level callables like instances with__call__or class objects won't match and will getc_call/c_returnevents instead ofcall/return. This diverges from CPython, where any Python-level__call__triggerscall/returnevents.That said, this is likely acceptable for an initial implementation since the main goal is to avoid double-tracing for
PyFunctionandPyBoundMethodwhich already emit events viawith_frame().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/vm/src/coroutine.rs (1)
203-228:⚠️ Potential issue | 🟡 Minor
close()returns the generator's return value instead ofNoneonReturn— deliberate?CPython's
gen_close()unconditionally returnsPy_Noneon success. Line 226 returns the actualExecutionResult::Return(value), which means if a generator catchesGeneratorExitand executesreturn some_value,close()will surface that value instead ofNone. This is an extremely rare edge case, but it's a semantic difference from CPython.If aligning with CPython is desired:
Proposed fix
- Ok(ExecutionResult::Return(value)) => Ok(value), + Ok(ExecutionResult::Return(_)) => Ok(vm.ctx.none()),
🧹 Nitpick comments (4)
crates/vm/src/ospath.rs (1)
330-346: Consider extracting shared logic withwith_filename.
with_filename_from_errnoduplicateswith_filename(lines 318–329) almost entirely — the only difference is the#[cfg(windows)]without_winerror()call. You could extract a common helper and pass a flag or closure for the Windows-specific transform, though given how small these methods are, this is not urgent.♻️ One way to reduce duplication
+ fn build_with_filename_inner<'a>( + error: &std::io::Error, + filename: impl Into<OsPathOrFd<'a>>, + strip_winerror: bool, + vm: &VirtualMachine, + ) -> crate::builtins::PyBaseExceptionRef { + use crate::exceptions::ToOSErrorBuilder; + let builder = error.to_os_error_builder(vm); + #[cfg(windows)] + let builder = if strip_winerror { builder.without_winerror() } else { builder }; + let builder = builder.filename(filename.into().filename(vm)); + builder.build(vm).upcast() + } + #[must_use] pub(crate) fn with_filename<'a>( error: &std::io::Error, filename: impl Into<OsPathOrFd<'a>>, vm: &VirtualMachine, ) -> crate::builtins::PyBaseExceptionRef { - use crate::exceptions::ToOSErrorBuilder; - let builder = error.to_os_error_builder(vm); - let builder = builder.filename(filename.into().filename(vm)); - builder.build(vm).upcast() + Self::build_with_filename_inner(error, filename, false, vm) } #[must_use] pub(crate) fn with_filename_from_errno<'a>( error: &std::io::Error, filename: impl Into<OsPathOrFd<'a>>, vm: &VirtualMachine, ) -> crate::builtins::PyBaseExceptionRef { - use crate::exceptions::ToOSErrorBuilder; - let builder = error.to_os_error_builder(vm); - #[cfg(windows)] - let builder = builder.without_winerror(); - let builder = builder.filename(filename.into().filename(vm)); - builder.build(vm).upcast() + Self::build_with_filename_inner(error, filename, true, vm) }As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."
crates/vm/src/stdlib/itertools.rs (1)
959-983: Reentrancy guard and cache-awareget_itemlook correct.The design mirrors CPython's
teedataobjectpattern well: serve from cache on fast path, guarditerable.next()with an atomicrunningflag, and append to cache on miss.One minor observation: on line 982,
values[index]relies on the invariant thatindexis always either< values.len()(served from the fast path on lines 965–967) or== values.len()(pushed on line 980). If this invariant were ever violated (e.g.,values.len() < index), it would panic at runtime. Since the tee iterators enforce sequential access, this is safe in practice—same assumption CPython makes—but a debug assertion could catch future regressions:💡 Optional: add a debug assertion
if values.len() == index { values.push(obj); } + debug_assert!( + index < values.len(), + "tee iterator cache invariant violated: index {index} >= len {}", + values.len() + ); Ok(PyIterReturn::Return(values[index].clone()))crates/vm/src/frame.rs (1)
466-479: Lockingself.object.traceon every instruction while tracing is active — acceptable but worth noting.The
self.object.trace.lock()inside the hot loop (line 473) is only reached whenvm.use_tracingis set, so normal execution is unaffected. However, when a trace function is active, this acquires the mutex on every instruction — even when the line hasn't changed. For a tighter fast path you could cache whether the per-frame trace is set outside the loop, or at least check the line-change condition before acquiring the lock.That said, this mirrors CPython's approach (check tracing flags, then consult per-frame state), and tracing is inherently slow, so this is fine for now.
crates/vm/src/protocol/callable.rs (1)
52-69: Unconditionalto_owned()clone for non-Python callables even when tracing is off.Line 60 creates an
Arcclone of the callable before checkinguse_tracinginsidetrace_event. When tracing is disabled (the overwhelmingly common case), this is a wasted atomic increment/decrement on every non-Python call (builtins, C extensions, etc.).A quick guard avoids the overhead:
Proposed optimization
} else { - let callable = self.obj.to_owned(); - vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?; - let result = (self.call)(self.obj, args, vm); - if result.is_ok() { - vm.trace_event(TraceEvent::CReturn, Some(callable))?; - } else { - let _ = vm.trace_event(TraceEvent::CException, Some(callable)); + if vm.use_tracing.get() { + let callable = self.obj.to_owned(); + vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?; + let result = (self.call)(self.obj, args, vm); + if result.is_ok() { + vm.trace_event(TraceEvent::CReturn, Some(callable))?; + } else { + let _ = vm.trace_event(TraceEvent::CException, Some(callable)); + } + result + } else { + (self.call)(self.obj, args, vm) } - result }
02f87b3 to
c6c4d45
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
crates/vm/src/coroutine.rs (1)
203-228:⚠️ Potential issue | 🟡 Minor
close()returns the generator's return value instead of alwaysNone.On line 226,
Ok(ExecutionResult::Return(value)) => Ok(value)returns whatever the generator produced. In CPython'sgen_close(genobject.c), the return value is explicitly discarded andPy_Noneis always returned. While this may not matter in practice (callers rarely inspectclose()'s return value), it's a semantic deviation from CPython.Consider returning
Ok(vm.ctx.none())for consistency:Proposed fix
Ok(ExecutionResult::Yield(_)) => { Err(vm.new_runtime_error(format!("{} ignored GeneratorExit", gen_name(jen, vm)))) } Err(e) if !is_gen_exit(&e, vm) => Err(e), - Ok(ExecutionResult::Return(value)) => Ok(value), + Ok(ExecutionResult::Return(_)) => Ok(vm.ctx.none()), _ => Ok(vm.ctx.none()),
🧹 Nitpick comments (2)
crates/vm/src/stdlib/itertools.rs (1)
959-983: Reentrancy guard logic is sound but not panic-safe.The two-phase approach (cache-first, then fetch-with-guard) is correct. The
runningflag properly prevents reentrant calls to the underlying iterator, and thevalues.len() == indexcheck on line 979 correctly handles the case where another thread already populated the cache entry.However, if
self.iterable.next(vm)on line 973 panics (unwind),runningremainstruepermanently, bricking the tee. Consider using a scope guard (e.g., a drop-based RAII wrapper) to ensurerunningis reset even on panic:Suggested panic-safe pattern
// Prevent concurrent/reentrant calls to iterable.next() if self.running.swap(true, Ordering::Acquire) { return Err(vm.new_runtime_error("cannot re-enter the tee iterator")); } - let result = self.iterable.next(vm); - self.running.store(false, Ordering::Release); + let result = { + struct ResetOnDrop<'a>(&'a AtomicBool); + impl Drop for ResetOnDrop<'_> { + fn drop(&mut self) { + self.0.store(false, Ordering::Release); + } + } + let _guard = ResetOnDrop(&self.running); + self.iterable.next(vm) + };This is a minor robustness improvement—VM calls generally shouldn't panic, but a drop guard is a cheap safety net.
crates/vm/src/vm/vm_new.rs (1)
554-560: Redundant call toerror.python_location()— consider reusing the earlier result.
python_location()is already called on line 541 but the result is immediately converted toPyInt, making it unavailable here. A small refactor could store the(u32, u32)tuple before converting, avoiding the duplicate call.♻️ Suggested refactor
- let (lineno, offset) = error.python_location(); - let lineno = self.ctx.new_int(lineno); - let offset = self.ctx.new_int(offset); + let (lineno_raw, offset_raw) = error.python_location(); + let lineno = self.ctx.new_int(lineno_raw); + let offset = self.ctx.new_int(offset_raw);Then at line 555:
if let Some((end_lineno, end_offset)) = error.python_end_location() { let (end_lineno, end_offset) = if narrow_caret { - let (l, o) = error.python_location(); - (l, o + 1) + (lineno_raw, offset_raw + 1) } else { (end_lineno, end_offset) };
- Add interactive REPL mode: auto-print expression results in single mode - Implement Destructor for PyGenerator and PyCoroutine - Add locals_dirty tracking and locals_to_fast() for frame sync - Add per-line tracing with prev_line tracking in execution loop - Fix gen_throw to close sub-iterator on GeneratorExit (gen_close_iter) - Pass callable object as arg in c_call/c_return/c_exception trace events - Distinguish [Errno] vs [WinError] for CRT vs Win32 API errors - Fix tee thread safety with AtomicBool running flag - Fix division error messages to match expected format
📦 Library DependenciesThe following Lib/ modules were modified. Here are their dependencies: [x] lib: cpython/Lib/doctest.py dependencies:
dependent tests: (29 tests)
[ ] lib: cpython/Lib/pdb.py dependencies:
dependent tests: (no tests depend on pdb) [ ] lib: cpython/Lib/concurrent dependencies:
dependent tests: (11 tests)
[x] lib: cpython/Lib/dbm dependencies:
dependent tests: (4 tests)
[ ] test: cpython/Lib/test/test_extcall.py (TODO: 30) dependencies: dependent tests: (no tests depend on extcall) [ ] test: cpython/Lib/test/test_generators.py (TODO: 12) dependencies: dependent tests: (no tests depend on generator) [x] lib: cpython/Lib/inspect.py dependencies:
dependent tests: (42 tests)
[ ] test: cpython/Lib/test/test_raise.py dependencies: dependent tests: (no tests depend on raise) [ ] test: cpython/Lib/test/test_sys.py (TODO: 15) dependencies: dependent tests: (209 tests)
[ ] lib: cpython/Lib/trace.py dependencies:
dependent tests: (1 tests)
Legend:
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 2
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
crates/vm/src/builtins/int.rs (2)
123-145:⚠️ Potential issue | 🟠 MajorError messages diverge from CPython — this may break doctest compatibility.
CPython uses
"integer division or modulo by zero"for//anddivmod, and"integer modulo by zero"for%. Only true division (/) uses"division by zero". Unifying all three to"division by zero"will cause any doctest that captures the exactZeroDivisionErrormessage for%,//, ordivmodto fail.The old
inner_divmodmessage reportedly had a typo ("integer division orModulo by zero"), so fixing that is welcome — but the replacement should match CPython.Proposed fix to match CPython messages
fn inner_mod(int1: &BigInt, int2: &BigInt, vm: &VirtualMachine) -> PyResult { if int2.is_zero() { - Err(vm.new_zero_division_error("division by zero")) + Err(vm.new_zero_division_error("integer modulo by zero")) } else { Ok(vm.ctx.new_int(int1.mod_floor(int2)).into()) } } fn inner_floordiv(int1: &BigInt, int2: &BigInt, vm: &VirtualMachine) -> PyResult { if int2.is_zero() { - Err(vm.new_zero_division_error("division by zero")) + Err(vm.new_zero_division_error("integer division or modulo by zero")) } else { Ok(vm.ctx.new_int(int1.div_floor(int2)).into()) } } fn inner_divmod(int1: &BigInt, int2: &BigInt, vm: &VirtualMachine) -> PyResult { if int2.is_zero() { - return Err(vm.new_zero_division_error("division by zero")); + return Err(vm.new_zero_division_error("integer division or modulo by zero")); }
123-145:⚠️ Potential issue | 🔴 CriticalFix error messages for integer zero-division operations to match CPython.
CPython uses distinct error messages for integer operations:
%(modulo):"integer modulo by zero"or"integer division or modulo by zero"//(floor division):"integer division or modulo by zero"divmod():"integer division or modulo by zero"/(true division):"division by zero"The current code incorrectly uses the generic
"division by zero"for all three integer operations (inner_mod,inner_floordiv,inner_divmod), which breaks CPython compatibility. This is especially problematic for doctests, which validate exact exception messages. Update these functions to use the correct messages matching CPython's behavior.
🤖 Fix all issues with AI agents
In `@crates/vm/src/frame.rs`:
- Around line 261-281: locals_to_fast currently leaves fastlocals[i] untouched
when locals.mapping().subscript(varname, vm) returns a KeyError, but it should
clear the fast slot like CPython's PyFrame_LocalsToFast; update the match in
locals_to_fast so that Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error)
sets fastlocals[i] = None (unbound) instead of doing nothing, ensuring deleted
keys in f_locals unbind the corresponding fastlocal; keep the existing Err(e) =>
return Err(e) path and retain locals_dirty store/release logic.
In `@crates/vm/src/protocol/callable.rs`:
- Around line 52-69: The code currently always calls self.obj.to_owned() for
non-Python callables, bumping refcounts even when tracing is disabled; change
the logic to first check the VM tracing flag (e.g., vm.use_tracing) and only
call self.obj.to_owned() when tracing is enabled, then call
vm.trace_event(TraceEvent::CCall, Some(callable.clone())) before invoking
(self.call) and vm.trace_event(TraceEvent::CReturn) /
vm.trace_event(TraceEvent::CException) afterwards using that owned `callable`;
otherwise, when tracing is disabled, directly invoke (self.call)(self.obj, args,
vm) without calling self.obj.to_owned() or any clone to avoid the hot-path
refcount bump.
| /// Sync locals dict back to fastlocals. Called before generator/coroutine resume | ||
| /// to apply any modifications made via f_locals. | ||
| pub fn locals_to_fast(&self, vm: &VirtualMachine) -> PyResult<()> { | ||
| if !self.locals_dirty.load(atomic::Ordering::Acquire) { | ||
| return Ok(()); | ||
| } | ||
| let code = &**self.code; | ||
| let mut fastlocals = self.fastlocals.lock(); | ||
| for (i, &varname) in code.varnames.iter().enumerate() { | ||
| if i >= fastlocals.len() { | ||
| break; | ||
| } | ||
| match self.locals.mapping().subscript(varname, vm) { | ||
| Ok(value) => fastlocals[i] = Some(value), | ||
| Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {} | ||
| Err(e) => return Err(e), | ||
| } | ||
| } | ||
| self.locals_dirty.store(false, atomic::Ordering::Release); | ||
| Ok(()) | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
locals_to_fast does not clear fastlocals when a key is deleted from f_locals.
When a varname is missing from the locals dict (KeyError on line 275), the corresponding fastlocals[i] slot is left unchanged. In CPython's PyFrame_LocalsToFast, a missing key causes the fast slot to be set to NULL (i.e., the variable becomes unbound). This means del frame.f_locals['x'] followed by a generator resume won't actually unbind x.
If this is intentional (e.g., scoped to what doctest needs), a comment would help. Otherwise:
Proposed fix
match self.locals.mapping().subscript(varname, vm) {
Ok(value) => fastlocals[i] = Some(value),
- Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
+ Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {
+ fastlocals[i] = None;
+ }
Err(e) => return Err(e),
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// Sync locals dict back to fastlocals. Called before generator/coroutine resume | |
| /// to apply any modifications made via f_locals. | |
| pub fn locals_to_fast(&self, vm: &VirtualMachine) -> PyResult<()> { | |
| if !self.locals_dirty.load(atomic::Ordering::Acquire) { | |
| return Ok(()); | |
| } | |
| let code = &**self.code; | |
| let mut fastlocals = self.fastlocals.lock(); | |
| for (i, &varname) in code.varnames.iter().enumerate() { | |
| if i >= fastlocals.len() { | |
| break; | |
| } | |
| match self.locals.mapping().subscript(varname, vm) { | |
| Ok(value) => fastlocals[i] = Some(value), | |
| Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {} | |
| Err(e) => return Err(e), | |
| } | |
| } | |
| self.locals_dirty.store(false, atomic::Ordering::Release); | |
| Ok(()) | |
| } | |
| /// Sync locals dict back to fastlocals. Called before generator/coroutine resume | |
| /// to apply any modifications made via f_locals. | |
| pub fn locals_to_fast(&self, vm: &VirtualMachine) -> PyResult<()> { | |
| if !self.locals_dirty.load(atomic::Ordering::Acquire) { | |
| return Ok(()); | |
| } | |
| let code = &**self.code; | |
| let mut fastlocals = self.fastlocals.lock(); | |
| for (i, &varname) in code.varnames.iter().enumerate() { | |
| if i >= fastlocals.len() { | |
| break; | |
| } | |
| match self.locals.mapping().subscript(varname, vm) { | |
| Ok(value) => fastlocals[i] = Some(value), | |
| Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => { | |
| fastlocals[i] = None; | |
| } | |
| Err(e) => return Err(e), | |
| } | |
| } | |
| self.locals_dirty.store(false, atomic::Ordering::Release); | |
| Ok(()) | |
| } |
🤖 Prompt for AI Agents
In `@crates/vm/src/frame.rs` around lines 261 - 281, locals_to_fast currently
leaves fastlocals[i] untouched when locals.mapping().subscript(varname, vm)
returns a KeyError, but it should clear the fast slot like CPython's
PyFrame_LocalsToFast; update the match in locals_to_fast so that Err(e) if
e.fast_isinstance(vm.ctx.exceptions.key_error) sets fastlocals[i] = None
(unbound) instead of doing nothing, ensuring deleted keys in f_locals unbind the
corresponding fastlocal; keep the existing Err(e) => return Err(e) path and
retain locals_dirty store/release logic.
| // Python functions get 'call'/'return' events from with_frame(). | ||
| // Bound methods delegate to the inner callable, which fires its own events. | ||
| // All other callables (built-in functions, etc.) get 'c_call'/'c_return'/'c_exception'. | ||
| let is_python_callable = self.obj.downcast_ref::<PyFunction>().is_some() | ||
| || self.obj.downcast_ref::<PyBoundMethod>().is_some(); | ||
| if is_python_callable { | ||
| (self.call)(self.obj, args, vm) | ||
| } else { | ||
| let callable = self.obj.to_owned(); | ||
| vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?; | ||
| let result = (self.call)(self.obj, args, vm); | ||
| if result.is_ok() { | ||
| vm.trace_event(TraceEvent::CReturn, Some(callable))?; | ||
| } else { | ||
| let _ = vm.trace_event(TraceEvent::CException, Some(callable)); | ||
| } | ||
| result | ||
| } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🛠️ Refactor suggestion | 🟠 Major
Unconditional clone of the callable on every non-Python call, even when tracing is off.
self.obj.to_owned() (line 60) bumps the refcount for every built-in call regardless of whether tracing is active. Since vm.use_tracing is usually false, gate the tracing path behind the check:
Proposed fix
if is_python_callable {
(self.call)(self.obj, args, vm)
+ } else if vm.use_tracing.get() {
+ let callable = self.obj.to_owned();
+ vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
+ let result = (self.call)(self.obj, args, vm);
+ if result.is_ok() {
+ vm.trace_event(TraceEvent::CReturn, Some(callable))?;
+ } else {
+ let _ = vm.trace_event(TraceEvent::CException, Some(callable));
+ }
+ result
} else {
- let callable = self.obj.to_owned();
- vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
- let result = (self.call)(self.obj, args, vm);
- if result.is_ok() {
- vm.trace_event(TraceEvent::CReturn, Some(callable))?;
- } else {
- let _ = vm.trace_event(TraceEvent::CException, Some(callable));
- }
- result
+ (self.call)(self.obj, args, vm)
}This avoids the to_owned() + clone() on the hot path when no tracer is registered.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Python functions get 'call'/'return' events from with_frame(). | |
| // Bound methods delegate to the inner callable, which fires its own events. | |
| // All other callables (built-in functions, etc.) get 'c_call'/'c_return'/'c_exception'. | |
| let is_python_callable = self.obj.downcast_ref::<PyFunction>().is_some() | |
| || self.obj.downcast_ref::<PyBoundMethod>().is_some(); | |
| if is_python_callable { | |
| (self.call)(self.obj, args, vm) | |
| } else { | |
| let callable = self.obj.to_owned(); | |
| vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?; | |
| let result = (self.call)(self.obj, args, vm); | |
| if result.is_ok() { | |
| vm.trace_event(TraceEvent::CReturn, Some(callable))?; | |
| } else { | |
| let _ = vm.trace_event(TraceEvent::CException, Some(callable)); | |
| } | |
| result | |
| } | |
| // Python functions get 'call'/'return' events from with_frame(). | |
| // Bound methods delegate to the inner callable, which fires its own events. | |
| // All other callables (built-in functions, etc.) get 'c_call'/'c_return'/'c_exception'. | |
| let is_python_callable = self.obj.downcast_ref::<PyFunction>().is_some() | |
| || self.obj.downcast_ref::<PyBoundMethod>().is_some(); | |
| if is_python_callable { | |
| (self.call)(self.obj, args, vm) | |
| } else if vm.use_tracing.get() { | |
| let callable = self.obj.to_owned(); | |
| vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?; | |
| let result = (self.call)(self.obj, args, vm); | |
| if result.is_ok() { | |
| vm.trace_event(TraceEvent::CReturn, Some(callable))?; | |
| } else { | |
| let _ = vm.trace_event(TraceEvent::CException, Some(callable)); | |
| } | |
| result | |
| } else { | |
| (self.call)(self.obj, args, vm) | |
| } |
🤖 Prompt for AI Agents
In `@crates/vm/src/protocol/callable.rs` around lines 52 - 69, The code currently
always calls self.obj.to_owned() for non-Python callables, bumping refcounts
even when tracing is disabled; change the logic to first check the VM tracing
flag (e.g., vm.use_tracing) and only call self.obj.to_owned() when tracing is
enabled, then call vm.trace_event(TraceEvent::CCall, Some(callable.clone()))
before invoking (self.call) and vm.trace_event(TraceEvent::CReturn) /
vm.trace_event(TraceEvent::CException) afterwards using that owned `callable`;
otherwise, when tracing is disabled, directly invoke (self.call)(self.obj, args,
vm) without calling self.obj.to_owned() or any clone to avoid the hot-path
refcount bump.
Summary by CodeRabbit
New Features
Bug Fixes
Performance & Stability