Skip to content

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Feb 9, 2026

Summary by CodeRabbit

  • New Features

    • REPL interactive mode prints top-level expression results automatically.
    • Expanded tracing with call/return/line and native-call events for richer debugging.
  • Bug Fixes

    • Standardized "division by zero" messages.
    • Improved list index bounds handling and file/path error reporting that preserves errno info.
    • Refined syntax-error caret positioning.
  • Performance & Stability

    • Improved generator/coroutine cleanup and destructor semantics.
    • Frame locals synchronization for consistency.
    • Concurrency-safe caching and reentrancy protection for tee iterator.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 9, 2026

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Adds REPL printing in the compiler; per-frame locals_dirty tracking and locals_to_fast syncing; extends tracing with Call/Return/Line and CCall/CReturn/CException and fires trace events around frames and C-callables; refines generator/coroutine destructor/close behavior and return types; improves OSError/WinError handling; fixes list indexing and tee reentrancy; narrows a syntax-error caret.

Changes

Cohort / File(s) Summary
Compiler (interactive/REPL)
crates/codegen/src/compile.rs
Add interactive: bool to Compiler; emit Print intrinsics for top-level Exprs in single/REPL compilation.
Frame & execution tracing
crates/vm/src/frame.rs, crates/vm/src/builtins/frame.rs, crates/vm/src/vm/mod.rs, crates/vm/src/vm/vm_new.rs
Add Frame::locals_dirty: AtomicBool, locals_to_fast() and call sites; set locals_dirty in f_locals; emit per-line Line trace events; fire Call/Return trace events around frames; adjust syntax error caret for a parse case.
Tracing & call protocol
crates/vm/src/protocol/callable.rs, crates/vm/src/protocol/mod.rs
Make TraceEvent crate-visible, add variants (Line, CCall, CReturn, CException), change trace_event to accept Option<PyObjectRef>, and emit C* events for non-Python callables.
Coroutine / Generator lifecycle
crates/vm/src/builtins/coroutine.rs, crates/vm/src/builtins/generator.rs, crates/vm/src/coroutine.rs
Expose Destructor for coroutine/generator types; change close() to return PyObjectRef; add del destructor implementations and suspended() accessor; materialize locals before send/throw; refine throw/close control flow and repr.
Frame exception construction
crates/vm/src/frame.rs (continued)
Refactor gen_throw/throw exception construction to use ExceptionCtor::try_from_object and attach traceback explicitly.
OSError / WinError handling
crates/vm/src/exceptions.rs, crates/vm/src/ospath.rs, crates/vm/src/stdlib/io.rs, crates/vm/src/stdlib/os.rs
Add OSErrorBuilder::without_winerror() and with_filename_from_errno(); switch stdlib call sites to use errno-aware builder to control WinError propagation/presentation.
List indexing bounds
crates/vm/src/builtins/list.rs
Use wrap_index() for integer indices and return IndexError("list index out of range") on invalid indices instead of unguarded access.
Itertools tee reentrancy
crates/vm/src/stdlib/itertools.rs
Add running: AtomicBool and a cache-aware fast path for tee get_item; guard re-entrancy and append fetched items to cache.
Integer division messages
crates/vm/src/builtins/int.rs
Standardize zero-division messages to "division by zero" across integer div/mod/divmod/floordiv.
Callable imports / minor hygiene
crates/vm/src/protocol/callable.rs, crates/vm/src/coroutine.rs
Add new imports (PyBoundMethod, PyFunction, TryFromObject) used by tracing/exception logic and callable type checks.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • ShaharNaveh
  • coolreader18

Poem

🐰
I nibbled bytes and traced each line,
REPL prints sparkle where values shine,
Generators tuck in, their endings neat,
Errors wear cleaner shoes for each OS street,
A rabbit hops — the patch feels sweet.

🚥 Pre-merge checks | ✅ 1 | ❌ 2
❌ Failed checks (1 warning, 1 inconclusive)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 54.39% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title partially relates to the changeset but is overly broad and doesn't capture the main scope of changes. The title mentions 'doctest' and 'test_generators' updates, but the PR includes extensive changes across frame locals synchronization, coroutine/generator destructors, tracing infrastructure, exception handling, list indexing, and error messages. Consider a more comprehensive title that reflects the broader scope or focus on the primary change.
✅ Passed checks (1 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

Code has been automatically formatted

The code in this PR has been formatted using:

  • cargo fmt --all
    Please pull the latest changes before pushing again:
git pull origin doctest

@youknowone youknowone marked this pull request as ready for review February 11, 2026 00:33
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
crates/vm/src/builtins/int.rs (1)

123-145: ⚠️ Potential issue | 🟠 Major

Fix error messages to match CPython for integer division operations.

CPython 3.12 uses:

  • inner_mod: "integer modulo by zero" (not "division by zero")
  • inner_floordiv: "integer division or modulo by zero" (not "division by zero")
  • inner_divmod: "integer division or modulo by zero" (not "division by zero")
crates/vm/src/coroutine.rs (1)

203-228: ⚠️ Potential issue | 🟡 Minor

close() returns the generator's return value instead of None.

CPython's gen_close() discards the generator's return value and always returns None:

Py_DECREF(retval);
Py_RETURN_NONE;

Here, ExecutionResult::Return(value) on line 226 passes through the generator's actual return value. If a generator has return 42 and the caller does result = gen.close(), CPython yields None but this implementation yields 42.

This is unlikely to matter in practice (the return value of .close() is almost never used), but it's a behavioral divergence from CPython.

Fix to match CPython
-            Ok(ExecutionResult::Return(value)) => Ok(value),
+            Ok(ExecutionResult::Return(_)) => Ok(vm.ctx.none()),
🤖 Fix all issues with AI agents
In `@crates/codegen/src/compile.rs`:
- Around line 2159-2166: The current interactive printing (guarded by
self.interactive && !self.ctx.in_func() && !self.ctx.in_class) also prints
expression statements nested inside compound blocks; restrict printing to true
top-level only by adding a check that there is no block nesting (e.g., require
ctx.block_depth == 0 or !self.ctx.in_block, or add/use a helper like
ctx.is_module_toplevel()) before emitting Instruction::CallIntrinsic1 { Print },
or alternatively move the echo logic out of the per-statement path into
compile_program_single so only top-level expressions are printed; update the
condition that currently references self.interactive, self.ctx.in_func(), and
self.ctx.in_class accordingly.

In `@crates/vm/src/frame.rs`:
- Around line 261-281: The locals_to_fast method clears locals_dirty before
syncing, so if mapping().subscript returns an Err and the function returns early
the dirty flag is lost; move the locals_dirty.store(false,
atomic::Ordering::Release) to after the for loop completes successfully (just
before returning Ok(())) so the flag is only cleared when all code.varnames have
been processed and fastlocals updated, ensuring a retry on error; update
references in locals_to_fast accordingly (keep use of fastlocals.lock() and the
existing error handling for key_error).
🧹 Nitpick comments (4)
crates/vm/src/vm/vm_new.rs (1)

555-560: narrow_caret logic is unreachable when python_end_location() returns None.

If the parser ever produces an InvalidStarredExpressionUsage error without an end location, narrow_caret = true would have no effect since this code is gated on if let Some(...). In practice this is likely fine since ruff should always provide end locations for this error type, but worth noting.

crates/vm/src/builtins/frame.rs (1)

46-49: Minor: locals_dirty is set even when self.locals(vm) fails.

If self.locals(vm) returns an Err, the caller never receives the locals dict, so marking locals_dirty = true is unnecessary. It's harmless (just triggers a no-op sync later), but for correctness:

Suggested fix
     fn f_locals(&self, vm: &VirtualMachine) -> PyResult {
-        let result = self.locals(vm).map(Into::into);
-        self.locals_dirty
-            .store(true, core::sync::atomic::Ordering::Release);
-        result
+        let result = self.locals(vm)?;
+        self.locals_dirty
+            .store(true, core::sync::atomic::Ordering::Release);
+        Ok(result.into())
     }
crates/vm/src/builtins/coroutine.rs (1)

155-169: Duplicated destructor logic between PyGenerator and PyCoroutine.

The del implementation here is identical to PyGenerator::del in generator.rs (lines 151-168). Consider extracting the common finalization logic into a method on Coro itself (e.g., Coro::finalize(&self, jen: &PyObject, vm: &VirtualMachine) -> PyResult<()>) and having both destructors delegate to it.

Sketch

Add to Coro:

pub fn finalize(&self, jen: &PyObject, vm: &VirtualMachine) -> PyResult<()> {
    if self.closed() || self.running() {
        return Ok(());
    }
    if self.frame.lasti() == 0 {
        self.closed.store(true);
        return Ok(());
    }
    if let Err(e) = self.close(jen, vm) {
        vm.run_unraisable(e, None, jen.to_owned());
    }
    Ok(())
}

Then both destructors become:

fn del(zelf: &Py<Self>, vm: &VirtualMachine) -> PyResult<()> {
    zelf.inner.finalize(zelf.as_object(), vm)
}

As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."

crates/vm/src/protocol/callable.rs (1)

55-56: is_python_callable check may miss other Python-level callables.

The check only covers PyFunction and PyBoundMethod. Other Python-level callables like instances with __call__ or class objects won't match and will get c_call/c_return events instead of call/return. This diverges from CPython, where any Python-level __call__ triggers call/return events.

That said, this is likely acceptable for an initial implementation since the main goal is to avoid double-tracing for PyFunction and PyBoundMethod which already emit events via with_frame().

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/vm/src/coroutine.rs (1)

203-228: ⚠️ Potential issue | 🟡 Minor

close() returns the generator's return value instead of None on Return — deliberate?

CPython's gen_close() unconditionally returns Py_None on success. Line 226 returns the actual ExecutionResult::Return(value), which means if a generator catches GeneratorExit and executes return some_value, close() will surface that value instead of None. This is an extremely rare edge case, but it's a semantic difference from CPython.

If aligning with CPython is desired:

Proposed fix
-            Ok(ExecutionResult::Return(value)) => Ok(value),
+            Ok(ExecutionResult::Return(_)) => Ok(vm.ctx.none()),
🧹 Nitpick comments (4)
crates/vm/src/ospath.rs (1)

330-346: Consider extracting shared logic with with_filename.

with_filename_from_errno duplicates with_filename (lines 318–329) almost entirely — the only difference is the #[cfg(windows)] without_winerror() call. You could extract a common helper and pass a flag or closure for the Windows-specific transform, though given how small these methods are, this is not urgent.

♻️ One way to reduce duplication
+    fn build_with_filename_inner<'a>(
+        error: &std::io::Error,
+        filename: impl Into<OsPathOrFd<'a>>,
+        strip_winerror: bool,
+        vm: &VirtualMachine,
+    ) -> crate::builtins::PyBaseExceptionRef {
+        use crate::exceptions::ToOSErrorBuilder;
+        let builder = error.to_os_error_builder(vm);
+        #[cfg(windows)]
+        let builder = if strip_winerror { builder.without_winerror() } else { builder };
+        let builder = builder.filename(filename.into().filename(vm));
+        builder.build(vm).upcast()
+    }
+
     #[must_use]
     pub(crate) fn with_filename<'a>(
         error: &std::io::Error,
         filename: impl Into<OsPathOrFd<'a>>,
         vm: &VirtualMachine,
     ) -> crate::builtins::PyBaseExceptionRef {
-        use crate::exceptions::ToOSErrorBuilder;
-        let builder = error.to_os_error_builder(vm);
-        let builder = builder.filename(filename.into().filename(vm));
-        builder.build(vm).upcast()
+        Self::build_with_filename_inner(error, filename, false, vm)
     }
 
     #[must_use]
     pub(crate) fn with_filename_from_errno<'a>(
         error: &std::io::Error,
         filename: impl Into<OsPathOrFd<'a>>,
         vm: &VirtualMachine,
     ) -> crate::builtins::PyBaseExceptionRef {
-        use crate::exceptions::ToOSErrorBuilder;
-        let builder = error.to_os_error_builder(vm);
-        #[cfg(windows)]
-        let builder = builder.without_winerror();
-        let builder = builder.filename(filename.into().filename(vm));
-        builder.build(vm).upcast()
+        Self::build_with_filename_inner(error, filename, true, vm)
     }

As per coding guidelines, "When branches differ only in a value but share common logic, extract the differing value first, then call the common logic once to avoid duplicate code."

crates/vm/src/stdlib/itertools.rs (1)

959-983: Reentrancy guard and cache-aware get_item look correct.

The design mirrors CPython's teedataobject pattern well: serve from cache on fast path, guard iterable.next() with an atomic running flag, and append to cache on miss.

One minor observation: on line 982, values[index] relies on the invariant that index is always either < values.len() (served from the fast path on lines 965–967) or == values.len() (pushed on line 980). If this invariant were ever violated (e.g., values.len() < index), it would panic at runtime. Since the tee iterators enforce sequential access, this is safe in practice—same assumption CPython makes—but a debug assertion could catch future regressions:

💡 Optional: add a debug assertion
             if values.len() == index {
                 values.push(obj);
             }
+            debug_assert!(
+                index < values.len(),
+                "tee iterator cache invariant violated: index {index} >= len {}",
+                values.len()
+            );
             Ok(PyIterReturn::Return(values[index].clone()))
crates/vm/src/frame.rs (1)

466-479: Locking self.object.trace on every instruction while tracing is active — acceptable but worth noting.

The self.object.trace.lock() inside the hot loop (line 473) is only reached when vm.use_tracing is set, so normal execution is unaffected. However, when a trace function is active, this acquires the mutex on every instruction — even when the line hasn't changed. For a tighter fast path you could cache whether the per-frame trace is set outside the loop, or at least check the line-change condition before acquiring the lock.

That said, this mirrors CPython's approach (check tracing flags, then consult per-frame state), and tracing is inherently slow, so this is fine for now.

crates/vm/src/protocol/callable.rs (1)

52-69: Unconditional to_owned() clone for non-Python callables even when tracing is off.

Line 60 creates an Arc clone of the callable before checking use_tracing inside trace_event. When tracing is disabled (the overwhelmingly common case), this is a wasted atomic increment/decrement on every non-Python call (builtins, C extensions, etc.).

A quick guard avoids the overhead:

Proposed optimization
         } else {
-            let callable = self.obj.to_owned();
-            vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
-            let result = (self.call)(self.obj, args, vm);
-            if result.is_ok() {
-                vm.trace_event(TraceEvent::CReturn, Some(callable))?;
-            } else {
-                let _ = vm.trace_event(TraceEvent::CException, Some(callable));
+            if vm.use_tracing.get() {
+                let callable = self.obj.to_owned();
+                vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
+                let result = (self.call)(self.obj, args, vm);
+                if result.is_ok() {
+                    vm.trace_event(TraceEvent::CReturn, Some(callable))?;
+                } else {
+                    let _ = vm.trace_event(TraceEvent::CException, Some(callable));
+                }
+                result
+            } else {
+                (self.call)(self.obj, args, vm)
             }
-            result
         }

@youknowone youknowone force-pushed the doctest branch 2 times, most recently from 02f87b3 to c6c4d45 Compare February 11, 2026 03:52
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
crates/vm/src/coroutine.rs (1)

203-228: ⚠️ Potential issue | 🟡 Minor

close() returns the generator's return value instead of always None.

On line 226, Ok(ExecutionResult::Return(value)) => Ok(value) returns whatever the generator produced. In CPython's gen_close (genobject.c), the return value is explicitly discarded and Py_None is always returned. While this may not matter in practice (callers rarely inspect close()'s return value), it's a semantic deviation from CPython.

Consider returning Ok(vm.ctx.none()) for consistency:

Proposed fix
             Ok(ExecutionResult::Yield(_)) => {
                 Err(vm.new_runtime_error(format!("{} ignored GeneratorExit", gen_name(jen, vm))))
             }
             Err(e) if !is_gen_exit(&e, vm) => Err(e),
-            Ok(ExecutionResult::Return(value)) => Ok(value),
+            Ok(ExecutionResult::Return(_)) => Ok(vm.ctx.none()),
             _ => Ok(vm.ctx.none()),
🧹 Nitpick comments (2)
crates/vm/src/stdlib/itertools.rs (1)

959-983: Reentrancy guard logic is sound but not panic-safe.

The two-phase approach (cache-first, then fetch-with-guard) is correct. The running flag properly prevents reentrant calls to the underlying iterator, and the values.len() == index check on line 979 correctly handles the case where another thread already populated the cache entry.

However, if self.iterable.next(vm) on line 973 panics (unwind), running remains true permanently, bricking the tee. Consider using a scope guard (e.g., a drop-based RAII wrapper) to ensure running is reset even on panic:

Suggested panic-safe pattern
             // Prevent concurrent/reentrant calls to iterable.next()
             if self.running.swap(true, Ordering::Acquire) {
                 return Err(vm.new_runtime_error("cannot re-enter the tee iterator"));
             }
-            let result = self.iterable.next(vm);
-            self.running.store(false, Ordering::Release);
+            let result = {
+                struct ResetOnDrop<'a>(&'a AtomicBool);
+                impl Drop for ResetOnDrop<'_> {
+                    fn drop(&mut self) {
+                        self.0.store(false, Ordering::Release);
+                    }
+                }
+                let _guard = ResetOnDrop(&self.running);
+                self.iterable.next(vm)
+            };

This is a minor robustness improvement—VM calls generally shouldn't panic, but a drop guard is a cheap safety net.

crates/vm/src/vm/vm_new.rs (1)

554-560: Redundant call to error.python_location() — consider reusing the earlier result.

python_location() is already called on line 541 but the result is immediately converted to PyInt, making it unavailable here. A small refactor could store the (u32, u32) tuple before converting, avoiding the duplicate call.

♻️ Suggested refactor
-        let (lineno, offset) = error.python_location();
-        let lineno = self.ctx.new_int(lineno);
-        let offset = self.ctx.new_int(offset);
+        let (lineno_raw, offset_raw) = error.python_location();
+        let lineno = self.ctx.new_int(lineno_raw);
+        let offset = self.ctx.new_int(offset_raw);

Then at line 555:

         if let Some((end_lineno, end_offset)) = error.python_end_location() {
             let (end_lineno, end_offset) = if narrow_caret {
-                let (l, o) = error.python_location();
-                (l, o + 1)
+                (lineno_raw, offset_raw + 1)
             } else {
                 (end_lineno, end_offset)
             };

youknowone and others added 4 commits February 11, 2026 16:46
- Add interactive REPL mode: auto-print expression results in single mode
- Implement Destructor for PyGenerator and PyCoroutine
- Add locals_dirty tracking and locals_to_fast() for frame sync
- Add per-line tracing with prev_line tracking in execution loop
- Fix gen_throw to close sub-iterator on GeneratorExit (gen_close_iter)
- Pass callable object as arg in c_call/c_return/c_exception trace events
- Distinguish [Errno] vs [WinError] for CRT vs Win32 API errors
- Fix tee thread safety with AtomicBool running flag
- Fix division error messages to match expected format
@github-actions
Copy link
Contributor

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[x] lib: cpython/Lib/doctest.py
[ ] test: cpython/Lib/test/test_doctest (TODO: 9)

dependencies:

  • doctest

dependent tests: (29 tests)

  • doctest: test_cmd test_code test_collections test_ctypes test_deque test_descrtut test_difflib test_doctest test_doctest2 test_enum test_extcall test_generators test_getopt test_heapq test_http_cookies test_itertools test_listcomps test_metaclass test_pep646_syntax test_pickle test_pickletools test_setcomps test_statistics test_syntax test_threading_local test_unpack test_unpack_ex test_weakref test_zipimport

[ ] lib: cpython/Lib/pdb.py
[ ] test: cpython/Lib/test/test_pdb.py

dependencies:

  • pdb (native: _pyrepl.utils, atexit, builtins, itertools, sys)
    • asyncio (native: _overlapped, _pyrepl.console, _remote_debugging, _winapi, asyncio.tools, collections.abc, concurrent.futures, errno, itertools, math, msvcrt, sys, time)
    • bdb (native: sys)
    • glob (native: itertools, sys)
    • io (native: _io, _thread, errno, sys)
    • json (native: json.tool, sys)
    • stat
    • tempfile (native: _thread, errno, sys)
    • tokenize (native: _tokenize, builtins, itertools, sys)
    • types
    • _colorize, cmd, code, codeop, contextlib, dis, inspect, linecache, os, pprint, re, selectors, signal, socket, textwrap, threading, token, traceback, typing, warnings, weakref

dependent tests: (no tests depend on pdb)

[ ] lib: cpython/Lib/concurrent
[ ] test: cpython/Lib/test/test_concurrent_futures (TODO: 18)
[ ] test: cpython/Lib/test/test_interpreters
[ ] test: cpython/Lib/test/test__interpreters.py
[ ] test: cpython/Lib/test/test__interpchannels.py
[ ] test: cpython/Lib/test/test_crossinterp.py

dependencies:

  • concurrent (native: _interpqueues, _interpreters, concurrent.futures, concurrent.futures._base, itertools, multiprocessing.connection, multiprocessing.queues, sys, time)
    • collections (native: _weakref, itertools, sys)
    • logging (native: atexit, errno, logging.handlers, sys, time)
    • multiprocessing (native: _multiprocessing, _winapi, array, atexit, collections.abc, errno, itertools, mmap, msvcrt, sys, time)
    • pickle (native: itertools, sys)
    • types
    • functools, os, queue, threading, traceback, weakref

dependent tests: (11 tests)

  • concurrent: test_asyncio test_concurrent_futures test_context test_genericalias
    • asyncio: test_asyncio test_contextlib_async test_inspect test_logging test_os test_sys_settrace test_unittest

[x] lib: cpython/Lib/dbm
[ ] test: cpython/Lib/test/test_dbm.py (TODO: 2)
[x] test: cpython/Lib/test/test_dbm_dumb.py
[ ] test: cpython/Lib/test/test_dbm_gnu.py
[ ] test: cpython/Lib/test/test_dbm_ndbm.py
[x] test: cpython/Lib/test/test_dbm_sqlite3.py

dependencies:

  • dbm

dependent tests: (4 tests)

  • dbm: test_dbm test_dbm_dumb test_dbm_sqlite3 test_shelve

[ ] test: cpython/Lib/test/test_extcall.py (TODO: 30)

dependencies:

dependent tests: (no tests depend on extcall)

[ ] test: cpython/Lib/test/test_generators.py (TODO: 12)
[ ] test: cpython/Lib/test/test_genexps.py
[ ] test: cpython/Lib/test/test_generator_stop.py
[ ] test: cpython/Lib/test/test_yield_from.py (TODO: 6)

dependencies:

dependent tests: (no tests depend on generator)

[x] lib: cpython/Lib/inspect.py
[ ] test: cpython/Lib/test/test_inspect (TODO: 50)

dependencies:

  • inspect

dependent tests: (42 tests)

  • inspect: test_abc test_argparse test_asyncgen test_builtin test_code test_collections test_coroutines test_decimal test_enum test_functools test_generators test_grammar test_inspect test_ntpath test_operator test_patma test_posixpath test_signal test_traceback test_type_annotations test_types test_typing test_unittest test_yield_from test_zipimport
    • asyncio: test_asyncio test_contextlib_async test_logging test_os test_sys_settrace test_unittest
    • bdb: test_bdb
    • dataclasses: test__colorize test_genericalias test_pprint test_regrtest test_zoneinfo
    • importlib.metadata: test_importlib
    • pydoc:
      • xmlrpc.server: test_docxmlrpc test_xmlrpc
    • rlcompleter: test_rlcompleter
    • trace: test_trace

[ ] test: cpython/Lib/test/test_raise.py

dependencies:

dependent tests: (no tests depend on raise)

[ ] test: cpython/Lib/test/test_sys.py (TODO: 15)
[ ] test: cpython/Lib/test/test_syslog.py (TODO: 2)
[ ] test: cpython/Lib/test/test_sys_setprofile.py (TODO: 14)
[ ] test: cpython/Lib/test/test_sys_settrace.py (TODO: 85)

dependencies:

dependent tests: (209 tests)

  • sys: regrtestdata test___all__ test__colorize test__locale test__osx_support test_android test_annotationlib test_argparse test_array test_ast test_asyncio test_audit test_bdb test_bigaddrspace test_bigmem test_bisect test_buffer test_builtin test_bytes test_bz2 test_c_locale_coercion test_calendar test_call test_cmath test_cmd test_cmd_line test_cmd_line_script test_code test_code_module test_codeccallbacks test_codecs test_collections test_compile test_compileall test_complex test_concurrent_futures test_context test_contextlib test_coroutines test_csv test_ctypes test_datetime test_dbm test_dbm_sqlite3 test_descr test_dict test_difflib test_dis test_doctest test_doctest2 test_docxmlrpc test_dtrace test_dynamic test_dynamicclassattribute test_email test_ensurepip test_enum test_enumerate test_eof test_except_star test_exceptions test_faulthandler test_fcntl test_file test_file_eintr test_fileinput test_fileio test_float test_fork1 test_format test_fractions test_frozen test_functools test_future_stmt test_generators test_genericpath test_getopt test_glob test_grammar test_gzip test_hash test_hashlib test_http_cookiejar test_httpservers test_importlib test_inspect test_int test_io test_iter test_itertools test_json test_largefile test_list test_locale test_logging test_long test_lzma test_mailbox test_marshal test_math test_memoryio test_memoryview test_metaclass test_mimetypes test_mmap test_msvcrt test_multiprocessing_fork test_multiprocessing_main_handling test_ntpath test_numeric_tower test_operator test_optparse test_ordered_dict test_os test_osx_env test_pathlib test_patma test_peepholer test_pickle test_pkg test_pkgutil test_platform test_plistlib test_posix test_posixpath test_print test_property test_pty test_pwd test_py_compile test_pyclbr test_pyexpat test_quopri test_raise test_range test_re test_regrtest test_repl test_reprlib test_resource test_runpy test_script_helper test_select test_selectors test_shutil test_signal test_site test_slice test_smtplib test_socket test_sqlite3 test_ssl test_stat test_statistics test_str test_strftime test_string_literals test_strtod test_struct test_subprocess test_support test_symtable test_sys test_sys_setprofile test_sys_settrace test_sysconfig test_syslog test_tarfile test_tempfile test_termios test_threadedtempfile test_threading test_threading_local test_threadsignals test_time test_timeit test_tomllib test_tools test_trace test_traceback test_type_comments test_types test_typing test_unicode_file test_unicode_file_functions test_unicodedata test_unittest test_univnewlines test_urllib test_urllib2 test_urllib2net test_urlparse test_utf8_mode test_uuid test_venv test_wait3 test_wait4 test_wave test_weakref test_webbrowser test_with test_wsgiref test_xml_etree test_xmlrpc test_zipapp test_zipfile test_zipfile64 test_zipimport test_zlib

[ ] lib: cpython/Lib/trace.py
[ ] test: cpython/Lib/test/test_trace.py (TODO: 14)

dependencies:

  • trace (native: gc, sys, time)
    • sysconfig (native: os.path, sys)
    • io, pickle, tokenize
    • dis, inspect, linecache, os, threading, token

dependent tests: (1 tests)

  • trace: test_trace

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
crates/vm/src/builtins/int.rs (2)

123-145: ⚠️ Potential issue | 🟠 Major

Error messages diverge from CPython — this may break doctest compatibility.

CPython uses "integer division or modulo by zero" for // and divmod, and "integer modulo by zero" for %. Only true division (/) uses "division by zero". Unifying all three to "division by zero" will cause any doctest that captures the exact ZeroDivisionError message for %, //, or divmod to fail.

The old inner_divmod message reportedly had a typo ("integer division orModulo by zero"), so fixing that is welcome — but the replacement should match CPython.

Proposed fix to match CPython messages
 fn inner_mod(int1: &BigInt, int2: &BigInt, vm: &VirtualMachine) -> PyResult {
     if int2.is_zero() {
-        Err(vm.new_zero_division_error("division by zero"))
+        Err(vm.new_zero_division_error("integer modulo by zero"))
     } else {
         Ok(vm.ctx.new_int(int1.mod_floor(int2)).into())
     }
 }

 fn inner_floordiv(int1: &BigInt, int2: &BigInt, vm: &VirtualMachine) -> PyResult {
     if int2.is_zero() {
-        Err(vm.new_zero_division_error("division by zero"))
+        Err(vm.new_zero_division_error("integer division or modulo by zero"))
     } else {
         Ok(vm.ctx.new_int(int1.div_floor(int2)).into())
     }
 }

 fn inner_divmod(int1: &BigInt, int2: &BigInt, vm: &VirtualMachine) -> PyResult {
     if int2.is_zero() {
-        return Err(vm.new_zero_division_error("division by zero"));
+        return Err(vm.new_zero_division_error("integer division or modulo by zero"));
     }

123-145: ⚠️ Potential issue | 🔴 Critical

Fix error messages for integer zero-division operations to match CPython.

CPython uses distinct error messages for integer operations:

  • % (modulo): "integer modulo by zero" or "integer division or modulo by zero"
  • // (floor division): "integer division or modulo by zero"
  • divmod(): "integer division or modulo by zero"
  • / (true division): "division by zero"

The current code incorrectly uses the generic "division by zero" for all three integer operations (inner_mod, inner_floordiv, inner_divmod), which breaks CPython compatibility. This is especially problematic for doctests, which validate exact exception messages. Update these functions to use the correct messages matching CPython's behavior.

🤖 Fix all issues with AI agents
In `@crates/vm/src/frame.rs`:
- Around line 261-281: locals_to_fast currently leaves fastlocals[i] untouched
when locals.mapping().subscript(varname, vm) returns a KeyError, but it should
clear the fast slot like CPython's PyFrame_LocalsToFast; update the match in
locals_to_fast so that Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error)
sets fastlocals[i] = None (unbound) instead of doing nothing, ensuring deleted
keys in f_locals unbind the corresponding fastlocal; keep the existing Err(e) =>
return Err(e) path and retain locals_dirty store/release logic.

In `@crates/vm/src/protocol/callable.rs`:
- Around line 52-69: The code currently always calls self.obj.to_owned() for
non-Python callables, bumping refcounts even when tracing is disabled; change
the logic to first check the VM tracing flag (e.g., vm.use_tracing) and only
call self.obj.to_owned() when tracing is enabled, then call
vm.trace_event(TraceEvent::CCall, Some(callable.clone())) before invoking
(self.call) and vm.trace_event(TraceEvent::CReturn) /
vm.trace_event(TraceEvent::CException) afterwards using that owned `callable`;
otherwise, when tracing is disabled, directly invoke (self.call)(self.obj, args,
vm) without calling self.obj.to_owned() or any clone to avoid the hot-path
refcount bump.

Comment on lines +261 to +281
/// Sync locals dict back to fastlocals. Called before generator/coroutine resume
/// to apply any modifications made via f_locals.
pub fn locals_to_fast(&self, vm: &VirtualMachine) -> PyResult<()> {
if !self.locals_dirty.load(atomic::Ordering::Acquire) {
return Ok(());
}
let code = &**self.code;
let mut fastlocals = self.fastlocals.lock();
for (i, &varname) in code.varnames.iter().enumerate() {
if i >= fastlocals.len() {
break;
}
match self.locals.mapping().subscript(varname, vm) {
Ok(value) => fastlocals[i] = Some(value),
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
}
self.locals_dirty.store(false, atomic::Ordering::Release);
Ok(())
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

locals_to_fast does not clear fastlocals when a key is deleted from f_locals.

When a varname is missing from the locals dict (KeyError on line 275), the corresponding fastlocals[i] slot is left unchanged. In CPython's PyFrame_LocalsToFast, a missing key causes the fast slot to be set to NULL (i.e., the variable becomes unbound). This means del frame.f_locals['x'] followed by a generator resume won't actually unbind x.

If this is intentional (e.g., scoped to what doctest needs), a comment would help. Otherwise:

Proposed fix
             match self.locals.mapping().subscript(varname, vm) {
                 Ok(value) => fastlocals[i] = Some(value),
-                Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
+                Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {
+                    fastlocals[i] = None;
+                }
                 Err(e) => return Err(e),
             }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
/// Sync locals dict back to fastlocals. Called before generator/coroutine resume
/// to apply any modifications made via f_locals.
pub fn locals_to_fast(&self, vm: &VirtualMachine) -> PyResult<()> {
if !self.locals_dirty.load(atomic::Ordering::Acquire) {
return Ok(());
}
let code = &**self.code;
let mut fastlocals = self.fastlocals.lock();
for (i, &varname) in code.varnames.iter().enumerate() {
if i >= fastlocals.len() {
break;
}
match self.locals.mapping().subscript(varname, vm) {
Ok(value) => fastlocals[i] = Some(value),
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {}
Err(e) => return Err(e),
}
}
self.locals_dirty.store(false, atomic::Ordering::Release);
Ok(())
}
/// Sync locals dict back to fastlocals. Called before generator/coroutine resume
/// to apply any modifications made via f_locals.
pub fn locals_to_fast(&self, vm: &VirtualMachine) -> PyResult<()> {
if !self.locals_dirty.load(atomic::Ordering::Acquire) {
return Ok(());
}
let code = &**self.code;
let mut fastlocals = self.fastlocals.lock();
for (i, &varname) in code.varnames.iter().enumerate() {
if i >= fastlocals.len() {
break;
}
match self.locals.mapping().subscript(varname, vm) {
Ok(value) => fastlocals[i] = Some(value),
Err(e) if e.fast_isinstance(vm.ctx.exceptions.key_error) => {
fastlocals[i] = None;
}
Err(e) => return Err(e),
}
}
self.locals_dirty.store(false, atomic::Ordering::Release);
Ok(())
}
🤖 Prompt for AI Agents
In `@crates/vm/src/frame.rs` around lines 261 - 281, locals_to_fast currently
leaves fastlocals[i] untouched when locals.mapping().subscript(varname, vm)
returns a KeyError, but it should clear the fast slot like CPython's
PyFrame_LocalsToFast; update the match in locals_to_fast so that Err(e) if
e.fast_isinstance(vm.ctx.exceptions.key_error) sets fastlocals[i] = None
(unbound) instead of doing nothing, ensuring deleted keys in f_locals unbind the
corresponding fastlocal; keep the existing Err(e) => return Err(e) path and
retain locals_dirty store/release logic.

Comment on lines +52 to +69
// Python functions get 'call'/'return' events from with_frame().
// Bound methods delegate to the inner callable, which fires its own events.
// All other callables (built-in functions, etc.) get 'c_call'/'c_return'/'c_exception'.
let is_python_callable = self.obj.downcast_ref::<PyFunction>().is_some()
|| self.obj.downcast_ref::<PyBoundMethod>().is_some();
if is_python_callable {
(self.call)(self.obj, args, vm)
} else {
let callable = self.obj.to_owned();
vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
let result = (self.call)(self.obj, args, vm);
if result.is_ok() {
vm.trace_event(TraceEvent::CReturn, Some(callable))?;
} else {
let _ = vm.trace_event(TraceEvent::CException, Some(callable));
}
result
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🛠️ Refactor suggestion | 🟠 Major

Unconditional clone of the callable on every non-Python call, even when tracing is off.

self.obj.to_owned() (line 60) bumps the refcount for every built-in call regardless of whether tracing is active. Since vm.use_tracing is usually false, gate the tracing path behind the check:

Proposed fix
         if is_python_callable {
             (self.call)(self.obj, args, vm)
+        } else if vm.use_tracing.get() {
+            let callable = self.obj.to_owned();
+            vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
+            let result = (self.call)(self.obj, args, vm);
+            if result.is_ok() {
+                vm.trace_event(TraceEvent::CReturn, Some(callable))?;
+            } else {
+                let _ = vm.trace_event(TraceEvent::CException, Some(callable));
+            }
+            result
         } else {
-            let callable = self.obj.to_owned();
-            vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
-            let result = (self.call)(self.obj, args, vm);
-            if result.is_ok() {
-                vm.trace_event(TraceEvent::CReturn, Some(callable))?;
-            } else {
-                let _ = vm.trace_event(TraceEvent::CException, Some(callable));
-            }
-            result
+            (self.call)(self.obj, args, vm)
         }

This avoids the to_owned() + clone() on the hot path when no tracer is registered.

📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// Python functions get 'call'/'return' events from with_frame().
// Bound methods delegate to the inner callable, which fires its own events.
// All other callables (built-in functions, etc.) get 'c_call'/'c_return'/'c_exception'.
let is_python_callable = self.obj.downcast_ref::<PyFunction>().is_some()
|| self.obj.downcast_ref::<PyBoundMethod>().is_some();
if is_python_callable {
(self.call)(self.obj, args, vm)
} else {
let callable = self.obj.to_owned();
vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
let result = (self.call)(self.obj, args, vm);
if result.is_ok() {
vm.trace_event(TraceEvent::CReturn, Some(callable))?;
} else {
let _ = vm.trace_event(TraceEvent::CException, Some(callable));
}
result
}
// Python functions get 'call'/'return' events from with_frame().
// Bound methods delegate to the inner callable, which fires its own events.
// All other callables (built-in functions, etc.) get 'c_call'/'c_return'/'c_exception'.
let is_python_callable = self.obj.downcast_ref::<PyFunction>().is_some()
|| self.obj.downcast_ref::<PyBoundMethod>().is_some();
if is_python_callable {
(self.call)(self.obj, args, vm)
} else if vm.use_tracing.get() {
let callable = self.obj.to_owned();
vm.trace_event(TraceEvent::CCall, Some(callable.clone()))?;
let result = (self.call)(self.obj, args, vm);
if result.is_ok() {
vm.trace_event(TraceEvent::CReturn, Some(callable))?;
} else {
let _ = vm.trace_event(TraceEvent::CException, Some(callable));
}
result
} else {
(self.call)(self.obj, args, vm)
}
🤖 Prompt for AI Agents
In `@crates/vm/src/protocol/callable.rs` around lines 52 - 69, The code currently
always calls self.obj.to_owned() for non-Python callables, bumping refcounts
even when tracing is disabled; change the logic to first check the VM tracing
flag (e.g., vm.use_tracing) and only call self.obj.to_owned() when tracing is
enabled, then call vm.trace_event(TraceEvent::CCall, Some(callable.clone()))
before invoking (self.call) and vm.trace_event(TraceEvent::CReturn) /
vm.trace_event(TraceEvent::CException) afterwards using that owned `callable`;
otherwise, when tracing is disabled, directly invoke (self.call)(self.obj, args,
vm) without calling self.obj.to_owned() or any clone to avoid the hot-path
refcount bump.

@youknowone youknowone changed the title Doctest Update doctest,test_generators from v3.14.2 and fix generator bugs Feb 11, 2026
@youknowone youknowone merged commit fa59402 into RustPython:main Feb 11, 2026
14 checks passed
@youknowone youknowone deleted the doctest branch February 11, 2026 08:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant