Skip to content

dealloc and finalize_modules #6934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
youknowone merged 3 commits into from
Feb 2, 2026
Merged

dealloc and finalize_modules #6934

youknowone merged 3 commits into from
Feb 2, 2026

Conversation

@youknowone
Copy link
Member

@youknowone youknowone commented Feb 1, 2026 •
edited by coderabbitai bot
Loading

Replace the absence of module finalization during interpreter shutdown with a 5-phase algorithm matching pylifecycle.c finalize_modules():

  1. Set special sys attributes to None, restore stdio
  2. Set all sys.modules values to None, collect module dicts
  3. Clear sys.modules dict
  4. Clear module dicts in reverse import order (2-pass _PyModule_ClearDict)
  5. Clear sys and builtins dicts last

This ensures del methods are called during shutdown and modules are cleaned up in reverse import order without hardcoded module names.

Summary by CodeRabbit

  • New Features

    • Added a staged VM shutdown that more thoroughly finalizes and clears modules to improve cleanup during exit.

  • Bug Fixes

    • Improved object deallocation and finalization flow to better handle destructors, weak references, and object lifecycle edge cases.

  • Chores

    • Updated spell-check dictionary to recognize an additional Python-related token.

✏️ Tip: You can customize this high-level summary in your review settings.

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Feb 1, 2026 •
edited
Loading

📝 Walkthrough

Walkthrough

Adds a staged module-finalization flow via a new VirtualMachine::finalize_modules() invoked from Interpreter::finalize, refactors object deallocation to use a new default_dealloc routed through the vtable, and adds a pylifecycle token to the spell-check dictionary.

Changes

Cohort / File(s) Summary
Module finalization
crates/vm/src/vm/mod.rs, crates/vm/src/vm/interpreter.rs		 Introduce VirtualMachine::finalize_modules() and helper steps; interpreter finalization now calls it after atexit. Implements phased nullification and staged clearing of sys.modules and module dicts in reverse import order.
Object deallocation / vtable
crates/vm/src/object/core.rs, crates/vm/src/object/traverse_object.rs		 Replace drop_dealloc_obj with default_dealloc; vtable field renamed from drop_dealloc to dealloc and initialized to default_dealloc::<T>. Deallocation now dispatched via vtable.
Spell-check dictionary
.cspell.dict/cpython.txt		 Add new token pylifecycle to the public cspell dictionary.

Sequence Diagram

sequenceDiagram
    participant Interp as Interpreter
    participant VM as VirtualMachine
    participant Sys as sys.modules
    participant Builtins as builtins
    participant Mods as ModuleDicts

    Interp->>VM: finalize_modules()
    activate VM
    VM->>Sys: Phase 1 — clear special sys attrs / restore stdio
    VM->>VM: Phase 2 — nullify sys.modules entries, collect weak refs
    VM->>Sys: Phase 3 — clear sys.modules dict
    VM->>Mods: Phase 4 — clear module dicts in reverse import order
    Mods->>Mods: two-pass clear (safeguard __main__)
    VM->>Builtins: Phase 5 — clear sys and builtins dicts
    VM-->>Interp: return
    deactivate VM
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Poem

🐰
Hop, hop, I tidy each module in line,
Reverse the imports, clear references fine,
Vtable whispers, dealloc takes flight,
Pylifecycle token hops into sight,
A rabbit’s shuffle — shutdown done right!

🚥 Pre-merge checks | ✅ 3
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Title check ✅ Passed The title 'dealloc and finalize_modules' covers both main changes in the PR: the refactoring of deallocation logic and the addition of the 5-phase module finalization algorithm, but is somewhat generic and does not emphasize which change is primary.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2026

Code has been automatically formatted

The code in this PR has been formatted using:

  • cargo fmt --all
    Please pull the latest changes before pushing again:
git pull origin finalize-modules

@youknowone youknowone marked this pull request as ready for review February 1, 2026 10:42
youknowone and others added 3 commits February 2, 2026 01:27
@youknowone
rewrite finalize_modules with phased algorithm
59739e6 
Replace the absence of module finalization during interpreter shutdown
with a 5-phase algorithm matching pylifecycle.c finalize_modules():

1. Set special sys attributes to None, restore stdio
2. Set all sys.modules values to None, collect module dicts
3. Clear sys.modules dict
4. Clear module dicts in reverse import order (2-pass _PyModule_ClearDict)
5. Clear sys and builtins dicts last

This ensures __del__ methods are called during shutdown and modules are
cleaned up in reverse import order without hardcoded module names.
@youknowone
dealloc the rigth way
bf76333
@youknowone @claude
fix finalize_modules: only clear __main__ dict, mark daemon thread te…
0db8e53 
…sts as expected failure

Without GC, clearing all module dicts during finalization causes __del__
handlers to fail (globals are None). Restrict Phase 4 to only clear
__main__ dict — other modules' globals stay intact for their __del__
handlers.

Mark test_daemon_threads_shutdown_{stdout,stderr}_deadlock as expected
failures — without GC+GIL, finalize_modules clears __main__ globals
while daemon threads are still running.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2026

📦 Library Dependencies

The following Lib/ modules were modified. Here are their dependencies:

[ ] test: cpython/Lib/test/test_builtin.py (TODO: 17)

dependencies:

dependent tests: (no tests depend on builtin)

[ ] lib: cpython/Lib/io.py
[ ] lib: cpython/Lib/_pyio.py
[ ] test: cpython/Lib/test/test_io.py (TODO: 62)
[ ] test: cpython/Lib/test/test_bufio.py (TODO: 2)
[ ] test: cpython/Lib/test/test_fileio.py
[ ] test: cpython/Lib/test/test_memoryio.py (TODO: 26)

dependencies:

  • io (native: _io, _thread, errno, sys)
    • _pyio
    • os (native: os.path, sys)
    • _collections_abc, abc, codecs, stat

dependent tests: (87 tests)

  • io: test__colorize test_android test_argparse test_ast test_asyncio test_bufio test_builtin test_bz2 test_calendar test_cmd test_cmd_line_script test_codecs test_compileall test_compiler_assemble test_concurrent_futures test_configparser test_contextlib test_csv test_dbm_dumb test_dis test_email test_enum test_file test_fileinput test_fileio test_ftplib test_getpass test_gzip test_hashlib test_httplib test_httpservers test_importlib test_inspect test_io test_json test_largefile test_logging test_lzma test_mailbox test_marshal test_memoryio test_memoryview test_mimetypes test_optparse test_pathlib test_pickle test_pickletools test_plistlib test_pprint test_print test_pty test_pulldom test_pyexpat test_regrtest test_robotparser test_shlex test_shutil test_site test_smtplib test_socket test_socketserver test_subprocess test_support test_sys test_tarfile test_tempfile test_threadedtempfile test_timeit test_tokenize test_traceback test_typing test_unittest test_univnewlines test_urllib test_urllib2 test_uuid test_wave test_wsgiref test_xml_dom_xmlbuilder test_xml_etree test_xml_etree_c test_xmlrpc test_zipapp test_zipfile test_zipimport test_zoneinfo test_zstd

[ ] test: cpython/Lib/test/test_sys.py (TODO: 16)
[ ] test: cpython/Lib/test/test_syslog.py (TODO: 2)
[ ] test: cpython/Lib/test/test_sys_setprofile.py (TODO: 22)
[ ] test: cpython/Lib/test/test_sys_settrace.py (TODO: 85)

dependencies:

dependent tests: (208 tests)

  • sys: regrtestdata test___all__ test__colorize test__locale test__osx_support test_android test_argparse test_array test_ast test_asyncio test_audit test_bdb test_bigaddrspace test_bigmem test_bisect test_buffer test_bufio test_builtin test_bytes test_bz2 test_c_locale_coercion test_calendar test_call test_cmath test_cmd test_cmd_line test_cmd_line_script test_code test_code_module test_codeccallbacks test_codecs test_collections test_compile test_compileall test_complex test_concurrent_futures test_context test_contextlib test_coroutines test_csv test_ctypes test_datetime test_dbm_sqlite3 test_descr test_dict test_difflib test_dis test_doctest test_doctest2 test_docxmlrpc test_dtrace test_dynamic test_dynamicclassattribute test_email test_ensurepip test_enum test_enumerate test_eof test_except_star test_exceptions test_faulthandler test_fcntl test_file test_file_eintr test_fileinput test_fileio test_float test_fork1 test_format test_fractions test_frozen test_functools test_future_stmt test_generators test_genericpath test_getopt test_glob test_grammar test_gzip test_hash test_hashlib test_http_cookiejar test_httpservers test_importlib test_inspect test_int test_io test_iter test_itertools test_json test_largefile test_list test_locale test_logging test_long test_lzma test_mailbox test_marshal test_math test_memoryio test_memoryview test_metaclass test_mimetypes test_mmap test_msvcrt test_multiprocessing_fork test_multiprocessing_main_handling test_ntpath test_numeric_tower test_operator test_optparse test_ordered_dict test_os test_osx_env test_pathlib test_patma test_peepholer test_pickle test_pkg test_pkgutil test_platform test_plistlib test_posix test_posixpath test_print test_property test_pty test_pwd test_py_compile test_pyclbr test_pyexpat test_queue test_quopri test_raise test_range test_re test_regrtest test_repl test_reprlib test_resource test_runpy test_script_helper test_select test_selectors test_shutil test_signal test_site test_slice test_smtplib test_socket test_sqlite3 test_ssl test_stat test_statistics test_str test_strftime test_string_literals test_strtod test_struct test_subprocess test_support test_symtable test_sys test_sys_setprofile test_sys_settrace test_sysconfig test_syslog test_tarfile test_tempfile test_termios test_threadedtempfile test_threading test_threading_local test_time test_timeit test_tomllib test_tools test_trace test_traceback test_type_comments test_types test_typing test_unicode_file test_unicode_file_functions test_unicodedata test_unittest test_univnewlines test_urllib test_urllib2 test_urllib2net test_urlparse test_utf8_mode test_uuid test_venv test_wait3 test_wait4 test_wave test_weakref test_webbrowser test_with test_wsgiref test_xml_etree test_xmlrpc test_zipapp test_zipfile test_zipfile64 test_zipimport test_zlib

Legend:

  • [+] path exists in CPython
  • [x] up-to-date, [ ] outdated

@youknowone youknowone changed the title finalize_modules dealloc and finalize_modules Feb 2, 2026
@youknowone youknowone merged commit 7004502 into RustPython:main Feb 2, 2026
13 of 14 checks passed
@youknowone youknowone deleted the finalize-modules branch February 2, 2026 01:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@youknowone