|
| 1 | +--- |
| 2 | +name: upgrade-pylib |
| 3 | +description: Upgrade a Python standard library module from CPython into RustPython using scripts/update_lib and then triage and mark remaining failures. |
| 4 | +allowed-tools: Bash(git add:*) Bash(git commit:*) Bash(git diff:*) Bash(cargo run:*) Bash(python3 scripts/update_lib quick:*) Bash(python3 scripts/update_lib auto-mark:*) |
| 5 | +--- |
| 6 | + |
| 7 | +# Upgrade Python Library from CPython |
| 8 | + |
| 9 | +Upgrade a Python standard library module from CPython to RustPython. |
| 10 | + |
| 11 | +## Arguments |
| 12 | + |
| 13 | +- Library name or path (examples: `inspect`, `asyncio`, `json`, `cpython/Lib/inspect.py`, `cpython/Lib/json/`) |
| 14 | + |
| 15 | +## Important: Report Tool Issues First |
| 16 | + |
| 17 | +If `scripts/update_lib` shows a bug, missing automation, or unexpected behavior, stop the upgrade and report the tooling issue first, including: |
| 18 | + |
| 19 | +1. What you were trying to do: |
| 20 | + |
| 21 | +- Library name. |
| 22 | +- Full command used. |
| 23 | + |
| 24 | +2. What went wrong or what is missing. |
| 25 | +3. Expected vs actual behavior. |
| 26 | + |
| 27 | +## Workflow |
| 28 | + |
| 29 | +1. Run quick upgrade with update_lib: |
| 30 | + |
| 31 | +- `python3 scripts/update_lib quick <module>` |
| 32 | +- or `python3 scripts/update_lib quick cpython/Lib/<module>.py` |
| 33 | +- or `python3 scripts/update_lib quick cpython/Lib/<module>/` |
| 34 | + |
| 35 | +This step copies library files, patches tests while preserving markers where possible, runs tests, auto-marks new failures, removes stale `@unittest.expectedFailure`, and creates a commit. |
| 36 | + |
| 37 | +If warnings like `WARNING: TestCFoo does not exist in remote file` appear, class structure changed and some markers must be restored manually. |
| 38 | + |
| 39 | +2. Review diff and restore RustPython-specific changes: |
| 40 | + |
| 41 | +- Run `git diff Lib/test/test_<module>`. |
| 42 | +- Restore only changes with explicit `RUSTPYTHON` comments. |
| 43 | +- Do not restore unrelated upstream CPython changes. |
| 44 | + |
| 45 | +3. Investigate failing dependent tests: |
| 46 | + |
| 47 | +- Get dependencies: |
| 48 | + |
| 49 | + ```shell |
| 50 | + cargo run --release -- scripts/update_lib deps <module> |
| 51 | + ``` |
| 52 | + |
| 53 | +- Find direct dependent test modules from the `- [ ] <module>:` line. |
| 54 | +- Run dependent tests and collect failures: |
| 55 | + |
| 56 | + ```shell |
| 57 | + cargo run --release -- -m test <test_modules...> 2>&1 | grep -E "^(FAIL|ERROR):" |
| 58 | + ``` |
| 59 | + |
| 60 | +- For each failing test identifier, investigate with the `investigate-test-failure` workflow. |
| 61 | + |
| 62 | +4. Mark remaining failures with auto-mark: |
| 63 | + |
| 64 | +- `python3 scripts/update_lib auto-mark Lib/test/test_<module>.py --mark-failure` |
| 65 | +- or `python3 scripts/update_lib auto-mark Lib/test/test_<module>/ --mark-failure` |
| 66 | + |
| 67 | +Note: `--mark-failure` marks all current failures including regressions; review carefully. |
| 68 | + |
| 69 | +5. Handle panics manually: |
| 70 | + |
| 71 | +- For tests that panic/crash/hang, use `@unittest.skip("TODO: RUSTPYTHON; ...")` rather than expectedFailure. |
| 72 | + |
| 73 | +6. Handle class-specific failures: |
| 74 | + |
| 75 | +- If only one subclass fails (for example `TestCFoo` but not `TestPyFoo`), move marker to the failing subclass override. |
| 76 | + |
| 77 | +7. Commit test-fix markers as a separate commit: |
| 78 | + |
| 79 | +```shell |
| 80 | +git add -u && git commit -m "Mark failing tests" |
| 81 | +``` |
| 82 | + |
| 83 | +## Example usage |
| 84 | + |
| 85 | +```text |
| 86 | +upgrade-pylib inspect |
| 87 | +upgrade-pylib json |
| 88 | +upgrade-pylib asyncio |
| 89 | +upgrade-pylib cpython/Lib/inspect.py |
| 90 | +upgrade-pylib cpython/Lib/json/ |
| 91 | +``` |
| 92 | + |
| 93 | +## Notes |
| 94 | + |
| 95 | +- The `cpython/` directory should contain the CPython source used for sync. |
| 96 | +- `scripts/update_lib` modes: |
| 97 | + - `quick`: patch + auto-mark |
| 98 | + - `migrate`: patch only |
| 99 | + - `auto-mark`: mark failures only |
| 100 | + - `copy-lib`: copy library files only |
| 101 | +- Patching transfers `@unittest.expectedFailure` and `@unittest.skip` with `TODO: RUSTPYTHON` markers where possible. |
| 102 | +- The tool does not preserve every RustPython-specific change; always review and restore explicit RUSTPYTHON-marked logic. |
0 commit comments