Skip to content

Commit a6b4ef7

Browse files
etaloofqingshi163youknowonecoolreader18
authored
Replace Python parser with ruff parser (RustPython#5494)
* stage1 * compiler pass build * introduce rustpython-compiler-source * stage2 * fixup * pass compile * Fix hello world compiler test * Fix code generation for if-elif-else statement * Fix code generation for lambda expression * Fix code generation for integers * Fix code generation for fstrings * Fix code generation for if statement * Fix code generation for if statement * Fix code generation for if statement * Fix code generation for fstring * Fix code generation for class definition * Replace feature flags * Initialize frozen core modules * Allow __future__ import after module doc comment * Disable ast module * Commit remaining fixes for compile errors in examples * Fix some warnings * Update ast stdlib module * Update ast stdlib module * Update ast stdlib module * Update ast stdlib module * Update ast stdlib module * Split ast stdlib module into files * Fix codegen for positional arguments with defaults * Update ast stdlib module * Update ast stdlib module * Extract string and constant handling from expression.rs * Always add required fields to AST nodes * Compile doc strings correctly again * Enable "ast" Cargo feature by default * Refactor compilation of big integer literal * Update ast stdlib module * Update ast stdlib module * Update ast stdlib module * Reset barebones example * Fix some left-over warnings * Undo accidential change * Adapt shell to ruff parser * Pin parser to v0.4.10 * fix clippy * Add TODO about interactive mode * Fix compilation of complex number expression * Remove moved code * Update test case to ruff v0.4.10 * Apply suggestion Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> * Apply suggestion Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> * Fix compilation of fstring expression * Fix compilation of fstring expression * Fix wasm compile errors * Attach correct source locations to ast objects * Fix some more wasm compile errors * Consider compile mode and enable AST stdlib module again * Fix incorrect AST source location end column * Fix compile error if "compiler" feature is not enabled * Fix regrtests * Fix some test_ast tests * Add source range to type ignore * Fix incompatibility with Rust 2024 edition * Fix todos by implementing missing ast conversions and deleting unused code * Appease clippy * Fix remaining ast tests * Fix remaining ast tests * Mark/fix remaining tests * Fix more * Hacky windows fix --------- Co-authored-by: Kangzhi Shi <shikangzhi@gmail.com> Co-authored-by: Jeong YunWon <jeong@youknowone.org> Co-authored-by: Jeong, YunWon <69878+youknowone@users.noreply.github.com> Co-authored-by: Noa <coolreader18@gmail.com>
1 parent 45c0fa0 commit a6b4ef7

70 files changed

Lines changed: 9981 additions & 6858 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎Cargo.lock‎

Lines changed: 125 additions & 91 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

‎Cargo.toml‎

Lines changed: 11 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -28,7 +28,7 @@ rustpython-compiler = { workspace = true }
2828
rustpython-pylib = { workspace = true, optional = true }
2929
rustpython-stdlib = { workspace = true, optional = true, features = ["compiler"] }
3030
rustpython-vm = { workspace = true, features = ["compiler"] }
31-
rustpython-parser = { workspace = true }
31+
ruff_python_parser = { workspace = true }
3232

3333
cfg-if = { workspace = true }
3434
log = { workspace = true }
@@ -94,7 +94,7 @@ x86_64-pc-windows-msvc = { triplet = "x64-windows-static-md", dev-dependencies =
9494
[workspace]
9595
resolver = "2"
9696
members = [
97-
"compiler", "compiler/core", "compiler/codegen",
97+
"compiler", "compiler/core", "compiler/codegen", "compiler/source",
9898
".", "common", "derive", "jit", "vm", "vm/sre_engine", "pylib", "stdlib", "derive-impl",
9999
"wasm/lib",
100100
]
@@ -108,6 +108,7 @@ repository = "https://github.com/RustPython/RustPython"
108108
license = "MIT"
109109

110110
[workspace.dependencies]
111+
rustpython-compiler-source = { path = "compiler/source" }
111112
rustpython-compiler-core = { path = "compiler/core", version = "0.4.0" }
112113
rustpython-compiler = { path = "compiler", version = "0.4.0" }
113114
rustpython-codegen = { path = "compiler/codegen", version = "0.4.0" }
@@ -121,15 +122,20 @@ rustpython-stdlib = { path = "stdlib", default-features = false, version = "0.4.
121122
rustpython-sre_engine = { path = "vm/sre_engine", version = "0.4.0" }
122123
rustpython-doc = { git = "https://github.com/RustPython/__doc__", tag = "0.3.0", version = "0.3.0" }
123124

125+
ruff_python_parser = { git = "https://github.com/astral-sh/ruff.git", tag = "v0.4.10" }
126+
ruff_python_ast = { git = "https://github.com/astral-sh/ruff.git", tag = "v0.4.10" }
127+
ruff_text_size = { git = "https://github.com/astral-sh/ruff.git", tag = "v0.4.10" }
128+
ruff_source_file = { git = "https://github.com/astral-sh/ruff.git", tag = "v0.4.10" }
129+
ruff_python_codegen = { git = "https://github.com/astral-sh/ruff.git", tag = "v0.4.10" }
124130
# rustpython-literal = { version = "0.4.0" }
125131
# rustpython-parser-core = { version = "0.4.0" }
126132
# rustpython-parser = { version = "0.4.0" }
127133
# rustpython-ast = { version = "0.4.0" }
128134
# rustpython-format= { version = "0.4.0" }
129135
rustpython-literal = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
130-
rustpython-parser-core = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
131-
rustpython-parser = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
132-
rustpython-ast = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
136+
# rustpython-parser-core = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
137+
# rustpython-parser = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
138+
# rustpython-ast = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
133139
rustpython-format = { git = "https://github.com/RustPython/Parser.git", version = "0.4.0", rev = "d2f137b372ec08ce4a243564a80f8f9153c45a23" }
134140
# rustpython-literal = { path = "../RustPython-parser/literal" }
135141
# rustpython-parser-core = { path = "../RustPython-parser/core" }

0 commit comments

Comments
 (0)