Repository navigation
Speedup compile time for rustpython-parser - #3652
Conversation
ec63ea3 to
9649ea5
Compare
|
|
||
| fn check_lalrpop() { | ||
| println!("cargo:rerun-if-changed=src/python.lalrpop"); | ||
| let sha3_line = BufReader::with_capacity(128, File::open("src/python.rs").unwrap()) |
There was a problem hiding this comment.
If we are going to generate this file, I think the output is expected not to be placed in repository but loaded form CI cache.
There was a problem hiding this comment.
I think it's fine to check it into the repo; while different cause it's a bootstrapping issue, lalrpop itself checks its generated parser into its repo, I feel like it's a decently common practice. While I do really like the idea of having everything necessary for the build be generated at build-time, that comes in conflict with the reality of longer build times. If it verifies that the generated file is up-to-date, I don't think I really see an issue with it.
There was a problem hiding this comment.
I think that's not because of lalrpop convention but a typical compiler bootstrapping issue. If they don't bundle generated parser to their repository, people who don't have lalrpop yet will never can generate lalrpop parser by themselves.
I am not sure if I am little bit out of trends, but as I understand, traditional software communities only commit the original source to the repository but attaching the configuration result to their source code distribution. We usually now don't distribute source code anymore. So we'd better to invest more on easier build from source, but still we can avoid to bundle the generated output itself when possible.
There was a problem hiding this comment.
practically, once after building the parser first, it will not take long time again because contributors and users never edit parsers. So this is concerning about CI, not the users or contributors, right?
then I think we can build this from CI and cache the result. There will be another dependency, but it will not be a big deal after first build because cache will be shared between the different CI runs.
There was a problem hiding this comment.
Another advantage to including the parser in the repository is that we can see how much of it is covered by our test suite.
There was a problem hiding this comment.
That point makes sense. We need rustfmt in that case.
|
Achh, newline issues on windows. |
TAKE MY BENCHMARKS WITH A GRAIN OF SALTHP Elitebook 745 G3 Current
|
| Targets: | rustpython 0.1.2 (lib, bin "rustpython") |
|---|---|
| Profile: | release |
| Fresh units: | 0 |
| Dirty units: | 375 |
| Total units: | 375 |
| Max concurrency: | 4 (jobs=4 ncpu=4) |
| Build start: | 2022-04-23T01:04:52Z |
| Total time: | 726.9s (12m 6.9s) |
| rustc: | rustc 1.60.0 (7737e0b5c 2022-04-04) Host: x86_64-unknown-linux-gnu Target: x86_64-unkown-linux-gnu |
| Max (global) rustc threads concurrency: | 0 |
This branch (2918395)
| Targets: | rustpython 0.1.2 (lib, bin "rustpython") |
|---|---|
| Profile: | release |
| Fresh units: | 0 |
| Dirty units: | 357 |
| Total units: | 357 |
| Max concurrency: | 4 (jobs=4 ncpu=4) |
| Build start: | 2022-04-23T01:38:57Z |
| Total time: | 575.9s (9m 35.9s) |
| rustc: | rustc 1.60.0 (7737e0b5c 2022-04-04) Host: x86_64-unknown-linux-gnu Target: x86_64-unkown-linux-gnu |
| Max (global) rustc threads concurrency: | 0 |
2918395 to
ad3dcb9
Compare
|
Should I rustfmt it? |
Saves about half a minute on a clean build, see timing reports:
Before
After