Repository navigation
Speed up import isal.igzip by not importing the typing module - #319
Conversation
|
This is very much a speedup that's tiny, but being me, I find each of them satisfying and I like how they add up when you consider users x runs |
rhpvorderman
left a comment
There was a problem hiding this comment.
Looks good to me, one minor thing so it can be done with even less code.
3d815c7 to
7420b66
Compare
Only type checkers need typing, and importing it also loads functools and copyreg on Python 3.14, and re, enum, contextlib and warnings on 3.11. igzip_threaded now uses built-in generics, and igzip annotates the mtime argument of compress() as int | None instead of Optional[SupportsInt]. Under callgrind, import isal.igzip costs 10.5M instead of 28.4M instructions on top of interpreter startup on Python 3.14, and 5.4M instead of 38.6M on 3.11.
7420b66 to
4f3faf7
Compare
|
As a bit of history context, when I started this library years ago the Thank you for all your work, I have to focus on other projects now as well so it will take some time before I get to all the PRs. |
No problem at all. Appreciate your reviews and feedback. Take all the time! The work never ends haha! |
import isal.igzipandimport isal.igzip_threadedalso import thetypingmodule, but the two modules use the names that they import from it only in annotations.typingin turn imports other modules, for example,functoolson Python 3.14 andre,enum, andcontextlibon Python 3.11, so every program that importsisal.igzippays for them.python -m isal.igziploads most of these modules anyway throughargparseandshutil, so on the command line this PR saves only the import oftypingitself.With this PR, neither module imports
typing:igzip.pyannotates themtimeparameter ofcompress()asint | Noneinstead ofOptional[SupportsInt]. Type checkers now report anmtimethat isn't anint, but at run timecompress()still convertsmtimewithint().igzip_threaded.pyuseslist,tuple, andX | Nonein its annotations instead ofList,Tuple, andOptional.The
test_import_does_not_load_cli_modulestest is nowtest_import_does_not_load_slow_modules, and it also checks thattypingisn't imported. That check fails ondevelop.I measured the wall-clock time of new processes that run
python -c "import isal.igzip", in paired rounds againstdevelop. The Linux numbers are from Docker on WSL2, and the Windows numbers are from native builds. The table shows the medians in milliseconds:developOn Linux with CPython 3.14,
python -c passtakes 7.6 ms, so importingisal.igzipadds about 5.2 ms to the process ondevelopand 2.1 ms with this PR. On the five platforms in the table,python -m isal.igzip -con a 64 KiB file finishes 1.3 to 2.4 ms sooner. A program that importstypingfor other reasons saves nothing. xopen and dnaio both importtyping, for example.The test suite passes on this branch on Windows with CPython 3.14 and 3.11.
Checklist