Skip to content

bpo-39595: Improve zipfile.Path performance#18406

Merged
jaraco merged 7 commits into
python:masterfrom
jaraco:feature/zipp-perf
Feb 12, 2020
Merged

bpo-39595: Improve zipfile.Path performance#18406
jaraco merged 7 commits into
python:masterfrom
jaraco:feature/zipp-perf

Conversation

@jaraco

@jaraco jaraco commented Feb 7, 2020 •
edited by bedevere-bot
Loading

Copy link
Copy Markdown
Member

In jaraco/zipp#32, I learned that the zipfile.Path and zipp.Path objects were O(n) for joinpath and entries, causing O(n^2) complexity for other operations, including those used by importlib.metadata. This change incorporates those changes, released with zipp 1.1 and 2.1.

https://bugs.python.org/issue39595

@codecov

codecov Bot commented Feb 7, 2020 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

Merging #18406 into master will increase coverage by 0.00%.
The diff coverage is n/a.

Impacted file tree graph

@@            Coverage Diff            @@
##           master   #18406     +/-   ##
=========================================
  Coverage   82.11%   82.12%             
=========================================
  Files        1955     1955             
  Lines      588928   583715   -5213     
  Branches    44428    44436      +8     
=========================================
- Hits       483611   479354   -4257     
+ Misses      95667    94715    -952     
+ Partials     9650     9646      -4
Impacted Files Coverage Δ
Lib/distutils/tests/test_bdist_rpm.py 30.00% <0.00%> (-65.00%) ⬇️
Lib/distutils/command/bdist_rpm.py 7.63% <0.00%> (-56.88%) ⬇️
Lib/test/test_urllib2net.py 76.92% <0.00%> (-13.85%) ⬇️
Lib/test/test_smtpnet.py 78.57% <0.00%> (-7.15%) ⬇️
Lib/ftplib.py 63.85% <0.00%> (-6.06%) ⬇️
Lib/test/test_ftplib.py 87.11% <0.00%> (-4.72%) ⬇️
Tools/scripts/db2pickle.py 17.82% <0.00%> (-3.97%) ⬇️
Tools/scripts/pickle2db.py 16.98% <0.00%> (-3.78%) ⬇️
Lib/test/test_socket.py 71.94% <0.00%> (-3.77%) ⬇️
Lib/distutils/tests/test_bdist_msi.py 56.25% <0.00%> (-3.75%) ⬇️
... and 335 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dc7a50d...093b637. Read the comment docs.

tirkarthi
tirkarthi reviewed Feb 9, 2020
View reviewed changes
Comment thread Lib/zipfile.py Outdated
@jaraco jaraco changed the title Improve zipfile.Path performance bpo-39595: Improve zipfile.Path performance Feb 9, 2020
@jaraco jaraco removed the skip issue label Feb 9, 2020
tirkarthi
tirkarthi reviewed Feb 11, 2020
View reviewed changes
Comment thread Lib/test/test_zipfile.py
@jaraco
jaraco merged commit e5bd736 into python:master Feb 12, 2020
@bedevere-bot

bedevere-bot commented Feb 12, 2020

Copy link
Copy Markdown

@jaraco: Please replace # with GH- in the commit message next time. Thanks!

@miss-islington

miss-islington commented Feb 12, 2020

Copy link
Copy Markdown
Contributor

Thanks @jaraco for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

@miss-islington

miss-islington commented Feb 12, 2020

Copy link
Copy Markdown
Contributor

I'm having trouble backporting to 3.8. Reason: 'Error 110 while writing to socket. Connection timed out.'. Please retry by removing and re-adding the needs backport to 3.8 label.

@jaraco
jaraco deleted the feature/zipp-perf branch February 12, 2020 02:58
@miss-islington

miss-islington commented Feb 12, 2020

Copy link
Copy Markdown
Contributor

Thanks @jaraco for the PR 🌮🎉.. I'm working now to backport this PR to: 3.8.
🐍🍒⛏🤖

@bedevere-bot

bedevere-bot commented Feb 12, 2020

Copy link
Copy Markdown

GH-18472 is a backport of this pull request to the 3.8 branch.

jaraco added a commit that referenced this pull request Feb 12, 2020
@miss-islington @jaraco
bpo-39595: Improve zipfile.Path performance (GH-18406) (GH-18472)
ed4d263 
* Improve zipfile.Path performance on zipfiles with a large number of entries.

* 📜🤖 Added by blurb_it.

* Add bpo to blurb

* Sync with importlib_metadata 1.5 (6fe70ca)

* Update blurb.

* Remove compatibility code

* Add stubs module, omitted from earlier commit

Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
(cherry picked from commit e5bd736)

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>

Co-authored-by: Jason R. Coombs <jaraco@jaraco.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@tirkarthi tirkarthi tirkarthi left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@jaraco @bedevere-bot @miss-islington @tirkarthi @the-knights-who-say-ni