You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
zipfile.ZipFile is closed when zipfile.Path is closed #88804
Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.
Show more details
GitHub fields:
assignee=Noneclosed_at=<Date2021-07-16.01:12:12.249>created_at=<Date2021-07-14.14:48:27.833>labels= ['3.8', 'type-bug', 'library']
title='zipfile.ZipFile is closed when zipfile.Path is closed'updated_at=<Date2021-07-16.13:15:04.558>user='https://github.com/christian-steinmeyer'
When executing the code below with the attached zip file (or any other that has one or more files directly at root level), I get a "ValueError: seek of closed file". It seems, the zipfile handle being part of the TestClass instance is being closed, when the zipfile.Path is garbage collected, when it is no longer referenced. Since zipfile.Path even takes a zipfile.Zipfile as an argument, I don't think it is intended? It surprised me at least.
import zipfile
class TestClass:
def __init__(self, path):
self.zip_file = zipfile.ZipFile(path)
def iter_dir(self):
return [each.name for each in zipfile.Path(self.zip_file).iterdir()]
def read(self, filename):
with self.zip_file.open(filename) as file:
print(file.read())
root = "zipfile.zip"
test = TestClass(root)
files = test.iter_dir()
test.read(files[0])
I'm not able to reproduce this on my machine; the script runs without any issue.
the TestClass instance is being closed
What do you mean by this statement? You aren't doing anything to TestClass or its instance ("test") in this script. They remain in scope, so they will always be referenced.
christian-steinmeyer commented on Jul 15, 2021
christian-steinmeyermannequin
MannequinAuthor
More actions
I work on macOS 11.4 (20F71) (Kernel Version: Darwin 20.5.0).
My python version is 3.8.9 and zipp is at 3.5.0 (but 3.4.1 behaves the same for me).
For me, this is behavior is reproducible.
Let me try to clarify what I mean.
test=TestClass(root) # this creates a zipfile handle (an instance of zipfile.ZipFile) at test.zip_filefiles=test.iter_dir() # this creates multiple instances of zipfile.Path() as part of the list comprehension and these are deferenced afterwards. I found that test.zip_file.fp is closed after this line executes, which to me suggests that the closing of the zipfile.Path also closes the zipfile.ZipFile that was used to create the zipfile.Path.test.read(files[0]) # this should in theory try to read from the test.zip_file for the first time, but fails because it is closed as per the above.
Here is the full stack trace:
Traceback (most recent call last):
File "test.py", line 20, in <module>
test.read(files[0])
File "test.py", line 12, in readwithself.zip_file.open(filename) asfile:
File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/zipfile.py", line 1530, in open
fheader = zef_file.read(sizeFileHeader)
File "/usr/local/opt/python@3.8/Frameworks/Python.framework/Versions/3.8/lib/python3.8/zipfile.py", line 763, in readself._file.seek(self._pos)
ValueError: seek of closed file
Here's a much simpler repro that avoids the class construction but triggers the same error:
import zipfile
zip_file = zipfile.ZipFile('zipfile.zip')
names = [each.name for each in zipfile.Path(zip_file).iterdir()]
with zip_file.open(names[0]) as file:
print(file.read())
My python version is 3.8.9 and zipp is at 3.5.0 (but 3.4.1 behaves the same for me).
It's not enough to have zipp 3.5.0. You need to use zipp.Path over zipfile.Path.
christian-steinmeyer commented on Jul 16, 2021
christian-steinmeyermannequin
MannequinAuthor
More actions
Thank you for the in depth look Jason!
Especially that last comment was very useful to me. Perhaps it would make sense to add something like this to the documentation of zipfile.
I'm not sure what would be the best hint, but perhaps in zipfile.Path's documentation a hint that zipp.Path can be used to access newer functionality even for older python versions (if what I understand is correct) might be useful to others as well. Because as of now, I cannot find an equivalent hint yet.
New changeset 29358e9 by Jason R. Coombs in branch 'main': bpo-44638: Add a reference to the zipp project and hint as to how to use it. (GH-27188) 29358e9
If a backport isn't desired (as mentioned in #84744 (comment)), would it at least be possible to put a note about this bug into the zipfile.Path docs of Python versions < 3.10? Would've saved me a couple of hours just now and I'm probably not the only one :-)
Activity
christian-steinmeyer commentedon Jul 14, 2021
When executing the code below with the attached zip file (or any other that has one or more files directly at root level), I get a "ValueError: seek of closed file". It seems, the zipfile handle being part of the
TestClassinstance is being closed, when thezipfile.Pathis garbage collected, when it is no longer referenced. Sincezipfile.Patheven takes azipfile.Zipfileas an argument, I don't think it is intended? It surprised me at least.tirkarthi commentedon Jul 14, 2021
This seems similar to https://bugs.python.org/issue40564
jdevries3133 commentedon Jul 15, 2021
I'm not able to reproduce this on my machine; the script runs without any issue.
What do you mean by this statement? You aren't doing anything to TestClass or its instance ("test") in this script. They remain in scope, so they will always be referenced.
christian-steinmeyer commentedon Jul 15, 2021
I work on macOS 11.4 (20F71) (Kernel Version: Darwin 20.5.0).
My python version is 3.8.9 and zipp is at 3.5.0 (but 3.4.1 behaves the same for me).
For me, this is behavior is reproducible.
Let me try to clarify what I mean.
jaraco commentedon Jul 16, 2021
I was able to replicate the error using the script as posted:
jaraco commentedon Jul 16, 2021
Here's a much simpler repro that avoids the class construction but triggers the same error:
jaraco commentedon Jul 16, 2021
Even simpler:
jaraco commentedon Jul 16, 2021
This also reproduces the failure:
Removing
del pathbypasses the issue. Something about the destructor for zipfile.Path is causing the closing of the handle for zip_file.jaraco commentedon Jul 16, 2021
Even simpler:
jaraco commentedon Jul 16, 2021
Changing the repro to:
I'm able now to test against zipfile or zipp. And I notice that the issue occurs only on zipp<3.2 or Python<3.10.
Looking at the changelog (https://zipp.readthedocs.io/en/latest/history.html#v3-2-0), it's clear now that this issue is a duplicate of bpo-40564 and the problem goes away using the original repro and Python 3.10:
The solution is to use zipp>=3.2 or Python 3.10.
jaraco commentedon Jul 16, 2021
It's not enough to have
zipp3.5.0. You need to usezipp.Pathoverzipfile.Path.christian-steinmeyer commentedon Jul 16, 2021
Thank you for the in depth look Jason!
Especially that last comment was very useful to me. Perhaps it would make sense to add something like this to the documentation of zipfile.
I'm not sure what would be the best hint, but perhaps in zipfile.Path's documentation a hint that zipp.Path can be used to access newer functionality even for older python versions (if what I understand is correct) might be useful to others as well. Because as of now, I cannot find an equivalent hint yet.
miss-islington commentedon Jul 16, 2021
New changeset 29358e9 by Jason R. Coombs in branch 'main':
bpo-44638: Add a reference to the zipp project and hint as to how to use it. (GH-27188)
29358e9
smheidrich commentedon Jun 4, 2022
If a backport isn't desired (as mentioned in #84744 (comment)), would it at least be possible to put a note about this bug into the
zipfile.Pathdocs of Python versions < 3.10? Would've saved me a couple of hours just now and I'm probably not the only one :-)