You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Bug]: LICENSE_QHULL is included in wheel only the second time #25212
The first time I run python3.10 setup.py bdist_wheel on both 3.6.0 and 3.7.0, the resulting wheel does not include matplotlib-0.0.0.dist-info/LICENSE_QHULL. The second time the LICENSE_QHULL file is included.
I cannot reproduce it locally (although with Python 3.9). Does the log print that it adds the license files? I get this at the end of the log and it seems to be in the wheel as well
Did you already have the Qhull code/license downloaded? The problem I see happens only the first time, so a git clean -dxff . should help reproduce the problem. I get the same wrong behaviour with py3.9 on a clean manylinux container.
This is a race condition between the qhull custom build step (which puts the license file in the right place) in setupext.py and the manifest generation (which seems to be controlled via setup.cfg).
Hopefully there is some nice setuptools hook we can call as part of the copying to re-discover the files (or just manually update the manifest metadata).
Same results for 3.7.0rc1 wheels, but not for wheels generated on the 3.6.x branch, which had all of the qhull license files (I just pulled the zip from the "zenodo doi" run shortly after 3.6.3 was released)
I spot checked a couple manylinux 3.6.3 wheels from pypi and found they had LICENSE_QHULL.
So this does seem to be relatively recent.
I spot checked a couple manylinux 3.6.3 wheels from pypi and found they had LICENSE_QHULL. So this does seem to be relatively recent.
Actually the problem for me happens also in 3.6.0, when built from source. PyPI does have the LICENSE_QHULL file, but it doesn't if you re-build it (from clean env) from the git tag v3.6.0.
Well, it may be true that newly built wheels have the same problem, it may still be caused by a relatively recent toolchain update, which is useful information.
diff --git a/setup.py b/setup.py
index bbaa56f3d6..f3b3b29c5b 100644
--- a/setup.py+++ b/setup.py@@ -42,12 +42,13 @@ from setupext import print_raw, print_status
# These are the packages in the order we want to display them.
+qhull_setup = setupext.Qhull()
mpl_packages = [
setupext.Matplotlib(),
setupext.Python(),
setupext.Platform(),
setupext.FreeType(),
- setupext.Qhull(),+ qhull_setup,
setupext.Tests(),
setupext.BackendMacOSX(),
]
@@ -267,6 +268,8 @@ if not (any('--' + opt in sys.argv
package_data.setdefault(key, [])
package_data[key] = list(set(val + package_data[key]))
+qhull_setup.download()+
setup( # Finally, pass this all along to setuptools to do the heavy lifting.
name="matplotlib",
description="Python plotting package",
diff --git a/setupext.py b/setupext.py
index 7cd5216a59..acba7ea70e 100644
--- a/setupext.py+++ b/setupext.py@@ -756,7 +756,7 @@ class Qhull(SetupPackage):
else:
cls._extensions_to_update.append(ext)
- def do_custom_build(self, env):+ def download(self):
if options.get('system_qhull'):
return
@@ -767,6 +767,12 @@ class Qhull(SetupPackage):
)
shutil.copyfile(toplevel / "COPYING.txt", "LICENSE/LICENSE_QHULL")
+ def do_custom_build(self, env):+ if options.get('system_qhull'):+ return++ self.download()+
for ext in self._extensions_to_update:
qhull_path = Path(f'build/qhull-{LOCAL_QHULL_VERSION}/src')
ext.include_dirs.insert(0, str(qhull_path))
A super quick and dirty workaround, not sure I like it, but it seems to work.
Essentially I copied the download/copy license file from do_custom_build and put an additional method which I call before handing off to setuptools at all (but after any config parsing, so the license file is not included if system_qhull is set to True)
One small difference (which may be important) is that the sdist includes the QHULL license file (which it didn't before) and is not actually distributing qhull at all.
Have had no success at pinning down what local version of build deps may have changed this behavior, but I did notice that the 3.6.x branch uses cibuildwheel 0.11.2, whereas the main branch (and 3.7.x) uses 0.12.0, so that could have something to do with it... though nothing jumped at me in looking at the changelog there, so not sure that is relevant (and certainly would expect something lower level to be the actual root cause even if that is the proximal cause)
Note that this is only fixed for our CI-built wheels. We are still not sure what changed in what package that caused 3.7 wheels to drop the license while 3.6 wheels were previously fine.
Activity
oscargus commentedon Feb 14, 2023
I cannot reproduce it locally (although with Python 3.9). Does the log print that it adds the license files? I get this at the end of the log and it seems to be in the wheel as well
Any other license files missing?
ret2libc commentedon Feb 14, 2023
Did you already have the Qhull code/license downloaded? The problem I see happens only the first time, so a
git clean -dxff .should help reproduce the problem. I get the same wrong behaviour with py3.9 on a clean manylinux container.tacaswell commentedon Feb 14, 2023
This is a race condition between the qhull custom build step (which puts the license file in the right place) in
setupext.pyand the manifest generation (which seems to be controlled viasetup.cfg).Hopefully there is some nice setuptools hook we can call as part of the copying to re-discover the files (or just manually update the manifest metadata).
ksunden commentedon Feb 14, 2023
The following wheels from the 3.7.0 final release DO NOT include LICENSE_QHULL:
i.e. all of the manylinux wheels except pypy 3.9 plus one 32 bit windows wheel
tacaswell commentedon Feb 14, 2023
We should fix this for 3.7.1 (which I expect to be with in the month) and going forward.
ksunden commentedon Feb 14, 2023
Same results for 3.7.0rc1 wheels, but not for wheels generated on the 3.6.x branch, which had all of the qhull license files (I just pulled the zip from the "zenodo doi" run shortly after 3.6.3 was released)
I spot checked a couple manylinux 3.6.3 wheels from pypi and found they had LICENSE_QHULL.
So this does seem to be relatively recent.
ret2libc commentedon Feb 14, 2023
Actually the problem for me happens also in 3.6.0, when built from source. PyPI does have the LICENSE_QHULL file, but it doesn't if you re-build it (from clean env) from the git tag v3.6.0.
ksunden commentedon Feb 14, 2023
Well, it may be true that newly built wheels have the same problem, it may still be caused by a relatively recent toolchain update, which is useful information.
ksunden commentedon Feb 15, 2023
A super quick and dirty workaround, not sure I like it, but it seems to work.
Essentially I copied the download/copy license file from
do_custom_buildand put an additional method which I call before handing off to setuptools at all (but after any config parsing, so the license file is not included ifsystem_qhullis set to True)One small difference (which may be important) is that the sdist includes the QHULL license file (which it didn't before) and is not actually distributing qhull at all.
Have had no success at pinning down what local version of build deps may have changed this behavior, but I did notice that the 3.6.x branch uses cibuildwheel 0.11.2, whereas the main branch (and 3.7.x) uses 0.12.0, so that could have something to do with it... though nothing jumped at me in looking at the changelog there, so not sure that is relevant (and certainly would expect something lower level to be the actual root cause even if that is the proximal cause)
BLD: Pre-download Qhull license to put in wheels
BLD: Pre-download Qhull license to put in wheels
BLD: Pre-download Qhull license to put in wheels
BLD: Pre-download Qhull license to put in wheels
QuLogic commentedon Mar 3, 2023
Note that this is only fixed for our CI-built wheels. We are still not sure what changed in what package that caused 3.7 wheels to drop the license while 3.6 wheels were previously fine.