gh-91172: Create a workflow for verifying bundled pip and setuptools - #31885
Conversation
| - uses: actions/checkout@v2 | ||
| - name: Compare checksums of bundled pip and setuptools to ones published on PyPI | ||
| run: | | ||
| package_names=("pip" "setuptools") |
There was a problem hiding this comment.
Perhaps this could be moved to a script (say, under Tools/scripts), so it can also be run locally?
There was a problem hiding this comment.
Good idea, but it looks like all scripts in Tools/scripts are written in Python.
This is a Bash script, and it may not work on some platforms.
Do you know how compatible with different platforms scripts located in Tools/scripts should be?
There was a problem hiding this comment.
Hmm, or perhaps the Misc directory? The devguide says:
Various tools with configuration files as found in the Misc directory
I don't know what compatibility is needed, but as this script is primarily intended for Ubuntu on the CI, I'd stick with that.
There was a problem hiding this comment.
Done. This is the latest run after I corrupted the pip wheel temporarily.
BTW, I added messages that GitHub has to show as annotations. But I could not see the annotations, maybe it does not show them for binary files…
Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
a94b7fe to
23683cb
Compare
This reverts commit e46f87d.
|
@illia-v the below is my attempt at a Python rewrite: """Compares checksums for wheels in :mod:`ensurepip` against the Cheeseshop."""
import hashlib
import json
from pathlib import Path
import re
from urllib.request import urlopen
PACKAGE_NAMES = ("pip", "setuptools")
ROOT = Path(__file__).parent.parent / "Lib/ensurepip"
WHEEL_DIR = ROOT / "_bundled"
ENSURE_PIP_INIT_PY_TEXT = (ROOT / "__init__.py").read_text(encoding="utf-8")
# Contains names of successfully verified packages
verified_packages = {*()}
for package_name in PACKAGE_NAMES:
# Find the package on disk
package_path = next(WHEEL_DIR.glob(f"{package_name}*.whl"), None)
if package_path is None:
continue
print(f"Verifying checksum for {package_path}.")
# Find the version of the package used by ensurepip
package_version_match = re.search(
f'_{package_name.upper()}_VERSION = "([^"]+)',
ENSURE_PIP_INIT_PY_TEXT
)
if package_version_match is None:
continue
package_version = package_version_match[1]
# Get the SHA 256 digest from the Cheeseshop
try:
raw_text = urlopen(f"https://pypi.org/pypi/{package_name}/json").read()
except (OSError, ValueError) as error:
continue
expected_digest = ""
release_files = json.loads(raw_text)["releases"][package_version]
for release_info in release_files:
if package_path.name != release_info["filename"]:
continue
expected_digest = release_info["digests"].get("sha256", "")
if expected_digest == "":
continue
# Compute the SHA 256 digest of the wheel on disk
actual_digest = hashlib.sha256(package_path.read_bytes()).hexdigest()
print(f"Expected digest: {expected_digest}")
print(f"Actual digest: {actual_digest}")
# The messages are formatted to be parsed by GitHub Actions.
# https://docs.github.com/en/actions/using-workflows/workflow-commands-for-github-actions#setting-a-notice-message
if actual_digest == expected_digest:
print(f"::notice file={package_path}::"
f"Successfully verified the checksum of the {package_name} wheel.\n")
verified_packages.add(package_name)
else:
print(f"::error file={package_path}::"
f"Failed to verify the checksum of the {package_name} wheel.\n")
# If we verified all packages, the set of package names and the set of verified
# packages will be equal.
if {*PACKAGE_NAMES} == verified_packages:
raise SystemExit(0)
# Otherwise we failed to verify all the packages.
raise SystemExit(1)A |
@AA-Turner thanks! Could you please push it to this branch or create a pull request to it? Also, what do you think about creating the BTW, isn't Warehouse the new name of what was called Cheese Shop? |
Well, Python Package Index used to be called Cheese Shop (based on a Monty Python sketch about a Cheese Shop that didn't have any cheese) -- the current codebase for PyPI is called Warehouse: https://github.com/pypa/warehouse/ |
|
GH-94121 is a backport of this pull request to the 3.11 branch. |
|
GH-94123 is a backport of this pull request to the 3.9 branch. |
|
GH-94124 is a backport of this pull request to the 3.8 branch. |
|
GH-94122 is a backport of this pull request to the 3.10 branch. |
|
I added my security branches. Rationale: if there is a security-related bump of |
|
GH-94126 is a backport of this pull request to the 3.7 branch. |

https://bugs.python.org/issue47016