This tool scans the repository for hardcoded references to specific dependency versions (like Python 3.7) that need to be upgraded or removed.
Run the script from the repository root:
python3 scripts/version_scanner/version_scanner.py -d <dependency> -v <version> [options]-d,--dependency: Name of the dependency (e.g., python, protobuf)-v,--version: Specific version to search for (e.g., 3.7, 4.25.8)-p,--path: Root directory to scan (defaults to current directory)--package: Specific subdirectory filter (useful for monorepos)--package-file: Path to a file containing a list of package directories to scan (e.g.,scripts/version_scanner/small_package_list.txt)--config: Path to the regex configuration file (defaults to scripts/version_scanner/regex_config.yaml)-o,--output: Path to the output CSV file (defaults to --.csv)--github-repo: GitHub repository URL base (defaults to https://github.com/googleapis/google-cloud-python)--branch: GitHub branch for links (defaults to main)
By default, the core scanner only depends on Python's standard library and pyyaml to read the configuration file.
If you want to use the Google Drive upload feature (--upload), you must install the optional Google API client dependencies:
pip install -r scripts/version_scanner/requirements.txtNote
This scanner is primarily intended for auditing handwritten code, configuration files, CI scripts, and documentation.
You do not need to scan or manually edit auto-generated GAPIC libraries. Any dependency updates for generated code are handled upstream by editing the generator templates in the gapic-generator-python repository. When the templates are updated, the changes naturally trickle downstream to correct all generated client libraries upon the next regeneration.
- Single-Line Matching Only: The scanner processes files line-by-line to ensure high performance and simplicity. Consequently, version declarations or dependency lists that span across multiple lines (such as multiline lists in a
setup.pyfile) will not be caught by the regex patterns.
The scanner uses a YAML configuration file (regex_config.yaml) to define rules and regex patterns.
You can create a .scannerignore file in the directory you are scanning (usually the repo root) to list directories to skip, one per line.
- Binary Ignores in
.scannerignore: Recursive wildcard ignores (e.g.,*.jpg) currently do not effectively ignore deeply nested binary files. The scanner logic should be investigated to support robust globbing or full-path suffix matching.
This prompt is provided as an example and outlines the approach to update Python packages to drop support for end-of-life Python runtimes (3.7, 3.8, 3.9) OR for deprecated dependencies, and ensure the packages are configured for modern Python. This may help speed up your ability to resolve version mismatches. This prompt is provided with no guarantees, your mileage may vary. LLMs may make mistakes, always double check the LLM's work and test thoroughly.
- One Branch Per Package: To keep PRs manageable and isolated, we suggest a dedicated worktree and branch for each package (e.g.,
feat/drop-<dependency>-<version>-<package-name>i.e.feat/drop-protobuf-4.25.8-google-cloud-bigquery). - Small & Reversible Commits: Group changes into logical commits (Metadata, Nox, Docs, Cleanup, Tests) following Conventional Commits.
Follow these steps for each package in the target list. Context and warnings are provided inline before the steps where they apply.
- Ensure
mainbranch is up to date. - Create the feature branch:
git checkout -b feat/drop-<dependency>-<version>-<package-name>.
- Run the
version_scannerfor the package to get a list of all occurrences of the dependency and version.[!TIP] Use
# version-scanner: ignoreorignore-next-linein code to silence true false-positives and maintain clean reports.
Before applying changes, review these standards to ensure consistency:
- Standard: Use
sys.version_info < (X, Y). - Rationale: Python compares tuples lexicographically, making this robust.
- Avoid:
sys.version_info.minor < Yor string conversions.
- Standard:
@pytest.mark.skipif(sys.version_info < (X, Y), reason="Requires Python X.Y+"). - Avoid: String-based conditions like
@pytest.mark.skipif("sys.version_info < ...").
- Standard:
session.python == "X.Y"(Nox uses strings). - Avoid:
float(session.python) < X.Y(fails for3.10).
- Polyfills: Remove dead
try/exceptblocks guarding polyfills for features now standard in 3.10+. - Obsolete Skips: Remove pytest skips for features now universally available.
- Use idiomatic python references to detect dependency versions and to compare against the target version.
Every reference to the dependency version found by the scanner must be dispositioned in one of these ways:
- Update: Update the reference if still necessary (e.g., changing
3.9to3.10in support files). - Delete: Delete if no longer relevant (dead code, obsolete comments).
- Pragma Ignore: Use
# version-scanner: ignoreor# version-scanner: ignore-next-linebut ONLY for immutable historical facts or true false positives. Do NOT use for things that might change in future upgrades.
- Update
setup.pyorpyproject.tomlmetadata andrequires-python. - Update
noxfile.pyto remove old versions from sessions. - Update
README.rstandCONTRIBUTING.rstdocumentation. - Remove compatibility code and skips based on the standards above.
- Sync Documentation: If the package has a
docsfolder containing aREADME.rst, copy the updated top-levelREADME.rstto overwrite it (unless it is a symlink). - Continue with the update process until all rows from the scan have been properly dispositioned.
- Run the
version_scanneragain. The result should be 0 matches (or only valid ignores).
Review these lessons learned when dealing with constraints:
- Lowest Runtime Constraints: The file for the lowest accepted runtime (e.g.,
constraints-3.10.txt) must have pins matching the lowest acceptable versions insetup.pyorpyproject.toml. - Philosophy on Warnings: Do not simply block warnings (like
sixorpkg_resources) to make tests pass. Bump the lower bounds of dependencies to versions that don't trigger warnings on the current lowest acceptable runtime. This protects customers who use strict warning filters. - SQLAlchemy Transition: For libraries supporting both 1.4 and 2.0, use
SQLALCHEMY_SILENCE_UBER_WARNING=1in specific legacy Nox sessions rather than silencing globally.
- Run unit tests using Nox (e.g.,
nox -s unit).[!TIP] Use
nox -s unit-3.10to save time when debugging specific runtime failures. - Run
blackenandlintsessions.
- Push the branch and create the PR using the template in the Appendix.
PR Template 1
This PR updates `<dependency>` to establish version x.y.z as the minimum supported version.
### Changes
* Configuration: Updated `setup.py` and `noxfile.py` to require <dependency> <version> and remove references to older versions.
* Cleanup: Removed dead code and polyfills no longer needed.
Fixes internal issue: http://b/482126936 🦕
The following guidelines are universal for AI assistants workin' in this repo and should be moved to .conductor files or Gemini memories:
- AI & LLM Guidelines for Verification:
- Use Git Worktrees to scan branches without switching.
- Run scanner from main branch pointing to worktree.
- Bypass env artifacts by worktree only checking out tracked files.
- Automated Bisection:
- Use
version_bisector.pyto find lowest workable versions. - Abort tests early as soon as collection succeeds to save time.
- Use
Footnotes
-
Adapted from the standard PR template used in this repository. ↩