Would it be possible for pre-commit's hook template to modify PATH before invoking pre-commit, when pre-commit is running in a virtualenv? This would allow pre-commit to run Python hooks installed into the virtualenv.
Currently, if pre-commit is installed in a virtualenv, git hooks invoke pre-commit without activating the virtualenv. More specifically, while the Python interpreter running pre-commit uses the virtualenv, the PATH and VIRTUAL_ENV environment variables are not modified to reflect the virtualenv. It is thus the responsibility of the user to activate the virtualenv in the shell invoking git.
As explained below (see Rationale), I believe allowing pre-commit to run hooks in the same virtualenv can offer significant advantages to some projects. This could be achieved by inserting three lines into the hook template (see Implementation Notes).
If you are interested in this feature, I would be happy to provide a PR with tests and documentation. Do you think this is a sensible idea? If so, should this be done always, or hidden behind a feature flag such as an option to the pre-commit install command? I would love to hear your thoughts!
Thanks for your work creating and maintaining so many awesome tools!
Rationale
For certain Python projects, it is beneficial to use dedicated tools to manage the environments and dependencies of Python-language pre-commit hooks, rather than pre-commit itself. Here are some of the advantages this can offer:
- Project dependencies can be declared in one place, and managed by one tool.
- Automated dependency upgrades (e.g. Dependabot) for hooks become possible.
- Additional dependencies (e.g. Flake8 extensions) are visible to the dependency management tool. (
pre-commit autoupdate is centered around the concept of Git repositories; it is -- and should be -- agnostic of language-level concepts such as PyPI versions, does not perform dependency resolution, and does not inspect additional dependencies.)
- Subdependencies (dependencies of dependencies) can be locked automatically, making checks more repeatable and deterministic.
pre-commit already supports this use case via repository-local system hooks. However, this requires that the virtualenv be activated in the calling shell. In some situations, explicitly activating the environment is not practical. Consider these examples:
Example 1: pre-commit and Nox
When running pre-commit in a Nox session, Python hooks can be installed into the environment associated with the Nox session, alongside pre-commit itself. This works fine when pre-commit is invoked by Nox: Nox activates the virtual environment for the pre-commit session, so the hooks are located on PATH. However, when pre-commit is invoked from git hooks, the virtualenv is not active, so pre-commit fails to locate the hooks.
Example 2: pre-commit and Poetry
Another example is the case of adding pre-commit and the Python hooks as dependencies to a Poetry-managed project. When invoking pre-commit with Poetry (poetry run pre-commit), the environment is activated by Poetry. But when pre-commit is invoked from git hooks, the hooks are no longer on PATH.
Example 3: pre-commit, Nox, and Poetry
For a more concrete example of an attempt to integrate pre-commit with both Nox and Poetry, see this tree, specifically the noxfile.py and .pre-commit-config.yaml files in the {{cookiecutter.project_name}} directory. The associated PR is cjolowicz/cookiecutter-hypermodern-python#397.
Implementation Notes
Virtualenv activation could be achieved by inserting three lines into the hook template:
diff --git a/pre_commit/resources/hook-tmpl b/pre_commit/resources/hook-tmpl
index 299144e..19fc7ea 100755
--- a/pre_commit/resources/hook-tmpl
+++ b/pre_commit/resources/hook-tmpl
@@ -26,6 +26,9 @@ ARGS.extend(sys.argv[1:])
DNE = '`pre-commit` not found. Did you forget to activate your virtualenv?'
if os.access(INSTALL_PYTHON, os.X_OK):
+ bindir = os.path.dirname(INSTALL_PYTHON)
+ if os.path.isfile(os.path.join(os.path.dirname(bindir), 'pyvenv.cfg')):
+ os.environ['PATH'] = os.pathsep.join((bindir, os.environ.get('PATH', '')))
CMD = [INSTALL_PYTHON, '-mpre_commit']
elif which('pre-commit'):
CMD = ['pre-commit']
The hook template checks if the Python interpreter used when installing pre-commit (INSTALL_PYTHON) is located in a virtualenv, by checking for the file pyvenv.cfg in the directory above the executable. If this is the case, the directory containing the Python interpreter is prepended to PATH. In a typical virtualenv, this is bin on UNIX-like systems, and Scripts on Windows. To keep the script short, the process environment is modified in place, rather than copied and passed to os.execvpe or subprocess.Popen.
Would it be possible for pre-commit's hook template to modify
PATHbefore invoking pre-commit, when pre-commit is running in a virtualenv? This would allow pre-commit to run Python hooks installed into the virtualenv.Currently, if pre-commit is installed in a virtualenv, git hooks invoke pre-commit without activating the virtualenv. More specifically, while the Python interpreter running pre-commit uses the virtualenv, the
PATHandVIRTUAL_ENVenvironment variables are not modified to reflect the virtualenv. It is thus the responsibility of the user to activate the virtualenv in the shell invoking git.As explained below (see Rationale), I believe allowing pre-commit to run hooks in the same virtualenv can offer significant advantages to some projects. This could be achieved by inserting three lines into the hook template (see Implementation Notes).
If you are interested in this feature, I would be happy to provide a PR with tests and documentation. Do you think this is a sensible idea? If so, should this be done always, or hidden behind a feature flag such as an option to the
pre-commit installcommand? I would love to hear your thoughts!Thanks for your work creating and maintaining so many awesome tools!
Rationale
For certain Python projects, it is beneficial to use dedicated tools to manage the environments and dependencies of Python-language pre-commit hooks, rather than pre-commit itself. Here are some of the advantages this can offer:
pre-commit autoupdateis centered around the concept of Git repositories; it is -- and should be -- agnostic of language-level concepts such as PyPI versions, does not perform dependency resolution, and does not inspect additional dependencies.)pre-commit already supports this use case via repository-local system hooks. However, this requires that the virtualenv be activated in the calling shell. In some situations, explicitly activating the environment is not practical. Consider these examples:
Example 1: pre-commit and Nox
When running pre-commit in a Nox session, Python hooks can be installed into the environment associated with the Nox session, alongside pre-commit itself. This works fine when pre-commit is invoked by Nox: Nox activates the virtual environment for the pre-commit session, so the hooks are located on
PATH. However, when pre-commit is invoked from git hooks, the virtualenv is not active, so pre-commit fails to locate the hooks.Example 2: pre-commit and Poetry
Another example is the case of adding pre-commit and the Python hooks as dependencies to a Poetry-managed project. When invoking pre-commit with Poetry (
poetry run pre-commit), the environment is activated by Poetry. But when pre-commit is invoked from git hooks, the hooks are no longer onPATH.Example 3: pre-commit, Nox, and Poetry
For a more concrete example of an attempt to integrate pre-commit with both Nox and Poetry, see this tree, specifically the noxfile.py and .pre-commit-config.yaml files in the
{{cookiecutter.project_name}}directory. The associated PR is cjolowicz/cookiecutter-hypermodern-python#397.Implementation Notes
Virtualenv activation could be achieved by inserting three lines into the hook template:
The hook template checks if the Python interpreter used when installing pre-commit (
INSTALL_PYTHON) is located in a virtualenv, by checking for the filepyvenv.cfgin the directory above the executable. If this is the case, the directory containing the Python interpreter is prepended toPATH. In a typical virtualenv, this isbinon UNIX-like systems, andScriptson Windows. To keep the script short, the process environment is modified in place, rather than copied and passed toos.execvpeorsubprocess.Popen.