Repository navigation
Gitlab cache configuration (as documented) leads to corrupted/invalid cache #625
Description
describe your issue
Following from the discussion at pre-commit/pre-commit#1133 , following the current documentation on how to configure GitLab's cache will result in errors such as:
An error has occurred: InvalidManifestError:
=====> /builds/kjfMHq72/0/owner/project/.cache/pre-commit/repoxhyml2wy/.pre-commit-hooks.yaml is not a file
Check the log at /builds/rZMs1Hzd/0/owner/project/.cache/pre-commit/pre-commit.log
This usually happens some days/weeks after the cache was first created.
The issue can be temporarily solved by clicking the Clearing runner caches button in GitLab's CI/CD section but a more reliable solution is desirable.
pre-commit --version
2.17.0
.pre-commit-config.yaml
---
repos:
- repo: https://github.com/pre-commit/pre-commit-hooks
rev: v4.0.1
hooks:
- id: trailing-whitespace
args: ["--markdown-linebreak-ext=md"]
- id: end-of-file-fixer
- id: fix-byte-order-marker
- id: fix-encoding-pragma
args: ["--remove"]
- id: check-executables-have-shebangs
- id: check-yaml
# Allows use of !reference in .gitlab-ci.yml
args: ["--unsafe"]
- id: check-symlinks
- id: check-docstring-first
- id: check-added-large-files
- id: check-merge-conflict
- id: debug-statements
- repo: https://github.com/pycqa/isort
# Formats, sorts and reorganizes imports
rev: 5.9.1
hooks:
- id: isort
args:
- "--profile"
- "black"
- "--filter-files"
- "--thirdparty"
- "mmpy_bot"
- repo: https://github.com/psf/black
# Code style formatting
rev: 20.8b1
hooks:
- id: black
- repo: https://github.com/pre-commit/mirrors-mypy
# static type analysis
rev: v0.812
hooks:
- id: mypy
- repo: https://gitlab.com/pycqa/flake8
# Checks the code for PEP8 violations and common pitfals
rev: 3.9.0
hooks:
- id: flake8
args:
- "--extend-ignore"
- "W503,E203,E501"
- "--per-file-ignores"
- "alembic/versions/*:E501"
- repo: https://github.com/myint/docformatter
# Formats docstrings following PEP 257
rev: v1.3.1
hooks:
- id: docformatter
args:
- "--in-place"
- repo: https://github.com/asottile/pyupgrade
# updates syntax to use constructs introduced in newer python versions
# e.g. f-strings instead of "{}".format()
rev: v2.13.0
hooks:
- id: pyupgrade
args:
- --py39-plus
- repo: https://github.com/jackdewinter/pymarkdown
rev: 0.9.1
hooks:
- id: pymarkdown
args:
# Disable line length rule and mandatory H1 at top-level
- --disable-rules=md013,md041
- scan
- repo: https://github.com/adrienverge/yamllint.git
# validates YAML syntax
rev: v1.26.1
hooks:
- id: yamllint
args:
- "--config-file"
- ".yamllint.yaml"
- "--format"
- "parsable"
- "--strict"
- repo: https://github.com/jumanjihouse/pre-commit-hook-yamlfmt
# formats YAML to correct indentation and alignment
rev: 0.1.0
hooks:
- id: yamlfmt
args:
- "--preserve-quotes"
- "--mapping"
- "2"
- "--sequence"
- "4"
- "--offset"
- "2"
- repo: https://github.com/pycqa/doc8
# sphinx rst style checker
rev: 0.9.0a1
hooks:
- id: doc8~/.cache/pre-commit/pre-commit.log (if present)
No response
Activity
asottile commentedon Jan 24, 2022
(I've moved this issue to the docs, since that's what should be adjusted)
for gitlab hosted the documented value has been working great -- the problem appears to only be for personal gitlab installations. perhaps because the working directory is variable there?
unode commentedon Jan 24, 2022
If relevant, we are using a kubernetes based runner and indeed it is a self-hosted gitlab installation.
asottile commentedon Jan 24, 2022
I'd recommend you sponsor the project if you're looking for free support by the way
asottile commentedon Jan 24, 2022
it looks like the builds directory is configurable -- but gitlab automatically inserts some tokens into it that make it not consistent: https://docs.gitlab.com/runner/best_practice/#build-directory
they say you can also write to other paths inside there but that it's discouraged -- perhaps that would get you a consistent path?
asottile commentedon Jan 24, 2022
another idea would be to have a cache key that uses the values that are changing -- then you'd invalidate your cache any time gitlab decided to use a different directory
asottile commentedon Apr 13, 2022
via #645