Skip to content

Apply linting to entire codebase & add CI workflow to check linting#3216

Merged
MMathisLab merged 84 commits intomainfrom
jaap/reformat_codebase
Mar 30, 2026
Merged

Apply linting to entire codebase & add CI workflow to check linting#3216
MMathisLab merged 84 commits intomainfrom
jaap/reformat_codebase

Conversation

@deruyter92
Copy link
Copy Markdown
Collaborator

@deruyter92 deruyter92 commented Feb 23, 2026

This PR is part of the task described in issue #3217

Motivation:
Linting has not been applied consistently in existing codebase. Following DeepLabCut/DeepLabCut-live#166, it would be great to introduce a modern and consistent linting configuration again, after reformatting the codebase.

Order of merging:

Changes:
(Copies the procedure from DeepLabCut/DeepLabCut-live#166, by @C-Achard )

  • Add format.yaml
  • Apply linting to the entire codebase by running:
pre-commit install
pre-commit run --all-files

Note: By running this command, the formatting could be applied easily in a separate PR. This requires step 2 to be completed (having an updated pyproject.toml which include the new linting configuration).

@C-Achard C-Achard added this to the Centralize linting update milestone Feb 23, 2026
@C-Achard C-Achard linked an issue Feb 23, 2026 that may be closed by this pull request
3 tasks
@deruyter92 deruyter92 requested review from MMathisLab and removed request for MMathisLab February 28, 2026 08:05
@deruyter92 deruyter92 added the enhancement New feature or request label Feb 28, 2026
@C-Achard C-Achard added the CI Related to CI/CD jobs and automated testing label Mar 3, 2026
@deruyter92 deruyter92 force-pushed the jaap/reformat_codebase branch from 88356ec to 8e65bd6 Compare March 16, 2026 09:18
C-Achard added 12 commits March 16, 2026 15:58
Bump pyproject-fmt to v2.18.1 and ruff-pre-commit to v0.15.6. Add docformatter (v1.7.7) with formatting args (--wrap-summaries/--wrap-descriptions=88, --in-place, --black). Keep ruff hooks (format + check) and existing validate-pyproject/trailing-whitespace hooks.
Introduce tools/ruff_report.py: a CLI script that runs `ruff` on given paths (using `--output-format=json --exit-zero`), parses the JSON results, and generates a readable Markdown report. The report groups issues by rule and file, includes a summary table, suggested triage order, table of contents, per-rule sections with detailed issue tables, and VS Code "Quick open" commands. The script also embeds short RULE_NOTES for common codes, accepts paths (default `.`) and `--output` (default `ruff-report.md`), and writes the generated Markdown to disk.
Add tools/trim_lines.py: a command-line helper that finds files with Ruff E501 violations and attempts to reduce them using autopep8, then normalizes formatting with Ruff. It scans paths via `ruff check --output-format=json`, supports batching (50 files), a configurable `--line-length` (default 88), and a `--check` dry-run mode. The script prints invoked commands and outputs, runs autopep8 with selected fixes, then runs `ruff --fix --unsafe-fixes` and `ruff format`. Requires ruff and autopep8.
Standardize docstring punctuation and formatting across modules, add stacklevel=2 to warnings.warn calls for clearer warning locations, and make zip/dict(zip()) usages more robust by passing strict=False where appropriate. Also include minor loop variable/name cleanups and CLI help text formatting fixes. These changes improve error tolerance when iterables differ in length, produce more helpful warning tracebacks, and tidy user-facing documentation strings.
Delete the generated ruff-report.md (large auto-generated lint report) and add tmp/* to .gitignore to avoid committing tool output. Also include an update to tools/ruff_report.py (script changes) alongside the cleanup.
Rework deeplabcut package initialization: use relative imports, add a logger and robust DEBUG env parsing, and replace eager imports of optional GUI/PyTorch features with a lazy-loading mechanism. Optional public names (GUI and transformer_reID) are resolved on first access via __getattr__ (with clear error messages when dependencies are missing) and cached for subsequent use; __dir__ and a consolidated __all__ improve discoverability. Removes previous noisy prints and light-mode fallbacks in favor of structured logging and explicit public API grouping.
C-Achard and others added 7 commits March 18, 2026 16:26
Detect changed Python files in the workflow and generate a Ruff Markdown report for changed files. Bump actions (checkout@v6, setup-python@v6), install ruff alongside pre-commit, and add a step that runs tools/ruff_report.py to produce tmp/ruff-report.md. The workflow now publishes a short top-section to the GitHub Actions summary, uploads the full report as an artifact, and fails the job if pre-commit reports failures.

Also add a dedicated step to output changed Python file paths and wire that into report generation, and adjust the pre-commit run to allow collecting results before explicitly failing the job. Update tools/ruff_report.py to invoke Ruff via `python -m ruff` (ensures the venv-installed tool is used) and fix usage examples.
Co-authored-by: Cyril Achard <cyril.achard@epfl.ch>
@deruyter92 deruyter92 marked this pull request as ready for review March 19, 2026 17:26
@C-Achard C-Achard self-requested a review March 19, 2026 19:20
deruyter92 added a commit to deruyter92/DeepLabCut that referenced this pull request Mar 20, 2026
deruyter92 added a commit to deruyter92/DeepLabCut that referenced this pull request Mar 20, 2026
deruyter92 and others added 6 commits March 20, 2026 13:45
* Update pre-commit config

* Update pyproject.toml with new linting

* update pyproject.toml remove tool.isort (already specified via ruff).

* Update uv.lock

* align pre-commit config with PR #3216

* fix CI python-package workflow: remove unauthorized channel `defaults`

* Align pyproject.toml with PR #3216

---------

Co-authored-by: Cyril Achard <cyril.achard@epfl.ch>
Change get_fmpose3d_inference_api to use None as the default for config_kwargs and initialize it to an empty dict inside the function. This avoids the shared mutable default argument ({}), preventing accidental state leakage across calls. Updated in deeplabcut/pose_estimation_pytorch/modelzoo/fmpose_3d/fmpose3d.py.
Small refactor in fmpose_3d inference: bind outer variables (predictions_2d, all_poses_3d) as default args for the nested _process_batch to ensure correct capture when used as a callback, and tidy up several long-wrapped statements for readability (create_empty_df call, dest_folder handling, logger.info, and the poses_3d_serialisable list comprehension). No functional changes intended.
Clean up pyproject.toml by removing the temporary [tool.yapf] linting configuration and its related comment. This deletes the previous based_on_style and indent_width settings no longer needed after linting changes.
@C-Achard
Copy link
Copy Markdown
Collaborator

@deruyter92 Small note, I'd be in favor of merging this as soon as possible, it'd be very helpful to have this be usable in other PRs.

@deruyter92 deruyter92 requested a review from MMathisLab March 27, 2026 09:48
@MMathisLab MMathisLab merged commit c190b94 into main Mar 30, 2026
30 of 31 checks passed
@MMathisLab MMathisLab deleted the jaap/reformat_codebase branch March 30, 2026 16:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CI Related to CI/CD jobs and automated testing enhancement New feature or request high-priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Task: Fix linting & and pre-commit-hooks and add to CI workflow

3 participants