feat: report checked value in JSON output for passing checks by shenxianpeng · Pull Request #522 · commit-check/commit-check

shenxianpeng · 2026-08-04T21:57:03Z

What

validate_all_detailed() now populates CheckOutcome.value on pass as well as fail, with the concrete value each validator actually checked.

Why

Structured consumers (--format json, the Python API, and commit-check-action) currently receive value: "" for every passing check. The action's success report wants to show what was validated (PR title, commit subject, branch name, author name/email) — that data is collected in the validators but thrown away on pass.

Changes

BaseValidator._checked_value — set by each validator to the value it checked
Validators now record: commit message (message, signoff, body, commit-type checks), subject (subject_*), author name/email, branch name, merge-base target, push refs (no_force_push)
validate_all_detailed() pass branch passes validator._checked_value through
CheckOutcome.value docstring updated
Value collection is opt-in (_collect_value, set by validate_all_detailed); text-mode validation stays free of the extra lookups and keeps values empty

No change to terminal output: the text path never printed values on pass, and it still doesn't.

Example

{
  "rule_id": "CC001",
  "check": "message",
  "status": "pass",
  "value": "feat: add login page"
}

Value semantics

value is the raw input the validator actually checked, identical on pass and fail (fail uses the same value recorded by _print_failure), except where noted:

Check(s)	`value` on pass	`value` on fail
`message`, `require_signed_off_by`, `require_body`, `allow_*_commits`	full commit message	full commit message
`subject_*`	subject (first line)	subject (first line)
`author_name` / `author_email`	the identity checked (git config, falling back to the last commit's author)	same
`branch`	branch name	branch name
`merge_base`	current branch name	current branch name
`no_force_push`	`local_ref -> remote_ref` per pushed ref (newline-separated when several) or `branch -> upstream`	same ref pair that triggered the failure
`ai_attribution`	the scanned commit message (forbid policy, no signature found)	comma-separated detected tool names

Notes for consumers:

Message-class values may be multi-line (subject + body); take the first line for a subject.
ai_attribution is the one check where pass/fail values have different shapes (message vs. tool list) by design.
When a check is skipped (e.g. author in ignore_authors, no upstream configured for force push), value stays "".

Tests

490 tests pass (was 479); added coverage for pass-value in JSON mode, message/subject/author/branch validators, ignore_authors, ai_attribution, multi-ref force push, and text-mode zero-overhead (_collect_value off skips the branch lookup)

Summary by CodeRabbit

Improvements
- Detailed validation results now show the specific commit information evaluated by each check, including passing checks.
- JSON validation output now includes the validated value for successful checks.

validate_all_detailed now populates CheckOutcome.value on pass as well as fail, with the concrete value each validator checked (commit subject, branch name, author name/email, push refs). Structured consumers such as --format json, the Python API, and commit-check-action can therefore show what was validated even when every check passed.

coderabbitai · 2026-08-04T21:57:22Z

📝 Walkthrough

Walkthrough

Validation results now retain the concrete value evaluated by each validator. Detailed and JSON outputs include these values for passing checks. Tests cover message, subject, author, and branch validation.

Changes

Validation value reporting

Layer / File(s)	Summary
Record checked values `commit_check/engine.py`	`CheckOutcome.value` documents the checked value. Validators record the relevant message, subject, author, branch, merge-base, signoff, body, commit type, or force-push value.
Expose and validate checked values `commit_check/engine.py`, `tests/engine_test.py`, `tests/main_test.py`	`validate_all_detailed` includes recorded values for passing checks. Tests verify detailed and JSON output values.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

commit-check/commit-check#512: Both PRs enrich CheckOutcome and validate_all_detailed() results with additional metadata.

Suggested labels: minor

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly and concisely describes the main change: reporting checked values for passing validation results in JSON output.

✨ Finishing Touches

📝 Generate docstrings

Create stacked PR
Commit on current branch

🧪 Generate unit tests (beta)

Create PR with unit tests
Commit unit tests in branch feature/checked-value-in-json-output

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

codecov · 2026-08-04T21:58:50Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.68%. Comparing base (5c39e5f) to head (433f6a7).

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #522      +/-   ##
==========================================
+ Coverage   97.40%   97.68%   +0.28%     
==========================================
  Files          12       12              
  Lines        1231     1254      +23     
==========================================
+ Hits         1199     1225      +26     
+ Misses         32       29       -3

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

coderabbitai

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@commit_check/engine.py`:
- Line 652: Update ForcePushValidator.validate() so each successful local_ref ->
remote_ref result is accumulated rather than overwriting self._checked_value,
and serialize the complete collection according to the CheckOutcome.value
contract consumed by validate_all_detailed(). Add a regression test covering a
successful multi-reference push and asserting every validated reference is
preserved.
- Around line 703-704: Update the `ignore_authors` rule setup around
`CommitTypeValidator` so `_checked_value` is assigned the resolved commit author
before `_should_skip_commit_validation()` evaluates it, rather than the commit
message. Preserve the empty result for ignored authors and verify both ignored
and non-ignored author paths.
- Line 914: Update AiAttributionValidator.validate() to assign _checked_value on
every concrete validation path, including successful returns after reading the
commit body and when the policy is not forbid. Preserve the existing failure
assignment and ensure validate_all_detailed() receives the actual ai_attribution
value for passing results.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

Push a commit to this branch (recommended)
Create a new PR with the fixes

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: d5c4c596-5bef-4f34-b6ea-d41350b8f40b

📥 Commits

Reviewing files that changed from the base of the PR and between 5c39e5f and 6a38436.

📒 Files selected for processing (3)

commit_check/engine.py
tests/engine_test.py
tests/main_test.py

- accumulate every validated ref pair in no_force_push instead of overwriting on multi-ref pushes - record the resolved commit author for the ignore_authors rule and keep the value empty when the author is ignored - record the scanned message for ai_attribution when forbid policy finds no signature; ignore policy remains a no-op without a value

codspeed-hq · 2026-08-04T22:15:11Z

Merging this PR will improve performance by ×3.4

⚡ 1 improved benchmark
✅ 426 untouched benchmarks
🆕 11 new benchmarks
⏩ 121 skipped benchmarks¹

Performance Changes

	Benchmark	`BASE`	`HEAD`	Efficiency
⚡	`test_empty_message_passes`	7.8 ms	2.3 ms	×3.4
🆕	`test_validate_all_detailed_author_reports_author_name`	N/A	2.2 ms	N/A
🆕	`test_validate_all_detailed_branch_reports_branch_name`	N/A	406.1 µs	N/A
🆕	`test_validate_all_detailed_reports_value_on_pass`	N/A	551.4 µs	N/A
🆕	`test_json_format_pass_reports_checked_value`	N/A	23.5 ms	N/A
🆕	`test_forbid_policy_clean_commit_records_message`	N/A	457.9 µs	N/A
🆕	`test_ignore_policy_records_no_value`	N/A	335.1 µs	N/A
🆕	`test_ignore_authors_records_resolved_author`	N/A	4 ms	N/A
🆕	`test_ignore_authors_skipped_keeps_value_empty`	N/A	2.4 ms	N/A
🆕	`test_multiple_push_refs_accumulate_checked_value`	N/A	1.5 ms	N/A
🆕	`test_upstream_fallback_structured_mode_records_value`	N/A	4.4 ms	N/A
🆕	`test_upstream_fallback_text_mode_skips_branch_lookup`	N/A	4.3 ms	N/A

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.

_{Comparing feature/checked-value-in-json-output (433f6a7) with main (5c39e5f)}

121 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩

The checked value now shares the branch name with the failure path instead of issuing a second git subprocess call, keeping the upstream fallback path at one branch-name lookup per run.

validate_all_detailed opts validators into value collection; text-mode validation stays free of the extra git subprocess calls (branch name lookup in the force-push upstream fallback, author resolution for ignore_authors), keeping the benchmarked pass paths unchanged.

sonarqubecloud · 2026-08-04T22:52:17Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

## What Adds the v2.13.1 entry to the changelog: - **New feature**: `--format json` and `validate_all_detailed()` now report the checked value on passing checks (PR [commit-check/commit-check#522](commit-check/commit-check#522)) - Highlights table row for 2.13.1, linking to the "Output for scripts and CI" recipe

shenxianpeng requested a review from a team as a code owner August 4, 2026 21:57

github-actions Bot added enhancement New feature or request tests Add test related changes labels Aug 4, 2026

coderabbitai Bot reviewed Aug 4, 2026

View reviewed changes

Comment thread commit_check/engine.py Outdated

Comment thread commit_check/engine.py

Comment thread commit_check/engine.py

shenxianpeng removed the tests Add test related changes label Aug 4, 2026

github-actions Bot added the tests Add test related changes label Aug 4, 2026

shenxianpeng added 2 commits August 5, 2026 01:29

refactor: reuse branch name lookup in force push upstream check

13ed4b6

The checked value now shares the branch name with the failure path instead of issuing a second git subprocess call, keeping the upstream fallback path at one branch-name lookup per run.

shenxianpeng removed the tests Add test related changes label Aug 4, 2026

shenxianpeng merged commit a0d3f77 into main Aug 4, 2026
28 checks passed

shenxianpeng deleted the feature/checked-value-in-json-output branch August 4, 2026 23:31

shenxianpeng mentioned this pull request Aug 4, 2026

docs: update changelog for v2.13.1 commit-check/commit-check.com#5

Merged

This was referenced Aug 7, 2026

fix: distinguish an absent commit message from an empty one #534

Merged

feat: report a skipped check as skipped, not as passed #537

Merged

Navigation Menu

Uh oh!

Uh oh!

feat: report checked value in JSON output for passing checks - #522

feat: report checked value in JSON output for passing checks#522
shenxianpeng merged 4 commits into
mainfrom
feature/checked-value-in-json-output

shenxianpeng commented Aug 4, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot commented Aug 4, 2026 •

edited

Loading

Walkthrough

Changes

Uh oh!

codecov Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

coderabbitai Bot left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed-hq Bot commented Aug 4, 2026 •

edited

Loading

Uh oh!

sonarqubecloud Bot commented Aug 4, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Uh oh!

Conversation

shenxianpeng commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What

Why

Changes

Example

Value semantics

Tests

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Uh oh!

codecov Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

coderabbitai Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

codspeed-hq Bot commented Aug 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Merging this PR will improve performance by ×3.4

Performance Changes

Footnotes

Uh oh!

sonarqubecloud Bot commented Aug 4, 2026

Quality Gate passed

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

shenxianpeng commented Aug 4, 2026 •

edited

Loading

coderabbitai Bot commented Aug 4, 2026 •

edited

Loading

codecov Bot commented Aug 4, 2026 •

edited

Loading

codspeed-hq Bot commented Aug 4, 2026 •

edited

Loading