Skip to content

review: treat a new negative assertion as a decision, not as coverage - #2861

Open
EmilianoU26 wants to merge 1 commit into
garrytan:mainfrom
EmilianoU26:pr2-negative-assertions
Open

review: treat a new negative assertion as a decision, not as coverage#2861
EmilianoU26 wants to merge 1 commit into
garrytan:mainfrom
EmilianoU26:pr2-negative-assertions

Conversation

@EmilianoU26

Copy link
Copy Markdown

An assertion that a behavior is ABSENT (not in, assertNotIn,
not.toContain, pytest.raises on a call that used to succeed) is part of the
behavior change, not test coverage — and it arrives in the diff wearing
coverage's clothes, so scope-creep detection walks past it.

Three things the review now does:

  • Scope check: a new negative assertion needs the stated-intent line that
    authorizes the removal. With nothing to cite, it is scope creep signed as
    coverage.
  • The testing specialist states the revert cost in the finding. Once merged,
    restoring the old behavior makes these tests fail, so whoever reverts looks
    like the author of the regression.
  • The one that is easy to miss: a PRE-EXISTING negative assertion emptied by
    this diff. A rename or removal makes a standing assert "<old token>" not in output true forever, so it stops guarding without ever going red. Grep the
    diff for renamed or deleted identifiers, then grep the suite for negative
    assertions naming the old token — including the ones inside tests that assert
    something else.

The scope-check bullet is edited in scripts/resolvers/review.ts, which feeds
both /review and /ship; review/SKILL.md and ship/sections/plan-completion.md
are the regenerated output. review/specialists/testing.md is hand-maintained.

Kept deliberately terse in the skeleton: /review sits close to its
maxSkeletonBytes ratchet, so the always-loaded bullet is one line and the
detail that earns its keep lives in the specialist, which loads on demand.
Parity suite: 21/21.

🤖 Generated with Claude Code

An assertion that a behavior is ABSENT (`not in`, `assertNotIn`,
`not.toContain`, `pytest.raises` on a call that used to succeed) is part of the
behavior change, not test coverage — and it arrives in the diff wearing
coverage's clothes, so scope-creep detection walks past it.

Three things the review now does:

- Scope check: a new negative assertion needs the stated-intent line that
  authorizes the removal. With nothing to cite, it is scope creep signed as
  coverage.
- The testing specialist states the revert cost in the finding. Once merged,
  restoring the old behavior makes these tests fail, so whoever reverts looks
  like the author of the regression.
- The one that is easy to miss: a PRE-EXISTING negative assertion emptied by
  this diff. A rename or removal makes a standing `assert "<old token>" not in
  output` true forever, so it stops guarding without ever going red. Grep the
  diff for renamed or deleted identifiers, then grep the suite for negative
  assertions naming the old token — including the ones inside tests that assert
  something else.

The scope-check bullet is edited in `scripts/resolvers/review.ts`, which feeds
both `/review` and `/ship`; `review/SKILL.md` and `ship/sections/plan-completion.md`
are the regenerated output. `review/specialists/testing.md` is hand-maintained.

Kept deliberately terse in the skeleton: `/review` sits close to its
`maxSkeletonBytes` ratchet, so the always-loaded bullet is one line and the
detail that earns its keep lives in the specialist, which loads on demand.
Parity suite: 21/21.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@trunk-io

trunk-io Bot commented Sep 13, 2026

Copy link
Copy Markdown

Merging to main in this repository is managed by Trunk.

  • To merge this pull request, check the box to the left or comment /trunk merge below.

After your PR is submitted to the merge queue, this comment will be automatically updated with its status. If the PR fails, failure details will also be posted here

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants