feat(ENG-RECORD-ANCHOR-RATCHET): parse the citation form the checker never read, and ratchet its rot - #851
Open
localai-bot wants to merge 1 commit into
Open
feat(ENG-RECORD-ANCHOR-RATCHET): parse the citation form the checker never read, and ratchet its rot#851localai-bot wants to merge 1 commit into
localai-bot wants to merge 1 commit into
Conversation
…never read, and ratchet its rot `check-agent-record.py` looked like it validated the record's `file.cpp:123` citations and did not. `LINK_RE` matches markdown links, so `local_line_anchors` only ever saw `[label](path#L505)`. Re-derived over the five matrices at `0e8b15d56`: 479 link anchors carrying an `#L` fragment against 2134 bare citations inside backtick spans, so 17.2% of the citation forms a reader sees were examined as citations at all. Three gaps compounded. The bare form was unparsed. `is_code_anchor` answers with `any()`, so one good link in a cell covered every rotted citation beside it. And `EVIDENCED_STATES` omits `ACTIVE` and `READY`, so 83 live rows got no anchor check. Inside the fraction that was examined, only the RANGE was checked -- which is why three stale anchors survived the 2026-08-13/14 campaign and had to be caught by a human reading. All three were in range. This parses both forms, classifies each resolvable citation as OK, STALE (the line exists but does not contain the symbol named beside it) or BROKEN (out of range, or a filename gone from a directory we own), and gates the STALE+BROKEN count against `scripts/record-anchor-baseline.json` the way the DSR ratchet in `check-device-leakage.py` gates leakage: a bucket above its baseline fails, and a bucket below it fails too, so a repair must lower the baseline in the same commit. Measured backlog at adoption is 40 -- 33 stale, 7 broken, over 800 correct -- and every one of the 40 was verified by hand against the cited file before the number was written down. They are deliberately NOT repaired here; the ratchet exists so they are fixed by whoever next touches each row. `is_code_anchor`'s `any()` stays for the STATE gate: a row is still evidenced by one good anchor. The ratchet counts every citation independently, which is the half `any()` was hiding. The symbol test is conservative by measurement, not by taste. It reads only the `code` and `tests` cells, never `upstream`, which is what keeps 1446 upstream `vllm/...py:123` references structurally out of the count instead of relying on a path heuristic that `tests/`, `cmake/`, `docs/` and `src/` all defeat. A symbol is inferred only from an immediately adjacent backtick span that looks like an identifier -- 4+ characters carrying `_`, `::`, `()` or an uppercase letter, and not starting with `_`, which was added after a measured false positive on `` the text-only `_ModelInfo` ``. About six citations in seven yield no symbol and are OK by construction. That polarity is the point: a checker that cries wolf gets disabled, and this one has to survive a four-figure backlog. Nine cases in `RecordAnchorRatchet`, all captured red before implementation -- the six the spec names plus the two gate directions and the tree/baseline pin. Five mutants red them: report-only, `EVIDENCED_STATES` restored, links-only, first-citation-only, and range-only. The load-bearing case is the cell holding one good link beside one rotted bare citation, which is the exact shape the rot hid in; links-only, first-citation-only and range-only all red it. The ratchet is a mode of the checker rather than a new script because it needs `parse_claim_rows` and `field_index`; a separate checker would have re-derived both and dragged a `CREATION_MUTATIONS` entry through `check-pr-size.py` for no gain. It runs in `agent-preflight.sh` through `check-agent-record`, and the `agent-record` CI job now calls `--report` so the offender list is printed on every run, the way `check-device-leakage --report` is wired below it. FOLLOWING_AGENTS_PROTOCOL Following-Agents-Protocol: true AI-Assisted: true Assisted-by: AGENT:claude-opus-5 [Claude Code]
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes the implementation half of #632, row
ENG-RECORD-ANCHOR-RATCHET(
SPIKE->ACTIVE), against the committed spec.agents/specs/record-anchor-ratchet.md.The defect, re-derived
check-agent-record.pylooked like it validated the record'sfile.cpp:123citations.
LINK_REmatches markdown links, solocal_line_anchorsonly eversaw
[label](path#L505). Re-derived over the five matrices at0e8b15d56bycounting
path:linetokens inside backtick spans:path:line(invisible)engine-matrix.mdmodel-matrix.mdkernel-matrix.mdquantization-matrix.mdbackend-matrix.md17.2% of citation forms were examined. Three gaps compounded: the bare form
was unparsed;
is_code_anchoranswers withany(), so one good link coveredevery rotted citation beside it; and
EVIDENCED_STATESomitsACTIVEandREADY, so 83 live rows got no anchor check at all. Inside the fraction thatwas checked, only the RANGE was checked, which is why three stale anchors
survived the 2026-08-13/14 campaign and had to be caught by a human reading.
All three were in range.
What this adds
`path:line`and`path:line-line`beside theexisting link form (
BARE_CITATION_RE,cell_citations).it), BROKEN (out of range, or a filename gone from a directory we own).
scripts/record-anchor-baseline.json, gated in both directionslike
scripts/device-leakage-baseline.json: above it fails, below it failsuntil the baseline is lowered in the same commit.
--reportprinting every offender with file, line and expected symbol.ACTIVEandREADYjoin the counted states.is_code_anchor'sany()stays for the STATE gate. A row is still evidenced byone good anchor; the ratchet counts every citation independently, which is the
half
any()was hiding.The measured backlog
40: 33 STALE, 7 BROKEN, over 800 OK. Every one of the 40 was verified by
hand against the cited file before the number was written down. Examples:
scripts/check-commit-trailers.py:38namesFORGE_ACCOUNT_EMAIL, which is at:47.src/vllm/entrypoints/openai/server_main.cpp:289nameskAcceptedInertArgs,which is at
:277.src/vllm/model_executor/models/registry.cpp:10-20is gone; the file is nowmodel_registry.cpp.examples/server/main.cppat lines 96 to 247. That file has 23lines.
They are deliberately not repaired here. The ratchet exists so they are
fixed by whoever next touches each row, which is the design the spec chose over
a bulk cleanup.
Where the conservative line is drawn
Precision over recall at five measured points:
codeandtestscells are read, neverupstream. That is whatkeeps 1446 upstream
vllm/...py:123references out of the countstructurally;
tests/,cmake/,docs/,src/andtools/all collidewith upstream references here, so a path heuristic cannot do it.
so llama.cpp's
src/llama-model.cppis skipped rather than blamed on us._,::,()or anuppercase letter, and not starting with
_. The leading-underscore rule wasadded after a measured false positive on
the text-only `_ModelInfo`,where the anchor was right.
About six citations in seven yield no symbol and are OK by construction.
Tests and mutation
Nine cases in
RecordAnchorRatchet, all captured RED before implementation(
AttributeError: module 'agent_record' has no attribute 'scan_record_anchors'x7, then 2 more). Five mutants red them:
test_new_rot_fails_the_gate,test_a_repair_fails_until_the_baseline_is_loweredRECORD_ANCHOR_STATESback toEVIDENCED_STATEStest_active_row_anchors_are_counted,test_baseline_matches_the_tree_exactlyanyrestored)test_one_good_link_does_not_cover_a_rotted_bare_citationThe load-bearing case is the cell holding one good link beside one rotted bare
citation, the exact shape the rot hid in. Links-only, first-citation-only and
range-only all red it. The tree was restored byte-for-byte after each mutant
(
sha256sum -cOK, five times plus a final check).Wiring
scripts/agent-preflight.shgates it throughcheck-agent-record(a red therenames the bucket that moved, and
runonly shows 12 lines, which is why theoffender list is not wired there). The
agent-recordCI job now calls--report, the waycheck-device-leakage --reportis wired below it.The ratchet is a mode of the checker rather than a new script because it needs
parse_claim_rowsandfield_index. A separate checker would have re-derivedboth and dragged a
CREATION_MUTATIONSentry throughcheck-pr-size.pyfor nogain.
Rebase onto #846, and the ratchet catching its own author
mainmoved the issue index out ofroadmap_v1.mdwhile this was in flight.Resolved by taking main's
roadmap_v1.mdwholesale and re-appending #632 to.agents/issue-index.md;tests/scripts/test_agent_record.pywas a union (bothsides appended a new class at the end, neither replaces the other).
The rebase then shifted every line this row's own
Our codecell cites, and theratchet reported all seven as STALE and reddened the gate. That is the first
thing it did on landing, so the row is repaired and the count is back at 40.
Gates
Host and load stated, because one of them is load-dependent.
scripts/check-agent-record.pymudler-ubuntu-box, 20 coresOK ... ANCHOR-ROT=40python3 tests/scripts/test_agent_record.pypython3 tests/scripts/test_check_gate_commands.pyscripts/agent-preflight.shtest_cpu_x86_llamacpp_floorcheck-commit-trailers.py --range <merge-base>..HEADcheck-commit-style.py --range <merge-base>..HEADcheck-pr-size.py --base origin/main --head HEADtests/scripts/test_cpu_x86_llamacpp_floor.pykairos-4db2, 14 cores, load 3.6RecordAnchorRatchettest_cpu_x86_llamacpp_flooris #618 and is not this change: the failing caseis
test_the_quiet_gate_does_not_see_the_harnesss_own_process_tree, which exitsNO_QUIET_WINDOWbecause the harness's own quiet-window precondition neverarrives on a loaded box (
busy=150% load=28.98). It passed on this same treecontent at load 20 on the dev box, and the identical tree was shipped to Thor
with
git archive | ssh tar -xand passes there at load 3.6. No changed pathtouches that harness.
Note for anyone re-running the record gate from a
git archivetree: it has no.git, socommit_exists()reports everyDONErow's closing commit missing.That is the instrument, not the record.
No GPU, no build, no network: this is a Python checker and its suite.
🤖 Generated with Claude Code