Skip to content

[finding] os-verify-lock.sh's VERDICT line reports the BATCH SCRIPT's exit, not each command's — a failing command inside a batch is announced as command-exit 0 #12288

Description

@os-warren

Filed by the domain:engine PM seat (session_01W6HFzyH98W1YaQXhJUJt6o), measured during #11567 (PR #12287) on 2026-08-25. ⛔ A defect in the verification instrument, which every dev in every lane is briefed to quote — so it costs more than one card if it stays unrecorded.

scripts/pm/** and the pm-dispatch brief text are governed surfaces, maintainer hand-merge only. Filing, not fixing.

What was measured, in the dev's own words

my first batched test run printed VERDICT command-exit 0 while runtime inside it was exit=1 — the wrapper reports the BATCH SCRIPT's exit, not each command's. The per-command capture is what caught it; I read the per-command exits, not the VERDICT, for that batch.

Why this is worse than an ordinary bug

scripts/pm/os-verify-lock.sh's VERDICT command-exit N line is exactly what dispatch briefs tell devs to quote as proof a suite passed — precisely because it is more trustworthy than a bare $? after a pipe. For a single command it is. For a batch, it reports the exit of the wrapper script, and a shell script's exit is its last command's unless set -e is in force — so a failure in the middle is announced as success.

⚠️ It fails in the green direction, on the one line a reviewer is told to trust. Every downstream control reports success: the lock was held, the command ran, the verdict says 0. Nothing is red anywhere.

This is the same family as the restore-leg hazards already recorded — an instrument that reports success while measuring the wrong thing — and it now has three members:

card instrument failure
#11539 ablation trap restore ran from the wrong cwd, did not restore, exited 0
#11648 git checkout -- PATH restored from the polluted index, exited 0
#12204 git checkout HEAD -- PATH restored perfectly, destroying uncommitted work; all checks pass
this os-verify-lock.sh VERDICT reports the batch's exit, not the failing command's

What a card here would decide

  1. Whether the wrapper should set -o pipefail / set -e, or aggregate per-command exits and report the worst, rather than the last.
  2. Failing that, whether the brief text should say plainly: the VERDICT line is authoritative for a SINGLE command only; for a batch, read the per-command exits. ⚠️ The current guidance ("quote the verdict line, never a bare $?") actively points devs at the wrong number for the batch case.
  3. Whether it should refuse a batch outright rather than answer a question it cannot answer correctly.

Not claimed

  • ⛔ Not a claim that any past result is wrong. The signature to look for is a batched run whose VERDICT reads 0 while a suite inside it reported failures in the body — this seat has not audited prior runs.
  • ⛔ No frequency measured — one occurrence, one seat, one day. It presumably affects only devs who batch, which the brief neither requires nor forbids.
  • ⛔ The exact remedy is not prescribed; the wrapper's own contract may have a reason for reporting what it does.

Dedup — bounded, not proven

⚠️ search_issues is not answering from this seat: repo:… is:issue is:open in:title finding returns total_count: 0 when at least six open issues carry "finding" in their titles. The positive control does not fire, so no zero from that channel is a reading (#11835). Direct issue_read of #11539, #11648 and #12204 confirms none covers this instrument. If a covering card exists elsewhere, close this as a duplicate.

Refs

#11567 / PR #12287 (where it was measured) · #11539 · #11648 · #12204 (the same instrument-reports-success family) · #11363 (verify-lock contention, the other standing hazard on this script)

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions