Skip to content

test(qa): discover expression surfaces by identity and through file-local aliases, and classify the five positions that were blind - #18106

Merged
os-warren merged 4 commits into
mainfrom
claude/issue-17630-expression-conformance-blind-mechanisms
Sep 14, 2026
Merged

os-warren merged 4 commits into
mainfrom
claude/issue-17630-expression-conformance-blind-mechanisms

Conversation

@os-warren

@os-warren os-warren commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Fixes #17630

Triage ruling of record: 5657520516direction 1 + 2 together, one PR, not 3, and classify all five in the same PR. The card body's line numbers are stale; everything below is re-measured on this branch's base a26a114d7.

What was blind

Discovery in the ADR-0058 D7 expression-conformance ratchet was a line regex anchored to the HEAD of the declaration, so a roster schema mounted anywhere but immediately after field: was invisible — including with its name literally on the line. Five declaring positions were unclassified while the ratchet reported a complete classification and stayed green.

position line on a26a114d7 mechanism
system/metrics.zod.ts:ServiceLevelIndicatorSchema.successCriteria :477 A — union member
system/tracing.zod.ts:TraceSamplingConfigSchema.condition :349 A — union member
ui/component.zod.ts:RecordAlertProps.visible :1594 A — union member
ui/action.zod.ts:actionObject.visible :1378 (alias :832) B — file-local alias
ui/action.zod.ts:actionObject.disabled :1393 (alias :832) B — file-local alias

The two action keys are spelled actionObject, not ActionSchema: the ratchet key is mechanically file:SCHEMA.FIELD, where SCHEMA is the enclosing column-0 declaration and the fields live in the actionObject factory that ActionSchema = lazySchema(() => actionObject().refine(…)) derives from. Kept mechanical on purpose — re-keying to the exported name would move coverage on rows nobody reads the diff of, which is #15500's lesson.

What changed

expression-conformance.test.ts — discovery only:

  1. Drop the head anchor. A roster name is matched by IDENTITY anywhere on a line (lookarounds, so ExpressionInputSchema does not fire inside CronExpressionInputSchema), attributed to the field: it mounts: the key on the same line, else the nearest preceding key at strictly smaller indentation, bounded by the enclosing top-level declaration. The indentation constraint is load-bearing — see ablation D below. Strict vs inclusive comparison is not (ablation C): every sibling key inside a structured union arm is deeper than the member line, so <= changes no key today; < is kept as the stricter rule, not as a measured distinction.
  2. Resolve aliases. A file-local const X = …RosterSchema… registers X for the rest of that file, at any indentation, including the multi-line spelling. A name already on the roster registers nothing, which is why system/settings-manifest.zod.ts:353's SettingsVisibilityInputSchema is correctly not treated as a hazard.
  3. Two new pins, because a widening that silently un-widens is this card's own defect wearing a new face:
    • a per-MECHANISM floor (SCAN_CONTROLS): head >= 37, inline >= 3, alias >= 2. The ratchet alone does not catch a re-narrowing done together with deleting the five rows — both of its sides then agree again, green, over a population five positions smaller.
    • no roster hit is DROPPED: a hit the scan cannot attribute is reported, not discarded. Discarding is exactly how the five stayed invisible. Non-declaring shapes (imports and their continuation lines, comment prose, barrel re-exports, the export type X = z.input typeof RosterSchema companion beside every roster member) are excluded structurally, never by a position-keyed allowlist.

expression-conformance.ledger.ts — four rows and an honest header:

row covers tier
cel-action-visible actionObject.visible enforced / fail-closed
cel-action-disabled actionObject.disabled enforced / fail-soft-log
cel-record-alert-visible RecordAlertProps.visible enforced / fail-closed
cel-declared-unwired-observability SLI successCriteria + trace-sampling condition experimental / unevaluated

Split rather than merged where the fail FACES differ: a faulting action visible hides the action, a disabled the renderer cannot evaluate leaves it runnable (objectui#3842/#3848/#3849 removed the closed face as a defect). One row stating one of those would silently speak for the other — #15500's lesson again.

The header restates the three surviving limits instead of the one it named (it also carried a dangling half-sentence, Two limits survive and / One limit survives, which this rewrite removes): an independently defined schema nobody registered; alias resolution being FILE-LOCAL, so shared/expression.zod.ts:PredicateInputSchema would still be invisible across files (latent — measured, zero slot users); and attribution being textual, which the unattributed-hits pin turns into a hard failure rather than silence.

Both fences held

Fence A — the answer is reached by reading packages/spec/src as TEXT. No new packages/spec export, no dependency on a built packages/spec, no import() of a spec module from the test. The diff touches no file under packages/spec/.

Fence B — no new tier and no tier redefined: the four rows use enforced/experimental and fail-closed/fail-soft-log/unevaluated, all pre-existing, and the two consumer-less positions apply the ledger's own PARSE-ONLY tier the way cron-knowledge-refresh, template-prompt and cel-advanced-policy already do. No behaviour changes — see below.

Discovery before/after, with controls

Replaying the two scans over the 200 .zod.ts files of packages/spec/src at a26a114d7:

files scanned                   200
head-anchored discovery (OLD)    37   <- CONTROL: the old regex still fires
raw roster identity hits         88   <- CONTROL: the identity scan sees the whole population
widened discovery (NEW)          42
  only in NEW                     5   <- exactly the five above
  only in OLD                     0   <- no regression
aliases registered                2   <- ActionConditionInputSchema, PredicateInputSchema
unattributed roster hits          0

Both controls fire, so the 5 is a reading rather than a search that could only answer one way. +2 manual RLS rows on each side: 39 -> 44 ratchet keys.

The widening is what found them, measured rather than asserted — the intermediate commit that carried the widened discovery with the ledger untouched fails with exactly these five and nothing else:

× is a sound conformance ledger + ratchet (ADR-0060 checkLedger)
  UNCLASSIFIED surface — add a ledger row (ADR-0060): system/metrics.zod.ts:ServiceLevelIndicatorSchema.successCriteria
  UNCLASSIFIED surface — add a ledger row (ADR-0060): system/tracing.zod.ts:TraceSamplingConfigSchema.condition
  UNCLASSIFIED surface — add a ledger row (ADR-0060): ui/action.zod.ts:actionObject.visible
  UNCLASSIFIED surface — add a ledger row (ADR-0060): ui/action.zod.ts:actionObject.disabled
  UNCLASSIFIED surface — add a ledger row (ADR-0060): ui/component.zod.ts:RecordAlertProps.visible
Test Files  1 failed (1) | Tests  1 failed | 6 passed (7)

With the four rows in place: Test Files 1 passed (1) · Tests 7 passed (7).

Reverse verification — four ablations, each restored and proved restored

Run from the committed state; each leg proves the mutation reached disk (occurrence counts of the deleted and injected text, plus the blob hash), then restores with git checkout HEAD -- PATH and proves git diff HEAD empty and the blob hash back at the HEAD blob 3807350190681792b4ece28bca545b0485bd4a30. No build sits between the edit and the run: the scan reads packages/spec/src as text at run time and vitest loads the test file directly, so a mutation reaches the measurement with nothing cached in between.

leg mutation observed reads as
A re-narrow to the head anchor (isRoster = DECLARES_EXPRESSION.test(line), isAlias = false) 3 failed / 4 passed — ratchet with all five STALE covers; mechanism floor inline 0 vs floor 3; unattributed-hits pin the widening is what finds the five
B kill the alias-binding line (LOCAL_BINDING never matches) 1 failed / 6 passed — unattributed-hits pin only ⚠️ not a clean isolation of mechanism B: with the binding skip gone, the multi-line-alias fallback still registers ActionConditionInputSchema, so both action keys stay discovered. What the pin caught is the roster's OWN definition lines becoming unplaceable. Reported as observed, not as predicted
C relax the attribution comparison (< to <=) 7 passed no key moves — the strict/inclusive distinction is not measurable on this tree, and the PR text above was corrected to say so
D drop the indentation constraint entirely (if (key)) 1 failed / 6 passedUNCLASSIFIED ... ServiceLevelIndicatorSchema.percentile + STALE covers ... ServiceLevelIndicatorSchema.successCriteria the constraint is what keeps attribution off a sibling inside the structured arm

Legs A and D are the ones that carry the claim; C is a negative result that corrected the text; B is reported honestly as a partial isolation rather than dressed up.

Verification

Derived with node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack against the actual changed paths, every exit code captured to disk before any pipe, then reconciled with --ran:

Run reconciliation — 53 derived, 52 run, 1 NOT-MEASURED, 0 UNRUN.
  NOT-MEASURED · DERIVED (1) — recorded exit 3, PREREQUISITE NOT MET:
    - pnpm check:dual-build-cjs-loads

check:dual-build-cjs-loads reads built output and refused its own prerequisite (eight packages have no dist/ in this worktree; it needs a full pnpm build). That is NOT MEASURED, not a finding — and this diff ships no source into any dist, so CI's Build Core measures it on a fresh checkout. Two gates needed a re-run rather than a verdict: check:doc-authoring was a real red on this diff (tracker ids inside runtime string prose — the ids moved to the adjacent // comments, the idiom the earlier rows already use) and is green; check:type-check-debt OOMed at --max-old-space-size=4096 under container contention, exit 3 = nothing measured, green at 8192.

Beyond the derived set, and named because the derivation does not name it: pnpm lint, the full unioneslint . --no-inline-config over 6,746 files (count read from --format json, not estimated), 0 errors, 0 warnings, at 16f8b1cca7, the final commit. No narrowing was used, so no narrowing needs justifying.

Also run: pnpm --filter @objectstack/dogfood typecheck green, with tsc --noEmit --listFiles confirming both changed files are in the program (a package whose typecheck excluded *.test.ts would have reported green over neither). pnpm --filter @objectstack/dogfood exec vitest run test/expression-conformance.test.ts — 7 passed. A control-character scan over both files (grep -naP for the C0 range and DEL) beyond check:nul-bytes: clean.

origin/main moved to 66aa2d98d6 while this ran; the only paths it moved are .claude/skills/pm-dispatch/references/contract-review.md and scripts/pm/check-governed-merges.mjs. packages/spec/src is untouched across that range, so nothing this diff measures moved, and the gate derivation was byte-identical across two runs two commits apart.

Nothing evaluates differently

Structural: the diff is two files, both under packages/qa/dogfood/test/, in a "private": true package that no runtime imports. No schema, producer or evaluator is touched, so no predicate's accepted set can have moved.

Measured anyway, at the mounted slot (never the schema in isolation), against the two non-evaluable envelopes the spec's own EVALUATED_EXPRESSION_SOURCE_REQUIRED names — an ast-only envelope, and a source blank after trimming:

slot                                                          good  ast-only  blank-source
ui/action.zod.ts:actionObject.visible                         true  true      true
ui/action.zod.ts:actionObject.disabled                        true  true      true
ui/component.zod.ts:RecordAlertProps.visible                  true  true      true
system/metrics.zod.ts:ServiceLevelIndicatorSchema.successCriteria true  true      true
system/tracing.zod.ts:TraceSamplingConfigSchema.condition     true  true      true
CONTROL automation/flow.zod.ts:FlowEdgeSchema.condition       true  false     false
CONTROL builtin-node-config.zod.ts:AssignmentExpressionValueSchema true  false     false

The two EvaluatedExpressionInputSchema-typed controls refuse both, which is what makes the five rows of true a reading.

Changeset

skip-changeset. @objectstack/dogfood is "private": true — it publishes nothing, so no released artefact moves. The diff adds no key, export or schema.

Acceptance notes

Provenance

Authored by the os-dev seat for the domain:cli lane, session session_01TbSMtGzMrtPwh925wDEZd5, under PM claim 5658038794 and triage ruling 5657520516. Measured on base a26a114d7; the lint union and the ratchet families were re-run on the final commit 16f8b1cca7.

Attribution is stated here in prose deliberately: this body was created through the MCP-free REST channel and then EDITED through it, and the edit path APPENDS its own footer block regardless of what the body already carries — measured on this very PR, where a session-URL footer I sent survived and a bare block landed underneath it, leaving two. So the footer below is the platform's, the session id above is the durable record, and this body is not re-sent to 'fix' the footer.


Generated by Claude Code

…y found

Discovery in the ADR-0058 D7 expression-conformance ratchet was anchored to the
HEAD of the declaration, so a roster schema mounted as a union member or behind
a file-local alias const was invisible — including with its name literally on
the line. Five declaring positions were unclassified while the ratchet reported
a complete classification.

Discovery now matches a roster name by IDENTITY anywhere on a line, attributes
it to the `field:` it mounts (nearest preceding key at strictly smaller
indentation, bounded by the enclosing top-level declaration), and resolves
file-local `const X = ...RosterSchema...` aliases for the rest of that file.
Two new pins keep it from silently un-widening: a per-MECHANISM floor, and a
refusal to DROP a roster hit the scan cannot attribute.

Four ledger rows classify all five positions — three first classifications of
enforced UI predicates, and one `unevaluated` / PARSE-ONLY row for the two
observability positions measured to have no consumer outside `packages/spec/src`.
No behaviour changes: all five accept the two non-evaluable envelopes exactly as
before.

Claude-Session: https://claude.ai/code/session_01TbSMtGzMrtPwh925wDEZd5
Co-authored-by: Claude <noreply@anthropic.com>
@os-warren os-warren added the skip-changeset PR has no user-facing published change; bypasses the changeset gate label Sep 14, 2026 — with Claude
@github-actions github-actions Bot added the tests label Sep 14, 2026
@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown
Contributor

📓 Docs Drift Check

Nothing in this diff resolved to a documentable surface (no symbol, route or SDK anchor derived from 0 changed package(s)), so this run has no opinion about the docs.

What this run could not see
  • a page that states a rule by its inputs shares no identifier with the emitter that implements the rule, so an emitter-only diff cannot list it — not on this run and not on any run. Measured on fix(driver-sql): emit varchar(maxLength) for a text field a declared index keys on #11430: content/docs/protocol/objectql/types.mdx documents the text-family column mapping by the ObjectQL type names it maps FROM (text / textarea / html) while the diff changed createColumn; it went unlisted, and it was the page that diff falsified, in four places. No shared token exists to detect this on, so a rule your change carries has to be re-read by hand in the pages that restate it.

Coarse fallback — 0 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): node scripts/docs-audit/affected-docs.mjs --json 82d8942ec89045312efa3d31a69ee4d4744bf828packageMentionDocs.

check:doc-authoring's sibling-package prose-id leg refuses `#NNNN` inside a
runtime string: an author, operator or generated surface reading the cell cannot
resolve a tracker id. The ids move to the adjacent `//` comments, which is the
idiom the rows already added by earlier cards use.

Claude-Session: https://claude.ai/code/session_01TbSMtGzMrtPwh925wDEZd5
Co-authored-by: Claude <noreply@anthropic.com>
…umed one

The ablation that mattered was the indentation constraint as a whole, not the
strict-vs-inclusive comparison: relaxing `<` to `<=` changes no key today
(measured — every sibling key inside a structured union arm is deeper than the
member line), while dropping the constraint entirely misattributes
`successCriteria` to `percentile`. The docblock now says which of the two is a
reading and which is a stricter rule kept for its own sake.

Claude-Session: https://claude.ai/code/session_01TbSMtGzMrtPwh925wDEZd5
Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/m skip-changeset PR has no user-facing published change; bypasses the changeset gate tests

Projects

None yet

2 participants