Skip to content

gate(ROCM): M4 for Qwen3.5-0.8B GDN -- the oracle-backed near-tie gate as the RED anchor of the forward-divergence fix - #559

Draft
VikashLoomba wants to merge 3 commits into
mudler:mainfrom
VikashLoomba:row/ROCM-M4-GDN-GATE
Draft

gate(ROCM): M4 for Qwen3.5-0.8B GDN -- the oracle-backed near-tie gate as the RED anchor of the forward-divergence fix#559
VikashLoomba wants to merge 3 commits into
mudler:mainfrom
VikashLoomba:row/ROCM-M4-GDN-GATE

Conversation

@VikashLoomba

@VikashLoomba VikashLoomba commented Aug 13, 2026

Copy link
Copy Markdown
Contributor

M4 for Qwen3.5-0.8B (GDN) — the oracle-backed gate, landed green-shaped (issue #41)

Stacked: #549 (0.6B lane + oracle) ← the dispatch fix (#570) ← this gate. The fix lands below the gate, so every commit in the stack is individually green on the gate host and the committed goldens are ones the tree can actually reproduce.

Gate

test_qwen35_paged_engine.cpp — the GDN sibling of the Qwen3-dense SACRED gate: standard 16-prompt battery through the full paged engine, hard anchor + ≤0.5-nat near-tie band, backend proof over the 15-op GDN set. Goldens captured from the pinned vLLM-ROCm oracle on this box (K=10 per-prompt, deterministic in every cell) with manifest.json recording the oracle identity; the checkpoint resolves via parity::HfSnapshot at the pinned HF revision 2fc06364 (check-snapshot-pins clean).

Verdict: 16/16 PASS (15/16 strict token-exact vs the oracle, 1/16 near-tie band, max gap 0.125 nats, 0 forward-divergent).

Review-sweep rework (all three findings accepted)

  1. CI-red manifest fixed: qwen35_0_8b_greedy is in PendingRunnerOps() (the qwen36_gguf_greedy precedent) — the manifest itself stays; its oracle identity is the point.
  2. Fail-safe by device: the gate exits 77 (CTest Skipped) on any non-ROCm device and when the pinned checkpoint is absent — never a false green, never a foreign engine compared against ROCm-derived goldens.
  3. Green-shaped landing: the committed goldens are the FIXED engine's oracle-re-derived pair. The pre-fix capture (13/16 forward-divergent; first-token divergences on 6 prompts at 0.375–1.062 nats, 5 of 6 over band) is kept as evidence in the spec + parity ledger, not as goldens no code can pass. The anchor-drift REQUIRE now reads "REGRESSION SUSPECTED — bisect first"; golden re-derivation is the last step of a justified re-capture, not the response to a failure.

Prose corrections from review

First-divergence maximum is 1.062 nats (not 1.188), 5 of the 6 token-0 divergences are over the band (not all), and the headline gap is at the first over-band position rather than a post-divergence maximum.

@localai-bot

Copy link
Copy Markdown
Collaborator

Reviewed as part of a sweep over the open external PRs. The measurement behind this is real — I recomputed the verdict independently from the committed goldens and got exactly your numbers: 13/16 fail, 3 pass, max gap 14125 mnats at (prompt 8, tok 8), 6 prompts differing at token 0. Golden provenance also checks out: greedy_ids.npy and our_ids.npy differ in 196 of 256 positions so they are not the same source, and the prompt token ids are distinctly Qwen3.5's rather than copied from the 0.6B lane. The manifest recording the oracle identity is a genuine improvement over the existing goldens, which record none.

Three things before this can land.

1. It would turn main's CPU CI red for every subsequent PR.

tests/parity/goldens/qwen35_greedy_0_8b/manifest.json declares "op": "qwen35_0_8b_greedy", and no runner is registered for it, so test_op_parity.cpp:1933 hits FATAL ERROR: no runner for op 'qwen35_0_8b_greedy' — add one before committing goldens. That takes down build-test-cpu and both sanitize-cpu jobs — all three are the same single ctest failure, not sanitizer diagnostics (no ASan report, no TSan race anywhere in the logs), and all three are green on main.

The guard is behaving correctly: the existing qwen3_greedy_0_6b/ goldens only avoid it by having no manifest at all. So please fix it by adding the runner arm next to the existing qwen3_5_mtp_head / gdn_packed_decode_bf16 precedents, or a PendingRunnerOps() entry — not by deleting the manifest, which is the best thing in this PR.

2. The gate does not fail safe by device.

const bool other_partial = metal || tenstorrent;kCPU and kCUDA set neither flag, so nothing returns and the gate proceeds to teacher-force a ROCm-captured anchor against whatever device is running, with the entire backend proof (OpRegistered, selections>0, declines==0) sitting inside if (rocm) and therefore skipped. On a CPU or CUDA box with that checkpoint cached you get either a hard REQUIRE "anchor drift" failure attributed to a CUDA regression that does not exist, or a green gate with no proof of which device ran. The only thing preventing this today is that the pinned revision is not cached on anyone else's box, which is luck rather than a fail-safe. Two lines fixes it: if (!rocm) { MESSAGE(...); std::exit(77); }.

Related, and cheap: the checkpoint-absent path at :118-123 returns, so doctest prints assertions: 0 | 0 passed | 0 failed and Status: SUCCESS! — indistinguishable in a log from a gate that loaded a model and matched an oracle. tests/CMakeLists.txt:34 already wires SKIP_RETURN_CODE 77 for every test and test_voxtral_e2e.cpp:55 has the canonical SkipGate() helper; this is issue #463's exact pattern.

3. The RED anchor does not have teeth against the defect it names.

The pass/fail decision is if (mn > kNearTieMnats) prompt_ok = false; where mn is read from the committed neartie_gap_mnats.npy. The engine's output enters only via the anchor REQUIRE(got == our_ids). So the committed RED is a property of a data file rather than of the code, and no forward fix can turn it green — the anchor REQUIRE fires first. #570 confirms this by replacing the goldens rather than passing the gate, which means the gate never had the chance to fail for the original defect.

The shape that works is to land the gate with GREEN goldens alongside #570, keeping the RED capture as recorded evidence in the spec and the parity ledger, where it already lives in full.

One more worth fixing while you are here: the anchor-drift message ends "— re-run qwen3-neartie-gap.py to refresh the gap golden". After #570, that REQUIRE is the only thing catching a reintroduction of the dtype-dispatch bug, and its failure text tells the next engineer to regenerate the golden. It should say a drift is a regression until proven otherwise.

Numbers to correct in the prose: the six token-0 divergences are 0.375/0.750/0.750/0.938/1.000/1.062 nats, so the max is 1.062 not 1.188, and only 5 of the 6 are over the 0.5-nat band. And worst_gap scans positions after the first over-band token, where the oracle is teacher-forced on an already-diverged prefix — the defensible headline is the gap at the first over-band position.

On CI: agent-record and pr-size are the unrebased-fork artifact (now fixed on our side in #619) and Windows is the known-broken arm — but build-test-cpu and both sanitizers are genuinely yours, per finding 1.

No AMD hardware here, so nothing was executed on ROCm; findings 1–3 are static, and the verdict arithmetic above came from your committed files.

@VikashLoomba

Copy link
Copy Markdown
Contributor Author

All three findings accepted and reworked (commit a77a3b3, rebased onto current main):

  1. CI-red fixed: qwen35_0_8b_greedy is in PendingRunnerOps() next to the qwen36_gguf_greedy precedent — the manifest stays (its oracle identity is the improvement you credited).
  2. Fail-safe by device: the gate exits 77 (CTest Skipped) on any non-ROCm device AND on absent checkpoint (the test_voxtral_e2e reports SUCCESS with zero assertions when its weights env var is unset (and ~40 more gates share the shape) #463 pattern) — no more foreign-engine-vs-ROCm-goldens comparison, no more 0-assertion "SUCCESS".
  3. Green-shaped landing: the stack is reordered — the kernel fix (fix(ROCM): AttnQkNormRopeGate output-dtype dispatch -- the 0.8B GDN forward divergence root-caused and fixed (RED anchor turns GREEN) #570) now lands BELOW this gate, so the committed goldens are the fixed engine's oracle-re-derived pair and every commit in the stack is individually green on the gate host. The RED capture (13/16, max first-divergence 1.062 nats — corrected per your arithmetic, 5 of 6 over band) lives as evidence in the spec + parity ledger. The anchor-drift REQUIRE now reads "REGRESSION SUSPECTED — bisect the engine change first"; golden re-derivation is the last step of a justified re-capture, never the response to a failure.

Gate on gfx1100: 16/16 PASS (15/16 strict, 1/16 band, max gap 0.125 nats); 0.6B gate re-run green.

…lden lane in the Qwen3-0.6B near-tie gate

Issue mudler#41 M4 milestone. The oracle is upstream vLLM AT THE PIN (555967922,
runtime 0.23.1rc1.dev1511+g555967922 -- verified == the parity-pin block) built
from source as a ROCm wheel for gfx1100 in rocm/vllm-dev:base and committed as
the image vllm-rocm-oracle:555967922-gfx1100 (recipe: .agents/specs/rocm-m4-oracle.md).

Gate result on this box (4x RX 7900 XTX, ROCm 7.14, enforce_eager, batch=1,
flock ~/gpu.lock): oracle K=10 per-prompt greedy deterministic in every cell;
our ROCm engine 16/16 prompts PASS -- 11/16 strict token-exact vs base greedy,
5/16 near-tie band, max teacher-forced gap 0.125 nats, 0 forward-divergent,
125/125 assertions, backend proof all 8 Qwen3-dense ops selections>0 and
declines==0 on device 5. The p0 France/Italy flip is a literal 0.0000-nat tie
in the oracle's own logits (the Metal M3b signature, reproduced on ROCm).

Change shape: the parity gate's Metal/Tenstorrent device-golden lane gains the
kROCM arm (identical anchor+band logic, ROCm's own oracle-backed golden pair);
dgx base goldens byte-untouched; the *_rocm goldens are additive.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
…he 0.8B GDN forward divergence, root-caused and fixed

Issue mudler#41. The ROCm dispatcher for the fused full-attention preamble keyed its
template on the SOURCE dtype (qgate.dtype) instead of the OUTPUT dtype like the
CUDA lane (LaunchAttnPreambleOut switches on q_out.dtype). The 0.8B bf16 model
runs bf16 QKV-projection output + f32 q/k/gate outs (the f32-attention path; no
FA-2 on ROCm), which the src-keyed dispatch mis-launched as all-bf16 -- writing
bf16 bits through the f32 out pointers. Evidence chain (gfx1100, flock):
CPU==pinned-oracle token-exact / ROCm diverges at tok0; VT_DUMP_ACT per-layer
residual dumps: layers 0-2 clean (<=9.6e-3), layer 3 (first full-attn) 2.1e-1;
sub-stage dumps: block_out 4.4e-1 with post_input_norm clean; VT_DUMP_ATTN op
dumps: fa0_q rms-rel 1.196 with qkv/gate clean; VT_ROCM_ATTN_CPU_REF=1 unchanged.

Fix: dispatch on q_out.dtype; bf16-out requires bf16-src (VT_CHECK).

Teeth: the cross-device AttnQkNormRopeGate case gains the bf16-src -> f32-out
combo at the real 0.8B dims (Hq=8, Hkv=2, Dh=256, rot=64); mutation-proven
(fails with the fix reverted, passes with it).

Gates on this branch (gfx1100, flock ~/gpu.lock): test_backend_cross_device
19/19 (346 assertions, incl. the new combo); test_qwen3_paged_engine 16/16
unchanged; e2e 0.8B 'The capital of France is' -> ' Paris.\nThe capital of
France is' == pinned oracle == our CPU. The M4 0.8B gate with the re-derived
GREEN goldens lands in the stacked sibling on row/ROCM-M4-GDN-GATE; full HIP
ctest zero-delta vs base measured there.

Also: three pre-existing hipcc/clang-only build breaks that block ANY full HIP
build on this toolchain: same-line double CAPTURE in
test_qwen3_5_gdn_spec_routing.cpp and test_ops_quant_repack.cpp, and
-Wunused-const-variable on kGdnLayers (uses compile out on HIP). Debug
instrumentation env-gated and off by default (VT_DUMP_ACT / VT_DUMP_ACT_SUB /
VT_DUMP_ATTN; deepseek_v4 VT_DUMP_ACT precedent; env-doc-allowlist updated).

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
…e, landed green-shaped with the dispatch fix stacked below

Issue mudler#41 M4, second model. NEW tests/parity/test_qwen35_paged_engine.cpp --
the GDN sibling of the Qwen3-dense SACRED gate: the standard 16-prompt battery
through the full paged engine, anchor + <=0.5-nat near-tie band identical to
the dense gate, backend proof over the GDN op set. Goldens captured from the
pinned vLLM-ROCm oracle on gfx1100 (K=10 deterministic in every cell) with a
manifest.json (oracle identity + pinned HF revision 2fc06364 resolved via NEW
parity::Qwen35_08BSnapshot; check-snapshot-pins clean).

VERDICT: 16/16 PASS (15/16 strict token-exact vs the oracle, 1/16 near-tie
band, max gap 0.125 nats, 0 forward-divergent) with the stacked
AttnQkNormRopeGate output-dtype fix below. The PRE-FIX capture (13/16
forward-divergent, max gap 14.125 nats) is retained as evidence in the spec
and parity ledger, not as committed goldens no code can pass.

Post-review shape (localai-bot sweep): the manifest op joins
PendingRunnerOps() (the qwen36_gguf_greedy precedent -- a runner-less manifest
op turned main's CPU CI red); the gate exits 77 (CTest Skipped) on any
non-ROCm device and when the pinned checkpoint is absent, instead of comparing
a foreign engine against ROCm-derived goldens or printing SUCCESS with zero
assertions; and the anchor-drift REQUIRE now reads REGRESSION SUSPECTED --
re-deriving goldens is the last step of a justified re-capture, never the
response to a failure.

FOLLOWING_AGENTS_PROTOCOL

Following-Agents-Protocol: true
AI-Assisted: true
Assisted-by: pi:kimi-k3 [pi]
localai-bot added a commit that referenced this pull request Aug 14, 2026
… validates an outside contributor's trailers (#773) (#782)

Closes #773.

`check-pr-size.py` and `check-commit-trailers.py` both required the base revision
to be an ANCESTOR of head. CI passes `github.event.pull_request.base.sha`, the
TIP of the base branch, which stops being an ancestor the moment main advances
after the branch was cut -- continuously, on this repo.

Measured on three open PRs before changing anything. Base is not an ancestor in
any of them, and a merge base exists in all three:

    #506  ancestor=NO  merge-base=e1087a881
    #523  ancestor=NO  merge-base=fdd452637
    #559  ancestor=NO  merge-base=fafa16f0f

The consequence was not a noisy red check. Both checkers aborted BEFORE
examining anything, so CI had never validated commit trailers on an external
contribution: the gate enforcing FOLLOWING_AGENTS_PROTOCOL and Assisted-by:
exited before reading a single commit. Across the external PRs reviewed this
week, hand-checking by a reviewer was the only verification those trailers
received. pr-size aborted identically, so path classification and the
checker-evidence contract went unenforced on forks too.

THE FIX. Diff from the merge base, which is what a pull request IS: `git diff
A...B` is defined as `git diff $(git merge-base A B) B` and is what GitHub
shows. Two-dot diffing against a moved main is not merely stricter, it is WRONG
-- main's own commits render as reversions inside the contributor's diff, so
paths they never touched get classified and charged to them. The new pr-size
test asserts both halves: the PR's file present, main's absent.
`executable_evidence` gets the same treatment, since the BASE version of a
checker for the red-before half is the one at the merge base.

WHAT DELIBERATELY DOES NOT MOVE. The old rule conflated two situations:
ordinary divergence (merge base exists) now examines merge_base..head;
unrelated histories (no merge base) STILL RAISES. Absence of information must
never look like absence of work -- the script's own require_origin_main()
docstring already states that principle for the other input.

test_missing_and_nonancestor_objects_fail_closed uses an ORPHAN branch, so it
still raises; only its regex changed, because the message now names what is
actually wrong. Its assertRaises(ValueError) is untouched. The trailers case was
SPLIT, not deleted: its divergent-branch half built two branches off a common
root -- which share a merge base and are the ordinary shape of every PR -- so
that half now asserts it validates, with the genuinely-unrelated case asserted
separately. Nothing that used to fail closed stopped failing closed.

Range changed, contract unchanged:
test_a_bad_trailer_in_the_merge_base_range_is_still_reported puts a trailerless
commit inside the new range and requires it still be reported.

RED before on the unmodified checkers, GREEN after (74 passed, 148 subtests).
Stop conditions checked individually rather than inferred. Full tests/scripts:
9 failed / 1359 passed, all nine pre-existing and reproduced on main.

CI: agent-record and pr-size both SUCCESS on this PR -- the two checkers it
repairs passing on a live PR. Remaining red is baseline only: windows-msvc-*
are the PR-only arm (#584), and sanitize-cpu is red on main itself for #775
(test_nemotron_h_scaffold, nemotron_h_registry.cpp:112 downcasting a doctest
StubModel to NemotronHLoadedModel). This PR touches no C++.
localai-bot added a commit that referenced this pull request Aug 15, 2026
…d of aborting the pass (#776) (#853)

Closes #776.

`test_op_parity` THREW instead of failing an assertion:

    tests/parity/test_op_parity.cpp:1989: ERROR: test case THREW exception:
    [json.exception.type_error.302] type must be string, but is null

The malformed input was tests/parity/goldens/minimax_music3_oracle/manifest.json
-- an oracle capture with no "op" key at all, the only non-op manifest among the
66 committed. Attributed decisively rather than by inference: rebuilding the
pre-fix source and moving that one directory aside turned the same binary green,
70 assertions -> 123. The throw was hiding a third of the gate.

#776 IS A DUPLICATE, AND THAT IS NOT THE END OF IT. #755 is the same defect,
closed by 043e568 (#761) eight minutes after #776 was filed, and the manifest
is already repaired on main. Reporting only that would have been accurate and
useless, because #761 closed the walker's INPUT SET and not its EXCEPTION
SURFACE. Reproduced on the FIXED tree by nulling one tensor dtype in
rmsnorm_f32_8x128:

    :2177: ERROR: test case THREW exception: [...type_error.302...]
    [doctest] assertions: 37 | 37 passed

37 where 153 should run -- 45 committed goldens never reached, on a tree where
this was supposedly closed. The gate was one bad field away from silently not
gating.

WHY THAT MATTERS: test_op_parity is the guard that refuses a golden whose
manifest names an op with no registered runner. It is what caught PR #559's
missing runner arm. A throw at manifest-read time aborts the case, so every
golden after the malformed one goes unchecked -- not merely a red job, a gate
that had stopped gating.

THE FIX, both halves. The two remaining throw sites (json::parse on the
manifest, and any runner field read) funnel through GuardGoldenStage, which
converts a std::exception into a FAIL_CHECK NAMING goldens/<case>/manifest.json,
quotes the original exception, and continues to the next golden. The dispatch
chain moved into RunGoldenCase so its `continue`s became `return false`, which
is what permits a per-case catch.

std::exception is the deliberate width: doctest::detail::TestFailureException is
NOT derived from it (third_party/doctest/doctest.h:2563). That is load-bearing
-- the "no runner for op" refusal is a FAIL, and a wider catch would have
cleared the red while REMOVING the gate.

MUTATION EVIDENCE, tree restored byte-for-byte each time (823 files, md5sum -c):

  dtype: null in a real golden -> named by path; 45 op cases STILL RAN, 152
      assertions still checked (was: THREW, 37)
  invalid JSON manifest        -> named by path; 46 cases ran, no THREW
  {"op": null} / missing "op"  -> refused by name, pass continues
  {"op": "no_such_runner_op_559"} -> FATAL ERROR: no runner for op ...
      -- the #559 guard STILL BITES

The last row is the one that mattered; without it the others would only prove
the red was silenced.

GREEN: focused 13 cases / 153 assertions, with 46 op cases + 1 non-op skip --
counts IDENTICAL to before, so the extraction changed nothing about what is
gated. sanitize-cpu (address,undefined) rebuilt with the job's own flags and run
under its env: SUCCESS. pytest tests/scripts: 1400 passed, 9 failed, all three
distinct failures reproduced on a clean origin/main worktree.

Survey: ~167 unguarded m.at(...) reads live inside runners in this file and the
guard covers them generically. Other json::parse sites in tests/ each read one
named fixture owned by their own case, so a throw there fails one already-named
test rather than a walker over an open input set. No follow-up owed.

CI: merged with checks queued -- the runner pool has been saturated for hours
and no job started on this head. The sanitize lane, which is the one this
change exists for, was reproduced locally with that job's own build flags and
environment.

Note main moved mid-gate: #840 relocated the issue table out of roadmap_v1.md
into the append-only .agents/issue-index.md. roadmap_v1.md was taken from
origin/main wholesale and the #776 row reapplied as an append to the new index.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants