feat(#1195): spin-then-park thread pool for the packed matmul kernels — 153 → 61 ms/step - #1196

…n the Pixel 8a

Row-partition threading for both orders of both kernels (feed-order and the
#1189 row-major variants), through a shared runner (skainet_row_threads).
The design is the product of a measured elimination on device, each variant
a cooled Qwen2.5-1.5B Q4_K_M decode step:

  single-threaded (#1190 baseline)            153 ms
  pthread_create/join per call                994 ms  (cpuidle wakeup × ~600/step)
  parked pool, static quarter chunks          115 ms  (big.LITTLE straggle)
  parked pool, 64-row stealing                227 ms  (short scattered windows)
  parked pool, guided grains                  306 ms  (still scheduler-starved)
  spin-then-park pool + guided grains          61 ms  ← shipped

The decisive piece is the spin: sub-millisecond parallel bursts separated by
sleeps never build per-thread utilization, so EAS/schedutil parks workers on
little cores at low clocks — the pool was slower than one pegged big core.
Workers spin (`yield`) on the job epoch for ~1 ms before parking on the
condvar: utilization stays pegged during decode, the scheduler answers with
big cores and full clocks, and everyone parks when work stops (no idle burn).
Same reason llama.cpp's pool spins.

Structure: kernels refactored around shared per-block terms + row-range
workers; entries quantize the activation to Q8 once (read-only across
threads) and hand the rows to skainet_run_rows. Guided grains off an atomic
cursor — long contiguous streams first, small tail last — degrade to an even
split on symmetric cores; nothing is tuned to one SoC. Threshold 512 keeps
GQA k/v projections (256 rows) single-threaded. MSVC (no pthreads) and any
pthread_create failure degrade to the caller's thread.

Bit-identity: workers own disjoint out[] ranges and per-row accumulation
order never changes. RowMajorMatmulParityTest grows threaded cases with two
oracles: threaded-vs-feed-order on permuted bytes, and threaded-vs-1536
independent single-row calls (pins the partition arithmetic itself).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>

…t fallback trap, measured

`-e residency heap` restores heap staging (default mapped) so staging
strategies can be A/B'd on models that fit the cap. First use found a trap
instead of an answer: SmolLM2-135M's hidden size (576) is not a 256-multiple,
so llama.cpp quantized most of its matrices as Q8_0 (k-quant fallback) — and
heap Q8_0 in canonical order without prepack has no BLOCKED_ROW_MAJOR kernel,
so dispatch silently served the decoding reference:

  heap + prepack=true              66 ms/step
  mapped + prepack=false       48,771 ms/step  (Q8_0 -> reference, ~800x)
  mapped + prepack=true            65 ms/step  (mapped rm + prepacked Q8_0)

Exactly #1193's "silent fallback" case and #1192's Q8_0 priority, now with
numbers. The Qwen runs never hit it (all dims 256-multiples, pure Q4_K/Q6_K).

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>