feat(android): M2-A5 on-device measurement harness (#1130) - #1188
Merged
Merged
Conversation
An instrumentation harness — deliberately a measurement, not an acceptance test, per #1130: it never calls PlanVsActual.check(), and it assumption-skips when no model file is present so connectedAndroidTest stays green on model-less runners. What one run records, on real ART hardware: - the MemoryPlans plan for the GGUF (header-only) against Runtime.maxMemory() as the budget — the real heap cap; - load under WeightResidency.MAPPED, wall time, RSS before/after and major faults during load (MemoryProbe); - a decode loop in the DecodeHarness shape over the REAL loaded weights: per step, every layer's attention + FFN projections as packed matmuls through KernelDispatch (JNI kernel pack installed when available) out of a recycled ForwardScope, one token per step into a model-scoped KV ring; RSS and major-fault deltas sampled per step; - ActualMemory replayed from the trace, PlanVsActual rendered into the report — with the loaded weights accounted as model-scope allocation events, since the loader's storages are not all sink-wired; - the report written to the app's external-files dir (m2a5-report.md), streamed to logcat under the M2A5 tag, and printed to stdout. The class KDoc carries the full adb recipe (push the model into the test app's external-files dir — /data/local/tmp is not readable by an app process under SELinux; run with -Pandroid.testInstrumentationRunnerArguments.class=...M2A5DeviceMeasurement; pull the report and paste it into the issue). Refs #1130 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ce numbers Three fixes from the first real runs on a Pixel 8a (Android 17): - weights load OUT_IN — the dispatcher wants [out, in], the file stores ne order; - KernelPacks.installPacked() alongside install() — install() alone bridges only FP32, and the decoding reference at per-element view get() is hours-scale for even a 135M model; - an opt-in prepack=true argument permutes each used weight once into the BLOCKED_INPUT_MAJOR order the packed JNI kernels read; the copies land in the trace as prepack adapters, so their memory cost is itself part of the measurement (70 MB for SmolLM2-135M's used layers). First recorded numbers (Pixel 8a, ART cap 256 MB), for #1130: - SmolLM2-135M Q4_K_M (100 MB): plan 185/256 MB "fits" — and it does: load 375 ms / 272 tensors, 2 major faults during load, decode 63 ms/step steady with the JNI packed kernels, ZERO major faults across steady-state decode; plan-vs-actual recorded with explained drifts. - Qwen2.5-1.5B Q4_K_M (1.06 GB): OutOfMemoryError at load — a 131 MB tensor allocation against the growth limit. Under MAPPED residency the quantized payloads still materialize as heap ByteArrays; the gap M2-A5 exists to expose, now measured. Refs #1130 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The measurement harness #1130 asked for — an instrumentation test that is deliberately a measurement, not an assertion (never calls
PlanVsActual.check(); assumption-skips without a model file soconnectedAndroidTeststays green on model-less runners).One run records, on real ART hardware: the
MemoryPlansplan againstRuntime.maxMemory()as budget; load underWeightResidency.MAPPEDwith RSS/major-faults before/after; a DecodeHarness-shaped loop over the real loaded weights throughKernelDispatch(JNI packed kernels viainstallPacked, optional one-time feed-orderprepackwhose memory cost lands in the trace as adapters); per-step RSS + major-fault deltas;PlanVsActualrendered into a report (m2a5-report.mdin the app's external-files dir, logcatM2A5, stdout). Full adb recipe in the class KDoc.First numbers are on the issue — recorded on a Pixel 8a: the 100 MB Q4_K_M fits-and-decodes case (2 load faults, 0 steady-state faults, 63 ms/step) and the 1.06 GB exceeds-cap case (OOM at load: quantized payloads still materialize on-heap under MAPPED residency — the measured gap).
🤖 Generated with Claude Code