You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
A model-footprint analysis CLI: given a model file, answer "will this fit on the target device?" in seconds — before anyone invests a day in converting it.
The concrete pain: there is no quick way to check whether a model fits on an embedded device with limited memory and compute (usable RAM ≈ 2.1 GB) before committing to a full conversion run. The tool should:
accept GGUF, safetensors and ONNX files and analyze them from headers/metadata only (no payload load — a multi-GB file is answered in seconds);
report the actual resident size of the weights — as stored, and (where the format carries encodings) as they would be held in memory after resolution, since a Q4 weight that has to dequantize costs ~8× its file size;
print a verdict against the device budget (fits / does not fit, with margin and actionable suggestions such as KV-cache quantization or a smaller context).
Header-only planning for GGUF — StreamingGGUFReader.planInput() builds a PlanInput from the tensor table + architecture metadata without reading payloads; MemoryPlans.plan() prices weights, KV cache and the forward slab against a Budget and renders a fit table with suggestions.
Per-device policy — PlannerProfile (e.g. MOBILE_2GB: reserve, mapped weights, auto KV quantization, strict mode) and DeviceFit/DeviceMemory for checking both RAM pools.
Resolved cost, not file cost — WeightFormResolver (what the file holds × profile × kernel capabilities) and AllocationResolver.explain() (P7: AllocationResolver — one resolver decides domain and scope SKaiNET#1143): the plan prices what will actually be held, and can say per tensor where it lands and why, pre-load.
A working CLI skeleton — skainet-apps/skainet-plan already does all of the above for GGUF and prints the plan.
Tracked as a coding issue in core: SKaiNET-developers/SKaiNET#1169 (safetensors + ONNX front ends, the ~2.1 GB budget flag and verdict ergonomics).
Research questions
ONNX weight enumeration without materialization — initializers live inside the protobuf graph; what is the cheapest reliable way to enumerate names/dims/dtypes (and external-data references) without loading tensor bytes, and how do quantized ONNX conventions (QDQ, MatMulInteger) map onto a resident-bytes estimate?
Runtime overhead model per format/runtime — file weights are a lower bound; what per-runtime headroom model (allocator overhead, graph buffers, KV cache for decoder models without GGUF-style metadata) makes the 2.1 GB verdict trustworthy rather than optimistic? Can we calibrate it against measured RSS on the device (plan-vs-actual, as SKEEP-003 M1 did for GGUF decode)?
Verdict quality — false "fits" is the expensive failure. What safety margin and phrasing keep the tool honest when geometry metadata is missing (safetensors/ONNX), where only weights + headroom can be priced?
Idea
A model-footprint analysis CLI: given a model file, answer "will this fit on the target device?" in seconds — before anyone invests a day in converting it.
The concrete pain: there is no quick way to check whether a model fits on an embedded device with limited memory and compute (usable RAM ≈ 2.1 GB) before committing to a full conversion run. The tool should:
What already exists in SKaiNET core
Most of the machinery landed with the SKEEP-003 memory-architecture arc (SKaiNET-developers/SKaiNET#932):
StreamingGGUFReader.planInput()builds aPlanInputfrom the tensor table + architecture metadata without reading payloads;MemoryPlans.plan()prices weights, KV cache and the forward slab against aBudgetand renders a fit table with suggestions.PlannerProfile(e.g.MOBILE_2GB: reserve, mapped weights, auto KV quantization, strict mode) andDeviceFit/DeviceMemoryfor checking both RAM pools.WeightFormResolver(what the file holds × profile × kernel capabilities) andAllocationResolver.explain()(P7: AllocationResolver — one resolver decides domain and scope SKaiNET#1143): the plan prices what will actually be held, and can say per tensor where it lands and why, pre-load.skainet-apps/skainet-planalready does all of the above for GGUF and prints the plan.Tracked as a coding issue in core: SKaiNET-developers/SKaiNET#1169 (safetensors + ONNX front ends, the ~2.1 GB budget flag and verdict ergonomics).
Research questions