Tags: SKaiNET-developers/SKaiNET
Tags
SKaiNET 0.52.0 — the engine stops silently running on the scalar floor KernelDispatch now installs itself on first use through the ServiceLoader-backed ViewKernelPack SPI — providers, the row-major Q-series packs and the ternary/BitNet packs (BITNET_B1_58 LUT gemv, fused BITNET_PLANES lm_head) are all discovered, so an application that never calls an install routine no longer loses every kernel to the ~1000x decoding-reference fallback, and a fallback that does happen says so once, loudly. Dense FP32 weights are served from any storage kind, decode-shaped matmuls get a dedicated FP32 GEMV path, and androidNativeArm32/Arm64 build and publish across the whole downstream chain. Validated downstream: SKaiNET-transformers decodes BitNet-b1.58-2B-4T at full speed with zero bootstrap code, token-for-token equal to bitnet.cpp and the HF BF16 reference.
SKaiNET 0.51.0 — ternary/BitNet weights join the memory-mapped weight… … story Off-heap storage removes the Android ART heap-cap OOM risk for ternary weights, a SEQUENTIAL-layout (NeoGPU-converted) GGUF now gets a true zero-copy mmap load with no repack at all, and a new AOT converter (I2sAotConverter) lets a build that owns its model pipeline pay the repack cost once, offline, instead of on every load.
SKaiNET 0.50.0 — mapped packed staging, threaded kernels, a truthful … …memory plan Model size on Android is now a page-cache question, not a heap question — and decode is 2.5x faster. - Packed-tensor mapped staging (#1189/#1190, #1192): every GGML block format (Q4_K, Q6_K, Q5_K, Q8_0, Q4_0, Q5_0, Q5_1) is served straight from the memory-mapped GGUF file under WeightForm(residency = MAPPED) — zero copies, zero relayout. Row-major (_rm) C kernels read canonical file order, reached via JNI direct-buffer entries on Android and FFM MemorySegment.ofBuffer on the JVM (#1191). Storage-polymorphic dispatch also serves un-prepacked heap weights (#1193), closing a measured 48,771 ms/step silent-fallback trap on mixed-quant models (now 33 ms/step). - Threaded packed matmuls (#1195/#1196): spin-then-park worker pool with guided row-grains — 153 -> 61 ms/step on a 1.5B, bit-identical results. - The memory plan charges mapped weights against device RAM, not the heap budget (#1190): resolver, plan and loader share one servability predicate. - No silent fallbacks: every packed-bridge punt to the decoding reference is a named trace event; the M2-A5 harness reports the count. Measured on a Pixel 8a (256 MB ART cap): Qwen2.5-1.5B-Instruct Q4_K_M (1.0 GB) loads in ~0.4 s with 566 KB of weight heap and decodes at 61-66 ms/step with zero steady-state major page faults. Third-party: ships the vendored NeoGPU ternary NEON kernel unchanged (hs_ml_ternary_neon.c, (c) 2024 NeoGPU Contributors, MIT, byte-identical to anjaustin/neogpu @ 0846b24); attribution in REUSE metadata and META-INF/THIRD-PARTY-NOTICES.md in the published artifacts (#1166). Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
SKaiNET 0.40.1 — correctness hotfix Fixes a silent-wrong-output bug (issue #968, PR #969): DefaultCpuOps.transpose() performed a shape-only relabel instead of a real block-grid byte permutation when transposing packed quantized weight data. Affected every format with more than one quantization block per row (Q4_0, Q5_0, Q5_1, Q8_0, Q4_K, Q5_K, Q6_K) — virtually every real quantized model — across all three kernel tiers (scalar, Panama-vector, and native/FFM/JNI/Kotlin-Native). The bug did not crash; it silently returned incorrect matmul output whenever this dispatch path was exercised. Fixed with a real O(bytes) block-grid permutation. Misaligned/unsupported input now throws IllegalArgumentException instead of silently truncating. Verified with a ground-truth regression test: two independently-built, bytewise-different packings of the same logical matrix, checked against a from-scratch block-sequential dequant reference — proves the bug before the fix and correctness after it, across all 7 formats. A downstream audit (SKaiNET-transformers) confirmed no shipped model output was actually corrupted: gemma, llama, apertus, and qwen all sidestepped the bug via an independent eager block-relayout step at weight-load time that runs before transpose() is ever invoked. This release closes the underlying engine gap regardless, since any future quantized-format integration that doesn't happen to use that relayout pattern would otherwise hit it silently. Correctness-only patch release. No new features, no API changes. Recommended upgrade priority: high.
SKaiNET 0.40.0 Big models fit on real devices, and SKaiNET reaches iOS/macOS natively. Off-heap/mmap tensor storage on Android shares the JVM's memory-mapped weight loading with the Android compilation: dense F32 tensors serve as zero-heap mapped views, and weight bytes live in OS-paged file-backed pages instead of the managed heap — outside the hard ART heap cap that limited practical model size on Android (#921). GGUF DEQUANTIZE_TO_FP32 no longer over-allocates: loading a 1.1B Q4_K_M model transiently needed >12 GB heap against a ~4.4 GB dense-FP32 floor. Three compounding causes in the GGUF loader and K-quant kernels are fixed, bringing peak live allocation down to ~1.05x of the dense FP32 size (#782). Q5_0/Q5_1 packed matmul reaches the native tier for the first time: new NEON C kernels are wired into the FFM (JVM), Kotlin/Native, and Android JNI providers, unblocking the packed Q5_1 path for Q5_K_M checkpoints under NATIVE_OPTIMIZED (#708). skainet-backend-native-cpu now publishes iOS/macOS Kotlin/Native targets (iosArm64, iosSimulatorArm64, macosArm64) with the Mach-O kernel static archive embedded via cinterop. A single Apple arm64 archive dispatches FEAT_DotProd at runtime, so one build serves A12 through M-series (#958, Native·linux and Native·apple. See CHANGELOG.md for full details.
SKaiNET 0.39.1 Scoped patch release on top of 0.39.0, containing exactly two changes: - FP32 primitive fast paths for the eager CPU ops and a cached DirectCpuExecutionContext.ops handle (#950, closes #949) — cuts per-op dispatch overhead on the eager path. - README: pointer to SKaiNET-transformers for LLM users (#952, closes #923). Deliberately excluded: the Q5_0/Q5_1 packed matmul kernels (#951) — they ship with the next minor release. The kernel-support matrix in this tag reflects the pre-#951 tiers. Docs (antora skainet_version, BOM snippets, kernel matrix) are aligned to 0.39.1 in this tree.
Release 0.39.0 On-device AI on Android becomes real. A JNI NEON kernel backend (skainet-backend-jni-cpu) brings the hand-tuned ARM matmul kernels to Android, where the FFM provider can never run: two .so tiers built from the same sources (baseline armv8-a and armv8.2-a+dotprod) are selected at load time from /proc/cpuinfo, so one artifact is safe from Cortex-A53 up. Measured ~24 tok/s SmolLM2-135M Q8_0 decode on a Pixel 8a versus ~3.8 scalar (6.4x), clearing the on-device usability bar. The provider auto-registers via ServiceLoader. Android GGUF loading no longer OOMs: createRandomAccessSource streams via positional FileChannel reads across io-gguf/-safetensors/-onnx instead of a full-file heap read. Published Kotlin/Native kernel klibs now embed their static archive, so downstream K/N consumers link with no setup, and a NEON body was added for the Q4_0 matmul kernel. Tensor-storage correctness pass: fail-fast on unsupported GGUF quant types instead of silently dropping weights, truthful ownership labels and real byte counts in the storage layer, a materializable FileBacked/Aliased transfer path, and a rank-safe default copyToFloatArray. See CHANGELOG.md for details.
PreviousNext