Tags: SKaiNET-developers/SKaiNET-transformers
Tags
0.55.0 -- transformers-only, still against engine 0.54.0 Adds asr-domain module + BackendProvider capabilities/options (#432): generic ASR task types (Transcription, DecodingOptions, FeatureFrames) moved up from the downstream ASR cartridge ecosystem, plus a defaulted BackendCapabilities/BackendOptions extension to BackendProvider. Builds on 0.54.1 (parallel attention heads, copy-free K/V).
SKaiNET-transformers 0.53.0 — lock-step with SKaiNET 0.53.0 Version lock-step with the engine is restored: this release ships against SKaiNET 0.53.0 (#397), which brings the billion-parameter export fixes (SKaiNET#1247) and the sharded SafeTensors ParametersLoader (SKaiNET#1246) this repository's Gemma 3n export and family loaders were waiting on. Everything accumulated under Unreleased since 0.40.2 ships here too: BitNet b1.58, the engine-loader migration (every GGUF weight loader a thin engine wrapper, MAPPED residency by default), the Gemma 3n DSL path with its StableHLO/IREE export harness, and the Qwen / Apertus decode and tool-calling fixes. Highlights - The Gemma 3n export actually emits (#396): the harness streams weights through the BufferResolver, so the >=2 GiB tied embedding (BufferHandle.Floats from the engine) reaches disk. Full 30-layer E2B: 15k-line StableHLO module, zero failure comments, 4.6 GB safetensors, under a minute — previously an OOM at a 46 GB heap. - SafeTensors loading rides the engine (#398 gemma, #400 shared decoder loader, #401 apertus + gemma3n): per-family hand-rolled materialization collapses onto ShardedSafeTensorsParametersLoader / SafeTensorsParametersLoader; each family keeps only its slot table and allowlist; Require(BF16)/Require(FP16) keep-native accepted on the SafeTensors lane; synthetic 2-shard fixture tests per loader. Voxtral and llm-core's legacy Q4 path wait on SKaiNET#1256. - skainet-decode on Android (#395, SKaiNET#1244): llm-apps:skainet-decode-core (shared DecodeSession with MemoryProbe sampling inside every decode span) and llm-apps:skainet-decode-android, the repository's first Android application — mapped GGUF load, fit check before allocating, one-thread decode, GenerationMetrics with page-fault and RSS rows. - Tool calling re-verified against the new engine: Gemma4E2BToolCallSmokeTest re-enabled (#399) — the real E2B checkpoint emits parseable <|tool_call> markup; FunctionGemmaOfficialGgufTest and QwenToolCallSmokeTest (Qwen3-1.7B) re-run green. - CI: smoke-reference.yml duplicate-key fix (#403) — no more phantom red "workflow file issue" runs on every push. Engine: sk.ainet:skainet-bom:0.53.0 Changelog: CHANGELOG.md, section [0.53.0] — 2026-09-02
SKaiNET-transformers 0.40.2 — republish of 0.40.1 (broken publish fix)
Ships against SKaiNET engine 0.40.1 (same engine pin as 0.40.1 -- this
is not an engine bump, see below).
Headline
--------
0.40.1's tag-triggered Maven Central publish (./gradlew publish) broke
partway through the module graph: :llm-inference:smollm2 declared
linuxX64()/linuxArm64() Kotlin/Native targets with no source to back
them (jvmMain-only export tooling, no commonMain at all), so
compileKotlinLinuxArm64 reported NO-SOURCE and produced no klib, but
the maven-publish plugin still registered a publication for that
target and generateMetadataFileForLinuxArm64Publication threw
FileNotFoundException trying to hash the nonexistent file, aborting
the build. By that point llm-api, llm-agent, llm-core,
transformer-core, llm-bom, llm-performance, llm-providers,
llm-inference:{apertus,bert,functiongemma,gemma,llama,moonshine,qwen},
and smollm2's own JVM publication had already published;
llm-inference:{t5,vec2text,voxtral,whisper} and all of llm-runtime:*
never got attempted.
0.40.2 is a straight republish of everything 0.40.1 was meant to be --
same content, same engine pin -- with the two unused target
declarations dropped from smollm2. Every other multiplatform module
was audited for the same declared-target-vs-actual-source mismatch;
none had it. 0.40.1 is superseded: use 0.40.2.
For the actual feature content of this release (tool-calling epic
substrate, packed-quant shared packer, FunctionGemma/SmolLM2 compiled
export modules, generic Android JNI compiled runtime, the packed-quant
regression fix), see the 0.40.1 tag/CHANGELOG entry -- unchanged here.
Fixed
-----
- Broken 0.40.1 release-workflow publish (#313): dropped smollm2's
unused linuxX64()/linuxArm64() target declarations, which had no
source and broke Maven Central metadata generation.
SKaiNET-transformers 0.40.1 — the compiled edge path lands Ships against SKaiNET engine 0.40.1, in lock-step: a transformers X.Y.Z release runs against engine X.Y.Z. Headline -------- FunctionGemma and SmolLM2 each get a standalone DSL -> StableHLO -> IREE export module, and a new generic Android JNI runtime (llm-runtime/iree-android) serves any compiled model the way skainet-backend-jni-cpu already serves the eager path. The tool-calling epic's shared substrate (#35) lands in three stacked PRs. Every packed-quant format gets a single hoisted packer with a pre-transposed-by-default fast path now that engine 0.40.0 shipped native Q5 kernels -- and 0.40.1 fixes a real matmul-corruption regression that the 0.40.0 -> 0.40.1 engine pin exposed on the classic (non-pre-transposed) packed path. Added ----- - Tool-calling epic substrate (#35), three stacked PRs: generateUntilStop promoted to llm-core, demo/agent CLI extracted out of kllama into llm-agent (#296, closes #37/#49-P1); HF-side chat-template auto-detection from tokenizer_config.json/chat_template.json/config.json plus registerable parser strategies (#297, closes #38/#40); AgentCli resolution diagnostics, detection/diagnostics tests, and validation against a real Qwen instruct GGUF (#299, closes #41/#42/#43/#44). - Packed Q5_0/Q5_1 converter path + shared block packer (#294, closes #170, implements #184 items 2-3): GemmaMemSegConverter keeps Q5_1/Q5_0 weights packed instead of FP32 dequant fallback, gated on hasPackedMatmulKernel() rather than engine version. The GGUF-block -> engine-tensor packing gemma/llama/apertus each carried privately is hoisted into sk.ainet.lang.nn.quant.BlockQuantPacking (transformer-core); PreTransposedWeight marks weights already in kernel-feed layout so linearProject can skip ops.transpose entirely. - FunctionGemma extracted into a standalone module (#302): :llm-inference:functiongemma owns the function-calling export/contract, moved verbatim (byte-identical, sha256-verified) from :llm-runtime:kgemma, which keeps deprecated delegating shims. :llm-runtime:gemma-iree gains manifest-driven support (GemmaManifest, GemmaKvDecoder.fromManifest, CompactToolCodec.fromManifest). - SmolLM2 compiled-export path (#305 epic): host-side StableHLO export for SmolLM2-135M-Instruct, the first export for the llama architecture (#306); a standalone :llm-inference:smollm2 module with the redecode-graph + DSL argMax tail, numerically verified end-to-end via iree-compile/ iree-run-module (#308). - Generic Android JNI runtime for the compiled path (#309): :llm-runtime:iree-android, the compiled-path counterpart to the engine's skainet-backend-jni-cpu -- binds external weights from a .irpa, invokes a named compiled function, model-agnostic. Both arm64-v8a/armeabi-v7a ABIs, both CPU and Vulkan HAL drivers built in. - Android Antora docs (#310): getting-started for the eager path and an eager-vs-compiled explanation page. Changed ------- - SKaiNET engine 0.39.1 -> 0.40.0 -> 0.40.1 (#307, #311). 0.40.0 brings native Q5_0/Q5_1 kernels, so the gemma/llama packed-weight converters flip to packPreTransposed by default wherever a packed kernel is confirmed available -- verified byte/token-identical greedy decode across the FP32 baseline, the JVM MemSeg path, and the Kotlin/Native board path on a real FunctionGemma-270M checkpoint. Fixed ----- - kllama native kernels now actually register on Linux Kotlin/Native (#301, closes #300): linuxX64/linuxArm64 publish the cinterop-embedded native kernels, but nothing ever called the engine's installNativeKernels() on those targets -- measured 0.6 -> 2.06 tok/s (3.4x) on SmolLM2-135M Q8_0 once fixed. - Packed-quant classic-path matmul corruption under engine 0.40.1 (#311): engine 0.40.1's ops.transpose became a physical canonical->kernel-native block-grid permutation (closing engine #968), but BlockQuantPacking.pack()'s classic path was already eagerly relayouting bytes to kernel-native order at load time -- so the weight got double-permuted at forward time, silently producing wrong matmul output on Q4_K/Q5_0/Q5_1 (not a crash). pack() now stores checkpoint bytes verbatim; the Apertus Q4_K/Q6_K converter (its own inlined, un-migrated relayout) is switched onto the shared packPreTransposed path. New parity-matrix, synthetic-Apertus, and llama quant-layout tests close the coverage gap that let this ship undetected. The pre-transposed production path -- what gemma/llama/kgemma actually serve -- was never affected. Root cause tracked upstream at SKaiNET-developers/SKaiNET#973: packed-quant byte order is an unwritten, contradictory contract across the engine and its converters.
SKaiNET-transformers 0.39.1 — the gemma function-calling day Ships against SKaiNET engine 0.39.1, in lock-step: a transformers X.Y.Z release runs against engine X.Y.Z. Headline -------- The compiled FunctionGemma path gets materially smaller, faster, and board-verified: the compact functional-token tool-calling format joins the ToolCallingSupport architecture, the tied embedding is exported once (weight archive ~832 -> ~512 MiB bf16), true-dynamic with_past export is the default, and GemmaKvDecoder is board-verified (SL2610 steady-state ~1740 ms/token, 2.1x the same-day re-decode baseline). Underneath, engine 0.39.1 removes the per-element eager-op overhead that dominated on-device decode on every non-JVM target. Added ----- - FunctionGemmaToolCallingSupport: the compact functional-token format (<tool_N>(...), CompactCodec) as a ToolCallingSupport implementation — parser strategy, byte-exact chat template, NATIVE-mode detection; the tool map is injectable (CompactToolCodec), so consumers can extend the tool set without a library change. First concrete slice of the #35 generalization. (#292) - Board-verified KV decode: GemmaKvDecoder with K-first outputs, raw .bin I/O, per-graph parameter archives (gemma-prefill.irpa / gemma-with-past.irpa), runbook in llm-runtime/gemma-iree/docs/. SL2610: steady-state ~1740 ms/token, 2.1x the same-day re-decode baseline. (#291) - On-device Android E2E SmolLM2 generation spike for the runtime facades. (#288, refs #272) Changed ------- - Engine pin skainet 0.39.0 -> 0.39.1: picks up the engine's primitive FP32 fast paths for the eager CPU ops and the cached DirectCpuExecutionContext.ops (engine #949) — the per-element overhead that dominated on-device decode (83% of SmolLM2-135M end-to-end on a Pixel 8a even with NEON matmul) is gone for every non-JVM target. - Tied embedding exported once: Gemma4WeightLoader aliases token_embd into output.weight; the FunctionGemma weight archive drops ~832 -> ~512 MiB bf16 with a single 262153x640 global. (#290, closes #260) - True-dynamic with_past export is the default: engine Dim.DYNAMIC tracing replaces the SENTINEL_PAST=7919 text rewrite (GEMMA_SENTINEL_PAST=1 is the rollback); the g165 Torq-fork compiler accepts the dynamic-dim MLIR on-board. (#290, refs #248) - Token embedding stays packed on the JVM eager path: the transformers-local RowDequantSource became a deprecated typealias to the engine type (fixing a latent gather mismatch), and GemmaMemSegConverter keeps row-sliceable token_embd layouts packed — ~0.49 GB less FP32 on Q8_0, decode byte-identical. (#289, closes #178/#234) Fixed ----- - Shared-KV cache variants trace correctly: SharedPositionalKVCache / PaddedSharedPositionalKVCache / OwnerReadOnlyKVCache no longer bake K=V=0 constants under embedConstants tracing (kvSharedLayers > 0, e.g. Gemma 4 E2B). (#290, closes #194)
SKaiNET-transformers 0.39.0 — Android NEON out of the box Ships against SKaiNET engine 0.39.0, in lock-step: a transformers X.Y.Z release runs against engine X.Y.Z. Headline -------- Android apps using the runtime facades (runtime-kllama, runtime-kgemma) now decode with native NEON kernels out of the box. The Android artifacts carry the engine's new sk.ainet.core:skainet-backend-jni-cpu AAR as a runtimeOnly dependency; it self-registers via ServiceLoader on ART and provides NEON kernels with runtime dotprod dispatch for Q8_0 / Q4_0 / Q4_K / Q5_K / Q6_K. Measured on a Pixel 8a (SmolLM2-135M Q8_0 decode-kernel throughput): ~24 tok/s vs ~3.8 tok/s scalar (~6.4x). Together with engine 0.39.0's Android streaming-GGUF fix (createRandomAccessSource is real on Android — the hard-OOM full-file load path is gone), this closes the mobile field-report arc tracked in engine issue #920. Added ----- - whisper-tiny authored end-to-end in the NN DSL (skainet-transformers-inference-whisper): encoder, fixed-masked-KV decoder, weights streamed from the HF safetensors checkpoint, MLIR + IRPA export harness for IREE. Verified against the ONNX-pipeline golden and on-device (Pixel Tensor G3, Vulkan). (#279) - SmolLM2 tool-calling support: SmolLMChatTemplate, parser strategy, and resolver registration in the agent loop. (#277, #272) - Cross-target SmolLM2-135M inference spike in kllama commonTest — one env-gated test giving directly comparable load/tok-s numbers on JVM, Linux native, and the iOS simulator. (#278, #272) - iOS artifacts for the runtime facades: kllama and kgemma publish iosArm64 + iosSimulatorArm64 klibs. (#274, #271) - Supported-targets matrix in the README. (#271) - Android NEON backend wiring as described above. (#286, #285) Changed ------- - SKaiNET engine 0.38.0 -> 0.39.0: Android JNI NEON backend, Android random-access GGUF loading, fail-fast on unsupported GGUF tensor types plus packed Q4_0/Q5_0/Q5_1 load branches, NEON Q4_0 kernel and cinterop-embedded kernel archives for Kotlin/Native, tensor-storage API hygiene. (#282) Fixed ----- - Gemma integration tests skip properly under JUnit 5 (Jupiter Assumptions; stray JUnit 4 dependency removed). (#280, #261) - Refreshed stale jvm binary-compatibility dumps for llm-agent, kllama, transformer-core — apiCheck green on clean checkouts. (#276, #275) Full details: CHANGELOG.md, section [0.39.0].
SKaiNET-transformers 0.38.0 Ships against SKaiNET engine 0.38.0 (first-class dynamic tensor shapes, Dim). Moonshine v2 streaming ASR authored end-to-end in the SKaiNET NN DSL (last vendor-ONNX graph gone), narrow-float KEEP_NATIVE fp16/bf16 weights, true-dynamic KV-cache decode graphs + fixed-max-pad cross-attention mask, Gemma token_embd row-dequant. See CHANGELOG.md [0.38.0].
PreviousNext