feat(kllama,kgemma): ship the Android NEON backend with the runtime facades (#285) - #286
Merged
Merged
Conversation
…acades (#285) Add sk.ainet.core:skainet-backend-jni-cpu (engine 0.39.0) as a runtimeOnly dependency of the androidMain source sets. The backend self-registers via ServiceLoader on ART, so presence on the runtime classpath is sufficient — Android consumers get NEON kernels (Q8_0/Q4_0/Q4_K/Q5_K/Q6_K, runtime dotprod dispatch) instead of the scalar Kotlin fallback. Version comes from the engine BOM re-exported by :llm-bom. Document the behavior in the README target-matrix section and CHANGELOG.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #285.
Engine 0.39.0 ships
sk.ainet.core:skainet-backend-jni-cpu— an AAR with NEON kernels for Q8_0 / Q4_0 / Q4_K / Q5_K / Q6_K and runtime dotprod dispatch, ServiceLoader-registered on ART. This PR makes the two Android runtime facades carry it, so an Android app that depends onruntime-kllamaorruntime-kgemmagets native kernels out of the box instead of silently falling back to the scalar Kotlin matmul (measured on a Pixel 8a: ~24 tok/s vs ~3.8 tok/s decode-kernel throughput on SmolLM2-135M Q8_0, engine PR SKaiNET#945).Changes:
gradle/libs.versions.toml: catalog entryskainet-backend-jniCpu(version-less; the engine BOM re-exported by:llm-bomcarries the 0.39.0 constraint — verified in the publishedskainet-bom-0.39.0.pom).llm-runtime/kllama+llm-runtime/kgemma:runtimeOnly(libs.skainet.backend.jniCpu)inandroidMain.runtimeOnlybecause the backend has no compile-time API surface — discovery is ServiceLoader-only.llm-inference/llama+transformer-coredirectly should add the AAR themselves; excluding the artifact opts back into pure Kotlin.[Unreleased].Verified locally:
:llm-runtime:kllama:dependencies --configuration androidRuntimeClasspathand the kgemma equivalent both resolveskainet-backend-jni-cpu -> 0.39.0(BOM-constrained).:llm-runtime:kllama:assembleAndroidMainand:llm-runtime:kgemma:assembleAndroidMainbuild clean.Cost note for reviewers: consuming apps gain four small
.sofiles (arm64-v8a + armeabi-v7a × baseline/dotprod tiers). If you'd rather keep the facades dependency-free and make this opt-in-by-docs instead, happy to flip the PR to a README-only change.