Context
feature/bert-dsl-embeddings already carries a substantial embedding stack that
develop doesn't have:
BertEmbeddingModel one-call factory (fromSafeTensors + fromHuggingFace)
in llm-providers, with tests
- the DSL BERT in
llm-inference/bert (legacy eager BERT stack removed on the
branch)
MiniLmExportHarness (jvmTest) — fixed-seq-bucket StableHLO export for the
on-device embedder cartridge, plus eager reference probes
- a
kbert-cli app in llm-apps
So "do we have a usable embedding model?" is really "how good is the one we
already have, and what does it take to land it?"
Ask — an evaluation pass with recorded numbers
- Correctness / parity: cosine similarity of branch embeddings vs the
reference implementation (sentence-transformers all-MiniLM-L6-v2) over a
fixed sentence set — eager path and StableHLO-exported path both, per
fixed-seq bucket.
- Retrieval quality: a small STS/retrieval benchmark (e.g. STS-B subset)
scored against the reference model's numbers, so regressions are visible in
review rather than discovered by users.
- Performance: embeddings/sec + resident memory on JVM, Android and iOS —
embedding models are the most credible near-term on-device story (they're
small and latency-tolerant), and the same mobile constraints we documented
for generative models (no native kernels, ART heap cap) apply here and
should be quantified.
- Merge path: what remains between the branch and develop (the branch also
removes the legacy eager BERT stack — that's a breaking change worth its own
review), and whether the export harness belongs in commonTest rather than
jvmTest so the parity probes run cross-target.
We can run the on-device measurements (physical Android + iPhone) and contribute
the parity/benchmark test harness.
Context
feature/bert-dsl-embeddingsalready carries a substantial embedding stack thatdevelop doesn't have:
BertEmbeddingModelone-call factory (fromSafeTensors+fromHuggingFace)in
llm-providers, with testsllm-inference/bert(legacy eager BERT stack removed on thebranch)
MiniLmExportHarness(jvmTest) — fixed-seq-bucket StableHLO export for theon-device embedder cartridge, plus eager reference probes
kbert-cliapp inllm-appsSo "do we have a usable embedding model?" is really "how good is the one we
already have, and what does it take to land it?"
Ask — an evaluation pass with recorded numbers
reference implementation (sentence-transformers
all-MiniLM-L6-v2) over afixed sentence set — eager path and StableHLO-exported path both, per
fixed-seq bucket.
scored against the reference model's numbers, so regressions are visible in
review rather than discovered by users.
embedding models are the most credible near-term on-device story (they're
small and latency-tolerant), and the same mobile constraints we documented
for generative models (no native kernels, ART heap cap) apply here and
should be quantified.
removes the legacy eager BERT stack — that's a breaking change worth its own
review), and whether the export harness belongs in commonTest rather than
jvmTest so the parity probes run cross-target.
We can run the on-device measurements (physical Android + iPhone) and contribute
the parity/benchmark test harness.