Skip to content

FunctionGemma as a standalone DSL->StableHLO->IREE module - #302

Merged
michalharakal merged 1 commit into
developfrom
feat/functiongemma-module
Aug 11, 2026
Merged

michalharakal merged 1 commit into
developfrom
feat/functiongemma-module

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

  • New :llm-inference:functiongemma module (the whisper/moonshine "one self-contained module" shape): FunctionGemmaSpec + FunctionGemmaContract (manifest.json emission) in commonMain, FunctionGemmaExportHarness + the exportFunctionGemma CLI/gradle task in jvmMain. Depends on :llm-inference:gemma for the architecture; owns the function-calling export/contract.
  • The three-graph export (gemma redecode, gemma_prefill, gemma_with_past) is moved verbatim from :llm-runtime:kgemma's FunctionGemmaExportgolden-equivalence verified: all three exported MLIRs and all three safetensors archives are byte-identical (sha256) to the pre-move kgemma export on the same FunctionGemma-270M checkpoint.
  • :llm-runtime:kgemma's FunctionGemmaExport/FunctionGemmaExportMain become @Deprecated delegating shims (deprecate-don't-delete); the pre-existing FunctionGemmaExportTest, FunctionGemmaInt8QuantTest, FunctionGemmaWithPastMlirDumpTest integration tests run unmodified against the shims and stay green. apiCheck is green (no public API change).
  • :llm-runtime:gemma-iree gains manifest-driven support (D3): GemmaManifest (plain-JSON parse — intentionally no dependency on functiongemma, to keep gemma-iree's lean target set), GemmaKvDecoder.fromManifest(...) sourcing architecture constants/arg orders from the contract (existing hardcoded defaults preserved for direct-constructor callers), and CompactToolCodec.fromManifest(...) sourcing the tool vocabulary from the manifest's toolMap.
  • Docker (skainet/iree-compiler:3.11.0) vmfb parity: gemma_prefill's compiled first token matches the board-verified oracle. Full multi-step greedy parity via x64 host-CPU run-module hit a pre-existing, non-regression divergence at the first generated special-vocabulary token — reproduces identically in bf16 and FP32, and independent of the KV-cache loop (a one-shot full-sequence forward reproduces it too), so it predates this move. Full diagnostic trail is in FunctionGemmaVmfbParityTest's class doc; flagged as a follow-up, not blocking.

Test plan

  • Golden-equivalence: new module's export vs. pre-move kgemma export, byte-identical (sha256) MLIR + safetensors x3 graphs
  • FunctionGemmaContractTest (pure, no checkpoint) — manifest arg/result order pinning
  • FunctionGemmaExportDumpTest (real checkpoint) — MLIR signatures match the contract, manifest.json round-trips
  • FunctionGemmaVmfbParityTest (real checkpoint + docker) — compile/convert-parameters/run-module succeed for all 3 graphs; prefill first-token oracle match
  • GemmaManifestTest — manifest parsing + CompactToolCodec.fromManifest
  • kgemma's pre-existing FunctionGemmaExportTest, FunctionGemmaInt8QuantTest, FunctionGemmaWithPastMlirDumpTest pass unmodified against the new shims
  • :llm-runtime:kgemma:apiCheck green
  • gemma-iree full jvmTest suite green (CompactCodec/ToolCallingSupport/ParserStrategy tests unaffected)

🤖 Generated with Claude Code

…EE module

New :llm-inference:functiongemma module (whisper/moonshine "one self-contained
module" shape): FunctionGemmaSpec + FunctionGemmaContract (manifest.json
emission, pure Kotlin) in commonMain, FunctionGemmaExportHarness + the
exportFunctionGemma CLI/gradle task in jvmMain. The three-graph export logic
(gemma redecode, gemma_prefill, gemma_with_past) is moved VERBATIM from
kgemma's FunctionGemmaExport — golden-equivalence verified: all three MLIRs
and all three safetensors archives are byte-identical (sha256) to the
pre-move kgemma export on the same FunctionGemma-270M checkpoint.

:llm-runtime:kgemma's FunctionGemmaExport/FunctionGemmaExportMain become
@deprecated delegating shims to the new module (deprecate-don't-delete); the
pre-existing FunctionGemmaExportTest, FunctionGemmaInt8QuantTest and
FunctionGemmaWithPastMlirDumpTest integration tests run unmodified against
the shims and stay green (apiCheck also green — no public API change).

:llm-runtime:gemma-iree gains manifest-driven support (D3): GemmaManifest
(plain-JSON parse of manifest.json — gemma-iree intentionally does NOT
depend on functiongemma, to keep its lean target set), GemmaKvDecoder.fromManifest
sourcing architecture constants/arg orders from the contract instead of
hardcoding them (existing hardcoded defaults preserved for direct-constructor
callers), and CompactToolCodec.fromManifest sourcing the tool vocabulary from
the manifest's toolMap.

Docker (skainet/iree-compiler:3.11.0) vmfb parity: gemma_prefill's compiled
first token matches the board-verified oracle. Full multi-step greedy parity
via x64 host-CPU run-module hit a pre-existing, non-regression divergence at
the first generated special-vocabulary token — reproduces identically in
bf16 and FP32, and independent of the KV-cache loop (a one-shot full-sequence
forward reproduces it), so it is not something this move introduced; see
FunctionGemmaVmfbParityTest's class doc for the full diagnostic trail.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@michalharakal
michalharakal merged commit a071fb9 into develop Aug 11, 2026
2 checks passed
@michalharakal
michalharakal deleted the feat/functiongemma-module branch August 11, 2026 20:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant