FunctionGemma as a standalone DSL->StableHLO->IREE module - #302
Merged
Merged
Conversation
…EE module New :llm-inference:functiongemma module (whisper/moonshine "one self-contained module" shape): FunctionGemmaSpec + FunctionGemmaContract (manifest.json emission, pure Kotlin) in commonMain, FunctionGemmaExportHarness + the exportFunctionGemma CLI/gradle task in jvmMain. The three-graph export logic (gemma redecode, gemma_prefill, gemma_with_past) is moved VERBATIM from kgemma's FunctionGemmaExport — golden-equivalence verified: all three MLIRs and all three safetensors archives are byte-identical (sha256) to the pre-move kgemma export on the same FunctionGemma-270M checkpoint. :llm-runtime:kgemma's FunctionGemmaExport/FunctionGemmaExportMain become @deprecated delegating shims to the new module (deprecate-don't-delete); the pre-existing FunctionGemmaExportTest, FunctionGemmaInt8QuantTest and FunctionGemmaWithPastMlirDumpTest integration tests run unmodified against the shims and stay green (apiCheck also green — no public API change). :llm-runtime:gemma-iree gains manifest-driven support (D3): GemmaManifest (plain-JSON parse of manifest.json — gemma-iree intentionally does NOT depend on functiongemma, to keep its lean target set), GemmaKvDecoder.fromManifest sourcing architecture constants/arg orders from the contract instead of hardcoding them (existing hardcoded defaults preserved for direct-constructor callers), and CompactToolCodec.fromManifest sourcing the tool vocabulary from the manifest's toolMap. Docker (skainet/iree-compiler:3.11.0) vmfb parity: gemma_prefill's compiled first token matches the board-verified oracle. Full multi-step greedy parity via x64 host-CPU run-module hit a pre-existing, non-regression divergence at the first generated special-vocabulary token — reproduces identically in bf16 and FP32, and independent of the KV-cache loop (a one-shot full-sequence forward reproduces it), so it is not something this move introduced; see FunctionGemmaVmfbParityTest's class doc for the full diagnostic trail. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This was referenced Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
:llm-inference:functiongemmamodule (the whisper/moonshine "one self-contained module" shape):FunctionGemmaSpec+FunctionGemmaContract(manifest.jsonemission) incommonMain,FunctionGemmaExportHarness+ theexportFunctionGemmaCLI/gradle task injvmMain. Depends on:llm-inference:gemmafor the architecture; owns the function-calling export/contract.gemmaredecode,gemma_prefill,gemma_with_past) is moved verbatim from:llm-runtime:kgemma'sFunctionGemmaExport— golden-equivalence verified: all three exported MLIRs and all three safetensors archives are byte-identical (sha256) to the pre-move kgemma export on the same FunctionGemma-270M checkpoint.:llm-runtime:kgemma'sFunctionGemmaExport/FunctionGemmaExportMainbecome@Deprecateddelegating shims (deprecate-don't-delete); the pre-existingFunctionGemmaExportTest,FunctionGemmaInt8QuantTest,FunctionGemmaWithPastMlirDumpTestintegration tests run unmodified against the shims and stay green.apiCheckis green (no public API change).:llm-runtime:gemma-ireegains manifest-driven support (D3):GemmaManifest(plain-JSON parse — intentionally no dependency onfunctiongemma, to keep gemma-iree's lean target set),GemmaKvDecoder.fromManifest(...)sourcing architecture constants/arg orders from the contract (existing hardcoded defaults preserved for direct-constructor callers), andCompactToolCodec.fromManifest(...)sourcing the tool vocabulary from the manifest'stoolMap.skainet/iree-compiler:3.11.0) vmfb parity:gemma_prefill's compiled first token matches the board-verified oracle. Full multi-step greedy parity via x64 host-CPUrun-modulehit a pre-existing, non-regression divergence at the first generated special-vocabulary token — reproduces identically in bf16 and FP32, and independent of the KV-cache loop (a one-shot full-sequence forward reproduces it too), so it predates this move. Full diagnostic trail is inFunctionGemmaVmfbParityTest's class doc; flagged as a follow-up, not blocking.Test plan
FunctionGemmaContractTest(pure, no checkpoint) — manifest arg/result order pinningFunctionGemmaExportDumpTest(real checkpoint) — MLIR signatures match the contract, manifest.json round-tripsFunctionGemmaVmfbParityTest(real checkpoint + docker) — compile/convert-parameters/run-module succeed for all 3 graphs; prefill first-token oracle matchGemmaManifestTest— manifest parsing +CompactToolCodec.fromManifestFunctionGemmaExportTest,FunctionGemmaInt8QuantTest,FunctionGemmaWithPastMlirDumpTestpass unmodified against the new shims:llm-runtime:kgemma:apiCheckgreengemma-ireefulljvmTestsuite green (CompactCodec/ToolCallingSupport/ParserStrategy tests unaffected)🤖 Generated with Claude Code