FunctionGemmaToolCallingSupport: compact functional tokens on the llm-agent provider architecture (#35, #36) - #292
Merged
Conversation
…l tokens on the llm-agent provider architecture (#35, #36) The compact <tool_N>(k="v")<end> format had no bridge into llm-agent's ToolCallingSupport / ToolCallParserStrategy / ChatTemplate stack. Add one, living in gemma-iree because the dependency direction demands it: llm-agent publishes js/wasm/android/ios targets gemma-iree does not have, so the provider is registered at runtime via ToolCallingSupportResolver.register() (prepended, so it outranks the generic Gemma provider that would otherwise claim FunctionGemma's plain gemma3 arch). - CompactToolCodec: CompactCodec's TOKEN_TO_NAME is now a constructor param (stock v10 six-tool map as default) — a 7th tool needs no library edit. The CompactCodec object stays, delegating, for source compat. - FunctionGemmaToolCallParserStrategy: gemma-iree ToolCall -> llm-agent ToolCall (flat string args as JsonPrimitives), formatName "functiongemma". - FunctionGemmaChatTemplate: reproduces byte-for-byte the prompt FunctionGemma.call() used to hardcode (no newline before <end_of_turn>; tools deliberately ignored — the vocabulary is baked into the checkpoint). - FunctionGemma.call() now applies the template; FunctionGemmaEagerTest verified green against the real Q5_K_M checkpoint (set_lights/state=on). - commonTest mirrors CompactCodecTest through the strategy + custom-map and exact-prompt tests. Prior art: feature/smoll's SmolLM provider (#272). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…real-model tests - llm-inference/gemma/README.md: stale 0.35.0 coordinates -> 0.39.0. - New sk.ainet.models.gemma.FunctionGemmaFixture (env-first GEMMA_GGUF with the documented dev-box path as fallback), mirroring kgemma's fixture; the seven real-GGUF tests now share it instead of seven hardcoded absolute paths. RealGemmaLoadTest verified green through the fixture. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 11, 2026
This was referenced Aug 11, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
First concrete slice of #35/#36: the compact
<tool_N>(k="v")<end>functional-token format (FunctionGemma v10, parsed byCompactCodec) is now wired into llm-agent'sToolCallingSupport/ToolCallParserStrategy/ChatTemplatearchitecture, following the SmolLM provider prior art from #272.Placement: gemma-iree, not llm-agent
Gemma4ToolCallingSupportand friends live in:llm-agent, but llm-agent publishes js/wasm/android/ios targets that:llm-runtime:gemma-ireedoes not have — so llm-agent cannot depend on gemma-iree. The provider therefore lives in gemma-iree commonMain (which gains anapi(":llm-agent")dependency) and is opted into at runtime:registerprepends, so the provider outranks the genericGemmaToolCallingSupportthat would otherwise claim FunctionGemma's plaingemma3arch and hand out the JSONfunctionCalltemplate the fine-tune was never trained on.Changes
CompactToolCodec—CompactCodec'sTOKEN_TO_NAMEis now a constructor parameter (stock v10 six-tool vocabulary as default): a 7th tool needs no library edit, e.g.CompactToolCodec(DEFAULT_TOKEN_TO_NAME + ("6" to "open_gripper")). TheCompactCodecobject stays, delegating, so existing call sites are source-compatible.FunctionGemmaToolCallParserStrategy— bridges gemma-ireeToolCall(tool, args)to llm-agentToolCall(id, name, JsonObject);formatName = "functiongemma";<tool_none>stays an explicit no-op.FunctionGemmaChatTemplate— reproduces byte-for-byte the promptFunctionGemma.call()used to hardcode (<start_of_turn>user\n…<end_of_turn>\n<start_of_turn>model\n; note: no newline before<end_of_turn>, unlikeGemmaChatTemplate).toolsare deliberately ignored — the vocabulary is baked into the checkpoint as special tokens.FunctionGemmaToolCallingSupport—family = "functiongemma", detection via family or<tool_*>tokenizer hints (arch is NOT a discriminator),ToolCallingMode.NATIVE.FunctionGemma.call()refactored onto the template (behavior unchanged).llm-inference/gemma/README.mdcoordinates 0.35.0 → 0.39.0; new env-firstFunctionGemmaFixturein llm-inference/gemma replaces seven hardcoded absolute GGUF paths (GEMMA_GGUFoverride, documented dev-box fallback).Tests
CompactCodecTestthrough the strategy (jvm + linuxX64 both green), plus custom-tool-map, unique-call-id, exact-prompt, multi-turn, andsupports()discrimination tests.CompactCodecTestuntouched — proves object-API compat.FunctionGemmaEagerTestagainst the real Q5_K_M checkpoint: passed, not skipped (tests=1 skipped=0 failures=0), emitting<tool_0>(state="on")<end>→set_lights/state=onthrough the new template.RealGemmaLoadTest(-PincludeIntegration) green through the new fixture../gradlew apiCheckgreen (no public-API dump changes in validated modules; kgemma's surface unchanged).🤖 Generated with Claude Code