W2b: HF-side chat-template auto-detection (#38) + registerable parser strategies (#40) - #297
Merged
Merged
Conversation
…rser strategies Completes the two genuinely-missing pieces of the detection/parsing layer (the resolver-side auto-detection with explicit-override-wins already shipped; see the #35 reconciliation): - #38: ModelMetadataExtraction.fromHuggingFaceConfig() parses tokenizer_config.json (chat_template as string or HF's named-template list, additional_special_tokens for hints), chat_template.json, and config.json (model_type -> family). kllama Main.kt now peeks these sidecar files for safetensors checkpoints and feeds the result into the same resolver path GGUF already used — auto-detection now works on the HF path too, and --template still wins. Malformed/missing JSON degrades to partial metadata, never throws. - #40: ToolCallParser.registerStrategy()/unregisterStrategy()/ registeredFormats() — family-specific parser strategies can be registered into the default chain at runtime, prepended so they outrank the built-ins, replace-by-formatName semantics mirroring ToolCallingSupportResolver.register(). Built-ins are not removable. - #39 needed no code: GenericToolCallingSupport + resolveOrFallback + GENERIC-mode logging already exist. 17 new unit tests (HF extraction incl. resolver end-to-end for qwen2/ llama3 safetensors sidecars; parser registration ordering/replacement/ unregistration). apiDump refreshed; apiCheck green. Part of #35; W2b, stacked on #296. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tests, Qwen real-GGUF validation, native-vs-generic docs Final PR of the W2 stack (#35 epic): - #41: AgentCli (chat/agent modes) now prints the same provider/mode/ reason resolution diagnostics ToolCallingDemo already printed, so native-vs-generic selection is visible in every CLI mode. - #42: ResolutionDiagnosticsTest pins resolveWithDiagnostics — explicit override reason, auto-detect reason, GENERIC fallback mode/reason, and extraction→diagnostics carry-through. (GGUF/HF extraction tests landed in the earlier PRs of this stack.) - #43: QwenToolCallSmokeTest, env-gated on QWEN_MODEL_PATH. Two stages: (1) metadata extraction + auto-resolution against the real file — header-only, runs in seconds; verified PASSING against Qwen2.5-0.5B-Instruct-F16.gguf (family=qwen, arch=qwen2, hints [<tool_call>, <tool_response>, <|im_start|>] → provider qwen, NATIVE, no explicit family). (2) full agent-loop round-trip over the real checkpoint via the same DSL path the CLI uses — currently fails for a reason outside this architecture: the DSL Qwen runtime emits degenerate text on real checkpoints (tracked as #118); documented in the test so it becomes the round-trip pin once #118 lands. - #44: README gains a "Tool calling: native vs. generic support" section with resolution-order docs and a family/provider/mode/format/ detection compatibility matrix; CHANGELOG entry for the W2 stack. Verification: llm-agent jvmTest 210 green; kllama jvmTest green (gated tests skip cleanly without env vars); apiCheck green on llm-core/llm-agent/kllama (no public API changes in this PR); gemma-iree jvmTest green (functiongemma provider registration). Note: :llm-apps:kllama-cli / :llm-apps:skainet-cli fail dependency resolution on develop (empty engine version on app classpaths), pre-existing and unrelated — demo validation therefore ran through the env-gated smoke tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 11, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
W2b — second of three stacked PRs for the tool-calling epic (#35). Stacked on #296 (W2a) — review only the last commit here until #296 merges.
#38 — best-effort auto-detection from HF-side configs
The resolver-side auto-detection (metadata → provider, explicit override wins) already shipped; what was missing was the tokenizer/config-side inspection called out in the issue. Now:
ModelMetadataExtraction.fromHuggingFaceConfig(tokenizerConfigJson, chatTemplateJson, modelConfigJson)parses:tokenizer_config.json—chat_templateas a plain string or HF's list-of-named-templates form ("default"entry wins, else first), plusadditional_special_tokens(strings or{"content": ...}objects) for tool-hint scanningchat_template.json— used when tokenizer_config has no templateconfig.json—model_type→ architecture → familyMain.ktnow peeks these sidecar files for safetensors checkpoints (peekHfMetadata) and feeds the result into the exact resolver path GGUF metadata already used.--template=...still wins (resolver contract, unchanged).#40 — registerable parser strategies
ToolCallParserStrategy+parseWithexisted; clean registration into the default chain did not. Now:ToolCallParser.registerStrategy(strategy)— prepended (outranks built-ins), replace-by-formatName, mirroringToolCallingSupportResolver.register()semantics exactlyToolCallParser.unregisterStrategy(formatName)— removes runtime registrations only; built-ins (hermes, llama3-function-tag, llama3-json) are permanentToolCallParser.registeredFormats()— chain snapshot in resolution order#39 — no code needed
GenericToolCallingSupport+resolveOrFallback+ GENERIC-mode diagnostics already exist (32b3b9e); verified, will be close-commented on the issue.Verification
:llm-agent:jvmTest: 206 tests (17 new), 0 failures — HF extraction (string/list templates, precedence, malformed JSON, end-to-end resolver checks for qwen2/llama3 sidecars) + parser registration (ordering, replacement, unregistration, built-in immutability):llm-agent:apiCheckgreen, apiDump refreshed:llm-runtime:kllama:compileKotlinJvmgreenRefs #38 #39 #40 #35
🤖 Generated with Claude Code