W2a: #49-P1 substrate — generateUntilStop in llm-core, demo/CLI out of kllama, shared ModelMetadata extraction (#37) - #296
Merged
Conversation
…ore, extract demo/agent CLI from kllama, shared ModelMetadata extraction (#37) Three moves that finish decoupling tool calling from the kllama runner (#49 Phase 1) and make capability metadata shared and testable (#37): - generateUntilStop + GenerateResult + tensor sampleFromLogits move to llm-core's sk.ainet.apps.llm, next to InferenceRuntime/PrefillStrategy. Any runner now gets EOS-aware generation without depending on the agent layer. llm-agent keeps typealias/delegating re-exports (same pattern as the earlier InferenceRuntime promotion), so no caller changes. - ToolCallingDemo, AgentCli, ListFilesTool, CalculatorTool move from :llm-runtime:kllama jvmMain to :llm-agent jvmMain, packages preserved. kgemma and skainet-cli previously imported these *from kllama* — the exact cross-runner coupling #49 Phase 1 calls out; both already depend on llm-agent directly and need no source changes. - New ModelMetadataExtraction in llm-agent: pure, format-agnostic best-effort extraction (GGUF field map -> family/architecture/ chat_template/tokenizerHints). kllama's private peekGgufMetadata now delegates to it; unit tests cover full/missing/malformed fields and that extracted metadata drives resolver auto-detection. apiDump refreshed for llm-core, llm-agent, kllama; apiCheck green. Part of the #35 epic; W2a of three stacked PRs (W2a/W2b/W2c). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rser strategies Completes the two genuinely-missing pieces of the detection/parsing layer (the resolver-side auto-detection with explicit-override-wins already shipped; see the #35 reconciliation): - #38: ModelMetadataExtraction.fromHuggingFaceConfig() parses tokenizer_config.json (chat_template as string or HF's named-template list, additional_special_tokens for hints), chat_template.json, and config.json (model_type -> family). kllama Main.kt now peeks these sidecar files for safetensors checkpoints and feeds the result into the same resolver path GGUF already used — auto-detection now works on the HF path too, and --template still wins. Malformed/missing JSON degrades to partial metadata, never throws. - #40: ToolCallParser.registerStrategy()/unregisterStrategy()/ registeredFormats() — family-specific parser strategies can be registered into the default chain at runtime, prepended so they outrank the built-ins, replace-by-formatName semantics mirroring ToolCallingSupportResolver.register(). Built-ins are not removable. - #39 needed no code: GenericToolCallingSupport + resolveOrFallback + GENERIC-mode logging already exist. 17 new unit tests (HF extraction incl. resolver end-to-end for qwen2/ llama3 safetensors sidecars; parser registration ordering/replacement/ unregistration). apiDump refreshed; apiCheck green. Part of #35; W2b, stacked on #296. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tests, Qwen real-GGUF validation, native-vs-generic docs Final PR of the W2 stack (#35 epic): - #41: AgentCli (chat/agent modes) now prints the same provider/mode/ reason resolution diagnostics ToolCallingDemo already printed, so native-vs-generic selection is visible in every CLI mode. - #42: ResolutionDiagnosticsTest pins resolveWithDiagnostics — explicit override reason, auto-detect reason, GENERIC fallback mode/reason, and extraction→diagnostics carry-through. (GGUF/HF extraction tests landed in the earlier PRs of this stack.) - #43: QwenToolCallSmokeTest, env-gated on QWEN_MODEL_PATH. Two stages: (1) metadata extraction + auto-resolution against the real file — header-only, runs in seconds; verified PASSING against Qwen2.5-0.5B-Instruct-F16.gguf (family=qwen, arch=qwen2, hints [<tool_call>, <tool_response>, <|im_start|>] → provider qwen, NATIVE, no explicit family). (2) full agent-loop round-trip over the real checkpoint via the same DSL path the CLI uses — currently fails for a reason outside this architecture: the DSL Qwen runtime emits degenerate text on real checkpoints (tracked as #118); documented in the test so it becomes the round-trip pin once #118 lands. - #44: README gains a "Tool calling: native vs. generic support" section with resolution-order docs and a family/provider/mode/format/ detection compatibility matrix; CHANGELOG entry for the W2 stack. Verification: llm-agent jvmTest 210 green; kllama jvmTest green (gated tests skip cleanly without env vars); apiCheck green on llm-core/llm-agent/kllama (no public API changes in this PR); gemma-iree jvmTest green (functiongemma provider registration). Note: :llm-apps:kllama-cli / :llm-apps:skainet-cli fail dependency resolution on develop (empty engine version on app classpaths), pre-existing and unrelated — demo validation therefore ran through the env-gated smoke tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 11, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
W2a — first of three stacked PRs executing the tool-calling architecture epic (#35) on the #49 Phase-1 substrate. Scope reconciliation: #35 (comment)
Closes the remaining #49 Phase-1 items and the extraction half of #37:
1.
generateUntilStoppromoted tollm-core(#49 P1, item 1)generateUntilStop,GenerateResult, and the tensorsampleFromLogitsoverload now live insk.ainet.apps.llm(llm-core), next toInferenceRuntime,PrefillStrategy, and the FloatArray sampler they build on.llm-agentkeeps typealias/delegating re-exports insk.ainet.apps.kllama.agent— the same backward-compat pattern used whenInferenceRuntimeitself was promoted. No caller changes anywhere (AgentLoop,KLlamaSession, prefill-equivalence tests all compile untouched).2. Demo/agent CLI extracted from kllama (#49 P1, item 3)
ToolCallingDemo,AgentCli,ListFilesTool,CalculatorToolmove from:llm-runtime:kllamajvmMain to:llm-agentjvmMain, packages preserved (sk.ainet.apps.kllama.cli).:llm-runtime:kgemmaand:llm-apps:skainet-cliwere importing these from kllama — exactly the cross-runner coupling Unified Model Pipeline with Decoupled Tool Calling #49 Phase 1 calls out. Both already depend onllm-agentdirectly, so they need zero source changes (verified by compiling both).3. Shared
ModelMetadataextraction (#37)ModelMetadataExtractionin llm-agent commonMain: pure, format-agnostic, best-effort — takes a GGUF key/value field map (or, for the helpers, a chat template + vocab) and producesModelMetadatawithfamily/architecture/chatTemplate/tokenizerHints/sourceFormat. Malformed/missing input degrades to partial metadata, never throws.peekGgufMetadatanow delegates to it — semantics preserved (same family mapping), plustokenizerHintspopulation that didn't exist before.#36 is intentionally not re-implemented here:
ToolCallingSupport/ToolCallingMode/ToolCallingSupportResolveralready shipped and have been exercised from outside llm-agent twice (SmolLM2 #277, FunctionGemma #292) — see the reconciliation comment.Verification
:llm-agent:jvmTest189,:llm-core:jvmTest107,:llm-runtime:kllama:jvmTest42 — all green:llm-core:apiCheck :llm-agent:apiCheck :llm-runtime:kllama:apiCheckgreen; apiDump refreshed (classes moved kllama→llm-agent, additions in llm-core):llm-runtime:kgemma:compileKotlinJvmand:llm-apps:skainet-cli:compileKotlingreen (downstream importers of the moved classes)Stacked: W2b (#38/#39/#40) and W2c (#41/#42/#43/#44-docs) follow on top of this branch.
Refs #49 #37 #36 #35
🤖 Generated with Claude Code