Skip to content

W2a: #49-P1 substrate — generateUntilStop in llm-core, demo/CLI out of kllama, shared ModelMetadata extraction (#37) - #296

Merged
michalharakal merged 6 commits into
developfrom
feat/tool-calling-w2a
Aug 11, 2026
Merged

michalharakal merged 6 commits into
developfrom
feat/tool-calling-w2a

Conversation

@michalharakal

Copy link
Copy Markdown
Contributor

Summary

W2a — first of three stacked PRs executing the tool-calling architecture epic (#35) on the #49 Phase-1 substrate. Scope reconciliation: #35 (comment)

Closes the remaining #49 Phase-1 items and the extraction half of #37:

1. generateUntilStop promoted to llm-core (#49 P1, item 1)

  • generateUntilStop, GenerateResult, and the tensor sampleFromLogits overload now live in sk.ainet.apps.llm (llm-core), next to InferenceRuntime, PrefillStrategy, and the FloatArray sampler they build on.
  • llm-agent keeps typealias/delegating re-exports in sk.ainet.apps.kllama.agent — the same backward-compat pattern used when InferenceRuntime itself was promoted. No caller changes anywhere (AgentLoop, KLlamaSession, prefill-equivalence tests all compile untouched).

2. Demo/agent CLI extracted from kllama (#49 P1, item 3)

  • ToolCallingDemo, AgentCli, ListFilesTool, CalculatorTool move from :llm-runtime:kllama jvmMain to :llm-agent jvmMain, packages preserved (sk.ainet.apps.kllama.cli).
  • :llm-runtime:kgemma and :llm-apps:skainet-cli were importing these from kllama — exactly the cross-runner coupling Unified Model Pipeline with Decoupled Tool Calling #49 Phase 1 calls out. Both already depend on llm-agent directly, so they need zero source changes (verified by compiling both).

3. Shared ModelMetadata extraction (#37)

  • New ModelMetadataExtraction in llm-agent commonMain: pure, format-agnostic, best-effort — takes a GGUF key/value field map (or, for the helpers, a chat template + vocab) and produces ModelMetadata with family/architecture/chatTemplate/tokenizerHints/sourceFormat. Malformed/missing input degrades to partial metadata, never throws.
  • kllama's private peekGgufMetadata now delegates to it — semantics preserved (same family mapping), plus tokenizerHints population that didn't exist before.
  • 11 new unit tests: full/missing/empty/wrong-typed fields, family mapping, hint scanning, and end-to-end "extracted metadata resolves the right provider".

#36 is intentionally not re-implemented here: ToolCallingSupport/ToolCallingMode/ToolCallingSupportResolver already shipped and have been exercised from outside llm-agent twice (SmolLM2 #277, FunctionGemma #292) — see the reconciliation comment.

Verification

  • :llm-agent:jvmTest 189, :llm-core:jvmTest 107, :llm-runtime:kllama:jvmTest 42 — all green
  • :llm-core:apiCheck :llm-agent:apiCheck :llm-runtime:kllama:apiCheck green; apiDump refreshed (classes moved kllama→llm-agent, additions in llm-core)
  • :llm-runtime:kgemma:compileKotlinJvm and :llm-apps:skainet-cli:compileKotlin green (downstream importers of the moved classes)

Stacked: W2b (#38/#39/#40) and W2c (#41/#42/#43/#44-docs) follow on top of this branch.

Refs #49 #37 #36 #35

🤖 Generated with Claude Code

michalharakal and others added 2 commits August 11, 2026 17:31
…ore, extract demo/agent CLI from kllama, shared ModelMetadata extraction (#37)

Three moves that finish decoupling tool calling from the kllama runner
(#49 Phase 1) and make capability metadata shared and testable (#37):

- generateUntilStop + GenerateResult + tensor sampleFromLogits move to
  llm-core's sk.ainet.apps.llm, next to InferenceRuntime/PrefillStrategy.
  Any runner now gets EOS-aware generation without depending on the agent
  layer. llm-agent keeps typealias/delegating re-exports (same pattern as
  the earlier InferenceRuntime promotion), so no caller changes.
- ToolCallingDemo, AgentCli, ListFilesTool, CalculatorTool move from
  :llm-runtime:kllama jvmMain to :llm-agent jvmMain, packages preserved.
  kgemma and skainet-cli previously imported these *from kllama* — the
  exact cross-runner coupling #49 Phase 1 calls out; both already depend
  on llm-agent directly and need no source changes.
- New ModelMetadataExtraction in llm-agent: pure, format-agnostic
  best-effort extraction (GGUF field map -> family/architecture/
  chat_template/tokenizerHints). kllama's private peekGgufMetadata now
  delegates to it; unit tests cover full/missing/malformed fields and
  that extracted metadata drives resolver auto-detection.

apiDump refreshed for llm-core, llm-agent, kllama; apiCheck green.
Part of the #35 epic; W2a of three stacked PRs (W2a/W2b/W2c).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rser strategies

Completes the two genuinely-missing pieces of the detection/parsing layer
(the resolver-side auto-detection with explicit-override-wins already
shipped; see the #35 reconciliation):

- #38: ModelMetadataExtraction.fromHuggingFaceConfig() parses
  tokenizer_config.json (chat_template as string or HF's named-template
  list, additional_special_tokens for hints), chat_template.json, and
  config.json (model_type -> family). kllama Main.kt now peeks these
  sidecar files for safetensors checkpoints and feeds the result into
  the same resolver path GGUF already used — auto-detection now works
  on the HF path too, and --template still wins. Malformed/missing JSON
  degrades to partial metadata, never throws.
- #40: ToolCallParser.registerStrategy()/unregisterStrategy()/
  registeredFormats() — family-specific parser strategies can be
  registered into the default chain at runtime, prepended so they
  outrank the built-ins, replace-by-formatName semantics mirroring
  ToolCallingSupportResolver.register(). Built-ins are not removable.
- #39 needed no code: GenericToolCallingSupport + resolveOrFallback +
  GENERIC-mode logging already exist.

17 new unit tests (HF extraction incl. resolver end-to-end for qwen2/
llama3 safetensors sidecars; parser registration ordering/replacement/
unregistration). apiDump refreshed; apiCheck green.

Part of #35; W2b, stacked on #296.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…tests, Qwen real-GGUF validation, native-vs-generic docs

Final PR of the W2 stack (#35 epic):

- #41: AgentCli (chat/agent modes) now prints the same provider/mode/
  reason resolution diagnostics ToolCallingDemo already printed, so
  native-vs-generic selection is visible in every CLI mode.
- #42: ResolutionDiagnosticsTest pins resolveWithDiagnostics — explicit
  override reason, auto-detect reason, GENERIC fallback mode/reason, and
  extraction→diagnostics carry-through. (GGUF/HF extraction tests landed
  in the earlier PRs of this stack.)
- #43: QwenToolCallSmokeTest, env-gated on QWEN_MODEL_PATH. Two stages:
  (1) metadata extraction + auto-resolution against the real file —
  header-only, runs in seconds; verified PASSING against
  Qwen2.5-0.5B-Instruct-F16.gguf (family=qwen, arch=qwen2, hints
  [<tool_call>, <tool_response>, <|im_start|>] → provider qwen, NATIVE,
  no explicit family). (2) full agent-loop round-trip over the real
  checkpoint via the same DSL path the CLI uses — currently fails for a
  reason outside this architecture: the DSL Qwen runtime emits
  degenerate text on real checkpoints (tracked as #118); documented in
  the test so it becomes the round-trip pin once #118 lands.
- #44: README gains a "Tool calling: native vs. generic support"
  section with resolution-order docs and a family/provider/mode/format/
  detection compatibility matrix; CHANGELOG entry for the W2 stack.

Verification: llm-agent jvmTest 210 green; kllama jvmTest green
(gated tests skip cleanly without env vars); apiCheck green on
llm-core/llm-agent/kllama (no public API changes in this PR);
gemma-iree jvmTest green (functiongemma provider registration).

Note: :llm-apps:kllama-cli / :llm-apps:skainet-cli fail dependency
resolution on develop (empty engine version on app classpaths),
pre-existing and unrelated — demo validation therefore ran through the
env-gated smoke tests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
W2c: AgentCli diagnostics (#41), detection tests (#42), Qwen real-GGUF validation (#43), native-vs-generic docs (#44)
W2b: HF-side chat-template auto-detection (#38) + registerable parser strategies (#40)
@michalharakal
michalharakal merged commit f132dbb into develop Aug 11, 2026
1 check passed
@michalharakal
michalharakal deleted the feat/tool-calling-w2a branch August 11, 2026 20:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant