W2c: AgentCli diagnostics (#41), detection tests (#42), Qwen real-GGUF validation (#43), native-vs-generic docs (#44) - #299
Merged
Conversation
…tests, Qwen real-GGUF validation, native-vs-generic docs Final PR of the W2 stack (#35 epic): - #41: AgentCli (chat/agent modes) now prints the same provider/mode/ reason resolution diagnostics ToolCallingDemo already printed, so native-vs-generic selection is visible in every CLI mode. - #42: ResolutionDiagnosticsTest pins resolveWithDiagnostics — explicit override reason, auto-detect reason, GENERIC fallback mode/reason, and extraction→diagnostics carry-through. (GGUF/HF extraction tests landed in the earlier PRs of this stack.) - #43: QwenToolCallSmokeTest, env-gated on QWEN_MODEL_PATH. Two stages: (1) metadata extraction + auto-resolution against the real file — header-only, runs in seconds; verified PASSING against Qwen2.5-0.5B-Instruct-F16.gguf (family=qwen, arch=qwen2, hints [<tool_call>, <tool_response>, <|im_start|>] → provider qwen, NATIVE, no explicit family). (2) full agent-loop round-trip over the real checkpoint via the same DSL path the CLI uses — currently fails for a reason outside this architecture: the DSL Qwen runtime emits degenerate text on real checkpoints (tracked as #118); documented in the test so it becomes the round-trip pin once #118 lands. - #44: README gains a "Tool calling: native vs. generic support" section with resolution-order docs and a family/provider/mode/format/ detection compatibility matrix; CHANGELOG entry for the W2 stack. Verification: llm-agent jvmTest 210 green; kllama jvmTest green (gated tests skip cleanly without env vars); apiCheck green on llm-core/llm-agent/kllama (no public API changes in this PR); gemma-iree jvmTest green (functiongemma provider registration). Note: :llm-apps:kllama-cli / :llm-apps:skainet-cli fail dependency resolution on develop (empty engine version on app classpaths), pre-existing and unrelated — demo validation therefore ran through the env-gated smoke tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Aug 11, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
W2c — final PR of the tool-calling stack (#35 epic). Stacked on #297 (W2b) — review only the last commit until #296/#297 merge.
#41 — AgentCli resolution diagnostics
ToolCallingDemoalready printedProvider: <family> (mode=…, reason: …);AgentCli(chat + agent modes) now prints the same line, so native-vs-generic selection is visible in every CLI mode. Explicit--templateselection and auto-detection both flow through the identical resolver path (unchanged).#42 — detection/diagnostics tests
ResolutionDiagnosticsTestpinsresolveWithDiagnostics: explicit-override reason, auto-detected reason, GENERIC fallback mode + reason, and GGUF-extraction→diagnostics carry-through. Together with the stack this gives: resolver 21 (pre-existing) + GGUF extraction 11 (#296) + HF-side detection & parser registration 17 (#297) + diagnostics 4 (here).#43 — Qwen validation against a real instruct GGUF
QwenToolCallSmokeTest(kllama jvmTest), env-gated onQWEN_MODEL_PATH, two stages:Qwen2.5-0.5B-Instruct-F16.gguf—family=qwen, arch=qwen2, hints=[<tool_call>, <tool_response>, <|im_start|>]→ providerqwen,mode=NATIVE, reasonauto-detected from metadata(no explicit family anywhere). This is the architecture-validation core of Validate the generalized design against existing Qwen support #43.DecoderGgufWeightLoader→QwenNetworkLoader→OptimizedLLMRuntimeDIRECT path the CLI uses): resolution, ChatSession wiring, Qwen template rendering, and the agent loop all execute — but the run surfaced that the DSL Qwen runtime currently emits degenerate text on real checkpoints (garbage multilingual tokens at temp 0; 57-minute CPU run captured in the test report). That is an inference-quality problem squarely in Phase 4 readiness: validate DSL Qwen output against HF reference on real GGUFs #118's scope (validate DSL Qwen output vs HF reference), not a tool-calling-architecture problem — the test documents this and becomes the round-trip pin once Phase 4 readiness: validate DSL Qwen output against HF reference on real GGUFs #118 lands.#44 — docs
[Unreleased]entry for the whole W2 stack.Verification
:llm-agent:jvmTest210 green;:llm-runtime:kllama:jvmTestgreen (gated tests skip cleanly without env);:llm-runtime:gemma-iree:jvmTestgreen (functiongemma provider registration against this stack):llm-core:apiCheck :llm-agent:apiCheck :llm-runtime:kllama:apiCheckgreen — no public API changes in this PRFound while validating (pre-existing, unrelated):
:llm-apps:kllama-cliand:llm-apps:skainet-clifail dependency resolution on clean develop (Could not find sk.ainet.core:skainet-lang-core:— empty version on the app-module classpath), so CLI-level demo validation isn't currently possible; worth a small separate fix.Refs #41 #42 #43 #44 #35 #118
🤖 Generated with Claude Code