Summary
QwenChatTemplate's KDoc states that Qwen 2.5 instruct models "use the same ChatML + Hermes-style contract" (QwenChatTemplate.kt:10). The tool-call format is the same, but the Qwen3 template with enableThinking = false pre-fills the generation prompt with an empty <think>\n\n</think> block (QwenChatTemplate.kt:24–28). Qwen2.5-0.5B-Instruct has never seen those tokens in that position and stops producing <tool_call> blocks.
Measured (host JVM, greedy, 8-utterance German TV golden set, 15 tools, 0.53.0)
| model |
template |
tool calls correct |
| Qwen2.5-0.5B-Instruct Q8_0 |
QwenChatTemplate(enableThinking = false) |
0–1 / 8 |
| Qwen2.5-0.5B-Instruct Q8_0 |
faithful port of the official Qwen2.5 chat_template (no think block, tools rendered per the 2.5 Jinja) |
4 / 8 |
| Qwen3-0.6B Q8_0 |
QwenChatTemplate(enableThinking = false) |
7 / 8 |
Proposal
Add a Qwen2.5 variant (QwenChatTemplate(thinking = Thinking.NONE) or a Qwen25ChatTemplate) that omits the think pre-fill and renders tools exactly as the official Qwen2.5 template does, and have ChatSession/AgentLoop pick it from the GGUF metadata (general.name/architecture version) instead of documenting the two families as equivalent. Happy to contribute the port (≈ 60 lines, verified against the HF Jinja for the tool and tool-response turns).
Context. Measured on 2026-09-03 while bringing FunctionGemma-270M up as an NLU cartridge on a MagentaTV One (Amlogic S905X5L, Android 14, armeabi-v7a-only 32-bit process, 4× ARMv8 @ 2.0 GHz, Mali GPU) with SKaiNET 0.53.0 + SKaiNET-transformers 0.53.0 from Maven Central. Harness and raw result files: tvv/nlu-llm-harness (local; can share on request).
Summary
QwenChatTemplate's KDoc states that Qwen 2.5 instruct models "use the same ChatML + Hermes-style contract" (QwenChatTemplate.kt:10). The tool-call format is the same, but the Qwen3 template withenableThinking = falsepre-fills the generation prompt with an empty<think>\n\n</think>block (QwenChatTemplate.kt:24–28). Qwen2.5-0.5B-Instruct has never seen those tokens in that position and stops producing<tool_call>blocks.Measured (host JVM, greedy, 8-utterance German TV golden set, 15 tools, 0.53.0)
QwenChatTemplate(enableThinking = false)chat_template(no think block, tools rendered per the 2.5 Jinja)QwenChatTemplate(enableThinking = false)Proposal
Add a Qwen2.5 variant (
QwenChatTemplate(thinking = Thinking.NONE)or aQwen25ChatTemplate) that omits the think pre-fill and renders tools exactly as the official Qwen2.5 template does, and haveChatSession/AgentLooppick it from the GGUF metadata (general.name/architecture version) instead of documenting the two families as equivalent. Happy to contribute the port (≈ 60 lines, verified against the HF Jinja for the tool and tool-response turns).Context. Measured on 2026-09-03 while bringing FunctionGemma-270M up as an NLU cartridge on a MagentaTV One (Amlogic S905X5L, Android 14,
armeabi-v7a-only 32-bit process, 4× ARMv8 @ 2.0 GHz, Mali GPU) with SKaiNET 0.53.0 + SKaiNET-transformers 0.53.0 from Maven Central. Harness and raw result files:tvv/nlu-llm-harness(local; can share on request).