Skip to content

llm-agent: QwenChatTemplate(enableThinking = false) pre-fills an empty <think> block, which Qwen2.5-Instruct never saw — tool calling drops to 0–1/8 vs 4/8 with the official Qwen2.5 template #409

Description

@michalharakal

Summary

QwenChatTemplate's KDoc states that Qwen 2.5 instruct models "use the same ChatML + Hermes-style contract" (QwenChatTemplate.kt:10). The tool-call format is the same, but the Qwen3 template with enableThinking = false pre-fills the generation prompt with an empty <think>\n\n</think> block (QwenChatTemplate.kt:24–28). Qwen2.5-0.5B-Instruct has never seen those tokens in that position and stops producing <tool_call> blocks.

Measured (host JVM, greedy, 8-utterance German TV golden set, 15 tools, 0.53.0)

model template tool calls correct
Qwen2.5-0.5B-Instruct Q8_0 QwenChatTemplate(enableThinking = false) 0–1 / 8
Qwen2.5-0.5B-Instruct Q8_0 faithful port of the official Qwen2.5 chat_template (no think block, tools rendered per the 2.5 Jinja) 4 / 8
Qwen3-0.6B Q8_0 QwenChatTemplate(enableThinking = false) 7 / 8

Proposal

Add a Qwen2.5 variant (QwenChatTemplate(thinking = Thinking.NONE) or a Qwen25ChatTemplate) that omits the think pre-fill and renders tools exactly as the official Qwen2.5 template does, and have ChatSession/AgentLoop pick it from the GGUF metadata (general.name/architecture version) instead of documenting the two families as equivalent. Happy to contribute the port (≈ 60 lines, verified against the HF Jinja for the tool and tool-response turns).

Context. Measured on 2026-09-03 while bringing FunctionGemma-270M up as an NLU cartridge on a MagentaTV One (Amlogic S905X5L, Android 14, armeabi-v7a-only 32-bit process, 4× ARMv8 @ 2.0 GHz, Mali GPU) with SKaiNET 0.53.0 + SKaiNET-transformers 0.53.0 from Maven Central. Harness and raw result files: tvv/nlu-llm-harness (local; can share on request).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions