fix(tools): avoid Qwen tool-call bridge collision - #110451
Conversation
Competing fix for #110442 alongside #110452 (filed minutes later). This PR renames to |
kvnloo
left a comment
There was a problem hiding this comment.
Both fix #110442 (bridge named tool_call collides with qwen's <tool_call> template tag). One of them should land. Posting this on both PRs since the comparison is the review.
Verdict: #110452 wins
- One-line single-sourced core (
tools/tool_search_catalog.py:20:TOOL_CALL_NAME = "invoke_tool") vs #110451's two frozensets + legacy constant + helper + validation carve-out across 15 files. - Better name.
invoke_toolbreaks thetool_*prefix shared withtool_search/tool_describe. The confusion vector was family resemblance to the template tag;tool_invokekeeps thetool_prefix and stays one token-swap away from the collision. (Also: this was never qwen-only — Nous's own Hermes prompt format trains on<tool_call>delimiters, so Hermes-family models were exposed to the same misread.) - #110452 did the live-model verification (qwen3:14b via ollama, per its body). For a model-behavior bug that's the only evidence class that counts — unit tests can't prove a model stops pattern-matching a name.
- #110451 left the desktop and evals behind (details below). #110452 updated both.
#110451's gaps (not covered by existing comments)
1. Desktop connector rendering regresses. apps/desktop/src/lib/connector-tools.ts:97 (if (name !== 'tool_call') return []) and connector-tool.tsx:382,391,406 (props.toolName === 'tool_call') still key on the old name. After the rename to tool_invoke, connector batch results stop rendering as batches — the renderer no longer recognizes the bridge. #110452 fixed all four sites (accepts both names).
2. The evals harness goes blind. evals/core_tool_deferral/tasks.py:380 (bridge-call counter), worker.py:293-295 (name match), and every evals/tool_search/tool_search_livetest*.py bridge tuple still look for "tool_call". Post-rename, bridge-call metrics silently undercount to zero. #110452 updated all of them.
3. Gratuitous lazy import in a hot path. agent/turn_tool_validation.py:42-56 — is_dispatchable_tool_name does from tools.tool_search_catalog import ... inside the function with except Exception: return False. No cycle exists: tools/__init__.py is explicitly documented side-effect-free, the catalog imports only stdlib + snowballstemmer, and only agent/turn_tool_round.py imports this module. If that import ever fails, validation silently goes strict for every tool call in every turn. Hoist it to module level.
4. The alias preserves the failure loop it claims to fix. Keeping tool_call dispatchable means a qwen model emitting bare tool_call out of template habit (not from the schema — the schema no longer advertises it) still dispatches → resolve_underlying_call shape error → replay → guardrail halt. Both PRs fix the reported trigger (advertised name == tag), but only #110452 gives the habit-emission case a clean unknown-tool signal instead of re-entering the loop.
5. TOOL_CALL_NAME = TOOL_INVOKE_NAME (tools/tool_search_catalog.py:25-26) makes the constant name a lie for every external reader — plugins, user scripts, anything grepping the codebase. #110452 keeps the stale constant name too, but at least it's still the single source of truth rather than an alias of a differently-named constant.
#110452's gaps (fairness)
1. No legacy hint on rejection. A stale tool_call — model habit, eval replay, mixed-version fleet mid-hermes update — gets a bare "unknown tool" with no pointer to the new name. Cheap, high-leverage fix: in the invalid-tool error path, special-case the legacy name with "did you mean invoke_tool?" That's the actual loop-breaking mechanism — the model learns the new name next turn — and it's strictly better than #110451's silent alias, which teaches nothing.
2. Docs need a repo-wide sweep, not one file. liuhao1024 already flagged website/docs/user-guide/features/tool-search.md; the stale bridge name also spans developer-guide docs (agent-loop, gateway-internals, architecture, adding-providers, context-compression-and-caching, …). One-file fix leaves the rest rotting.
Suggested synthesis
Land #110452 + (a) the "did you mean invoke_tool?" hint for the legacy name in the invalid-tool error, (b) the docs sweep, (c) keep the issue's two-tool repro (tool_call vs renamed bridge, same prompt) as the documented acceptance check — that's the test that actually proves the fix, and it should be re-runnable, not a one-off claim in a PR body.
What does this PR do?
Renames the advertised deferred-tool bridge to
tool_invokeso Qwen-family models no longer confuse it with their native<tool_call>template delimiter. Existing literaltool_callbridge calls remain accepted as a dispatch-only compatibility alias.Symptom
With tool deferral enabled, Qwen-family models could emit malformed calls named
tool_callafter interpreting the advertised bridge name as their XML-like wrapper tag. The bridge rejected the malformed shape and the turn could repeat until its guardrail halted it.Impact
Users of Qwen-family and similar local models could lose all deferred tool use for an affected turn, including normal direct calls that followed the malformed bridge interaction.
Bug Cause
Trigger:
tools/tool_search_catalog.pyadvertised the deferred invocation bridge astool_call.Causal chain:
tool_callin the tool schema and collides it with its<tool_call>output tag.Why it is wrong: a model-facing function name must not reuse a native tool-call template delimiter.
Working sibling / contrast:
tool_searchandtool_describeremain model-facing bridge names without this delimiter collision.Ruled out: changing the loop guardrail would not address the malformed call emitted before dispatch.
Fix
The schema now advertises
tool_invokewhile reserving and recognizing legacytool_callonly at dispatch, planner, executor, and TUI lifecycle seams. Agent-loop validation allows the alias only whentool_invokeis actually model-visible; unrelated unknown tools remain rejected.Related Issue
Fixes #110442
Type of Change
Changes Made
tools/tool_search_catalog.pyandtools/tool_search.py— introduce the advertisedtool_invokebridge and keeptool_callas a non-advertised legacy alias.model_tools.py,agent/tool_dispatch_helpers.py,agent/tool_executor.py, andtui_gateway/tool_progress.py— accept both bridge spellings through existing dispatch, planning, execution, and lifecycle paths.agent/turn_tool_validation.pyandagent/turn_tool_round.py— allow the legacy alias only while the advertised bridge is present.tests/agent/test_run_agent.pyandtests/tools/— cover Qwen collision prevention, legacy dispatch, planner behavior, and ordinary invalid-tool rejection.How to Test
scripts/run_tests.sh tests/tools/test_tool_search.py tests/tools/test_model_tools.py tests/tools/test_deferral_fixes.py tests/tools/test_connector_local_batches.py tests/tools/test_connector_bridge_wiring.py -q— 133 passed.scripts/run_tests.sh tests/tools/test_tool_search.py -k qwen_tool_call_tag_is_not_advertised_as_the_deferred_bridge -q— failed onorigin/mainbefore the fix and passes after it.python -m pytest -q tests/agent/test_run_agent.py -k 'legacy_tool_call_bridge_alias or non_bridge_invalid_name'— 2 passed.Checklist
Code
Documentation & Housekeeping
cli-config.yaml.example: N/ACONTRIBUTING.mdorAGENTS.md: N/A