Skip to content

fix: balance callback lifecycle for hallucinated tool calls#4998

Open
giulio-leone wants to merge 2 commits intogoogle:mainfrom
giulio-leone:fix/hallucinated-tool-callback-lifecycle
Open

fix: balance callback lifecycle for hallucinated tool calls#4998
giulio-leone wants to merge 2 commits intogoogle:mainfrom
giulio-leone:fix/hallucinated-tool-callback-lifecycle

Conversation

@giulio-leone
Copy link
Copy Markdown

Summary

Fixes #4775

When an LLM hallucinates a tool name that doesn't exist in tools_dict, _get_tool() raises ValueError. Previously, on_tool_error_callback fired immediately — before before_tool_callback and outside the OpenTelemetry tracer span. This caused plugins that push/pop spans (e.g. BigQueryAgentAnalyticsPlugin's TraceManager) to pop the parent agent span, corrupting the trace stack for all subsequent tool calls in the session.

Root Cause

The callback lifecycle contract is:

before_tool_callback → (tool execution OR on_tool_error_callback) → after_tool_callback

For hallucinated tools, the old code path was:

on_tool_error_callback → (return or raise)  # before_tool_callback never called!

This violated the lifecycle invariant — plugins that push a span in before_tool_callback never get to push, but on_tool_error_callback still pops, corrupting the stack.

Fix

Move the ValueError handling inside _run_with_trace() so that:

  1. before_tool_callback always fires first (balanced push)
  2. The error is surfaced within the OTel span context
  3. on_tool_error_callback fires after before_tool_callback

Applied to both handle_function_calls_async and handle_function_calls_live code paths.

Testing

  • 2 new tests in test_plugin_tool_callbacks.py:
    • test_hallucinated_tool_fires_before_and_error_callbacks: Verifies callback order (before → error)
    • test_hallucinated_tool_raises_when_no_error_callback: Verifies ValueError propagates correctly
  • All 12 callback tests pass
  • Full unit test suite: 4727 passed, 0 regressions

⚠️ This reopens #4808 which was accidentally closed due to fork deletion.

When an LLM hallucinates a tool name, _get_tool() raises ValueError.
Previously, on_tool_error_callback fired immediately — before
before_tool_callback and outside the OTel tracer span.  This caused
plugins that push/pop spans (e.g. BigQueryAgentAnalyticsPlugin's
TraceManager) to pop the parent agent span, corrupting the trace
stack for all subsequent tool calls.

Move the ValueError handling inside _run_with_trace() so that:
1. before_tool_callback always fires first (balanced push)
2. The error is surfaced within the OTel span context
3. on_tool_error_callback fires after before_tool_callback

Applied to both handle_function_calls_async and
handle_function_calls_live code paths.

Fixes google#4775
@adk-bot adk-bot added the core [Component] This issue is related to the core interface and implementation label Mar 25, 2026
@rohityan rohityan self-assigned this Mar 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

core [Component] This issue is related to the core interface and implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Unbalanced tool lifecycle callbacks for hallucinated tools cause TraceManager stack corruption in plugins

3 participants