Skip to content

fix: TECH-1717 agent & chat reliability fixes + smoke test coverage - #613

Open
DoganK01 wants to merge 2 commits into
devfrom
TECH-1717
Open

fix: TECH-1717 agent & chat reliability fixes + smoke test coverage#613
DoganK01 wants to merge 2 commits into
devfrom
TECH-1717

Conversation

@DoganK01

@DoganK01 DoganK01 commented Jun 13, 2026

Copy link
Copy Markdown
Contributor
  • RetryExecutionTimeoutError is now retried; GuardrailValidationError warns before re-raise; an explicit per-call retry= overrides the agent-level setting (previously unreachable dead code).
  • Concurrency — per-run state (run_id / output / tool counts) isolated in a ContextVar (agent/run_state.py); concurrent do_async/astream on one agent no longer clobber each other.
  • Chat session state — a failed invoke latches SessionState.ERROR; reopen() recovers non-destructively (history + cumulative duration); invoking a closed session raises SessionClosedError.
  • Usagetool_calls are counted into the usage registry (from ModelResponse.tool_calls).
  • Memorynum_last_messages groups history by user-turn boundary, so the limit holds through tool round-trips.
  • Heartbeatexecute_heartbeat / aexecute_heartbeat gain a print= opt-in (silent by default).
  • Refactor — reuse the ModelResponse.tool_calls property instead of inline ToolCallPart filtering (recorder + 3 call sites).

Usage-metrics accuracy

Three pre-existing telemetry/display defects (low severity, no wrong outputs), fixed and regression-tested:

  • Pipeline step countMemorySaveStep was the only non-streaming step that never self-finalized, so runs logged 23/24 steps and dropped step_statuses['memory_save']. It now finalizes in a finally like every sibling → 24/24.
  • Tool execution time — tool wall time was folded only into the per-run snapshot, so registry-derived Agent.usage.tool_execution_time stayed 0. Now recorded as a kind="tool" registry entry via record_tool_execution_time, through the single AgentRunOutput.add_tool_execution_time chokepoint (no double-count). upsonic_execution_time is unchanged (the same elapsed is added to both duration and tool_execution_time).
  • Cached cost — the per-call cost panels priced cached input at full rate, over-estimating vs the cache-aware chat.usage.cost. Panels now price cache-aware via get_estimated_cost_from_usage (backward-compatible: no cache tokens → identical output).

- retry: ExecutionTimeoutError is now retried; GuardrailValidationError warns before re-raise; an explicit per-call retry= overrides the agent-level setting (was unreachable dead code)
- concurrency: isolate per-run state in a ContextVar (new agent/run_state.py) so concurrent do_async/astream no longer clobber each other's run_id / output / tool counts
- chat: a failed invoke latches SessionState.ERROR; reopen() recovers non-destructively (history + cumulative duration preserved); invoking a closed session raises SessionClosedError
- usage: tool_calls are counted into the registry from ModelResponse.tool_calls
- memory: num_last_messages groups history by user-turn boundary, so the limit holds through tool round-trips
- heartbeat: execute_heartbeat / aexecute_heartbeat gain a print= opt-in (silent by default)
- refactor: reuse the ModelResponse.tool_calls property instead of inline ToolCallPart filtering
- tests: unit tests for every fix; env-based smoke model selection (conftest + helper); concurrency + chat-lifecycle smoke tests; smoke fixes pinning model-specific tests
- docs: sync explanation docs (agent / chat / memory / utils)
@notion-workspace

Copy link
Copy Markdown

…ed cost)

- MemorySaveStep now self-finalizes in a finally block like every other
  non-streaming step, so execution_stats counts all steps (24/24 instead of
  23/24) and records step_statuses['memory_save'].
- Record tool execution time into the usage registry via a new kind='tool'
  entry (record_tool_execution_time), wired through the single chokepoint
  AgentRunOutput.add_tool_execution_time, so Agent.usage.tool_execution_time
  reflects real tool wall time instead of staying 0.
- Price the per-call cost panels cache-aware (get_estimated_cost_from_usage)
  so the displayed Estimated Cost matches the cache-aware chat.usage.cost
  instead of charging cached input at full rate.
- Sync the step-finalization note in explanation/agent/agent.md; add unit and
  smoke regression tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant