Skip to content

fix(provider): validate final Chat Completions tool pairing before sending - #10752

Open
bizu314926-collab wants to merge 1 commit into
AstrBotDevs:masterfrom
bizu314926-collab:fix/provider-final-tool-pairing
Open

bizu314926-collab wants to merge 1 commit into
AstrBotDevs:masterfrom
bizu314926-collab:fix/provider-final-tool-pairing

Conversation

@bizu314926-collab

@bizu314926-collab bizu314926-collab commented Oct 25, 2026 •

Copy link
Copy Markdown

Direct Provider calls, summary requests, and custom_extra_body.messages can bypass the history repair in #10342. For example, an assistant declaring calls a and b followed only by result a currently reaches the SDK. The existing final sanitizer removes orphaned or duplicate receipts but does not reject an incomplete declaration group, and it only examines the base messages.

This change validates the effective final Chat Completions message sequence before SDK invocation in both streaming and non-streaming requests. A remaining structural error stops locally without sending the request or trying another key/provider.

Related to #10338; complements #10342. The separation was discussed in this COMMENT review. This PR contains no ContextManager/ContextTruncator changes or supplemental tests for that PR, and does not replace its historical repair.

Modifications

  • Validate final messages after Provider overrides and the existing sanitizer. If extra_body contains messages, validate that effective override even when its value is None.

  • Require one contiguous response per declared call, allowing any result order within a parallel group. Reject missing/invalid IDs, incomplete groups and duplicate declaration IDs within a group. Separate completed groups may reuse IDs.

  • Preserve existing base-message orphan/duplicate receipt cleanup; make its ID scan type-safe so malformed structures reach the local validation error.

  • Rethrow only the new local ToolMessageProtocolError before API recovery, and stop runner fallback only for that type. This also prevents diagnostic values such as index=429 or pending_count=429 from being mistaken for rate limiting. Remote error classification, retry predicates and fallback configuration are unchanged.

  • Add 99 parameterized tests. Add matching tool receipts to two existing empty-content normalization fixtures while keeping their original normalization assertions.

  • This is NOT a breaking change.

Compatibility consideration: ID-less modern tool calls are rejected on the guarded Chat Completions path. #10342 deliberately preserves all-ID-less history in the generic context layer; this stricter final-adapter policy is explicitly offered for maintainer review. Already repaired base orphan/duplicate receipts remain accepted. Raw extra_body.messages overrides are validated without that repair.

This is pairing validation, not complete API schema validation. Empty message lists and legacy function_call/function matching remain outside its contract. It covers inherited Chat Completions send paths; Responses and native Gemini/Anthropic adapters are not claimed to be covered.

Test Results

Fresh verification against master da8e7587a8df578225dd111374e0581557c4f40f:

Source Environment (Windows) Result
This patch on master Python 3.12.10 / OpenAI 3.37.0 / pydantic 2.14.1 365 passed
This patch on master Python 3.13.15 / OpenAI 3.0.0 / pydantic 2.13.4 365 passed
master + #10342 delta + this patch Python 3.13.15 / OpenAI 3.0.0 / pydantic 2.13.4 372 passed

The #10342 head tested was e7e117424258fb3ae4cc58400025ae49ae7fb97d. Its four changed paths are disjoint from this patch's five paths. Applying its rebased delta and then this patch succeeded without conflicts.

Relevant test command:

python -m pytest -q tests/agent tests/test_conversation_checkpoint.py tests/test_openai_source.py tests/test_tool_loop_agent_runner.py tests/test_openai_tool_pairing.py tests/test_openai_responses_source.py tests/test_openai_thinking_tags.py tests/test_request_retry.py

The local runs used an additional external test launcher to isolate runtime data and deny network access. All model responses came from in-memory SDK substitutes. OpenAI/Azure tests built actual SDK request JSON with a fail-if-send in-memory transport; no SDK request was sent. Only the Windows standard library's private asyncio socketpair connection to its own listener was allowed by the launcher. No external model calls or production changes were made.

The prior red baseline for the same 99 new cases was 63 failed / 36 passed on unmodified master 0e997ccd9e8b47930c830e69db6a657dfbf44d9d. This baseline-update pass retained the approved patch byte-for-byte. The Python 3.12 run has one pre-existing audioop deprecation warning; there are no failed or skipped cases in the fresh matrices.

Ruff 0.15.22: project format/check passed (524 files unchanged), and all five changed files including tests passed explicit check/format-check. git diff --cached --check passed. Added-line secret/private-data scanning and independent diff review found no blocking issue; new fixtures use synthetic IDs, keys and .invalid hosts.

Checklist

Summary by Sourcery

Validate final Chat Completions tool-call pairings locally and stop malformed requests before transmission or fallback.

Bug Fixes:

  • Block Chat Completions requests with incomplete, invalid, duplicate, or orphaned tool-call/result pairings before they reach the SDK.
  • Prevent local tool-pairing validation errors from triggering provider or key fallback, including when diagnostics contain rate-limit-like values.

Enhancements:

  • Validate the effective final message sequence for both streaming and non-streaming requests, including custom message overrides, while preserving existing receipt cleanup and legacy message compatibility.

Tests:

  • Add comprehensive parameterized coverage for final tool-pairing validation, request blocking, fallback prevention, provider paths, overrides, and SDK request construction.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!

Sourcery assessment

Approved.


Sourcery is free for open source - if you like our reviews please consider sharing them ✨

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant