Repository navigation
Preserve reasoning items across multi-turn Responses API calls - #26
Conversation
Reasoning output items returned by gpt-5 and o-series models were silently dropped during response parsing, so the encrypted_content required to round-trip the reasoning back on the next turn was lost. Reasoning items now become thought-channel MessagePart instances attached to the next message or function_call candidate, and they are re-emitted as top-level reasoning input items on subsequent requests. Reasoning token counts from output_tokens_details are also threaded into TokenUsage. Thought-signature and thought-token support is runtime-gated on the php-ai-client SDK so the plugin remains compatible with SDK versions below 1.3.0 (the gating no-ops on older SDKs).
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
…blob The OpenAI Responses API expects all three fields back when echoing a reasoning item, but the SDK's MessagePart::$thoughtSignature is a single opaque string. Encode the trio as JSON on inbound and decode on outbound so id and summary are preserved across multi-turn calls, not just the encrypted_content blob. Co-authored-by: Takshil-Kunadia <takshilkunadia@gmail.com>
The Responses API constraint that function_call / function_response parts must
be the only part in a wire Message applies to the serialized message body,
not to the in-memory Message DTO. Thought-channel parts are emitted as
separate top-level `reasoning` input items by getReasoningInputItems() and
skipped by getMessageInputItem(), so they never reach the wire Message and
must not count toward the 'only part' rule.
Before this fix, when the API returned [reasoning, function_call] (the common
case for reasoning-capable models invoking tools), parseFunctionCallOutputToCandidate
built a Candidate with a Message of [reasoning_part, function_call_part]. On
the next turn, prepareInputParam called validateMessages first, which counted
both parts, found the function_call, and threw:
InvalidArgumentException: Function call parts must be the only part in a
message for the OpenAI Responses API.
The fix filters thought-channel parts out before counting and type-checking,
keeping the rule identical for genuine wire content.
felixarntz
left a comment
There was a problem hiding this comment.
@superdav42 This looks great so far!
One early point of feedback: Could you change the PHP AI Client requirement of the package to 1.3.0 please? We don't need to maintain back compat with those older versions, especially because WordPress uses 1.3.0+ internally, so it would never be relevant there.
And for any other project, I think it's reasonable to expect them to upgrade too.
Per review feedback: WordPress ships php-ai-client 1.3+ internally and other consumers can be expected to upgrade, so the defensive method_exists() / ReflectionMethod guards around getChannel(), getThoughtSignature(), and the 4th TokenUsage parameter are no longer needed. Changes: - composer.json: require-dev wordpress/php-ai-client ^1.3 || dev-trunk (was ^0.4 || dev-trunk). - OpenAiTextGenerationModel::getReasoningInputItems(): drop the method_exists guards; call $part->getChannel()->isThought() and $part->getThoughtSignature() directly. - OpenAiTextGenerationModel::validateMessages(): drop the same guard. - OpenAiTextGenerationModel::getMessageInputItem(): drop the same guard. - OpenAiTextGenerationModel::parseReasoningOutputToPart(): drop the method_exists fallback (the method is always present at >=1.3) and the @phpstan-ignore-next-line on the MessagePart constructor. - OpenAiTextGenerationModel::buildTokenUsage(): drop the ReflectionMethod check and always pass $thoughtTokens to TokenUsage (4-param signature is part of the 1.3 contract). Also resolves the @SInCE n.e.x.t placeholders to @SInCE 1.1.0 (next minor for new functionality) and trims verbose docblocks.
|
Done in ad97bfc.
Also resolved Verified locally with aidevops.sh v3.17.14 plugin for OpenCode v1.15.6 with claude-sonnet-4-6 spent 26m and 40 tokens on this as a headless worker. |
There was a problem hiding this comment.
Pull request overview
This PR updates the OpenAI Responses API adapter to preserve and round-trip “reasoning” output items across multi-turn conversations by converting them into thought-channel message parts (inbound) and re-emitting them as top-level reasoning input items (outbound). It also extends token accounting to include reasoning/thought tokens when present.
Changes:
- Parse
output[].type === 'reasoning'into thought-channelMessageParts, buffer them, and attach them to the subsequent message / function_call candidate. - Serialize thought-channel parts with a stored signature back into top-level
{type:'reasoning', id, encrypted_content, summary}input items before the corresponding message on the next turn. - Introduce thought-token extraction from
usage.output_tokens_details.reasoning_tokensand adjust dev dependency constraints/lock metadata.
Reviewed changes
Copilot reviewed 2 out of 3 changed files in this pull request and generated 7 comments.
| File | Description |
|---|---|
| src/Models/OpenAiTextGenerationModel.php | Implements buffering, parsing, and round-tripping of Responses API reasoning items plus thought token usage handling. |
| composer.json | Changes dev constraint for wordpress/php-ai-client to ^1.3. |
| composer.lock | Updates lock metadata (plugin-api-version). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| $thoughtTokens = $details['reasoning_tokens']; | ||
| } | ||
|
|
||
| return new TokenUsage($inputTokens, $outputTokens, $totalTokens, $thoughtTokens); |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
…-pr-26-to-require-wordpress-php-ai
# Conflicts: # composer.lock
# Conflicts: # composer.json # composer.lock # src/Provider/OpenAiProvider.php
|
@felixarntz back to you for review/merge here |
Co-authored-by: Darin Kotter <darin.kotter@gmail.com>
Co-authored-by: Darin Kotter <darin.kotter@gmail.com>
Summary
Reasoning output items returned by
gpt-5and o-series models are currently dropped during response parsing. Theencrypted_contentblob the Responses API expects back on the next turn is lost, so multi-turn conversations with reasoning models silently degrade in quality and the OpenAI docs guidance on round-tripping reasoning items cannot be followed.This PR threads reasoning items through
OpenAiTextGenerationModelin both directions:output[].type === 'reasoning'items are turned into thought-channelMessagePartinstances and attached to the nextmessage/function_callcandidate's message parts (alongside the answer text, matching the Anthropic provider's pattern). The reasoning item's{id, encrypted_content, summary}trio is packed into a JSON blob stored on the part'sthoughtSignatureso all three fields are recoverable on round-trip. The summary text is also surfaced as the part's text content for human-readable consumption.{type: 'reasoning', id, encrypted_content, summary}input items before the corresponding message item. The signature blob is JSON-decoded to restore the original shape.usage.output_tokens_details.reasoning_tokensis now threaded intoTokenUsage::$thoughtTokenswhen present.Resolves #13.
Credit to @Takshil-Kunadia — the JSON-blob signature encoding (commit 2) was cherry-picked from #20.
Why reasoning is attached to the existing candidate, not emitted as a separate one
GenerativeAiResult::toText(),toFile(),toMessage()and friends all inspect only the first candidate (php-ai-client/src/Results/DTO/GenerativeAiResult.php:189). Becauseoutput[]from the Responses API always lists reasoning items before the corresponding message item, an alternative design that emits each reasoning item as its ownCandidatewould put the reasoning candidate first —toText()would then throwRuntimeException('No text content found in first candidate')for every gpt-5 / o-series response, breaking the canonicalwp_ai_client_prompt()->generate_text()API.Attaching reasoning as additional thought-channel parts on the answer's candidate keeps
toText()working (it filters to content-channel parts) and matches how the Anthropic provider already surfacesthinkingblocks.SDK compatibility
The thought-signature and thought-token DTO surface (
MessagePart::$thoughtSignature,TokenUsage::$thoughtTokens) shipped inwordpress/php-ai-client1.3.0 (#216). This plugin's currentcomposer.jsonconstraint^0.4 || dev-trunkstraddles that boundary, so all new code paths are runtime-gated:MessagePart::getThoughtSignature()/ 3-argMessagePartconstructor — guarded bymethod_exists(MessagePart::class, 'getThoughtSignature').TokenUsageconstructor — guarded byReflectionMethod::getNumberOfParameters().method_exists($part, 'getChannel').On pre-1.3 SDKs the new code paths no-op cleanly; on 1.3+ they activate. No dependency bump needed.
Verification
Static analysis:
composer phpstanclean against the locked SDK 0.4.2 (CI's resolution).Coding standards:
composer phpcsclean.toText()regression check: the canonical answer-extraction path returns"Hello world."when the response contains a leading reasoning item (would throw under an alternative design that emits reasoning as a standalone first candidate).End-to-end round-trip test (mock HTTP transporter, live WP stack, real-shape Responses API body with
id+encrypted_content+summary):The verifier instantiates
OpenAiTextGenerationModelthrough the liveProviderRegistry, swaps in a recording mockHttpTransporterInterface(the configured project API key lacksapi.responses.writescope, so the real network call returns 401), and confirms the patched parse + serialize paths handle real-shape gpt-5 response bodies correctly across two consecutive turns.Backwards compatibility: re-ran the same verifier against
php-ai-client 0.4.2(the locked SDK version) — patch silently no-ops, no fatals, parser behaves identically to currenttrunkfor callers without reasoning items.Risk
output[].type === 'reasoning'(gpt-5, o-series with reasoning enabled).Message::getParts()and don't filter by channel will now see those parts.GenerativeAiResult::toText()filters to content channel and is unaffected.