Skip to content

fix: replay OpenAI web searches inline without item IDs - #31743

Merged
ethanndickson merged 1 commit into
codagt-1430-inprocess-mcp-media-capfrom
ethan/openai-web-search-inline-replay
Nov 10, 2026
Merged

ethanndickson merged 1 commit into
codagt-1430-inprocess-mcp-media-capfrom
ethan/openai-web-search-inline-replay

Conversation

@ethanndickson

Copy link
Copy Markdown
Member

Closes CODAGT-1425

Stacked on #31386.

coder/fantasy#78, which is already on main, fixed the searches with no reasoning before them by sending them inline without their id. Stored chats still send a search as an item_reference when a reasoning item came right before it, though. The API pairs any stored ID with that reasoning item and rejects the whole request if it's missing, and a reference doesn't save any tokens over an inline item anyway.

This bumps fantasy to pick up coder/fantasy#77, which just never references a search, so stored chats now replay earlier searches inline, like unstored ones. It also drops the id from replayed computer calls, which have the same problem.

I've had to stack this on the CODAGT-1430 PRs, as coder/fantasy#77 merged after coder/fantasy#73 and coder/fantasy#74, so any pin that has it has them too. TestOpenAIResponsesNoStaleWebSearchReplay now covers a stored chat, and I deleted TestOpenAIResponsesFullReplayPairsReasoningAndWebSearch, as it asserted the order of references we no longer send.

@linear-code

linear-code Bot commented Nov 4, 2026

Copy link
Copy Markdown

CODAGT-1425

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-09T06:07:56.316235Z 0673d09 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: f9ee699e4d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ethanndickson
ethanndickson force-pushed the ethan/openai-web-search-inline-replay branch from f9ee699 to 0533a70 Compare November 4, 2026 07:02
@ethanndickson
ethanndickson force-pushed the ethan/openai-web-search-inline-replay branch from 0533a70 to 8210b3f Compare November 4, 2026 07:16
@ethanndickson
ethanndickson marked this pull request as ready for review November 4, 2026 07:54
@ethanndickson
ethanndickson force-pushed the ethan/openai-web-search-inline-replay branch from 8210b3f to 26b6781 Compare November 5, 2026 08:12
@ethanndickson
ethanndickson force-pushed the ethan/openai-web-search-inline-replay branch from 26b6781 to 0673d09 Compare November 9, 2026 06:04
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Swish!

Reviewed commit: 0673d090f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ethanndickson
ethanndickson force-pushed the ethan/openai-web-search-inline-replay branch from 0673d09 to 628d85d Compare November 9, 2026 06:48
Pins coder/fantasy#77 (04dc16d). Stored chats now replay earlier web searches inline, like unstored ones.
@ethanndickson
ethanndickson force-pushed the ethan/openai-web-search-inline-replay branch from 628d85d to 0554e90 Compare November 9, 2026 07:32
@ethanndickson
ethanndickson merged commit d4edf16 into main Nov 10, 2026
36 of 56 checks passed
@ethanndickson
ethanndickson deleted the ethan/openai-web-search-inline-replay branch November 10, 2026 13:06
ethanndickson added a commit that referenced this pull request Nov 10, 2026
Part of CODAGT-1430
Relates to CODAGT-1541

Stacked on #31743.

This bumps fantasy to pick up coder/fantasy#84 and coder/fantasy#85.

A Chat Completions tool message can only hold text, so fantasy sends
tool result media (images, audio, and now PDFs from #31384) in a `role:
"user"` message right after it. That gives whatever the tool returned
the authority of the user's own words, which Codex flagged on #31383 as
a prompt injection risk for MCP tools that return untrusted files.

coder/fantasy#84 starts that user message with a text part naming the
tool call and saying the content is tool output, not instructions from
the user. It only changes the Chat Completions paths (`openai` without
the Responses API, `openaicompat`, OpenRouter and Vercel); Anthropic and
Responses keep the media inside the tool result. AI Bridge records the
last user message's text as the prompt, so on the step after a tool
returns media, it'll now record the label. I think that's fine for now.

coder/fantasy#85 fixes the output token limit field. The OpenAI provider
sent `max_tokens` unless the model ID looked like a reasoning model, and
reasoning models reject it:

```
Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
```

That breaks any reasoning model whose ID fantasy doesn't recognise, e.g.
an Azure deployment name, which doesn't say what model it serves (Azure
providers start using this client in #31952). The OpenAI provider now
sends `max_completion_tokens` on every Chat Completions request, which
is what OpenAI deprecated `max_tokens` in favour of. `openaicompat`,
OpenRouter and Vercel keep `max_tokens`, so self-hosted servers behind
the OpenAI Compatible provider are unaffected. I've added a test that
checks the field on each route, and the models docs no longer say Max
Completion Tokens is only for reasoning models.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants