Repository navigation
fix: route Azure OpenAI through the OpenAI client - #31952
ethanndickson wants to merge 4 commits into
Conversation
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c072f4748f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Breezy! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Azure OpenAI providers fell through to the OpenAI-compatible client, which drops every provider_options.openai setting and always speaks Chat Completions with max_tokens. Azure now shares the OpenAI client. An unset use_responses_api defaults to Chat Completions for Azure, because a deployment name does not identify the model, and the admin UI shows Use Responses API and Reasoning Model for Azure. Fixes #31714
…type ModelFromConfig checked reasoning_mode against the client name. Azure and Bedrock OpenAI-format models use the OpenAI client, so the runtime check let them through. newModel now runs the same check as the API, with the configured AI provider type, and returns a config error.
ab8e1b1 to
761fea0
Compare
Part of CODAGT-1430 Relates to CODAGT-1541 Stacked on #31743. This bumps fantasy to pick up coder/fantasy#84 and coder/fantasy#85. A Chat Completions tool message can only hold text, so fantasy sends tool result media (images, audio, and now PDFs from #31384) in a `role: "user"` message right after it. That gives whatever the tool returned the authority of the user's own words, which Codex flagged on #31383 as a prompt injection risk for MCP tools that return untrusted files. coder/fantasy#84 starts that user message with a text part naming the tool call and saying the content is tool output, not instructions from the user. It only changes the Chat Completions paths (`openai` without the Responses API, `openaicompat`, OpenRouter and Vercel); Anthropic and Responses keep the media inside the tool result. AI Bridge records the last user message's text as the prompt, so on the step after a tool returns media, it'll now record the label. I think that's fine for now. coder/fantasy#85 fixes the output token limit field. The OpenAI provider sent `max_tokens` unless the model ID looked like a reasoning model, and reasoning models reject it: ``` Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead. ``` That breaks any reasoning model whose ID fantasy doesn't recognise, e.g. an Azure deployment name, which doesn't say what model it serves (Azure providers start using this client in #31952). The OpenAI provider now sends `max_completion_tokens` on every Chat Completions request, which is what OpenAI deprecated `max_tokens` in favour of. `openaicompat`, OpenRouter and Vercel keep `max_tokens`, so self-hosted servers behind the OpenAI Compatible provider are unaffected. I've added a test that checks the field on each route, and the models docs no longer say Max Completion Tokens is only for reasoning models.
|
@codex review |
|
@codex security review |
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
🛡️ Codex Security ReviewSecurity review completed. No security issues were found in this pull request. Reviewed commit: Only the user who started this review can view the report in Codex. ℹ️ About Codex security reviews in GitHubThis is an experimental Codex feature. Security reviews are triggered when:
Once complete, Codex will leave suggestions, or a comment if no findings are found. |
Documentation CheckBlocking
Findings for agents- id: coverage-1
persona: coverage
class: missing
blocking: true
file: docs/coder-agents/models.md
line: 289
quote: "Azure OpenAI models use the Chat Completions API unless you set **Use Responses API** to On, because the deployment name doesn't identify the model."
evidence:
- coderd/x/chatd/chatprovider/chatprovider.go:148-150
- coderd/x/chatd/media_sanitize.go
rule: null
problem: Turning on **Use Responses API** for an Azure model silently drops audio tool-result media, and the note does not mention it.
fix: Add to the Azure note that the Responses route accepts images but not audio or PDFs.
verified: true
- id: coverage-2
persona: coverage
class: missing
blocking: true
file: docs/coder-agents/models.md
line: 289
quote: "Azure OpenAI uses the same options as OpenAI except Reasoning Mode."
evidence:
- site/src/api/chatModelOptionsGenerated.json:90
rule: null
problem: The **Reasoning Model** override is now exposed for Azure, but the Azure note still says Azure uses the same options as OpenAI except Reasoning Mode.
fix: Clarify that the **Reasoning Model** override is available for Azure, distinct from Reasoning Mode.
verified: true
- id: language-1
persona: language
class: rule
blocking: true
file: docs/coder-agents/models.md
line: 289
quote: "Azure OpenAI models use the Chat Completions API unless you set **Use Responses API** to On, because the deployment name doesn't identify the model."
evidence:
- docs/coder-agents/platform-controls/virtual-desktop.md:65
rule: docs/.style/style-guide/formatting.md#bold-for-ui-elements
problem: The option value **On** is literal UI text and should be bolded.
fix: "Azure OpenAI models use the Chat Completions API unless you set **Use Responses API** to **On**, because the deployment name doesn't identify the model."
verified: trueAutomated review via Coder Agents. Reply to a finding's thread with the reason if the finding is wrong. |
| The **OpenAI Compatible** type is a catch-all for any service that exposes an OpenAI-compatible chat completions endpoint. | ||
| Use it to connect to self-hosted models, internal gateways, or third-party proxies like LiteLLM. | ||
|
|
||
| Use the **Azure OpenAI** type for GPT-4o and newer OpenAI models. |
There was a problem hiding this comment.
(Newcomer) The paragraph says which models use the Azure OpenAI type, but it doesn't tell the reader how to check a deployment against that rule. Point them to the model each deployment serves, which the note below says the deployment name doesn't show.
| Use the **Azure OpenAI** type for GPT-4o and newer OpenAI models. | |
| Use the **Azure OpenAI** type for GPT-4o and newer OpenAI models. Check the model that each deployment serves before you choose a type, because the deployment name doesn't show it. |
Closes #31714
Closes CODAGT-1541
Azure OpenAI providers fell through to the OpenAI-compatible client, which ignores every
provider_options.openaisetting and sendsmax_tokensunless the deployment name looks like a reasoning model. So an Azure reasoning deployment with any other name failed with:To fix, Azure now shares the OpenAI client, which sends
max_completion_tokenssince the fantasy bump in #31746. A deployment name doesn't say what model it serves, so an unset Use Responses API defaults to Chat Completions for Azure, the same way Bedrock already defaults to Responses. The model form now also shows Use Responses API and Reasoning Model for Azure, so admins can move a deployment to Responses, or mark it as a reasoning model so we send reasoning effort and droptemperatureandtop_p.Older Azure models reject
max_completion_tokens. Azure doesn't publish a list, but it's the pre-gpt-4o versions that aren't on Azure's retired list:turbo-2024-04-091106-preview,0125-previewandvision-preview1106and0125I've only seen
turbo-2024-04-09fail, but the rest are the same generation, so I'd expect them to fail too. gpt-4o, gpt-4o-mini, gpt-4.1 and newer all accept it. If anyone still runs one of the models above, they can workaround it by adding the deployment under an OpenAI Compatible provider instead.The same goes for non-OpenAI models served from an Azure resource, e.g. DeepSeek or Kimi. They rely on the OpenAI-compatible client to read
reasoning_contentand send it back, which the OpenAI client doesn't do, so they belong under an OpenAI Compatible provider too. I've added a couple of lines to the docs on which Azure models go where.This also means any OpenAI options saved on an Azure model now take effect, where they were silently ignored before. As with OpenAI providers, a model with any of those options set also sends
store: true.I've run this against a real Azure
gpt-5.4deployment, which rejectsmax_tokenswith the error above. Chats with tool calls and reasoning effort work on Chat Completions and with Use Responses API on, and Azure acceptsstorewhether it's set totrueorfalse.