Skip to content

fix: route Azure OpenAI through the OpenAI client - #31952

Open
ethanndickson wants to merge 4 commits into
mainfrom
issue-31714-e16j
Open

ethanndickson wants to merge 4 commits into
mainfrom
issue-31714-e16j

Conversation

@ethanndickson

@ethanndickson ethanndickson commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Closes #31714
Closes CODAGT-1541

Azure OpenAI providers fell through to the OpenAI-compatible client, which ignores every provider_options.openai setting and sends max_tokens unless the deployment name looks like a reasoning model. So an Azure reasoning deployment with any other name failed with:

Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.

To fix, Azure now shares the OpenAI client, which sends max_completion_tokens since the fantasy bump in #31746. A deployment name doesn't say what model it serves, so an unset Use Responses API defaults to Chat Completions for Azure, the same way Bedrock already defaults to Responses. The model form now also shows Use Responses API and Reasoning Model for Azure, so admins can move a deployment to Responses, or mark it as a reasoning model so we send reasoning effort and drop temperature and top_p.

Older Azure models reject max_completion_tokens. Azure doesn't publish a list, but it's the pre-gpt-4o versions that aren't on Azure's retired list:

  • gpt-4 turbo-2024-04-09
  • gpt-4 1106-preview, 0125-preview and vision-preview
  • gpt-35-turbo 1106 and 0125

I've only seen turbo-2024-04-09 fail, but the rest are the same generation, so I'd expect them to fail too. gpt-4o, gpt-4o-mini, gpt-4.1 and newer all accept it. If anyone still runs one of the models above, they can workaround it by adding the deployment under an OpenAI Compatible provider instead.

The same goes for non-OpenAI models served from an Azure resource, e.g. DeepSeek or Kimi. They rely on the OpenAI-compatible client to read reasoning_content and send it back, which the OpenAI client doesn't do, so they belong under an OpenAI Compatible provider too. I've added a couple of lines to the docs on which Azure models go where.

This also means any OpenAI options saved on an Azure model now take effect, where they were silently ignored before. As with OpenAI providers, a model with any of those options set also sends store: true.

I've run this against a real Azure gpt-5.4 deployment, which rejects max_tokens with the error above. Chats with tool calls and reasoning effort work on Chat Completions and with Use Responses API on, and Azure accepts store whether it's set to true or false.

@linear-code

linear-code Bot commented Nov 10, 2026

Copy link
Copy Markdown

CODAGT-1541

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 10, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T01:22:14.527838Z 761fea0 Manual request
🔒 Security Review ✅ Completed 2026-11-12T01:25:42.710125Z 761fea0 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c072f4748f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Breezy!

Reviewed commit: ab8e1b1ee6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from codagt-1430-fantasy-media-label to main November 10, 2026 13:06
@ethanndickson
ethanndickson temporarily deployed to main November 10, 2026 13:06 — with GitHub Actions Inactive
Azure OpenAI providers fell through to the OpenAI-compatible client, which drops every provider_options.openai setting and always speaks Chat Completions with max_tokens. Azure now shares the OpenAI client. An unset use_responses_api defaults to Chat Completions for Azure, because a deployment name does not identify the model, and the admin UI shows Use Responses API and Reasoning Model for Azure.

Fixes #31714
…type

ModelFromConfig checked reasoning_mode against the client name. Azure and
Bedrock OpenAI-format models use the OpenAI client, so the runtime check
let them through. newModel now runs the same check as the API, with the
configured AI provider type, and returns a config error.
ethanndickson added a commit that referenced this pull request Nov 10, 2026
Part of CODAGT-1430
Relates to CODAGT-1541

Stacked on #31743.

This bumps fantasy to pick up coder/fantasy#84 and coder/fantasy#85.

A Chat Completions tool message can only hold text, so fantasy sends
tool result media (images, audio, and now PDFs from #31384) in a `role:
"user"` message right after it. That gives whatever the tool returned
the authority of the user's own words, which Codex flagged on #31383 as
a prompt injection risk for MCP tools that return untrusted files.

coder/fantasy#84 starts that user message with a text part naming the
tool call and saying the content is tool output, not instructions from
the user. It only changes the Chat Completions paths (`openai` without
the Responses API, `openaicompat`, OpenRouter and Vercel); Anthropic and
Responses keep the media inside the tool result. AI Bridge records the
last user message's text as the prompt, so on the step after a tool
returns media, it'll now record the label. I think that's fine for now.

coder/fantasy#85 fixes the output token limit field. The OpenAI provider
sent `max_tokens` unless the model ID looked like a reasoning model, and
reasoning models reject it:

```
Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.
```

That breaks any reasoning model whose ID fantasy doesn't recognise, e.g.
an Azure deployment name, which doesn't say what model it serves (Azure
providers start using this client in #31952). The OpenAI provider now
sends `max_completion_tokens` on every Chat Completions request, which
is what OpenAI deprecated `max_tokens` in favour of. `openaicompat`,
OpenRouter and Vercel keep `max_tokens`, so self-hosted servers behind
the OpenAI Compatible provider are unaffected. I've added a test that
checks the field on each route, and the models docs no longer say Max
Completion Tokens is only for reasoning models.
@ethanndickson
ethanndickson temporarily deployed to main November 10, 2026 13:07 — with GitHub Actions Inactive
@ethanndickson
ethanndickson marked this pull request as draft November 10, 2026 14:20
@ethanndickson
ethanndickson marked this pull request as ready for review November 12, 2026 01:19
@ethanndickson
ethanndickson temporarily deployed to main November 12, 2026 01:19 — with GitHub Actions Inactive
@ethanndickson

Copy link
Copy Markdown
Member Author

@codex review

@ethanndickson

Copy link
Copy Markdown
Member Author

@codex security review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 761fea0e4c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@chatgpt-codex-connector

Copy link
Copy Markdown

🛡️ Codex Security Review

Security review completed. No security issues were found in this pull request.

Reviewed commit: 761fea0e4c

View security finding report

Only the user who started this review can view the report in Codex.

ℹ️ About Codex security reviews in GitHub

This is an experimental Codex feature. Security reviews are triggered when:

  • You comment "@codex security review"
  • A regular code review gets triggered (for example, "@codex review" or when a PR is opened), and you’re opted in so security review runs alongside code review

Once complete, Codex will leave suggestions, or a comment if no findings are found.

@ethanndickson
ethanndickson temporarily deployed to main November 12, 2026 01:53 — with GitHub Actions Inactive
Comment thread docs/coder-agents/models.md Resolved
Comment thread docs/coder-agents/models.md Resolved
Comment thread docs/coder-agents/models.md Resolved
@coderagents

coderagents Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Documentation Check

Blocking

  • docs/coder-agents/models.md (open) - The Azure note should mention that the Responses route accepts images but not audio or PDFs, since audio tool results are silently dropped when Use Responses API is set to On.
  • docs/coder-agents/models.md (open) - The Azure note should state that the Reasoning Model override is available for Azure, distinct from Reasoning Mode.
  • docs/coder-agents/models.md (open) - The option value On should be bolded for consistency with the UI-element formatting rule. (fixed in 26612e5)
Findings for agents
- id: coverage-1
  persona: coverage
  class: missing
  blocking: true
  file: docs/coder-agents/models.md
  line: 289
  quote: "Azure OpenAI models use the Chat Completions API unless you set **Use Responses API** to On, because the deployment name doesn't identify the model."
  evidence:
    - coderd/x/chatd/chatprovider/chatprovider.go:148-150
    - coderd/x/chatd/media_sanitize.go
  rule: null
  problem: Turning on **Use Responses API** for an Azure model silently drops audio tool-result media, and the note does not mention it.
  fix: Add to the Azure note that the Responses route accepts images but not audio or PDFs.
  verified: true

- id: coverage-2
  persona: coverage
  class: missing
  blocking: true
  file: docs/coder-agents/models.md
  line: 289
  quote: "Azure OpenAI uses the same options as OpenAI except Reasoning Mode."
  evidence:
    - site/src/api/chatModelOptionsGenerated.json:90
  rule: null
  problem: The **Reasoning Model** override is now exposed for Azure, but the Azure note still says Azure uses the same options as OpenAI except Reasoning Mode.
  fix: Clarify that the **Reasoning Model** override is available for Azure, distinct from Reasoning Mode.
  verified: true

- id: language-1
  persona: language
  class: rule
  blocking: true
  file: docs/coder-agents/models.md
  line: 289
  quote: "Azure OpenAI models use the Chat Completions API unless you set **Use Responses API** to On, because the deployment name doesn't identify the model."
  evidence:
    - docs/coder-agents/platform-controls/virtual-desktop.md:65
  rule: docs/.style/style-guide/formatting.md#bold-for-ui-elements
  problem: The option value **On** is literal UI text and should be bolded.
  fix: "Azure OpenAI models use the Chat Completions API unless you set **Use Responses API** to **On**, because the deployment name doesn't identify the model."
  verified: true

Automated review via Coder Agents. Reply to a finding's thread with the reason if the finding is wrong.

@ethanndickson
ethanndickson temporarily deployed to main November 12, 2026 03:28 — with GitHub Actions Inactive
@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 06:01 — with GitHub Actions Inactive
@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 06:07 — with GitHub Actions Inactive
@ethanndickson
ethanndickson temporarily deployed to main November 13, 2026 06:40 — with GitHub Actions Inactive

@coderagents coderagents Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Automated doc-check review of the latest commit, generated by Coder Agents.

The **OpenAI Compatible** type is a catch-all for any service that exposes an OpenAI-compatible chat completions endpoint.
Use it to connect to self-hosted models, internal gateways, or third-party proxies like LiteLLM.

Use the **Azure OpenAI** type for GPT-4o and newer OpenAI models.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Newcomer) The paragraph says which models use the Azure OpenAI type, but it doesn't tell the reader how to check a deployment against that rule. Point them to the model each deployment serves, which the note below says the deployment name doesn't show.

Suggested change
Use the **Azure OpenAI** type for GPT-4o and newer OpenAI models.
Use the **Azure OpenAI** type for GPT-4o and newer OpenAI models. Check the model that each deployment serves before you choose a type, because the deployment name doesn't show it.

This branch was previously deployed

1 inactive deployment
main — d5cd6370 Deployed Nov 13, 2026 by ethanndickson via cla #255788
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Azure OpenAI returned an unexpected error: 'max_tokens' is not supported with this model

1 participant