feat(opencode): add copilot specific provider to properly handle copilot reasoning tokens#8900
Conversation
|
Hey! Your PR title Please update it to start with one of:
Where See CONTRIBUTING.md for details. |
|
The following comment was made by an LLM, it may be inaccurate: Related PRs FoundPR #5346: feat: reasoning text for Gemini 3 Pro in GH Copilot Why it's related: This PR is explicitly mentioned in the description as similar work. PR #8900 builds on the approach from #5346 but extends it to properly handle PR #5877: fix(github-copilot): auto-route GPT-5+ models to Responses API Why it's related: Deals with GitHub Copilot routing and API handling, which is in the same domain as the copilot-specific provider changes. |
|
Thanks for your contribution! This PR doesn't have a linked issue. All PRs must reference an existing issue. Please:
See CONTRIBUTING.md for details. |
|
Hello, I can confirm the reasoning works with Gemini 2.5 Pro however it doesn't seem to be the case with GPT 5.2 Codex / GPT 5.2 and Opus 4.5 |
|
@Coruscant11 thanks for trying! GPT-5 variants use the responses API version, which this PR does not affect. I need to check Opus - it's possible that reasoning needs to be explicitly enabled. |
Document findings from investigating Claude extended thinking via GitHub Copilot: - Responses API rejects Claude models entirely - Chat API accepts thinking params but ignores them - Gemini models DO return reasoning_text, Claude does not - Feature blocked on GitHub enabling it server-side Include test scripts used during investigation for future reference. See also: PR anomalyco#8900 which adds proper reasoning handling for Gemini. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
Here's the debugging I did so far -- lmk if there's something wrong with my methodology! I added basic logging for the opaque signatures 1) received by the API and 2) received by Then I ran a basic test conversation: The logs indicated that
The research I did earlier indicates that this because the AI SDK version currently being used (5.0.x?) doesn't automatically propagate Hope this helps @SteffenDE :) |
|
I also remember that @rekram1-node (not pinging rn) was quite knowledgeable on the Copilot API/AI SDK relationship so he might be able to give some insights |
|
@aadishv the place you log in convertToOpenAICompatibleChatMessages is wrong though, it’s just after initializing the variable so you’ll never see opaque not being undefined there. Try logging here https://github.com/aadishv/opencode/blob/5776493d4add243cf2bb155ed3989dac835328a9/packages/opencode/src/provider/sdk/copilot/chat/openai-compatible-chat-language-model.ts#L320, which is where I tested and did see it working. But as mentioned I’m only on my phone right now - will verify tomorrow :) I was also using ai-sdk v5 in another project from which I adapted the code and it‘s definitely passing the metadata back. |
|
@aadishv the reasoning_opaque is part of the previous assistant messages, not the user message. So your check would need to be
|
Source: @ai-sdk/openai-compatible@1.0.30 https://github.com/vercel/ai/tree/@ai-sdk/openai-compatible@1.0.30/packages/openai-compatible/src/chat Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Consolidate all copilot provider code under src/provider/sdk/copilot/: - Move responses/ folder from openai-compatible - Move provider setup files - Rename openai-compatible-provider.ts to copilot-provider.ts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Add type-only imports for verbatimModuleSyntax compliance - Update provider.ts to import from ./sdk/copilot - Update index.ts to reference renamed copilot-provider.ts Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add Copilot-specific cache control to all messages for prompt caching: - System messages: use content array format with cache control on each part - User messages: add cache control at message level - Assistant messages: add cache control at message level - Tool messages: add cache control at message level Also update OpenAICompatibleSystemMessage type to support content array format. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add Copilot-specific reasoning fields to response and streaming schemas: - reasoning_text: the actual reasoning text (like reasoning_content) - reasoning_opaque: opaque signature for multi-turn reasoning Update doGenerate and doStream to: - Extract reasoning from reasoning_text in addition to reasoning_content/reasoning - Include reasoning_opaque in providerMetadata for multi-turn context - Emit reasoning-end BEFORE text-start or tool-input-start when they arrive in the same chunk (critical for proper event ordering) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…er options - Change provider options key from openaiCompatible to copilot - Add reasoning_text and reasoning_opaque extraction from assistant messages - Extract reasoningOpaque from providerOptions.copilot on any message part - Use null instead of empty string for content when assistant has no text - Add Copilot-specific types to OpenAICompatibleMessage interfaces: - CopilotCacheControl type for cache control fields - reasoning_text and reasoning_opaque on assistant messages - Update tests for new implementation Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update copilot-provider.ts to import OpenAICompatibleChatLanguageModel from local chat folder instead of @ai-sdk/openai-compatible, enabling the Copilot-specific features: - copilot_cache_control on all messages - reasoning_text/reasoning_opaque schema support - Proper reasoning-end event ordering in streaming Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add validation to throw InvalidResponseDataError if multiple reasoning_opaque values are received in a single response, as only one thinking part per response is supported. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Remove reasoning_content and reasoning fields from schemas and extraction logic. Copilot only uses reasoning_text for thinking tokens, so we don't need the fallback chain. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
The new copilot provider looks for providerOptions.copilot, not providerOptions.openaiCompatible. This ensures that custom options are passed through. Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
5f712a1 to
82ca0a3
Compare
|
@SteffenDE Perhaps you can refer to this https://github.com/caozhiyuan/copilot-api/tree/all/src/routes/messages, which supports claude thinking & interleaved thinking (not native, by prompt), gemini thinking, and gpt responses API. The copilot models API will return model information, such as support for responses API / chat completions / message API (message API may be supported in the next version of claude or the next major version of vscode). |
|
I discovered through copilot-api that it's possible to bypass the max_prompt_tokens limit, allowing even Claude models to exceed 200k, though I'm not sure if this is a bug in Copilot. However, I haven't abused this. |
|
I keep hitting bugs when I try using this, lots of {"error":{"message":"","code":"not_found"}} |
|
okay I played w/ this a bit, I think it's good now |
|
seems need change "Openai-Intent" to "conversation-agent" and variants logic change to if (model.id.includes("claude")) { |
|
@rekram1-node can add a Flag.OPENCODE_EXPERIMENTAL_xxx (default false) for claude message api ? |
|
@caozhiyuan sure, but note that it does have a little bit to it, did you see I actually was using it in prod for a while but we ran into rate limit issues for users of it |
What's this for again? |
|
lol u know the api rlly well |
@rekram1-node for chat completions api , claude thinking need conversation-agent. |
|
@caozhiyuan I'm pretty sure the opencode client is actually blocked on copilots messages proxy now. It just 404's on messages now for me (unless I spoof being vscode, with a token obtained with the vscode client id). Could just be me but the fact that it works with the vscode auth makes me think its actually blocked And yeah this does need to handle claude vs. gemini |
|
working on a fix for all that |
Does this actually make a meaningful difference or is this a consistency thing I think copilot team was telling me this header may not be necessary but i forget |
@rekram1-node This morning I tried the dev branch code, and the Claude model wasn't outputting thinking information. After changing the |
|
It won't output thinking unless u use thinking variant tho, hit ctrl+t and youll see it, we should default to it on tho prolly |
|
@caozhiyuan I explicitly tested thinking with both intent header values and it did not make a difference |
|
cool, ill go over my pr one more time make sure everything still works and then merge it and should fix some small issues |
|
It's possible that Copilot has relaxed its restrictions. Previously, when thinking budget was first supported, only agent calls were supported. This morning, I installed the latest version of OpenCode via npm. Typing |
|
@rekram1-node It would be better if it could support different variations of the thinking budget. |
|
Yeah we cna do that im game |
…lot reasoning tokens (anomalyco#8900) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Aiden Cline <63023139+rekram1-node@users.noreply.github.com> Co-authored-by: Aiden Cline <aidenpcline@gmail.com>
…lot reasoning tokens (anomalyco#8900) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Aiden Cline <63023139+rekram1-node@users.noreply.github.com> Co-authored-by: Aiden Cline <aidenpcline@gmail.com>
…lot reasoning tokens (anomalyco#8900) Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com> Co-authored-by: Aiden Cline <63023139+rekram1-node@users.noreply.github.com> Co-authored-by: Aiden Cline <aidenpcline@gmail.com>






What does this PR do?
This PR adds a copilot specific provider for the copilot completions API (there's already copilot specific code for the responses API). The code already states that this code is only meant for copilot, so I decided to rename the folder accordingly for clarity. While it would be great to not need this, I currently don't see a way to not have the copilot specifics (apart from extracting it into a separate repository).
It is similar to #5346, but it changes the completions code to properly store the
reasoning_opaquefield and send it back to the copilot API.This PR is based on code I wrote for tidewave. I used Claude to implement the same changes based on a fresh copy of the upstream openai-compatible provider: https://github.com/vercel/ai/tree/%40ai-sdk/openai-compatible%401.0.30/packages/openai-compatible/src/chat
There are multiple small commits to make reviewing this easier and I also added tests for the important cases I encountered when handling the reasoning fields.
In the past, the Copilot API failed if the reasoning signature (
reasoning_opaque) was not sent back for the Gemini 3 models. At some point GitHub seems to have changed this. Note though that the models still behave differently if the reasoning tokens are not passed back. For example, in Tidewave we've often seen Copilot's Gemini 2.5 spiral into a loop when omitting the reasoning, while it doesn't do that when including those.How did you verify your code works?
You can chat with Gemini 2.5 Pro (3 Pro Preview is currently broken because of #8829, but it works in the Tidewave version of the code and all the fields are the same).
Closes #6864.