Repository navigation
feat(schema): add streamUsage compatibility to turn off stream_options - #57255
Merged
Merged
Conversation
Contributor
Author
There was a problem hiding this comment.
Works as described. I ran opencode run from source against a local OpenAI-compatible server that records request bodies, with "compatibility": { "streamUsage": false } on the provider. On the base, the key is ignored and every request still sends stream_options: { include_usage: true }. With this PR, stream_options is left out and the run completes normally. The openai-chat, config/provider and model-resolver tests pass, and so does typecheck in ai, schema and core.
Public API impact:
- Added: optional
compatibility.streamUsage(boolean, defaults totrue) on models and providers in the config schema, in the OpenAPIModelCompatibility/ProviderCompatibilityand in the generated client types. Backwards compatible. - Renamed in the published
@opencode/aipackage:LanguageModelCompatibility.supportsUsageInStreaming→streamUsage. Only direct SDK callers that set the old field tofalseare affected: TypeScript flags it, and untyped callers would quietly start sendingstream_optionsagain. Existing configs are unaffected.
No blocking concerns.
Related: #31156
8 of 10 tasks
3 of 6 tasks
3 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #57230 (the
stream_optionshalf;storewas handled in #57239).Chat Completions always sends
stream_options: { include_usage: true }, and there was no way to turn it off from config. Strict OpenAI-style APIs that reject unknown fields (Sarashina in #57230, Databricks AI Gateway in #31156) fail with a 400.Changes:
streamUsageto the public model/providercompatibilityschema."compatibility": { "streamUsage": false }on a provider or model dropsstream_optionsfrom the request. Defaults totrue, so behavior is unchanged unless set.supportsUsageInStreamingtostreamUsageso config and the AI package share one name, and removes the always-true detector plus Cohere's redundanttrue.packages/protocol/openapi.json, and theservices/wwwOpenAPI docs. Only the twostreamUsagehunks are included; the generator also produced unrelated drift already ondev, which is left out.Example:
{ "providers": { "example": { "package": "@opencode/ai/providers/openai-compatible", "settings": { "baseURL": "https://api.example.com/v1" }, "compatibility": { "streamUsage": false } } } }Without usage chunks, token and cost numbers for that provider are only as good as what the API returns without
include_usage.Testing:
bun testinpackages/ai: all pass except the two Bedrock PDF cassettes, which fail only because of the host'sAWS_REGION; they pass with it unset. New test:streamUsage: falseomitsstream_options.bun run typecheckinpackages/aiandpackages/schema;tscforpackages/coresrc and tests.packages/coreconfig/provider, normalization, and model-resolver tests. New test: provider-levelcompatibility.streamUsage: falsereaches the resolved runtime model.Requested by: @rekram1-node (Aiden Cline via Slack)