Skip to content

feat(server): list experimental voice models - #57194

Merged
kitlangton merged 1 commit into
devfrom
voice-models
Oct 21, 2026
Merged

kitlangton merged 1 commit into
devfrom
voice-models

Conversation

@kitlangton

@kitlangton kitlangton commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Why

iOS and desktop clients need to discover which voice models the server can start (and which transports each supports) before initiating a call, rather than guessing a model and hitting a 503 Service Unavailable.

What

  • Extend Realtime.engines (packages/ai/src/experimental/realtime.ts) with each engine's default model (gpt-live-1, gpt-realtime-2.1, grok-voice-3.0, gemini-3.8-live) and supported transports, and validate connect against resolved.transports so the listing and session handshake cannot drift.
  • Add Voice.Service.models() (packages/core/src/voice.ts), sharing findConnection(provider) with Voice.Service.session(input) to check whether an API key or non-empty environment variable connection exists without resolving secret material or calling providers.
  • Expose GET /api/experimental/voice/models (experimental.voice.models) in @opencode/protocol, @opencode/server, @opencode/client, and OpenAPI specs.
  • Known limitation: Integration tracks one active connection per provider; voice picks the first stored API key or non-empty environment variable connection and skips OAuth/ChatGPT sign-ins.

Call-site example

const { models, default: defaultModel } = await client.voice.models()
// {
//   models: [
//     { model: "gpt-live-1", provider: "openai", transports: ["webrtc"], available: true },
//     { model: "gpt-realtime-2.1", provider: "openai", transports: ["webrtc", "websocket"], available: true },
//     { model: "grok-voice-3.0", provider: "xai", transports: ["websocket"], available: false },
//     { model: "gemini-3.8-live", provider: "google", transports: ["websocket"], available: false },
//   ],
//   default: "gpt-live-1",
// }

Verification

  • packages/ai: bun typecheck, bun test test/realtime.test.ts
  • packages/core: bun typecheck, bun run script/test.ts test/voice.test.ts
  • packages/server: bun typecheck, bun test test/voice.test.ts
  • packages/client: bun typecheck, bun test test/promise.test.ts, bun run check:generated
  • packages/protocol & services/www: bun run check:generated

Added by the opencode bot (AI-generated).

Public API: adds GET /api/experimental/voice/models and client.voice.models() (backwards compatible)

@kitlangton
kitlangton marked this pull request as ready for review October 21, 2026 06:13
@kitlangton
kitlangton merged commit 79fd806 into dev Oct 21, 2026
19 checks passed
@kitlangton
kitlangton deleted the voice-models branch October 21, 2026 06:13

@opencode-agent opencode-agent Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No blocking problems found. I ran opencode serve from this branch with XAI_API_KEY set and OPENAI_API_KEY set to an empty string. GET /api/experimental/voice/models listed all four models, with only grok-voice-3.0 available and set as the default. Asking voice/session for a transport the model doesn't support now returns a 400 from the shared transports check. The targeted tests and type checks pass in ai, core, server and client.

Public API (additive, backwards compatible):

  • Adds GET /api/experimental/voice/models (experimental.voice.models), with new OpenAPI schemas VoiceModel, VoiceTransportType and VoiceModelsResponse.
  • Adds the client method client.voice.models().
  • Adds Realtime.engines and the type RealtimeTransportType to @opencode/ai/experimental.
  • Changes the error message for a model/transport mismatch on voice/session, e.g. xAI Grok Voice only supports WebSocket transport → grok-voice-3.0 only supports websocket transport. It is still a 400 InvalidRequestError.

vleonbonnet added a commit to vleonbonnet/opencode that referenced this pull request Oct 21, 2026
Upstream lists experimental voice models (anomalyco#57194) and edits the Go FAQ (anomalyco#57663). No conflicts; the generated OpenAPI documents and client types match their sources.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant