Repository navigation
Conversation
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 041b6717bb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
No |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8cd37e73ed
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
No |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review |
|
No |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b27474d408
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Can't wait for the next one! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
Readiness record for b27474d:
Generated with |
Turn on the AI Gateway Responses WebSocket endpoint for users with the user-scoped ai-gateway-responses-websocket experiment. G2a shipped the dormant plumbing; this wires it up. - aibridge: the OpenAI provider's bridged Responses route serves WebSocket upgrades when the request owner has the experiment on and the gateway attached a socket acquirer. The handler acquires a socket lease, frees the HTTP concurrency slot, dials upstream under the lease with isolated headers and centralized-key failover, accepts the client, relays text frames through responsesws.Session with frame limits and close-status mapping, and records every interception before releasing the lease. Every other route, provider, and user keeps the 501. - aibridge: passthrough routes refuse WebSocket upgrades with 501 for everyone instead of tunneling them with the centralized key and no record. - aibridged: attach the socket registry as the acquirer, mapping cap refusals to 429 and shutdown to 503, and bound socket closing to half of the shutdown deadline. - Tests: library acceptance tests against a fake upstream WebSocket, Server-level capacity, eviction, lifetime and shutdown tests, and embedded and standalone gateway tests.
- Record the end of open interceptions without waiting for the upstream close handshake. An upstream that does not answer the going-away close held recording for 5 seconds, past the socket share of a shutdown, so interceptions open at shutdown were never ended. - Record a tool result create only after the accountant recorded the tool call it answers, so coderd can link the interception to its parent. - Refuse a socket over the per-user or per-replica cap with 426, on which Codex falls back to HTTP, instead of 429, which it retries until it fails the turn. - Serve Responses WebSocket mode only for providers of type openai, not for OpenAI-compatible providers built on the same provider. - Refuse a client handshake that websocket.Accept would refuse before taking a lease or dialing upstream.
Remove a comment that restated a variable name and reflow an overlong shutdown comment. No behavior change. _Generated with xum. Model: anthropic:claude-opus-5-5. Thinking: high_ Signed-off-by: Thomas Kosiewski <tk@coder.com>
…and URL scheme Extend TestResponsesWebSocketUpstreamHeaders so the client's Sec-WebSocket-Protocol, Sec-WebSocket-Extensions, and Origin headers must not reach upstream; each is in the strip list but no test failed when it was removed. Add TestResponsesWebSocketURL for the https to wss mapping and query preservation, which integration tests against plain HTTP fake upstreams never reach.
…unted tool calls A create carrying function_call_output used to wait for every queued accounting job, with no time bound. It now waits only when its call ID was announced by a response on this session whose interception has not ended, and for at most one record timeout. The wait also ends with the caller, the session or the accountant. Interceptions release waiters when they end, are forgotten after a failed start, are handed off and ended on overload, or are swept at session end. --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…retry and actor headers Address Codex review findings on the Responses WebSocket handler: - A client that leaves before its upgrade now cancels the upstream handshake, so the lease is released at once instead of after the 30 s handshake timeout. - A refusal relays upstream's retry-after-ms hint as Retry-After in whole seconds, as the HTTP Responses path does. - Client handshake headers are stripped before actor headers are applied, so an actor header configured at one of those names still reaches upstream. --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…nt Firewall headers Address Codex review findings on the Responses WebSocket handler: - The concurrency slot is now released only after both the upstream and the client handshakes completed, so stalled upstream handshakes stay bounded by MaxConcurrency, as the ConcurrencySlotRelease contract says. - Agent Firewall correlation headers are validated before the lease is taken (400 on partial or malformed headers, as over HTTP) and recorded on every interception of the socket. --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
… its create is written Remote UAT found prompts lost under load: when upstream answered a create before Send marked the write done, the response opened its own interception without a prompt and the create ended without its response. - A create is marked in flight under the lock before its write. The lane's head create binds a response.created while written or in flight, since upstream answers only creates and creates on a lane are written one at a time. - A steer's continuation, recognized by a previous_response_id naming a response with a response.steer.accepted, never binds to a create. An ambiguous or unmarked response does not bind to a create still being written, which ends without a response as before. - A create whose write fails after binding a response ends with the write error once its queued jobs ran, so their records stay on it; later events of that response open their own interception. Usage is recorded exactly once. TestResponseDuringBlockedWrite now expects the binding, and the recording invariants model only protocol-valid upstream behavior. --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
b27474d to
834a98d
Compare
Serves OpenAI Responses WebSocket mode on the bridged
/<provider>/v1/responsesroute for users with the user-scopedai-gateway-responses-websocketexperiment. Builds on the dormant plumbing from #30245. Resolves AIGOV-698 items 1, 2 and 5, and delivers the gateway side of AIGOV-410.What changes
aibridge/contextgainsSocketLease/SocketAcquirer.coderd/aibridgedinstallsacquireSocket, adaptingSockets().Acquire, next to the admission callback.openai(newconfig.OpenAI.ResponsesWebSocket, set bybuildProvider), with the experiment on and an acquirer present. openai-compat, Azure, Google, OpenRouter, Vercel, Copilot and proxy mode keep today's 501 or 404.aibridge/responses_websocket.go), in order:OpenAI-Betais forwarded as Codex sends it. Centralized keys fail over on handshake errors the HTTP path fails over on.responsesws.Session(a binary client frame closes with 1003).Session.Close, then release the lease last.response.createdthat arrives while it is written or in flight. Before, a response that beatpublishopened its own prompt-less interception and the create ended without its response; remote UAT under load lost first-turn prompts this way. A steer's continuation, recognized by aprevious_response_idnaming a response with aresponse.steer.accepted, never binds to a create; ambiguous or unmarked responses do not bind to a create still being written. A write that fails after binding ends the create with the write error once its queued jobs ran, so usage is recorded exactly once. The residual case is documented onbindCreatedLocked.Plan deviations
TestResponsesWebSocketCapacityasserts 426.ConcurrencySlotReleasecontract states. The open socket still holds no slot.TestResponsesWebSocketHandshakeHoldsConcurrencySlotcovers it.TestResponseDuringBlockedWritenow expects the in-flight binding (with a comment on why the rule changed), the racedresponse.createdcase moved out ofTestRacedErrorKeepsCause, andTestRecordingInvariantsnow models only protocol-valid upstream behavior: upstream answers only creates, continuations follow an accepted steer, and a failed write breaks the connection.aibridged.Serverand bridge pool behindhttpmw.ConcurrencyLimit, not through a full coderd. Experiment on/off, recording, the compat 501 and the passthrough 501 run through the embedded/api/v2/ai-gatewayand the standalone gateway.Testing
go test -racepasses on./aibridge/...,./coderd/aibridged/...and./coderd/aibridge/..., plus the embedded and standalone gateway WebSocket tests. golangci-lint is clean.Follow-ups (not in this PR)
coder_ai_gateway_*interception, prompt, token and tool metrics. Both are tracked on AIGOV-698.supports_websockets = falseguidance stays until then.generate:false) shows as an empty 0/0 thread.📋 Implementation Plan
G2b: Responses WebSocket endpoint (dispatch, upstream dial, transport, acceptance tests)
Stack #30212: main <- X1 #30210 <- E1 #30211 <- G1 #30198 <- G1b #30218 <- G2a #30245 <- G2b.
Base branch:
tk/ai-gateway-responses-ws-plumbing(head d6c39cc). Resolves Linear AIGOV-698 (items listed below) and delivers AIGOV-410's gateway side.G2b turns the endpoint on for users with the user-scoped experiment
ai-gateway-responses-websocket(X1). With the experiment off, behavior is unchanged except for the passthrough fix in section 6, which applies to all users.1. Injection (from the G2a surface)
aibridge/context/context.go: addtype SocketLease interface{ Context() context.Context; Release() },type SocketAcquirer func(ctx context.Context, actorID, provider string) (SocketLease, error), plusWithSocketAcquirer/SocketAcquirerFromContext. Re-export inaibridge/api.golike the G2a admission helpers.coderd/aibridged/http.go:(s *Server) acquireSocket(...)adaptings.Sockets().Acquire(sockets.go:118). Return a nil interface on error (never a typed nil*SocketLease). MapErrSocketActorLimitandErrSocketReplicaLimitto a 429*intercept.ResponseErrorandErrShutdownto 503, since aibridge cannot import aibridged errors. Install it next to the admission callback (http.go:172).recthatNewRequestBridgealready passes to its handler closures (bridge.go:88). Never wrap it inAsyncRecorder(itsRecordInterceptionis a no-op).2. Dispatch (aibridge/bridge.go:153)
/<providerName>/v1/responses), the provider type is OpenAI (not Copilot, not Azure),agplaibridge.ResponsesWebSocketEnabled(ctx)is true, and a socket acquirer is on the context. Otherwise keep today's 501.3. Handler order and failure paths (frozen)
aibcontext.ActorFromContext) and provider name. Refusal: plain HTTP error before upgrade (429 or 503 as mapped), keep the HTTP concurrency slot until here.agplaibridge.ConcurrencySlotReleaseFromContext(ctx)()right after acquire succeeds (AIGOV-698 item 1).lease.Context()with a bounded handshake timeout:https://api.openai.com/v1/) withhttps->wss,http->ws, path joined withresponses. Preserve query parameters the HTTP path forwards.BuildUpstreamHeaders(headers/headers.go:230), then remove client handshake headers (Sec-WebSocket-Key,Sec-WebSocket-Version,Sec-WebSocket-Extensions,Sec-WebSocket-Protocolunless OpenAI requires a subprotocol), cookies, and any Coder auth headers; install only the resolved credential.Authorizationleft after Coder token stripping) is passed through with no failover. Centralized pool: choose a key per handshake attempt with the same walker and error marking as HTTP (intercept/responses/streaming.go:151-200,credential/credential.go:104), retrying only on handshake failures that HTTP would fail over on, bounded by the pool size and the handshake timeout.websocket.Accept, compression disabled unless both sides need it). If accept fails, close the upstream connection.SetReadLimit(responsesws.MaxClientFrameBytes)on the client andSetReadLimit(responsesws.MaxUpstreamFrameBytes)on upstream (G2a constants). A frame over the limit closes with 1009 (message too big).responsesws.NewSession(lease.Context(), ...)with Options from the HTTP path: Provider, Recorder (baserec), Admit (aibcontext.CreateAdmissionFromContext(ctx)), Logger, Client (aibclient.GuessClient(r)), ClientSessionID (aibclient.GuessSessionID), UserAgent, CredentialKind/Hint from the resolved credential. Carry the actor metadata into the lease context.SendandRecv->client, text messages only; a binary client message closes with 1003.httpapi.WSWatcher(coderd/httpapi/websocket.go:47-107) pings the client; only import it if aibridge can without a cycle, else an equivalent ping loop.StatusGoingAwayto both; internal error ->StatusInternalErrorwithhttpapi.WebsocketCloseSprintf-style text.Session.Close(records every open interception), thenlease.Release()last. Never reconnect or replay after the upstream handshake succeeded.4. Shutdown sub-deadline (AIGOV-698 item 2)
Server.Shutdowndrains sockets before canceling its lifecycle. Give socket closure a bounded share of the deadline (going-away close to every socket, then wait for releases), so session accounting still runs while the coderd DRPC connection is up and the rest of shutdown keeps its time. Test that shutdown with open sockets finishes within the deadline and every interception is ended.5. Metrics
G2a's socket gauge and refusal counter stay. Add an upstream handshake failure counter (by provider and status class) and a socket duration histogram only if they follow existing aibridge metric naming; regenerate
scripts/metricsdocgen/generated_metricsand the prometheus docs if metrics change.6. Passthrough WebSocket tunnels (AIGOV-698 item 5, all users)
Verified: passthrough routes (
aibridge/passthrough.go:28-79; OpenAI/conversations,/models,/responses/; Copilot every non-bridged route) tunnel WebSocket upgrades throughhttputil.ReverseProxy, with the operator's centralized pool key injected when the client has no BYOK key, and nothing is recorded. Docs say WebSocket is unsupported (docs/ai-coder/ai-gateway/clients/codex.md:147-171). Reject upgrades in the passthrough closure (passthrough.go:69, beforeproxy.ServeHTTP) with 501, for every provider and user. Tests: an upgrade on a passthrough route returns 501 and never reaches the fake upstream; ordinary passthrough HTTP is unchanged (extendTestPassthroughRoutes).7. Tests (activation gate; all in this PR)
response.created, deltas,response.completedwith usage,error, close codes).Sec-WebSocket-Key, no cookie); BYOK passthrough; centralized key failover on handshake 401/429 then success on the next key, and no failover after 101; upstream handshake failure -> plain HTTP error, client not upgraded; client accept failure closes upstream; oversized client and upstream frames -> 1009; binary client frame -> 1003; upstream close code propagated; max lifetime -> going away (fake clock)./api/v2/ai-gateway(enterprisestartEmbeddedAIGateway, aibridge_recording_test.go:133) and the standalone gateway: experiment on/off, recording visible, per-actor cap -> 429 before upgrade, an open socket holds a lease but no MaxConcurrency slot, bridge pool eviction does not close an open socket, bounded shutdown with open sockets, proxy mode opens no socket, passthrough upgrade -> 501.go test -raceon every touched package; existing tests unchanged.8. Docs
Out of scope here (G3 follows): the Codex client page and the rollout flag. Keep
supports_websockets = falseguidance until G3.Acceptance criteria
-race;make genandmake fmtclean; golangci-lint clean; no em/en dashes; full pre-push hook on push.supports_websockets = true, a Codex CLI session against the dev deployment's gateway completes several tool-using turns over WebSocket, and the Sessions page shows the interceptions with tokens and prompts; with the experiment off, Codex falls back to HTTP and still works. Capture the HTTP 101 and the records as evidence.Delivery
One PR (activation and its safety tests together), draft, base
tk/ai-gateway-responses-ws-plumbing. Expected roughly 1,500 to 2,000 lines including tests. If a real boundary appears, the only allowed split is: (a) a dormant transport adapter and dial logic with unit tests, nothing reachable, then (b) dispatch, lifecycle integration and the embedded/standalone acceptance tests. Do not split acceptance tests into a follow-up. Report to the coordinator before splitting.Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:high