Skip to content

feat: serve OpenAI Responses WebSocket mode behind its experiment - #30260

Closed
ThomasK33 wants to merge 8 commits into
tk/ai-gateway-responses-ws-plumbingfrom
tk/ai-gateway-responses-ws-endpoint
Closed

ThomasK33 wants to merge 8 commits into
tk/ai-gateway-responses-ws-plumbingfrom
tk/ai-gateway-responses-ws-endpoint

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Oct 1, 2026 •

Copy link
Copy Markdown
Member

Serves OpenAI Responses WebSocket mode on the bridged /<provider>/v1/responses route for users with the user-scoped ai-gateway-responses-websocket experiment. Builds on the dormant plumbing from #30245. Resolves AIGOV-698 items 1, 2 and 5, and delivers the gateway side of AIGOV-410.

What changes

  • Injection: aibridge/context gains SocketLease / SocketAcquirer. coderd/aibridged installs acquireSocket, adapting Sockets().Acquire, next to the admission callback.
  • Dispatch: a WebSocket upgrade is served only on the exact bridged Responses route of a provider of type openai (new config.OpenAI.ResponsesWebSocket, set by buildProvider), with the experiment on and an acquirer present. openai-compat, Azure, Google, OpenRouter, Vercel, Copilot and proxy mode keep today's 501 or 404.
  • Handler (aibridge/responses_websocket.go), in order:
    1. Validate the client handshake and the Agent Firewall headers (400 on partial or malformed headers, as over HTTP).
    2. Acquire a lease.
    3. Dial upstream under the lease context with isolated headers: only the resolved credential is sent, with no Coder auth, cookies or client handshake headers. OpenAI-Beta is forwarded as Codex sends it. Centralized keys fail over on handshake errors the HTTP path fails over on.
    4. Accept the client, release the HTTP concurrency slot, apply the G2a frame limits (1009 when exceeded), and pump text frames through responsesws.Session (a binary client frame closes with 1003).
    5. Map close codes. Lease end from max lifetime or shutdown closes with 1001.
    6. Tear down with Session.Close, then release the lease last.
  • Recording: every interception of a socket carries the socket's Agent Firewall session ID and sequence number, like an HTTP interception.
  • Shutdown: socket closing gets a bounded share of the shutdown deadline. The upstream reader runs on its own goroutine, so open interceptions are recorded without waiting for an unresponsive peer's close frame.
  • Passthrough: WebSocket upgrades on passthrough routes now return 501 for every provider and user. Before this change they were tunneled with the operator's key and nothing was recorded.
  • Prompt attribution (G2a session change): a create is marked in flight under the lock before its upstream write, and the lane's head create binds a response.created that arrives while it is written or in flight. Before, a response that beat publish opened its own prompt-less interception and the create ended without its response; remote UAT under load lost first-turn prompts this way. A steer's continuation, recognized by a previous_response_id naming a response with a response.steer.accepted, never binds to a create; ambiguous or unmarked responses do not bind to a create still being written. A write that fails after binding ends the create with the write error once its queued jobs ran, so usage is recorded exactly once. The residual case is documented on bindCreatedLocked.
  • Lineage: a create answering a tool call waits until the response that announced that call ID is accounted, so its parent link is found. Only call IDs this session is still accounting for wait, at most one record timeout, and the wait ends with the caller, session or accountant.

Plan deviations

  • Socket cap refusals return 426 Upgrade Required, not 429. This applies to the per-user and per-replica caps. Codex CLI falls back to HTTPS at once on 426, but gives up on 429 ("exceeded retry limit") and fails the turn. A local Codex CLI 0.159.3 run against a fake upstream confirmed both. Shutdown keeps 503: Codex retries it and then falls back, which gives a load balancer time to move the client to a healthy replica. TestResponsesWebSocketCapacity asserts 426.
  • The concurrency slot is released after both handshakes, not right after the lease. Stalled upstream handshakes therefore stay bounded by MaxConcurrency, as the ConcurrencySlotRelease contract states. The open socket still holds no slot. TestResponsesWebSocketHandshakeHoldsConcurrencySlot covers it.
  • G2a tests changed. The plan said existing tests stay unchanged. TestResponseDuringBlockedWrite now expects the in-flight binding (with a comment on why the rule changed), the raced response.created case moved out of TestRacedErrorKeepsCause, and TestRecordingInvariants now models only protocol-valid upstream behavior: upstream answers only creates, continuations follow an accepted steer, and a failed write breaks the connection.
  • No new metrics. The plan made them optional. The G2a socket gauge and refusal counter stay.
  • Where the cap tests run. The per-actor cap, bridge eviction, lifetime and shutdown tests run on the real aibridged.Server and bridge pool behind httpmw.ConcurrencyLimit, not through a full coderd. Experiment on/off, recording, the compat 501 and the passthrough 501 run through the embedded /api/v2/ai-gateway and the standalone gateway.

Testing

  • Automated: go test -race passes on ./aibridge/..., ./coderd/aibridged/... and ./coderd/aibridge/..., plus the embedded and standalone gateway WebSocket tests. golangci-lint is clean.
  • Remote dogfood UAT: run with Codex CLI through Coder Agents on an isolated develop.sh deployment with a local fake upstream. It passed in round 2: tool-using turns over WebSocket (HTTP 101, no fallback), Sessions showing tokens and prompts, HTTPS fallback with the experiment off, passthrough 501, 426 at the cap, and interceptions open at shutdown ending. Round 4 found the prompt-loss race fixed here; round 5 on the final head passed: 275 first turns with immediate follow-ups across 5 runs under heavy load, all with their prompts and 0 prompt-less roots (the previous head lost 5 of 110 in the same harness), plus Codex WebSocket turns and the 426, 501 and fallback regressions.

Follow-ups (not in this PR)

  • WebSocket creates do not consult or update circuit breakers yet, and are missing from the coder_ai_gateway_* interception, prompt, token and tool metrics. Both are tracked on AIGOV-698.
  • The Codex client docs page and rollout guidance (G3); supports_websockets = false guidance stays until then.
  • Codex's warmup create (generate:false) shows as an empty 0/0 thread.
  • Already true before this PR: auth errors return 400/403 rather than 401, and the 501 body is plain text.

📋 Implementation Plan

G2b: Responses WebSocket endpoint (dispatch, upstream dial, transport, acceptance tests)

Stack #30212: main <- X1 #30210 <- E1 #30211 <- G1 #30198 <- G1b #30218 <- G2a #30245 <- G2b.
Base branch: tk/ai-gateway-responses-ws-plumbing (head d6c39cc). Resolves Linear AIGOV-698 (items listed below) and delivers AIGOV-410's gateway side.

G2b turns the endpoint on for users with the user-scoped experiment ai-gateway-responses-websocket (X1). With the experiment off, behavior is unchanged except for the passthrough fix in section 6, which applies to all users.

1. Injection (from the G2a surface)

  • aibridge/context/context.go: add type SocketLease interface{ Context() context.Context; Release() }, type SocketAcquirer func(ctx context.Context, actorID, provider string) (SocketLease, error), plus WithSocketAcquirer / SocketAcquirerFromContext. Re-export in aibridge/api.go like the G2a admission helpers.
  • coderd/aibridged/http.go: (s *Server) acquireSocket(...) adapting s.Sockets().Acquire (sockets.go:118). Return a nil interface on error (never a typed nil *SocketLease). Map ErrSocketActorLimit and ErrSocketReplicaLimit to a 429 *intercept.ResponseError and ErrShutdown to 503, since aibridge cannot import aibridged errors. Install it next to the admission callback (http.go:172).
  • The session recorder is the synchronous base rec that NewRequestBridge already passes to its handler closures (bridge.go:88). Never wrap it in AsyncRecorder (its RecordInterception is a no-op).

2. Dispatch (aibridge/bridge.go:153)

  • Serve the WebSocket handler only when all hold: the request is a WebSocket upgrade, the route is exactly the provider's bridged Responses route (/<providerName>/v1/responses), the provider type is OpenAI (not Copilot, not Azure), agplaibridge.ResponsesWebSocketEnabled(ctx) is true, and a socket acquirer is on the context. Otherwise keep today's 501.
  • Proxy mode (aibridge/x/proxy/router.go) is not wired: add a test that an upgrade there does not open a socket (today 404).
  • Never pass MCP configuration or tools to the session, for any user.

3. Handler order and failure paths (frozen)

  1. Acquire the lease with the verified actor ID (aibcontext.ActorFromContext) and provider name. Refusal: plain HTTP error before upgrade (429 or 503 as mapped), keep the HTTP concurrency slot until here.
  2. Release the HTTP concurrency slot: agplaibridge.ConcurrencySlotReleaseFromContext(ctx)() right after acquire succeeds (AIGOV-698 item 1).
  3. Dial upstream under lease.Context() with a bounded handshake timeout:
    • URL: the provider's base URL (default https://api.openai.com/v1/) with https->wss, http->ws, path joined with responses. Preserve query parameters the HTTP path forwards.
    • Headers: start from BuildUpstreamHeaders (headers/headers.go:230), then remove client handshake headers (Sec-WebSocket-Key, Sec-WebSocket-Version, Sec-WebSocket-Extensions, Sec-WebSocket-Protocol unless OpenAI requires a subprotocol), cookies, and any Coder auth headers; install only the resolved credential.
    • Credentials: BYOK (Authorization left after Coder token stripping) is passed through with no failover. Centralized pool: choose a key per handshake attempt with the same walker and error marking as HTTP (intercept/responses/streaming.go:151-200, credential/credential.go:104), retrying only on handshake failures that HTTP would fail over on, bounded by the pool size and the handshake timeout.
    • If every attempt fails, return a plain HTTP error to the client mapped like the HTTP path (upstream 401/403/429/5xx, or 502 for network errors). The client was never upgraded.
  4. Accept the client (websocket.Accept, compression disabled unless both sides need it). If accept fails, close the upstream connection.
  5. SetReadLimit(responsesws.MaxClientFrameBytes) on the client and SetReadLimit(responsesws.MaxUpstreamFrameBytes) on upstream (G2a constants). A frame over the limit closes with 1009 (message too big).
  6. responsesws.NewSession(lease.Context(), ...) with Options from the HTTP path: Provider, Recorder (base rec), Admit (aibcontext.CreateAdmissionFromContext(ctx)), Logger, Client (aibclient.GuessClient(r)), ClientSessionID (aibclient.GuessSessionID), UserAgent, CredentialKind/Hint from the resolved credential. Carry the actor metadata into the lease context.
  7. Pumps: client->Send and Recv->client, text messages only; a binary client message closes with 1003. httpapi.WSWatcher (coderd/httpapi/websocket.go:47-107) pings the client; only import it if aibridge can without a cycle, else an equivalent ping loop.
  8. Close mapping: client close -> close upstream with the same code; upstream close -> close client with the same code and reason (truncated to the close-reason limit); lease context end with the max-lifetime or shutdown cause -> StatusGoingAway to both; internal error -> StatusInternalError with httpapi.WebsocketCloseSprintf-style text.
  9. Teardown: Session.Close (records every open interception), then lease.Release() last. Never reconnect or replay after the upstream handshake succeeded.

4. Shutdown sub-deadline (AIGOV-698 item 2)

Server.Shutdown drains sockets before canceling its lifecycle. Give socket closure a bounded share of the deadline (going-away close to every socket, then wait for releases), so session accounting still runs while the coderd DRPC connection is up and the rest of shutdown keeps its time. Test that shutdown with open sockets finishes within the deadline and every interception is ended.

5. Metrics

G2a's socket gauge and refusal counter stay. Add an upstream handshake failure counter (by provider and status class) and a socket duration histogram only if they follow existing aibridge metric naming; regenerate scripts/metricsdocgen/generated_metrics and the prometheus docs if metrics change.

6. Passthrough WebSocket tunnels (AIGOV-698 item 5, all users)

Verified: passthrough routes (aibridge/passthrough.go:28-79; OpenAI /conversations, /models, /responses/; Copilot every non-bridged route) tunnel WebSocket upgrades through httputil.ReverseProxy, with the operator's centralized pool key injected when the client has no BYOK key, and nothing is recorded. Docs say WebSocket is unsupported (docs/ai-coder/ai-gateway/clients/codex.md:147-171). Reject upgrades in the passthrough closure (passthrough.go:69, before proxy.ServeHTTP) with 501, for every provider and user. Tests: an upgrade on a passthrough route returns 501 and never reaches the fake upstream; ordinary passthrough HTTP is unchanged (extend TestPassthroughRoutes).

7. Tests (activation gate; all in this PR)

  • A fake upstream OpenAI Responses WebSocket server (httptest + coder/websocket) that records handshake headers and scripts server events (response.created, deltas, response.completed with usage, error, close codes).
  • Library level (aibridge/internal/integrationtest): experiment off -> 501; on -> create/created/completed relayed byte for byte with interception, prompt and token records; unknown client events forwarded unchanged; per-create admission refusal -> error event, socket stays open; MCP-configured user gets no tool injection (forwarded create identical); header isolation (upstream sees only the resolved credential, no Coder token, no client Sec-WebSocket-Key, no cookie); BYOK passthrough; centralized key failover on handshake 401/429 then success on the next key, and no failover after 101; upstream handshake failure -> plain HTTP error, client not upgraded; client accept failure closes upstream; oversized client and upstream frames -> 1009; binary client frame -> 1003; upstream close code propagated; max lifetime -> going away (fake clock).
  • Through /api/v2/ai-gateway (enterprise startEmbeddedAIGateway, aibridge_recording_test.go:133) and the standalone gateway: experiment on/off, recording visible, per-actor cap -> 429 before upgrade, an open socket holds a lease but no MaxConcurrency slot, bridge pool eviction does not close an open socket, bounded shutdown with open sockets, proxy mode opens no socket, passthrough upgrade -> 501.
  • go test -race on every touched package; existing tests unchanged.

8. Docs

Out of scope here (G3 follows): the Codex client page and the rollout flag. Keep supports_websockets = false guidance until G3.

Acceptance criteria

  1. Every test in section 7 passes with -race; make gen and make fmt clean; golangci-lint clean; no em/en dashes; full pre-push hook on push.
  2. Experiment off: no behavior change on bridged routes (still 501 for upgrades); passthrough upgrades now 501 for everyone.
  3. Codex CLI dogfood (remote UAT): with the experiment on for the UAT user and supports_websockets = true, a Codex CLI session against the dev deployment's gateway completes several tool-using turns over WebSocket, and the Sessions page shows the interceptions with tokens and prompts; with the experiment off, Codex falls back to HTTP and still works. Capture the HTTP 101 and the records as evidence.

Delivery

One PR (activation and its safety tests together), draft, base tk/ai-gateway-responses-ws-plumbing. Expected roughly 1,500 to 2,000 lines including tests. If a real boundary appears, the only allowed split is: (a) a dormant transport adapter and dial logic with unit tests, nothing reachable, then (b) dispatch, lifecycle integration and the embedded/standalone acceptance tests. Do not split acceptance tests into a follow-up. Report to the coordinator before splitting.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@linear-code

linear-code Bot commented Oct 1, 2026

Copy link
Copy Markdown

AIGOV-698

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-01T20:27:25.368532Z b27474d Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@ThomasK33
ThomasK33 added this pull request to stack #30212 October 1, 2026 17:51

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 041b6717bb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/responses_websocket.go Resolved
Comment thread aibridge/responses_websocket.go Resolved
Comment thread aibridge/responses_websocket.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

No pull_request ci run started within 2 minutes of pushing 8cd37e7, so I dispatched ci manually: run https://github.com/coder/coder/actions/runs/36905789626 (workflow_dispatch, headSha 8cd37e7).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8cd37e73ed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/responses_websocket.go Resolved
Comment thread aibridge/bridge.go Resolved
Comment thread aibridge/bridge.go Resolved
Comment thread aibridge/bridge.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

No pull_request ci run started within 2 minutes of pushing d68283e, so I dispatched ci manually: run https://github.com/coder/coder/actions/runs/36910370555 (workflow_dispatch, headSha d68283e).

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: d68283eb03

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@ThomasK33

Copy link
Copy Markdown
Member Author

No pull_request ci run started within 2 minutes of pushing b27474d, so I dispatched ci manually: run https://github.com/coder/coder/actions/runs/36920262686 (workflow_dispatch, headSha b27474d).

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b27474d408

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread aibridge/responses_websocket.go Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Can't wait for the next one!

Reviewed commit: b27474d408

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33

Copy link
Copy Markdown
Member Author

Readiness record for b27474d:

  • CI: ci run https://github.com/coder/coder/actions/runs/36920262686 is green (26/26 checks). One rerun was needed for test-go-race-pg, which hit a DATA RACE in cli/ssh.go (rawSSHCopier) against tailnet configmaps. This PR does not touch either file.
  • Codex: 5 rounds; round 5 is clean on this head. Two round-2 findings are deferred to Linear AIGOV-698: circuit breakers and interception metrics for WebSocket creates.
  • Remote dogfood UAT (Codex CLI through Coder Agents, isolated dev deployment): round 5 passed on this head with 275 first turns with immediate follow-ups and 0 prompt-less roots. Codex WebSocket tool turns, the 426 cap, the 501 when the experiment is off, the HTTPS fallback and the passthrough 501 all behaved as expected.
  • Local: go test -race passes on the touched packages, golangci-lint is clean, and the full pre-push hook ran on every push.
  • Independent recommendation: ready, with follow-ups tracked on AIGOV-698. Those follow-ups also list the paths only unit tests cover (steer continuation, write failure after bind) and the unmeasured shutdown share.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Turn on the AI Gateway Responses WebSocket endpoint for users with the
user-scoped ai-gateway-responses-websocket experiment. G2a shipped the
dormant plumbing; this wires it up.

- aibridge: the OpenAI provider's bridged Responses route serves
  WebSocket upgrades when the request owner has the experiment on and
  the gateway attached a socket acquirer. The handler acquires a socket
  lease, frees the HTTP concurrency slot, dials upstream under the lease
  with isolated headers and centralized-key failover, accepts the client,
  relays text frames through responsesws.Session with frame limits and
  close-status mapping, and records every interception before releasing
  the lease. Every other route, provider, and user keeps the 501.
- aibridge: passthrough routes refuse WebSocket upgrades with 501 for
  everyone instead of tunneling them with the centralized key and no
  record.
- aibridged: attach the socket registry as the acquirer, mapping cap
  refusals to 429 and shutdown to 503, and bound socket closing to half
  of the shutdown deadline.
- Tests: library acceptance tests against a fake upstream WebSocket,
  Server-level capacity, eviction, lifetime and shutdown tests, and
  embedded and standalone gateway tests.
- Record the end of open interceptions without waiting for the upstream
  close handshake. An upstream that does not answer the going-away close
  held recording for 5 seconds, past the socket share of a shutdown, so
  interceptions open at shutdown were never ended.
- Record a tool result create only after the accountant recorded the tool
  call it answers, so coderd can link the interception to its parent.
- Refuse a socket over the per-user or per-replica cap with 426, on which
  Codex falls back to HTTP, instead of 429, which it retries until it
  fails the turn.
- Serve Responses WebSocket mode only for providers of type openai, not
  for OpenAI-compatible providers built on the same provider.
- Refuse a client handshake that websocket.Accept would refuse before
  taking a lease or dialing upstream.
Remove a comment that restated a variable name and reflow an overlong
shutdown comment. No behavior change.

_Generated with xum. Model: anthropic:claude-opus-5-5. Thinking: high_

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…and URL scheme

Extend TestResponsesWebSocketUpstreamHeaders so the client's
Sec-WebSocket-Protocol, Sec-WebSocket-Extensions, and Origin headers must
not reach upstream; each is in the strip list but no test failed when it
was removed. Add TestResponsesWebSocketURL for the https to wss mapping
and query preservation, which integration tests against plain HTTP fake
upstreams never reach.
…unted tool calls

A create carrying function_call_output used to wait for every queued
accounting job, with no time bound. It now waits only when its call ID
was announced by a response on this session whose interception has not
ended, and for at most one record timeout. The wait also ends with the
caller, the session or the accountant. Interceptions release waiters
when they end, are forgotten after a failed start, are handed off and
ended on overload, or are swept at session end.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…retry and actor headers

Address Codex review findings on the Responses WebSocket handler:

- A client that leaves before its upgrade now cancels the upstream
  handshake, so the lease is released at once instead of after the
  30 s handshake timeout.
- A refusal relays upstream's retry-after-ms hint as Retry-After in whole
  seconds, as the HTTP Responses path does.
- Client handshake headers are stripped before actor headers are applied,
  so an actor header configured at one of those names still reaches
  upstream.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…nt Firewall headers

Address Codex review findings on the Responses WebSocket handler:

- The concurrency slot is now released only after both the upstream and
  the client handshakes completed, so stalled upstream handshakes stay
  bounded by MaxConcurrency, as the ConcurrencySlotRelease contract
  says.
- Agent Firewall correlation headers are validated before the lease is
  taken (400 on partial or malformed headers, as over HTTP) and recorded
  on every interception of the socket.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
… its create is written

Remote UAT found prompts lost under load: when upstream answered a create
before Send marked the write done, the response opened its own
interception without a prompt and the create ended without its response.

- A create is marked in flight under the lock before its write. The
  lane's head create binds a response.created while written or in
  flight, since upstream answers only creates and creates on a lane are
  written one at a time.
- A steer's continuation, recognized by a previous_response_id naming a
  response with a response.steer.accepted, never binds to a create. An
  ambiguous or unmarked response does not bind to a create still being
  written, which ends without a response as before.
- A create whose write fails after binding a response ends with the
  write error once its queued jobs ran, so their records stay on it;
  later events of that response open their own interception. Usage is
  recorded exactly once.

TestResponseDuringBlockedWrite now expects the binding, and the
recording invariants model only protocol-valid upstream behavior.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33
ThomasK33 force-pushed the tk/ai-gateway-responses-ws-endpoint branch from b27474d to 834a98d Compare October 2, 2026 08:16
@github-actions github-actions Bot added the stale This issue is like stale bread. label Nov 10, 2026
@github-actions github-actions Bot closed this Nov 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

stale This issue is like stale bread.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant