Repository navigation
Conversation
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d79281d1be
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
… bounds Adds the server-side contracts the Responses WebSocket endpoint will use. Nothing becomes reachable: WebSocket upgrades still get 501. - Per-create admission: every authorized gateway request carries an aibridge.CreateAdmissionFunc. Each response.create counts against the same rate limiter and key as the HTTP request (exposed by httpmw.RateLimitByAuthToken as a context value) and rechecks the AI budget. Refusals are OpenAI-shaped errors: 429 rate_limit_exceeded, 403 for an exceeded budget, 500 when the check fails. - Socket ownership: aibridged.Server owns a SocketRegistry with per-actor (16) and per-replica (512) caps that count connecting sockets, a 60 minute maximum lifetime, and shutdown that ends every lease. Lease contexts are detached from the per-user bridge, so bridge eviction does not end sockets. Adds open and refusal metrics. - MaxConcurrency no longer applies to WebSocket upgrades to /v1/responses, since a socket would hold a slot for its lifetime; the socket caps bound them instead. - responsesws: frame size limits, a byte bound with backpressure on frames read ahead of Recv, and accounting of frames over extract.MaxEventBytes without copying them.
…Concurrency
The exemption cleaned the path before matching, so an upgrade to
/v1/responses/ or /v1/responses/{id}, a passthrough route the reverse
proxy tunnels upstream, skipped the concurrency limit for the tunnel's
lifetime. Match the clean escaped path exactly instead.
Signed-off-by: Thomas Kosiewski <tk@coder.com>
…ockets first Upgrades to /v1/responses skipped MaxConcurrency before authentication, so upgrades with fake tokens could pile up handlers waiting on coderd. Drop the path exemption: every request holds its slot as before, and ConcurrencyLimit now carries an aibridge.ConcurrencySlotRelease that a Responses WebSocket handler calls once its socket holds a lease, so only the open socket leaves the limit and the socket caps bound it. Server.Shutdown now ends the sockets before it cancels the lifecycle, which closes the coderd connection that a closing socket's last records need.
d6c39cc to
7b846eb
Compare
Adds the server-side contracts the AI Gateway Responses WebSocket endpoint (next layer) will use. Nothing becomes reachable: WebSocket upgrades to the gateway still return 501.
Part of AIGOV-698 and AIGOV-410. Stacked on #30218.
Changes
aibridge.CreateAdmissionFunc(context helpers inaibridge/context;responsesws.AdmitFuncis now an alias). Eachresponse.createcounts one request against the same rate limiter instance and key as HTTP (httpmw.RateLimitByAuthTokennow exposes aRateLimitConsumeron the request context; HTTP responses are unchanged), then re-checks the AI budget over DRPC. Refusals are*intercept.ResponseErrors, so the client gets an error event and keeps its socket: 429rate_limit_exceeded, 403 with the HTTP budget message (type and codeinsufficient_quota), and 500 with theErrBudgetCheckmessage when the check fails (fail closed).aibridged.Serverowns aSocketRegistry(Server.Sockets()):Acquire(ctx, actorID, provider)returns a lease whose context keeps the request's values but not its cancellation, so per-user bridge eviction (pool TTL, provider reloads) does not end sockets. Caps are constants (16 per actor, 512 per replica) and count sockets fromAcquire, so connecting sockets count. A lease ends after 60 minutes (OpenAI's own WebSocket limit) withErrSocketLifetime;Server.Shutdownfirst refuses new leases, cancels all withErrShutdown, and waits up to its deadline for releases, and only then cancels its lifecycle, so closing sockets can still record through the coderd connection. Detached sockets do not re-authorize: they are bounded by the lifetime, and every create runs admission. New metricsresponses_websockets_openandresponses_websockets_refusals_total.httpmw.ConcurrencyLimitholds a slot for the handler's lifetime, which for a socket would be up to 60 minutes. Admitted requests now carry anagplaibridge.ConcurrencySlotRelease(released once; later calls and the release on return do nothing). The next layer calls it afterSocketRegistry.Acquiresucceeds, so the handshake, authorization and budget check stay bounded byMaxConcurrency, and only the open socket moves to the socket caps. No request skips the limit in this PR.responseswssession bounds.MaxClientFrameBytes = routing.MaxRequestBodyBytesandMaxUpstreamFrameBytes = 16 MiB. Frames read ahead ofRecvare bounded by 8 MiB as well as 16 frames, with backpressure on the upstream reader. Accounting jobs for frames overextract.MaxEventBytesare queued without the frame bytes, so they cannot trigger the queue overload error. The package doc states the worst-case memory, and theaccountLoopcomment now explains thepublishextraction.Why detaching the lease context is enough
RequestBridge.Shutdownonly cancels its in-flight request contexts and shuts down its MCP proxies. The recorder, providers, key pools and logger it holds stay usable after shutdown, so a session that keeps them only needs a context that bridge shutdown does not cancel. No refcount on the bridge is needed.Validation
go test -raceon the touched packages, golangci-lint,make fmt,make gen, and the full pre-push hook on push./api/v2/ai-gateway/openai/v1/responsesstill 501 with theai-gateway-responses-websocketexperiment on and off. An earlier path-based exemption was replaced by the early release above after UAT and review findings.Generated with
xum• Model:anthropic:claude-opus-5-5• Thinking:high