Repository navigation
docs: disclose AI Gateway reverse proxy accounting limits - #29727
Conversation
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
fd534b6 to
a5d87a9
Compare
|
/coder-agents-review |
|
Chat: Review posted | View chat Review historydeep-review v0.13.0 | Round 2 | Last posted: Round 2, 17 findings (2 P2, 8 P3, 7 P4), COMMENT. Review Finding inventoryFindings
Contested and acknowledgedCRF-12 (P3, codersdk/deployment.go:2108) - All 21 completed CI jobs fail at head with no explanation
CRF-12 panel closure (R2, 1/1)
Law analysisNot run. Effective additions 36 (threshold 1000). Round logRound 1Round type: round 1, full panel (focused docs PR with one Go file). Panel: ging-go (trigger: .go changed), pen-botter (documentation), pariston, mafuuu (core), gon, leorio (floor), razor, mafu-san (agent-authored), knov (wildcard). Netero first pass: 2 P3, no P0. Cross-check: the actor-header disclosure (CRF-3) converged across 8 reviewers; Pen Botter, Mafuuu, Leorio, Razor, Knov reproduced it with scratch tests; orchestrator confirmed Downgrades: CRF-11 Gon P2 to P4. Keep-argument: the comment is exported to swagger x-enum-comments, so redundant and undefined wording ships in the API contract. Downgrade reason: no reader misreads behavior from the redundancy; the behavioral gap at that line is CRF-1. CRF-12 Mafu-san P2 to P3. Keep-argument: gen and lint-docs back the PR's consistency claim and are red. Downgrade reason: Netero ran clidocgen, the golden tests, and markdownlint locally and all passed, and the Reviewed against dfe3d87..9ce714f. Round 2Fix round (same 13 files, no renames or signature changes, effective additions +27). Churn guard PROCEED: CRF-1 to CRF-11 author fixed in d641f33, CRF-12 contested. Netero (advisory): CRF-19, CRF-20 new P4. Panel: mafu-san (dispute check-in, same agent), gon, leorio (floor; comments and doc text touched), pen-botter, mafuuu, pariston, razor, knov (raised CRF-3/4/6/7 whose text changed; same agents), ging-go (.go changed; same agent), zoro (wildcard, new agent). Cross-check: CRF-12 closed 1/1. CRF-4 re-raised by Pen Botter with a reproduction (Claude Platform BYOK request reached upstream with an empty Anthropic-Workspace-Id). CRF-11 re-raised by Gon ("Unsafe:" and restated identifier kept; new clause is CRF-20). Leorio CRF-21 verified by orchestrator: About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
This PR adds operator warnings and option-text disclosures for the ai-gateway-reverse-proxy experiment; 2 P2, 6 P3, 3 P4. The budget, restart, Bedrock routing, and lifecycle-only recording claims match the code at head; the gaps are what the warnings leave out.
Out of scope (needs a ticket or explicit acceptance by a human):
aibridge/x/proxy/handler.go:100: forwarding does not delete client values at configured actor-header destinations, so with the experiment on, an upstream proxy that trusts those headers receives a client-chosen identity.aibridge/x/proxy/handler.go:153: proxy mode forwards a bridged request even whenRecordInterceptionfails, while interception mode returns 500 (aibridge/bridge.go:240-243), so a recorder outage produces unrecorded provider traffic.aibridge/x/proxy/handler.go:57: the forwarding handler never installs the Claude Platform transport that setsAnthropic-Workspace-Id, so Claude Platform BYOK or pooled-key requests are likely sent without it (unverified against AWS).aibridge/x/proxy/router.go:70: Bedrock bridged routes answer with Go's bare404 page not found, which gives the client no cause to act on.aibridge/passthrough.go:82: passthrough routes forward clientX-AI-Bridge-Actor*headers in both modes, so "Client headers starting with X-AI-Bridge-Actor are always removed" is false for those routes.
🤖 This review was automatically generated with Coder Agents.
|
/coder-agents-review |
There was a problem hiding this comment.
Round 2 of the reverse proxy disclosure review: 1 P2, 1 P3, and 5 P4 open. The fixes for the MCP condition, the experiment name, the actor-header forwarding disclosure, per-process mode selection, best-effort recording, route terminology, spend scope, and the duplicated budget sentence match the code at 0687c9e331. The CI finding is closed: run 36835726001 at the round 1 head succeeded, and the red jobs came from the cancelled run 36835725519. The Claude Platform provider bullet is reopened, and the experiment comment cleanup is incomplete.
Out of scope (needs a ticket or explicit acceptance by a human):
docs/ai-coder/ai-gateway/reference.md:86: the actor-header forwarding section says AI Gateway uses account values, not client-supplied headers, which is false in reverse proxy mode for custom destination names, andsetup.md:125sends operators there without any link to the reverse proxy warning.aibridge/x/proxy/handler.go:100: forwarding still does not delete client values at configured actor-header destinations (unchanged from round 1).aibridge/x/proxy/handler.go:57: the forwarding handler still does not apply the Claude Platform transport, so intercepted requests carry noAnthropic-Workspace-Id(unchanged from round 1).
🤖 This review was automatically generated with Coder Agents.
| "Client headers starting with X-AI-Bridge-Actor are always removed.", | ||
| "Client headers starting with X-AI-Bridge-Actor are always removed. " + | ||
| "When the ai-gateway-reverse-proxy experiment selects reverse proxy mode, this option has no effect on intercepted routes: " + | ||
| "AI Gateway adds no actor headers and forwards client-supplied values at configured actor-header destinations unchanged, " + |
There was a problem hiding this comment.
P4 [CRF-19] "forwards client-supplied values at configured actor-header destinations unchanged" is false for the default destinations. (Netero)
The defaults are X-AI-Bridge-Actor-ID and X-AI-Bridge-Actor-Metadata-Username (codersdk/deployment.go:2078, :2088), and PrepareClientHeaders deletes every X-AI-Bridge-Actor* header (aibridge/headers/headers.go:215-219). Only custom names outside that prefix pass through, which is what monitoring.md already says. Use "custom actor-header destinations that don't start with X-AI-Bridge-Actor".
🤖
| ExperimentWorkspaceCapableLicensing Experiment = "workspace-capable-licensing" // Counts only users holding the workspace-create permission toward the license seat limit. | ||
| ExperimentAIGatewaySeatExclusion Experiment = "ai-gateway-seat-exclusion" // Excludes AI Gateway (AI Bridge) usage from AI Governance seat consumption. | ||
| ExperimentAIGatewayReverseProxy Experiment = "ai-gateway-reverse-proxy" // Uses stateless reverse proxy routing when MCP injection is not configured. | ||
| ExperimentAIGatewayReverseProxy Experiment = "ai-gateway-reverse-proxy" // Unsafe: when no MCP servers are configured, uses reverse proxy routing that records no token usage or spend for intercepted routes, so budgets cannot limit them. |
There was a problem hiding this comment.
P4 [CRF-20] "so budgets cannot limit them" contradicts the budget check that still runs in reverse proxy mode. (Netero)
coderd/aibridged/http.go:165-179 calls IsBudgetExceeded and returns 403 before routing, so a user already over budget from other traffic is still refused; cost-controls.md says this correctly. This text ships in swagger x-enum-comments. Say these requests add no spend toward budgets, or drop the clause.
🤖
| > Reverse proxy mode also changes how intercepted routes behave: | ||
| > | ||
| > - Requests to Amazon Bedrock providers return `404 page not found`. | ||
| > - Requests to Anthropic providers that would authenticate with AWS credentials, such as Claude Platform for AWS or the `CODER_AI_GATEWAY_BEDROCK_*` settings, return `501 Not Implemented`. |
There was a problem hiding this comment.
P3 [CRF-21] The 501 bullet names "the CODER_AI_GATEWAY_BEDROCK_* settings" as an AWS credential source, but those settings no longer configure any provider. (Leorio)
providers.md:22-23 says leftover provider environment variables are ignored and lists the CODER_AI_GATEWAY_BEDROCK_* variables for removal, and cli/aibridged.go builds the anthropic provider with a nil Bedrock config; no non-test Go file references AI_GATEWAY_BEDROCK. An operator with those variables still set reads the bullet as saying they are active. Name only Claude Platform for AWS here; the round 1 comment on this bullet suggested the removed settings, which was wrong.
🤖
| - AI Gateway [enabled and configured](./setup.md) with at least one provider. | ||
|
|
||
| > [!WARNING] | ||
| > The unsafe `ai-gateway-reverse-proxy` experiment disables usage accounting for [requests to intercepted routes](./reference.md#supported-apis). |
There was a problem hiding this comment.
P4 [CRF-22] "disables usage accounting" is undefined, and the same warning then lists what is still recorded and checked. (Gon)
Lines 26 and 29 say start and end are still recorded and budget checks still include other spend, so a reader of line 22 cannot tell what stops. Name it, matching monitoring.md:15: "doesn't record token usage or accrue spend for requests to intercepted routes".
🤖
| > [!WARNING] | ||
| > The unsafe `ai-gateway-reverse-proxy` experiment switches an AI Gateway process to reverse proxy mode when no MCP servers are configured. | ||
| > [AI Governance cost control](./cost-controls.md) explains when the experiment takes effect, how to confirm the mode, and how to turn it off. | ||
| > In reverse proxy mode, AI Gateway makes a best-effort record of only when each request to an intercepted route starts and ends. |
There was a problem hiding this comment.
P4 [CRF-23] The recording limits are written out twice (cost-controls.md:26-28 and monitoring.md:15-17), and the enforcement instruction is copied from cost-controls.md:30 to spend-management.md:14. (Zoro)
The code says model and token accounting are "not yet supported" (coderd/aibridged/aibridged.go:267), so this text is expected to change, and each change needs the same edit in two or three callouts. Keep the full description in cost-controls.md and keep only the page-specific consequence on the other two pages, with the existing link.
🤖
0687c9e to
5f41412
Compare
322862c to
fc5048e
Compare
5f41412 to
69ebfc0
Compare
fc5048e to
7c37e31
Compare
6a95d01 to
e1fe6a1
Compare
7c37e31 to
e8e5a1f
Compare
998377f to
fdb903b
Compare
79d997a to
609c5f7
Compare
fdb903b to
c6f166c
Compare
6a99493 to
1ecb969
Compare
1e82141 to
f646a8e
Compare
1ecb969 to
cc8820d
Compare
f646a8e to
e7a7633
Compare
1807aff to
7f45457
Compare
e7a7633 to
e2dae58
Compare
7f45457 to
2c38889
Compare
00195a8 to
cabb446
Compare
1aa60d0 to
6df3d59
Compare
3d1ec95 to
a071803
Compare
8673318 to
8dfba0e
Compare
bfb19c7 to
e907262
Compare
8dfba0e to
8b0c48b
Compare
e907262 to
e580bd7
Compare
7b69d45 to
cbcb84b
Compare
e580bd7 to
4892f12
Compare
Generated by Coder Agents.
…ecycle-and-key-9' into pawel/aigov-615-reverse-proxy-handler-with-recorder-lifecycle-and-key-10
Document the unsafe reverse proxy experiment's missing token, model, content, and spend accounting, along with budget/backfill limitations, unsupported Bedrock routes, and actor-header restrictions. Explain that start recording must succeed before forwarding and end recording is asynchronous and best effort. Keep CLI help, configuration references, and generated API descriptions consistent.
Depends on #29726, which activates forwarding. This PR contains only disclosures, option descriptions, and generated outputs. The restack also corrects the AWS-authentication disclosure and distinguishes custom actor-header names from stripped actor-prefixed headers.
Scope and implementation decisions
Disclosures follow the synchronous-start lifecycle in #29726. Usage and spend are not backfilled; blank model or usage values are not evidence of zero provider spend. Turning off the experiment requires restarting every affected control plane replica and standalone gateway. No forwarding, recorder-schema, or identity changes belong to this slice.
The prerequisite stack is shared body capture (#30398) → provider wire forwarding and Router-owned transports (#30399) → existing key-pool failover (#30400) → terminal response observation (#30226) → lifecycle recording and activation (#29726). Validation and the passthrough nil-body sibling (#30401) are unchanged.
Generated by Coder Agents on behalf of @pawbana.