Repository navigation
feat: use the resolved organization for AI Gateway budgets and cost (1/2) - #31120
Conversation
d3e6cfd to
e6405e8
Compare
e6405e8 to
9372bb8
Compare
9372bb8 to
776f1bc
Compare
776f1bc to
d9bd6c9
Compare
d9bd6c9 to
035aef5
Compare
AI budget resolution searches all of a user's groups across every organization. Whichever group wins determines the organization. #31120 needs to resolve inside a known organization, and it needs to know which organization site-wide resolution picked. Add an `organizationID uuid.NullUUID` argument to `budget.ResolveUserAIBudget` and `budget.ResolveUserEffectiveGroup`. When it is set, count only that organization's override, group budgets and Everyone group. An organization that is unknown, deleted or that the user is not a member of resolves to nothing. When it is null, resolve exactly as before. Return the winning group's organization in `budget.EffectiveGroup.OrganizationID`. Read the override through a new resolution-only query, `GetUserAIBudgetOverrideWithOrganization`, so that the CRUD query `GetUserAIBudgetOverride` and the admin endpoints stay untouched. Add a nullable `organization_id` filter to `GetHighestGroupAIBudgetByUser` and `GetUserEveryoneFallbackGroup`. Every caller passes null, so behaviour is unchanged. There is no migration and no proto change. Scoping to an organization can lift a limit: a user capped in org B has no limit in org A if A has no budgets. AIGOV-711 intends this, and it takes effect in #31120. Part of AIGOV-715. --- *This PR was generated by Coder Agents on behalf of @SasSwart.*
035aef5 to
56918ae
Compare
| // The organization comes from IsAuthorized. Empty means site-wide | ||
| // resolution, as sent by gateways older than v1.6. |
There was a problem hiding this comment.
I think this comment should be clearer about two things:
- An empty org ID is only expected from gateways older than v1.6. From v1.6,
IsAuthorizedshould always returns an org or rejects the request, so a >v1.6 gateway should never send an empty org. - We trust that the org ID is valid and that the user is a member of it. That's fine because the gateway sends back the org it got from
IsAuthorized, but the comment should say so.
ssncferreira
left a comment
There was a problem hiding this comment.
My main concern is backward compatibility 😕 We have to keep the empty-org path for gateways older than v1.6, and coderd can't tell those apart from a v1.6 gateway that sends no org. So we should make sure a request through a v1.6+ gateway can never go ahead without an org. Otherwise it quietly falls back to site-wide budget and cost, which is easy to miss.
The first step is for coderd's org resolution to return an error when the user has no org, instead of an empty result. Then IsAuthorized always returns an org or an error. As a safety net in case that ever changes, the v1.6 gateway should also reject the request when IsAuthorized returns no org. That's the only place the gateway gets the org from coderd, and everything after it reuses that org, so a single check covers the whole request. Older gateways don't have this check, so they keep working.
I think the tests matter most, because nothing else enforces this. They should check that a user with no org is rejected and that a v1.6 gateway never proceeds without one.
| func TestAuthorization_Delegated(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| // A valid delegated request resolves to the Everyone group of this |
There was a problem hiding this comment.
nit: What is a "delegated request"? Is it a chatd request, those can also include workspaces, right?
There was a problem hiding this comment.
Those can include workspaces, but in AIGOV-713 we're going to make them send the org that the chat belongs to. So we don't rely on the workspace for org resolution. I will fix this code comment, though.
…1/2) IsAuthorized resolves the request organization once: the workspace org for a workspace session token, otherwise the org of the user's site-wide effective group. It returns the org and its source. aibridged sends the org to IsBudgetExceeded, which resolves the budget inside it, and stamps the org and source on RecordInterception. Cost resolves the effective group inside the org stored on the interception. A null org keeps today's site-wide behaviour. The proto moves to v1.6: IsAuthorizedResponse.organization_source and IsBudgetExceededRequest.organization_id. The chat organization input is left for AIGOV-713. The org-aware price lookup follows in AIGOV-716b.
…n, in the org resolver
IsAuthorized now returns ErrNoOrganization when no organization resolves, instead of an empty result that fell back to site-wide budget and cost. The v1.6 gateway also rejects an authorization response without an organization, so a v1.6+ gateway never proceeds without one. Gateways older than v1.6 still send an empty organization to IsBudgetExceeded and keep working.
6196933 to
985edfa
Compare
#31121) Cost looks up model prices site-wide only, so an organization's own prices never apply, even though #31118 now stores the request organization on each interception. Pass the interception's stored organization to `GetAIModelPriceForProvider`. The lookup already falls back from the organization's price to the site-wide custom price, then to the price book. An interception without an organization (old rows, old gateways) uses only site-wide prices, as before. | Stored org | Org price | Site-wide custom | Price used | |---|---|---|---| | none | any | any | site-wide custom, then price book | | B | live | any | B's price | | B | soft-deleted or absent | present | site-wide custom | | B | absent | absent | price book, or NULL cost | Remove `TODO(AIGOV-716)`. Make `TestRecordTokenUsageAuthorized` store an organization and assert that its price wins under the strict authorizer. Part of AIGOV-716. Stacked on #31120. --- *This PR was generated by Coder Agents on behalf of @SasSwart.*
IsAuthorizedresolves the workspace organization, but uses it only to reject requests; nothing downstream reads it. The budget check and cost each resolve the group site-wide, so a user working in an org B workspace can be billed in org A, and no interception records its organization.Resolve the request organization once, in
IsAuthorized. A workspace session token resolves to the workspace's organization (sourceworkspace). Any other request, including chatd requests until AIGOV-713, resolves to the organization of the user's site-wide effective group (sourceeffective_group). Return the source inIsAuthorizedResponse.Have aibridged pass the organization to
IsBudgetExceeded, which resolves the budget inside it, and stamp the organization and source onRecordInterception. Resolve cost's effective group inside the organization stored on the interception. When there is no organization, behave site-wide as before.Move
resolveOrganizationand the workspace checks intoorgresolver.go, and leave a slot for the chat organization (AIGOV-713). Bump the aibridged API to v1.6, addingIsAuthorizedResponse.organization_sourceandIsBudgetExceededRequest.organization_id.aibridged checks only the shape of the organization (both fields or neither, and a valid non-nil UUID), not the source value. That lets a newer coderd add a source without breaking older gateways.
Behaviour changes to review:
IsAuthorized, instead of 500 fromIsBudgetExceeded.IsAuthorized.GET /api/v2/users/{user}/ai/spend) stays site-wide, so it can disagree with where a request was charged.The org-aware price lookup follows in #31121.
Part of AIGOV-716. Stacked on #31119.
This PR was generated by Coder Agents on behalf of @SasSwart.