Skip to content

feat: use the resolved organization for AI Gateway budgets and cost (1/2) - #31120

Merged
SasSwart merged 4 commits into
mainfrom
sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests
Oct 30, 2026
Merged

SasSwart merged 4 commits into
mainfrom
sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests

Conversation

@SasSwart

@SasSwart SasSwart commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

IsAuthorized resolves the workspace organization, but uses it only to reject requests; nothing downstream reads it. The budget check and cost each resolve the group site-wide, so a user working in an org B workspace can be billed in org A, and no interception records its organization.

Resolve the request organization once, in IsAuthorized. A workspace session token resolves to the workspace's organization (source workspace). Any other request, including chatd requests until AIGOV-713, resolves to the organization of the user's site-wide effective group (source effective_group). Return the source in IsAuthorizedResponse.

Have aibridged pass the organization to IsBudgetExceeded, which resolves the budget inside it, and stamp the organization and source on RecordInterception. Resolve cost's effective group inside the organization stored on the interception. When there is no organization, behave site-wide as before.

Move resolveOrganization and the workspace checks into orgresolver.go, and leave a slot for the chat organization (AIGOV-713). Bump the aibridged API to v1.6, adding IsAuthorizedResponse.organization_source and IsBudgetExceededRequest.organization_id.

aibridged checks only the shape of the organization (both fields or neither, and a valid non-nil UUID), not the source value. That lets a newer coderd add a source without breaking older gateways.

Behaviour changes to review:

  • Spend follows the workspace's organization. A user whose budgets are all in org A has no limit in an org B workspace if B has none.
  • A database error while resolving the organization now returns 403 from IsAuthorized, instead of 500 from IsBudgetExceeded.
  • Requests without a workspace make up to three extra budget queries in IsAuthorized.
  • The user spend status endpoint (GET /api/v2/users/{user}/ai/spend) stays site-wide, so it can disagree with where a request was charged.

The org-aware price lookup follows in #31121.

Part of AIGOV-716. Stacked on #31119.


This PR was generated by Coder Agents on behalf of @SasSwart.

@linear-code

linear-code Bot commented Oct 21, 2026

Copy link
Copy Markdown

AIGOV-716

@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from d3e6cfd to e6405e8 Compare October 26, 2026 10:38
@SasSwart
SasSwart added this pull request to stack #31264 October 26, 2026 12:00
@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from e6405e8 to 9372bb8 Compare October 28, 2026 08:35

@SasSwart SasSwart left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Self Review

Comment thread coderd/aibridged/proto/version.go Resolved
Comment thread coderd/aibridged/http.go Resolved
Comment thread coderd/aibridged/http.go Resolved
@SasSwart
SasSwart requested a review from ssncferreira October 28, 2026 08:45
@SasSwart
SasSwart marked this pull request as ready for review October 28, 2026 08:45
@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from 9372bb8 to 776f1bc Compare October 28, 2026 08:52
@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from 776f1bc to d9bd6c9 Compare October 28, 2026 08:59
@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from d9bd6c9 to 035aef5 Compare October 28, 2026 13:25
Base automatically changed from sas/aigov-715-limit-effective-group-resolution-to-an-organization to main October 29, 2026 10:14
SasSwart added a commit that referenced this pull request Oct 29, 2026
AI budget resolution searches all of a user's groups across every
organization. Whichever group wins determines the organization. #31120
needs to resolve inside a known organization, and it needs to know which
organization site-wide resolution picked.

Add an `organizationID uuid.NullUUID` argument to
`budget.ResolveUserAIBudget` and `budget.ResolveUserEffectiveGroup`.
When it is set, count only that organization's override, group budgets
and Everyone group. An organization that is unknown, deleted or that the
user is not a member of resolves to nothing. When it is null, resolve
exactly as before. Return the winning group's organization in
`budget.EffectiveGroup.OrganizationID`.

Read the override through a new resolution-only query,
`GetUserAIBudgetOverrideWithOrganization`, so that the CRUD query
`GetUserAIBudgetOverride` and the admin endpoints stay untouched. Add a
nullable `organization_id` filter to `GetHighestGroupAIBudgetByUser` and
`GetUserEveryoneFallbackGroup`.

Every caller passes null, so behaviour is unchanged. There is no
migration and no proto change.

Scoping to an organization can lift a limit: a user capped in org B has
no limit in org A if A has no budgets. AIGOV-711 intends this, and it
takes effect in #31120.

Part of AIGOV-715.

---

*This PR was generated by Coder Agents on behalf of @SasSwart.*
@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from 035aef5 to 56918ae Compare October 29, 2026 10:14
@mtojek
mtojek requested review from mtojek and removed request for johnstcn October 29, 2026 10:53
Comment thread coderd/aibridge/factory.go Resolved
Comment thread coderd/aibridged/http.go Resolved
Comment thread coderd/aibridgedserver/orgresolver.go Resolved
Comment thread coderd/aibridgedserver/orgresolver.go Resolved
@SasSwart
SasSwart requested a review from mtojek October 29, 2026 14:25

@mtojek mtojek left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

Comment thread coderd/aibridgedserver/orgresolver.go Resolved
Comment on lines +987 to +988
// The organization comes from IsAuthorized. Empty means site-wide
// resolution, as sent by gateways older than v1.6.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this comment should be clearer about two things:

  • An empty org ID is only expected from gateways older than v1.6. From v1.6, IsAuthorized should always returns an org or rejects the request, so a >v1.6 gateway should never send an empty org.
  • We trust that the org ID is valid and that the user is a member of it. That's fine because the gateway sends back the org it got from IsAuthorized, but the comment should say so.

@ssncferreira ssncferreira left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My main concern is backward compatibility 😕 We have to keep the empty-org path for gateways older than v1.6, and coderd can't tell those apart from a v1.6 gateway that sends no org. So we should make sure a request through a v1.6+ gateway can never go ahead without an org. Otherwise it quietly falls back to site-wide budget and cost, which is easy to miss.

The first step is for coderd's org resolution to return an error when the user has no org, instead of an empty result. Then IsAuthorized always returns an org or an error. As a safety net in case that ever changes, the v1.6 gateway should also reject the request when IsAuthorized returns no org. That's the only place the gateway gets the org from coderd, and everything after it reuses that org, so a single check covers the whole request. Older gateways don't have this check, so they keep working.

I think the tests matter most, because nothing else enforces this. They should check that a user with no org is rejected and that a v1.6 gateway never proceeds without one.

Comment thread coderd/aibridged/proto/aibridged.proto Resolved
Comment thread coderd/aibridged/http.go Resolved
func TestAuthorization_Delegated(t *testing.T) {
t.Parallel()

// A valid delegated request resolves to the Everyone group of this

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: What is a "delegated request"? Is it a chatd request, those can also include workspaces, right?

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Those can include workspaces, but in AIGOV-713 we're going to make them send the org that the chat belongs to. So we don't rely on the workspace for org resolution. I will fix this code comment, though.

Comment thread coderd/aibridgedserver/orgresolver.go Resolved
Comment thread coderd/aibridgedserver/orgresolver.go Resolved
…1/2)

IsAuthorized resolves the request organization once: the workspace org
for a workspace session token, otherwise the org of the user's site-wide
effective group. It returns the org and its source. aibridged sends the
org to IsBudgetExceeded, which resolves the budget inside it, and stamps
the org and source on RecordInterception. Cost resolves the effective
group inside the org stored on the interception. A null org keeps
today's site-wide behaviour.

The proto moves to v1.6: IsAuthorizedResponse.organization_source and
IsBudgetExceededRequest.organization_id. The chat organization input is
left for AIGOV-713. The org-aware price lookup follows in AIGOV-716b.
IsAuthorized now returns ErrNoOrganization when no organization resolves,
instead of an empty result that fell back to site-wide budget and cost. The
v1.6 gateway also rejects an authorization response without an organization,
so a v1.6+ gateway never proceeds without one. Gateways older than v1.6 still
send an empty organization to IsBudgetExceeded and keep working.
@SasSwart
SasSwart force-pushed the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch from 6196933 to 985edfa Compare October 30, 2026 11:19
@SasSwart
SasSwart merged commit 15365c6 into main Oct 30, 2026
27 checks passed
@SasSwart
SasSwart deleted the sas/aigov-716-use-the-resolved-organization-throughout-ai-gateway-requests branch October 30, 2026 11:35
SasSwart added a commit that referenced this pull request Oct 30, 2026
#31121)

Cost looks up model prices site-wide only, so an organization's own
prices never apply, even though #31118 now stores the request
organization on each interception.

Pass the interception's stored organization to
`GetAIModelPriceForProvider`. The lookup already falls back from the
organization's price to the site-wide custom price, then to the price
book. An interception without an organization (old rows, old gateways)
uses only site-wide prices, as before.

| Stored org | Org price | Site-wide custom | Price used |
|---|---|---|---|
| none | any | any | site-wide custom, then price book |
| B | live | any | B's price |
| B | soft-deleted or absent | present | site-wide custom |
| B | absent | absent | price book, or NULL cost |

Remove `TODO(AIGOV-716)`. Make `TestRecordTokenUsageAuthorized` store an
organization and assert that its price wins under the strict authorizer.

Part of AIGOV-716. Stacked on #31120.

---

*This PR was generated by Coder Agents on behalf of @SasSwart.*
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants