Repository navigation
feat(coderd): price token usage with organization prices - #30719
Conversation
135b4d4 to
0958a4f
Compare
0958a4f to
cc7f0f8
Compare
cc7f0f8 to
198ffd3
Compare
| // to the model reported by the provider, and compute cost. Organization | ||
| // prices need the organization the request was made in, which | ||
| // interceptions do not carry yet, so only site-wide prices match. | ||
| // TODO: pass the organization once interceptions carry it. |
There was a problem hiding this comment.
nit: link to the relevant issue, I believe AIGOV-716
198ffd3 to
3df39da
Compare
c71041f to
d370e05
Compare
Adds `ai_model_configs`: an organization's AI Governance configuration for one model on one configured provider. Token prices are its first settings. Migration `000630` is additive and needs no backfill. - Rows are soft deleted with a `deleted` flag. A surrogate `id` is the primary key, and a partial unique index allows one live row per `(organization_id, provider_id, model)`. Cascades on delete of the organization or provider. The organization is a column, as in other organization-scoped tables. - A check constraint rejects an empty model. - Pricing per provider ID is what lets `openai-compat` gateways be priced, and lets providers of the same type have different prices. - Site-wide prices per provider type stay in the existing `ai_model_prices` table, unchanged. Recording which price was used on each token usage record is deferred until its design settles. **Used later:** #30718 adds the queries that read and write the table, and #30719 looks prices up from it. The rest of the diff is generated. --- *This PR was generated by Coder Agents on behalf of @SasSwart.*
d370e05 to
4fc368b
Compare
4fc368b to
ae98f91
Compare
…#31116) Adds @ssncferreira and @SasSwart as code owners of `ai_providers.sql`, `ai_provider_keys.sql` and `aicostcontrol.sql`, alongside @ibetitsmike and @johnstcn. The `codersdk/aiproviders*.go` rules are unchanged. Both current owners are on PTO, which leaves the organization AI model price stack (#30718, #30719) without anyone who can give the required code owner review on `aicostcontrol.sql`. > Generated by Coder Agents on behalf of @SasSwart.
ssncferreira
left a comment
There was a problem hiding this comment.
LGTM, but I think the query is not falling back to the site prices if all prices are set to NULL
| WHERE ai_model_configs.organization_id = sqlc.narg('organization_id')::uuid | ||
| AND ai_model_configs.provider_id = @provider_id::uuid | ||
| AND ai_model_configs.model IN (@model::text, @provider_model::text) | ||
| AND ai_model_configs.deleted = FALSE |
There was a problem hiding this comment.
I think this query isn't quite right yet 🤔 There will be cases where an ai_model_configs row exists without any prices, for example, once we backfill it from the Agents models. In that case, it should fall back to the site-wide pricing, but as it stands this branch would pick the row, and its NULL prices would be counted as zero, so the model would be recorded as free.
We should also add a test for this case.
| -- most specific first: the organization's price for the provider, a site-wide | ||
| -- custom price, then the price book. The requested model is preferred over the |
There was a problem hiding this comment.
nit: it might be easier to read if this is written as a numbered list (same as before)
| func aiModelPriceSeed(t *testing.T, keys map[string]string, inputPrice int64) json.RawMessage { | ||
| t.Helper() | ||
| elem := map[string]any{ | ||
| "input_price": inputPrice, |
There was a problem hiding this comment.
Since we treat a price layer as a whole and never fall back to individual prices, it would be good to add that in a test. The helpers only set input_price, so org and site-wide rows always have NULL for the other prices, and a per-price fallback to the catalog would still pass. Could we add a case where the site-wide row sets all four prices and check that the org row's NULL prices stay NULL?
Adds the store queries for organization AI model prices in `ai_model_configs`, with authorization. They cover basic CRUD on one price, keyed by `(organization_id, provider_id, model)`. - `GetOrganizationAIProviderModelPrice` reads one price. - `UpsertOrganizationAIProviderModelPrice` sets one price, with typed params. It rewrites an existing row only when a price differs, so `updated_at` records the last change. - `DeleteOrganizationAIProviderModelPrice` soft deletes one price. Reads and upserts only match live rows, so setting a price after a delete inserts a new live row. All three authorize against `ResourceAiModelPrice.InOrg(org)`. The new `AiModelPriceInOrg` case in `TestRolePermissions` pins that only owners and org admins pass. A TODO in dbauthz notes that this should become a resource for AI Governance model config. Each query has its own test. The upsert tests cover that identical prices leave `updated_at` unchanged, that changed prices advance it, and that an empty model is rejected. **Used later:** #30719 seeds its lookup tests with the upsert. #30720 serves all three through the org price API. --- *This PR was generated by Coder Agents on behalf of @SasSwart.*
ae98f91 to
a192961
Compare
|
Rebased onto
The rebase squashed the earlier branch commits into 05ded5b, since they targeted tables that no longer exist. Comment generated by Coder Agents on behalf of @SasSwart. |
Prices token usage by the most specific match: the organization's price for the configured provider, then a site-wide custom price, then the price book. The requested model is still preferred over the model the provider reports.
GetAIModelPriceForProviderreplacesGetAIModelPriceByProviderModel, which is removed. It is oneUNION ALLof primary-key lookups, ordered by requested model first, then by specificity. dbauthz checksResourceAiModelPrice, scoped to the org when one is given.coderd/aibridge/pricestests read seeded prices throughGetAIModelPrices.Interceptions don't carry an organization yet, so
resolveTokenUsageCostpasses none and only site-wide prices apply at runtime for now (AIGOV-716). Org matching is covered by database tests with explicit org IDs. For the same reason, the unpriced-models report still counts only site-wide prices.Used later: prices set through the API in #30720 take effect here once interceptions carry an organization.
This PR was generated by Coder Agents on behalf of @SasSwart.