feat: per-user per-model chat compaction threshold overrides by ibetitsmike · Pull Request #23412 · coder/coder

ibetitsmike · 2026-03-22T23:52:20Z

What

Adds per-user per-model auto-compaction threshold overrides. Users can now customize the percentage of context window usage that triggers chat compaction, independently for each enabled model.

Why

The compaction threshold was previously only configurable at the deployment level (chat_model_configs.compression_threshold). Different users have different preferences — some want aggressive compaction to keep costs low, others prefer higher thresholds to retain more context. This gives users control without requiring admin intervention.

Architecture

Storage: Reuses the existing user_configs table (no migration needed). Overrides are stored as key/value pairs with keys shaped chat_compaction_threshold:<modelConfigID> and integer percent values.

API: Three new experimental endpoints under /api/experimental/chats/config/:

GET /user-compaction-thresholds — list all overrides for the current user
PUT /user-compaction-thresholds/{modelConfig} — upsert an override (validates model exists and is enabled, validates 0–100 range)
DELETE /user-compaction-thresholds/{modelConfig} — clear an override (idempotent)

Runtime resolution: In coderd/chatd/chatd.go, a new resolveUserCompactionThreshold() helper runs at the start of each chat turn (inside runChat()), after the model config is resolved but before CompactionOptions is built. If a valid override exists, it replaces modelConfig.CompressionThreshold. The threshold source (user_override vs model_default) is logged with each compaction event.

Precedence: effectiveThreshold = userOverride ?? modelConfig.CompressionThreshold

UI: New "Context Compaction" subsection in the Agents → Settings → Behavior tab, placed after Personal Instructions. Shows one row per enabled model with the system default, a number input for the override, and Save/Reset controls.

Testing

9 API subtests covering CRUD, validation (boundary values 0/100, out-of-range rejection), upsert behavior, idempotent delete, user isolation, and non-existent model config
4 dbauthz tests (16 scenarios) verifying ActionReadPersonal / ActionUpdatePersonal on all query methods
4 Storybook stories with play functions (Default, WithOverrides, Loading, Error)

Implementation plan

Phase 1 — Tests

Backend API tests in coderd/chats_test.go (9 subtests)
Database auth wrapper tests in coderd/database/dbauthz/dbauthz_test.go (4 methods)
Frontend stories in UserCompactionThresholdSettings.stories.tsx (4 stories)

Phase 2 — Backend preference surface

4 SQL queries in coderd/database/queries/users.sql (list, get, upsert, delete)
make gen to propagate into generated artifacts
Auth/metrics wrappers in dbauthz and dbmetrics
SDK types and client methods in codersdk/chats.go
HTTP handlers and routes in coderd/chats.go and coderd/coderd.go
Key prefix constant shared between handlers and runtime

Phase 3 — Runtime override

resolveUserCompactionThreshold() helper in coderd/chatd/chatd.go
Override injection in runChat() before building CompactionOptions
threshold_source field added to compaction log

Phase 4 — Settings UI

API client methods and React Query hooks in site/src/api/
UserCompactionThresholdSettings component extracted from SettingsPageContent
Per-model mutation tracking (only the active row disables during save)
100% warning, "System default" label, helpful empty state copy

Phase 5 — Refactor and review fixes

Consolidated key prefix constant in codersdk
Explicit PUT range validation (not just struct tags)
GET handler gracefully skips malformed rows instead of 500
Boundary value, upsert, and non-existent model config tests
UX improvements: per-model mutation state, aria-live on errors

coder-tasks · 2026-03-22T23:57:25Z

Documentation Check

Updates Needed

docs/ai-coder/agents/architecture.md — The "Context compaction" section (line ~86) says the threshold is a fixed setting, but users can now override it per model. Add a note that the threshold can be customized per model in Agents → Settings → Behavior.
docs/ai-coder/agents/models.md — The Compression Threshold row in the general options table describes it as the percentage at which compaction triggers, but does not mention it is a system default that individual users can override. Update the description to reflect that users can set per-model overrides in their personal settings.

Automated review via Coder Tasks

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0c7017bce6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderd/chats.go

ibetitsmike · 2026-03-23T00:05:02Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3948c9e3e6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderd/chats.go

ibetitsmike · 2026-03-23T00:24:53Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fafde313b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderd/exp_chats.go

site/src/pages/AgentsPage/AgentSettingsPageView.tsx

ibetitsmike · 2026-03-23T08:33:58Z

@codex review

ibetitsmike · 2026-03-23T08:44:52Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fafde313b6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

site/src/pages/AgentsPage/UserCompactionThresholdSettings.tsx

site/src/pages/AgentsPage/AgentSettingsPageView.tsx

DanielleMaywood

Unsure on the best practices so converse with your agent, but it feels like I'm expecting to see a <form> somewhere but I'm not.

site/src/pages/AgentsPage/UserCompactionThresholdSettings.tsx

ibetitsmike · 2026-03-23T09:46:46Z

@codex review

ibetitsmike · 2026-03-23T10:02:09Z

@codex review

chatgpt-codex-connector · 2026-03-23T10:18:25Z

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ibetitsmike · 2026-03-23T10:23:24Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5db337a048

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

site/src/pages/AgentsPage/UserCompactionThresholdSettings.tsx

ibetitsmike · 2026-03-23T10:49:25Z

@codex review

chatgpt-codex-connector · 2026-03-23T11:08:25Z

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ibetitsmike · 2026-03-23T11:21:21Z

@codex review

chatgpt-codex-connector · 2026-03-23T11:32:45Z

Codex Review: Didn't find any major issues. More of your lovely PRs please.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

ibetitsmike · 2026-03-23T11:36:52Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c30d97c979

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderd/database/queries/users.sql

coderd/chats.go

coderd/exp_chats.go

coderd/chats_test.go

ibetitsmike · 2026-03-23T12:09:08Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2f3ef6f5bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

coderd/database/queries/users.sql

ibetitsmike · 2026-03-23T12:30:42Z

@codex review

…ompaction thresholds

…d endpoints

…ld handler

…server instance

…unction

…ebase

ibetitsmike · 2026-03-23T22:40:33Z

@codex review

chatgpt-codex-connector · 2026-03-23T22:56:21Z

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

feedback adressed

github-actions bot assigned ibetitsmike Mar 22, 2026

ibetitsmike marked this pull request as ready for review March 22, 2026 23:54

ibetitsmike changed the title ~~feat(coderd): per-user per-model chat compaction threshold overrides~~ feat: per-user per-model chat compaction threshold overrides Mar 22, 2026

chatgpt-codex-connector bot reviewed Mar 22, 2026

View reviewed changes

coderd/chats.go Outdated Show resolved Hide resolved

ibetitsmike force-pushed the mike/user-compaction-thresholds branch from 0c7017b to 3948c9e Compare March 23, 2026 00:04

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

coderd/chats.go Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

coderd/exp_chats.go Show resolved Hide resolved

site/src/pages/AgentsPage/AgentSettingsPageView.tsx Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

site/src/pages/AgentsPage/UserCompactionThresholdSettings.tsx Outdated Show resolved Hide resolved

site/src/pages/AgentsPage/AgentSettingsPageView.tsx Show resolved Hide resolved

DanielleMaywood reviewed Mar 23, 2026

View reviewed changes

site/src/pages/AgentsPage/UserCompactionThresholdSettings.tsx Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

site/src/pages/AgentsPage/UserCompactionThresholdSettings.tsx Outdated Show resolved Hide resolved

DanielleMaywood requested a review from johnstcn March 23, 2026 11:40

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

coderd/database/queries/users.sql Outdated Show resolved Hide resolved

johnstcn reviewed Mar 23, 2026

View reviewed changes

coderd/database/queries/users.sql Outdated Show resolved Hide resolved

coderd/chats.go Show resolved Hide resolved

coderd/chats.go Show resolved Hide resolved

coderd/exp_chats.go Show resolved Hide resolved

coderd/chats_test.go Outdated Show resolved Hide resolved

chatgpt-codex-connector bot reviewed Mar 23, 2026

View reviewed changes

coderd/database/queries/users.sql Outdated Show resolved Hide resolved

ibetitsmike added 21 commits March 23, 2026 22:39

fix(coderd): add PUT validation and graceful GET error handling for c…

bb34721

…ompaction thresholds

test(coderd): add boundary, upsert, and non-existent model config tests

ca8d3ff

fix(site): address review findings for compaction threshold settings

aa97918

docs: clarify PR title scope rules with CI enforcement note

db599da

fix: remove swagger annotations from experimental compaction threshol…

5a5a291

…d endpoints

fix: use system context for model config lookup in compaction thresho…

894db15

…ld handler

fix(site): move mutation handlers to useMutation definition

44a67aa

fix(site): use per-row mutation tracking and surface model-config errors

3d314d4

fix(site): track per-model pending state for concurrent saves

e6085d2

fix(site): fall back to model name when display_name is empty

f175a60

fix(site): show user compaction threshold override in chat UI

cbc0f2f

fix: address review feedback on compaction threshold backend

1a8c87c

refactor(coderd): consolidate compaction threshold tests into single …

e4a186e

…server instance

fix: remove invalid ESCAPE clause from compaction threshold query

d138605

fix: revert threshold clamping to skip behavior matching runtime

75bfefc

refactor(site): extract compression threshold resolution to utility f…

f51063f

…unction

fix: align compaction threshold with experimental API surface after r…

48b0f28

…ebase

fix(site): address review feedback on compaction threshold UI

e3a2c33

fix: revert feature-branch-only AGENTS.md additions

40396ee

fix: share chat compaction threshold keys

19b0aa5

fix: move compaction threshold SDK methods to ExperimentalClient

3b4a437

ibetitsmike force-pushed the mike/user-compaction-thresholds branch from 342059c to 3b4a437 Compare March 23, 2026 22:40

fix(site): only fetch model configs on behavior tab to fix flaky story

6683983

ibetitsmike requested a review from mafredri March 23, 2026 23:47

ibetitsmike merged commit 82f965a into main Mar 23, 2026
29 checks passed

ibetitsmike deleted the mike/user-compaction-thresholds branch March 23, 2026 23:48

github-actions bot locked and limited conversation to collaborators Mar 23, 2026

Conversation

ibetitsmike commented Mar 22, 2026

What

Why

Architecture

Testing

Phase 1 — Tests

Phase 2 — Backend preference surface

Phase 3 — Runtime override

Phase 4 — Settings UI

Phase 5 — Refactor and review fixes

Uh oh!

coder-tasks bot commented Mar 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Documentation Check

Updates Needed

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

DanielleMaywood left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot commented Mar 23, 2026

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot commented Mar 23, 2026

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot commented Mar 23, 2026

Uh oh!

ibetitsmike commented Mar 23, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

coder-tasks bot commented Mar 22, 2026 •

edited

Loading