Skip to content

fix: run queued messages on the chat's model when theirs is unavailable - #31632

Merged
mafredri merged 7 commits into
mainfrom
mathias/chat-queued-model-fallback
Nov 12, 2026
Merged

mafredri merged 7 commits into
mainfrom
mathias/chat-queued-model-fallback

Conversation

@mafredri

@mafredri mafredri commented Nov 2, 2026 •

Copy link
Copy Markdown
Member

A queued message whose stored model is no longer available (model disabled, provider disabled, or model deleted) ran on the organization default, even when the chat was on another available model. The chat's last model then moved to the default, so the picker changed after the turn.

  • Promotion (resolveQueuedMessageModelConfigID) now tries the queued row's model, then the chat's last model, then the organization default. This is the order SendMessage already uses for a message sent without a model.
  • Saving a queued edit with the picker untouched now sends the picker's model when the row's model is no longer selectable (isUnavailableHistoricalModelID, the rule history edits use). An available row still keeps its model unless the user picks one during the edit.

A = organization default, B = the unavailable model, C = the available model the chat last ran on.

Case Before After
Queued B row promoted at turn end, chat on C runs on A, chat moves to A runs on C
Queued B row promoted, chat on B runs on A runs on A
Queued B row saved with the picker untouched (picker shows C) model_config_id omitted, row keeps B, runs on A sends C, runs on C
Queued available row saved with the picker untouched omitted, row keeps its model unchanged

EditMessage without a model uses a separate resolver (resolveFallbackModelConfigID with the edited message's model, chat_mutations.go) that still skips the chat's last model. This PR leaves it as is; when the edited message's model is unavailable, the UI already sends the picker's model.

coderd/x/chatd/ARCHITECTURE.md describes the new promotion order in "Organization-local model selection and fallback".

Tests:

  • TestPromoteQueuedMessageResolvesOrganizationModel/disabled_model_falls_back_to_chat_model (chatstate). Before the fix it gets the default instead of the chat's model.
  • AgentChatPage.test.tsx "saving a queued edit sends the picker's model when the row's model is no longer available", added to the existing queued-save table. Before the fix it gets model_config_id: undefined.

Refs CODAGT-971

🤖 This PR was created with the help of Coder Agents on behalf of @mafredri, and will be reviewed by a human. 🏂🏻

@linear-code

linear-code Bot commented Nov 2, 2026

Copy link
Copy Markdown

CODAGT-971

@mafredri

This comment has been minimized.

@coder-agents-review

coder-agents-review Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-11-02 16:11 UTC by @mafredri
Spend: $10.39 / $100.00

Review history
  • R1 (2026-11-02): 13 reviewers, 3 Nit, 2 P2, 1 P3, 1 P4, COMMENT. Review

deep-review v0.13.0 | Round 1 | 3d0fd60..49c87f4

Last posted: Round 1, 7 findings (2 P2, 1 P3, 1 P4, 3 Nit), COMMENT. Review

Finding inventory

Finding inventory: PR #31632

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 P2 Open transitions_test.go:190 "falls back to local default" subtests (and chatd queued-promotion tests) now return via chat model; default branch untested R1 Netero, Bisky Yes
CRF-2 P2 Open transitions.go:358 Promotion checks candidates with daemon auth, so an ACL-removed row model is kept, overwrites last model, and generation runs on the default R1 Pariston Yes
CRF-3 P3 Open transitions.go:350 Doc comment ties the enabled/org check to the row model only and omits error behavior R1 Gon P2, Leorio Nit Yes
CRF-4 P4 Open transitions.go:362 Valid: chat.LastModelConfigID != uuid.Nil cannot be false (NOT NULL FK column) R1 Bisky Yes
CRF-5 Nit Open ARCHITECTURE.md:857 Rewritten sentence merges promotion and generation paths with different "stored model" meanings R1 Leorio Yes
CRF-6 Nit Open transitions_test.go:302 Adjacent uuid params; modelConfigID does not say it is the queued row's model R1 Gon Yes
CRF-7 Nit Open AgentChatPage.test.tsx:355 Bare boolean pickC table column and generic want R1 Gon Yes
CRF-8 P3 Dropped by orchestrator (client change is not redundant: picker can show a user-selected model that differs from the chat's last model, AgentChatPage.tsx:468-483) submitChatTurn.ts:376 Client override redundant with server fallback; two timings R1 Ryosuke No
CRF-9 Nit Dropped by orchestrator (resolveEditModelConfigID's selectable-and-differs rule would overwrite available queued rows; a flag for one call site adds indirection) submitChatTurn.ts:377 Inline copy of resolveEditModelConfigID rule R1 Ryosuke No
CRF-10 P2 Dropped by orchestrator (comment already states why: picker shows chat's model; Leorio reviewed it as accurate) submitChatTurn.ts:370 Comment restates ternary R1 Gon No
CRF-11 Nit Dropped by orchestrator (squash merge keeps the PR title, which is within limit) commits Two commit subjects exceed 72 chars R1 Leorio No
CRF-12 OOS Out of scope (body) chatd.go:884 Three model fallback implementations with different availability rules R1 Netero, Ryosuke, Pariston, Zoro No
CRF-13 OOS Out of scope (body) chat_mutations.go:397 EditMessage without a model skips the chat's last model; comment at :390 claims send parity R1 Ryosuke, Melody, Mafuuu, Bisky, Leorio No
CRF-14 Note Note (body) submitChatTurn.ts:378 Client pins picker model at save time; server alone resolves at promotion time R1 Ryosuke, Pariston, Zoro No
CRF-15 Note Note (body) submitChatTurn.ts:378 Rows on a model the user has no provider key for now save with the picker's model R1 Nami No
CRF-16 Note Note (body) transitions.go:360 Promotion paths pass a chat snapshot read before the transaction's own inserts R1 Mafuuu No

Contested and acknowledged

Round log

Round 1

Netero first pass: 1 P2 (CRF-1), panel proceeded. Full panel (round 1): ging-go, ging-ts, ging-react, nami, ryosuke, melody (specialists); pariston, mafuuu (core); bisky, hisoka, gon, leorio (pool); zoro (wildcard). Multi-domain (Go + TS), 12 + wildcard. Result: 2 P2, 1 P3, 1 P4, 3 Nit posted; 4 dropped; 2 OOS; 3 Notes. Gon P2 on transitions.go:350 settled at P3 (comment ambiguity, no behavior defect). CI test-go-pg and test-go-pg-17 failed; logs unreadable (gh HTTP 401), local runs of affected packages pass. Reviewed against 3d0fd60..49c87f4.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-02T16:13:21.903808Z 49c87f4 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

This comment has been minimized.

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR makes queued-message promotion try the chat's last model before the organization default, and makes a queued edit with an untouched picker send the picker's model when the row's model is not selectable. Findings: 2 P2, 1 P3, 1 P4, 3 Nit.

CI jobs test-go-pg and test-go-pg-17 fail at 49c87f4. The review runner could not read their logs (GitHub API returned 401), and the chatstate, chatd, and coderd queue tests pass locally at this head, so whether the failures come from this diff is unverified. Read the failed test names in those logs before merging.

Out of scope (needs a ticket or explicit acceptance by a human):

  • coderd/x/chatd/chatd.go:884: resolveFallbackModelConfigID, resolveModelConfig (chatd.go:1931), and chatstate.resolveQueuedMessageModelConfigID each implement "use this model if available, else the default" with different rules (daemon vs owner authorization, IsNotAuthorizedError handling, DeriveEffectiveChatModelConfigs vs an IsDefault scan), so the same unavailable model can resolve to different models on the send, promotion, and generation paths.
  • coderd/x/chatd/chat_mutations.go:397: EditMessage without a model passes the edited message's model to resolveFallbackModelConfigID, which skips the chat's last model, so an API client editing a message whose model is unavailable gets the organization default; the comment at chat_mutations.go:390 says this path resolves "like a normal message send", which is no longer true.

Notes:

  • submitChatTurn.ts:378 writes the picker's model into the row at save time. Without that change, the server would pick the chat's last model at promotion time. The two differ when the picker shows a user-selected model other than the chat's last model, or when the chat's last model changes between save and promotion.
  • submitChatTurn.ts:378: modelOptions excludes models whose provider the user has no key for (utils/modelOptions.ts:258-263), so a row on such a model now saves with the picker's model, as a history edit does.
  • transitions.go:360: every promotion path passes a chat read before the transaction inserts messages (for example FinishInterruption inserts partial messages before advanceQueue), so the chat-model candidate is the value from before those inserts. The result is the same today because a turn runs only on chat.LastModelConfigID or the default.

🤖 This review was automatically generated with Coder Agents.

chat database.Chat,
queued database.ChatQueuedMessage,
) (uuid.NullUUID, error) {
//nolint:gocritic // Queue promotion needs daemon access to deployment model configuration.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 [CRF-2] A queued row whose model was removed from the owner's model ACL still runs on the organization default and moves the chat off its model, because promotion checks candidates with dbauthz.AsChatd. (Pariston)

GetEnabledChatModelConfigByID under AsChatd returns the row's model B, so promotion keeps B; InsertChatMessages sets last_model_config_id to B (chats.sql:1020), then resolveModelConfig (chatd.go:1938-1961) reads B with the owner's authorization, gets a not-authorized error from the ACL check (dbauthz.go:4160-4166), and selects the default. That is the first row of the PR table (runs on A, chat moves to A) for the ACL case, while the frontend already treats B as unavailable because modelOptions is owner-filtered. This is inferred from the code and not run; a chatd test with an ACL-restricted row model, a chat on an available model, and an owner outside the ACL would confirm it. Check the candidates with the chat owner's model authorization, as resolveModelConfig does.

🤖

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reproduced through the API on this branch and on main: a queued row on an ACL-removed model is kept by promotion, the turn runs on the default, and the chat moves to it. Same on main, so not caused by this PR. Tracked in CODAGT-1521 with the repro patch, together with the three fallback paths using different rules.

🤖 Posted using /amend-review skill via Coder Agents.

Comment thread coderd/x/chatd/chatstate/transitions.go Resolved
Comment thread coderd/x/chatd/chatstate/transitions.go Resolved
Comment thread coderd/x/chatd/ARCHITECTURE.md Resolved
@mafredri

mafredri commented Nov 2, 2026

Copy link
Copy Markdown
Member Author

In reply to CRF-12 (P3, coderd/x/chatd/chatd.go:884): #31632 (review)

Tracked in CODAGT-1521.


In reply to CRF-13 (P3, coderd/x/chatd/chat_mutations.go:397): #31632 (review)

Tracked in CODAGT-1521, including the comment at chat_mutations.go that claims send parity.


In reply to AMREM-1 (P3, .github/workflows/ci.yaml:1): #31632 (review)

Not caused by this PR: TestProjectSkillsFollowProjectAccess and TestChatTurnStopsAtStepLimit do not promote queued messages, the second also failed on main, and both passed on rerun.

🤖 Posted using /amend-review skill via Coder Agents.

@mafredri
mafredri marked this pull request as ready for review November 2, 2026 18:27
@mafredri
mafredri requested a review from a team as a code owner November 2, 2026 18:27
@pr-queue-manager
pr-queue-manager Bot requested a review from jaaydenh November 2, 2026 18:28
…eued rows

A queued row whose model is no longer available was promoted onto the
organization default even when the chat ran on another available model,
which also moved the chat to the default. Try the chat's last model
before the default, the same order as a message sent without a model.
…le queued rows

Saving a queued edit without touching the picker omitted the model, so a
row whose model was no longer available kept it until promotion. Send
the picker's model in that case, the rule history edits already use.
The fallback-to-default cases put the chat on the organization default,
so promotion returned the chat's model and never reached the default
lookup. Their chats now use an unavailable model of the same kind as
the queued row, so the asserted model can only come from the default.
The doc comment read as if only the queued row's model was checked.
The chat's last model is a NOT NULL column, so its candidate is always
valid.
@mafredri
mafredri force-pushed the mathias/chat-queued-model-fallback branch from da2e103 to 853a70a Compare November 12, 2026 12:27
@mafredri
mafredri temporarily deployed to main November 12, 2026 12:28 — with GitHub Actions Inactive
@mafredri
mafredri enabled auto-merge (squash) November 12, 2026 12:38
@mafredri
mafredri merged commit 3086d4f into main Nov 12, 2026
49 of 52 checks passed
@mafredri
mafredri deleted the mathias/chat-queued-model-fallback branch November 12, 2026 12:48
@mafredri
mafredri temporarily deployed to main November 12, 2026 12:48 — with GitHub Actions Inactive
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 12, 2026

This branch was previously deployed

1 inactive deployment
main — 853a70a1 Deployed Nov 12, 2026 by mafredri via cla #254901
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants