Repository navigation
chore(coderd): store a delivery mode on queued chat messages - #30073
Conversation
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
… guard (#30096) Adds the backend-only seam that lets a later `Publish` operation admit an automation message inside chatd's existing create-chat and send-message transactions, and the guard that drops stale automation input when the queue promotes it. Only Go test callbacks use the seam in this PR, so ordinary chat behavior does not change. Closes CODAGT-1208 Part of CODAGT-1135 > Stack: PR 2, on top of #30085 (schema, RBAC resource, and audit entries). No migration changes; this PR adds and extends sqlc queries only. ARCHITECTURE.md prose was drafted by an agent at the user's request; the user will review it. ## Changes - **Admission seam.** `chatstate.AdmitFunc` is a nil-default `AdmitInTx` field on `chatd.CreateOptions`, `chatd.SendMessageOptions`, `chatstate.CreateChatInput`, and `chatstate.SendMessageInput`. `CreateChat` runs it after inserting the chat and before the initial history, and requires exactly one initial user message. `SendMessage` runs it on the locked chat before any write. The callback returns an `AutomationProvenance` (automation ID, input ID, queue generation), which is validated and stamped on the message. A callback error rolls back the whole transaction, so no chat, message, or queue row survives. `Server.CreateChat` and `Server.SendMessage` keep their signatures. - **Provenance.** `chatstate.Message.Automation` is written to queued rows (`automation_id`, `input_id`, `queue_generation`) and history rows (`automation_id`, `input_id`). `InsertChatMessages` and `InsertChatQueuedMessageWithCreator` gain the matching parameters, and `messageFromQueuedRow` copies them on promotion. - **Promotion guard.** `chatstate.LockAutomations` is the only way to lock automation rows: it locks them `FOR UPDATE` in ascending ID order, running as chatd. The chatd subject gains read on `chat_automation`. A queued automation row passes only if its automation exists, is enabled, and has the row's `queue_generation`. Two helpers apply the check: - `nextPromotableQueueHead` serves `sendMessageE1`, `FinishTurn`, and `FinishInterruption`. A head without an automation costs no extra query. Otherwise it locks every automation in the queue up front, deletes failing heads until one passes, and leaves rows behind that head alone. If the queue empties, `FinishTurn` and `FinishInterruption` land in `waiting`, as from `R0` and `I0`. - `guardQueuedRows` serves explicit `PromoteQueuedMessage`. A failing row is deleted and the transition returns `Rejected` without an error, so the delete commits. `Server.PromoteQueued` then returns `ErrQueuedMessageNotFound` (HTTP 404) without reordering, substituting a message, or changing status. - The steer paths (`DeliverSteerMessages` and the steer batches from #30073/#30074) add one `guardQueuedRows` call each, whichever PR lands second. - **Deadlocks.** `database.IsDeadlockError` is new. `ChatMachine.Update` reruns the transaction, at most 3 attempts with a fresh publish buffer each, only when an attempt that took automation locks hit a deadlock abort. Attempts without automation locks, and nested Updates, never retry. The `SendMessage` and `PromoteQueued` closures reset their results so a rerun is safe. - **Transition matrix.** Adds the outputs the guard can produce, such as `R1 -> FinishTurn -> W` and `E1 -> SendMessage -> R0`. - **`ARCHITECTURE.md`.** Describes the admission seam, provenance, the queue promotion guard, the lock order, and the deadlock retry, and adds the guard's outcomes to the state diagram. ## Validation - Go tests on real PostgreSQL cover the ticket's checks: - A callback error leaves no chat, message, or queue row. - A guard rejection deletes only the rejected heads, on every promotion path and for each failure reason (missing, disabled, disabled with the same generation, generation mismatch). - Explicit promotion of a failing row returns not found, and the delete is committed. - Provenance round-trips through queue and history. - `LockAutomations` works through dbauthz. - `Update` retries only when an attempt that took automation locks hits a deadlock. - A concurrency test of acceptance, promotion, and disable across mixed automation IDs sees zero deadlock aborts. - Mutation checks confirmed that the guard, retry, and permission tests fail when their behavior is removed. - Remote dogfood UAT passed on a real dev instance: sending, queueing, head and explicit promotion, interrupting, and deleting queued messages all behaved as before. --- _Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
632582f to
c140e42
Compare
|
@codex review |
c140e42 to
028f65a
Compare
| @@ -0,0 +1,33 @@ | |||
| -- Take the lock this migration needs before changing anything, retrying | |||
| -- short waits so that concurrent chatd traffic cannot deadlock with it. See | |||
| -- 000609_chat_automations.up.sql for the reasoning. | |||
There was a problem hiding this comment.
We should remove the lock-retry block. Every pending migration runs in one transaction, so if an earlier one already holds chats, chatd requests hang behind the block for 2 minutes and then the upgrade fails. Without the block, Postgres cancels one request after 1s and the upgrade succeeds. My agent reproduced this, and 000615 already fails this way on upgrades from 610.
There was a problem hiding this comment.
Oh okay interesting. I copied it verbatim from prior migrations so i assumed it was fine?
There was a problem hiding this comment.
As I understand it, it's fine on dogfood due to deploying every migration individually, but can fail on release.
There was a problem hiding this comment.
@ethanndickson on some further analysis, this method of locking trades one issue for another. But essentially, my previous statement was wrong.
If there's a release, and first migration that locks is 509, that's the migration that can cause a problem. All future migrations that do the same lock are essentially no-op because that lock is retained throughout the migration.
See #31167 for more context.
Essentially, I think we shouldn't do this type of locking at all, and find a better way to deal with issues resulting from rolling upgrades. Personally I would like to see a method where "writes" are revoked from old replicas during upgrade.
5e23b29 to
a1439bd
Compare
802a23e to
aba1026
Compare
a97485f to
2cfb041
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
2cfb041 to
61262fd
Compare
Add a chat_busy_behavior enum ('queue', 'steer') and a busy_behavior
column on chat_queued_messages. Every current insert stores 'queue', so
behavior does not change. A later change delivers 'steer' rows at the
next step boundary.
61262fd to
2c514df
Compare
Queued chat messages now store a delivery mode. Later in this stack,
steermessages go in before the next model call instead of at turn end, so chatd needs to know which queued rows are steer rows.The migration adds the enum
chat_busy_behavior(queue,steer) and the columnchat_queued_messages.busy_behavior. It'sNOT NULL DEFAULT 'queue', so existing rows getqueue, and chatstate still passesqueueon every insert for now, so this doesn't change behaviour. The oldInsertChatQueuedMessagequery doesn't set the column and just gets the default, but only tests use it.There's no
interruptvalue on purpose. An interrupt isn't a delivery mode, it's something that happens to the chat, and #31174 stops the API accepting it as a busy behaviour.This supersedes #29783.
Part of CODAGT-1140