Skip to content

perf(coderd/x/chatd): finish the turn in the step commit transaction - #30270

Closed
cstyan wants to merge 2 commits into
callum/chatd-04-single-writefrom
callum/chatd-07-finish-turn-in-step
Closed

cstyan wants to merge 2 commits into
callum/chatd-04-single-writefrom
callum/chatd-07-finish-turn-in-step

Conversation

@cstyan

@cstyan cstyan commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

When the decision pass over the history a step commits would finish the turn, the generation loop re-dispatched a task iteration (a ReadLock, prepareGeneration, and a second row lock) only to reach the same decision. commitGenerationStep now applies FinishTurn in the same Update as CommitStep when nothing else needs that extra round: hooks disabled (the stop hook runs between the two transitions), an empty queue (R1 promotion can fail on model config resolution and must not roll back the step), and no consumed compaction request (that commit already wrote the row). stepFinishesTurn carries the reasoning for why the merge changes nothing but the number of transactions.

Database impact: one fewer transaction and one fewer row lock acquisition per turn that ends after a committed step.

Combined with the other PRs: based on #30267; without #30268 the finish branch sets committed from the FinishTurn result and skips the in-callback reload, and once #30268's UpdateReturning lands those lines reduce to finished = true, so expect a small conflict in commitGenerationStep for whichever merges second.

Depends on #30267.

Series: #30264 harness jitter, #30265 transition metrics, #30266 lock-free stream reads, #30267 single-write transitions, #30268 commit-write result and RBAC cache, #30269 non-blocking acquisition, #30270 finish turn in the step commit.


Generated by Coder Agents on behalf of @cstyan.

cstyan added 2 commits October 1, 2026 20:29
Postgres re-runs every foreign key check on an UPDATE whose old row version
was written by the current transaction. A transition used to bump
snapshot_version first and then update the chats row again from the message
history trigger, UpdateChatExecutionState, the attempt increment, the retry
state write, and one queue-version trigger per queued row. Each follow-up
write re-checked owner_id, organization_id, and last_model_config_id with
FOR KEY SHARE on parent rows shared by every chat, so all transitions
serialized on the users, organizations, and chat_model_configs heap pages
(pg_buffercache showed every backend queued on those three pages).

Each transition now ends with exactly one chats UPDATE that also advances
snapshot_version: UpdateChatExecutionState, IncrementChatGenerationAttempt,
UpdateChatRetryState, or the new BumpChatSnapshotVersion, which Update also
issues when a callback performed no commit write. LockChatForTransition
replaces LockChatAndBumpSnapshotVersion and writes nothing. The revision
trigger stamps snapshot_version + 1 (the version the transaction commits);
the history and queue-version triggers are dropped and the commit write
records those changes from flags set by the message and queue helpers.
InsertChat starts a chat at version 0 so CreateChat still lands on 1/1.
UpdateChatExecutionState keeps clearing retry_state whenever the chat
leaves running.

The lock also changes from FOR UPDATE to FOR NO KEY UPDATE (as do
GetChatByIDForUpdate, LockChatByID, and the context-resource sync):
transitions never change chats.id, and FOR UPDATE conflicted with the FOR
KEY SHARE locks that foreign-key child writes such as the batched
heartbeat upserts take on the chat row, so heartbeats and transitions
convoyed against each other.

The lock's RETURNING row, extended with has_queued, seeds the Tx. The first
loadState in a transaction returns it with no round trip. This is not a
cache across writes: every transition validates before it writes, so that
first call always precedes any execution-state mutation and sees exactly
the locked row. Later loadState calls read from the database, so bundles
of transitions still validate against the latest state. Tx.Current lets a
callback inspect state without consuming the seed; the HTTP handlers, the
task starters, and the worker's acquisition and abandon paths use it
instead of re-reading the chat under the lock. Publication keeps reading
the committed row after the commit write.

Regression tests pin the single-write contract and the round-trip budget:
a transition validates from the seed and updates the chats row exactly
once, a metadata-only callback still advances snapshot_version once, and a
bundle re-reads exactly once after the seed is consumed.

Emulation (48 clients, same host, pgbench with the generated statements):
LWLock:BufferContent fell from ~30% to ~15% of backend samples, insert and
exec-state latencies dropped by a quarter, and organizations left the hot
page list. Remaining shared-page work is the per-message model config FK
check and index maintenance on the commit write.
When the decision pass over the history a step commits would finish
the turn, apply FinishTurn in the same Update as CommitStep instead of
re-dispatching a task iteration (ReadLock, prepareGeneration, and a
second row lock) only to reach the same decision.

The merge is limited to the cases where it changes nothing but the
number of transactions: hooks disabled (the stop hook runs between
the two transitions), an empty queue (R1 promotion can fail on model
config resolution and must not roll back the step), and no consumed
compaction request (that commit already wrote the row).

A completed turn without hooks therefore traces as a single
generate_assistant step whose commit also completes the turn span.
@linear-code

linear-code Bot commented Oct 29, 2026

Copy link
Copy Markdown

GRU-114

@cstyan cstyan closed this Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant