Repository navigation
perf(coderd): bound the initial chat stream snapshot to the client cursor - #29761
Conversation
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9bba8c1b78
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f70de36ddb
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review The previous round's P1 asking for TODOs in
|
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
Writing review of the description and the new comments: one accuracy fix, one question, and comment rewordings. I checked claims against the code but did not otherwise review the logic.
The description says database "rows read drop from the whole chat to 2". No index on chat_messages includes revision, so the bounded query still reads every row of the chat to filter by revision; what drops is rows returned. The diff below fixes that, moves the benchmark caveats before the table, and trims the delivery record (earlier-round history, the comment-cleanup bullet, the Codex P1/P2 priority labels). It also uses one term per concept: "deleted row" instead of "tombstone", which appears nowhere in non-test code on main, plus "initial snapshot", "fetch", and "history".
@@ -1,4 +1,4 @@
-Opening a chat on the Agents page loads its transcript over REST and then connects the chat stream with `after_id` set to the newest loaded message. The stream's initial database snapshot ignored that cursor and read every row of the chat on each connection. Any soft-deleted row left behind by an earlier message edit also turned the connection into a `history_reset` followed by a full transcript replay.
+Opening a chat on the Agents page loads its message history over REST and then connects the chat stream with `after_id` set to the newest loaded message. The stream's initial database snapshot ignored that cursor and fetched every row of the chat on each connection. A soft-deleted row from any earlier message edit also turned every connection into a `history_reset` and a full history replay.
-The initial snapshot now resolves `after_id` to that message's revision and only reads rows changed at or after it. A tombstone resets the client's history only when the client could still hold the row: its id is at or below the cursor and it was deleted after the cursor's revision. Cursors that cannot bound the read keep the full scan: a deleted cursor still produces one `history_reset` with the surviving history, and an unknown id keeps the documented numeric cutoff. A message id from another chat, live or deleted, is ignored, so the client receives the full history. Live updates after the initial sync are unchanged.
+The initial snapshot now fetches only rows changed at or after the `after_id` message's revision. A deleted row resets the client's history only when the client could still hold it: its id is at or below the cursor and it was deleted after the cursor's revision. Deleted and unknown cursors keep fetching the whole chat, as before: a deleted cursor produces one `history_reset` with the surviving history, and an unknown id still skips messages with ids at or below it. A cursor from another chat, deleted or not, is ignored, so the client receives the full history. Live updates after the initial snapshot are unchanged.
@@ -6,3 +6,3 @@ The initial snapshot now resolves `after_id` to that message's revision and only
-Remote UAT round 4 compared the merge base `956eafbb75` with `5a170a7c9b6` on the same seeded database. It timed the gap from websocket dial to the initial snapshot frame, using 30 samples per cell in interleaved batches. Numbers are medians.
+A remote UAT run timed websocket dial to the initial snapshot frame on the merge base `956eafbb75` and on `5a170a7c9b6`, with 30 interleaved samples per cell; numbers are medians. Both used one seeded database of synthetic rows shaped from real chats (tool results about 8 KB), with a hot cache, one sequential client, and a loaded shared host, so treat the numbers as relative, not as production latencies.
@@ -19,5 +19,5 @@ Remote UAT round 4 compared the merge base `956eafbb75` with `5a170a7c9b6` on th
-On reconnect, database rows read drop from the whole chat to 2 (the cursor lookup plus the bounded read). At 50 messages, and in the no-cursor and deleted-cursor cases, the difference is within the base-vs-base noise. Caveats: synthetic rows shaped from real chats, with tool results of about 8 KB; hot cache; one sequential client; a loaded shared host. Treat the numbers as relative, not as production latencies.
+With the cursor at the newest message, the snapshot queries return 2 rows instead of the whole chat (the cursor lookup plus the bounded fetch). At 50 messages the difference is also within the noise between repeated `main` runs.
-Follow-up for the human author: the message synchronization section of `coderd/x/chatd/ARCHITECTURE.md` still says the initial sync always fetches after `history_version` and resets on any tombstone. It needs the cursor-bounded first fetch and the conditional tombstone reset. Xum did not edit that file, per the author's standing instruction.
+Follow-up for the human author: the message synchronization section of `coderd/x/chatd/ARCHITECTURE.md` still says every sync fetches rows after `history_version` and resets on any deleted row. Xum did not edit that file, per the author's standing instruction.
@@ -26,10 +26,6 @@ Follow-up for the human author: the message synchronization section of `coderd/x
-- Remote Coder Agents UAT ([chat](https://dogfood.cdr.dev/agents/3bf66d40-1761-43f4-ab5d-059e1ac323ea)): PASS on the final head `5a170a7c9b6` with no regressions. The statement log confirmed the revision-bounded read for a live cursor, and the round covered deleted, foreign, and unknown cursors, edits, grouped commits, and reconnects. Round 1 passed on `9bba8c1b78f`; round 2 on `f70de36ddb1` found the deleted foreign cursor gap that Codex round 2 also reported. Three minor findings remain, all identical on `main`.
-- Remote UAT round 4 (performance, same chat) on `5a170a7c9b6` vs `956eafbb75`: results in Performance above; the statement logs confirmed the intended query path in every cell. Isolation proof and leak audit clean, no functional anomalies.
+- Remote Coder Agents UAT ([chat](https://dogfood.cdr.dev/agents/3bf66d40-1761-43f4-ab5d-059e1ac323ea)) on `5a170a7c9b6`: PASS, no regressions. The statement log showed the revision-bounded fetch, and the run covered deleted and unknown cursors, cursors from other chats, edits, grouped commits, and reconnects. Three minor findings remain, all identical on `main`.
- Local validation on `5a170a7c9b6`: `go test ./coderd/x/chatd`, `go test ./coderd -run '^TestStreamChat$'`, `go test ./coderd/database/dbauthz -run '^TestMethodTestSuite$'`, `go test -race` on the stream tests, and `make pre-commit`.
-- Comment cleanup pass (agent) before the PR opened: tightened comments in `9bba8c1b78f`.
-- [Codex round 1](https://github.com/coder/coder/pull/29761#pullrequestreview-5284702021) on `9bba8c1b78f`: one P2, a cursor from another chat with higher ids returned an empty history (also one of the UAT findings). Confirmed red with the reordered `TestStreamChat/AfterIDOtherChat` and fixed in `f70de36ddb1`.
-- [Codex round 2](https://github.com/coder/coder/pull/29761#pullrequestreview-5284855535) on `f70de36ddb1`: a P2, a deleted cursor from another chat still returned an empty history. Confirmed red with the extended `TestStreamChat/AfterIDOtherChat` and fixed in `5a170a7c9b6` by resolving the cursor with `GetChatMessageByIDForStream`, which includes deleted rows. A P1 asked for TODOs in the chatd `ARCHITECTURE.md`; not applied per the author's standing instruction (see the follow-up above).
-- [Codex round 3](https://github.com/coder/coder/pull/29761#issuecomment-5786154753) on `5a170a7c9b6`: no findings.
-- CI on `5a170a7c9b6`: all checks passed.
+- Codex [round 1](https://github.com/coder/coder/pull/29761#pullrequestreview-5284702021) found that a cursor from another chat, with an id above every id in this chat, returned an empty history, and [round 2](https://github.com/coder/coder/pull/29761#pullrequestreview-5284855535) found the same for a deleted cursor from another chat. `TestStreamChat/AfterIDOtherChat` failed before each fix (`f70de36ddb1`, `5a170a7c9b6`). Round 2 also asked for TODOs in the chatd `ARCHITECTURE.md`; see the follow-up above. [Round 3](https://github.com/coder/coder/pull/29761#issuecomment-5786154753) on `5a170a7c9b6`: no findings.
+- CI on `5a170a7c9b6`: all checks passed or were skipped.
🤖 This review was automatically generated with Coder Agents.
mafredri
left a comment
There was a problem hiding this comment.
Follow-up to my earlier review: the same "tombstone" and "foreign cursor" wording in places that review missed.
🤖 This review was automatically generated with Coder Agents.
mafredri
left a comment
There was a problem hiding this comment.
One correctness issue, plus two things that break on current main.
A live after_id row does not show that the client applied earlier deletions. The initial sync treats the cursor row's revision as the point the client's history is current to, and skips the reset for older deleted rows. The web client can hold a newer ID without having applied an earlier edit:
- Tab A loads the chat; its stream socket is down.
- Tab B edits an earlier message. A misses the
history_reset. - A sends a message.
submitChatTurn.tsstores the response, so A's newest ID is now above the edit. - A reconnects with that ID. The fetch starts at the new row's revision, finds no deleted rows, and sends no reset.
A keeps showing the deleted messages and never gets their replacements. On main the same sequence recovers, because any deleted row forces the reset. The test below fails on 98098cf461 (expected [5 6 7 8], actual [3 4 7 8]) and passes on the merge base 956eafbb75. A socket drop between the frames of a multi-frame reset gives the same result, because the client advances its ID on buffered rows before it applies the reset.
An ID cannot distinguish "loaded a fresh page after the edit" from "held the old rows and got a newer ID from a send", so the server cannot fix this from after_id alone. The client has to report which history version it has applied. #29662 does this: the messages page and status events carry history_version, and the stream accepts it as after_revision. I'd suggest taking that approach here; I'll close #29662 once this lands. Its known gaps, so they don't carry over:
- The client ref stores a chat ID, but
connect()never checks it, so a page withouthistory_versionreuses the previous chat's version. history_versionis returned onafter_idpages withhas_more; a caller that passes it skips the unread rows. Return it only on the newest page, as withqueued_messages.- The version only advances on status changes, so a reconnect after a live edit during a running turn replays the full history again.
- Returning to a cached chat restores the cached page's old version, which costs one more full reset.
- With
after_revisionset, drop theafter_idfilter: it skips rows another tab added while the socket was down (also broken onmain).
On a merge with current main:
TestStreamLoopInitialSyncBoundsFetchToCursorRevisionfails in all 6 cases: the mock expectsInTx(..., nil)andGetChatByIDForShare, which #30266 replaced with a read-only repeatable-read snapshot.genfails:queries.sql.goneeds regenerating for the newqueued_message_idcolumn.
Reproduction test (drop into coderd/)
package coderd_test
import (
"slices"
"testing"
"github.com/google/uuid"
"github.com/stretchr/testify/require"
"github.com/coder/coder/v2/coderd/coderdtest"
"github.com/coder/coder/v2/codersdk"
"github.com/coder/coder/v2/testutil"
)
// A tab whose stream is down misses an edit made elsewhere, then sends a
// message and stores the response, as submitChatTurn.ts does. Its newest
// message ID is now above the edit. After reconnecting with that ID, the
// tab's messages must match the chat's live history.
func TestStreamChatMissedEditThenSend(t *testing.T) {
t.Parallel()
ctx := testutil.Context(t, testutil.WaitLong)
client := newChatClient(t)
firstUser := coderdtest.CreateFirstUser(t, client.Client)
_ = createChatModel(t, client)
waitIdle := func(chatID uuid.UUID) {
t.Helper()
require.Eventually(t, func() bool {
chat, err := client.GetChat(ctx, chatID)
return err == nil && chat.Status == codersdk.ChatStatusWaiting
}, testutil.WaitLong, testutil.IntervalFast)
}
liveIDs := func(chatID uuid.UUID) []int64 {
t.Helper()
page, err := client.GetChatMessages(ctx, chatID, nil)
require.NoError(t, err)
require.False(t, page.HasMore)
ids := make([]int64, 0, len(page.Messages))
for _, m := range page.Messages {
ids = append(ids, m.ID)
}
slices.Sort(ids)
return ids
}
chat, err := client.CreateChat(ctx, codersdk.CreateChatRequest{
OrganizationID: firstUser.OrganizationID,
Content: []codersdk.ChatInputPart{{Type: codersdk.ChatInputPartTypeText, Text: "first prompt"}},
})
require.NoError(t, err)
waitIdle(chat.ID)
// The tab loads the newest page; its stream then goes down.
page, err := client.GetChatMessages(ctx, chat.ID, nil)
require.NoError(t, err)
held := map[int64]bool{}
var promptID int64
for _, m := range page.Messages {
held[m.ID] = true
if m.Role == codersdk.ChatMessageRoleUser {
promptID = m.ID
}
}
require.NotZero(t, promptID)
// Another tab edits the prompt, deleting it and its reply.
_, err = client.EditChatMessage(ctx, chat.ID, promptID, codersdk.EditChatMessageRequest{
Content: []codersdk.ChatInputPart{{Type: codersdk.ChatInputPartTypeText, Text: "edited elsewhere"}},
})
require.NoError(t, err)
waitIdle(chat.ID)
// This tab sends and stores the response.
sent, err := client.CreateChatMessage(ctx, chat.ID, codersdk.CreateChatMessageRequest{
Content: []codersdk.ChatInputPart{{Type: codersdk.ChatInputPartTypeText, Text: "sent while the stream was down"}},
})
require.NoError(t, err)
require.False(t, sent.Queued)
newest := sent.Messages
if len(newest) == 0 && sent.Message != nil {
newest = []codersdk.ChatMessage{*sent.Message}
}
require.NotEmpty(t, newest)
var afterID int64
for _, m := range newest {
held[m.ID] = true
}
for id := range held {
afterID = max(afterID, id)
}
waitIdle(chat.ID)
// Reconnect and apply the initial sync.
events, closer, err := client.StreamChat(ctx, chat.ID, &codersdk.StreamChatOptions{AfterID: &afterID})
require.NoError(t, err)
defer closer.Close()
for done := false; !done; {
select {
case <-ctx.Done():
require.FailNow(t, "timed out waiting for the initial stream sync")
case event, ok := <-events:
require.True(t, ok, "stream closed before the initial sync completed")
switch event.Type {
case codersdk.ChatStreamEventTypeHistoryReset:
clear(held)
case codersdk.ChatStreamEventTypeMessage:
held[event.Message.ID] = true
case codersdk.ChatStreamEventTypeStatus:
done = true
}
}
}
got := make([]int64, 0, len(held))
for id := range held {
got = append(got, id)
}
slices.Sort(got)
require.Equal(t, liveIDs(chat.ID), got, "the tab still shows deleted messages or is missing their replacements")
}🤖 This review was automatically generated with Coder Agents.
…rsor Every chat stream connect loaded the chat's entire message history from Postgres, then dropped the rows at or before after_id in Go. Any soft-deleted row from a past edit also forced a history reset, so those chats re-sent their whole transcript over the WebSocket on each connect. Resolve the cursor message's revision first and fetch only revisions from that point on. A cursor that an edit has since removed still falls back to the full scan so the deletion surfaces as a history reset.
…shot An edit soft-deletes the discarded turn and inserts its replacement in one transaction, so those tombstones share the replacement's revision. A client reconnecting with the replacement as its after_id cursor pulled them into the bounded fetch and received a redundant history_reset plus a full replay. Only tombstones at or below the cursor that were deleted after the cursor's revision can invalidate history the client saw, so the snapshot now resets on those alone and skips the rest. Adds mock coverage for same-revision truncation, tombstones after the cursor, and a held row deleted later, plus HTTP coverage for an edited cursor, an unknown cursor, and a cursor from another chat.
The initial stream sync looked up after_id with GetChatMessageByID, which skips deleted rows, so a deleted message from another chat looked unknown and kept the numeric cutoff, emptying the snapshot when its id was higher. Resolve the cursor with GetChatMessageByIDForStream, which includes tombstones: another chat's message, live or deleted, clears the cutoff, and a same-chat tombstone keeps the unbounded scan because deleting it moved its revision. Unknown ids keep the numeric cutoff.
Main (#29808) now decides the first-sync history reset in historyResetRequired: it resets only when message after_id is missing or deleted. The rebase keeps that rule instead of this branch's deletedRowRequiresReset, which gave the same result for suffix deletions. Under that rule a deleted cursor in the bounded fetch already forces the reset, because the fetch includes the cursor row itself. The deleted-cursor fallback to a full scan is therefore dropped, and initialSyncCursor bounds the fetch for any cursor of this chat. The cursor-bound mock test is folded into main's TestStreamLoopInitialSyncTombstones table, which now answers the cursor lookup and the revision query from the rows of every chat and states the expected bound per case. New cases cover a message committed with after_id, tombstones committed with after_id, an unknown after_id without tombstones, and live or deleted cursors from another chat. queries.sql.go is regenerated for main's queued_message_id column.
98098cf to
7bed785
Compare
|
@mafredri Thanks. I rebased on Missed edit then send (not changed here). Since your review, #29808 merged, and Mock tests after #30266. Fixed. The cursor-bound mock test is now part of the
Also: with the reset rule from
|
|
@codex review
|
|
Codex Review: Didn't find any major issues. Already looking forward to the next diff. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Opening a chat on the Agents page loads its transcript over REST and then connects the chat stream with
after_idset to the newest loaded message. The first sync of the stream ignores that cursor when it queries the database: it fetches every row of the chat on each connection, and then sends only the rows aboveafter_id.The first sync now resolves
after_idto that message's revision and fetches only rows changed at or after it. The history reset rule is the one onmain(#29808): the stream resets only when messageafter_idis deleted or missing. A deleted cursor still produces onehistory_resetwith the surviving history, because the bounded query includes the cursor row. An unknown ID keeps the full query. A message ID from another chat, live or deleted, is ignored, so the client receives the full history. Live updates after the first sync are unchanged.Performance
Measured on dogfood's database, which runs
main, withEXPLAIN (ANALYZE, BUFFERS, SERIALIZE BINARY)of the first-sync query. The chats are the 554 chats updated in the past day, picked by row count.mainstarts at revision 0. This PR starts at the revision of the newest live message, which is theafter_idthe web client sends.mainoutputmainfirst / repeat runrevision. The saving is the TOAST reads, serialization, transfer, and decoding of rows thatmainthen discards. Most ofmain's time is serialization: 153 ms of 193 ms for the largest chat.Not changed
A tab whose stream was down can miss an edit, then send a message and store the response, so its newest ID is above the edit. On reconnect, message
after_idis live and the stream sends no reset, so the tab keeps the deleted messages.mainhas done the same since #29808, and this PR does not change it (mafredri's reproduction givesexpected [5 6 7 8], actual [3 4 7 8]on both). An ID cannot show which edits the client applied. #29662 adds a client-reported version for that.Delivery record
mainat6ebefb83254, keeping the fix(coderd/x/chatd): reset stream history only if after_id is deleted #29808 reset rule and perf(coderd/x/chatd): make chat stream reads lock-free #30266 read-only snapshot reads. The cursor-bound mock test is part ofTestStreamLoopInitialSyncTombstones.go test ./coderd/x/chatd/...,go test ./coderd -run '^TestStreamChat$',go test ./coderd/database/dbauthz -run '^TestMethodTestSuite$'. Mutants of the cursor bound (no- 1, no bound, cursor from another chat kept) each failTestStreamLoop*.7bed7855e41found nothing.