Repository navigation
fix(coderd/x/chatd): coalesce deltas when persisting interrupted turns - #29661
Conversation
ddb90bd to
8973b1a
Compare
|
@codex review |
|
Chat: Review posted | View chat Review historydeep-review v0.13.0 | Round 5 | Last posted: Round 5, 24 findings (6 P3, 12 P4, 6 Nit), APPROVE. Review Finding inventoryFindings
Contested and acknowledgedCRF-2 (P4, message_conversion.go:796) - streamedRunIndex duplicates len(assistantParts)-1
CRF-28 (Nit, message_conversion.go:879) - appendAssistantPart doc first clause restates body
CRF-29 (Nit, message_conversion_test.go:1027) - Benchmark comment restates fixture; cap not tied to defaultMaxEpisodeBytes
Round logRound 1Round type: round 1 (full panel). Netero: 1 P3, 1 P4, no P0, panel proceeds. Round 2Churn guard: PROCEED. 12 author fixed, CRF-2 contested. Round type: restructure (appendStreamedDelta deleted, appendAssistantPart rewritten to own the merge, new sameTime helper, new reasoning-split behavior from 8394793). Round 3Churn guard: PROCEED. CRF-22..27, CRF-30 author fixed; CRF-28, CRF-29 contested (Gon). Round type: fix round (commits only address R2 findings; chatloop_internal_test.go added to the diff only as the CRF-25 test). Round 4Churn guard: PROCEED. CRF-34 author fixed (44544e2). Round type: fix round (only change since R3 head 5d7f912 in PR files is a 2-line doc comment edit in appendAssistantPart). Round 5Churn guard: PROCEED. CRF-38 author fixed (description edit). Round type: fix round. Rebase onto 782fb30 only; About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8973b1a1d8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
This PR merges adjacent text and reasoning deltas into one part each when an interrupted turn is persisted. Findings: 4 P3, 6 P4, 3 Nit. The P3s are an unreachable and untested merge path, a benchmark sized above the episode buffer cap, and two comments that state the wrong contract.
Out of scope (needs a ticket or explicit acceptance by a human):
coderd/x/chatd/generation.go:1414:publishdiscardsErrEpisodeFullfromAddPart, so an interrupted turn longer than about 18 KB of streamed text is persisted without its tail and nothing is logged.coderd/x/chatd/messagepartbuffer/message_part_buffer.go:309: the episode cap charges each delta its full JSON envelope (about 400 bytes for 7 characters), so 1 MiB holds about 18 KB of text.codersdk/chats.go:485:ChatMessagePartserializes every empty field (367 bytes for a 7-character text part), which inflates the buffer, stream replay, and stored completed messages.coderd/x/chatd/message_conversion.go:882: interrupted rows stored before this PR keep one part per token, so/messagespages containing them stay multi-megabyte.site/src/pages/AgentsPage/components/ChatConversation/messageParsing.ts:23: whitespace-only parts are dropped, so those older rows render with words and paragraphs joined.- Repository
_test.gofiles: 48 benchmarks usefor range b.Nagainst 9 usingb.Loop().
🤖 This review was automatically generated with Coder Agents.
8973b1a to
d6b5985
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Bravo. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
This round re-reviews the restructured merge: appendAssistantPart now does every append, and a reasoning run ends when CreatedAt changes. Findings: 2 P3, 4 P4, 3 Nit.
Twelve round-1 findings are fixed; the explanation for keeping streamedRunIndex is accepted. The P3s are an untested mid-stream run close and a field comment whose rule contradicts appendAssistantPart.
Out of scope (needs a ticket or explicit acceptance by a human):
coderd/x/chatd/message_conversion.go:828: every interrupted reasoning part getsCompletedAt = interruptedAt, including blocks whoseReasoningEndalready fired, so a finished block shows a duration that runs to the interrupt.coderd/x/chatd/message_conversion.go:789:modelStreamedAssistantis true for tool calls and reasoning too, so the name says "streamed" where it means "the model produced content".
In reply to IC_kwDOGkVX1s8AAAABaCX6JA:
P4 [CRF-26] The PR description says a seven-character part "serializes to about 367 bytes, so the 1 MiB episode buffer stops after about 2,585" deltas, but 1 MiB / 367 B is 2,857. (Leorio)
The buffer charges serializedPartBytes (messagepartbuffer/message_part_buffer.go:658), which marshals the {seq, role, part} envelope: 406 B for that part, and 1,048,576 / 406 = 2,582. State both numbers so the benchmark sizing can be checked.
🤖
🤖 This review was automatically generated with Coder Agents.
d6b5985 to
5d7f912
Compare
|
In reply to CRF-26 (P4, Partly. 1 MiB / 367 B is not 2,585, but 2,582 is not right either: the envelope is 403 to 406 B as
|
|
@codex review
|
|
Codex Review: Didn't find any major issues. You're on a roll. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
/coder-agents-review
|
There was a problem hiding this comment.
This round checks the fixes for round 2. Findings: 1 P4.
Seven round-2 findings are fixed: CRF-22 through CRF-27 and CRF-30. The new tests fail when their targets are mutated. The explanations for the two doc-comment Nits are accepted.
Out of scope (needs a ticket or explicit acceptance by a human):
coderd/x/chatd/chatloop/chatloop.go:1000: buffered reasoning deltas carry no provider block ID, so interrupted-turn persistence has to infer block identity from the start time.
🤖 This review was automatically generated with Coder Agents.
5d7f912 to
44544e2
Compare
|
@codex review
|
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
There was a problem hiding this comment.
This round checks the fix for the same-microsecond reasoning merge (now documented in the appendAssistantPart comment). Findings: 1 P4, in the PR description. It is in the item below, because it has no code line to attach to. No code findings remain.
In reply to IC_kwDOGkVX1s8AAAABaluopA:
P4 [CRF-38] The PR description's Fix section says reasoning deltas split on CreatedAt "and each block keeps its own part", which the new appendAssistantPart comment contradicts: blocks that start in the same microsecond merge. (Leorio)
clockNow rounds to the microsecond (chatloop.go:955, dbtime.go:17), so two ReasoningStart events in one microsecond produce equal CreatedAt. Suggested: "Reasoning deltas carry their block's start time, rounded to the microsecond, as CreatedAt, so a reasoning run ends when CreatedAt changes; two blocks that start in the same microsecond merge into one part."
🤖
🤖 This review was automatically generated with Coder Agents.
44544e2 to
b48cabb
Compare
|
In reply to CRF-38 (P4, The Fix section now says reasoning blocks that start in the same microsecond merge into one part, matching the
|
|
$@codex review\n/coder-agents-review\n\n> 🤖 This review was automatically generated with Coder Agents. |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
/coder-agents-review
|
|
Codex Review: Didn't find any major issues. Swish! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
An interrupted turn was persisted from the buffered stream deltas one part per token. A completed turn takes another path and stores one text part per run, so the two shapes diverged: a 28 KB interrupted response became 3,987 text parts and 1 MB of JSON, each part repeating the same empty fields, and every client that loaded the page parsed and retained thousands of objects for it. Interrupts are routine (stop button, a new message sent while running), so long chats accumulate these. Merge adjacent text and reasoning deltas into the previous part while converting buffered parts, mirroring how processStepStream accumulates a run: keep the latest non-empty provider metadata, the first start time and the latest completion time. Tool-call deltas were already merged. The new test fails on the previous code with 8 parts where a completed turn would store 4.
Merging each delta with prev.Text += part.Text copied the whole run so far on every token: a 3,987-delta interrupted turn allocated 61 MB to persist 28 KB of text. Keep the open run in a strings.Builder and write it back when the run closes. Same input now allocates 2.6 MB. Benchmark added; the tool-call accumulators already used builders.
"Linear" named no variable, and a benchmark comment said allocations "must" grow linearly when a benchmark performs no such check. Say linear in N, and tell the reader to compare B/op across the two sizes.
…cannot show The comments narrated the merge steps and call sites visible below them. Keep the reason for the builder and the parity with processStepStream; drop the rest.
Reasoning deltas from consecutive provider blocks are adjacent in the buffer, so merging by type alone joined them into one part and dropped the second block's start time. Every delta of a block carries the block's start time as CreatedAt, so a changed CreatedAt starts a new part, as the completed path keeps one part per block.
No producer sets ProviderMetadata or CompletedAt on text or reasoning deltas, and every reasoning delta in a run now has the same CreatedAt, so the merges never changed a value and no test reached the metadata branch. The run keeps its first delta's fields.
toolCall appended its placeholder directly and had to close the open run itself, and the merge helper returned a bool its name did not explain. appendAssistantPart now either merges a delta into the open run or closes the run and appends, so it is the only code that grows assistantParts.
closeStreamedRun resets the builder whenever a run was open, so the builder is already empty when a new run opens.
The 1 MiB episode buffer stops accepting seven-byte text deltas after about 2,585, so the 4,000-delta case measured an input the interrupt path cannot receive. Use 2,500, and b.Loop so setup stays out of the timing without ResetTimer.
On main each whitespace-only delta was its own part, and both the frontend and prompt replay skip whitespace-only text parts, so words and paragraphs ran together. Merged into the run, the whitespace is kept; the test now covers it.
The comments claimed parity with completed turns and metadata merging that the code does not do, described an earlier commit, and left out that the run's part holds stale text until the run closes. State the merge rule and the stale-text rule, drop the comment that restated closeStreamedRun, and say why the test feeds whitespace-only deltas.
…microsecond merge
An interrupted turn merged every adjacent text delta into one part with the first delta's fields, so narration followed by answer text was stored as one narration part. The web client folds narration into the working block, so the partial answer disappeared into it after the interrupt. A text run now ends when Narration changes, which is also where the web client splits streamed text.
cd5edc7 to
94698e8
Compare
…ect the episode-cap count The benchmark comment said a 1 MiB episode holds about 2,585 seven-byte deltas; on current main the part serializes to 385 bytes and the buffer charges 421 to 424 bytes, so it holds about 2,475. The whitespace test comment no longer says the frontend drops whitespace-only parts, which #30551 changed.
An interrupted turn is built from the buffered stream deltas and was stored with one
ChatMessagePartper delta, whileprocessStepStreamgives a completed turn one part per block. On a benchmark deployment around 2026-09-21, a 28,539-character interrupted response was stored as 3,987 text parts and 1 MB of JSON, and a/messagespage with interrupted turns was 11.7 MB, against 117 KB for 50 completed messages. On currentmain, a seven-character delta serializes to a 385-byte part. Interrupts are routine (the stop button, or sending while a turn runs), so long chats collect these rows. Whitespace-only deltas were also separate parts, which prompt replay drops, so words and paragraphs ran together in the next turn's prompt.bufferedPartsToPartialMessagesnow joins adjacent deltas of the same type into one part, which keeps the first delta's fields. A text run ends at a part of another type, at a tool result, or where thenarrationflag changes. Each reasoning delta'sCreatedAtis its block's start time, so a reasoning run ends whereCreatedAtchanges. The open run accumulates in astrings.Builder, so persisting N deltas is linear in N.Limits: text deltas have no block identity, so adjacent text blocks still merge, and two reasoning blocks that start in the same microsecond merge. Rows already stored are not rewritten.
These tests fail on
mainand pass here:TestProcessStepStreamSetsReasoningDeltaCreatedAtToBlockStartpasses on both; it checks thatprocessStepStreamsets each reasoning delta'sCreatedAtto its block's start time, which the reasoning split depends on.BenchmarkBufferedPartsToPartialMessages_StreamedTextDeltas, one call persisting N seven-byte deltas as one run:mainB/opmainns/opFound while investigating side-by-side chat rendering performance (#29596). Stream history replay on connect is #29662.
Test output on
main