Skip to content

feat: load whole chat turns per history page - #30524

Merged
ibetitsmike merged 8 commits into
mainfrom
mike/codagt-1372-whole-turn-pages
Oct 9, 2026
Merged

ibetitsmike merged 8 commits into
mainfrom
mike/codagt-1372-whole-turn-pages

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 8, 2026 •

Copy link
Copy Markdown
Collaborator

Closes CODAGT-1372.

With collapsed working blocks, a long agent turn renders as one row, but chat history pages hold a fixed 50 messages. A 400-step turn took about eight sequential page loads to fill, and its block showed partial labels ("at least", "or more") until then.

GET /api/v2/chats/{chat}/messages now returns turn_start_id on newest-first pages: the ID of the user prompt that starts the turn containing the page's oldest message. It comes from a new GetChatTurnStartID query on the existing idx_chat_messages_user_prompts index, so there is no migration. Soft-deleted prompts and model-only user rows never start a turn.

The history query then fills each page back to that prompt with requests of up to 200 messages, capped at 1000 messages per page, so a page holds whole turns and a complete block renders without partial labels. Past the cap, or when a fill request fails or returns no messages, the page keeps what loaded and ends mid-turn, and the next page continues it, as before. Because the fill runs inside the page fetch, a chat now waits for its fill requests (one to five) before it first renders: with an injected 500 ms per-request delay, a chat whose newest turn hits the cap appeared after 3.8 s instead of 1.1 s.

Screenshots

From the dogfood UAT, the same 200-step turn with its next history request held, so each label shows only what loaded before it:

Before, on the base commit 29e876df4da, after two 50-message pages: "Worked for at least 1m 35s (48 steps or more)", partly hidden by the loading indicator.

Before: partial label for a 200-step turn

After, on 186a9b053a0, after the first page: "Worked for 6m 39s (200 steps)".

After: exact label for the same 200-step turn

A 750-step turn, larger than one 1000-message page, is exact once the next page loads:

After: 750-step turn, expanded

A malformed fill response keeps the loaded messages and the partial label instead of showing "Failed to load chat", as ecdfb27ec29 did:

Malformed fill response: partial label, no error screen

Testing

  • Go: TestGetChatMessages_TurnStartID (pages spanning turns, deleted prompts, model-only user rows, polling and empty pages) and the dbauthz suite entry.
  • Vitest: fill request parameters, the 1000-message cap, failed, malformed, and empty fill responses, and servers without turn_start_id.
  • Dogfood UAT through Coder Agents: PASS on 186a9b053a0 (round 4): whole-turn first pages, the 1000-message cap and its continuation, fill failures, a live turn, and API checks for deleted prompts and model-only user rows. Round 3 on ecdfb27ec29 failed: a malformed fill failed the chat load, and an empty fill reporting has_more repeated forever. 7266f468290 fixes both, with regression tests.

Review record

  • Test audit with mutation checks: the Go test now covers a page spanning two turns, which catches a lookup keyed on the newest message. A re-audit of later test changes changed nothing; the one surviving mutation drops the !older.has_more stop, costing one extra request when the prompt is deleted mid-fill.
  • Frontend review (8 roles, 5 rounds): fixed doc, naming, loose-type, and test findings. Its suggestions to catch only the fill request (DF5) and drop the empty-page stop (SD4) caused the round 3 failures and are reverted. Kept: SD1 (MAX_MESSAGES_PER_REQUEST is used once, named like MESSAGES_PAGE_SIZE) and round 5 SD4 (!older.has_more is the endpoint's pagination signal). Declined DF6: the queued_messages refetch race predates this change, and a failed refetch fill only unloads older history that reloads on scroll.
  • Codex: no findings on 186a9b053a0.
  • Known: a hung fill delays the first render (no client timeout); fills are not cancelled when the user leaves; the top of history costs one extra empty request; injected malformed fill rows break the page as they do on the first page.

Xum opened this PR on behalf of @ibetitsmike.

Expose turn_start_id on newest-first chat message pages: the ID of the
user prompt that starts the turn containing the page's oldest message.
The chat history query extends each page back to that prompt, up to
1000 messages, so a collapsed working block loads in one page fetch.

Linear: CODAGT-1372
Address frontend review: a failed turn fill request now keeps the
messages already loaded instead of failing the page, and the helper and
tests are simplified.
Replace the edit-simulation fixture, which no mutation distinguished, with a page spanning two turns so a lookup keyed on the newest message instead of the oldest fails.
Only a failed fill request keeps the loaded page now; other errors in the fill loop fail the query. Name the endpoint's per-request limit, drop a loop guard the server contract makes unreachable, and shorten the turn_start_id field docs.
… docs

An empty fill response already reports has_more false, so has_more alone ends the loop. The field docs again name the case where no prompt starts the oldest message's turn.
@linear-code

linear-code Bot commented Oct 8, 2026

Copy link
Copy Markdown

CODAGT-1372

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-09T10:52:45.918057Z 596ca83 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: ecdfb27ec2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…is unusable

Remote UAT found two regressions from narrowing the fill error handling: a fill response whose body is not a messages page failed the whole chat, and an empty page that still reported has_more repeated the same request without end. Guard the fill request together with the use of its response, and stop on an empty page.
…ose types

Model the unusable fill body as a plain object behind one explicit cast instead of JSON.parse, fail the fill after a successful one so the test checks that loaded fill rows survive, reset the mock so unexpected requests behave the same in any test order, and state why the page keeps the first response's has_more.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Delightful!

Reviewed commit: 186a9b053a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike marked this pull request as ready for review October 9, 2026 09:11
@pr-queue-manager
pr-queue-manager Bot requested a review from ThomasK33 October 9, 2026 09:12
Comment thread coderd/database/queries/chats.sql Outdated
Comment on lines +534 to +536
-- Returns the newest live user prompt at or before the message, which is
-- the prompt that starts the message's turn. Backed by
-- idx_chat_messages_user_prompts.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yap

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed it in 596ca83; the query reads fine without it.

Xum replied on behalf of @ibetitsmike.

Comment thread coderd/exp_chats_test.go Outdated
func TestGetChatMessages_TurnStartID(t *testing.T) {
t.Parallel()

ctx := testutil.Context(t, testutil.WaitLong)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ctx should be declared directly above first usage

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

will fix

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Moved ctx directly above the soft delete, its first use, in 596ca83.

Xum replied on behalf of @ibetitsmike.

Comment on lines +1507 to +1551
const fetchMessagesPage = async (
chatId: string,
beforeId: number | undefined,
): Promise<TypesGen.ChatMessagesResponse> => {
const page = await API.experimental.getChatMessages(chatId, {
before_id: beforeId,
limit: MESSAGES_PAGE_SIZE,
});
const turnStartId = page.turn_start_id;
if (turnStartId === undefined) {
return page;
}

const messages = [...page.messages];
while (messages.length < MAX_MESSAGES_PER_PAGE) {
const oldestId = messages[messages.length - 1].id;
if (oldestId <= turnStartId) {
break;
}
try {
const older = await API.experimental.getChatMessages(chatId, {
// after_id is exclusive, so this keeps the prompt itself.
after_id: turnStartId - 1,
before_id: oldestId,
limit: Math.min(
MAX_MESSAGES_PER_REQUEST,
MAX_MESSAGES_PER_PAGE - messages.length,
),
});
messages.push(...older.messages);
// An empty page cannot move the cursor, whatever has_more says.
if (!older.has_more || older.messages.length === 0) {
break;
}
} catch {
// A failed fill keeps what loaded: the page ends mid-turn and the
// next page resumes from its oldest message.
break;
}
}

// Keep the first response's has_more; a fill's has_more covers only this
// turn.
return { ...page, messages };
};

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious, is there a better way to do this with React Query? We're basically bypassing any React Query cache of getChatMessages here. Maybe this is absolutely fine because we're using this in chatMessagesForInfiniteScroll. Unfortunately my memory of this area is a little poor

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll check

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we're bypassing anything here. getChatMessages has no per-request cache; the only cache for history is the infinite query under chatMessagesKey. These extra requests are just how one page gets built, and the merged page lands in that cache like before. Refetch and invalidation also rerun this function for every loaded page, so pages always start at their turn's prompt.

The React Query-native version would be one request per page with the component calling fetchNextPage until we reach turn_start_id. That would render a half-loaded turn between requests, which is the "at least N steps" label this PR fixes.

One gap is real: we didn't pass the query's abort signal, so leaving a chat didn't cancel requests still in flight. That predates this PR (the single request didn't pass one either). #30542, stacked on this PR, threads the signal into the first request and every fill request, so cancelling the query now aborts them.

Xum replied on behalf of @ibetitsmike.

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. More of your lovely PRs please.

Reviewed commit: 596ca83fed

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@DanielleMaywood DanielleMaywood left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once this has merged I think we should verify that GetChatTurnStartID has good performance characteristics in very long chats. (I'm assuming it should be don't wanna make any assumptions)

@ibetitsmike
ibetitsmike added this pull request to stack #30543 October 9, 2026 11:00
@ibetitsmike
ibetitsmike merged commit 782fb30 into main Oct 9, 2026
32 checks passed
@ibetitsmike
ibetitsmike deleted the mike/codagt-1372-whole-turn-pages branch October 9, 2026 11:32
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 9, 2026
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

Once this has merged I think we should verify that GetChatTurnStartID has good performance characteristics in very long chats. (I'm assuming it should be don't wanna make any assumptions)

Yup, queuing it up after deployment @DanielleMaywood

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants