Repository navigation
feat: load whole chat turns per history page - #30524
Conversation
Expose turn_start_id on newest-first chat message pages: the ID of the user prompt that starts the turn containing the page's oldest message. The chat history query extends each page back to that prompt, up to 1000 messages, so a collapsed working block loads in one page fetch. Linear: CODAGT-1372
Address frontend review: a failed turn fill request now keeps the messages already loaded instead of failing the page, and the helper and tests are simplified.
Replace the edit-simulation fixture, which no mutation distinguished, with a page spanning two turns so a lookup keyed on the newest message instead of the oldest fails.
Only a failed fill request keeps the loaded page now; other errors in the fill loop fail the query. Name the endpoint's per-request limit, drop a loop guard the server contract makes unreachable, and shorten the turn_start_id field docs.
… docs An empty fill response already reports has_more false, so has_more alone ends the loop. The field docs again name the case where no prompt starts the oldest message's turn.
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…is unusable Remote UAT found two regressions from narrowing the fill error handling: a fill response whose body is not a messages page failed the whole chat, and an empty page that still reported has_more repeated the same request without end. Guard the fill request together with the use of its response, and stop on an empty page.
…ose types Model the unusable fill body as a plain object behind one explicit cast instead of JSON.parse, fail the fill after a successful one so the test checks that loaded fill rows survive, reset the mock so unexpected requests behave the same in any test order, and state why the page keeps the first response's has_more.
|
@codex review |
|
Codex Review: Didn't find any major issues. Delightful! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
| -- Returns the newest live user prompt at or before the message, which is | ||
| -- the prompt that starts the message's turn. Backed by | ||
| -- idx_chat_messages_user_prompts. |
There was a problem hiding this comment.
Removed it in 596ca83; the query reads fine without it.
Xum replied on behalf of @ibetitsmike.
| func TestGetChatMessages_TurnStartID(t *testing.T) { | ||
| t.Parallel() | ||
|
|
||
| ctx := testutil.Context(t, testutil.WaitLong) |
There was a problem hiding this comment.
ctx should be declared directly above first usage
There was a problem hiding this comment.
Moved ctx directly above the soft delete, its first use, in 596ca83.
Xum replied on behalf of @ibetitsmike.
| const fetchMessagesPage = async ( | ||
| chatId: string, | ||
| beforeId: number | undefined, | ||
| ): Promise<TypesGen.ChatMessagesResponse> => { | ||
| const page = await API.experimental.getChatMessages(chatId, { | ||
| before_id: beforeId, | ||
| limit: MESSAGES_PAGE_SIZE, | ||
| }); | ||
| const turnStartId = page.turn_start_id; | ||
| if (turnStartId === undefined) { | ||
| return page; | ||
| } | ||
|
|
||
| const messages = [...page.messages]; | ||
| while (messages.length < MAX_MESSAGES_PER_PAGE) { | ||
| const oldestId = messages[messages.length - 1].id; | ||
| if (oldestId <= turnStartId) { | ||
| break; | ||
| } | ||
| try { | ||
| const older = await API.experimental.getChatMessages(chatId, { | ||
| // after_id is exclusive, so this keeps the prompt itself. | ||
| after_id: turnStartId - 1, | ||
| before_id: oldestId, | ||
| limit: Math.min( | ||
| MAX_MESSAGES_PER_REQUEST, | ||
| MAX_MESSAGES_PER_PAGE - messages.length, | ||
| ), | ||
| }); | ||
| messages.push(...older.messages); | ||
| // An empty page cannot move the cursor, whatever has_more says. | ||
| if (!older.has_more || older.messages.length === 0) { | ||
| break; | ||
| } | ||
| } catch { | ||
| // A failed fill keeps what loaded: the page ends mid-turn and the | ||
| // next page resumes from its oldest message. | ||
| break; | ||
| } | ||
| } | ||
|
|
||
| // Keep the first response's has_more; a fill's has_more covers only this | ||
| // turn. | ||
| return { ...page, messages }; | ||
| }; |
There was a problem hiding this comment.
I'm curious, is there a better way to do this with React Query? We're basically bypassing any React Query cache of getChatMessages here. Maybe this is absolutely fine because we're using this in chatMessagesForInfiniteScroll. Unfortunately my memory of this area is a little poor
There was a problem hiding this comment.
I don't think we're bypassing anything here. getChatMessages has no per-request cache; the only cache for history is the infinite query under chatMessagesKey. These extra requests are just how one page gets built, and the merged page lands in that cache like before. Refetch and invalidation also rerun this function for every loaded page, so pages always start at their turn's prompt.
The React Query-native version would be one request per page with the component calling fetchNextPage until we reach turn_start_id. That would render a half-loaded turn between requests, which is the "at least N steps" label this PR fixes.
One gap is real: we didn't pass the query's abort signal, so leaving a chat didn't cancel requests still in flight. That predates this PR (the single request didn't pass one either). #30542, stacked on this PR, threads the signal into the first request and every fill request, so cancelling the query now aborts them.
Xum replied on behalf of @ibetitsmike.
|
@codex review |
|
Codex Review: Didn't find any major issues. More of your lovely PRs please. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
DanielleMaywood
left a comment
There was a problem hiding this comment.
Once this has merged I think we should verify that GetChatTurnStartID has good performance characteristics in very long chats. (I'm assuming it should be don't wanna make any assumptions)
Yup, queuing it up after deployment @DanielleMaywood |
Closes CODAGT-1372.
With collapsed working blocks, a long agent turn renders as one row, but chat history pages hold a fixed 50 messages. A 400-step turn took about eight sequential page loads to fill, and its block showed partial labels ("at least", "or more") until then.
GET /api/v2/chats/{chat}/messagesnow returnsturn_start_idon newest-first pages: the ID of the user prompt that starts the turn containing the page's oldest message. It comes from a newGetChatTurnStartIDquery on the existingidx_chat_messages_user_promptsindex, so there is no migration. Soft-deleted prompts and model-only user rows never start a turn.The history query then fills each page back to that prompt with requests of up to 200 messages, capped at 1000 messages per page, so a page holds whole turns and a complete block renders without partial labels. Past the cap, or when a fill request fails or returns no messages, the page keeps what loaded and ends mid-turn, and the next page continues it, as before. Because the fill runs inside the page fetch, a chat now waits for its fill requests (one to five) before it first renders: with an injected 500 ms per-request delay, a chat whose newest turn hits the cap appeared after 3.8 s instead of 1.1 s.
Screenshots
From the dogfood UAT, the same 200-step turn with its next history request held, so each label shows only what loaded before it:
Before, on the base commit
29e876df4da, after two 50-message pages: "Worked for at least 1m 35s (48 steps or more)", partly hidden by the loading indicator.After, on
186a9b053a0, after the first page: "Worked for 6m 39s (200 steps)".A 750-step turn, larger than one 1000-message page, is exact once the next page loads:
A malformed fill response keeps the loaded messages and the partial label instead of showing "Failed to load chat", as
ecdfb27ec29did:Testing
TestGetChatMessages_TurnStartID(pages spanning turns, deleted prompts, model-only user rows, polling and empty pages) and the dbauthz suite entry.turn_start_id.186a9b053a0(round 4): whole-turn first pages, the 1000-message cap and its continuation, fill failures, a live turn, and API checks for deleted prompts and model-only user rows. Round 3 onecdfb27ec29failed: a malformed fill failed the chat load, and an empty fill reportinghas_morerepeated forever.7266f468290fixes both, with regression tests.Review record
!older.has_morestop, costing one extra request when the prompt is deleted mid-fill.MAX_MESSAGES_PER_REQUESTis used once, named likeMESSAGES_PAGE_SIZE) and round 5 SD4 (!older.has_moreis the endpoint's pagination signal). Declined DF6: thequeued_messagesrefetch race predates this change, and a failed refetch fill only unloads older history that reloads on scroll.186a9b053a0.