Repository navigation
refactor(coderd/x/chatd): delete the CompactionOptions and compactionOutcome mirrors - #30984
Conversation
Compaction is the recovery path for a chat near its context window, but its summary request sent the whole history, the tool definitions, and a larger output cap than the turn that overflowed. The tool-less retry only ran for HTTP 400 rejections, so a context-window error reported inside an HTTP 200 stream (OpenAI Responses API, Bedrock Mantle) wedged the chat: every turn and every /compact failed the same way. - Detect context-window rejections reported inside a stream: fantasy stream errors without a status code and Responses error events. - When the request without tools is still too large, retry with the largest tool results cut to a common size. Send that request first when the full request would leave under 16k tokens for the summary. - Classify a conversation that still does not fit as a non-retryable config error that tells the user how to recover. - Count the messages after the last step, usually its tool results, in the compaction trigger, so a large tool result compacts the history before it reaches the model. - Cap a single tool result at 256 KB on large-context models. - Record the summary request shape on the compaction span and log each rejected request.
…igger Follow-ups from a review of the compaction overflow fix: - Count pending user messages in the compaction trigger, so a large pasted message compacts the history before it reaches the model. The summary request still excludes them because they follow the compacted history. - Log only summary requests the model rejects for their size, under the span attribute keys, and stop after a rejected elided request. - Rename the span attributes to summary_requests_sent and summary_request_kind, type the request kind, and unexport the fields GenerateCompaction derives. - Tell the user that editing a message deletes every message after it. - Cover an elided request on a summary window smaller than the history, a non-transient in-stream error, the rejection log, and the 256 KB cap.
…timate The compaction trigger and the summary estimate counted only the messages after the last assistant message. An interrupted step persists its partial response without usage, so the estimate skipped that response and every message between it and the step that reported the usage. Both now start after the assistant message whose step reported the usage. isContextTooLargeError now trusts the error code fantasy appends to an OpenAI Responses failure. A rate limit or server error whose text mentions the context window no longer turns into a non-retryable compaction error. Also adds tests for when the elided summary request goes first and for case-insensitive overflow wording.
… windows The elided summary request halved the summary model's window for the history, then added the summary prompt and the pre_compact hint. With a 16 KiB hint in a window under about 13k tokens, the smallest request could not fit. The history now gets half of the window those leave. Tool error results are now elided like text results and stay errors, so large error output no longer makes a conversation unrecoverable. Text and other non-image, non-PDF attachments now count by size in the compaction estimate. Images and PDFs keep the per-part estimate because providers bill them by dimensions and pages.
…ummary When a compaction override model omits the chat's tool definitions, the elide-first check still counted them through the trigger step's usage. Large definitions could make compaction elide tool results before it tried the full history, which would have fit without them. The check now leaves the omitted definitions out before it elides. While the full history fits either way, the output cap still counts them, which keeps a margin for providers that reject input plus max_tokens over the window.
…s twice When a step reports only TotalTokens and OutputTokens, the context tokens fall back to the total, which already includes the output. The summary estimate added the output again, so it could elide tool results that fit. A tool call that publishes attachments persists them as an assistant message without usage, and the prompt omits that message. The trigger counted it as an unmeasured assistant message, so the trailing estimate started before the measured step and counted its prompt again. Only assistant messages that the prompt replays now count.
…izing A summary override model sanitizes the prompt, and that can drop an assistant message, such as an interrupted OpenAI response that holds only reasoning. The summary estimate then applied the original count of unmeasured assistants to the sanitized prompt. It counted history that the measured step already covers, so the estimate could exceed the window and elide tool results that fit. The preparer now estimates the trailing input from the original prompt and passes it as SummaryTrailingTokens.
…rompt The trigger left out the measured step's output, which the next request replays, so a large tool call could push that request past the threshold without compacting first. The trigger now adds the output to the trailing estimate, as the context gauge in the UI does. The post-compaction over-limit check still reads only the measured prompt. The prompt for an Anthropic model drops an assistant message that holds only provider tool history it cannot replay, such as an interrupted web search call without its result. latestPromptUsage still counted that message as unmeasured, so the trailing estimate counted the measured step's history again and compacted early. It now sanitizes each unmeasured message the same way before it counts it.
… the window When the summary requests omit the tool definitions, GenerateCompaction subtracted their estimate only when that alone cleared the elide-first check. Near the window the definitions then stayed in the estimate, so the output cap could not clamp to the room the tool-less request leaves: a 110k estimate with 20k of omitted definitions in a 100k window kept the configured cap instead of about 10k. Subtract them whenever the elide-first check fires and decide it again. They still stay in the estimate as a margin while the summary has room to spare.
…allback The elided summary request now cuts large text attachments the same way it cuts local tool results, keeping their filename and media type. Images and PDFs are left as they are. A history whose size comes from attached documents can now be summarized without the user editing earlier messages.
…llback Providers that reject text attachments get them inlined as user text before compaction runs, so the elided request now also cuts large text parts in user messages. System and assistant text are not cut.
…imating them Drop the token estimates this branch added (the trailing-token trigger, the elide-first check, the per-request summary cap, and the elision budget) and the 256 KB tool result cap. When the model rejects a summary request for its size, the next request is thinner: first without tool definitions, then with tool results cut to 32 KiB, then to 4 KiB. A step that cuts nothing is skipped. Only locally executed text and error tool results are cut. The in-stream rejection match, the non-retryable recovery error, and the Warn log for each rejected request stay.
…of GenerateCompactionOptions normalizedCompactionGenerateConfig copied 21 fields of GenerateCompactionOptions one by one into CompactionOptions, which only the compaction helpers read, so each new option (most recently Logger) had to be added to both structs and to the copier. The helpers now take GenerateCompactionOptions, and normalizedCompactionGenerateConfig applies its defaults to its parameter, which is already a copy. GenerateCompaction still reports the requested ThresholdPercent and ContextLimit from its own opts. TestGenerateCompactionSummaryUsesToolDefinitions repeated TestGenerateCompactionSummaryPreservesCallOptions. Its one extra input, a tool description, moves into the kept test.
…mpactionOutcome compactionOutcome repeated the fields of chatloop.CompactionResult so that generateCompaction could convert one into the other. buildCompactionMessagesInput now holds the chatloop.CompactionResult itself.
|
@codex review
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex Review: Didn't find any major issues. Another round soon, please! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review
|
|
Codex Review: Didn't find any major issues. Chef's kiss. Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
|
@codex review
|
|
Codex Review: Didn't find any major issues. 👍 Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Deletes two structs that each repeat another struct field for field.
chatloop.CompactionOptionsrepeated 21 fields ofGenerateCompactionOptions, andnormalizedCompactionGenerateConfigcopied them one by one, so each new option had to be added in three places, as #30766 did forLogger.compactionOutcomeinchatdrepeated the 11 fields ofchatloop.CompactionResultonly so thatgenerateCompactioncould convert one into the other. The PR also deletesTestGenerateCompactionSummaryUsesToolDefinitions, which repeatsTestGenerateCompactionSummaryPreservesCallOptions.Why it is safe
publishCompactionError,startCompactionDebugRun, andgenerateCompactionSummarynow takeGenerateCompactionOptions. Every read of the normalized config, inGenerateCompactionand in those helpers, uses one of the 21 fields that the copier copied without change. None reads the 7 fields that onlyGenerateCompactionOptionshas.normalizedCompactionGenerateConfigapplies the same four defaults and the same threshold gate to its parameter, which is already a copy. SoGenerateCompactionstill reports the requestedThresholdPercentandContextLimitfrom its ownopts. If it readconfig.ThresholdPercentinstead,TestGenerateCompaction_RecordsRequestedTriggerwould fail.compactionOutcome(outcome)conversion only because the two structs have identical fields, sobuildCompactionMessagesInputcan hold thechatloop.CompactionResultitself. The fourTestBuildCompactionMessages_*tests that build it keep their assertions.git grep -w CompactionOptionsandgit grep compactionOutcomefind no matches.Review focus
generateCompactionSummarykeeps itsmodelandmessagesparameters, so its 15 direct test calls change only the type name. It now also receivesModelandMessagesfields with the same values, and it does not read them.Validation
go test -race -count=1 ./coderd/x/chatd/chatloop/passes.go test -count=1 -p 4 -parallel 4 ./coderd/x/chatd/...passes (20 packages).golangci-lint run ./coderd/x/chatd/...reports 0 issues, and the pre-commit hook passed.Stack
#30766 has merged, so this PR now targets
mainby itself.