Skip to content

refactor(coderd/x/chatd): delete the CompactionOptions and compactionOutcome mirrors - #30984

Merged
ibetitsmike merged 17 commits into
mainfrom
mike/chatd-compaction-mirrors
Oct 20, 2026
Merged

ibetitsmike merged 17 commits into
mainfrom
mike/chatd-compaction-mirrors

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Deletes two structs that each repeat another struct field for field. chatloop.CompactionOptions repeated 21 fields of GenerateCompactionOptions, and normalizedCompactionGenerateConfig copied them one by one, so each new option had to be added in three places, as #30766 did for Logger. compactionOutcome in chatd repeated the 11 fields of chatloop.CompactionResult only so that generateCompaction could convert one into the other. The PR also deletes TestGenerateCompactionSummaryUsesToolDefinitions, which repeats TestGenerateCompactionSummaryPreservesCallOptions.

Why it is safe

  • publishCompactionError, startCompactionDebugRun, and generateCompactionSummary now take GenerateCompactionOptions. Every read of the normalized config, in GenerateCompaction and in those helpers, uses one of the 21 fields that the copier copied without change. None reads the 7 fields that only GenerateCompactionOptions has.
  • normalizedCompactionGenerateConfig applies the same four defaults and the same threshold gate to its parameter, which is already a copy. So GenerateCompaction still reports the requested ThresholdPercent and ContextLimit from its own opts. If it read config.ThresholdPercent instead, TestGenerateCompaction_RecordsRequestedTrigger would fail.
  • Go allows the compactionOutcome(outcome) conversion only because the two structs have identical fields, so buildCompactionMessagesInput can hold the chatloop.CompactionResult itself. The four TestBuildCompactionMessages_* tests that build it keep their assertions.
  • Six mutants of the summary call each fail the deleted test on fix(coderd/x/chatd): make compaction fit an overflowing context #30766: no tools, no call options, no summary prompt, one tool dropped, the caller's history changed, and tool descriptions stripped. The kept test failed only the first five, because its tool had no description. With that description moved into it, the kept test fails all six.
  • git grep -w CompactionOptions and git grep compactionOutcome find no matches.

Review focus

generateCompactionSummary keeps its model and messages parameters, so its 15 direct test calls change only the type name. It now also receives Model and Messages fields with the same values, and it does not read them.

Validation

  • go test -race -count=1 ./coderd/x/chatd/chatloop/ passes.
  • go test -count=1 -p 4 -parallel 4 ./coderd/x/chatd/... passes (20 packages).
  • golangci-lint run ./coderd/x/chatd/... reports 0 issues, and the pre-commit hook passed.

Stack

#30766 has merged, so this PR now targets main by itself.

Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description.

Compaction is the recovery path for a chat near its context window, but
its summary request sent the whole history, the tool definitions, and a
larger output cap than the turn that overflowed. The tool-less retry
only ran for HTTP 400 rejections, so a context-window error reported
inside an HTTP 200 stream (OpenAI Responses API, Bedrock Mantle) wedged
the chat: every turn and every /compact failed the same way.

- Detect context-window rejections reported inside a stream: fantasy
  stream errors without a status code and Responses error events.
- When the request without tools is still too large, retry with the
  largest tool results cut to a common size. Send that request first
  when the full request would leave under 16k tokens for the summary.
- Classify a conversation that still does not fit as a non-retryable
  config error that tells the user how to recover.
- Count the messages after the last step, usually its tool results, in
  the compaction trigger, so a large tool result compacts the history
  before it reaches the model.
- Cap a single tool result at 256 KB on large-context models.
- Record the summary request shape on the compaction span and log each
  rejected request.
…igger

Follow-ups from a review of the compaction overflow fix:

- Count pending user messages in the compaction trigger, so a large
  pasted message compacts the history before it reaches the model. The
  summary request still excludes them because they follow the compacted
  history.
- Log only summary requests the model rejects for their size, under the
  span attribute keys, and stop after a rejected elided request.
- Rename the span attributes to summary_requests_sent and
  summary_request_kind, type the request kind, and unexport the fields
  GenerateCompaction derives.
- Tell the user that editing a message deletes every message after it.
- Cover an elided request on a summary window smaller than the history,
  a non-transient in-stream error, the rejection log, and the 256 KB cap.
…timate

The compaction trigger and the summary estimate counted only the messages
after the last assistant message. An interrupted step persists its partial
response without usage, so the estimate skipped that response and every
message between it and the step that reported the usage. Both now start
after the assistant message whose step reported the usage.

isContextTooLargeError now trusts the error code fantasy appends to an
OpenAI Responses failure. A rate limit or server error whose text mentions
the context window no longer turns into a non-retryable compaction error.

Also adds tests for when the elided summary request goes first and for
case-insensitive overflow wording.
… windows

The elided summary request halved the summary model's window for the
history, then added the summary prompt and the pre_compact hint. With a
16 KiB hint in a window under about 13k tokens, the smallest request could
not fit. The history now gets half of the window those leave.

Tool error results are now elided like text results and stay errors, so
large error output no longer makes a conversation unrecoverable.

Text and other non-image, non-PDF attachments now count by size in the
compaction estimate. Images and PDFs keep the per-part estimate because
providers bill them by dimensions and pages.
…ummary

When a compaction override model omits the chat's tool definitions, the
elide-first check still counted them through the trigger step's usage.
Large definitions could make compaction elide tool results before it
tried the full history, which would have fit without them.

The check now leaves the omitted definitions out before it elides. While
the full history fits either way, the output cap still counts them, which
keeps a margin for providers that reject input plus max_tokens over the
window.
…s twice

When a step reports only TotalTokens and OutputTokens, the context tokens
fall back to the total, which already includes the output. The summary
estimate added the output again, so it could elide tool results that fit.

A tool call that publishes attachments persists them as an assistant
message without usage, and the prompt omits that message. The trigger
counted it as an unmeasured assistant message, so the trailing estimate
started before the measured step and counted its prompt again. Only
assistant messages that the prompt replays now count.
…izing

A summary override model sanitizes the prompt, and that can drop an
assistant message, such as an interrupted OpenAI response that holds
only reasoning. The summary estimate then applied the original count of
unmeasured assistants to the sanitized prompt. It counted history that
the measured step already covers, so the estimate could exceed the
window and elide tool results that fit.

The preparer now estimates the trailing input from the original prompt
and passes it as SummaryTrailingTokens.
…rompt

The trigger left out the measured step's output, which the next request
replays, so a large tool call could push that request past the
threshold without compacting first. The trigger now adds the output to
the trailing estimate, as the context gauge in the UI does. The
post-compaction over-limit check still reads only the measured prompt.

The prompt for an Anthropic model drops an assistant message that holds
only provider tool history it cannot replay, such as an interrupted web
search call without its result. latestPromptUsage still counted that
message as unmeasured, so the trailing estimate counted the measured
step's history again and compacted early. It now sanitizes each
unmeasured message the same way before it counts it.
… the window

When the summary requests omit the tool definitions, GenerateCompaction
subtracted their estimate only when that alone cleared the elide-first
check. Near the window the definitions then stayed in the estimate, so
the output cap could not clamp to the room the tool-less request leaves:
a 110k estimate with 20k of omitted definitions in a 100k window kept
the configured cap instead of about 10k.

Subtract them whenever the elide-first check fires and decide it again.
They still stay in the estimate as a margin while the summary has room
to spare.
…allback

The elided summary request now cuts large text attachments the same way
it cuts local tool results, keeping their filename and media type.
Images and PDFs are left as they are. A history whose size comes from
attached documents can now be summarized without the user editing
earlier messages.
…llback

Providers that reject text attachments get them inlined as user text
before compaction runs, so the elided request now also cuts large text
parts in user messages. System and assistant text are not cut.
…imating them

Drop the token estimates this branch added (the trailing-token trigger,
the elide-first check, the per-request summary cap, and the elision
budget) and the 256 KB tool result cap. When the model rejects a summary
request for its size, the next request is thinner: first without tool
definitions, then with tool results cut to 32 KiB, then to 4 KiB. A step
that cuts nothing is skipped. Only locally executed text and error tool
results are cut.

The in-stream rejection match, the non-retryable recovery error, and the
Warn log for each rejected request stay.
…of GenerateCompactionOptions

normalizedCompactionGenerateConfig copied 21 fields of
GenerateCompactionOptions one by one into CompactionOptions, which only
the compaction helpers read, so each new option (most recently Logger)
had to be added to both structs and to the copier. The helpers now take
GenerateCompactionOptions, and normalizedCompactionGenerateConfig applies
its defaults to its parameter, which is already a copy. GenerateCompaction
still reports the requested ThresholdPercent and ContextLimit from its
own opts.

TestGenerateCompactionSummaryUsesToolDefinitions repeated
TestGenerateCompactionSummaryPreservesCallOptions. Its one extra input, a
tool description, moves into the kept test.
…mpactionOutcome

compactionOutcome repeated the fields of chatloop.CompactionResult so
that generateCompaction could convert one into the other.
buildCompactionMessagesInput now holds the chatloop.CompactionResult
itself.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, posted this comment for @ibetitsmike.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T23:21:50.059233Z 034b20e Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 792b20b3c8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, posted this comment for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Chef's kiss.

Reviewed commit: a64d2477ca

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike marked this pull request as ready for review October 19, 2026 12:17
Base automatically changed from mike/chatd-compaction-fit to main October 19, 2026 22:30
main now has #30766 as squash commit 8245457. The conflicts in
compaction.go and compaction_internal_test.go keep this branch's side,
except compactionContextTooLargeMessage, which keeps main's /clear text.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, posted this comment for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 034b20eb76

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike merged commit 63c116f into main Oct 20, 2026
33 checks passed
@ibetitsmike
ibetitsmike deleted the mike/chatd-compaction-mirrors branch October 20, 2026 09:30
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants