Skip to content

refactor: delete chatdebug FinalizeStale and the stale threshold - #30989

Merged
ibetitsmike merged 6 commits into
mainfrom
mike/chatdebug-delete-finalize-stale
Oct 20, 2026
Merged

ibetitsmike merged 6 commits into
mainfrom
mike/chatdebug-delete-finalize-stale

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Deletes chatdebug.Service.FinalizeStale and everything that existed only for it. Nothing in production calls FinalizeStale since #26270, and #30988 removed the heartbeats that kept live runs out of its reach. This is the second of three PRs that delete the stale debug-run sweep.

  • chatdebug/service.go: deletes FinalizeStale, SetStaleAfter, DefaultStaleThreshold, the staleAfterNanos field and WithClock. The service keeps its real clock for the timestamps it writes.
  • chatd.go: the debug service factory no longer calls SetStaleAfter(inFlightChatStaleAfter * 3). inFlightChatStaleAfter stays, because the chat worker uses it.
  • chatdebug/types.go: deletes EventKindFinalize and BroadcastPubsubChannel. FinalizeStale was the only publisher without a chat ID, so PubsubChannel loses its uuid.Nil branch. Nothing in production subscribes to chat_debug:* channels.
  • Deletes the four FinalizeStale service tests, and TestFinalizeStaleChatDebugRows_TerminalStatusAlignment together with codersdk.ChatDebugTerminalStatuses, which only that test used. The test compared that list with the NOT IN lists in FinalizeStaleChatDebugRows and nothing else: no other debug query or constraint lists the statuses. The next PR deletes that query.
  • Comments that named the stale sweep now say what happens instead: debugCleanupClockSkew, errRunFinalized, CreateStep, UpdateStep, the InsertChatDebugStep and UpdateChatDebugStep queries, the edit cleanup in chat_mutations.go, the two notes on removed tests in chatd_internal_test.go, and four comments in chatd_test.go.

Why it is safe: FinalizeStale and SetStaleAfter had no production caller other than the factory line this PR removes, and nothing calls WithClock after #30988. make gen/db changes only the regenerated query comments. #30988 explains why no reader of updated_at needs the sweep and why dbpurge cannot delete a live run.

Review focus: the debugCleanupClockSkew comment in chatd_debug.go. The value stays 30 * time.Second. Runs that start inside that buffer survive the fast cleanup; the old comment said the stale sweep would then finalize them, but dbpurge deletes them once they age past the debug retention window.

Validation: make gen/db, go build ./..., and go vet on the touched packages. go test -count=1 passes for ./coderd/x/chatd/chatdebug/..., ./coderd/x/chatd/ -run 'Debug|EditMessage|ArchiveChat', ./coderd/database/ -run 'ChatDebug|TestIsAuthorizedQuery|TestInsertChatMessagesOrderContract|ConnectionLogs' and ./codersdk/. The pre-commit hook passed.

Stack

This stack deletes the stale debug-run sweep, then simplifies the chatdebug debug-run, stream wrapper and step numbering code, drops the indexes the sweep used, and last deletes a runner debug field that lost its only reader with the heartbeats. The debug-run and stream wrapper refactors each sit on a test-only layer that pins the behavior they keep.

  1. refactor(coderd): delete the chat debug run and step heartbeats #30988 Delete the chat debug run and step heartbeats
  2. This PR: Delete FinalizeStale and the stale threshold
  3. refactor: delete the FinalizeStaleChatDebugRows query #30995 Delete the FinalizeStaleChatDebugRows query
  4. test(coderd/x/chatd): pin the debug runs that turns, titles and status labels record #30933 Pin the debug runs that turns, titles and status labels record
  5. refactor(coderd/x/chatd): create and finalize debug runs through one helper #30934 Create and finalize debug runs through one helper
  6. test(coderd/x/chatd/chatdebug): table the stream wrapper tests and cover their status rules #30980 Table the stream wrapper tests and cover their status rules
  7. refactor(coderd/x/chatd/chatdebug): share one finalize-once lifecycle between the stream wrappers #30981 Share one finalize-once lifecycle between the stream wrappers
  8. refactor(coderd): number chat debug steps in the database #31007 Number chat debug steps in the database
  9. refactor(coderd/x/chatd/chatdebug): delete the process-global step counters #31008 Delete the process-global step counters
  10. chore(coderd/database): drop the chat debug stale-sweep indexes #31009 Drop the chat debug stale-sweep indexes
  11. refactor(coderd/x/chatd): drop the unused runnerCtx from the runner debug turn #31010 Drop the unused runnerCtx from the runner debug turn

Xum, an AI agent, opened this PR for @ibetitsmike and wrote this description.

The heartbeats only refreshed updated_at on in-progress debug runs and
steps so that FinalizeStale would not sweep them. FinalizeStale has no
production caller, so the heartbeats kept nothing alive. Delete both
launchers, Service.TouchStep, Service.TouchRun, the threshold-change
wake-up channel, and the TouchChatDebugStepAndRun and
TouchChatDebugRunUpdatedAt queries with their authz wrappers.
FinalizeStale has no production caller. The stale threshold,
SetStaleAfter, WithClock, the finalize event and its broadcast channel
existed only for it and its tests. Delete them, the SetStaleAfter call
in chatd.go, and the terminal-status alignment test together with
codersdk.ChatDebugTerminalStatuses, which only that test used. Comments
that pointed at the stale sweep now describe dbpurge instead.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-19T12:36:07.906592Z 1ae01fd Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 0c393e59b8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…he WithClock doc

This PR deletes the heartbeat tickers, so the WithClock doc no longer
names them.
…alize-stale

# Conflicts:
#	coderd/x/chatd/chatdebug/service.go
… stale sweep

The edit cleanup comment and the notes on removed archive and stop
tests named the stale-finalization sweep, which this PR deletes. The
edit cleanup test now scopes its "only DeleteAfterMessageID removes
the run" argument to the test, since the chat delete cascade and
dbpurge also remove runs.
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: 1ae01fd7aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ibetitsmike
ibetitsmike marked this pull request as ready for review October 19, 2026 12:13
@pr-queue-manager
pr-queue-manager Bot requested a review from ThomasK33 October 19, 2026 12:14

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1ae01fd7aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread codersdk/chats.go Resolved
@ibetitsmike

Copy link
Copy Markdown
Collaborator Author

@codex review

Xum, an AI agent, requested this review for @ibetitsmike.

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Another round soon, please!

Reviewed commit: 1ae01fd7aa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Base automatically changed from mike/chatdebug-delete-heartbeats to main October 20, 2026 20:32
ibetitsmike added a commit that referenced this pull request Oct 20, 2026
Deletes the chat debug heartbeats. Model calls and streams started a
goroutine that touched the step and its run on a ticker, and runner
debug turns started a second one that touched the run. Both existed so
that `FinalizeStale` would not mark a live run as interrupted, but
nothing in production calls `FinalizeStale` since #26270. Mike chose to
delete the stale sweep as dead code. This PR is the first of three that
do it.

- `chatdebug/model.go`: `Generate`, `GenerateObject` and both stream
wrappers no longer start `launchHeartbeat`.
- `active_turn_debug.go`: the runner debug turn no longer starts
`LaunchRunHeartbeat`.
- `chatdebug/service.go`: deletes `TouchStep`, `TouchRun`,
`LaunchRunHeartbeat`, `heartbeatInterval`, and the channel that
`SetStaleAfter` closed to wake the heartbeats. The `WithClock` doc no
longer names the heartbeat tickers.
- Deletes the `TouchChatDebugStepAndRun` and
`TouchChatDebugRunUpdatedAt` queries with their dbauthz wrappers and
tests, and `TestLaunchHeartbeat`. `TestFinalizeStaleChatDebugRows` now
backdates its run with `UpdateChatDebugRun` instead of the deleted
test-only query.

**Why it is safe:** the heartbeats wrote only `updated_at` on
`chat_debug_runs` and `chat_debug_steps`. These read it:

| Reader | Why it does not need the heartbeat |
|---|---|
| `FinalizeStaleChatDebugRows` and the partial indexes
`idx_chat_debug_runs_stale` and `idx_chat_debug_steps_stale` | No
production caller. The next two PRs delete the query, and #31009 drops
the indexes in a migration. |
| `DeleteOldChatDebugRuns` (dbpurge) | See the next paragraph. |
| `DebugPanel.tsx` run sort | Sorts by `started_at \|\| updated_at`, and
`started_at` is `NOT NULL`, so it never falls back to `updated_at`. |
| Debug run and step API responses | Return `updated_at` as data.
Nothing uses it to decide whether a run is alive. |

dbpurge deletes runs whose `updated_at` is older than the debug
retention window, whatever their status, and their steps cascade. The
window is `agents_chat_debug_retention_days`: 30 days by default, and 0
turns the purge off, so dbpurge never deletes a run that was written in
the last day. A run's `updated_at` still moves when the run is created,
when each step is inserted (the `InsertChatDebugStep` CTE), and on every
run update and finalize. So dbpurge could only delete a live run if one
model call plus the tool calls after it ran for more than a day. The
runner finalizes its debug run when the runner context ends, a
`requires_action` wait times out after 5 minutes (the case `TouchRun`
covered), and chatloop aborts a stream that is silent for
`--chat-stream-silence-timeout`, 10 minutes by default (0 turns that
guard off).

Mike accepted the cost: a run or step orphaned by a crashed replica
stays `in_progress` until dbpurge deletes it. That is already true on
`main`, because nothing runs the sweep.

This does not touch the chat worker's `chat_heartbeats` loop that
ARCHITECTURE.md describes, and no ARCHITECTURE.md sentence about debug
runs becomes false.

**Review focus:** nothing uses `WithClock` after this PR, but deleting
it here would take this PR past 500 changed lines, so it goes with
`FinalizeStale` in the next PR.

Validation: `make gen/db`, `go build ./coderd/...`, and `golangci-lint
run` on the touched packages (0 issues). `go test -count=1` passes for
`./coderd/x/chatd/chatdebug/...`, `./coderd/database/dbauthz/ -run
TestMethodTestSuite`, `./coderd/database/ -run ChatDebug`,
`./coderd/x/chatd/ -run Debug` and `./coderd/database/dbpurge/`. The
pre-commit hook passed.

## Stack

This stack deletes the stale debug-run sweep, then simplifies the
chatdebug debug-run, stream wrapper and step numbering code, drops the
indexes the sweep used, and last deletes a runner debug field that lost
its only reader with the heartbeats. The debug-run and stream wrapper
refactors each sit on a test-only layer that pins the behavior they
keep.

1. This PR: Delete the chat debug run and step heartbeats
2. #30989 Delete `FinalizeStale` and the stale threshold
3. #30995 Delete the `FinalizeStaleChatDebugRows` query
4. #30933 Pin the debug runs that turns, titles and status labels record
5. #30934 Create and finalize debug runs through one helper
6. #30980 Table the stream wrapper tests and cover their status rules
7. #30981 Share one finalize-once lifecycle between the stream wrappers
8. #31007 Number chat debug steps in the database
9. #31008 Delete the process-global step counters
10. #31009 Drop the chat debug stale-sweep indexes
11. #31010 Drop the unused `runnerCtx` from the runner debug turn

> Xum, an AI agent, opened this PR for @ibetitsmike and wrote this
description.
@ibetitsmike
ibetitsmike merged commit 3673588 into main Oct 20, 2026
29 checks passed
@ibetitsmike
ibetitsmike deleted the mike/chatdebug-delete-finalize-stale branch October 20, 2026 20:54
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants