Repository navigation
feat: refactor the chatd worker - #26109
Conversation
d01e533 to
2c55f38
Compare
0e10473 to
0840442
Compare
0840442 to
965ee76
Compare
mafredri
left a comment
There was a problem hiding this comment.
Only made it to messagepartbuffer so far, but I'm noticing a lot of added code is missing comments making it hard to review. I'll continue reviewing but will be harder to be thorough in these parts.
| // one. | ||
| func New(ctx context.Context, logger slog.Logger, db database.Store, vals *codersdk.DeploymentValues, reg prometheus.Registerer, auditor *atomic.Pointer[audit.Auditor], opts ...Option) io.Closer { | ||
| // The auditor pointer is accepted for compatibility with other background | ||
| // services. Dbpurge does not emit audit logs directly. |
There was a problem hiding this comment.
This is some LLM-based reasoning right here, let's remove it if it's no longer used.
| var purgedChats, purgedChatFiles, purgedChatDebugRuns int64 | ||
| if chatRetentionConfigErr == nil { | ||
| purgedChats, purgedChatFiles, archivedChats, err = i.purgeChatsInTx(ctx, tx, start, chatRetentionDays, chatAutoArchiveDays) | ||
| if chatRetentionErr == nil { |
There was a problem hiding this comment.
I'm a bit worried about the cleverness of this code. The distance between getting the error and using it is too large and unusual for Go code, it'd be better to set a flag rather than re-use the err here.
|
|
||
| // purgeChatsInTx MUST BE CALLED WITH A TRANSACTION | ||
| func (i *instance) purgeChatsInTx(ctx context.Context, tx database.Store, start time.Time, chatRetentionDays, chatAutoArchiveDays int32) (purgedChats, purgedChatFiles int64, archivedChats []database.AutoArchiveInactiveChatsRow, err error) { | ||
| func (*instance) purgeChatsInTx(ctx context.Context, tx database.Store, start time.Time, chatRetentionDays int32) (purgedChats, purgedChatFiles int64, err error) { |
There was a problem hiding this comment.
(Side-note/future work: It would be nice to add support for database/Store to assert InTx and error out if not.)
There was a problem hiding this comment.
Created a Linear issue to track this: https://linear.app/codercom/issue/CODAGT-618/assert-transaction-context-in-database-store
| // LaunchRunHeartbeat starts a goroutine that periodically touches an | ||
| // open run until done is closed or ctx is canceled. | ||
| func (s *Service) LaunchRunHeartbeat(ctx context.Context, runID uuid.UUID, chatID uuid.UUID, done <-chan struct{}) { | ||
| if s == nil || runID == uuid.Nil || chatID == uuid.Nil || done == nil { |
There was a problem hiding this comment.
This needs to either be part of the method documentation or return an error, same for TouchRun.
| interval := s.heartbeatInterval() | ||
| ticker := s.clock.NewTicker(interval, "chatdebug", "run-heartbeat") | ||
| defer ticker.Stop() | ||
| resetTicker := func() { |
| if episode == nil || !episode.created { | ||
| return ErrEpisodeNotFound | ||
| } |
There was a problem hiding this comment.
Suggestion: helper method (e.g. getEpisode) that handles not-found check.
| @@ -0,0 +1,492 @@ | |||
| package messagepartbuffer | |||
There was a problem hiding this comment.
I think this package could use a lot more documentation, also on the methods to clarify why the code is doing what it's doing. As-is, it's very hard to review for implementaiton matching intent.
| if !episode.created { | ||
| episode.created = true | ||
| close(episode.createdCh) | ||
| } |
There was a problem hiding this comment.
This is done in a few places, could be extracted to a method like episode.markCreated() or perhaps episode.finalize().
| episode := b.episodeLocked(key) | ||
| subscriber := &episodeSubscriber{ | ||
| out: make(chan Part), | ||
| notifyCh: make(chan struct{}, 1), |
There was a problem hiding this comment.
A comment here explaing why it's buffered would be useful for readers. Perhaps also why the unbuffered channels are unbuffered/safe.
| @@ -0,0 +1,377 @@ | |||
| package messagepartbuffer_test | |||
There was a problem hiding this comment.
I suspect adding goleak to this package would surface some straggling goroutines. The logic seems to me like it's "eventually torn down". Worth considering if it should be added and whether or not code-paths should wait for teardown to happen.
There was a problem hiding this comment.
Deferred with high priority. Add goleak to all packages affected by the refactor.
2c55f38 to
96fdf6f
Compare
965ee76 to
3bbda07
Compare
mafredri
left a comment
There was a problem hiding this comment.
I've glanced everything, but I still have a few more core-files to review in chatd with proper focus.
| if d.created { | ||
| return d.contextLocked(ctx) | ||
| } | ||
| if d.disabled || d.finalized { |
There was a problem hiding this comment.
Won't d.created check above always bypass d.finalized, what are the ramifications of allowing that?
There was a problem hiding this comment.
Moved the d.finalized check before d.created.
| for { | ||
| select { | ||
| case tick := <-ticker.C: | ||
| w.archiveOnce(ctx, dbtime.Time(tick).UTC()) |
There was a problem hiding this comment.
Minor: A slow archiveOnce or congested DB connection may create a constant archival loop. I'd suggest archiveOnce -> createTicker -> select -> archiveOnce + ticker.Reset(intval), but I leave it to the author. I don't think this will run often enough for this to become an issue.
| return | ||
| } | ||
|
|
||
| archiveCutoff := dbtime.StartOfDay(start).Add(-time.Duration(autoArchiveDays) * 24 * time.Hour) |
There was a problem hiding this comment.
We should have docs on why we've e.g. selected UTC 00:00 - days to be the cutoff.
|
|
||
| archived := make([]autoArchivedChat, 0, len(rows)) | ||
| for _, row := range rows { | ||
| family, err := w.archiveCandidateSafely(ctx, row) |
There was a problem hiding this comment.
I think archival = activity. Right now it reflects the last chat activity meaning an archived chat could almost immediately be wiped by dbpurge.
There was a problem hiding this comment.
Address now by adding a postgres trigger that automatically bumps updated_at.
There was a problem hiding this comment.
Update: every chat state transition already bumps updated_at via the UpdateChatExecutionState query, archival included. Nothing needs to be done here now, but adding a Postgres trigger would be more robust. Deferred.
There was a problem hiding this comment.
Not going to address since UpdateChatExecutionState already handles it well enough.
| // truth for clients. | ||
| type generatedChatTitle struct { | ||
| mu sync.RWMutex | ||
| title string |
There was a problem hiding this comment.
Suggestion: An atomic value seems more appropriate here.
| } | ||
|
|
||
| func (s *taskStarter) StartGeneration(ctx context.Context, input chatWorkerTaskStartInput) error { | ||
| if s.server == nil { |
There was a problem hiding this comment.
Do we really need all these runtime checks for required options?
| return s.finishGenerationError(ctx, machine, input, 0, err, generationAttemptNotRequired) | ||
| } | ||
| if prepared.Messages == nil { | ||
| prepared.Messages = messages |
There was a problem hiding this comment.
Isn't this already handled in prepareGeneration? It either returns an err or it returns input.Messages.
There was a problem hiding this comment.
Removed the redundant code.
| select { | ||
| case <-ticker.C: | ||
| if err := m.syncOnce(m.ctx); err != nil { | ||
| m.opts.Logger.Warn(m.ctx, "chatworker runner sync failed", slogError(err)) |
There was a problem hiding this comment.
If ctx is cancelled, this is noise.
| select { | ||
| case m.cleanupDoneCh <- key: | ||
| case <-m.ctx.Done(): | ||
| m.mu.Lock() |
There was a problem hiding this comment.
Caution: Let's be extra careful about potential deadlocks between m.wg.Wait and m.mu.Lock.
There was a problem hiding this comment.
Deferred follow up: ensure this is corrected from a concurrency perspective
3bbda07 to
c48520d
Compare
96fdf6f to
b504e1d
Compare
| ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second) | ||
| defer cancel() | ||
| _ = server.chatWorker.WaitIdle(ctx) |
There was a problem hiding this comment.
I'd caution against a hard-coded timeout here and instead accept a context.Context as a function parameter.
| routeStateHint func(context.Context, runnerStateUpdate) | ||
| requestCleanup func(context.Context, runnerKey) | ||
| afterInterruptionOutcome func(context.Context, interruptionOutcome) error |
There was a problem hiding this comment.
nit, non-blocking: see my comment on taskSideEffectRecorder. Extracting these to an interface would make the seam clearer and easier to mock ro spy on in tests.
| type taskSideEffectRecorder struct { | ||
| mu sync.Mutex | ||
| hints []runnerStateUpdate | ||
| cleanups []runnerKey | ||
| interrupts []interruptionOutcome | ||
| } |
There was a problem hiding this comment.
nit, non-blocking: this is doing a reasonable job as a test spy, but it also highlights that taskStarter has a number of dependencies that cause side-effects. Extracting these to an interface would make the seam clearer, plus then you could potentially use gomock here to handle the test assertions.
| for _, rec := range targets { | ||
| select { | ||
| case rec.stateCh <- state: | ||
| case <-rec.done: |
There was a problem hiding this comment.
Is this intended to also be an explicit return?
There was a problem hiding this comment.
No. We should continue here. Addressed.
| key := runnerKey{ChatID: req.ChatID, RunnerID: req.RunnerID} | ||
| m.mu.Lock() | ||
| if _, ok := m.runners[key]; ok { | ||
| m.opts.Logger.Warn(m.ctx, "invalid spawn request: chat runner already spawned", slog.F("key", key)) |
There was a problem hiding this comment.
I'd rather we either
a) log it at debug if it's just something that can happen and not to be concerned about
b) log it at error if it's a true logic error so that we can potentially catch it in tests
There was a problem hiding this comment.
log as error, this would be a true logic error
| return | ||
| } | ||
| if _, ok := m.cleaning[key]; ok { | ||
| m.opts.Logger.Warn(m.ctx, "invalid spawn request: chat runner in cleanup", slog.F("key", key)) |
There was a problem hiding this comment.
Same as above; we should either downgrade this to debug if it's not critical or raise it to error so we can catch it in tests
There was a problem hiding this comment.
Downgrade to debug now
There was a problem hiding this comment.
Update: this should be an error for the same reason as https://github.com/coder/coder/pull/26109/changes#r3387343995
| select { | ||
| case <-ticker.C: | ||
| if err := m.heartbeatOnce(m.ctx); err != nil { | ||
| m.opts.Logger.Warn(m.ctx, "chatworker heartbeat failed", slogError(err)) |
There was a problem hiding this comment.
I think Mathias' above comment is also applicable here -- skip logging this if it's a context canceled error.
| } | ||
| // Detach from the request lifetime so title generation can finish | ||
| // even after the create response is written. | ||
| titleCtx := context.WithoutCancel(ctx) |
There was a problem hiding this comment.
Should we still bound this with a separate timeout?
mafredri
left a comment
There was a problem hiding this comment.
Gone through the last files now, review done for this PR.
| defaultHeartbeatInterval = 9 * time.Second | ||
| defaultHeartbeatCleanupEvery = 30 * time.Second | ||
| defaultHeartbeatStaleSeconds = int32(30) | ||
| defaultArchiveInterval = 10 * time.Minute |
There was a problem hiding this comment.
Why do we run the archival loop every 10 minutes when we only apply the cutoff once every 24h? Don't know if we need to consider ticker going out-of-sync with clock but even once per hour should suffice.
There was a problem hiding this comment.
Changed to once per hour
| // result is committed and exits the local-tool commit as | ||
| // stale. Workspace context is persisted by the | ||
| // persist_workspace_context generation action in a later | ||
| // pass. |
There was a problem hiding this comment.
I understand this note exists in part because of the old implementation. But the way I read this is that it's easy to mis-use the implementation. Is that true or does the implementation guard against making a mistake like this? If it guards, is this comment even necessary? To further prevent the class of bug we should rename appendRootChatTools to appendRootChatToolsWithoutWorkspaceContext or something similar making the intent clear.
There was a problem hiding this comment.
Deferred. We should take a look at refactoring workspace context gathering in general.
| cleanup := prepared.Cleanup | ||
| if cleanup == nil { | ||
| cleanup = func() {} | ||
| } |
There was a problem hiding this comment.
This should happen after prepared is created (before decision so that there's only ever one cleanup path, matches code below).
| decisionChat := locked | ||
| if prepared.Chat.ID != uuid.Nil { | ||
| decisionChat = prepared.Chat | ||
| } |
There was a problem hiding this comment.
I feel we're jugging too many variables here and leaving old-variables referenceable that are no longer of use. It might be better focusing on using one name like chat instead of locked at top, and overriding once locked is no longer useful, etc.
| if errors.Is(err, sql.ErrNoRows) { | ||
| return errTaskExpectedExit | ||
| } | ||
| if err != nil { | ||
| return xerrors.Errorf("load chat: %w", err) | ||
| } |
There was a problem hiding this comment.
| if errors.Is(err, sql.ErrNoRows) { | |
| return errTaskExpectedExit | |
| } | |
| if err != nil { | |
| return xerrors.Errorf("load chat: %w", err) | |
| } | |
| if err != nil { | |
| if errors.Is(err, sql.ErrNoRows) { | |
| return errTaskExpectedExit | |
| } | |
| return xerrors.Errorf("load chat: %w", err) | |
| } |
Same case as in feedback on previous PRs, prefer structures that prevent mis-use. (Logic: all cases of err are handled after this one if statement.)
| } | ||
|
|
||
| func (s *taskStarter) StartAbandon(ctx context.Context, input chatWorkerTaskStartInput) error { | ||
| machine := chatstate.NewChatMachine(s.opts.Store, s.opts.Pubsub, input.ChatID, chatstate.Options{}) |
There was a problem hiding this comment.
Just noticed the chatstate.Options is reserved for future use. Are there actual plans for what this would entail? Seems like a premature optimization right now.
There was a problem hiding this comment.
Address now. Remove it
| if !ownedByTask(locked, input) { | ||
| mismatch = true | ||
| return errTaskExpectedExit | ||
| } | ||
| if err := verifyTaskFence(locked, input, input.Status, taskFenceOptions{requireHistory: true, allowArchived: true}); err != nil { | ||
| return err | ||
| } | ||
| if _, err := tx.Abandon(chatstate.AbandonInput{}); err != nil { |
There was a problem hiding this comment.
Feels like this is something that should be enforced by the machine rather than verified in "each" Update. 🤔
| } | ||
|
|
||
| // Start starts the acquisition and runner manager loops. | ||
| func (w *chatWorker) Start(ctx context.Context) error { |
There was a problem hiding this comment.
A better option is to rename this ctx parentCtx and use ctx inline. This helps avoid bugs where the wrong ctx is referenced in the function body.
| if manager == nil || manager.idle() { | ||
| return nil | ||
| } | ||
| timer := w.opts.Clock.NewTimer(10*time.Millisecond, "chatworker", "wait-idle") |
There was a problem hiding this comment.
Magic number, consider const + docs.
| defer wsCtx.close() | ||
|
|
||
| parts, discoveredSkillsPart, _, expectedAgentID := server.fetchContextForBuild(ctx, chat, &wsCtx, logger) | ||
| _ = discoveredSkillsPart |
b504e1d to
7e3810e
Compare
47d102f to
bb37788
Compare
7e3810e to
61bb879
Compare
bb37788 to
fd9d643
Compare
5b6d034 to
85a3bab
Compare
fd9d643 to
c9de302
Compare
c9de302 to
b688bfb
Compare
PR 4 of the chatd refactor. Implements the `/chats/$id/stream` and `/chats/$id/stream/parts` endpoints. The CI doesn't pass on purpose. Only the tip of the chatd refactor PR stack will pass CI. Previous PR in the stack: #26109
Enable goleak in chatd tests and fix some leaks. Addresses #26109 (comment)
Prevents slow chat auto-archive runs from causing a constant archival loop by resetting the ticker only after each run completes. Also documents the UTC midnight cutoff used for archive eligibility so chats with activity on the same UTC calendar date stay eligible or ineligible for the full day. Addresses deferred review comments: - #26109 (comment) - #26109 (comment) Generated by Coder Agents and closely reviewed by Hugo.
…implementation (#26508) Address deferred review feedback for `messagepartbuffer` by documenting its episode lifecycle, extracting the repeated episode lookup and finalization helpers, and documenting subscriber channel buffering decisions. Addresses these PR #26109 comments: - #26109 (comment) - #26109 (comment) - #26109 (comment) - #26109 (comment)
Addresses the deferred `coderd/x/chatd/generation.go` review comments from PR #26109: [required generation dependencies](#26109 (comment)), [scoped chat variables](#26109 (comment)), [generation state error handling](#26109 (comment)), [generation attempt return values](#26109 (comment)), [generation fence verification](#26109 (comment)), and [chatdebug outcome recording](#26109 (comment)). This makes generation task dependencies explicit, packages generation attempt episode state into a struct, and centralizes generation task fence checks for generation transitions. Generated by Coder Agents, closely reviewed by Hugo.
PR 3 of the chatd refactor. Refactors the chatd worker to use the chatd state machine.
The CI doesn't pass on purpose. Only the tip of the chatd refactor PR stack will pass CI.
Previous PR in the stack: #25923