Skip to content

feat: open a chat_turn span per prompt - #29771

Merged
jscottmiller merged 25 commits into
scott/chatd-lifecycle/2-provider-stagesfrom
scott/chatd-lifecycle/3-turn-span
Sep 30, 2026
Merged

jscottmiller merged 25 commits into
scott/chatd-lifecycle/2-provider-stagesfrom
scott/chatd-lifecycle/3-turn-span

Conversation

@jscottmiller

@jscottmiller jscottmiller commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

TL;DR: Wraps each prompt in a chat_turn root span with every lifecycle stage parented under it. The span records one outcome per turn (completed, interrupted, error, abandoned), and the new coderd_chatd_turn_outcomes_total counter counts those outcomes. Turns stay correct when tasks are canceled, replaced, or retried.

Each prompt runs inside a chat_turn root span owned by the runner (runnerTurnSpan). The generation step, prepare, MCP connect, stream, TTFT, provider attempt, commit, thinking, tool call, compaction, and retry backoff stages are parented under it, activating the stages from #29770. chat_turn spans are standalone roots: the turn runs on a worker, often on another replica, and no trace context is persisted.

Lifecycle

  • Start. Each generation loop iteration calls Ensure, which opens the turn if none is open and returns a turnToken. The span is backdated to the trigger: the latest of the last user prompt, a dynamic tool's result message after it, and a pending compaction request. An acquisition stage covers trigger to pickup.
  • Stale trigger / takeover. A trigger at or before the previous anchor opens the turn at now with no acquisition and counts stale_anchor. A runner that took the chat over from an owner with a stale heartbeat (TakenOver, from acquisition) opens its first turn at now with no acquisition, so the dead owner's run is not reported as pickup delay.
  • Newer prompt. Ensure on the current turn with a newer trigger closes it and opens a new one. The closed turn is abandoned unless the task that ran it records an outcome before releasing it.
  • Finish. After FinishTurn or EnterRequiresAction commits, the step calls Complete; Settle closes the span after the step's own stage ends, so the finishing step is inside the turn. A chat waiting in requires_action is outside any turn; the generation resumed by its tool results opens a new one.
  • Emission. Closing a turn fixes its end time; its span is emitted once every task that joined it (Ensure takes the task ID) has called Release, which runTask does after the task returns. A finishing task whose FinishTurn promoted a queued prompt therefore still counts its turn as completed even when the promoted task's Ensure runs before its Complete. End at runner exit emits whatever is left.
  • Failure. A step that commits FinishError, or fails closed, invalidates the turn as error after the commit. Errors the task runner retries, including task timeouts, keep the same turn open; an expected, non-retryable exit records no outcome and leaves the turn open until the interrupt task, a newer prompt, or runner exit closes it. A canceled generation task records no outcome and cannot obtain a turn token, so a replaced task never touches its successor's turn.
  • Interrupt. The interrupt task marks the open turn interrupted before FinishInterruption commits and closes it after. Only a stop (by the user or a parent agent) is interrupted; it ends with unset span status, since error status is reserved for error turns. runner shutdown, loss of ownership, and message edits close the turn as abandoned with unset status.
  • Promotion. Every promoting transition (FinishTurn, FinishInterruption, a send to an errored chat, PromoteQueued) returns the promoted message's queued time in its result, and one recordQueueWait helper records a standalone queue_wait to the promotion. The promoted turn opens at the promotion time.

chat_turn is observed on coderd_chatd_stage_duration_seconds only for completed turns, so the histogram is completed-turn latency; every turn still emits its span.

coderd_chatd_turn_outcomes_total{outcome, chat_kind} counts every closed turn exactly once (completed, interrupted, error, abandoned), matching the span's turn_outcome, under the chat-stage-metrics experiment. Turn time accounting is in draft #29772.

Stages and identity

  • tool_call is a live stage around each local tool call in chatloop; the tool runs on its context, and it ends errored only when the tool fails to execute. Calls that never run (inactive, unknown, or rejected by policy) record no stage.
  • mcp_connect carries connected and failed server counts and ends errored only when servers were configured and none connected.
  • The compaction stage carries the model that runs the summary.
  • withStageIdentity puts scope and chat kind on the turn context. Detached background work runs in background scope as its own trace roots.
  • The advisor tool's nested model call appears only as its enclosing tool_call.

ARCHITECTURE.md gains a "Lifecycle tracing" section.

Review decisions
  • Turn identity stays timestamp-based with targeted fixes (clamped acquisition, rotation on a newer trigger, invalidate after commit, fail-closed invalidation, settle closing invalidated turns) plus a takeover flag; keying turns on message IDs would not detect takeover and cannot express interrupted/abandoned outcomes.
  • Outcome classification: the interrupt task owns interrupted, and cancellation or a fence mismatch alone never records an outcome, so shutdown, ownership loss, and edits are abandoned. Only error turns set error span status. Remaining ordering races between a step and an interrupt are documented as known limitations in ARCHITECTURE.md.
  • Turns are emitted after the tasks that ran them return, not when they close, so an outcome a replaced task records while unwinding reaches its own turn; the successor task is not delayed. Chosen over an after-commit, pre-publish hook on ChatMachine.Update, which would fix only the FinishTurn ordering.
  • requires_action ends the turn rather than pausing it, so chat_turn measures chatd time; a prompt with client tools counts as several turns.
  • Promoted turns anchor at promotion time and queue_wait is always standalone, so chat_turn never includes time spent queued behind another turn.
  • The former capacity_wait stage (feat(coderd): record a capacity_wait stage on chat acquisition #29805) was dropped: it was a per-worker lower bound, excluded from accounting, and needed its own SQL and bookkeeping. feat(coderd): record a capacity_wait stage on chat acquisition #29805's remaining helpers are folded into this PR.

Supersedes #28972, #28973, and #29805.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.

@jscottmiller

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review

coder-agents-review Bot commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-09-29 02:23 UTC by @jscottmiller

Review history
  • R3 (2026-09-25): 15 reviewers, 10 Nit, 8 P2, 24 P3, 16 P4, COMMENT. Review
  • R4 (2026-09-25): 15 reviewers, 13 Nit, 8 P2, 28 P3, 21 P4, COMMENT. Review
  • R5 (2026-09-28): 13 reviewers, 13 Nit, 8 P2, 28 P3, 27 P4, COMMENT. Review
  • R6 (2026-09-29), 13 Nit, 8 P2, 28 P3, 27 P4, COMMENT. Review
  • R7 (2026-09-29): 15 reviewers, 14 Nit, 8 P2, 30 P3, 34 P4, COMMENT. Review

deep-review v0.13.0 | Round 7 | dc303a5..562baea

Last posted: Round 7, 86 findings (8 P2, 30 P3, 34 P4, 14 Nit), COMMENT. Review

Finding inventory

Finding inventory: PR #29771

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 P2 Author fixed (5ffb40e) generation.go:444 Turn span wiring in StartGeneration and the finish paths has no test with a recording tracer R1 Netero Yes
CRF-2 P3 Author fixed (moved to base 5be82a7, #29805) worker.go:329 forgetCapacityWait on a non-capacity skip is never exercised R1 Netero Yes
CRF-3 P3 Author fixed (5ffb40e) turn_trace.go:172 runnerTurnSpan.Context has no production caller; tests use it instead of the context Ensure returns R1 Netero Yes
CRF-4 P3 Author fixed (5ffb40e) generation_preparer.go:405 mcp_connect span always ends with nil, even when every MCP server failed R1 Netero Yes
CRF-5 P3 Author fixed (5ffb40e) generation.go:719 Canceled retry_backoff path is never executed R1 Netero Yes
CRF-6 Nit Author fixed (retitled feat(coderd)) PR title (.github/workflows/contrib.yaml:203) Title scope coderd/x/chatd excludes the coderd/database files, so the title CI job fails R1 Netero Yes
CRF-7 Note Dropped by orchestrator (CI status with no code location; not reproducible from the diff; carried in review body) CI fmt job fmt job fails at head; not reproduced locally R1 Netero No
CRF-8 P2 Author fixed (c303eda) chatstate/transitions.go:1513 queue_wait recorded only for FinishTurn and PromoteQueued promotions; FinishInterruption and sendMessageE1 promotions record nothing, and ARCHITECTURE.md says PromoteQueued is the only other path R2 Netero Yes
CRF-9 P3 Author fixed (c303eda) generation.go:1216 compaction span and its override model label have no test R2 Netero Yes
CRF-10 P3 Author fixed (c303eda; closure removed, tool_call moved to chatloop) generation.go:1052 production tool_call record closure (span name, parent, tool_name) never executed by a test R2 Netero Yes
CRF-11 P3 Author fixed (c303eda) chatd.go:2171 PromoteQueued call to recordQueueWait has no test R2 Netero Yes
CRF-12 P3 Author fixed (c303eda; promotion no longer reaches Complete) generation.go:1688 FinishTurn to next-turn promotion chain (PromotedQueuedAt forwarding) has no end-to-end test R2 Netero Yes
CRF-13 P4 Author fixed (61546de) chatd.go:4913 chat kind plus organization context pair built by hand in several places R2 Netero Yes
CRF-14 P2 Author fixed (c303eda) turn_trace.go:137 Clamped anchor still records acquisition (and backdates the reopened chat_turn), so a same-prompt reopen counts the previous turn as acquisition R2 Luffy, Melody, Ryosuke, Pariston, Mafuuu, Bisky, Leorio, Zoro Yes
CRF-15 P2 Author fixed (282311c) turn_trace.go:118 Ensure hands a new prompt's task the open turn's token; edits during requires_action always join the old turn, and a late Invalidate from a canceled task marks the new prompt's turn interrupted R2 Melody, Pen Botter, Takumi, Mafuuu Yes
CRF-16 P2 Author fixed (c303eda, 5b142d6) generation.go:1699 finishGenerationError invalidates before FinishError commits; a retried commit splits one prompt into two chat_turn spans R2 Melody P2, Pariston P3, Mafuuu P3 Yes
CRF-17 P2 Author fixed (c303eda) turn_trace.go:125 A new runner taking over a running chat records acquisition from the original prompt across the previous runner's work R2 Pen Botter P3, Pariston P2 Yes
CRF-18 P3 Author fixed (c303eda) turn_trace.go:258 chat_turn anchor differs by promotion path: FinishTurn promotions start at queuedAt and overlap the prior turn, other paths start at promotion time R2 Luffy, Ryosuke Yes
CRF-19 P3 Author fixed (c303eda) generation.go:577 chat_turn stays open through requires_action, so its duration includes client tool wait; undocumented and untested R2 Pen Botter P3, Bisky P3, Mafuuu P4 Yes
CRF-20 P3 Author fixed (c303eda) generation.go:1432 Fail-closed hook path commits FinishError without Invalidate, so the turn closes as abandoned with OK status R2 Ryosuke Yes
CRF-21 P4 Author fixed (c303eda) turn_trace.go:237 Settle leaves an invalidated turn open until the next Ensure or runner teardown R2 Ryosuke Yes
CRF-22 P3 Author fixed (c303eda, 61546de) generation.go:522 provider attribute is ResolvedProvider on prepare/generation_step/thinking but Model.Provider() on tool_call/stream/ttft R2 Gon Yes
CRF-23 P3 Author fixed (c303eda) generation.go:971 toolCallStageRecorder rebuilds per-call timing and names that chatloop.executeTools already has; record tool_call inside chatloop instead R2 Zoro Yes
CRF-24 P3 Author fixed (c303eda) stage_internal_test.go:404 TestServerInflightContextIsBackgroundScoped chat_kind assertion passes without inflightChatContext; organization not asserted R2 Bisky Yes
CRF-25 P4 Author fixed (c303eda) chatd_test.go:14822 TestActiveServer_TracesChatTurn does not assert organization_name, so the runner's resolver wiring is untested R2 Bisky Yes
CRF-26 P3 Author fixed (c303eda) turn_trace.go:86 Ensure doc says a trigger older than the anchor is clamped; code clamps equal too, and line 121 and the PR text state the rule differently R2 Gon P2, Leorio P4 Yes
CRF-27 P2 Author fixed (c303eda) chatloop/stage.go:74 TurnOutcomeAbandoned doc says no failure is recorded, but expected-exit invalidations end abandoned spans with error status R2 Gon Yes
CRF-28 P3 Author fixed (c303eda) turn_trace.go:132 Comment explains why promoted turns record no acquisition at a line promoted turns never reach R2 Gon Yes
CRF-29 P3 Author fixed (c303eda) turn_trace.go:159 Comment says this context gives every stage of the turn the chat kind and organization; stages get them from contextLocked R2 Gon Yes
CRF-30 Nit Author contested; panel closed R3 (1/1 accept) generation.go:461 Four doc comments restate the code below them (turnContinues, turnOutcomeForError, settleLocked, turnPromotion) R2 Gon Yes
CRF-31 P4 Author fixed (c303eda) stage_internal_test.go:716 Test comment mentions an End error that the test never passes R2 Gon Yes
CRF-32 P4 Author contested; panel closed R3 (1/1 accept) turn_trace.go:265 End disables the instance permanently and its err parameter is always nil R2 Gon Yes
CRF-33 P4 Author fixed (c303eda) generation_preparer.go:814 mcpConnectOutcome reads as a classifier but sets span attributes R2 Gon Yes
CRF-34 P4 Author fixed (c303eda) generation.go:1586 PromotedMessage != nil guard repeats the zero-value contract of PromotedQueuedAt R2 Gon Yes
CRF-35 P4 Author fixed (c303eda) turn_trace_internal_test.go:37 Test files swapped relative to the code they test R2 Gon Yes
CRF-36 Nit Author fixed (c303eda) generation.go:444 Local turnToken shadows the type turnToken R2 Gon Yes
CRF-37 Nit Author fixed (c303eda) chatloop/stage.go:57 AttrMCPConnected/AttrMCPFailed break the Attr naming pattern R2 Gon Yes
CRF-38 Nit Author fixed (c303eda) options.go:68 Same *runnerTurnSpan named turnSpan on runner and Turn on the task input R2 Gon Yes
CRF-39 Nit Author fixed (c303eda) generation.go:1160 compactionStage holds a StageModel R2 Gon Yes
CRF-40 Nit Author fixed (c303eda) stage_internal_test.go:417 sort.Slice where the package uses slices.SortFunc R2 Ging-Go Yes
CRF-41 Nit Author fixed (c303eda) stage_internal_test.go:379 context.WithCancel(context.Background()) plus Cleanup instead of t.Context() R2 Ging-Go Yes
CRF-42 P4 Author fixed (c303eda) generation_preparer.go:829 mcp_connect error gives a failure count but no server slug or cause R2 Leorio Yes
CRF-43 P4 Dropped by orchestrator (PR text with no code location; carried as one sentence in review body) PR description "activating the PR 2 stages" does not name the PR R2 Leorio No
CRF-44 P4 Author fixed (c303eda) chatd.go:2167 refreshedOK guard in PromoteQueued is always true; empty chat kind/organization branch is dead R2 Zoro Yes
CRF-45 OOS Out of scope (body) capacity.go:88 recordCapacityWait (#29805) builds the same chat kind plus organization context pair R2 Netero No
CRF-46 Note Dropped by orchestrator (praise; asks for no change) chatd.go:4878 inflightContext span stripping is correct; feature justified R2 Luffy No
CRF-47 Note Dropped by orchestrator (verification record; used to judge CRF-23 index alignment) generation.go:1052 tool_call dispatch-index pairing verified R2 Melody No
CRF-48 Note Dropped by orchestrator (praise; asks for no change) ARCHITECTURE.md:793 Lifecycle tracing section explains standalone root and two-step close R2 Leorio No
CRF-49 P3 Author fixed (282311c) turn_trace.go:186 No test passes a stale token to Complete or Settle; removing the ownership check leaves every test passing R3 Netero Yes
CRF-50 Nit Author fixed (retitled feat: without scope) PR title (.github/workflows/contrib.yaml:203) Title scope coderd excludes docs/ and scripts/ files added in c303eda, so the title job fails again R3 Netero Yes
CRF-51 P2 Author fixed (282311c) generation.go:473 turnOutcomeForError labels every task cancellation interrupted: shutdown, ownership loss, and edits close as interrupted with error status, and an edit's outcome depends on goroutine order R3 Razor P2, Pariston P3, Mafuuu P3, Pen Botter P4 Yes
CRF-52 P3 Author fixed (282311c) generation.go:441 An interrupt between generation steps closes the turn as abandoned with OK status; StartInterrupt never marks the turn R3 Takumi P3, Melody P4, Mafuuu P4 Yes
CRF-53 P3 Author fixed (b34b2c3) turn_trace.go:256 interrupted turns end with error status, so every user stop is an errored root span R3 Pen Botter Yes
CRF-54 P3 Author fixed (282311c) turn_trace.go:263 chat_turn histogram observes every outcome with no outcome label, and prometheus.md does not state the chat_turn window R3 Luffy P3, Pen Botter P3 Yes
CRF-55 P3 Author fixed (282311c) generation.go:464 turnContinues omits errors runTaskWithRetry retries (ErrTransitionNotAllowed, ErrInvalidState), so each retry closes an error turn and opens a stale_anchor turn R3 Melody Yes
CRF-56 P3 Author fixed (282311c) generation.go:450 No test reaches the turnContinues guard in StartGeneration; RetryKeepsTurnOpen retries inside one step R3 Bisky Yes
CRF-57 P3 Author fixed (282311c) chatloop.go:1333 executeSingleTool doc omits inactive and unknown tools, which return a nil error without running R3 Gon P3, Leorio P4 Yes
CRF-58 P4 Author fixed (b34b2c3) chatloop.go:1382 Inactive or unknown tool calls record an OK tool_call stage and a near-zero histogram sample; exclusive-policy rejections record none R3 Melody Yes
CRF-59 P4 Author contested; panel closed R4 (1/1 accept) turn_trace.go:124 Stale-anchor check compares a database-clock trigger with a replica-clock lastAnchorAt after a turn opened at now R3 Knuckle Yes
CRF-60 P4 Open turn_trace.go:86 Ensure doc lists two no-acquisition cases; a zero trigger is a third and counts no stale_anchor R3 Leorio Yes
CRF-61 P4 Author fixed (282311c) turn_trace.go:61 lastAnchorAt comment is false when the trigger is ahead of the replica clock R3 Leorio Yes
CRF-62 P4 Author fixed (282311c) chatd.go:2147 PromoteQueuedMessageResult lacks PromotedQueuedAt; the caller's InsertedMessage guard is untested (mutation passes) R3 Ryosuke, Razor Yes
CRF-63 P4 Author fixed (282311c; assertion removed) turn_trace_internal_test.go:480 EndTime assertion after End reads an ended-span snapshot and cannot fail R3 Bisky Yes
CRF-64 Nit Author fixed (282311c) generation_preparer.go:827 reason fallback written as if-block where the package uses cmp.Or R3 Ging-Go Yes
CRF-65 OOS Out of scope (body) chatloop.go:1102 executeTools takes 18 positional parameters copied from ExecuteLocalToolsOptions (predates the PR; the PR adds two) R3 Ryosuke No
CRF-66 Note Dropped by orchestrator (analysis record; asks for no change) turn_trace.go:308 turnTriggerTime parses turn tool messages each step, a subset of rows the step already parses R3 Killua No
CRF-67 Note Dropped by orchestrator (analysis record; asks for no change) chatd.go:4918 organizationName cache is bounded by organization count; failed lookups retry per call R3 Killua No
CRF-68 Note Dropped by orchestrator (behavior already stated in the doc comment; span-only attribute) chatd.go:4911 Cached organization names go stale across a rename until restart R3 Ryosuke No
CRF-69 Note Dropped by orchestrator (verification record) chatd.go:4897 GetOrganizationByID, chatd read permission, and PromotedQueuedAt sourcing checked at the Go/SQL boundary R3 Knuckle No
CRF-70 P3 Author fixed (b34b2c3; call removed) generation.go:451 No test runs the Invalidate(abandoned) call in StartGeneration; the true branch of stepAbandonsTurn has count 0 R4 Netero Yes
CRF-71 P4 Author fixed (b34b2c3; stepAbandonsTurn removed) generation.go:466 stepAbandonsTurn copies runTaskWithRetry's exit rule instead of sharing one predicate, and its doc restates the condition without naming runTaskWithRetry R4 Netero P4, Gon P2 Yes
CRF-72 P4 Author fixed (b34b2c3) tasks.go:347 Ambiguous-commit return in StartInterrupt skips Settle and recordQueueWait, so the interrupted turn stays open R4 Netero Yes
CRF-73 Note Note (body) generation.go:466 A timed-out attempt that returns a plain expected exit is not retried and records no outcome, contradicting ARCHITECTURE's timed-out-attempt sentence R4 Netero No
CRF-74 P3 Author fixed (b34b2c3) tasks.go:269 OpenToken has no ctx check, so a canceled interrupt task can mark the edited prompt's turn interrupted R4 Takumi P3, Ryosuke P4, Pariston P4 Yes
CRF-75 P3 Author fixed (b34b2c3) generation.go:450 A step that commits after the user's stop commits hits a fence mismatch and closes the turn as abandoned with error status R4 Takumi Yes
CRF-76 P3 Author contested; panel closed R5 (1/1 accept) turn_trace.go:291 Non-completed turns produce no metric, so chat_turn latency falls and no failure series exists during a provider outage R4 Chopper Yes
CRF-77 P4 Author fixed (b34b2c3, 43ec647) chatloop/stage.go:67 TurnOutcomeInterrupted doc says the user stopped the turn; parent agents interrupting subagents produce it too R4 Mafuuu, Leorio, Chopper Yes
CRF-78 P4 Author fixed (b34b2c3) chatloop/metrics.go:174 Help text says chat_turn runs from the prompt; client-tool and compaction turns start at their trigger R4 Pen Botter Yes
CRF-79 P4 Author fixed (b34b2c3) tasks.go:270 No test fails when the interrupt task's Invalidate moves after the FinishInterruption commit R4 Bisky Yes
CRF-80 Nit Author fixed (b34b2c3, 676c4fd) turn_trace.go:278 Three new comments restate the code (closeLocked doc, requireInterrupted, StaleTokenCannotCloseReplacement) R4 Gon Yes
CRF-81 Nit Author fixed (b34b2c3) tasks.go:269 Local turnToken shadows the turnToken type again R4 Gon Yes
CRF-82 Nit Author fixed (b34b2c3) turn_trace_internal_test.go:670 TestFinishGenerationErrorIgnoresCanceledContext names behavior the function does not have R4 Gon Yes
CRF-83 OOS Out of scope (body) chatloop/chatloop.go:1216 executeTools uses wg.Add/go/Done where wg.Go applies (base lines) R4 Ging-Go No
CRF-84 OOS Out of scope (body) ARCHITECTURE.md:75 I0/I1 state table says the user interrupted; parent agents reach the same states (base text) R4 Mafuuu No
CRF-85 OOS Out of scope (body) chatloop/metrics.go:124 coderd_chatd_chats{state} gauge is registered and documented but never set (base code) R4 Chopper No
CRF-86 P4 Author accepted R7 (untested; needs commit fault injection; no ticket) tasks.go:349 The Settle and recordQueueWait added to StartInterrupt's ambiguous-commit return have no test (block count 0) R5 Netero Yes
CRF-87 OOS Out of scope (body) tasks.go:348 The ambiguous-commit return in StartInterrupt skips DebugTurn.RecordOutcome (base code) R5 Netero No
CRF-88 P4 Author fixed (PR description, R7) generation.go:450 PR description's Failure bullet says an expected, non-retryable exit abandons the turn; since b34b2c3 that step records no outcome and leaves the turn open R5 Leorio, Pen Botter, Ryosuke Yes
CRF-89 P4 Author contested; panel closed R7 (1/1 accept) chatloop/chatloop.go:1358 executeSingleTool takes a startRun callback so runCall can start tool_call between resolution and tool.Run; splitting resolution from the run removes the callback R5 Ryosuke Yes
CRF-90 P4 Author contested; panel closed R7 (1/1 accept) generation.go:831 recordThinkingStages records thinking stages in chatd from timestamps GenerateAssistant already holds, and guards a length mismatch chatloop cannot produce R5 Zoro Yes
CRF-91 P4 Author fixed (562baea) turn_trace_internal_test.go:581 TestRunnerTurnSpanCanceledOpenToken doc credits the zero token for the stopped turn closing abandoned; the Ensure rotation closes it R5 Gon Yes
CRF-92 P4 Author contested; panel closed R7 (1/1 accept) turn_trace.go:200 Unset status for interrupted turns holds only because the one caller passes nil; the Invalidate and closeLocked docs do not say so and three tests pass an error R5 Gon Yes
CRF-93 Note Note (body) generation.go:453 A canceled generation task's Settle can close the stopped turn before FinishInterruption commits; outcome unchanged, span ends earlier R5 Takumi No
CRF-94 OOS Out of scope (body) generation.go:1512 The generation finish paths (finishGenerationTurnWithoutHook, finishGenerationTurn, enterRequiresAction, finishGenerationError) skip committedStateAfterUpdateError, so an ambiguous commit skips Complete or Invalidate (base error handling) R5 Pariston No
CRF-95 P2 Dropped by orchestrator (text the R4 review requested under CRF-77; accurate) tasks.go:265 StartInterrupt comment repeats the interrupt-source list from TurnOutcomeInterrupted R5 Gon No
CRF-96 Nit Dropped by orchestrator (accurate comments that each carry a fact the code does not show; trims save about one line each) turn_trace.go:241 OpenToken doc, generation.go:451 Settle comment, and chatloop.go:1153 runCall comment are longer than the facts they carry R5 Gon No
CRF-97 P4 Open worker_internal_test.go:425 TestWorkerRunnerReleasesTurnOnTaskExit ignores testutil.Eventually's result and indexes [0], so a regression panics the package test binary R7 Netero Yes
CRF-98 Nit Dropped by orchestrator (premise disproved: each committed step runs in a new task with a new ID, runner.go:162-174 and :210, so two step tasks do hold one turn) turn_trace_internal_test.go:671 firstStep and secondStep name task IDs as steps; holders are keyed by task and steps of one task share its ID R7 Netero No
CRF-99 P3 Open turn_trace_internal_test.go:126 TestRunnerTurnSpanRotatesOnNewerTrigger passes uuid.Nil, so the stale Invalidate finds no turn and the test asserts abandoned; in production the holding task's Invalidate reaches the turn R7 Bisky P3, Gon P4 Yes
CRF-100 P3 Open turn_trace.go:137 The uuid.Nil exemption in hold and Release exists only for tests; 40 unit-test Ensure calls run emit-on-close turns production never creates R7 Bisky Yes
CRF-101 P4 Open chatloop/stage.go:62 562baea deleted the TurnOutcome value docs; no Go comment defines interrupted or error or the span status each outcome sets R7 Leorio, Gon Yes
CRF-102 P4 Open turn_trace.go:194 Release doc says the ordering rule is so stages end before emission; the rule protects outcomes the task records while unwinding R7 Mafuuu Yes
CRF-103 P4 Open ARCHITECTURE.md:810 "a task canceled after Ensure records no outcome" is false when the transition committed first, as the same paragraph's FinishTurn example shows; PR description repeats it R7 Razor Yes
CRF-104 P4 Open tasks.go:268 PR description's Interrupt bullet names only user and parent-agent stops; promoting a queued message on a running chat (and API clients) also produce interrupted R7 Pen Botter Yes
CRF-105 P4 Open turn_trace.go:233 End closes the current turn in a block the next loop repeats, and pendingLocked sorts tokens for callers that do not depend on order R7 Zoro, Bisky Yes
CRF-106 P4 Open chatd.go:4859 inflightContext doc says reqCtx's values are kept for tracing; the body the PR changed strips the span context and sets background scope R7 Gon Yes
CRF-107 Nit Open turn_trace_internal_test.go:147 TestRunnerTurnSpanClampsStaleAnchor and the Review decisions bullet say clamped; a stale trigger now starts the turn at now R7 Leorio Yes
CRF-108 P4 Dropped by orchestrator (PR text with no code location; carried as one sentence in review body) PR description #29805 helpers sentence is stale after 1cf3bed, and nothing says why stages dropped organization_name R7 Leorio No
CRF-109 P4 Dropped by orchestrator (the stored value is End's err, whose always-nil status CRF-32's R3 closure accepted) turn_trace.go:58 closeErr and closeLocked's err parameter carry a value that is always nil R7 Zoro No
CRF-110 Nit Dropped by orchestrator (the parameter becomes the span start when zero, which startAt names) turn_trace.go:109 startLocked names its anchor parameter startAt while the caller passes anchorAt R7 Gon No
CRF-111 Nit Dropped by orchestrator (the current turn is the only unclosed turn, so OpenToken is accurate) turn_trace.go:209 OpenToken returns t.current; rename to CurrentToken or the field to open R7 Gon No

R2 severity decisions:

  • CRF-16: Melody P2 vs Pariston/Mafuuu P3. Upward case: Melody traces two triggers (transient DB error through normalizeTaskTransitionError, and task timeout through runTaskWithRetry) and the chain into the clamped-acquisition bug. Higher wins: P2.
  • CRF-17: Pen Botter P3 vs Pariston P2 (inside his clamp finding). Same consequence class as CRF-14, on every mid-turn takeover. Higher wins: P2.
  • CRF-26: Gon P2 vs Leorio P4. Keep argument: the Ensure doc is where a maintainer learns the clamp rule, and it excludes the equal case the clamp exists for. Downgrade argument: three adjacent statements give the correct rule (the line-121 comment, the base's stale_anchor constant doc "did not follow the previous turn's anchor", and the test comment), and the code is correct, so the consequence is one misleading paragraph. The downgrade argument is stronger: P3. Gon's line-121 finding (duplicate clamp comment, P2) merged here, since the two comments disagree.
  • CRF-27: Gon P2 kept. Keep argument: exported doc for a telemetry value contradicts the span status (reproduced). Downgrade argument (ARCHITECTURE.md states the error status) is weaker, since developers read the constant's doc.
  • CRF-28: Gon P2 to P3. Keep argument: a reader assumes Ensure handles promoted turns. Downgrade: the statement is true and the promoted path is in settleLocked; no behavior is misdescribed, only placement.
  • CRF-29: Gon P2 to P3. Keep argument: a developer could delete contextLocked's setting believing startLocked covers stages. Downgrade: contextLocked carries its own comment explaining why it sets them.
  • CRF-30: Gon's four P2 redundancy findings (generation.go:461, :469, turn_trace.go:248, :66) grouped as one Nit. Keep argument: restated docs drift from code. Downgrade: all four are accurate today; comment wording is the Nit definition.
  • CRF-31: Gon P2 to P4. Keep argument: the test comment names an End error that does not exist. Downgrade: test-only comment; the assertion is correct.
  • CRF-32: Gon P3 to P4. Keep argument: a future mid-runner End call would silently disable all later turns. Downgrade: one caller, at runner teardown; naming and an unused parameter.
  • CRF-33: Gon P3 to P4. Keep argument: the side effect hides inside an argument expression. Downgrade: one call site, behavior correct.
  • CRF-43: Keep argument: a reader cannot tell which stage definitions this PR activates. Drop reason: no code location for the CLI; P4; carried in the body.

CRF-7 drop gate. Keep argument: a red fmt job blocks merge and the author has to act on it. Drop reason: it has no code location, Netero's formatter run over every changed file and go mod tidy -diff at head both come back clean, and #29772 (base = this head) does not list fmt among its failures, so there is no code change to request. The action (read the job log or re-run it) is carried in the review body.

R3 severity decisions:

  • CRF-51: Razor P2 vs Pariston P3 and Mafuuu P3, plus Pen Botter's P4 edit race and Pariston's edit-race Note folded in. Upward case: three reviewers reproduced shutdown producing interrupted with error status, Razor reproduced edits producing interrupted in 30 of 30 runs, and the PR description and TurnOutcomeAbandoned doc promise abandoned for both, so turn_outcome cannot count user stops. Downward case: TurnOutcomeInterrupted's doc and ARCHITECTURE's Invalidate bullet describe the mechanism (task cancellation) accurately, and the attribute is experiment-gated telemetry. Razor's evidence is the broadest and the consequence is the same class as the R2 P2 misattributions; higher wins: P2. Contradiction recorded: Pen Botter proposes closing a rotated turn as interrupted; Razor proposes abandoned for edits per the existing doc. The comment asks for one outcome per cause without choosing.
  • CRF-52: Takumi P3 vs Melody P4 and Mafuuu P4. Upward case: a user stop between steps is labeled abandoned with OK status, and ARCHITECTURE's "interrupted when the task context is done" rule does not hold; the window reopens after every committed step. Downward case: inferred from code, not reproduced, and the window is short. Failure to reproduce does not weaken a race finding; higher wins: P3.
  • CRF-57: Gon P3 vs Leorio P4. Upward case: the doc is the contract the only caller reads to decide span status, and it misstates two of three nil-error paths. Downward case: private function, one caller, ARCHITECTURE states the rule. Same class as CRF-26 (doc omits cases), which was P3; higher wins: P3.
  • CRF-65 moved to out of scope: the positional parameter list predates the PR; the two parameters the PR adds have distinct types, and the refactor is not needed for the PR's goal.
  • CRF-66 drop gate. Keep argument: turnTriggerTime JSON-parses messages on every step. Drop reason: Killua shows the same step already loads and converts those rows, so no new order of cost; Killua asks for no change.
  • CRF-67 drop gate. Keep argument: failed organization lookups repeat per call. Drop reason: only while the database is failing, when those paths already query; asks for no change.
  • CRF-68 drop gate. Keep argument: a trace grouped by organization splits across a rename until restart. Drop reason: the organizationName doc comment states this and the attribute is span-only (no metric label); nothing to change.
  • CRF-69 drop gate. Keep argument: none beyond the record; it is Knuckle's verification of the Go/SQL boundary.
  • Verification-only Notes with no row (they confirm prior fixes and ask for nothing): Ging-Go (CRF-40, CRF-41), Luffy (CRF-14, CRF-18), Pen Botter (CRF-15, CRF-17, CRF-19), Leorio (CRF-42, CRF-26).

R4 severity decisions:

  • CRF-70: Netero framed it as a re-raise of CRF-56. Recorded as a new finding: the test CRF-56 asked for (task timeout keeps the turn open) was added, and the untested abandoned branch comes from the stepAbandonsTurn rewrite.
  • CRF-71: Netero P4 (duplicated predicate) and Gon P2 (doc restates it and names no runTaskWithRetry) merged, since one shared predicate or one pointer fixes both. Keep argument for P2: a maintainer who edits the runner's exit switch has no pointer to this copy, and a new non-retried case then leaves the turn open. Downgrade argument: the consequence is a turn closed late (next Ensure or runner exit) for a hypothetical future exit case, the doc is accurate today, and Netero's concrete consequence is rated P4. Downgrade stronger: P4.
  • CRF-74: Takumi P3 vs Ryosuke P4 and Pariston P4. Upward case: same consequence class as CRF-15 (a canceled task changes its successor's turn): the edited prompt's completed turn permanently reads interrupted with error status, reproduced at unit level by Takumi and Pariston. Downward case: the window is a few instructions between ReadLock and OpenToken. Severity follows consequence; higher wins: P3.
  • CRF-80: Gon's three P2 restatement findings (turn_trace.go:278, chatd_test.go:14906, turn_trace_internal_test.go:525) grouped as one Nit. Keep argument: restated comments drift, and the closeLocked sentence omits the reason for the completed-only rule. Downgrade: all three are accurate today, and comment wording with no behavior effect is the Nit definition (the R2 precedent grouped the same class as CRF-30). Contradiction recorded: Leorio's Note calls the stepAbandonsTurn doc sufficient; Gon's doc point on it is folded into CRF-71.
  • CRF-76: Chopper P3 kept. Keep argument: a p99 alert on chat_turn falls during a provider outage because failed turns are excluded, and no metric counts turn outcomes. Downward argument: stream and provider_attempt windows observe failures on the same histogram, and the completed-only rule was one of the two options the R3 review offered. The downward argument does not restore a per-turn failure signal: P3.
  • CRF-77: Mafuuu, Leorio, and Chopper each P4; merged.
  • CRF-83 to CRF-85 moved to out of scope: git diff -U0 e6f36ac5a2..HEAD has no hunk covering chatloop.go:1216, ARCHITECTURE.md:75, or metrics.go:124.
  • Verification-only Notes with no row: Killua (lifecycles), Kurapika (authorization scoping), Luffy (CRF-54), Pen Botter (CRF-51, CRF-54), Takumi (CRF-15), Leorio (stepAbandonsTurn doc and commit message), Knuckle (PromotedQueuedAt boundary).

R5 severity decisions:

  • CRF-76: dispute check-in closed 1/1 (see Contested and acknowledged).
  • CRF-88: Leorio, Pen Botter, and Ryosuke each P4; merged. Orchestrator check: the public REST API body of feat: open a chat_turn span per prompt #29771 at 676c4fd has the Failure bullet "an expected, non-retryable exit abandons it" while its Review decisions bullet says "cancellation or a fence mismatch alone never records an outcome"; generation.go:450-453 calls only Settle, which returns early without an outcome (turn_trace.go:229). Location: Ryosuke's generation.go:450, since the description has no line.
  • CRF-89: Ryosuke P4. Contradiction recorded: Pariston says the callback is not a redundant layer because runCall's deferred recover must end the span, and passing the tracer into executeSingleTool would need a second recover. That argument answers a different alternative (tracer passed in). Ryosuke's split has runCall call the resolve step, start the stage, and call the run itself, so runCall's one recover still covers tool.Run. Keep argument: the span's existence depends on a captured assignment inside a callback and on StageSpan.End accepting nil; the split makes the no-stage rule follow from control flow. Downward argument: the callback is eight lines, tested (RejectedCallsRecordNoStage fails when startRun moves above either check, per Netero and Bisky), and the split restructures a function with a result-metrics defer that must still run for rejections. Neither changes behavior; P4 kept.
  • CRF-90: Zoro P4. Verified: chatloop.go:964-965 appends to both reasoning slices in the same branch; the only production caller (generation.go:800) passes outcome.Step from GenerateAssistant, so the index >= len(step.ReasoningCompletedAt) return cannot run in production. Same class as CRF-23 (P3, stage timing rebuilt in chatd); the consequence here is smaller (no timing rebuilt, only the recording site and an unreachable guard with its test), so P4. recordThinkingStages exists since 60fc554 (R1); no earlier finding asked for the guard.
  • CRF-91: Gon P2 to P4. Keep argument: a maintainer who trusts the doc misreads which turn the zero token protects, the confusion CRF-74 was about. Downgrade: test-only comment; the assertion at :596 guards the right thing and fails under Gon's mutation (Should be zero, but was 2); CRF-31 (test comment naming behavior that does not exist) was P4. Downgrade stronger: P4.
  • CRF-92: Gon P4 (Invalidate accepts an error with TurnOutcomeInterrupted; tests at turn_trace_internal_test.go:140, :457, :624 pass one) and Gon's P2 on the closeLocked doc (turn_trace.go:271, framed as a re-raise of CRF-80) merged, since both describe one unstated rule: interrupted carries a nil error. Not recorded as a CRF-80 re-raise: CRF-80 was about restating comments, and this is a missing contract that b34b2c3 introduced by passing nil at tasks.go:272. Keep argument for P2: the closeLocked doc is where a maintainer learns the error rule. Downgrade: the docs do not contradict the code (a nil error replacing err reports no failure, and TurnOutcomeInterrupted's doc states unset status), and the one production caller passes nil (Melody, Mafuuu, Netero). Downgrade stronger: P4.
  • CRF-93: Takumi Note, reproduced with a scratch runnerTurnSpan test (stopped=1 openAfterGenSettle=0, one turn outcome=interrupted). Keep argument: ARCHITECTURE.md's Interrupt bullet says the interrupt task settles the turn after the commit. Takumi asks for no change: outcome unchanged, interrupted turns are not observed on the histogram. Note (body).
  • CRF-94 moved to out of scope: the missing ambiguous-commit handling is in base error paths (machine.Update error returns in finishGenerationTurnWithoutHook, finishGenerationTurn, enterRequiresAction, finishGenerationError; grep committedStateAfterUpdateError finds callers only in tasks.go), where base debug outcomes skip the same way; CRF-87 set the precedent for the StartInterrupt branch. Pariston listed it under Out of scope.
  • CRF-95 drop gate. Keep argument: a second copy of the interrupt-source list can drift from TurnOutcomeInterrupted's doc. Drop reason: the R4 review asked for this text in this comment (CRF-77 named the StartInterrupt comment as one of four places; Chopper verified it at tasks.go:265), the text is accurate, and asking to remove it reverses the review's own request.
  • CRF-96: Gon's three P2 restatement findings (turn_trace.go:241, generation.go:451, chatloop.go:1153) grouped as one Nit per the CRF-30 and CRF-80 precedent, then dropped. Keep argument: AGENTS.md requires substantive, concise comments, and restatements drift. Drop reason: each comment is accurate and carries one fact the code line does not show (why OpenToken checks ctx; that a failed step with no outcome leaves the turn open after stepAbandonsTurn's removal, which Gon calls a fact that matters; that rejected calls record no stage), and each proposed trim saves about one line.
  • Verification-only Notes with no row: Leorio (ARCHITECTURE.md:833 tool_call bullet), Pen Botter (CRF-53, CRF-78), Takumi (CRF-74, CRF-75 ordering), Bisky (CRF-75 needs no test), Pariston (framing analysis; every turn left open has a closer).

R7 severity decisions:

  • CRF-89, CRF-90, CRF-92: dispute check-ins closed 1/1 each (see Contested and acknowledged).
  • CRF-60 re-raised (Gon P3, Leorio P4, both at turn_trace.go:70): 562baea cut the Ensure doc to "joins taskID to the current turn, or starts one anchored at triggerAt", deleting the case list CRF-60's fix added. Verified at head: a zero triggerAt leaves anchorAt zero, and takeover and stale triggers zero it (turn_trace.go:94-101), so all three start at now with no acquisition; line 84 also closes a finished or invalidated current turn whatever the trigger. Recorded as a re-raise (status Open, reply on PRRT_kwDOGkVX1s6l46ud) because the original omission persists; Sev stays P4 (inventory columns other than Status do not change). Keep argument for Gon's P3: two statements are wrong for common inputs. Downward: one caller (generation.go:444), unexported, ARCHITECTURE's Start, Stale trigger, and Takeover bullets state every case; CRF-60 was P4 for the same class. P4.
  • CRF-99: Bisky P3 vs Gon P4 (Gon's point is the comment at :122, "the stale task invalidates"; merged). Upward case: Bisky reproduced the sequence with task IDs and Release after the Invalidate (first outcome=interrupted status=Error), so the test asserts that a stale task's outcome is discarded, the opposite of the rule cf34118 introduced; CRF-24 (a test asserting what production does not do) was P3. Downward: no production caller invalidates a superseded turn as interrupted (the interrupt task uses OpenToken, which returns only current), and TestRunnerTurnSpanSupersededTurnWaitsForHolder covers the production rule. Reviewers disagree; higher wins: P3.
  • CRF-100: Bisky P3. Verified: hold and Release skip uuid.Nil (turn_trace.go:137, :197); the only production Ensure (generation.go:444) passes input.TaskID, set from uuid.New() at runner.go:210; 40 Ensure calls in turn_trace_internal_test.go pass uuid.Nil. Keep argument: the branch exists only so tests can skip Release, it switches emission to emit-on-close silently, a future production caller with a zero ID would lose late outcomes with no error, and CRF-99 shows a test built on it asserting a non-production result. Downward: the four holder tests and TestWorkerRunnerReleasesTurnOnTaskExit cover the production mode, and the other holder-less tests check anchoring and rotation, which holders do not change. Structural alternative that removes the class behind CRF-99; kept at P3. Interaction: fixing CRF-100 as proposed fixes CRF-99.
  • CRF-101: Leorio P4 and Gon P4 (Gon framed it as a re-raise of CRF-77) merged. Recorded as new: CRF-77's defect (a doc naming only the user) no longer exists; the new defect is that no Go doc defines interrupted and error or the span status of each. Interaction: CRF-92 closed partly because the doc it measured against was deleted; restoring a status sentence brings that promise back, which Gon accepted on the ground that the one production caller passes nil.
  • CRF-102: Mafuuu P4. Verified: closeLocked fixes endAt (turn_trace.go:257) and emitIfReleasedLocked reads finished and outcome once at emission, so the ordering protects the outcome; the struct doc (turn_trace.go:26-28) states that reason, the Release doc states the other.
  • CRF-103: Razor P4. Verified: ARCHITECTURE.md:810 has "a task canceled after Ensure records no outcome" two sentences after the FinishTurn example of a canceled task recording completed; the PR description's Failure bullet has "A canceled generation task records no outcome".
  • CRF-104: Pen Botter P4. Verified: PromoteQueuedMessage moves R1 and I1 to interrupting (chatstate/transitions.go:889-895); the description's Interrupt bullet reads "Only a stop (by the user or a parent agent) is interrupted".
  • CRF-105: Zoro P4 and Bisky P4 merged. Zoro verified that removing lines 233-235 and pendingLocked passes the turn tests with -count=5. Contradiction recorded: Pariston's Note (turn_trace.go:236) says the loop is the part that does nothing in production, since End runs after waitForTasks and only the unclosed current turn remains; the loop still implements the tested "regardless of holders" contract, so the redundant part is the block. Pariston's Note is not posted separately.
  • CRF-106: Gon P3 to P4. Keep argument: a caller who trusts the doc expects detached work to join the caller's trace. Downgrade: the next two lines strip the span context and set ScopeBackground, the inline comment states the span rule, and the function is unexported. Downgrade stronger: P4. In scope: e9a0b17 changed the body the doc describes.
  • CRF-107: Leorio Nit. Verified: TestRunnerTurnSpanClampsStaleAnchor at turn_trace_internal_test.go:147; the stale_anchor help text (metrics.go:172) says "turn started now". CRF-82 precedent: a test name for behavior the code does not have is a Nit.
  • CRF-108 drop gate. Keep argument: reviewers and the squash commit cannot tell why organization_name was dropped (1cf3bed and a5e1978 have no body). Drop reason: PR text with no code location (CRF-43 precedent); one sentence in the review body carries the request.
  • CRF-109 drop gate. Keep argument: a reader of emitIfReleasedLocked must trace four callers to learn a close error cannot occur. Drop reason: the stored value is End's err; CRF-32 (closed R3, 1/1) accepted that it is always nil, and removing it either makes End ignore its argument or reopens CRF-32's parameter removal.
  • CRF-110 drop gate. Keep argument: one value has two names across one call. Drop reason: startLocked reassigns the parameter to now when it is zero and uses it as the span start, which startAt names.
  • CRF-111 drop gate. Keep argument: current and "open" name one turn. Drop reason: closeLocked zeroes current, so the current turn is the only unclosed one and OpenToken is accurate.
  • CRF-98 drop gate. Keep argument: test names should match production structure. Drop reason: Netero's premise that two steps never hold one turn separately is false: each committed step changes the history version, processState cancels the task and spawns a new one (runner.go:162-174) with a new ID (runner.go:210), and that task's Ensure joins the same turn.
  • Verification-only Notes with no row: Luffy (holder set justified), Pariston (chatd_test.go:15151 end-to-end test does not reproduce the promotion race; predates cf34118 and claims no such coverage), Pen Botter (CRF-88), Takumi (CRF-74, CRF-75, CRF-93), Knuckle (PromotedQueuedAt boundary, mutation), Melody (holder and emission pairings), Mafuuu (holder lifecycle), Ryosuke (emission layering), Ging-Go (CRF-40, CRF-41, CRF-64).

Law analysis

  • Head SHA: c2202f0
  • Effective LOC: +1762 -27 (17 files; 790 production, 972 test, 12 generated). Test density 55.2%.
  • Verdict: Split (vertical).
  • Enforcement: Mandatory.
  • Slice A (land first): capacity wait. capacity.go, the worker.go acquisition hooks, history_version on GetChatWorkerAcquisitionCandidates plus generated code, capacity_internal_test.go, ARCHITECTURE "Capacity wait", and the helpers chatKindAttr and Server.organizationName (first consumer). About 140 production, 165 test, 17 generated lines.
  • Slice B: this PR rebased on slice A. Turn span lifecycle, generation-path stages, background scoping of inflightContext, API-driven queue_wait. Do not split further: generation stages without the turn context publish scope=background for turn work (scopeFromContext default), and background detachment must land no later than the turn context.
  • Orchestrator seam check: chatKindAttr and organizationName are absent at base aacc28a709; newStageTestTracer exists at base; capacity_internal_test.go references neither runnerTurnSpan nor Ensure.
  • History: this PR supersedes feat(coderd/x/chatd): open a chat_turn span per prompt #28972 (turn span) and feat(coderd/x/chatd): record capacity wait as a lifecycle stage #28973 (capacity wait, stacked on feat(coderd/x/chatd): open a chat_turn span per prompt #28972), which was the same split in the opposite order. Whether the consolidation was reviewer-requested is unknown (gh returned 401); the review body asks the author.
  • R2: split carried out. Capacity wait is now feat(coderd): record a capacity_wait stage on chat acquisition #29805, whose head 5be82a7 is this PR's base. This PR at 5ffb40e: +1676 -24, 13 files, all under coderd/x/chatd. Law not re-run (effective additions fell by 86, below the +500 growth trigger).
  • R3: feat(coderd): record a capacity_wait stage on chat acquisition #29805 was closed without merging and capacity wait was dropped from the stack (commit b02f956 "refactor: drop the capacity_wait stage"; the capacity commits now appear in this branch's history with a net-zero diff). Base is now adeadee, the head of feat: trace chatd provider stages and count only output parts as first token #29770. This PR at 61546de: +2476 -78, 27 files (733 production, 1743 test). Growth since Law's R1 analysis (+1762) is +714, so Law re-runs this round.
  • R3 analysis: head 61546de. Effective LOC +2476 -78 (27 files; 733 production, 1743 test, 0 generated). Verdict: Don't split. Enforcement: Advisory. Four concerns (turn span lifecycle, stage identity and background detachment, generation-path live stages, queue_wait on promotion) form one dependency chain rooted in the turn span; vertical slices B (live stages, about 170 production) and C (queue_wait, about 65 production) are possible but small, share one risk domain, and would split TestActiveServer_TracesChatTurn. Horizontal cuts leave consumer-less layers. The R1 mandatory split is resolved by removing capacity wait from the diff.
  • R4 analysis: head 282311c, base e6f36ac. loc-analysis.json: +3034 -325 (27 files; 1012 production, 2022 test). Law excludes the regenerated docs/admin/integrations/prometheus.md reflow (+248 -248, DO NOT EDIT header) and generated_metrics: 764 production added, 68 removed, 2022 test. Verdict: Don't split. Enforcement: Advisory. Same four concerns as R3; no new concern; the >3000 trigger holds only by counting the regenerated reflow.

Contested and acknowledged

CRF-30 (Nit, generation.go:461) - four doc comments restate the code

  • Finding: Gon: the doc comments on turnContinues, turnOutcomeForError, settleLocked, and turnPromotion restate the code below them; keep only what the code cannot show (who retries the turn, what errTaskExpectedExit covers).
  • Author defense (R3): "Keeping these: they state constraints the code does not show, such as which failures the task runner retries and which exits count as abandoned. turnPromotion no longer exists."
  • Panel closure (R3, 1/1): Gon, the agent that raised it (gon PR29771 R2, resumed with the R3 prompt), accepted: turnContinues and turnOutcomeForError each carry one fact the body cannot show (the task runner retries the turn; errTaskExpectedExit means a fence or history mismatch), both comments are accurate at 61546de, and turnPromotion and settleLocked no longer exist. Residual cost Gon named: the restating clauses can go stale if a condition is added without editing the doc. Note: Melody's CRF-55 shows the turnContinues doc's claim about what the runner retries does not match runTaskWithRetry; that is a separate finding about behavior, not about restatement.

CRF-32 (P4, turn_trace.go:265) - End disables the instance permanently; err always nil

  • Finding: Gon: after End, ended is true and every later Ensure returns (ctx, 0), but the doc says only that the span closes; the only production call is End(nil) at runner teardown. Proposed renaming to Shutdown() without a parameter and documenting that later calls are no-ops.
  • Author defense (R3): "Not renaming: End matches the StageSpan vocabulary used across the stack, and a rename would add churn without changing behavior." The reply does not address the always-nil err parameter or the undocumented permanent disable.
  • Panel closure (R3, 1/1): Gon, the agent that raised it (gon PR29771 R2, resumed with the R3 prompt), accepted: StageSpan.End must also be called once, so the shared name tells a reader the same thing; the only caller is runner teardown (runner.go:91), so no turn is lost today; the always-nil err is dead input, not a defect. The current doc reads "End closes the chat_turn span. Later calls are ignored."

CRF-53 (P3, turn_trace.go:256) - interrupted turns end with error status

  • Finding: Pen Botter: closeLocked ends the span with the invalidation error, so every user stop (and, at R3, every edit) is an error-status chat_turn root; operators alerting on error roots or deriving error rates count one error per stop. Proposed ending interrupted turns without error status and keeping the cause in turn_outcome.
  • Author defense (R4): "Keeping error status: we decided interrupted turns end with error status. The span now carries a clear chat interrupted error and turn_outcome=interrupted, so trace queries can separate stops from failures."
  • Panel re-raise (R4): Pen Botter, the agent that raised it (pen-botter PR29771 R3, resumed with the R4 prompt), re-raised with new evidence: after 282311c an edit closes as abandoned with Unset status (EditAbandonsTurn asserts codes.Unset) while a stop or a running-chat promote closes with Error status (requireInterrupted asserts codes.Error), so two user actions that stop a turn report different statuses; turn_outcome already separates stops from failures, and the "we decided" defense links no discussion or Review decisions entry. Needs a human decision if the error status is intended.

CRF-58 (P4, chatloop.go:1382) - rejected tool calls record an OK tool_call stage

  • Finding: Melody: calls rejected by executeSingleTool as inactive or unknown record a tool_call stage with OK status and a near-zero histogram sample, while applyExclusiveToolPolicy rejections record no stage. Proposed starting the span after the tool resolves, or returning an error from both rejection branches.
  • Author defense (R4): "Not changing: a tool_call span records every call the model made, including rejected ones. Its error status is reserved for execution failures, and the rejection is in the tool result the model sees." The reply does not address the exclusive-policy rejections that record no stage.
  • Panel re-raise (R4): Melody, the agent that raised it (melody PR29771 R3, resumed with the R4 prompt), re-raised: the defense says a tool_call span records every call the model made, but two of three rejection paths record none (applyExclusiveToolPolicy returns at chatloop.go:543-559 before executeTools; admitStepToolCalls hook denials at generation.go:895-905 never reach it), and ARCHITECTURE.md:828 makes the same claim. Asks for one rule across the three paths, in code or in ARCHITECTURE.md.

CRF-59 (P4, turn_trace.go:124) - stale-anchor check mixes clocks

  • Finding: Knuckle: triggers are PostgreSQL now() values but lastAnchorAt holds the replica clock after a turn that opened at now, so replica-ahead skew misclassifies a prompt created shortly after as stale_anchor with no acquisition. Proposed tracking lastTriggerAt = max(lastTriggerAt, triggerAt) and comparing with it.
  • Author defense (R4): "Not changing: the proposed lastTriggerAt comparison would let turns overlap after a takeover. The skew between the database clock and this replica is milliseconds and is counted as stale_anchor."
  • Panel closure (R4, 1/1): Knuckle, the agent that raised it (knuckle PR29771 R3, resumed with the R4 prompt), accepted: with NTP the skew is milliseconds; a prompt inserted into history exists only after the previous turn finished (seconds later) and a queued prompt anchors at its later promotion time; and the proposed fix is wrong because created_at is the inserting transaction's start time, so a row committed after a takeover turn opened can carry an earlier trigger that lastTriggerAt would rotate into a span starting before the takeover turn.

CRF-76 (P3, turn_trace.go:291) - non-completed turns produce no metric

  • Finding: Chopper: closeLocked observes only completed turns and turn_outcome is span-only, so a metrics-only deployment has no turn failure signal and chat_turn p99 falls during a provider outage. Proposed a coderd_chatd_turns_total{outcome, chat_kind} counter incremented for every outcome.
  • Author defense (R5): "Not changing: turn_outcomes_total in feat: account chatd turn time by category #29772 counts every closed turn by outcome and is the failure signal. chat_turn latency covers completed turns by design, so it measures reply latency without mixing in stops and failures."
  • Panel closure (R5, 1/1): Chopper, the agent that raised it (chopper PR29771 R4, resumed with the R5 prompt), accepted: origin/scott/chatd-lifecycle/4-turn-accounting (feat: account chatd turn time by category #29772) contains this PR's head and adds coderd_chatd_turn_outcomes_total{outcome, chat_kind} incremented for every outcome (chatloop/metrics.go:316 on that branch), plus coderd_chatd_turn_time_seconds_total{category, chat_kind, outcome}, so the provider-outage case gets a failure series. Residual gap Chopper named: this PR alone records no failed, interrupted, or abandoned turn in any metric until feat: account chatd turn time by category #29772 merges; every stage metric is behind the chat-stage-metrics experiment. Orchestrator check: git merge-base --is-ancestor 676c4fd7c4 FETCH_HEAD succeeded, git grep TurnOutcomesTotal found the counter at metrics.go:108, :200, :316 on that branch, and the public REST API shows feat: account chatd turn time by category #29772 open, not draft, base scott/chatd-lifecycle/3-turn-span.

CRF-86 (P4, tasks.go:349) - ambiguous-commit settle in StartInterrupt has no test

  • Finding: Netero: the Settle(stoppedTurn) and recordQueueWait calls on StartInterrupt's ambiguous-commit return (FinishInterruption committed, machine.Update returned an error) have no test; coverage block tasks.go:348.74,352.4 had count 0, so deleting either line passes every test. Proposed letting taskRecordingPubsub.onPublish (tasks_test.go:1528) return an error and adding a TestInterruptTaskInterruptsOpenTurn variant where the FinishInterruption publish fails after the commit.
  • Author accepted (R7): "Accepted as a coverage gap for this stack. Reaching the ambiguous-commit return needs commit fault injection that the tests do not have, and I am not adding it here." No ticket. The reply does not address the proposed onPublish error hook. Leaving the branch untested needs a human decision.

CRF-89 (P4, chatloop/chatloop.go:1358) - startRun callback instead of splitting resolution from the run

  • Finding: Ryosuke: executeSingleTool gained a 15th parameter, the startRun callback, only so runCall can start tool_call between tool resolution and tool.Run; the stage exists only if the callback's captured assignment runs, the deferred End relies on a nil receiver for calls that never ran, and startRun != nil exists for tests that pass nil. Proposed splitting the alias, active, and lookup checks into a function that returns the resolved tool or the rejection result, with runCall starting the stage and running the tool.
  • Author defense (R7): "Keeping the callback. Splitting resolution from the run would restructure executeSingleTool to move one span start, and the callback is the smaller change." The reply does not address the nil-receiver End or the test-only nil check.
  • Panel closure (R7, 1/1): Ryosuke, the agent that raised it (ryosuke PR29771 R5, resumed with the R7 prompt), accepted: executeSingleTool is internal with one production caller, nothing breaks at runtime, and TestExecuteLocalToolsToolCallStage/RejectedCallsRecordNoStage (chatloop/stage_internal_test.go:612-646) fails if an inactive or unknown call records a tool_call span or histogram sample, so the no-stage rule is enforced by a test though not by control flow.

CRF-90 (P4, generation.go:831) - thinking stages recorded in chatd with an unreachable guard

  • Finding: Zoro: recordThinkingStages records thinking stages in chatd from timestamps chatloop.GenerateAssistant already holds next to Stages and StageModel, and its index >= len(step.ReasoningCompletedAt) return, doc sentence, and StopsAtFirstUnpairedStart subtest cover input production never builds (processStepStream appends both slices in the same branch). Proposed recording thinking in GenerateAssistant and deleting recordThinkingStages and its test.
  • Author defense (R7): "Keeping this as is for this stack. Moving thinking recording into chatloop is a refactor with no behavior change, and the length guard is one line." The reply does not address the StopsAtFirstUnpairedStart subtest. 562baea shortened the doc to "pairs reasoning start and completion timestamps by index".
  • Panel closure (R7, 1/1): Zoro, the agent that raised it (zoro PR29771 R5, resumed with the R7 prompt), accepted: the defended code leaves a three-line guard and the StopsAtFirstUnpairedStart subtest (generation_internal_test.go:161) covering input processStepStream never produces and records thinking one package away from stream, with no runtime effect; 562baea removed the misleading doc sentence about unpaired parts.

CRF-92 (P4, turn_trace.go:200) - unset status for interrupted turns depends on the caller passing nil

  • Finding: Gon: TurnOutcomeInterrupted's doc promised unset span status, but Invalidate accepts an error with any outcome, only the caller at tasks.go:272 passing nil keeps the promise, neither the Invalidate nor the close doc says so, and three tests pass xerrors.New(...) with TurnOutcomeInterrupted (one closes an interrupted turn with error status). Proposed stating in the Invalidate doc that err is nil for TurnOutcomeInterrupted or having Invalidate ignore err for that outcome, and passing nil in those tests.
  • Author defense (R7): "Keeping this as is. The only caller that invalidates with interrupted passes nil, and enforcing that inside Invalidate would add a rule for a case no caller produces." The reply rejects the enforcement option only. At 562baea the TurnOutcomeInterrupted doc in chatloop/stage.go is deleted, ARCHITECTURE.md:808 still says interrupted turns leave the status unset, and turn_trace_internal_test.go:126, :432, and :587 still pass an error with that outcome.
  • Panel closure (R7, 1/1): Gon, the agent that raised it (gon PR29771 R5, resumed with the R7 prompt), accepted: StartInterrupt (tasks.go:268) is the only production caller that passes TurnOutcomeInterrupted and it passes nil, so every interrupted turn ends with unset status as ARCHITECTURE.md states, and 562baea deleted the TurnOutcomeInterrupted doc whose promise the finding measured against. Note: in the same report Gon asks (CRF-101) to restore a one-line TurnOutcomeInterrupted doc that includes "status unset".

Round log

Round 1

Netero + Law (infrastructure). Netero: 1 P2, 4 P3, 1 Nit, 1 Note (Note dropped as CI status, carried in body). Law: mandatory vertical split (capacity wait into its own PR landing first). Panel skipped per Law mandatory-split gate; event COMMENT because the CLI rejects REQUEST_CHANGES without a panel. Reviewed against aacc28a..c2202f0.

Round 2

Churn guard PROCEED: CRF-1 to CRF-6 claimed fixed (CRF-1, 3, 4, 5 verified by Netero and Bisky; CRF-2 moved to #29805; CRF-6 retitled). Law split carried out (#29805 is the new base). Tool v0.12.0. Netero: 1 P2, 4 P3, 1 P4 (no P0, panel proceeds). Round type: first panel round for this PR and a restructure (split, rebase, new test file content), so full panel. Required specialists: ging-go (.go), Killua (generation loop, span allocation), Luffy (new feature), Melody (paired Complete/Settle and stage enumerations), Pen Botter (ARCHITECTURE.md), Ryosuke (Go), Takumi (mutex, task cancellation). Core: Pariston, Mafuuu. Bisky (tests changed), Gon and Leorio (once-per-PR floor, never appeared). 12 exceeds the 6-8 focused target because of the specialists above plus the floors. Wildcard: Zoro (random from distilled set). New posted this round (Netero plus panel): 6 P2, 13 P3, 10 P4, 7 Nit; convergence on the clamped acquisition (8 reviewers) and the token race (4). Killua: no findings. 3 praise/verification Notes dropped, 1 PR-text P4 carried in body, 1 OOS. Reviewed against 5be82a7..5ffb40e.

Round 3

Tool v0.13.0. Churn guard PROCEED: CRF-8 to CRF-44 (posted, open; CRF-43 was not posted) classified as 34 author-fixed claims and 2 contested (CRF-30, CRF-32); the classifications are author claims, not verified fixes. Branch rebased onto #29770 (adeadee); #29805 closed unmerged and capacity wait dropped (b02f956). Inventory validation of non-posted items against 61546de: CRF-7 disposition holds (context.md lists fmt as failed, but the job concluded cancelled, and gofumpt, gci, and markdown-table-formatter report no changes on the 24 changed .go files and docs/admin/integrations/prometheus.md); CRF-43 is moot (description now names #29770); CRF-45 is moot (recordCapacityWait no longer exists; capacity.go is not in the diff); CRF-46 to CRF-48 asked for no change and none of their assumptions changed a disposition. No reopenings.
Law: Don't split, Advisory (recorded under Law analysis). Netero (advisory, post-panel): 1 P3 (CRF-49), 1 Nit (CRF-50, title scope again excludes changed files). Round type: restructure. Since the R2 panel, 14 files entered the PR diff (among them chatloop/chatloop.go, chatloop/metrics.go, runner_manager.go, tasks.go, worker.go, docs/admin/integrations/prometheus.md, scripts/metricsdocgen/generated_metrics), the turn lifecycle in turn_trace.go and the tool_call stage were rewritten (toolCallStageRecorder removed, tool_call moved into chatloop.executeTools, executeSingleTool signature changed), and effective additions grew from +1676 to +2476 (+800 > 500). Full panel. Required specialists: ging-go (.go), Killua (generation loop, per-tool-call spans, organizationNames cache), Knuckle (queue_wait and turn anchors read DB-defaulted created_at values and compare them with the replica clock), Kurapika (new dbauthz.AsChatd read in organizationName, organization name exported to traces), Luffy (new feature), Melody (tool dispatch in executeTools, Complete/Settle pairing, four promoting transitions), Pen Botter (ARCHITECTURE.md, prometheus.md), Ryosuke (Go package boundaries across chatloop, chatstate, chatd), Takumi (runnerTurnSpan mutex, task cancellation, runner shutdown). Core: Pariston, Mafuuu. Bisky (tests changed), Leorio (restructure). Gon: dispute check-in for CRF-30 and CRF-32 (reuse agent 60423ae5, gon PR29771 R2) and restructure; not counted toward the size target. Size target 9-12 (auth-context code makes this multi-domain); 13 counted because of the 9 required specialists plus the core and the Bisky and Leorio floors. Wildcard: Razor (shuf over Meruem, Chopper, Kite, Zoro, Knov, Razor).
Panel results: Gon (reused agent 60423ae5, gon PR29771 R2, via message_agent) accepted both contested defenses, so CRF-30 and CRF-32 closed 1/1, and verified its fixed findings. Kurapika: no findings. Killua: no findings (2 Notes dropped). CRF-15 re-raised by Takumi (stale task's late Ensure gets the successor's token; unit-level reproduction); status set to Open with a reply on thread PRRT_kwDOGkVX1s6lqT8t. Convergence: turn_outcome classification (CRF-51: Razor, Pariston, Mafuuu, Pen Botter; shutdown reproduced by three reviewers, edits by Razor), inter-step interrupt (CRF-52: Takumi, Melody, Mafuuu), chat_turn histogram (CRF-54: Luffy, Pen Botter), executeSingleTool doc (CRF-57: Gon, Leorio), PromoteQueuedMessageResult (CRF-62: Ryosuke, Razor). Interactions: CRF-55 (turnContinues logic) and CRF-56 (guard untested) touch the same guard; CRF-57 and CRF-58 share the rejection branches; CRF-59 and CRF-61 both concern lastAnchorAt's clock. New this round (Netero plus panel): 1 P2, 7 P3, 6 P4, 2 Nit; 1 OOS (CRF-65); 4 Notes dropped (CRF-66 to CRF-69). CI at head: fmt, gen, lint, lint-docs, check-build, test-e2e, offlinedocs, and all test-go jobs concluded cancelled (public jobs API); title failed (CRF-50). Event COMMENT (no P0 or P1). Reviewed against adeadee..61546de.

Round 4

Tool v0.13.0 (unchanged). Branch rebased onto e6f36ac (new head of #29770, scott/chatd-lifecycle/2-provider-stages); git range-diff shows the round 3 commits unchanged except 04cb2d2 (the original feat commit, conflict-resolved against the new base). Fix commits since the rebased round 3 head 60c61df: fe9b6f7 (test import) and 282311c. Churn guard PROCEED: 14 author-fixed claims (CRF-15, CRF-49 to CRF-52, CRF-54 to CRF-57, CRF-60 to CRF-64; CRF-50 by retitle to feat: open a chat_turn span per prompt) and 3 contested (CRF-53, CRF-58, CRF-59); the classifications are author claims, not verified fixes. The two failing title jobs in context.md ran at 17:24:15 and 17:24:16 on 282311c; a third contrib run at 17:24:42 on the same head succeeded (public runs API), so CRF-50's failure evidence is stale. Only contrib runs exist for 282311c. Inventory validation of non-posted items against 282311c: CRF-7 holds (no fmt failure at head); CRF-43 and CRF-45 stay moot; CRF-46 to CRF-48 unchanged; CRF-65 holds (chatloop.go changed only the executeSingleTool doc); CRF-66 to CRF-68 hold (turnTriggerTime and organizationName unchanged); CRF-69 was a verification record, and the PromotedQueuedAt sourcing it checked now comes from PromoteQueuedMessageResult (reviewers verify). No reopenings. Effective additions +3034 (1012 production, 2022 test); growth since Law's R3 analysis (+2476) is +558, so Law re-runs. Of the 492 changed lines in docs/admin/integrations/prometheus.md, git diff -w shows 2 real changes (the separator row and the stage_duration_seconds help text); the rest is table re-padding.
Law: Don't split, Advisory (recorded under Law analysis; Law notes prometheus.md carries a DO NOT EDIT header, so the table reflow is regenerated output). Netero (advisory, post-panel): CRF-70 (P3, abandoned branch untested; Netero framed it as a re-raise of CRF-56, recorded as a new finding because the requested test was added and the untested branch comes from the stepAbandonsTurn rewrite), CRF-71 (P4), CRF-72 (P4), CRF-73 (Note, body). Round type: restructure. loc-analysis.json effective additions grew from +2476 at the R3 panel to +3034 (+558 > 500; the growth includes the regenerated table reflow), and the outcome classification was rewritten rather than patched (turnContinues and turnOutcomeForError removed, stepAbandonsTurn added, StartInterrupt now invalidates the open turn through the new OpenToken). Full panel. Required specialists (triggers against the PR diff, unchanged from R3): ging-go, Killua, Knuckle, Kurapika, Luffy, Melody, Pen Botter, Ryosuke, Takumi. Core: Pariston, Mafuuu. Bisky (tests changed), Gon and Leorio (restructure). Dispute check-in: Pen Botter (CRF-53, agent 6ba33796), Melody (CRF-58, agent 92e00722), Knuckle (CRF-59, agent 069749e6), resumed with message_agent; they are also required specialists. Size target 9-12; 14 counted, above target because of the 9 required specialists, the core, Bisky, and the Gon and Leorio restructure floor. Wildcard: Chopper (shuf over Meruem, Chopper, Kite, Zoro, Knov, Razor).
Panel results: dispute check-ins: Pen Botter re-raised CRF-53, Melody re-raised CRF-58, Knuckle accepted CRF-59 (closed 1/1); all three were the R3 agents resumed with message_agent. Prior fixes verified by reviewers at 282311c: CRF-15 (Netero mutation, Takumi), CRF-49 (Netero, Bisky), CRF-51 (Netero, Pariston, Mafuuu), CRF-52 (Netero, Melody, Mafuuu; CRF-72 gap), CRF-54 (Netero, Luffy, Bisky, Pen Botter), CRF-55 (Melody), CRF-56 (Bisky), CRF-57 (Gon), CRF-60 and CRF-61 (Leorio), CRF-62 (Netero, Ryosuke mutation, Knuckle), CRF-63 (Bisky), CRF-64 (Ging-Go); CRF-50 verified by the passing contrib run at 17:24:42. No findings: Ging-Go (1 OOS), Killua, Kurapika, Luffy, Knuckle. Convergence: OpenToken without a ctx check (CRF-74: Takumi, Ryosuke, Pariston; two unit reproductions), interrupted doc vs parent-agent interrupts (CRF-77: Mafuuu, Leorio, Chopper). Interactions: CRF-75's fix (record nothing on a fence mismatch) removes the Invalidate call that CRF-70 finds untested and the predicate CRF-71 finds duplicated; CRF-74 and CRF-81 are on the same line; CRF-76 is a consequence of the completed-only option the R3 review offered for CRF-54. New posted (Netero plus panel): 4 P3, 5 P4, 3 Nit; 3 OOS; 1 Note (body); 2 re-raises (CRF-53, CRF-58) as thread replies. Only contrib runs exist for 282311c (public runs API), so fmt, lint, and Go tests have not run on this commit. Event COMMENT (no P0 or P1). Reviewed against e6f36ac..282311c.

Round 5

Tool v0.13.0 (unchanged). Branch rebased onto ddc808b (new head of #29770); git range-diff shows round 4 commits unchanged except 8aa78f5, 433862f (capacity commits, net zero), and 4ba169c (rebased 282311c, conflict-resolved). Fix commits since 4ba169c: b34b2c3 (code, tests, regenerated metric docs), 43ec647 (ARCHITECTURE.md), 676c4fd (test comment). Churn guard PROCEED: 13 author-fixed claims (CRF-53 and CRF-58, which the panel re-raised in R4, plus CRF-70 to CRF-72, CRF-74, CRF-75, CRF-77 to CRF-82) and 1 contested (CRF-76); classifications are author claims, not verified fixes. Inventory validation of non-posted items against 676c4fd: CRF-83 holds (git diff -U0 ddc808b901..HEAD has no hunk at chatloop.go:1217, the wg.Add(1) line), CRF-84 holds (ARCHITECTURE.md hunk starts at 785; the I0 row is at 75), CRF-85 holds (metrics.go hunks at 47, 174, 188; the chats gauge is at 127); CRF-73 (Note, body) needs no reopening: with stepAbandonsTurn removed, a fence mismatch records nothing on any path, which ARCHITECTURE.md now states. No reopenings. Law not re-run: effective additions +3113, growth +79 since Law's R4 analysis (+3034). b34b2c3 again re-pads the whole prometheus.md table (+246 -246).
Netero (advisory, post-panel): CRF-86 (P4; Netero framed it as a re-raise of CRF-72, recorded as a new finding because the requested code change was made and only its test is missing), CRF-87 (OOS). Round type: fix round. Since the R4 panel (282311c, rebased as 4ba169c): no files added, renamed, or moved (27 files both rounds); no exported signature change outside unexported types (runnerTurnSpan.OpenToken gained ctx; executeSingleTool gained startRun); the changes remove or patch code the R4 findings named (stepAbandonsTurn and its call deleted, span start moved behind tool resolution, nil error for interrupted); the PR goal is unchanged; effective additions grew +79. Panel step 1: Chopper (contested CRF-76, dispute check-in with agent 16d8c274 via message_agent); personas with open findings whose code the new commits touch: Pen Botter (CRF-53, CRF-78), Melody (CRF-58), Takumi (CRF-74, CRF-75), Ryosuke and Pariston (CRF-74), Mafuuu and Leorio (CRF-77), Bisky (CRF-79), Gon (CRF-80 to CRF-82). Step 2 (triggers against files changed since the R4 panel): ging-go (.go), Killua (tool_call span lifecycle moved into executeSingleTool), Melody (tool dispatch), Pen Botter (ARCHITECTURE.md, prometheus.md), Ryosuke (Go), Takumi (ctx check under the turn mutex); Knuckle, Kurapika, Luffy not triggered (no SQL, auth, or feature surface changed). Gon and Leorio floors met by step 1. 12 panel reviewers. Wildcard: Zoro (shuf over Meruem, Kite, Zoro, Knov, Razor).
Panel results: dispute check-in: Chopper (the R4 agent, resumed with message_agent) accepted CRF-76, closed 1/1; orchestrator confirmed #29772 is open, not draft, based on scott/chatd-lifecycle/3-turn-span, contains 676c4fd, and defines TurnOutcomesTotal. Prior fixes verified by reviewers at 676c4fd: CRF-53 (Pen Botter, Bisky, Netero), CRF-58 (Melody, Netero mutation, Bisky, Leorio, Mafuuu), CRF-70 and CRF-71 (Netero; code removed), CRF-72 (Netero; code correct, test missing as CRF-86), CRF-74 (Takumi, Ryosuke, Pariston, Bisky mutation), CRF-75 (Takumi, Pariston; Bisky found a test would pin deleted code), CRF-77 (Chopper, Mafuuu, Leorio), CRF-78 (Pen Botter), CRF-79 (Bisky and Netero mutations), CRF-80 to CRF-82 (Gon, Netero). CRF-73 stays Note (body); moot per Netero. No findings: Chopper, Melody, Takumi (1 Note), Pariston (1 OOS), Mafuuu, Bisky, Ging-Go, Killua. Convergence: the stale PR description Failure bullet (CRF-88: Leorio, Pen Botter, Ryosuke). Contradiction: Ryosuke (split resolution from the run) vs Pariston (callback justified) on CRF-89. Interactions: CRF-86 (untested ambiguous-commit settle in StartInterrupt) and CRF-94 (generation finish paths have no ambiguous-commit handling) are the same failure mode on different transitions; CRF-92's nil-error rule is what makes CRF-53's fix hold. Reviewers ran targeted go test (including -race -count=3) in chatd and chatloop at 676c4fd; all passed. Public check-runs API for 676c4fd lists only title, cla, release-labels, and community-label, so fmt, lint, and the Go test jobs have not run on this commit. New posted (Netero plus panel): 6 P4. Body: 2 OOS (CRF-87, CRF-94), 1 Note (CRF-93). Dropped: CRF-95, CRF-96. No re-raises. Event COMMENT (no P0 or P1). Reviewed against ddc808b..676c4fd.

Round 6

BLOCKED. No review. Tool v0.13.0 (unchanged). Base dc303a5, head 562baea. Branch rebased (R5 head 676c4fd is now 4e3eb96) plus five commits: cf34118 (emit a turn after the tasks that ran it return; Ensure takes a task ID, new Release), 2bc9e3e (ARCHITECTURE.md), 1cf3bed and a5e1978 (drop the organization name from stage identity), 562baea (trim comments). Churn guard BLOCKED: 5 of 6 open findings silent (CRF-86, CRF-88, CRF-89, CRF-90, CRF-92: no reply on their threads and the requested change not made at head), 1 addressed without a reply (CRF-91: 562baea deleted the wrong test doc). Every top-level author comment since R5 is /coder-agents-review. No Netero, Law, or panel ran, so cf34118 and 1cf3bed are unreviewed. Posted a body-only COMMENT listing the silent findings and the response each needs.

Round 7

Tool v0.13.0 (unchanged). Head 562baea, base dc303a5 (same as R6; no commits since). The author replied on all five R6-silent threads and edited the PR description. Churn guard PROCEED: CRF-88 addressed (description Failure bullet now says the step records no outcome and leaves the turn open), CRF-86 acknowledged (coverage gap, no ticket), CRF-89, CRF-90, CRF-92 contested. Inventory validation of non-posted items against 562baea (last validated at 676c4fd; R6 was blocked): CRF-83 holds (chatloop.go hunks at 1185 and 1323; wg.Add(1) at 1201), CRF-84 holds (ARCHITECTURE.md hunk 785-842), CRF-85 holds (metrics.go hunks 158, 172; chats gauge at 111), CRF-87 holds (tasks.go hunk +345,2 covers only the Settle and recordQueueWait lines), CRF-94 holds (committedStateAfterUpdateError callers only at tasks.go:344 and :482), CRF-93 holds (Settle still closes the turn and fixes its end time; emission now waits for Release, which does not move the end time), CRF-66 holds (turnTriggerTime unchanged), CRF-67 to CRF-69 are moot (1cf3bed removed organizationName, GetOrganizationByID, and dbauthz.AsChatd from the diff). No reopenings. Law not re-run: effective additions +3056, growth +22 since Law's R4 analysis (+3034).
Netero (advisory, post-panel): CRF-97 (P4), CRF-98 (Nit).
Round type: restructure. Since the R5 panel (676c4fd, rebased as 4e3eb96): cf34118 rewrote runnerTurnSpan (turn_trace.go +/-351: one open turn replaced by a map of pending turnStates with holders; emission moved from close to the last Release) and changed signatures (Ensure gained a task ID, newRunnerTurnSpan lost a parameter, new Release, new StageSpan.EndTurn); 1cf3bed removed organization identity from stages. No files added or renamed (27 files); effective additions fell 57 since the R5 panel. Full panel. Required specialists (triggers against the PR diff): ging-go (.go), Killua (pending-turn map and holder lifecycle), Knuckle (PromotedQueuedAt sourced from queued rows' created_at in chatstate/transitions.go), Luffy (new feature), Melody (tool_call dispatch), Pen Botter (ARCHITECTURE.md, prometheus.md), Ryosuke (Go package boundaries), Takumi (mutex, task release, shutdown). Kurapika not triggered: 1cf3bed removed the only authorization code (dbauthz.AsChatd in organizationName), and no endpoint, token, or input handling is in the diff. Core: Pariston, Mafuuu. Pool: Bisky (tests changed), Gon and Leorio (restructure floor). Dispute check-in: Ryosuke (CRF-89, agent fe9e1f15), Gon (CRF-92, agent fbca2126), Zoro (CRF-90, agent a47b970a), each the R5 agent resumed with message_agent; Ryosuke and Gon are also on the panel for their triggers, Zoro does not count toward the size target. Size target 9-12; 13 counted, above target because of the 8 required specialists, the core, Bisky, and the Gon and Leorio restructure floor. Wildcard: Razor (shuf over Meruem, Chopper, Kite, Knov, Razor).
Panel results: dispute check-ins: Ryosuke accepted CRF-89, Zoro accepted CRF-90, Gon accepted CRF-92 (each closed 1/1; all three were the R5 agents resumed with message_agent). CRF-86 stays acknowledged without a ticket; the body asks for a human decision. CRF-60 re-raised (Gon, Leorio): 562baea deleted the Ensure case list its fix added. Prior fixes verified at 562baea: CRF-88 (Pen Botter), CRF-91 (Netero), CRF-74 and CRF-75 (Takumi), CRF-58 (Melody), CRF-14 to CRF-19, CRF-51, CRF-52 (Mafuuu); CRF-93 holds (Takumi). The emission redesign itself was checked by Takumi (FinishTurn promotion ordering, runner exit after waitForTasks, -race -count=3), Melody (Ensure/Release pairing, outcome precedence), Pariston (holder check mutation fails four tests), Knuckle (PromotedQueuedAt mutation), and Mafuuu (holder lifecycle, coverage). No findings: Ging-Go, Killua, Knuckle, Luffy (1 Note), Melody, Takumi, Pariston (2 Notes), Ryosuke. Convergence: test fidelity after cf34118 (CRF-99 and CRF-100: Bisky, Gon; CRF-97: Netero), doc deletions in 562baea (CRF-60 re-raise: Gon, Leorio; CRF-101: Leorio, Gon; CRF-102: Mafuuu), End redundancy (CRF-105: Zoro, Bisky; Pariston Note). Interactions: CRF-100's fix removes CRF-99's cause; CRF-101's restored status sentence brings back the promise CRF-92 measured against. Dropped: CRF-98, CRF-108 (carried in body), CRF-109, CRF-110, CRF-111. New posted (Netero plus panel): 2 P3, 7 P4, 1 Nit; 1 re-raise reply (CRF-60). Body: OOS CRF-83 to CRF-85, CRF-87, CRF-94 are not repeated (R4 and R5 bodies listed them); CRF-86 human decision; CRF-108 sentence. Public check-runs for 562baea at setup: success (6 passed, 2 skipped). Event COMMENT (no P0 or P1). Reviewed against dc303a5..562baea.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First-pass review: Netero's mechanical pass plus Law's decomposition analysis. The full persona panel has not reviewed this PR yet. It will review once the split below is done.

What this does well: production code reads time only from the injected clock (w.opts.Clock.Now(), stages.Now()); the new history_version column on the candidate query is matched by the generated scan; the new spans at prepare, commit, compaction, retry backoff, and generation step end with the step's real error; and turn tokens make Complete, Invalidate, and Settle no-ops for a task whose turn was already replaced. Test density is 55%. Netero reports that build, vet, and DB=ci go test ./coderd/x/chatd/ pass at 84.4% statement coverage. Netero raised 1 P2, 4 P3, 1 Nit, and 1 Note.

Requested change: split this PR. Law returned a mandatory vertical split. Capacity wait is a second feature in a different subsystem. It changes the worker acquisition loop every chat passes through (acquireOnce, acquireCandidate), adds history_version to the acquisition candidate query, keeps per-worker refusal state that is pruned only when a candidate batch is short, and can issue GetOrganizationByID on the acquisition goroutine on a cache miss. Its correctness argument (history-version matching, prune only on a short batch, the multi-replica lower bound) is independent of the turn lifecycle's (stale-task tokens, Complete/Settle ordering, Invalidate after Complete). Reviewed together, the acquisition change reads as a footnote to roughly 1,500 lines of tracing.

Proposed cut:

  1. Slice A, lands first: feat(coderd): record capacity_wait stage on acquisition. It contains capacity.go, the worker.go hooks, the chats.sql column and generated code, capacity_internal_test.go, the ARCHITECTURE "Capacity wait" section, and chatKindAttr plus Server.organizationName, since slice A is their first consumer. Roughly 140 production, 165 test, and 17 generated lines. The scope is coderd rather than Law's coderd/x/chatd because the slice touches coderd/database (see CRF-6).
  2. Slice B: this PR rebased on slice A, keeping the turn span lifecycle, the generation-path stages, background scoping of inflightContext, and the API-driven queue_wait.

Do not split slice B further. scopeFromContext defaults to ScopeBackground, so generation stages landed without the turn context would publish scope=background for turn work. inflightContext must also detach no later than the turn context starts carrying scope=turn.

The seam checks out. recordCapacityWait uses only RecordAs, ContextWithChatKind, and ContextWithOrganization, which exist at the base, plus the two helpers. capacity_internal_test.go references neither runnerTurnSpan nor Ensure, and its newStageTestTracer helper exists at the base.

This needs a human answer: this PR supersedes #28972 ("open a chat_turn span per prompt") and #28973 ("record capacity wait as a lifecycle stage", stacked on #28972). That was the split Law asks for, in the opposite order. If a reviewer asked for the two to be combined, say so on this PR. That is a human decision and it overrides this verdict. Otherwise, split.

CI: title fails because of CRF-6. fmt also fails, but the cause is not in the diff as far as we can tell. Netero ran the Go formatters and make fmt/markdown on every changed file with no diff, go mod tidy -diff exits clean at head, and #29772, whose base is this PR's head, does not list fmt among its failures. Read the job log or re-run it. Storybook fails here and on #29772, and neither PR changes files under site/.

As Law put it: "Pattern caught: two features disguised as one."


.github/workflows/contrib.yaml:203-218

Nit [CRF-6] The title scope coderd/x/chatd does not contain every changed file, so the title CI job fails. (Netero)

.github/workflows/contrib.yaml:203-218 fails when any changed file is outside the scope. The PR changes coderd/database/queries.sql.go and coderd/database/queries/chats.sql. AGENTS.md requires the scope to be "a real filesystem path containing every changed file".

Netero's fix: retitle to feat(coderd): open a chat_turn span per prompt. After the split, only slice A touches coderd/database, so slice B could keep coderd/x/chatd.

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/generation.go Resolved
Comment thread coderd/x/chatd/worker.go Resolved
Comment thread coderd/x/chatd/turn_trace.go Resolved
Comment thread coderd/x/chatd/generation_preparer.go Resolved
Comment thread coderd/x/chatd/generation.go Resolved
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/3-turn-span branch from c2202f0 to 7a5cd22 Compare September 23, 2026 19:37
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/3-turn-span branch from 7a5cd22 to 9185854 Compare September 23, 2026 20:27
@jscottmiller jscottmiller changed the title feat(coderd/x/chatd): open a chat_turn span per prompt feat(coderd): open a chat_turn span per prompt Sep 23, 2026
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/3-turn-span branch from 9185854 to 5ffb40e Compare September 23, 2026 21:04
@jscottmiller
jscottmiller removed this pull request from stack #29773 September 23, 2026 21:05
@jscottmiller
jscottmiller changed the base branch from scott/chatd-lifecycle/2-provider-stages to scott/chatd-lifecycle/3-capacity-wait September 23, 2026 21:05
@linear-code

linear-code Bot commented Sep 24, 2026

Copy link
Copy Markdown

CODAGT-1102

@jscottmiller

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR opens a chat_turn root span per prompt in the chat runner and parents the generation stages under it. This round has 6 P2, 13 P3, 10 P4, and 7 Nit findings.

The round 1 findings are fixed (recording-tracer test, runnerTurnSpan.Context removed, mcp_connect outcome, canceled retry_backoff test, title), and capacity wait moved to #29805, which is now this PR's base.

Four findings share one cause: a turn is identified by a clamped timestamp held in one runner's memory, and its outcome by which helper ran. Keying the turn on the trigger message ID or history version, and deriving the outcome from the status the step committed, would remove the conditions behind the clamped acquisition, the token handoff to a new prompt, the takeover acquisition, and the fail-closed outcome (inferred from the code, not implemented).

The PR description's "activating the PR 2 stages" should name the PR number.

Out of scope (needs a ticket or explicit acceptance by a human):

  • coderd/x/chatd/capacity.go:88: recordCapacityWait (#29805) builds the same chat kind plus organization context pair by hand, so a shared helper added in this PR would leave that copy separate.

coderd/x/chatd/generation.go:577

P3 [CRF-19] A turn that enters requires_action is neither completed nor invalidated, so chat_turn includes the time a client takes to submit dynamic tool results, up to requiresActionTimeout (5 minutes). (Pen Botter, Bisky, Mafuuu)

Settle does nothing because the turn is not finished, and the task spawned after CompleteRequiresAction or CancelRequiresAction gets the same token from Ensure. Neither ARCHITECTURE.md's Lifecycle tracing section nor any test states this, so a reader of the chat_turn histogram cannot separate server latency from client wait. Either record the wait as its own stage or state in ARCHITECTURE.md that chat_turn includes it, and add a TestActiveServer_TracesChatTurn case that enters requires_action and asserts the turn count and the turn's end time.

🤖

coderd/x/chatd/generation.go:1432

P3 [CRF-20] The fail-closed hook path commits FinishError without calling Invalidate, so the turn closes at runner teardown as turn_outcome=abandoned with OK status. (Ryosuke)

commitGenerationStep returns the nil result of afterGenerationOutcome to StartGeneration, which neither invalidates (no error) nor settles (not finished). The same PostCommitError without a commit goes through finishGenerationError and records error, so a hook failure is reported two ways. Call input.Turn.Invalidate(input.TurnToken, chatloop.TurnOutcomeError, commitHooks.PostCommitError) after the commit succeeds, or classify the outcome from the committed chat status in StartGeneration.

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/chatstate/transitions.go Resolved
Comment thread coderd/x/chatd/generation.go Resolved
Comment thread coderd/x/chatd/generation.go Resolved
Comment thread coderd/x/chatd/chatd.go Resolved
Comment thread coderd/x/chatd/generation.go Resolved
Comment thread coderd/x/chatd/generation.go Resolved
Comment thread coderd/x/chatd/stage_internal_test.go Resolved
Comment thread coderd/x/chatd/stage_internal_test.go Resolved
Comment thread coderd/x/chatd/generation_preparer.go Resolved
Comment thread coderd/x/chatd/chatd.go Resolved
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/3-turn-span branch from 5ffb40e to 61546de Compare September 25, 2026 05:04
@jscottmiller
jscottmiller removed this pull request from stack #29806 September 25, 2026 05:05
Scott Miller added 25 commits September 30, 2026 13:44
The acquisition loop remembers when each chat was first refused a
concurrent-agent capacity slot, with the chat's history version at
that refusal, and records a capacity_wait stage from that refusal
when it later acquires the chat at the same version. Waits are
dropped on non-capacity skips and pruned when the candidate batch is
complete. The acquisition candidates query selects history_version.

Adds Server.organizationName, a cached organization name lookup for
stage span attributes, and chatKindAttr.
Remove the per-worker capacity wait bookkeeping, the acquisition
candidate columns it read, and its stage, docs, and tests. Keep the
organization name cache and chatKind, and add withStageIdentity, which
puts a stage's scope, chat kind, and organization on a context.
finishGenerationError classified the turn outcome from the task context
after the FinishError commit. That commit publishes the status change
that cancels the task, so a provider error could be recorded as
interrupted. Classify before the commit and invalidate with the
precomputed outcome.

Generated by Coder Agents.
Drop tests that repeat other coverage, merge overlapping outcome
subtests, and share span and anomaly helpers with the external test
package. Pin turn anchors with a mock clock and fix the unsettled turn
test so it no longer passes through trigger rotation.

Cover the gaps the review found: the runner handing the takeover flag
to its turn span, runner shutdown abandoning an open turn, fail-closed
post_tool_use failures, the stop hook finish path and its queue_wait,
mcp_connect failure wiring, a provider retry staying in one turn, the
interrupted outcome, and outcome classification when the finishing
commit cancels the task.

Generated by Coder Agents.
- Set a stage's scope, chat kind, and organization through
  withStageIdentity in runnerTurnSpan, inflightChatContext, and
  recordQueueWait; startLocked reuses contextLocked.
- recordQueueWait reads the tracer clock, so promoting callers pass only
  the committed chat and the queued time.
- Drop hand-set provider attributes that StageModel now carries.
- Describe the turn scope as latency attributable to a prompt.
Only the interrupt task records interrupted: it marks the open turn
before FinishInterruption commits and settles it afterwards, so a stop
between steps is also counted. A canceled generation task records no
outcome, so shutdown, loss of ownership, and message edits close the
turn as abandoned with unset status, and a canceled task's late Ensure
gets the zero token instead of its replacement's.

A failed step abandons its turn only for an expected, non-retryable
exit on a live context; every error the task runner retries continues
the turn. chat_turn is observed on the stage histogram only for
completed turns. PromoteQueuedMessageResult reports PromotedQueuedAt
like the other promoting transitions.
runnerTurnSpan keeps every turn that has not been emitted, keyed by
turn token. Closing a turn fixes its end time; its span is emitted once
every task that joined it has called Release, which runTask does after
the task returns. An outcome a replaced task records while unwinding,
such as Complete after a FinishTurn that promoted a queued prompt, now
reaches its own turn instead of the turn closing as abandoned.

Generated by Coder Agents.
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/3-turn-span branch from 1be3031 to 93913d8 Compare September 30, 2026 14:02
@jscottmiller
jscottmiller merged commit fc0af7f into main Sep 30, 2026
32 checks passed
jscottmiller pushed a commit that referenced this pull request Sep 30, 2026
…t token (#29770)

**TL;DR:** Traces the model provider call: a `stream` stage, a nested
`time_to_first_token` stage, and one `provider_attempt` per HTTP round
trip, labelled with provider type and model. Behavior change:
`coderd_chatd_ttft_seconds` now counts only a real output part as the
first token, so error, warning, and finish parts no longer count.

Instruments the provider call.

**Stream and time to first token.** `GenerateAssistant` records a
`stream` stage from stream open to close and a nested
`time_to_first_token` stage. The TTFT window closes once, on the first
streamed output part: any part other than warnings, finish, or error,
including start markers such as `text_start`. A closed window observes
the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same
elapsed time. A failed open, an error part, or a release without an
output part ends the TTFT span with the error (a silence timeout is
classified as such) and observes nothing. The `stream` span ends once,
with the error `GenerateAssistant` returns, so content-filter and
no-output attempts end errored. Failed `stream` windows are observed
like successful ones. The attempt is released before the stream span
ends, so the TTFT window always falls inside its stream. The compaction
summary stream passes a nil tracer, so it records neither.

Without #29771, production emits only `provider_attempt`; #29771 passes
the tracer to `GenerateAssistant`.

**Provider attempts.** The provider HTTP transport is wrapped in
`stageSpanRoundTripper`, which emits one `provider_attempt` stage per
round trip, so retries each get a span. The span closes on response
headers; a status >= 400 marks it errored without changing what the
client sees.

`resolveModelCall` resolves the `StageModel` (wire provider, configured
`provider_type`, model, effective reasoning effort) and threads it to
the transport, so span attributes and labels match the call.
`provider_type` is read from `ai_providers.type`, so types outside the
fantasy set (copilot) keep their configured name. `chatd.New` builds the
tracer with the server clock.

**Behavior change to `coderd_chatd_ttft_seconds`:** only an output part
counts as a first token. Error parts no longer close the window as a
success, and warnings and finish parts are skipped; the help text says
so.

`StageSpan.End` returns the span's elapsed window, which `ttft_seconds`
observes. Callers pass `Stages` and `StageModel` through
`GenerateAssistantOptions`; a nil tracer discards (TTFT still records
`ttft_seconds`). `Server` gains the `stages` tracer and a
`TracerProvider` option, wired from `coderd`.

<details>
<summary>Review decisions</summary>

- Start markers close the TTFT window: they are the first sign the model
is producing output, and waiting for a delta would fold hidden reasoning
time into TTFT.
- Failed `stream` and `provider_attempt` windows are observed: that time
was spent. TTFT is the exception because a first token that never
arrived has no duration.
- `http_host` was dropped from `provider_attempt`; every request goes
through the local AI Gateway transport, so it was constant.
- The advisor tool's nested model call is intentionally not instrumented
as `stream`/TTFT.

</details>

Supersedes #28970 and #28971.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 30, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants