Repository navigation
feat: open a chat_turn span per prompt - #29771
jscottmiller merged 25 commits into
Conversation
6f34bf7 to
cf518ba
Compare
cf518ba to
8bb2f71
Compare
8bb2f71 to
c2202f0
Compare
|
/coder-agents-review |
|
Chat: Review posted | View chat Review history
deep-review v0.13.0 | Round 7 | Last posted: Round 7, 86 findings (8 P2, 30 P3, 34 P4, 14 Nit), COMMENT. Review Finding inventoryFinding inventory: PR #29771Findings
R2 severity decisions:
CRF-7 drop gate. Keep argument: a red fmt job blocks merge and the author has to act on it. Drop reason: it has no code location, Netero's formatter run over every changed file and R3 severity decisions:
R4 severity decisions:
R5 severity decisions:
R7 severity decisions:
Law analysis
Contested and acknowledgedCRF-30 (Nit, generation.go:461) - four doc comments restate the code
CRF-32 (P4, turn_trace.go:265) - End disables the instance permanently; err always nil
CRF-53 (P3, turn_trace.go:256) - interrupted turns end with error status
CRF-58 (P4, chatloop.go:1382) - rejected tool calls record an OK tool_call stage
CRF-59 (P4, turn_trace.go:124) - stale-anchor check mixes clocks
CRF-76 (P3, turn_trace.go:291) - non-completed turns produce no metric
CRF-86 (P4, tasks.go:349) - ambiguous-commit settle in StartInterrupt has no test
CRF-89 (P4, chatloop/chatloop.go:1358) - startRun callback instead of splitting resolution from the run
CRF-90 (P4, generation.go:831) - thinking stages recorded in chatd with an unreachable guard
CRF-92 (P4, turn_trace.go:200) - unset status for interrupted turns depends on the caller passing nil
Round logRound 1Netero + Law (infrastructure). Netero: 1 P2, 4 P3, 1 Nit, 1 Note (Note dropped as CI status, carried in body). Law: mandatory vertical split (capacity wait into its own PR landing first). Panel skipped per Law mandatory-split gate; event COMMENT because the CLI rejects REQUEST_CHANGES without a panel. Reviewed against aacc28a..c2202f0. Round 2Churn guard PROCEED: CRF-1 to CRF-6 claimed fixed (CRF-1, 3, 4, 5 verified by Netero and Bisky; CRF-2 moved to #29805; CRF-6 retitled). Law split carried out (#29805 is the new base). Tool v0.12.0. Netero: 1 P2, 4 P3, 1 P4 (no P0, panel proceeds). Round type: first panel round for this PR and a restructure (split, rebase, new test file content), so full panel. Required specialists: ging-go (.go), Killua (generation loop, span allocation), Luffy (new feature), Melody (paired Complete/Settle and stage enumerations), Pen Botter (ARCHITECTURE.md), Ryosuke (Go), Takumi (mutex, task cancellation). Core: Pariston, Mafuuu. Bisky (tests changed), Gon and Leorio (once-per-PR floor, never appeared). 12 exceeds the 6-8 focused target because of the specialists above plus the floors. Wildcard: Zoro (random from distilled set). New posted this round (Netero plus panel): 6 P2, 13 P3, 10 P4, 7 Nit; convergence on the clamped acquisition (8 reviewers) and the token race (4). Killua: no findings. 3 praise/verification Notes dropped, 1 PR-text P4 carried in body, 1 OOS. Reviewed against 5be82a7..5ffb40e. Round 3Tool v0.13.0. Churn guard PROCEED: CRF-8 to CRF-44 (posted, open; CRF-43 was not posted) classified as 34 author-fixed claims and 2 contested (CRF-30, CRF-32); the classifications are author claims, not verified fixes. Branch rebased onto #29770 (adeadee); #29805 closed unmerged and capacity wait dropped (b02f956). Inventory validation of non-posted items against 61546de: CRF-7 disposition holds (context.md lists fmt as failed, but the job concluded cancelled, and gofumpt, gci, and markdown-table-formatter report no changes on the 24 changed .go files and docs/admin/integrations/prometheus.md); CRF-43 is moot (description now names #29770); CRF-45 is moot (recordCapacityWait no longer exists; capacity.go is not in the diff); CRF-46 to CRF-48 asked for no change and none of their assumptions changed a disposition. No reopenings. Round 4Tool v0.13.0 (unchanged). Branch rebased onto e6f36ac (new head of #29770, Round 5Tool v0.13.0 (unchanged). Branch rebased onto ddc808b (new head of #29770); Round 6BLOCKED. No review. Tool v0.13.0 (unchanged). Base dc303a5, head 562baea. Branch rebased (R5 head 676c4fd is now 4e3eb96) plus five commits: cf34118 (emit a turn after the tasks that ran it return; Round 7Tool v0.13.0 (unchanged). Head 562baea, base dc303a5 (same as R6; no commits since). The author replied on all five R6-silent threads and edited the PR description. Churn guard PROCEED: CRF-88 addressed (description Failure bullet now says the step records no outcome and leaves the turn open), CRF-86 acknowledged (coverage gap, no ticket), CRF-89, CRF-90, CRF-92 contested. Inventory validation of non-posted items against 562baea (last validated at 676c4fd; R6 was blocked): CRF-83 holds (chatloop.go hunks at 1185 and 1323; About deep-reviewCRF = Coder Review Finding (P0-P4, Nit, Note)
|
There was a problem hiding this comment.
First-pass review: Netero's mechanical pass plus Law's decomposition analysis. The full persona panel has not reviewed this PR yet. It will review once the split below is done.
What this does well: production code reads time only from the injected clock (w.opts.Clock.Now(), stages.Now()); the new history_version column on the candidate query is matched by the generated scan; the new spans at prepare, commit, compaction, retry backoff, and generation step end with the step's real error; and turn tokens make Complete, Invalidate, and Settle no-ops for a task whose turn was already replaced. Test density is 55%. Netero reports that build, vet, and DB=ci go test ./coderd/x/chatd/ pass at 84.4% statement coverage. Netero raised 1 P2, 4 P3, 1 Nit, and 1 Note.
Requested change: split this PR. Law returned a mandatory vertical split. Capacity wait is a second feature in a different subsystem. It changes the worker acquisition loop every chat passes through (acquireOnce, acquireCandidate), adds history_version to the acquisition candidate query, keeps per-worker refusal state that is pruned only when a candidate batch is short, and can issue GetOrganizationByID on the acquisition goroutine on a cache miss. Its correctness argument (history-version matching, prune only on a short batch, the multi-replica lower bound) is independent of the turn lifecycle's (stale-task tokens, Complete/Settle ordering, Invalidate after Complete). Reviewed together, the acquisition change reads as a footnote to roughly 1,500 lines of tracing.
Proposed cut:
- Slice A, lands first:
feat(coderd): record capacity_wait stage on acquisition. It containscapacity.go, theworker.gohooks, thechats.sqlcolumn and generated code,capacity_internal_test.go, the ARCHITECTURE "Capacity wait" section, andchatKindAttrplusServer.organizationName, since slice A is their first consumer. Roughly 140 production, 165 test, and 17 generated lines. The scope iscoderdrather than Law'scoderd/x/chatdbecause the slice touchescoderd/database(see CRF-6). - Slice B: this PR rebased on slice A, keeping the turn span lifecycle, the generation-path stages, background scoping of
inflightContext, and the API-drivenqueue_wait.
Do not split slice B further. scopeFromContext defaults to ScopeBackground, so generation stages landed without the turn context would publish scope=background for turn work. inflightContext must also detach no later than the turn context starts carrying scope=turn.
The seam checks out. recordCapacityWait uses only RecordAs, ContextWithChatKind, and ContextWithOrganization, which exist at the base, plus the two helpers. capacity_internal_test.go references neither runnerTurnSpan nor Ensure, and its newStageTestTracer helper exists at the base.
This needs a human answer: this PR supersedes #28972 ("open a chat_turn span per prompt") and #28973 ("record capacity wait as a lifecycle stage", stacked on #28972). That was the split Law asks for, in the opposite order. If a reviewer asked for the two to be combined, say so on this PR. That is a human decision and it overrides this verdict. Otherwise, split.
CI: title fails because of CRF-6. fmt also fails, but the cause is not in the diff as far as we can tell. Netero ran the Go formatters and make fmt/markdown on every changed file with no diff, go mod tidy -diff exits clean at head, and #29772, whose base is this PR's head, does not list fmt among its failures. Read the job log or re-run it. Storybook fails here and on #29772, and neither PR changes files under site/.
As Law put it: "Pattern caught: two features disguised as one."
.github/workflows/contrib.yaml:203-218
Nit [CRF-6] The title scope coderd/x/chatd does not contain every changed file, so the title CI job fails. (Netero)
.github/workflows/contrib.yaml:203-218fails when any changed file is outside the scope. The PR changescoderd/database/queries.sql.goandcoderd/database/queries/chats.sql. AGENTS.md requires the scope to be "a real filesystem path containing every changed file".
Netero's fix: retitle to feat(coderd): open a chat_turn span per prompt. After the split, only slice A touches coderd/database, so slice B could keep coderd/x/chatd.
🤖
🤖 This review was automatically generated with Coder Agents.
c2202f0 to
7a5cd22
Compare
7a5cd22 to
9185854
Compare
9185854 to
5ffb40e
Compare
|
/coder-agents-review |
There was a problem hiding this comment.
This PR opens a chat_turn root span per prompt in the chat runner and parents the generation stages under it. This round has 6 P2, 13 P3, 10 P4, and 7 Nit findings.
The round 1 findings are fixed (recording-tracer test, runnerTurnSpan.Context removed, mcp_connect outcome, canceled retry_backoff test, title), and capacity wait moved to #29805, which is now this PR's base.
Four findings share one cause: a turn is identified by a clamped timestamp held in one runner's memory, and its outcome by which helper ran. Keying the turn on the trigger message ID or history version, and deriving the outcome from the status the step committed, would remove the conditions behind the clamped acquisition, the token handoff to a new prompt, the takeover acquisition, and the fail-closed outcome (inferred from the code, not implemented).
The PR description's "activating the PR 2 stages" should name the PR number.
Out of scope (needs a ticket or explicit acceptance by a human):
coderd/x/chatd/capacity.go:88:recordCapacityWait(#29805) builds the same chat kind plus organization context pair by hand, so a shared helper added in this PR would leave that copy separate.
coderd/x/chatd/generation.go:577
P3 [CRF-19] A turn that enters requires_action is neither completed nor invalidated, so chat_turn includes the time a client takes to submit dynamic tool results, up to requiresActionTimeout (5 minutes). (Pen Botter, Bisky, Mafuuu)
Settle does nothing because the turn is not finished, and the task spawned after CompleteRequiresAction or CancelRequiresAction gets the same token from Ensure. Neither ARCHITECTURE.md's Lifecycle tracing section nor any test states this, so a reader of the chat_turn histogram cannot separate server latency from client wait. Either record the wait as its own stage or state in ARCHITECTURE.md that chat_turn includes it, and add a TestActiveServer_TracesChatTurn case that enters requires_action and asserts the turn count and the turn's end time.
🤖
coderd/x/chatd/generation.go:1432
P3 [CRF-20] The fail-closed hook path commits FinishError without calling Invalidate, so the turn closes at runner teardown as turn_outcome=abandoned with OK status. (Ryosuke)
commitGenerationStep returns the nil result of afterGenerationOutcome to StartGeneration, which neither invalidates (no error) nor settles (not finished). The same PostCommitError without a commit goes through finishGenerationError and records error, so a hook failure is reported two ways. Call input.Turn.Invalidate(input.TurnToken, chatloop.TurnOutcomeError, commitHooks.PostCommitError) after the commit succeeds, or classify the outcome from the committed chat status in StartGeneration.
🤖
🤖 This review was automatically generated with Coder Agents.
5ffb40e to
61546de
Compare
The acquisition loop remembers when each chat was first refused a concurrent-agent capacity slot, with the chat's history version at that refusal, and records a capacity_wait stage from that refusal when it later acquires the chat at the same version. Waits are dropped on non-capacity skips and pruned when the candidate batch is complete. The acquisition candidates query selects history_version. Adds Server.organizationName, a cached organization name lookup for stage span attributes, and chatKindAttr.
Remove the per-worker capacity wait bookkeeping, the acquisition candidate columns it read, and its stage, docs, and tests. Keep the organization name cache and chatKind, and add withStageIdentity, which puts a stage's scope, chat kind, and organization on a context.
finishGenerationError classified the turn outcome from the task context after the FinishError commit. That commit publishes the status change that cancels the task, so a provider error could be recorded as interrupted. Classify before the commit and invalidate with the precomputed outcome. Generated by Coder Agents.
Drop tests that repeat other coverage, merge overlapping outcome subtests, and share span and anomaly helpers with the external test package. Pin turn anchors with a mock clock and fix the unsettled turn test so it no longer passes through trigger rotation. Cover the gaps the review found: the runner handing the takeover flag to its turn span, runner shutdown abandoning an open turn, fail-closed post_tool_use failures, the stop hook finish path and its queue_wait, mcp_connect failure wiring, a provider retry staying in one turn, the interrupted outcome, and outcome classification when the finishing commit cancels the task. Generated by Coder Agents.
- Set a stage's scope, chat kind, and organization through withStageIdentity in runnerTurnSpan, inflightChatContext, and recordQueueWait; startLocked reuses contextLocked. - recordQueueWait reads the tracer clock, so promoting callers pass only the committed chat and the queued time. - Drop hand-set provider attributes that StageModel now carries. - Describe the turn scope as latency attributable to a prompt.
Only the interrupt task records interrupted: it marks the open turn before FinishInterruption commits and settles it afterwards, so a stop between steps is also counted. A canceled generation task records no outcome, so shutdown, loss of ownership, and message edits close the turn as abandoned with unset status, and a canceled task's late Ensure gets the zero token instead of its replacement's. A failed step abandons its turn only for an expected, non-retryable exit on a live context; every error the task runner retries continues the turn. chat_turn is observed on the stage histogram only for completed turns. PromoteQueuedMessageResult reports PromotedQueuedAt like the other promoting transitions.
…calls, and turn ordering limits
runnerTurnSpan keeps every turn that has not been emitted, keyed by turn token. Closing a turn fixes its end time; its span is emitted once every task that joined it has called Release, which runTask does after the task returns. An outcome a replaced task records while unwinding, such as Complete after a FinishTurn that promoted a queued prompt, now reaches its own turn instead of the turn closing as abandoned. Generated by Coder Agents.
Generated by Coder Agents.
Generated by Coder Agents.
Generated by Coder Agents.
1be3031 to
93913d8
Compare
…t token (#29770) **TL;DR:** Traces the model provider call: a `stream` stage, a nested `time_to_first_token` stage, and one `provider_attempt` per HTTP round trip, labelled with provider type and model. Behavior change: `coderd_chatd_ttft_seconds` now counts only a real output part as the first token, so error, warning, and finish parts no longer count. Instruments the provider call. **Stream and time to first token.** `GenerateAssistant` records a `stream` stage from stream open to close and a nested `time_to_first_token` stage. The TTFT window closes once, on the first streamed output part: any part other than warnings, finish, or error, including start markers such as `text_start`. A closed window observes the stage and the pre-existing `coderd_chatd_ttft_seconds` with the same elapsed time. A failed open, an error part, or a release without an output part ends the TTFT span with the error (a silence timeout is classified as such) and observes nothing. The `stream` span ends once, with the error `GenerateAssistant` returns, so content-filter and no-output attempts end errored. Failed `stream` windows are observed like successful ones. The attempt is released before the stream span ends, so the TTFT window always falls inside its stream. The compaction summary stream passes a nil tracer, so it records neither. Without #29771, production emits only `provider_attempt`; #29771 passes the tracer to `GenerateAssistant`. **Provider attempts.** The provider HTTP transport is wrapped in `stageSpanRoundTripper`, which emits one `provider_attempt` stage per round trip, so retries each get a span. The span closes on response headers; a status >= 400 marks it errored without changing what the client sees. `resolveModelCall` resolves the `StageModel` (wire provider, configured `provider_type`, model, effective reasoning effort) and threads it to the transport, so span attributes and labels match the call. `provider_type` is read from `ai_providers.type`, so types outside the fantasy set (copilot) keep their configured name. `chatd.New` builds the tracer with the server clock. **Behavior change to `coderd_chatd_ttft_seconds`:** only an output part counts as a first token. Error parts no longer close the window as a success, and warnings and finish parts are skipped; the help text says so. `StageSpan.End` returns the span's elapsed window, which `ttft_seconds` observes. Callers pass `Stages` and `StageModel` through `GenerateAssistantOptions`; a nil tracer discards (TTFT still records `ttft_seconds`). `Server` gains the `stages` tracer and a `TracerProvider` option, wired from `coderd`. <details> <summary>Review decisions</summary> - Start markers close the TTFT window: they are the first sign the model is producing output, and waiting for a delta would fold hidden reasoning time into TTFT. - Failed `stream` and `provider_attempt` windows are observed: that time was spent. TTFT is the exception because a first token that never arrived has no duration. - `http_host` was dropped from `provider_attempt`; every request goes through the local AI Gateway transport, so it was constant. - The advisor tool's nested model call is intentionally not instrumented as `stream`/TTFT. </details> Supersedes #28970 and #28971. 🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.
TL;DR: Wraps each prompt in a
chat_turnroot span with every lifecycle stage parented under it. The span records one outcome per turn (completed,interrupted,error,abandoned), and the newcoderd_chatd_turn_outcomes_totalcounter counts those outcomes. Turns stay correct when tasks are canceled, replaced, or retried.Each prompt runs inside a
chat_turnroot span owned by the runner (runnerTurnSpan). The generation step, prepare, MCP connect, stream, TTFT, provider attempt, commit, thinking, tool call, compaction, and retry backoff stages are parented under it, activating the stages from #29770.chat_turnspans are standalone roots: the turn runs on a worker, often on another replica, and no trace context is persisted.Lifecycle
Ensure, which opens the turn if none is open and returns aturnToken. The span is backdated to the trigger: the latest of the last user prompt, a dynamic tool's result message after it, and a pending compaction request. Anacquisitionstage covers trigger to pickup.acquisitionand countsstale_anchor. A runner that took the chat over from an owner with a stale heartbeat (TakenOver, from acquisition) opens its first turn at now with noacquisition, so the dead owner's run is not reported as pickup delay.Ensureon the current turn with a newer trigger closes it and opens a new one. The closed turn isabandonedunless the task that ran it records an outcome before releasing it.FinishTurnorEnterRequiresActioncommits, the step callsComplete;Settlecloses the span after the step's own stage ends, so the finishing step is inside the turn. A chat waiting inrequires_actionis outside any turn; the generation resumed by its tool results opens a new one.Ensuretakes the task ID) has calledRelease, whichrunTaskdoes after the task returns. A finishing task whoseFinishTurnpromoted a queued prompt therefore still counts its turn ascompletedeven when the promoted task'sEnsureruns before itsComplete.Endat runner exit emits whatever is left.FinishError, or fails closed, invalidates the turn aserrorafter the commit. Errors the task runner retries, including task timeouts, keep the same turn open; an expected, non-retryable exit records no outcome and leaves the turn open until the interrupt task, a newer prompt, or runner exit closes it. A canceled generation task records no outcome and cannot obtain a turn token, so a replaced task never touches its successor's turn.interruptedbeforeFinishInterruptioncommits and closes it after. Only a stop (by the user or a parent agent) isinterrupted; it ends with unset span status, since error status is reserved forerrorturns. runner shutdown, loss of ownership, and message edits close the turn asabandonedwith unset status.FinishTurn,FinishInterruption, a send to an errored chat,PromoteQueued) returns the promoted message's queued time in its result, and onerecordQueueWaithelper records a standalonequeue_waitto the promotion. The promoted turn opens at the promotion time.chat_turnis observed oncoderd_chatd_stage_duration_secondsonly for completed turns, so the histogram is completed-turn latency; every turn still emits its span.coderd_chatd_turn_outcomes_total{outcome, chat_kind}counts every closed turn exactly once (completed,interrupted,error,abandoned), matching the span'sturn_outcome, under thechat-stage-metricsexperiment. Turn time accounting is in draft #29772.Stages and identity
tool_callis a live stage around each local tool call inchatloop; the tool runs on its context, and it ends errored only when the tool fails to execute. Calls that never run (inactive, unknown, or rejected by policy) record no stage.mcp_connectcarries connected and failed server counts and ends errored only when servers were configured and none connected.withStageIdentityputs scope and chat kind on the turn context. Detached background work runs inbackgroundscope as its own trace roots.advisortool's nested model call appears only as its enclosingtool_call.ARCHITECTURE.md gains a "Lifecycle tracing" section.
Review decisions
interrupted, and cancellation or a fence mismatch alone never records an outcome, so shutdown, ownership loss, and edits areabandoned. Onlyerrorturns set error span status. Remaining ordering races between a step and an interrupt are documented as known limitations in ARCHITECTURE.md.ChatMachine.Update, which would fix only theFinishTurnordering.requires_actionends the turn rather than pausing it, sochat_turnmeasures chatd time; a prompt with client tools counts as several turns.queue_waitis always standalone, sochat_turnnever includes time spent queued behind another turn.capacity_waitstage (feat(coderd): record a capacity_wait stage on chat acquisition #29805) was dropped: it was a per-worker lower bound, excluded from accounting, and needed its own SQL and bookkeeping. feat(coderd): record a capacity_wait stage on chat acquisition #29805's remaining helpers are folded into this PR.Supersedes #28972, #28973, and #29805.
🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.