Skip to content

feat: account chatd turn time by stage and category - #28974

Closed
jscottmiller wants to merge 1 commit into
scott/chatd-stages/6-capacity-waitfrom
scott/chatd-stages/7-turn-accounting
Closed

jscottmiller wants to merge 1 commit into
scott/chatd-stages/6-capacity-waitfrom
scott/chatd-stages/7-turn-accounting

Conversation

@jscottmiller

@jscottmiller jscottmiller commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Superseded by #29772. Closed in favor of the reduced stack #29773.

When a turn finishes normally, its stages are rolled up and emitted once, labelled by chat_kind plus stage, category, or outcome where the family has one. The turn families carry no model or organization label: a turn is not a model-scoped unit (subagents, compaction summaries, and model switches can run inside one), and per-model time is on coderd_chatd_model_stage_duration_seconds. The first model a turn resolves is stamped on the chat_turn span. Interrupted, errored, and abandoned turns emit no accounting.

Family Level What
turn_outcomes_total{outcome, chat_kind} basic every closed turn exactly once: completed, interrupted, error, abandoned
turn_time_seconds_total{category, chat_kind}, turns_total{chat_kind} basic an exclusive partition of turn wall time; rate(a)/rate(b) is mean seconds per turn by category
turn_time_seconds, turn_time_share {category, chat_kind} full per-turn distribution of each category's seconds and share; every category is emitted every turn so shares sum to 1
turn_stage_seconds{stage, chat_kind} full total seconds per stage per turn; stages overlap, so this is a profile, not a partition

completed equals turns_total except for non-positive-duration turns (stage_anomalies_total{reason="nonpositive_turn"}). Keeping the partition at basic as counters costs 16 series per chat kind instead of the 418 the two histograms would. The accumulator runs at every level, so the partition is complete at basic even for stages the level leaves span-only. Per-turn histograms use an 11-edge ladder from 1s to 1h; turn_time_share uses twentieths. Occurrence counts and per-stage shares per turn are left to traces.

How the partition is built

Nested stages share wall time with their parents, so summing durations over-counts. The partition assigns each stage's own time (duration minus its attributing children) to one category: a TurnAccumulator rides on the turn context, each attributing stage reports its full duration to its parent when it ends, and the parent's category receives what is left.

chat_turn                                        own time -> category
├── acquisition | queue_wait                     full     -> scheduling
└── generation_step (repeats per step)           own      -> chatd_overhead, or
    │                                                        tool_execution when the
    │                                                        step ran local tools
    ├── prepare                                  own      -> preparation
    │   └── mcp_connect                          own      -> preparation
    ├── retry_backoff                            own      -> retry_backoff
    ├── stream                                   own      -> streaming, or
    │   │                                                    provider_error on failure
    │   ├── time_to_first_token                  own      -> time_to_first_token, or
    │   │                                                    provider_error on failure
    │   └── provider_attempt  (overlaps TTFT)    not categorized
    ├── thinking              (inside stream)    not categorized
    ├── tool_call             (inside step)      not categorized
    ├── commit                                   own      -> persistence
    └── compaction                               own      -> compaction
                                                 remainder -> unattributed

provider_attempt, thinking, tool_call, and capacity_wait appear in the per-stage totals but never claim time from a parent: each lies inside a stage that is already categorized. Only turn-scoped stages report to the accumulator, live or reconstructed, so background work never lands in a turn; EndWithoutObservation reports too, so a stream that never produced a token still counts. Categories that sum to more than the turn are emitted as measured and counted as overattributed.

Also: SetGenerationAction replaces the raw attribute call at the step site; ARCHITECTURE.md gains a "Turn accounting" section.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.

@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from 2eec66e to 948b3e7 Compare September 11, 2026 19:17
@jscottmiller jscottmiller removed the experimental Changes that might not necessarily be merged, until its approved to proceed with. label Sep 14, 2026
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch 2 times, most recently from 186d94e to 4bb7e06 Compare September 14, 2026 20:34
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from 4bb7e06 to 52c1757 Compare September 14, 2026 22:06
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from 52c1757 to 323511d Compare September 15, 2026 21:39
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from 323511d to 7cbf824 Compare September 16, 2026 03:12
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from 7cbf824 to 6957249 Compare September 17, 2026 16:55
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch 2 times, most recently from b9ee64c to d9a6dac Compare September 17, 2026 21:26
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch 2 times, most recently from ea98e23 to d49be1a Compare September 21, 2026 22:52
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from d49be1a to c3dceec Compare September 23, 2026 01:37
Each chat_turn carries a TurnAccumulator that turn-scoped stages report
to as they end. Stages that partition the turn form an attribution tree
so a step's own time is split into disjoint categories, with tool
execution separated from chatd overhead by the step's generation
action. When a turn that Complete marked finished settles, its per-stage
totals and category partition are observed once on the turn
histograms; a turn closed interrupted, in error, or abandoned drops its
accounting and is counted under that outcome, so every closed turn
has exactly one outcome. Turns whose duration is not positive or whose
categories overrun it are counted as anomalies.

The per-turn families are labelled by chat kind only; a turn is not a
model-scoped unit, and per-model time is on
coderd_chatd_model_stage_duration_seconds. The organization name and
reasoning effort stay span attributes. coderd_chatd_turn_time_seconds_total,
turns_total, and turn_outcomes_total are registered at the basic and
full stage metrics levels; the per-turn distributions
(turn_stage_seconds, turn_time_seconds, turn_time_share) are registered
only at full. The per-turn histograms use an 11-bucket ladder from 1s
to 1h, since sub-second resolution carries nothing for a value summed
over a turn.
@jscottmiller
jscottmiller force-pushed the scott/chatd-stages/7-turn-accounting branch from c3dceec to 22937d0 Compare September 23, 2026 02:15
@github-actions github-actions Bot locked and limited conversation to collaborators Sep 23, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants