Repository navigation
feat: add the chat-stage-metrics-full experiment - #29775
Closed
jscottmiller wants to merge 5 commits into
Closed
jscottmiller wants to merge 5 commits into
jscottmiller wants to merge 5 commits into
Conversation
added 4 commits
September 23, 2026 02:38
Add a StageTracer that emits one span and one stage_duration_seconds observation per chat lifecycle stage, with typed Stage, Scope, and ChatKind values, a model_stage_duration_seconds histogram for the provider-bound stages labeled by provider_type and model, and a stage_anomalies_total counter for dropped observations. The metric families are registered when the chat-stage-metrics experiment is enabled; spans are emitted regardless. Extract the generation step body into its own function so the step has a boundary for a deferred stage end.
Contributor
Docs previewCheck off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here. |
Behind the chat-stage-metrics-full experiment, observe every lifecycle stage on the stage histograms with a 16-edge ladder, add thinking and compaction to the model histogram, and register the per-turn distribution families turn_stage_seconds, turn_time_seconds, and turn_time_share. Without it the families and stage set are unchanged.
jscottmiller
force-pushed
the
scott/chatd-stage-metrics-full
branch
from
September 23, 2026 03:09
3fc5822 to
65fd415
Compare
jscottmiller
force-pushed
the
scott/chatd-lifecycle/4-turn-accounting
branch
14 times, most recently
from
September 29, 2026 03:26
ff9b980 to
dd49b8e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the
chat-stage-metrics-fullexperiment (not inExperimentsSafe), which widens the chat lifecycle stage metrics from #29769 through #29772 at a higher series count. Without it nothing changes.With both
chat-stage-metricsandchat-stage-metrics-fullenabled:coderd_chatd_stage_duration_seconds, includinggeneration_step,prepare,thinking, andcompaction, which are otherwise span-only.0.05 20 120 600), so a query written against either reads the same edges.thinkingandcompactionjoincoderd_chatd_model_stage_duration_seconds.turn_time_seconds{category, chat_kind},turn_time_share{category, chat_kind}turn_stage_seconds{stage, chat_kind}Per-turn histograms use an 11-edge ladder from 1s to 1h;
turn_time_shareuses twentieths. Cost per replica perchat_kindis roughly 900 series (15 stages x 19, plus the three distributions over 13 stages and 11 categories) against ~180 without it, and 95 per(chat_kind, model)against 45.fullwithout the base experiment registers nothing.chatloop.NewMetricskeeps registering the base set only; tests that need the full set useMetricsOptions{StageMetrics: true, FullStageMetrics: true}.Dashboard rows for these families are not included; they belong with the dashboard in #29735 once it lands.
🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.