Skip to content

feat(coderd): record a capacity_wait stage on chat acquisition - #29805

Closed
jscottmiller wants to merge 5 commits into
scott/chatd-lifecycle/2-provider-stagesfrom
scott/chatd-lifecycle/3-capacity-wait
Closed

jscottmiller wants to merge 5 commits into
scott/chatd-lifecycle/2-provider-stagesfrom
scott/chatd-lifecycle/3-capacity-wait

Conversation

@jscottmiller

@jscottmiller jscottmiller commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

The acquisition loop remembers when each chat was first refused a concurrent-agent capacity slot, together with the chat's history version at that refusal (candidates skipped because their pool already refused one this pass are remembered too). When the same worker later acquires the chat at the same history version, it records a capacity_wait stage from that first refusal to the acquisition, as a span and as an observation on coderd_chatd_stage_duration_seconds. GetChatWorkerAcquisitionCandidates now selects history_version for this.

Chats admitted on their first attempt record nothing. A remembered refusal for an earlier history version is discarded, so a chat refused here, acquired elsewhere, and later re-prompted while every batch stayed full is not charged the old wait. The wait start is dropped when a candidate is skipped for a non-capacity reason (owned by a live runner, archived, not runnable, or gone) and pruned when the candidate batch is shorter than its limit, since a chat missing from a truncated batch may still be waiting. The map is touched only by the acquisition goroutine.

The refusal history is per worker and in memory while the limit is deployment-wide, so the value is a lower bound: a chat refused on one replica and acquired by another is measured from the acquiring replica's first refusal, and a restart discards history. All replicas wake together on the chat:ownership hint, so the undercount is bounded by about one 30s acquisition interval rather than growing with replica count.

Also adds Server.organizationName, a cached organization ID to name lookup used as a span attribute (an organization label is not added to any metric), and chatKindAttr. Both are used by the turn span in the next PR.

ARCHITECTURE.md gains a "Capacity wait" section.

🤖 This PR was generated by Coder Agents on behalf of @jscottmiller.

@jscottmiller

Copy link
Copy Markdown
Contributor Author

/coder-agents-review

@coder-agents-review

coder-agents-review Bot commented Sep 24, 2026 •

Copy link
Copy Markdown
Contributor

Chat: Review posted | View chat
Requested: 2026-09-24 15:09 UTC by @jscottmiller

Review history
  • R1 (2026-09-24): 13 reviewers, 3 Nit, 4 P2, 4 P3, 7 P4, COMMENT. Review

deep-review v0.13.0 | Round 1 | 3809e82..5be82a7

Last posted: Round 1, 18 findings (4 P2, 4 P3, 7 P4, 3 Nit), COMMENT. Review

Finding inventory

Finding inventory: PR #29805

Findings

# Sev Status Location Summary Round Reviewer Posted
CRF-1 P2 Open worker.go:335 No test drives refusal then acquisition through the worker to assert a capacity_wait stage R1 Netero Yes
CRF-2 P2 Open worker.go:230 Wait start only recorded once chat enters the 2x batch candidate window; undercount grows with queue depth, contradicting the 30s bound claim R1 Pariston P2, Luffy P2, Leorio P3 (merged CRF-11) Yes
CRF-3 P2 Open capacity.go:78 Next PR's DB-anchored acquisition stage covers the same window; per-worker map design is a less accurate subset R1 Luffy P2, Pariston (in P2 fix) Yes
CRF-4 P2 Open capacity.go:83 Chat interrupted while waiting records capacity_wait although never admitted (Admit bypasses non-running) R1 Kite P2, Takumi P3, Pariston P3, Mafuuu P3 Yes
CRF-5 P3 Open capacity.go:84 history_version fence misses another replica's acquire/abandon, so wait can include that replica's ownership R1 Knuckle P3, Ryosuke P3, Mafuuu P4, Kite P4, Pariston Note Yes
CRF-6 P4 Open worker.go:216 capacityWaitSince entries for chats acquired elsewhere persist while batches stay full; bound undocumented R1 Killua P4, Kite Note Yes
CRF-7 P4 Open worker.go:335 recordCapacityWait runs before Spawn; failed spawn yields two observations for one wait R1 Kite Yes
CRF-8 P3 Open capacity.go:87 No test asserts the organization attribute on the capacity_wait span R1 Bisky Yes
CRF-9 P4 Open capacity_internal_test.go:125 Prune test depends on ChatMachine nil-publisher validation order R1 Bisky Yes
CRF-10 P3 Open chatloop/metrics.go:173 stage_duration_seconds help text does not say when capacity_wait is observed or what it measures R1 Leorio Yes
CRF-11 P3 Dropped by orchestrator (merged into CRF-2; proposed 30s bound text contradicted by reproduced window undercount) ARCHITECTURE.md:668 Lower-bound paragraph omits the magnitude of the undercount R1 Leorio No
CRF-12 P4 Open capacity_internal_test.go:88 "every batch was full" conflates truncated batch with full pool R1 Leorio Yes
CRF-13 P3 Open worker.go:33 Field comment omits the acquisition-goroutine ownership rule R1 Gon P2, Leorio P4 Yes
CRF-14 Nit Open chatd.go:4889 Debug log message does not say the lookup failed R1 Leorio Yes
CRF-15 P4 Open ARCHITECTURE.md:666 Skip-reason list omits deleted chats R1 Gon P4, Leorio Nit Yes
CRF-16 P4 Open chatd.go:4877 PR description says helpers are used by the next PR; they are already used here R1 Leorio (Note) Yes
CRF-17 P4 Open chatd.go:177 Untyped sync.Map forces an unreachable type-assert fallthrough; use syncmap.Map R1 Gon P4, Ging-Go Nit Yes
CRF-18 Nit Open worker.go:34 capacityWaitSince name implies time values R1 Gon Yes
CRF-19 Nit Open chatd.go:4898 chatKindAttr returns a ChatKind, not an attribute R1 Gon Yes
CRF-20 P2 Dropped by orchestrator (comment explains why historyVersion is kept; describes behavior per repo comment rule) capacity.go:52 capacityWait doc restates field names R1 Gon No
CRF-21 P2 Dropped by orchestrator (history_version is monotonic per dump.sql:2219, so "newer" is accurate; doc describes behavior) capacity.go:61 noteCapacityRefused doc narrates body, "newer" claimed wrong R1 Gon No
CRF-22 P2 Dropped by orchestrator (doc states when nothing is recorded and why; caller-facing behavior) capacity.go:71 recordCapacityWait doc narrates early returns R1 Gon No
CRF-23 P2 Dropped by orchestrator (second sentence states the consequence; Gon's replacement is that sentence) capacity.go:96 forgetCapacityWait doc restates delete R1 Gon No
CRF-24 P2 Dropped by orchestrator (doc states cache lifetime and failure retry contract a caller cannot see from the signature) chatd.go:4872 organizationName doc restates branches R1 Gon No
CRF-25 P2 Dropped by orchestrator (one-line doc; "top-level" describes root, no behavior misstatement) chatd.go:4897 chatKindAttr doc restates body R1 Gon No
CRF-26 P2 Dropped by orchestrator (comment states why an unattempted candidate is noted; Gon's replacement carries the same content) worker.go:228 Comment restates the if condition R1 Gon No
CRF-27 Nit Dropped by orchestrator (sibling chatd.go:527 uses "does not own" in the RBAC-owner sense; wording follows file convention) chatd.go:4886 nolint justification says chatd does not own the chat R1 Kurapika No

Contested and acknowledged

Law analysis

Not run. Effective additions 386 (threshold 1000).

Round log

Round 1

Full panel (round 1). Netero first pass: 1 P2 (CRF-1), no P0, panel proceeded. Specialists by trigger: Ging-Go (.go), Knuckle (query change), Killua (acquisition loop, cache), Takumi (lock-free map, sync.Map), Kurapika (dbauthz.AsChatd read), Ryosuke (Go package layering), Luffy (new stage feature). Core: Pariston, Mafuuu. Pool: Bisky (tests), Gon and Leorio (once-per-PR floor). 12 panel reviewers at the multi-domain target, plus wildcard Kite (random draw from distilled set). Kurapika: no findings.

Cross-check: CRF-2 convergent (Pariston reproduced 20 of 25 refused chats remembered after one pass); Leorio's proposed 30s-bound doc text contradicts it and was merged (CRF-11). CRF-4 convergent across four reviewers, two reproductions (Takumi, Mafuuu); Kite's P2 wins the tiebreak (tail inflation on the metric used to size the limit). CRF-5 convergent across five reviewers at P3 to Note; P3 wins. CRF-3 structural alternative verified against stacked commit 5ffb40e ARCHITECTURE.md (acquisition anchored at prompt created_at). CRF-6: Killua's proposed expiry assumes a waiting chat stays in the window; interrupting, requires-action, and other-pool rows can displace it (chats.sql:2502-2507), noted in the comment. Gon rated eight comment-restatement items P2; seven dropped (CRF-20 to CRF-26) because the comments describe behavior and meet the repo comment rule, and the eighth (field ownership rule) merged with Leorio at P3 (CRF-13). Ging-Go Nit and Gon P4 on the untyped sync.Map merged at P4 (CRF-17). Kurapika's nolint wording Nit dropped (CRF-27). Leorio's PR-description Note raised to P4 (CRF-16) because it asks for a change. No out-of-scope items. Reviewed against 3809e82..5be82a7.

About deep-review

CRF = Coder Review Finding (P0-P4, Nit, Note)

Reviewer Focus
Bisky tests
Chopper ops/errors
Churn-guard change verification
Ging language modernization
Gon naming
Hisoka edge cases
Killua perf
Kite change integrity
Knov contracts
Knuckle SQL
Komugi flake/determinism
Kurapika security
Law decomposition
Leorio docs
Luffy product
Mafu-san process
Mafuuu contracts
Melody dispatch/pairing
Meruem structural
Nami frontend
Netero mechanical checks
Pariston premise testing
Pen-botter product gaps
Razor verification
Robin duplication
Ryosuke Go arch
Takumi concurrency
Zoro shape

🤖 Managed by Coder Agents.

@coder-agents-review coder-agents-review Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This PR records a capacity_wait stage from a worker's first capacity refusal of a chat to its acquisition, and adds an organization name cache and a chat kind helper. Findings: 4 P2, 4 P3, 7 P4, 3 Nit.


coderd/x/chatd/chatloop/metrics.go:173

P3 [CRF-10] The stage_duration_seconds help text lists capacity_wait as observed but does not say when it is observed or what it measures, and this PR adds its only producer. (Leorio)

Operators read the generated docs/admin/integrations/prometheus.md:244, not ARCHITECTURE.md, and will assume one observation per turn covering the whole wait. The stage is observed only for chats the acquiring replica refused at least once and starts at that replica's first refusal, so dividing its count by the chat_turn count does not give the fraction of turns that waited. Add that to the help string and run make gen.

🤖

🤖 This review was automatically generated with Coder Agents.

Comment thread coderd/x/chatd/worker.go Resolved
Comment thread coderd/x/chatd/worker.go Resolved
Comment thread coderd/x/chatd/capacity.go Resolved
Comment thread coderd/x/chatd/capacity.go Resolved
Comment thread coderd/x/chatd/capacity.go Resolved
Comment thread coderd/x/chatd/ARCHITECTURE.md Resolved
Comment thread coderd/x/chatd/chatd.go Resolved
Comment thread coderd/x/chatd/chatd.go Resolved
Comment thread coderd/x/chatd/worker.go Resolved
Comment thread coderd/x/chatd/chatd.go Resolved
Scott Miller added 5 commits September 25, 2026 04:42
The acquisition loop remembers when each chat was first refused a
concurrent-agent capacity slot, with the chat's history version at
that refusal, and records a capacity_wait stage from that refusal
when it later acquires the chat at the same version. Waits are
dropped on non-capacity skips and pruned when the candidate batch is
complete. The acquisition candidates query selects history_version.

Adds Server.organizationName, a cached organization name lookup for
stage span attributes, and chatKindAttr.
Remove the per-worker capacity wait bookkeeping, the acquisition
candidate columns it read, and its stage, docs, and tests. Keep the
organization name cache and chatKind, and add withStageIdentity, which
puts a stage's scope, chat kind, and organization on a context.
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/3-capacity-wait branch from 5be82a7 to b02f956 Compare September 25, 2026 05:04
@jscottmiller
jscottmiller force-pushed the scott/chatd-lifecycle/2-provider-stages branch from 3809e82 to adeadee Compare September 25, 2026 05:04
@jscottmiller
jscottmiller removed this pull request from stack #29806 September 25, 2026 05:05
@jscottmiller

Copy link
Copy Markdown
Contributor Author

Folded into #29771: capacity_wait was dropped in the architecture cleanup, and the remaining helpers (organization name cache, chatKind, withStageIdentity) now land as part of #29771.

Generated by Coder Agents on behalf of @jscottmiller.

@github-actions github-actions Bot locked and limited conversation to collaborators Sep 25, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant