Skip to content

feat: label automation messages in agent chats - #30167

Merged
ThomasK33 merged 29 commits into
tk/codagt-1211b-automations-run-nowfrom
tk/codagt-1215-automations-chat-labels
Oct 2, 2026
Merged

ThomasK33 merged 29 commits into
tk/codagt-1211b-automations-run-nowfrom
tk/codagt-1215-automations-chat-labels

Conversation

@ThomasK33

@ThomasK33 ThomasK33 commented Sep 30, 2026 •

Copy link
Copy Markdown
Member

Closes CODAGT-1215. Part of CODAGT-1135 (Automations for Coder Agents, RFC step 9).

Chat messages and queued inputs delivered by a chat automation (webhook, schedule, Run now) now show a label: Automation run · <automation name> · input <first 8 chars of input ID>. A tooltip holds the full automation and input IDs. While names load, the label omits the name part. When the name cannot be resolved (deleted automation, a shared-chat viewer who cannot read it, or experiment off), the label shows the automation ID instead. Ordinary messages have no label.

Changes

  • Backend (minimal, SDK exposure only): codersdk.ChatMessage and codersdk.ChatQueuedMessage gain automation_id and input_id (omitted when unset). db2sdk fills them from the existing columns, so REST messages and the stream message and queue_update events carry them. No behavior change in the lower layers.
  • Frontend: API.experimental.getChatAutomations, a chatAutomations query, and useChatAutomationNames. The hook fetches GET /api/experimental/organizations/{org}/chat-automations only when the user-scoped chat-automations experiment is on and the chat holds automation input. The AutomationLabel component uses the shared Badge and Tooltip, and is wired into ConversationTimeline (history) and QueuedMessagesList (queue). The chat store refetches names when new automation input arrives, so automations created or renamed after page load show their current name.
  • Out of scope, per the ticket: the occurrence label and the requested-versus-resolved effort display.

Validation

  • Vitest: the automations list is requested for the chat's organization only when the chat holds automation input and the experiment is on; useChatAutomationNames reports loading until the list resolves and then maps IDs to names; the chat store refetches names on new live automation input. Go: TestChatMessageAutomationProvenance for both converters.
  • Storybook (Pixel): ConversationTimeline/AutomationUserMessages and QueuedMessagesList/AutomationMessages (named, deleted by ID, long name at phone width, no label on ordinary items) and QueuedMessagesList/AutomationMessagesLoading (name still loading: no UUID flash).
  • Remote dogfood UAT on a develop.sh instance with the experiment on: PASS in round 2. It covered webhook input in an existing chat (history and queue), a schedule, a new_chat first message, the ID fallback after deletion, and no label on ordinary messages. Screenshots are in a comment below. Run now was not on the base branch during UAT; it uses the same fields and path.
  • coder-frontend-review self-review: all roles PASS, after fixes on each update.

Known limitations (follow-ups under CODAGT-1135)

  • A shared-chat viewer who does not own the automation sees its ID, because automation RBAC has no chat ACL (CODAGT-1239).
  • Copying the label text yields three lines, because the name span truncates separately (CODAGT-1238).

Stack

Stacked on #30085, #30096, #30112, #30117, #30127, #30134, #30160 (1211 7a) and #30166 (1211 7b). Base: tk/codagt-1211b-automations-run-now.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Chat messages and queued messages already record which automation and
input delivered them. Expose both on codersdk.ChatMessage and
codersdk.ChatQueuedMessage through the db2sdk converters so REST and
stream consumers can label automation input.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
Messages and queued inputs delivered by a chat automation now show an
"Automation run" label with the automation name and a short input ID.
Names come from the organization's chat automations list, fetched only
when the chat-automations experiment is on and the chat holds automation
input. A deleted or unreadable automation is shown by its ID.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
Live automation input now invalidates the chat automations list, so a
label from an automation created or renamed after the page loaded shows
the current name without a reload. The store invalidates on a durable
automation message and on a queue_update that adds automation input.

Long automation names now truncate on their own, so the input ID stays
visible. The tooltip shows the full name, or explains that the name is
unavailable because the automation was deleted or is not readable.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
… query

The only caller of chatAutomations() sets its own enabled condition, so
the factory's enabled option never took effect. Also tighten the
useChatAutomationNames doc comment.
…nd queue refresh

Add a ChatPageTimeline test that proves resolved automation names reach
history labels, with a deleted automation shown by ID and no label on
ordinary messages. Add a chat store case that already queued automation
input does not refetch the automation list.
Neutral tooltip copy for an unresolved automation name, shorter comments,
mock-prefixed fixtures, and an inlined single-use story constant.
…now' into tk/codagt-1215-automations-chat-labels
…now' into tk/codagt-1215-automations-chat-labels
@linear-code

linear-code Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

CODAGT-1135

CODAGT-1215

@ThomasK33

Copy link
Copy Markdown
Member Author

Dogfood UAT evidence (remote develop.sh instance, chat-automations experiment on). The verdict was PASS in round 2 on 607d638. Later commits change only tooltip copy, comments, tests, and lower-layer merges. Remote chat: https://dogfood.cdr.dev/agents/dd78c53c-da9b-4820-8c00-05c7533c5af5

Webhook input in an existing chat, with the label and tooltip:
Webhook message with automation label
Tooltip with automation name and full IDs

Queued webhook input, with the label and tooltip:
Queued automation input with label

Scheduled message:
Scheduled message with label

First message of a new_chat automation:
New chat first message with label

After the automation is deleted, the label shows its ID:
Deleted automation shown by ID

Ordinary messages and queued items have no label:
Ordinary typed message without label
Ordinary queued message without label

Run now was not on the base branch during UAT. It uses the same automation_id/input_id path.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 30, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-01T11:38:56.515547Z b52ec0c Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 112058251d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e1ca6544a7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. 👍

Reviewed commit: 198c31804d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

@ThomasK33

Copy link
Copy Markdown
Member Author

Readiness record for head 198c318.

  • CI: ci run 36682838129 (workflow_dispatch on 198c318) passed, including required. GitHub created no pull_request runs for the last two pushes because mergeability stayed UNKNOWN, a known issue in this stack.
  • Codex: 3 rounds, with a clean verdict on 198c318. Round 1 had 3 findings: 2 fixed, 1 declined (same-ID queued provenance across a rolling deploy; the ID-only queue compare predates this PR). Round 2 had 1 finding, fixed. There are no unresolved threads.
  • Frontend self-review: coder-frontend-review passed all 8 roles on every update.
  • UAT: remote dogfood round 2 PASS; evidence is in the comment above.
  • Independent check: ready with tracked follow-ups, CODAGT-1238 and CODAGT-1239.
  • Size: about 730 hand-written lines (312 product, 418 tests and stories) plus 124 generated.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…now' into tk/codagt-1215-automations-chat-labels
@ThomasK33

Copy link
Copy Markdown
Member Author

Merged the new 7b head (25b8907, which changes one line of coderd/x/chatd/ARCHITECTURE.md) into this branch. The new head is 91550ed. This PR's own diff did not change:

  • git diff 6417debe10..198c31804d | git patch-id --stable gives 6f1e5a6d365b0dc804a011c63dab5f47a57a2d30.
  • git diff 25b89076c1..91550edbdf | git patch-id --stable gives 6f1e5a6d365b0dc804a011c63dab5f47a57a2d30.

The only difference between 198c318 and 91550ed is the lower-layer ARCHITECTURE.md line. The clean Codex verdict on 198c318 therefore still covers this PR's changes, so I did not request a new round. CI for the new head is ci run 36684329074 (dispatched by hand, because no pull_request run was created while mergeability is UNKNOWN).


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…gt-1215-automations-chat-labels

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

sync with main d9c13dc: no content conflicts; make gen produced no diff


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ThomasK33
ThomasK33 marked this pull request as ready for review October 1, 2026 09:39
@ThomasK33
ThomasK33 requested a review from a team as a code owner October 1, 2026 09:39
Only chatAutomationNameMap uses it in this layer, so knip reported it as
an unused export.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

sync with main 35abc22: stopped exporting the chatAutomations query (knip flagged it as unused after the review fix)


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

The loading and error label states are covered by the QueuedMessagesList
and ConversationTimeline stories, so the timeline tests keep only the
automations request assertions.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

sync with main 35abc22: dropped the rendered label checks Codex flagged (FE1); the stories cover those states. Upper layers pick this up in the next sync pass


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…gt-1215-automations-chat-labels

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

sync with main d11809f: merged #30166


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ibetitsmike ibetitsmike left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Xum re-reviewed this PR on behalf of @ibetitsmike using GPT-6.1 Sol (high) Exec subagents.

The prior hook, required timeline-name prop, tooltip-status and tooltip-story fixes remain intact at the current pin. New findings: Standards 0, Spec 0, Frontend 0; the optional label status and remaining rendered-label wait stay on their existing threads, and the fixture nit remains optional. At this head, the Go provenance test, 9 focused Vitest cases, TypeScript check and 5 Chromium story setups passed. These setup checks are not Pixel or visual approval; fresh required CI/Pixel verdicts and remote UAT were not verified.

…gt-1215-automations-chat-labels

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

sync with main a328bee: merged #30166


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…gt-1215-automations-chat-labels

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30166


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…gt-1215-automations-chat-labels

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30166


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…gt-1215-automations-chat-labels

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30166 (main dc07908)


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

AutomationLabel no longer defaults nameStatus to "settled", so every
caller states the automations list status explicitly instead of
silently falling back to the automation ID.

- ConversationTimeline: ChatMessageItem requires automationNameStatus.
  Rows without an automation (and the live row) pass a module-level
  "settled" constant so memoized rows still skip re-rendering when the
  list status changes.
- QueuedMessagesList requires automationNames. AgentChatInput keeps the
  prop optional because the create form and loading skeleton have no
  queue, and defaults it to an explicit empty, settled constant.
- ChatPageContent.test: replace the queuedAutomationInput factory, which
  was called twice with identical arguments, with a
  mockQueuedAutomationInput fixture spread from MockChatQueuedMessage.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…ame defaults

Pass the "settled" literal to rows without an automation instead of a
named constant, since a string prop is already stable for memo. Move
NO_AUTOMATION_NAMES off the pill sizing comment and document the
optional automationNames prop where it is declared.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

Review fixes: AutomationLabel's nameStatus is now required and passed explicitly (rows without an automation get the "settled" literal, which is stable for memo); QueuedMessagesList requires automationNames; ChatPageContent.test.tsx uses a mockQueuedAutomationInput constant spread from MockChatQueuedMessage.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ThomasK33

Copy link
Copy Markdown
Member Author

Review follow-ups in head c551108:

  • Fixture nit from Mike's review: a7a0564 replaces the queuedAutomationInput(...) factory in ChatPageContent.test.tsx with a mockQueuedAutomationInput constant spread from MockChatQueuedMessage, used in both tests.
  • Required label status: see the reply on the thread.
  • Danielle's hook thread: the hook was removed in 2c3fd7c (reply on that thread); chatAutomationNameMap adds a select to the chatAutomations query options and callers use useQuery directly.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…automation request tests

The experiment-off tests waited on a rendered automation label or the
Send now button before asserting that no automations request was sent.
They now wait until the component has built its automation name query,
which happens only after the dashboard has loaded the experiments, and
then check the request spy. The label rendering stays covered by the
AutomationUserMessages and AutomationMessages stories.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
… mocked module

oxlint's consistent-type-imports rule forbids import() type annotations.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

@ibetitsmike ibetitsmike left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Xum validated this PR and approved it on behalf of @ibetitsmike.

The original test-contract finding is fixed: the maintained tests wait for the real query factory and assert the API outcome, not rendered copy. Fresh production-gate removal controls independently reproduce the forbidden request for history, queue experiment, and queue missing-input guards; the unchanged tests pass again with production restored. The required-status and shared-fixture corrections remain intact.

Independent GPT-6.1 Sol (high) Standards and Spec verification found no remaining own-layer blocker. Approval is tied to this exact head and base, current local behavioral/lint/type validation, unchanged PR scope, all-green CI, and an immediate live gate.

…ions-chat-labels

Brings main ff158c0 into this layer. No conflicts.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…ions-chat-labels

Brings the new_chat automation title source fix from #30134.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33
ThomasK33 merged commit b96f1b9 into main Oct 2, 2026
32 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 2, 2026
@ThomasK33
ThomasK33 deleted the tk/codagt-1215-automations-chat-labels branch October 2, 2026 15:55
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants