Skip to content

docs: add chat automations documentation - #30220

Merged
ThomasK33 merged 30 commits into
tk/codagt-1212c-automations-tool-togglefrom
tk/codagt-1216-automations-docs
Oct 2, 2026
Merged

ThomasK33 merged 30 commits into
tk/codagt-1212c-automations-tool-togglefrom
tk/codagt-1216-automations-docs

Conversation

@ThomasK33

Copy link
Copy Markdown
Member

Closes CODAGT-1216
Part of CODAGT-1135

Adds an early-access page for chat automations in Coder Agents: docs/ai-coder/agents/automations.md. It covers enabling the chat-automations experiment and the off experiment rule as the operator kill switch, webhooks (endpoint, bearer secret, 256 KiB JSON body, responses, single and multi use, rotation), schedules (cron and time zone, server previews, no catch-up, Run now), When busy, the manage_automations tool and heartbeats, the per-owner cap and the queue share, and the accepted risks (spend, silent failures, prompt injection).

It also adds the manifest entry (early access, after Chat Sharing), a glossary entry, a manage_automations row in the built-in tools table, and a link from the experiment-gated features list in Platform Controls.

The page documents the code as built on this stack, not the RFC, where they differ. Notable differences: reasoning effort is only offered for new chat targets; schedules accept an occurrence up to 60 seconds late and never replay older ones; the dashboard has no delete control, so delete is documented through the API; manage_automations never returns webhook secrets.

Dogfood UAT followed the page step by step on a develop.sh instance built from this branch (experiment rule on and off, webhook creation and curl deliveries with the documented status codes we could trigger, schedule previews and Run now, the Manage automations switch), and passed.

Where the page's claims come from
Claim Source
Experiment name, user-scoped codersdk/deployment.go (ExperimentChatAutomations, ExperimentsUserScoped)
Behavior when the experiment is off coderd/exp_chat_automations.go (requireChatAutomations, events handler), coderd/x/chatd/automation_schedules.go (scan skips owners, keeps cursors)
Management routes and fields coderd/coderd.go, codersdk/chatautomations.go, coderd/x/chatd/automations.go (validation)
Owner-only update, enable, rotate, run coderd/x/chatd/automations.go (ErrAutomationOwnerOnly)
Webhook endpoint, 256 KiB limit, responses coderd/exp_chat_automations.go (events handler, writeChatAutomationError), coderd/exp_chats.go (maxChatRequestBodyBytes)
Event data wrapping coderd/x/chatd/automations_publish.go (automationEventText)
Rotating a used single-use webhook returns 409 coderd/x/chatd/automations.go (RotateAutomationSecret), AutomationWebhookFields.tsx (button hidden)
Events rate limit coderd/httpmw/ratelimit.go (RateLimitByEndpointKey), --api-rate-limit default 512
Cron, time zone, previews coderd/schedule/cron/cron.go, coderd/exp_chat_automations.go (chatAutomationNextRunCount = 5, schedule preview)
30 s scan, 60 s grace, no catch-up coderd/x/chatd/automation_schedules.go (automationScheduleInterval, automationScheduleGrace)
When busy, queue share coderd/x/chatd/automations_publish.go (busy check, max(1, MaxQueuedMessagesPerChat/2)), codersdk/chats.go (default 20)
manage_automations gating and containment coderd/x/chatd/automations_tool.go, coderd/exp_chats.go (manage_automations_enabled checks), AgentChatInput.tsx
Per-owner cap codersdk/deployment.go (--chat-max-automations-per-owner), codersdk/chats.go (default 50)
Skipped and missed log messages coderd/x/chatd/automation_schedules.go
Audit fields enterprise/audit/table.go (prompt and webhook_secret_hash are secret)

Identifiers kept off the page because the reader never types or receives them: queue_generation, schedule_revision, schedule_next_run_at, and the internal error names above.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Add an early access page for chat automations: enabling the
chat-automations experiment and the experiment rule kill switch,
webhook automations (endpoint, bearer secret, 256 KiB JSON body,
responses, single and multi use, rotation), schedule automations
(cron, time zone, server previews, no catch-up, Run now), targets,
When busy, the manage_automations tool and heartbeats, limits, and
known risks.

Register the page in the manifest, add a glossary entry, and link it
from the experiments list and the built-in tools table.

_Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high_

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…bhook_use

An occurrence that came due while the experiment was off runs late only if a
30 s scan sees the experiment on inside the 60 s grace window, so in practice
the experiment must be back on within 30 s. Also note that an update request
ignores webhook_use instead of rejecting it.
Drop the Prompt row from the limits table (prompts have no length cap; the
request table already marks the field required) and a repeated new chat
qualifier on reasoning effort.

Signed-off-by: Thomas Kosiewski <tk@coder.com>
…-toggle' into tk/codagt-1216-automations-docs
…d rotate fixes

Rotating the secret of a used single-use webhook now returns 409, and
manage_automations never returns webhook secrets. Note that update,
enable and run_now are rechecked when the change is applied.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33
ThomasK33 added this pull request to stack #30097 October 1, 2026 03:26
@linear-code

linear-code Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

CODAGT-1135

CODAGT-1216

@github-actions

github-actions Bot commented Oct 1, 2026 •

Copy link
Copy Markdown
Contributor

Docs preview

Check off each page once it's been reviewed. If a page changes in a later push, its checkbox clears automatically so it gets a fresh look. Pages not yet wired into the docs navigation aren't listed here.

@ThomasK33

Copy link
Copy Markdown
Member Author

Manual CI run for head d7709b0aa5e9496175add1612e5244663687a773: https://github.com/coder/coder/actions/runs/36810532346 (dispatched because pushes to stacked PRs may not start pull_request CI).

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-01T04:42:50.322491Z 5149bd3 Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7709b0aa5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Keep the note on one line, declare one audience, describe off as
stopping automations, show the off command output, split target steps
into one instruction each, and stop promising a future endpoint path.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…-toggle' into tk/codagt-1216-automations-docs
@ThomasK33

Copy link
Copy Markdown
Member Author

Manual CI run for head 0bc969a2c5ba2ec49340511dd8856f157e8bebc3: https://github.com/coder/coder/actions/runs/36811701487 (dispatched because pushes to stacked PRs may not start pull_request CI).

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0bc969a2c5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Disclose that the tool can read, turn off and delete any of the owner's
automations in the organization, note that turning the experiment off
does not remove queued prompts, and link Run now refusals to the
webhook response table.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

Manual CI run for head 9b606a9cfc1ae2d583dc27871b4771babef8e52f: https://github.com/coder/coder/actions/runs/36812641108 (dispatched because pushes to stacked PRs may not start pull_request CI).

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9b606a9cfc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Mark kind and target_mode as required, say a minute schedule can start
a turn every minute, and note that chat-dependent webhook refusals are
also logged.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

Manual CI run for head 34cac3a2bf3e41fc56e7dae4d7f9e6cca4489b69: https://github.com/coder/coder/actions/runs/36813646679 (dispatched because pushes to stacked PRs may not start pull_request CI).

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 34cac3a2bf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Split the schedule steps and add the organization step, make recovery
after the experiment returns best effort, note silent schedule states,
and limit the relay advice to senders that can't set a bearer header.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

Manual CI run for head b3c1c3e7e52ed084d1fd5f26c579c8a2b593c15a: https://github.com/coder/coder/actions/runs/36815020331 (dispatched because pushes to stacked PRs may not start pull_request CI).

@ThomasK33

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b3c1c3e7e5

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread docs/manifest.json Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
Comment thread docs/ai-coder/agents/automations.md Resolved
@ThomasK33

Copy link
Copy Markdown
Member Author

sync with main a328bee: merged #30219


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…codagt-1216-automations-docs

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30219


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…codagt-1216-automations-docs

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30219 (main 15382eb)


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…codagt-1216-automations-docs

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30219 (main dc07908)


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…codagt-1216-automations-docs

No conflicts. make gen produced no diff.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

merged #30219


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…le recovery

Add manage_automations to the platform tools table in the architecture
page, which enumerates every built-in tool.

Describe the late run of an occurrence that came due while the
experiment was off as best effort: lock or hook delays can still push
it past the 60 second window. Turning the experiment back on before the
due time remains the way to be sure it runs.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
…codagt-1216-automations-docs

No conflicts.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

Docs review fixes: manage_automations in the architecture page's platform tools table; the late recovery of an occurrence missed while the experiment was off is described as best effort. Merged #30219.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ThomasK33

Copy link
Copy Markdown
Member Author

Review follow-ups in head cc508e0, both in 9675918:

  • Mike's re-review, recovery wording in docs/ai-coder/agents/automations.md: an occurrence that came due while the experiment was off now "can run once, late" if the experiment is back on at a check inside the window, followed by "This recovery is best effort: lock or hook delays can still push the occurrence past the window." The sentence telling users to turn the experiment back on before the due time to be sure is unchanged.
  • Documentation Check on #30214: manage_automations added to the platform tools table in docs/ai-coder/agents/architecture.md.

Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

Turning the experiment back on before an occurrence is due avoids the
late recovery path, but it does not guarantee the occurrence runs.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33

Copy link
Copy Markdown
Member Author

On the recovery wording held from review 5383780861 (thread r4151796827): docs/ai-coder/agents/automations.md no longer implies a guarantee. The text in "What happens when automations are off" now reads:

An occurrence that came due while the experiment was off can run once, late, if the experiment is back on at a check inside that window.
This recovery is best effort: lock or hook delays can still push the occurrence past the window.
Otherwise the occurrence counts as missed and the schedule continues from its next time.
To avoid depending on this recovery, turn the experiment back on before the occurrence is due.

The best-effort sentence landed in 9675918. Commit dc0d910 replaces the last overstatement, "To be sure an occurrence runs, turn the experiment back on before its due time." No other docs page repeats the recovery claim. "When a schedule runs" describes only the 60-second lateness limit and that missed occurrences are never replayed.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

@ibetitsmike ibetitsmike left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Xum reviewed this PR on behalf of @ibetitsmike using GPT-6.1 Sol (high) Exec subagents.

The original certainty-wording finding is fixed. One own-layer documentation-style issue remains below, so I am withholding approval. This is not a runtime scheduling defect; the accepted functional boundaries remain unchanged.

Comment thread docs/ai-coder/agents/automations.md Outdated
### Silent failures

Skipped and missed schedule runs appear only in the server logs, as `chat automation schedule occurrence skipped` and `chat automation schedule occurrence missed` messages.
A schedule whose target chat is archived or deleted, or whose owner isn't active, doesn't run and logs nothing.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: Follow the docs word-choice policy

Both whose clauses use the schedule as their antecedent. docs/.style/style-guide/word-choice.md ("Whose for people, not things") explicitly disallows that inanimate construction. This policy is documentation-only, so a clean Vale run does not check it. The scheduling behavior described here is correct; preserve it while changing the wording.

Suggested change
A schedule whose target chat is archived or deleted, or whose owner isn't active, doesn't run and logs nothing.
A schedule doesn't run or produce logs when its target chat is archived or deleted, or when its owner isn't active.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 8a58767. Silent failures now uses your wording: "A schedule doesn't run or produce logs when its target chat is archived or deleted, or when its owner isn't active."

I also checked every hand-written docs page this stack adds or changes (automations.md, architecture.md, chat-search-syntax.md, index.md, platform-controls/index.md, glossary.md) against the word-choice rules that Vale doesn't check. There was no other inanimate "whose". The same commit fixes the other findings, all on this page:

  • "the server" / "server logs" / "server options" now name the control plane, coderd (introduced as "the process that runs the control plane") and coder server (Naming Coder's components).
  • "keep the URL easy to update" is now "keep the URL configurable" (Don't assume simplicity or difficulty).

markdownlint (pnpm run lint-docs) and Vale report 0 errors.


Generated with xum • Model: anthropic:claude-opus-5-5 • Thinking: high

…s page

Replace the inanimate "whose" in Silent failures with the reviewer's
wording, name the control plane, `coderd` and `coder server` instead
of "the server", and drop "easy" from the experimental API note.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_

@ibetitsmike ibetitsmike left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Xum validated this PR and approved it on behalf of @ibetitsmike.

The reported wording findings are fixed. The unchanged head, base, body and seven docs blobs match the completed independent Standards and Spec reviews; those manual verdicts were carried, not rerun.

All six fresh exact-head docs checks passed: Markdownlint, tables, manifest validation, formatted generated-manifest byte comparison, manifest JSON, and local links/anchors (169 local passes; external URLs not fetched).

Exact-commit CI run 37012079913 passed, including required; its 26 check runs succeeded or were skipped. The PR rollup omits that dispatch run. Pixel / Review remains pending and explicitly non-required. This docs-only approval does not claim Pixel validation or merge readiness. No fresh Go, frontend, full-suite or remote UAT checks were run.

…omations-docs

Brings main ff158c0 into this layer. No conflicts.

---
_Generated with [`xum`](https://github.com/coder/xum) • Model: `anthropic:claude-opus-5-5` • Thinking: `high`_
@ThomasK33
ThomasK33 merged commit 7f3d353 into main Oct 2, 2026
30 checks passed
@github-actions github-actions Bot locked and limited conversation to collaborators Oct 2, 2026
@ThomasK33
ThomasK33 deleted the tk/codagt-1216-automations-docs branch October 2, 2026 15:55
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants