Skip to content

chore: add an experiment registry gated by lint and code owners - #30919

Open
ibetitsmike wants to merge 3 commits into
mainfrom
mike/experiments-registry
Open

ibetitsmike wants to merge 3 commits into
mainfrom
mike/experiments-registry

Conversation

@ibetitsmike

@ibetitsmike ibetitsmike commented Oct 18, 2026 •

Copy link
Copy Markdown
Collaborator

Adds experiments/registry.yaml, an inventory of every experiment on main today, plus a lint check and code owners that make it the gate for experiment flag changes. Adding, renaming, or removing a flag in codersdk.ExperimentsKnown fails make lint until the registry changes, and CODEOWNERS routes every registry change to @spikecurtis, @mafredri, and @johnstcn.

Registry

The scan covered every marker the repo uses: the 18 flags in codersdk.ExperimentsKnown, the x/ directories, exp_ files, /api/experimental routes, docs/manifest stage labels, and the frontend experiments.includes(...) gates. Each entry records the stage, the date it entered that stage, the flag, the paths it owns, its experimental routes and its docs page. A tools section lists exp_ files that are dev tools, not features (scale tests, debug commands).

29 entries: 18 flagged experiments, 11 unflagged ones. Stage comes from the docs label. A feature behind a hidden flag with no label is early access.

Drift the scan found, recorded in each entry's notes:

  • Coder Agents and agent skills are GA, but their code is still under x/ and some routes are still under /api/experimental.
  • Chat projects lost the chat-projects flag in feat: graduate chat projects out of the chat-projects experiment #31354, but the docs label is early access and the API is still under /api/experimental. Sharing moved to the new chat-project-sharing flag, which has no docs page.
  • mcp-server-http: the docs label the page beta, but the endpoint needs a hidden flag.
  • notifications: nothing reads the flag since chore: remove notifications experiment #14869, but it is still a known experiment.
  • chat-virtual-desktop and chat-acp-subagents: the flags gate chatd only. The desktop stream route and the agent's /api/v0/desktop and /api/v0/acp APIs are always on.
  • Debug-logging docs name /api/v2 routes that do not exist.
  • startup-coordination and external-workspaces are early access by docs label only. No flag gates them.

Left out: JetBrains Toolbox (labelled beta, but no gate or experimental code) and the removed agents and chat-automations flags. No owner field yet, since there is no team mapping to fill it.

Lint

make lint/check-experiment-registry (part of make lint, so CI's lint job runs it) runs scripts/checkexperimentregistry. It fails when:

  • a flag in codersdk.ExperimentsKnown has no registry entry,
  • a registry flag is not in codersdk.ExperimentsKnown,
  • two entries claim the same flag or key,
  • a stage or since value is invalid, or the file has fields outside the schema.

It checks flags only. Paths, routes, and docs are not linted, so renaming a file does not need a registry owner's approval.

Code owners

experiments/registry.yaml and scripts/checkexperimentregistry/ are owned by @spikecurtis, @mafredri, and @johnstcn. GitHub needs one owner's approval, not all three. On main CODEOWNERS review is not required today, so the gate holds only once a ruleset (or branch protection) requires code owner review for these paths; that needs a repository admin.

Testing

Xum acted for @ibetitsmike to update this PR.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 18, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-18T10:11:25.165856Z 82bf2fe PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@linear-code

linear-code Bot commented Oct 21, 2026

Copy link
Copy Markdown

CODAGT-1116

Update experiments/registry.yaml for main: chat projects lost its flag
(#31354), chat-project-sharing and chat-acp-subagents are new flags, and
#31300 removed the computer use provider routes.

Add make lint/check-experiment-registry, which fails when the registry
flags differ from codersdk.ExperimentsKnown, and make the registry and
its lint owned by @spikecurtis, @mafredri, and @johnstcn.
@ibetitsmike ibetitsmike changed the title chore(experiments): add registry of existing experiments chore: add an experiment registry gated by lint and code owners Oct 29, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant