Repository navigation
chore: add an experiment registry gated by lint and code owners - #30919
Open
ibetitsmike wants to merge 3 commits into
Open
ibetitsmike wants to merge 3 commits into
ibetitsmike wants to merge 3 commits into
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Update experiments/registry.yaml for main: chat projects lost its flag (#31354), chat-project-sharing and chat-acp-subagents are new flags, and #31300 removed the computer use provider routes. Add make lint/check-experiment-registry, which fails when the registry flags differ from codersdk.ExperimentsKnown, and make the registry and its lint owned by @spikecurtis, @mafredri, and @johnstcn.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds
experiments/registry.yaml, an inventory of every experiment onmaintoday, plus a lint check and code owners that make it the gate for experiment flag changes. Adding, renaming, or removing a flag incodersdk.ExperimentsKnownfailsmake lintuntil the registry changes, andCODEOWNERSroutes every registry change to @spikecurtis, @mafredri, and @johnstcn.Registry
The scan covered every marker the repo uses: the 18 flags in
codersdk.ExperimentsKnown, thex/directories,exp_files,/api/experimentalroutes,docs/manifeststage labels, and the frontendexperiments.includes(...)gates. Each entry records the stage, the date it entered that stage, the flag, the paths it owns, its experimental routes and its docs page. Atoolssection listsexp_files that are dev tools, not features (scale tests, debug commands).29 entries: 18 flagged experiments, 11 unflagged ones. Stage comes from the docs label. A feature behind a hidden flag with no label is early access.
Drift the scan found, recorded in each entry's
notes:x/and some routes are still under/api/experimental.chat-projectsflag in feat: graduate chat projects out of the chat-projects experiment #31354, but the docs label is early access and the API is still under/api/experimental. Sharing moved to the newchat-project-sharingflag, which has no docs page.mcp-server-http: the docs label the page beta, but the endpoint needs a hidden flag.notifications: nothing reads the flag since chore: remove notifications experiment #14869, but it is still a known experiment.chat-virtual-desktopandchat-acp-subagents: the flags gate chatd only. The desktop stream route and the agent's/api/v0/desktopand/api/v0/acpAPIs are always on./api/v2routes that do not exist.startup-coordinationandexternal-workspacesare early access by docs label only. No flag gates them.Left out: JetBrains Toolbox (labelled beta, but no gate or experimental code) and the removed
agentsandchat-automationsflags. Noownerfield yet, since there is no team mapping to fill it.Lint
make lint/check-experiment-registry(part ofmake lint, so CI's lint job runs it) runsscripts/checkexperimentregistry. It fails when:codersdk.ExperimentsKnownhas no registry entry,flagis not incodersdk.ExperimentsKnown,stageorsincevalue is invalid, or the file has fields outside the schema.It checks flags only. Paths, routes, and docs are not linted, so renaming a file does not need a registry owner's approval.
Code owners
experiments/registry.yamlandscripts/checkexperimentregistry/are owned by @spikecurtis, @mafredri, and @johnstcn. GitHub needs one owner's approval, not all three. OnmainCODEOWNERS review is not required today, so the gate holds only once a ruleset (or branch protection) requires code owner review for these paths; that needs a repository admin.Testing
go test ./scripts/checkexperimentregistry/: table tests for each failure above, unknown fields, and exit codes.make lint/check-experiment-registrypasses on this branch, and failed before the registry update with the three flag mismatches from feat: graduate chat projects out of the chat-projects experiment #31354 and feat: show available ACP harnesses in chat context #31059.x/andexp_file belongs to an entry or a tool. Five routes are missing from swagger; each was confirmed in the router.