Skip to content

ci: build docs PR previews on per-PR preview sites - #30698

Merged
nickvigilante merged 13 commits into
mainfrom
vigilante/docs-pr-previews
Nov 12, 2026
Merged

nickvigilante merged 13 commits into
mainfrom
vigilante/docs-pr-previews

Conversation

@nickvigilante

@nickvigilante nickvigilante commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Docs previews at coder.com/docs/@<branch> share coder.com's origin and cookies with the real site. This publishes each pull request's docs as its own site on docs.cdr-preview.dev, a domain used only for previews (repository variable DOCS_PREVIEW_DOMAIN), and points the existing docs-preview comment's per-page links and checkboxes at it.

URLs

  • https://pr-<n>.docs.cdr-preview.dev: the PR head, re-aliased on every build. A slower build of an older commit doesn't take it back.
  • https://pr-<n>-<first 9 of sha>.docs.cdr-preview.dev: one per built commit, listed newest first in a collapsible "Preview sites" list in the PR's docs preview comment, so reviewers can step through the branch.
  • Every URL is built from vars.DOCS_PREVIEW_DOMAIN (aliases, comment links, cleanup); nothing hard-codes the domain. Previews moved off coderite.app, which also hosts login-protected internal sites, so a preview page whose content any coder/coder writer controls would have been same-site with them. cdr-preview.dev hosts nothing else, and the docs previews sit under docs. so other preview kinds can get their own subdomain later. The jobs fail with a clear error if it is unset or a vercel.app name, because *.vercel.app names are first-come across all Vercel accounts and anyone could claim a preview's name before our deploy or after cleanup frees it.
  • Every preview carries the docs CSP and the other headers from docs-engine's apps/offlinedocs/vercel.json, plus X-Robots-Tag: noindex, which the deploy job prepends to the Build Output API config.json routes without touching the existing ones.
  • On merge or close, every deployment tagged pr=<n> and its aliases are deleted. A weekly groom (Mondays 07:23 UTC, also workflow_dispatch) deletes previews of any closed PR, in case a close event was missed.

Who gets previews

Only PRs from a branch in coder/coder, on every push that touches docs/**. Every job in the deploy workflow, and the close path of the cleanup, requires github.event.pull_request.head.repo.full_name == github.repository.

Fork PRs get no preview for now. Building fork content in a pull_request_target run would share the base branch's Actions cache scope, so a fork could poison main's cache, and it would sit beside the private engine source. Fork previews come in a follow-up once the build runs in a no-network sandbox (or the public builder kit ships). That follow-up will gate on a maintainer stating the exact approved commit, not on a label, so a push after approval can't slip in.

One comment, posted only after a deploy

docs-preview.yaml is now a reusable workflow (workflow_call only). The deploy workflow's comment job calls it after deploy succeeds, so the per-page links never point at a preview that doesn't exist yet. I chose this over moving the per-page logic into the deploy job: the existing comment logic and its mapper test stay as they are, and only the trigger changes.

Each PR gets at most one bot comment. The deploy job posts nothing; it passes the built commit (short_sha) to docs-preview.yaml, which adds a collapsible "Preview sites for each built commit" list to its comment, newest first, read back from the comment on each run and capped at 100. That comment is only posted when a non-generated page or an image changed: a PR that changes only docs/reference/** (generated CLI and API pages), or nothing previewable, gets no comment, and an earlier one is deleted. Engineers found the earlier two-comment version too noisy. If the comment is deleted that way and a later push brings it back, its commit list starts over; the earlier previews still exist until the PR closes.

Three jobs, so PR content never runs beside a secret

  1. engine: checks DOCS_PREVIEW_DOMAIN first, reads the docs-engine commit pinned in docs/.engine-ref (from the PR head, falling back to the base for a branch that predates the pin), mints a short-lived token from a GitHub App installed only on coder/docs-engine, downscoped to contents: read on that one repository, checks out with persist-credentials: false, and uploads a tarball plus its sha256 as a job output. Never touches PR content.
  2. build: no secrets, contents: read only. Verifies the engine tarball against the digest, sparse-checks out the PR's docs/, refuses a docs/ that is or contains a symbolic link (one could pull runner files, such as the private engine source, into a public preview), builds apps/offlinedocs with DOCS_ROOT as docs-engine's own docs-pr-preview.yml does, and uploads only the static out/ export.
  3. deploy: in the docs-preview environment with VERCEL_PREVIEW_TOKEN. Verifies the engine tarball against the digest again (a compromised build job could replace an artifact, but not a job output), refuses an export holding anything but regular files and directories, lays the static files out with the engine's build:output script, adds noindex, runs vercel deploy --prebuilt with meta pr and sha, and sets both aliases through the Vercel API. It holds only pull-requests: read, for the check that the PR head hasn't moved before re-aliasing; the comment job posts the comment. It runs the engine's trusted script rather than taking config.json from the build job, because a Build Output directory from another job could carry serverless functions or rewritten routes.

docs-pr-preview-cleanup.yaml keeps pull_request_target: closed so a PR can't change its own cleanup; it checks out nothing and runs no PR content, only Vercel and GitHub API calls. Fork PRs have no previews, so their close is skipped.

PR-chosen strings never reach a log line unfiltered: the comment workflow prints changed reference-page paths with every character outside a path allowlist as ?, so a file name can't forge a :: or ##[ workflow command, and the deploy job's errors name no path.

Engine pin

docs/.engine-ref now pins docs-engine main at eadb477, the engine coder.com/docs launched with on 2026-10-28 (was e3c55c2; an earlier revision of this PR pinned 5584d67). The commands the workflow runs are unchanged between them. e3c55c2 already had the build:output script, but the newer engine refuses a symlinked docs root, renders search hits without raw HTML, drops author-chosen heading ids that could break hydration, and pins Node 22.23.4, which the build and deploy jobs now use too. The commands the workflow runs (pnpm turbo run build --filter=@coder/offlinedocs and apps/offlinedocs/scripts/build-output.mjs) are unchanged at 5584d67.

The pin must be a commit on docs-engine main. The engine job checks docs-engine out with full history and fails before packing unless git merge-base --is-ancestor "$REF" origin/main passes. Once docs-engine is public, a commit pushed to any fork of it can be fetched by SHA through coder/docs-engine, so the 40-hex format check alone would let a PR build with unreviewed engine code. test-docs-preview-mapper.sh pins that the check runs in the engine job, before the pack step, against a full-history checkout.

Before this can run

Checked on 2026-11-12: IT registered cdr-preview.dev in Vercel on team team_tGkWfhEGGelkkqUUm9nXq17r, with Vercel nameservers, so *.docs.cdr-preview.dev needs no DNS or wildcard step. IT also set docs.cdr-preview.dev as the coder-docs-pr-preview project's own preview deployment suffix and removed coderite.app from it, so the project's generated deployment URLs land on the preview domain too, not on the team's coderite.app suffix; the first deploy will confirm that. The project has Vercel Authentication and password protection off. The secrets DOCS_ENGINE_APP_ID and DOCS_ENGINE_APP_PRIVATE_KEY exist. DOCS_PREVIEW_DOMAIN is set to docs.cdr-preview.dev. The docs-preview environment exists (created 2026-10-28) with no protection rules and no branch policy, and holds VERCEL_PREVIEW_ORG_ID and VERCEL_PREVIEW_PROJECT_ID, and VERCEL_PREVIEW_TOKEN was stored on 2026-11-12.

  • GitHub App (org owner; requested in DOCS-1739). Confirm the app behind DOCS_ENGINE_APP_ID is installed on coder/docs-engine only ("Only select repositories") with Repository permissions Contents: Read-only and Metadata: Read-only and nothing else, at https://github.com/organizations/coder/settings/installations. The workflow also downscopes each token to contents: read on docs-engine, so the installation is the outer bound.
  • Repository variable, set on 2026-11-12. Set it at repository level, not on the environment, because the engine job and docs-preview.yaml run outside it:
    gh variable set DOCS_PREVIEW_DOMAIN -R coder/coder --body docs.cdr-preview.dev
  • Vercel Authentication off on coder-docs-pr-preview (prj_RiVa1voPHmsfbEXkasFDXmYqAr0L): checked on 2026-11-12, ssoProtection and passwordProtection both report enabled: false.
  • Preview domain (IT): cdr-preview.dev registered with Vercel nameservers, and docs.cdr-preview.dev set as the project's preview deployment suffix.
  • Project-scoped Vercel token, 90-day expiry (created 2026-11-12, expires about 2027-02-10; a rotation reminder is set for 2027-02-09). The three checks below returned 200, 404 and 403. Create it at https://vercel.com/account/settings/tokens, scoped to the coder-docs-pr-preview project only (vercel tokens create --project is refused for non-owners). Then check, with the new token in $VERCEL_PREVIEW_TOKEN, that it can list the project's deployments (200), reach the team-level alias API (404, not 403), and cannot see the production coder-docs project (403 or 404, never 200):
    curl -sS -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $VERCEL_PREVIEW_TOKEN" \
      "https://api.vercel.com/v6/deployments?teamId=team_tGkWfhEGGelkkqUUm9nXq17r&projectId=prj_RiVa1voPHmsfbEXkasFDXmYqAr0L&limit=1"
    curl -sS -o /dev/null -w '%{http_code}\n' -X DELETE -H "Authorization: Bearer $VERCEL_PREVIEW_TOKEN" \
      "https://api.vercel.com/v2/aliases/pr-0.docs.cdr-preview.dev?teamId=team_tGkWfhEGGelkkqUUm9nXq17r"
    curl -sS -o /dev/null -w '%{http_code}\n' -H "Authorization: Bearer $VERCEL_PREVIEW_TOKEN" \
      "https://api.vercel.com/v6/deployments?teamId=team_tGkWfhEGGelkkqUUm9nXq17r&projectId=prj_pC4y7hjUgcsYYNoSDL7pNJnafzh3&limit=1"
  • docs-preview environment: it exists with no branch policy (because pull_request runs on the merge ref) and its two variables are set. Only the token is left:
    gh secret set VERCEL_PREVIEW_TOKEN -R coder/coder --env docs-preview
    gh secret set prompts for the token so it stays out of shell history.
  • Security sign-off (Lisa), on these points:
    • Preview domain. Previews now live on docs.cdr-preview.dev, which hosts nothing else, so they are no longer same-site with the login-protected sites on coderite.app. Previews are same-site with each other, so one PR's preview could set a cookie the others receive. Anything added to cdr-preview.dev later should stay outside docs. and not rely on cookies scoped to the parent domain. The docs CSP allows inline scripts, and any coder/coder writer controls a preview's content.
    • Any coder/coder writer can reach VERCEL_PREVIEW_TOKEN by editing the workflow on a branch, as with any secret a pull_request deploy uses. The token is scoped to the one preview project to bound that; required reviewers on the environment would close it, at the cost of an approval per preview.
    • The engine commit comes from the PR's docs/.engine-ref, so the engine code that lays out the export in the deploy job is PR-chosen, but only among commits on docs-engine main. It runs before, not in, the step that holds the token.
  • Mark ready for review once the items above are done and a docs push to this PR has produced a working preview.

Testing

test-docs-preview-mapper.sh checks that docs-preview.yaml reads the domain from vars.DOCS_PREVIEW_DOMAIN, builds links on pr-<n>.<domain> from the validated value with no coder.com/docs/@ or vercel.app link, and runs only as a reusable workflow called after deploy. It also checks the per-commit list (this commit first, earlier ones carried, repeats and other PRs' lines dropped, capped at 100), that the list round-trips through the comment it renders without touching checkbox state, and that the deploy job posts no comment and passes its commit on. All 58 checks pass (run with GNU coreutils, sed, grep and awk on macOS); the new workflow check failed before the change. actionlint at the repo's pinned commit, with ShellCheck, reports nothing, and so does the repo's pinned zizmor 1.11.0 (--strict-collection --persona=regular). The noindex step was run against a config.json from docs-engine's real build-output.mjs: the route count went from 11 to 12 with the CSP route intact. With the variable, environment and token in place, the engine and build jobs pass on this PR, including the check that the pinned engine commit is on docs-engine main. The first deploy failed at vercel pull with "Could not retrieve Project Settings": Vercel CLI 59.3.0 looks up the token's team (403 for a project-scoped token) and gives up, although the token reads the project (200). Run locally with the same token, 65.11.0 skips that lookup and pulls the settings, so the workflow now pins 65.11.0, the newest release more than a week old. The deploy, alias and cleanup calls have not completed in CI yet.


🤖 Built with AI assistance.

Docs previews at coder.com/docs/@<branch> share coder.com's origin and
cookies with the real site. Publish each PR's docs instead as its own site
on Vercel's domain, which is on the Public Suffix List:
coder-docs-pr-<number>.vercel.app for the PR head and
coder-docs-pr-<number>-<sha>.vercel.app for every built commit.

docs-pr-preview-deploy builds with docs-engine's offline export in three
jobs, so PR content never runs beside a secret: engine mints a short-lived
docs-engine read token and packs the source pinned in docs/.engine-ref;
build has no secrets and compiles the PR's docs/; deploy, in the
docs-preview environment, lays the static files out with the engine's
script (verified against the engine job's digest), adds X-Robots-Tag:
noindex, deploys prebuilt, sets both aliases and keeps one PR comment.
Branch PRs build on every push; fork PRs build only when a maintainer adds
the docs-preview label, and each new push removes it.

docs-pr-preview-cleanup deletes a PR's previews when it closes and grooms
previews of closed PRs weekly. docs-preview's per-page links now point at
the PR's preview site.

Assisted-by: AI
@linear-code

linear-code Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

DOCS-1001

DOCS-1506

A fork build in a pull_request_target run shares the base branch's Actions
cache scope, so a fork could poison main's cache, and it would sit beside
the private engine source. Drop the fork path: no pull_request_target build
or label handling, and every job requires the head repository to be this
one. The cleanup keeps pull_request_target: closed, which checks out and
runs nothing from the PR.

vercel.app names are first-come across all Vercel accounts, so anyone could
claim a preview's name before the deploy or after cleanup. Build every
preview URL from the repository variable DOCS_PREVIEW_DOMAIN instead, and
fail when it is unset or a vercel.app name.

docs-preview.yaml is now a reusable workflow that the deploy workflow calls
after a successful deploy, so its per-page links never point at a preview
that does not exist yet.

Assisted-by: AI
@nickvigilante nickvigilante changed the title ci: build docs PR previews on per-PR vercel.app sites ci: build docs PR previews on per-PR preview sites Oct 13, 2026
Move docs/.engine-ref from e3c55c2 to 5584d67. The newer engine refuses a
docs root that is a symbolic link, renders search hits without raw HTML,
drops author-chosen heading ids that could break hydration, and pins Node
to the patched 22.23.4 release. The build commands the preview runs
(`turbo run build --filter=@coder/offlinedocs` and
`scripts/build-output.mjs`) are unchanged at that commit.

Assisted-by: AI
- Refuse a docs/ tree with symbolic links before the build, and a static
  export holding anything but regular files and directories before the
  job with the Vercel token copies and uploads it.
- Print changed reference page paths with only allowlisted characters,
  since a PR-chosen file name could otherwise read as a "##[" workflow
  command.
- Build the comment's links from the validated domain, and test that the
  domain still comes from vars.DOCS_PREVIEW_DOMAIN.
- Use docs-preview.cdr.dev in the examples and test fixtures.
- Run Node 22.23.4, as the engine pins, and move harden-runner,
  create-github-app-token and upload-artifact to the pins the rest of
  the repository uses.

Assisted-by: AI
The preview domain is the team's Vercel domain coderite.app, read as
before from vars.DOCS_PREVIEW_DOMAIN. Only the header comment and the
test fixtures name it.

Assisted-by: AI
The new docs launched on coder.com from docs-engine main today, so
previews should build with the same engine. The commands this workflow
runs (the offlinedocs build and its build:output script) are unchanged
since the previous pin.

Assisted-by: AI
docs/.engine-ref comes from the pull request, so the PR chooses which docs-engine commit runs. Once docs-engine is public, a commit pushed to any fork of it can be fetched by SHA through coder/docs-engine, so a format check alone would let a PR build with unreviewed engine code. The engine job now checks out full history and fails before packing unless the pinned commit is an ancestor of docs-engine main.

Assisted-by: AI
@nickvigilante
nickvigilante temporarily deployed to main November 10, 2026 17:45 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 10, 2026 17:54 — with GitHub Actions Inactive
Previews move off coderite.app, which also hosts login-protected internal
sites, to docs.cdr-preview.dev, a domain used only for previews. The
domain still comes from DOCS_PREVIEW_DOMAIN; only the comments and test
fixtures named the old one.

Since the domain is now docs-only, the aliases drop their coder-docs-
prefix: pr-<n> for the PR head and pr-<n>-<sha:0:9> per commit. The
deploy, comment and cleanup workflows build the same names.

Assisted-by: AI
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 15:55 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 15:56 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 16:06 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 16:11 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 16:13 — with GitHub Actions Inactive
The preview token is scoped to the one preview project. Vercel CLI 59.3.0
looks up the token's user and team before every command and gives up when
the team lookup returns 403, so vercel pull failed with "Could not retrieve
Project Settings" although the token can read the project. 65.11.0 skips
that lookup and pulls the project's settings with the same token. It is
the newest release more than a week old.

Assisted-by: AI
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 16:25 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 16:27 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to docs-preview November 12, 2026 16:28 — with GitHub Actions Inactive
@github-actions

Copy link
Copy Markdown
Contributor

Docs preview deployment

This pull request's head: https://pr-30698.docs.cdr-preview.dev

Every built commit keeps its own preview, newest first:

Previews are not indexed by search engines and are deleted when this pull request is merged or closed.

@nickvigilante
nickvigilante marked this pull request as ready for review November 12, 2026 16:29
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 16:29 — with GitHub Actions Inactive
Engineers found two bot comments per docs PR too noisy. The deploy job no
longer posts its own "Docs preview deployment" comment. It passes the
built commit to docs-preview.yaml, whose existing comment gains a
collapsible list of every built commit's preview, newest first, read back
from the comment on each run and capped at 100.

docs-preview.yaml already posts nothing, and deletes its comment, when a
PR changes only generated reference pages or no page or image, so a
reference-only or non-docs PR now gets no comment at all. The deploy job's
pull-requests permission drops to read.

Assisted-by: AI
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 17:22 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to docs-preview November 12, 2026 17:26 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 17:26 — with GitHub Actions Inactive

@ianjevans ianjevans left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the per-commit previews as well as the overall. I'm 50/50 whether the links should be target=_blank.

@nickvigilante
nickvigilante merged commit 13e673e into main Nov 12, 2026
43 of 44 checks passed
@nickvigilante
nickvigilante deleted the vigilante/docs-pr-previews branch November 12, 2026 19:10
@nickvigilante
nickvigilante temporarily deployed to main November 12, 2026 19:10 — with GitHub Actions Inactive
@nickvigilante
nickvigilante temporarily deployed to docs-preview November 12, 2026 19:10 — with GitHub Actions Inactive
@github-actions github-actions Bot locked and limited conversation to collaborators Nov 12, 2026
@nickvigilante

Copy link
Copy Markdown
Collaborator Author

@ianjevans I'm indifferent. I'll merge this as-is and then launch a new PR for opening those links in a new tab (we should have an external link icon next to each link to signal to contributors and reviewers that it opens in a new tab).

This branch was previously deployed

2 inactive deployments
docs-preview — 0f8010b7 Deployed Nov 12, 2026 by nickvigilante via cleanup #1
main — 0f8010b7 Deployed Nov 12, 2026 by nickvigilante via cla #255223
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants