Skip to content

ci: automate the release — draft notes, sign in CI, publish behind approval - #63

Open
ndemianc wants to merge 2 commits into
developfrom
ci/automated-release
Open

ci: automate the release — draft notes, sign in CI, publish behind approval#63
ndemianc wants to merge 2 commits into
developfrom
ci/automated-release

Conversation

@ndemianc

Copy link
Copy Markdown
Contributor

Replaces the hybrid model (CI builds unsigned → you sign on your laptop → six gh commands by hand) with three chained workflows. The only manual steps left are the two that need judgement: writing the prose, and deciding to ship.

prepare-release.yml  →  you edit the PR  →  tag-on-merge.yml  →  release.yml
(drafts the notes)      (the prose)         (pushes the tag)     (builds, signs,
                                                                  notarizes, waits)

There is no version file to bump — build-macos.sh derives the version from git describe --tags, so the tag is the version.

Release notes

draft-release-notes.mjs already existed and already did the hard part: it fills in every fact (commit range, PRs, previous tag, suite/case counts, compare URL) and deliberately leaves <!-- TODO --> where judgement is needed. Its own header makes the case: "a changelog auto-generated from commit subjects is the reason most release notes go unread."

So prepare-release.yml opens a PR with that draft instead of committing it, and tag-on-merge.yml refuses to tag while a TODO or the scaffolding block survives — or when the first line doesn't name the version being tagged. The automation can't ship scaffolding, and it can't ship last release's notes either.

Signing

notarize.sh already accepted APPLE_ID + TEAM_ID + APP_SPECIFIC_PASSWORD as the CI alternative to a local keychain profile, so no build script changed. The workflow imports the cert into a temporary keychain in RUNNER_TEMP, runs the existing make-dmg.sh, verifies with codesign + stapler + spctl, and deletes the keychain in an always() step.

Two details are load-bearing and easy to omit:

  • set-key-partition-list — without it codesign blocks on a GUI keychain prompt nobody can click, and the job hangs until timeout rather than failing.
  • 6h keychain lock timeout — the 5-minute default re-locks during notarization, and the next codesign fails with a misleading "user interaction is not allowed".

Also fetch-depth: 0 on the build checkout: the version stamp comes from git describe --tags, and its failure mode is silent — it warns and ships a build whose About box reads 1.126.0, the Code-OSS base.

The gate

Publishing is deploying (§5): the Squirrel updater installs a published release on every existing install at its next check, no staged rollout, and the rollback pin cannot un-update anyone who already took it.

So the publish job sits in a release Environment with required reviewers. Everything before it is reversible — a branch, a tag, a draft. That step is not. Post-publish it polls the update feed and warns rather than fails, since the release is already correct at that point.

The draft job also refuses to publish unless all four assets are present. Losing only the x64 job would otherwise strand every Intel user silently, because the feed serves per-arch.

Before this can run

Six secrets, listed in RELEASING.md §7 — and one thing that is easy to skip and would quietly defeat the whole design:

Settings → Environments → release → Required reviewers. GitHub creates a missing environment implicitly with no protection rules, so without this the publish job runs straight through and the gate is decorative.

The trade-off, stated

The old §7 described this exact design as the road not taken: "puts your signing identity in the cloud — the hybrid flow above deliberately doesn't." That's now an accepted cost. The doc records what limits the blast radius (temp keychain, always-delete, secrets unavailable to fork PRs), the revocation path, and how to revert to hybrid if you'd rather not hold it.

Verification

actionlint — which shellchecks every run: block — clean on all three workflows.

The notes gate was extracted from the shipped workflow and exercised, so what was tested is what runs:

Input Expected Result
Real finished v1.0.4 notes, tagged v1.0.4 pass
Real generated draft (TODOs + scaffolding) fail ✅ both errors
TODOs removed, scaffolding left fail
v1.0.4 notes presented as v1.0.5 fail — stale
generate → fill → delete scaffolding pass

…proval

Replaces the hybrid model (CI builds unsigned, you sign on your laptop, then
hand-juggle six `gh` commands) with three chained workflows. The only manual
steps left are the two that need judgement: writing the prose, and deciding to
ship.

  prepare-release.yml  →  you edit the PR  →  tag-on-merge.yml  →  release.yml
  (drafts the notes)      (the prose)         (pushes the tag)     (builds, signs,
                                                                    notarizes, waits)

**Notes.** `draft-release-notes.mjs` already filled in every fact and left TODO
markers where judgement is needed — its header argues that a changelog generated
from commit subjects is why most release notes go unread. So the workflow opens a
PR with that draft rather than committing it, and `tag-on-merge.yml` refuses to
tag while a TODO or the scaffolding block survives, or when the first line doesn't
name the version being tagged. Automation cannot ship scaffolding, and it cannot
ship last release's notes either.

**Signing.** `notarize.sh` already took APPLE_ID + TEAM_ID + APP_SPECIFIC_PASSWORD
as the CI alternative to a local keychain profile, so no build script changed. The
workflow imports the Developer ID cert into a temporary keychain in RUNNER_TEMP,
runs the existing `make-dmg.sh`, verifies with codesign + stapler + spctl, and
deletes the keychain in an `always()` step. Two details that are load-bearing:
`set-key-partition-list` (without it codesign blocks on a GUI prompt nobody can
click and the job hangs to timeout) and a 6h keychain lock timeout (the 5-minute
default re-locks mid-notarization and the next codesign fails with a misleading
"user interaction is not allowed").

Also `fetch-depth: 0` on the build checkout: `build-macos.sh` stamps the version
from `git describe --tags`, and its failure mode is silent — it warns and ships a
build whose About box reads 1.126.0, the Code-OSS base.

**The gate.** Publishing is deploying: the Squirrel updater installs a published
release on every existing install at its next check, with no staged rollout, and
the rollback pin cannot un-update anyone who already took it. So the publish job
sits in a `release` Environment with required reviewers. Everything before it is
reversible — a branch, a tag, a draft. That step is not. After publishing it polls
the update feed and warns (not fails — the release is already correct) if the feed
hasn't picked it up.

The draft job also refuses to publish unless all four assets are present. Losing
only the x64 job would otherwise strand every Intel user silently, since the feed
serves per-arch.

RELEASING.md §7 rewritten: the six secrets, the environment setup (called out
hard — GitHub creates a missing environment with NO protection rules, so skipping
it makes the gate decorative), the new flow, and the trade-off the old §7 named
when it described this as the road not taken: the signing identity now lives in
the cloud. Revocation path and blast-radius limits documented, along with how to
go back.

Verified: actionlint (which shellchecks every run: block) clean on all three.
The notes gate was extracted from the workflow and exercised — real v1.0.4 notes
pass; a real generated draft fails on TODOs and scaffolding; TODOs-removed-but-
scaffolding-left still fails; v1.0.4 notes presented as v1.0.5 fail as stale; and
generate → fill → delete scaffolding passes.
Copilot AI lite review requested due to automatic review settings August 14, 2026 17:17

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Automates the macOS release process end-to-end in GitHub Actions (draft notes → human edits PR → tag on merge → build/sign/notarize → publish behind an Environment approval gate), replacing the previous hybrid “CI builds unsigned, local signing” flow.

Changes:

  • Adds a 3-step chained release pipeline via prepare-release.yml, tag-on-merge.yml, and updated release.yml.
  • Moves signing/notarization into CI using a temporary keychain and adds an explicit publish approval gate via the release Environment.
  • Updates docs/RELEASING.md to document the new workflow, secrets, and operational guardrails.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.

File Description
docs/RELEASING.md Replaces the hybrid CI release runbook with the new automated, gated release process and required setup.
.github/workflows/prepare-release.yml New manual workflow to draft RELEASE-NOTES.md and open a release/vX.Y.Z PR.
.github/workflows/tag-on-merge.yml New workflow to validate finished notes and push the vX.Y.Z tag on release PR merge.
.github/workflows/release.yml Updates release workflow to sign/notarize in CI, draft a release with all assets, then publish behind Environment approval.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread .github/workflows/tag-on-merge.yml Outdated
- uses: actions/checkout@v7
with:
# The merge commit on the base branch, not the PR head — that is the state being released.
ref: ${{ github.event.pull_request.base.ref }}
Comment thread .github/workflows/tag-on-merge.yml Outdated
Comment on lines +51 to +55
if grep -qF 'TODO' RELEASE-NOTES.md; then
echo "::error file=RELEASE-NOTES.md::Unfilled TODO markers remain:"
grep -nF 'TODO' RELEASE-NOTES.md | sed 's/^/ /'
FAILED=1
fi
Comment thread .github/workflows/release.yml Outdated
# one is there to click, and the job hangs until it times out rather than failing.
security set-key-partition-list -S apple-tool:,apple:,codesign: -s -k "$KEYCHAIN_PASSWORD" "$KEYCHAIN" >/dev/null
# Put it on the search list so codesign finds the identity by name.
security list-keychain -d user -s "$KEYCHAIN" login.keychain-db
Comment on lines +39 to +49
steps:
- uses: actions/checkout@v7
with:
# draft-release-notes.mjs walks `prevTag..HEAD` and reads `git tag --list`, so a shallow
# clone would silently produce an empty or wrong range — the class of quiet mistake this
# whole pipeline exists to remove.
fetch-depth: 0

- uses: actions/setup-node@v7
with:
node-version: "24"
**Tag the merge commit, by SHA.** `tag-on-merge.yml` checked out `base.ref`, which
resolves to whatever develop points at when the job starts — so anything merged in
the window after the release PR got swept into the tag. Since the build stamps its
version from `git describe --tags`, that ships code the release notes do not
describe, silently. Now checks out `pull_request.merge_commit_sha`.

**Narrow the TODO gate to the generator's marker.** It grepped for the bare word
`TODO`, so prose legitimately containing it — describing a known gap, quoting a
code comment — would block a finished release with no fix available except
rewording. It now matches `<!-- TODO`, which is the question the gate is actually
asking: did you fill in the placeholders draft-release-notes.mjs left?

**Refuse to prepare a release off develop.** The Run-workflow dropdown accepts any
ref, and the PR targets whatever you picked — but tag-on-merge.yml only listens for
PRs into develop, so a release PR opened elsewhere merges cleanly and then never
tags. A dead end with no error. Fails fast now, and RELEASING.md §7 says so.

**`list-keychains`, plural.** The review said the singular form "isn't a valid
subcommand" and that signing "will fail before codesign runs" — that part is not
right: `security` accepts unambiguous subcommand prefixes, and `security
list-keychain -d user` exits 0 today. Changed anyway, because prefix-matching is
not a name: one more Apple subcommand starting `list-keychain…` and it breaks.

Gate re-verified, six cases, harness fixed (the previous run generated drafts from
outside the repo, so the generator correctly refused and wrote an empty file —
which failed the first-line check for the wrong reason):

  real v1.0.4 notes @ v1.0.4        pass
  same notes claimed as v1.0.5      fail — stale
  raw generated draft               fail — TODOs + scaffolding
  filled + scaffolding removed      pass
  prose containing the word TODO    pass   ← was blocked before this change
  a real <!-- TODO --> marker       fail

actionlint clean.
@ndemianc

Copy link
Copy Markdown
Contributor Author

All four addressed in 4e4175b. Three were right; one had the right suggestion for the wrong reason.

Tag the merge commit — correct, and the most consequential of the four. base.ref resolves to whatever develop points at when the job starts, so anything merged in the window after the release PR got swept into the tag. Because the build stamps its version from git describe --tags, that ships code the notes don't describe — silently, and only discoverable afterwards. Now ref: ${{ github.event.pull_request.merge_commit_sha }}.

Narrow the TODO gate — correct. Matching the bare word would block a finished release whose prose legitimately says "TODO" (describing a known gap, quoting a code comment), with no fix available except rewording. Now keyed on <!-- TODO, which is the question the gate is actually asking.

Fail fast off develop — correct, and a genuinely nasty failure mode: the PR merges cleanly and then simply never tags. No error, no artifact, just a release that never happens. Now refuses up front, and RELEASING.md §7 shows Use workflow from: develop in the runbook.

list-keychains — changed, but the stated reason isn't right. security accepts unambiguous subcommand prefixes, so the singular form works today:

$ security list-keychain -d user
    "/Users/…/Library/Keychains/login.keychain-db"
$ echo $?
0

So signing would not have failed. Worth fixing regardless: prefix-matching isn't a name, and one more Apple subcommand starting list-keychain… turns it ambiguous. The plural is what security help documents.

Gate re-verified

Six cases, run against the logic extracted from the shipped workflow:

Input Expected Result
Real finished v1.0.4 notes @ v1.0.4 pass
Same notes claimed as v1.0.5 fail — stale
Raw generated draft fail — TODOs + scaffolding
Filled + scaffolding removed pass
Prose containing the word "TODO" pass (blocked before this change)
A real <!-- TODO --> marker fail

One note on that run: my first attempt reported false failures because the harness invoked draft-release-notes.mjs from outside the repo, where it correctly refuses and writes nothing — so the empty file failed the first-line check for the wrong reason. Harness fixed; the table above is from the corrected run.

actionlint clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants