fix(ci): gate every extension's tests, not just levelcode-ai - #25
Merged
Conversation
The pre-build test job did `cd extensions/levelcode-ai` before globbing, so extensions/levelcode-updater/test/*.test.js never ran in CI — including the regression test added in the S3 fix, which is the only thing standing between a feed change and the updater's Download button handing users a raw .app.zip. It protected the release it shipped in and nothing after that. The step now globs extensions/*/test/*.test.js, so a new suite is gated the moment it is added and there is no list here to drift out of sync. Requires in these tests are file-relative, so running from the repo root needs no cd. Also guards the vacuous pass: a zero-match glob previously reported success while gating nothing, which is the same class of bug as the one being fixed. It now fails with a ::error:: annotation. Verified locally under `bash -e` (the Actions default shell): - real run: 14 test files pass, up from 13 - zero-match glob: exits 1 with the annotation - a deliberately failing suite: aborts the job, exit 1 - release.yml still parses; all 3 jobs intact Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR strengthens the release workflow’s pre-build gate by ensuring all extension unit tests run before spending macOS build minutes, preventing silent gaps where new extension test suites are not executed in CI.
Changes:
- Run
nodetests acrossextensions/*/test/*.test.jsinstead of onlyextensions/levelcode-ai. - Add a “vacuous pass” guard: fail the job if the glob matches zero test files, with a
::error::annotation.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| # ran here, including the one guarding the updater's Download button against serving a raw | ||
| # .app.zip. Globbing every extension means a new suite is gated the moment it is added, with no | ||
| # list here to keep in sync. Requires are file-relative, so running from the repo root is fine. | ||
| run: | |
Addresses the PR #25 review: the step uses `shopt`, a bash builtin, but relied on the runner default rather than declaring the shell. True today on ubuntu-latest; pwsh on a Windows runner, where the step would break if the job were ever copied. Worth noting the fix is not purely defensive — `shell: bash` is not the same as the default. It runs `bash --noprofile --norc -eo pipefail` rather than `bash -e`, so the gate also stops inheriting profile files and gains pipefail. Both are improvements here, and neither changes this step's behaviour (it has no pipes). Updated the inline comment that credited "the Actions default" for the abort, since `-e` now comes from the pinned shell. Re-verified under the exact flags `shell: bash` invokes (`bash --noprofile --norc -eo pipefail`): - real run: 14 test files pass, exit 0 - zero-match glob: exit 1 with the ::error:: annotation - a deliberately failing suite: aborts, exit 1 - release.yml parses; test/build/draft-release intact; shell resolves to bash Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The gap
The pre-build test job did this:
The
cdmeantextensions/levelcode-updater/test/*.test.jsnever ran in CI — including the regression test added in the auto-update S3 fix, the one thing standing between a feed change and the Download button handing users a raw.app.zip. It protected the release it shipped in and nothing after that.The fix
Glob every extension instead, so a new suite is gated the moment it's added and there's no list here to drift out of sync. Requires in these tests are file-relative, so running from the repo root needs no
cd.It also guards the vacuous pass: a zero-match glob previously reported success while gating nothing — the same class of bug being fixed. It now fails with a
::error::annotation rather than going quietly green.Verification
Run locally under
bash -e, the Actions default shell:release.ymltest,build,draft-releaseintactNote
release.ymlis the only workflow in the repo, so this gate runs on tag push. It does not run on PRs — worth considering separately, but out of scope here.🤖 Generated with Claude Code