Repository navigation
Description
I maintain zanadir, which reads a repository's CI and reports which kinds of checks it runs and whether each one can actually fail the build. I ran it on stackql and wanted to share what it found, with a suggestion to run it in CI.
There's a lot here already: golangci-lint with gosec and about fifty linters, actionlint, CodeQL, robot suites on every platform including WSL and Docker, and a benchmark threshold gate. A few checks can't fail the way they look like they should:
- The go.sum coherence check compares
go.sumwith a copy of itself. Inbuild.yml(Linux Build, "Check go sum coherence") the step runscp go.sum go.sum.bkpand thendiff go.sum go.sum.bkpwith nothing in between, so it always prints "go.sum check passed".lint.ymlrunsgo mod tidybefore golangci-lint but never checks whether that changed anything, so a PR with a stalego.modorgo.sumpasses both. Replacing the lint step withgo mod tidy -difffails when the committed files aren't tidy and prints the diff:- name: Check go.mod and go.sum are tidy run: go mod tidy -diff
- A failing benchmark doesn't fail the Benchmark step. Both Benchmark steps (Linux and Linux arm64) run
go test -bench ... ./... | tee ${outFile}and then test$?. Without ashell:key the step runs asbash -e {0}(see the run logs), so$?is tee's and the "Benchmarking run failed" branch can't be reached. Benchmarks only run here, so a driver benchmark that hitsb.Fatalfjust drops out of the benchstat comparison and the step passes.set -o pipefailat the top of both steps, orshell: bash, fixes it. - The release integration test steps can never run. Five "Run robot integration tests" steps are gated on
startsWith(env.STATE_SOURCE_TAG, 'build-release'). The "Git Ref Parse" steps writeSOURCE_TAGto$GITHUB_STATE, which only feeds an action's own post step, soSTATE_SOURCE_TAGis never set (no step environment in the logs has it).startsWith(github.ref_name, 'build-release'), which the traffic-light steps already use, would make them reachable. If releases now come only frommainbuilds, as the release doc describes, they could go instead. - The workflow-level
GH_ACCESS_TOKENreaches every step.build.ymlsets it fromACTIONS_PRIVATE_PACKAGE_SECRETin the top-levelenv, so it is passed to tag-referenced third-party actions such asjiro4989/build-deb-action@v3(its container starts with-e GH_ACCESS_TOKEN) andsnok/install-poetry@v1, and theinsteadOfrewrite leaves it in~/.gitconfigfor the test steps. Everystackqlandstackql-labsmodule ingo.modis public, andlint.ymldownloads them all without a token, so the secret, the rewrite andGOPRIVATEcan probably go. Pinning third-party actions by SHA, kept current by agithub-actionsentry independabot.yml, would also help. - Nothing fails on known vulnerabilities. Dependabot opens update PRs, but no job gates on advisories.
govulncheck ./...in both Go modules reports the ones your code reaches, including in the standard library of the Go toolchain in use, the kind of problem [SECURITY] Published release v0.12.778 is built with go1.25.3 and still trusts the compromised eMudhra intermediates; cut a Go 1.26.10 release #857 tracked by hand. - Smaller:
rhysd/actionlint:1.6.23is from early 2023, which is why.github/actionlint.yamlhas to listubuntu-24.04and the arm runners as self-hosted; recent releases know them. The .NET SDK wrapper targetsnet8.0;net9.0, and both went out of support on 10 November 2026. Coverage (go test -coverprofile) and a secret scan such as Gitleaks on PRs are missing.
To run zanadir itself, add a job to lint.yml:
zanadir:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
with:
persist-credentials: false
- uses: MustacheCase/zanadir-action@059d8ee8b790fb5a3d70716ca5cb79fda35c67cf # v1.41
with:
dir: .
excluded-categories: Performance Testing,Supply ChainIt reports without failing the build; fail-on-weakened: true makes it gate. Today it scores 4 of 11 categories: linting (golangci-lint), unit tests, SAST (CodeQL) and workflow security (actionlint). Performance testing and supply chain are excluded because zanadir doesn't yet recognise the go test -bench and benchstat gate or npm provenance, which is a gap on our side.
Happy to send a PR for any of these, starting with the go.sum and benchmark checks.
Activity
jeffreyaven commentedon Nov 14, 2026
Thanks @tomersein for the detailed report and the offer to help. We'd welcome a focused PR fixing the
go.mod/go.sumconsistency check and ensuring benchmark failures fail the build on both Linux amd64 and arm64.Please keep the existing tests passing and include verification that the updated checks catch the failures described. I'll review it promptly and make sure you're credited for the contribution.
ci: make the go.mod/go.sum and benchmark checks able to fail (#921)