Skip to content

ci: make the go.mod/go.sum and benchmark checks able to fail - #921

Merged
general-kroll-4-life merged 1 commit into
stackql:mainfrom
tomersein:ci/tidy-and-benchmark-checks
Nov 15, 2026
Merged

general-kroll-4-life merged 1 commit into
stackql:mainfrom
tomersein:ci/tidy-and-benchmark-checks

Conversation

@tomersein

Copy link
Copy Markdown
Contributor

Description

Two checks in .github/workflows/build.yml could never fail, as described in #920.

  • go.mod/go.sum consistency (linuxbuild). The old step copied go.sum to go.sum.bkp and diffed it against that copy, so it always passed. It now runs go mod tidy -diff (Go 1.23+). The command fails and prints the diff when go.mod or go.sum isn't tidy, without modifying either file.
  • Benchmarks (linuxbuild and linuxarmbuild). go test -bench ... | tee ${outFile} was followed by if [ "$?" = "0" ]. Actions runs run: blocks as bash -e {0} without pipefail, so $? was tee's exit status and a failing benchmark (b.Fatalf) passed the step. Both steps now set -o pipefail and test the pipeline in the if directly. The rest of each step, including the benchstat threshold comparison, is unchanged.

Type of change

  • Bug fix (non-breaking change to fix a bug).
  • Feature (non-breaking change to add functionality).
  • Breaking change.
  • Other (eg: documentation change). Please explain.

Issues referenced.

Evidence

The new tidy check passes on the current tree. go mod tidy -diff on main (5c8c614) with Go 1.27.3 exits 0 with an empty diff, so this PR doesn't fail on today's go.mod/go.sum.

The new checks catch what the old ones missed. Each step's old and new script was run under bash -e, as Actions runs it, against a throwaway Go module:

Case Old step New step
go.mod requires a module nothing imports, go.sum empty exit 0, "go.sum check passed" exit 1, prints the go.mod/go.sum diff
tidy module exit 0 exit 0
benchmark calls b.Fatalf exit 0, although the log shows --- FAIL: BenchmarkThing exit 1, "Benchmarking run failed"
passing benchmark exit 0 exit 0

The benchmark step is identical in both jobs, so the same change covers amd64 and arm64. pipefail also makes a failing benchstat ... | tee fail the step, which seems right, since the threshold comparison can't run without it. The comparisons=$(... | grep ...) pipeline already ended in grep, so its behaviour doesn't change.

Checklist:

  • A full round of testing has been completed, and there are no test failures as a result of these changes. (This PR's CI is the test: only workflow steps change.)
  • The changes are covered with functional and/or integration robot testing. (n/a, CI-only change)
  • The changes work on all supported platforms. (Both Linux benchmark jobs; the go.sum check runs only on Linux.)
  • Unit tests pass locally, as per the developer guide. (No Go code changes.)
  • Robot tests pass locally, as per the developer guide. (No Go code changes.)
  • Linter passes locally, as per the developer guide. (No Go code changes.)

The go.sum coherence check copied go.sum and diffed it against the copy, so
it always passed. It now runs `go mod tidy -diff`, which fails and prints
the diff when go.mod or go.sum isn't tidy, without changing either.

The Benchmark steps in linuxbuild and linuxarmbuild piped `go test -bench`
into tee and then checked $?. Actions runs `run:` as `bash -e` without
pipefail, so $? was tee's and a failing benchmark passed the step. Both now
set pipefail and test the pipeline directly.

Refs stackql#920

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues were identified for the CI changes intended to catch dependency inconsistencies and benchmark failures.

No open issues 🎉

What changed in this PR

This PR addresses two ineffective CI checks from #920, allowing dependency inconsistencies and benchmark failures to fail the build.

Changes:

  • Replaces the self-comparison of go.sum with go mod tidy -diff, checking both module files without modifying them.
  • Enables pipefail and directly checks benchmark pipelines in both Linux jobs so tee no longer hides failures.
File Description
.github/​workflows/​build.yml Makes module tidiness and Linux amd64/arm64 benchmark checks enforce failures.

Review effort: Balanced

@general-kroll-4-life

Copy link
Copy Markdown
Contributor

@tomersein thank you for your contribution!

@general-kroll-4-life
general-kroll-4-life merged commit ae159ee into stackql:main Nov 15, 2026
32 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants