fix(ci): re-enable additional jobs gha cache saves by davdhacs · Pull Request #19116 · stackrox/stackrox

davdhacs · 2026-02-19T17:13:27Z

Description

PR #19069 removed all GHA cache save: false settings, but check-generated-files modifies the working tree which breaks hashFiles('**/go.sum') in the GOMODCACHE post-step, causing cache save failures. Reverted in #19108 (chat discussion in https://redhat-internal.slack.com/archives/CELUQKESC/p1771501400340929).

Save hash of go.sum files at the start and re-use it for saving. This avoid re-reading the workspace filesystem after the job completes and there may be part of the filesystem that are not readable because of permissions.
Re-enable cache saves for the disabled-save jobs (unit-tests, style-check, scanner-db-integration-tests, and check-generated-files).
Keep disabled: the check-generated-files save of the GOMODCACHE and GOCACHE. check-generated-files still prevents access to the go.sum files because of file system permissions changes by the job under operator/ (to be fixed in follow-up PRs)

Partially generated by AI.

Testing and quality

CI results are inspected

How I validated my change

Debug re-run (EACCES failure):
https://github.com/stackrox/stackrox/actions/runs/22172586856/job/64730778461
Force-save test:
https://github.com/stackrox/stackrox/actions/runs/22374447570/job/64761298810?pr=19116
Run with restored cache:
https://github.com/stackrox/stackrox/actions/runs/22375205467/job/64763723220?pr=19116
Normal run (reverted, restore-only):
https://github.com/stackrox/stackrox/actions/runs/22379932019/job/64778159539?pr=19116

…files PR #19069 removed all save: false, but check-generated-files modifies the working tree (regenerates protos, removes mocks) which breaks hashFiles('**/go.sum') in the GOMODCACHE post-step, causing cache save failures. Reverted in #19108. Re-enable saves for all other jobs (unit-tests, style-check, scanner-db-integration-tests) while keeping save: false only for check-generated-files where the hashFiles failure occurs. Partially generated by AI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

openshift-ci · 2026-02-19T17:13:31Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

sourcery-ai

Hey - I've reviewed your changes and they look great!

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

rhacs-bot · 2026-02-19T17:40:14Z

Images are ready for the commit at e163405.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.11.x-329-ge163405092.

codecov · 2026-02-19T17:56:22Z

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 49.64%. Comparing base (82657e5) to head (b5418bf).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master   #19116   +/-   ##
=======================================
  Coverage   49.64%   49.64%           
=======================================
  Files        2698     2698           
  Lines      203075   203075           
=======================================
+ Hits       100817   100824    +7     
+ Misses      94737    94730    -7     
  Partials     7521     7521

Flag	Coverage Δ
go-unit-tests	`49.64% <ø> (+<0.01%)`	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

sourcery-ai

Hey - I've reviewed your changes and they look great!

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

.github/workflows/style.yaml

porridge · 2026-02-24T08:15:30Z

Also if savedefaults totrue, maybe remove the true` lines?

… jobs `kubebuilder edit --plugins=helm/v2-alpha` (run by `make -C operator/ chart` during check-generated-files) scaffolds `operator/.github/` with mode 0700. The `actions/cache@v5` post step re-evaluates `hashFiles('**/go.sum')` on the host as the runner user, who cannot traverse this root-owned 0700 directory, causing EACCES. Compute the hash once upfront in a step output and reference that in cache keys so the post step uses the resolved string instead of re-traversing the filesystem. This also removes the `save: false` workaround from check-generated-files, re-enabling GOMODCACHE saves for that job. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

davdhacs · 2026-02-24T19:49:38Z

Also if savedefaults totrue, maybe remove the true` lines?

Good call. I'll remove them.

TEMPORARY: remove the default-branch-only gate on cache saves so the actions/cache@v5 post step actually runs on this PR. This verifies the pre-computed hashFiles fix works when caches are saved (not just restored). Revert this commit before merging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

This reverts commit 44056d4.

…ication Use go-mod-v1-pr19116- and go-build-v1-pr19116- prefixes so we can confirm caches are freshly saved (seeding run) and then restored (follow-up run). Also force save on all events. Revert before merging. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

porridge · 2026-02-26T06:22:56Z

The plan sounds good, but I'm busy with a release blocker at the moment.

kubebuilder v4.13 has the perms fix

davdhacs · 2026-03-02T20:08:52Z

Filed kubernetes-sigs/kubebuilder#5502 regarding the permissions. In the meantime I'll add a fixup chmod after the kubebuilder call into the chart target.

🥳 The fix in kubebuilder went through already and is on v4.13.0: kubernetes-sigs/kubebuilder@bb95ce2

Temporarily remove the master-only save restriction so all jobs save their caches on this PR run. This validates that saves work correctly for all jobs (especially check-generated-files with save: false). TO REVERT after confirming cache save+restore works. Partially generated by AI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

davdhacs · 2026-03-03T01:41:31Z

fails. I didn't look into if we have kubebuilder pinned (I think it was just "v4"), but it looks like we didn't get the updated version yet.

porridge · 2026-03-03T11:18:56Z

fails. I didn't look into if we have kubebuilder pinned (I think it was just "v4"), but it looks like we didn't get the updated version yet.

😅 #19264

davdhacs · 2026-03-03T15:59:50Z

I'll wait on this for the kubebuilder upgrade to be dependabot-pr'd and merged. Then this will turn green and we'll be saving new cache entries again :)

davdhacs · 2026-03-04T18:48:41Z

🥳 kubebuilder updated to 4.13.0 in #19268

davdhacs · 2026-03-05T16:58:59Z

/retest

davdhacs · 2026-03-05T17:00:53Z

/retest

davdhacs · 2026-03-09T18:31:51Z

/retest-required

Revert the force-save test. CI run on 699f101 confirmed all 67 jobs passed with cache saves enabled, including check-generated-files (kubebuilder fix resolved the go.sum modification issue). Retains the github.sha TAG for per-commit build cache keys. Partially generated by AI. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

davdhacs · 2026-03-11T14:59:09Z

Removed test-setup (always save) because the check-generated run saved correctly now with the updated kubebuilder (kubebuilder does not now set the file permissions that had blocked the hashFiles previously).

openshift-ci · 2026-03-11T22:34:57Z

@davdhacs: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/ocp-4-12-nongroovy-e2e-tests	`e163405`	link	false	`/test ocp-4-12-nongroovy-e2e-tests`
ci/prow/ocp-4-20-nongroovy-e2e-tests	`e163405`	link	false	`/test ocp-4-20-nongroovy-e2e-tests`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

davdhacs added the ci-save-cache Enable Go cache saves on this PR branch for testing label Feb 19, 2026

openshift-ci bot added the do-not-merge/work-in-progress label Feb 19, 2026

github-actions bot added area/ci ai-review labels Feb 19, 2026

sourcery-ai bot reviewed Feb 19, 2026

View reviewed changes

davdhacs added 2 commits February 19, 2026 11:32

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

5bdb621

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

ae04fac

davdhacs removed the ci-save-cache Enable Go cache saves on this PR branch for testing label Feb 19, 2026

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

8ba5dab

davdhacs marked this pull request as ready for review February 23, 2026 22:09

davdhacs requested a review from a team as a code owner February 23, 2026 22:09

openshift-ci bot removed the do-not-merge/work-in-progress label Feb 23, 2026

davdhacs requested review from dashrews78, janisz and porridge February 23, 2026 22:10

sourcery-ai bot reviewed Feb 23, 2026

View reviewed changes

davdhacs requested a review from tommartensen February 24, 2026 00:50

porridge reviewed Feb 24, 2026

View reviewed changes

.github/workflows/style.yaml Outdated Show resolved Hide resolved

davdhacs and others added 5 commits February 24, 2026 12:51

remove save:true (default)

c72922c

Revert "test(ci): force cache saves on PR to verify post-step fix"

863c65a

This reverts commit 44056d4.

empty commit

ac8d44d

davdhacs added 3 commits February 25, 2026 07:48

disable check-generated cache save

f1770a2

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

c8d5884

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

5b25ee9

enable cache on check-generated

e33f616

kubebuilder v4.13 has the perms fix

davdhacs added 2 commits March 3, 2026 07:49

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

3936e46

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

802b296

davdhacs changed the title ~~fix(ci): re-enable gha cache saves, except check-generated~~ fix(ci): re-enable additional jobs gha cache saves Mar 3, 2026

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

05d4148

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

8d6b404

janisz approved these changes Mar 6, 2026

View reviewed changes

davdhacs and others added 3 commits March 9, 2026 15:01

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

699f101

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

b5418bf

davdhacs requested a review from janisz March 11, 2026 14:59

Merge branch 'master' into davdhacs/gha-cache-save-all-fixed

e163405

davdhacs merged commit 6ac212c into master Mar 12, 2026
87 of 89 checks passed

davdhacs deleted the davdhacs/gha-cache-save-all-fixed branch March 12, 2026 16:48

davdhacs mentioned this pull request Mar 24, 2026

perf(ci): trim stale GOCACHE entries between runs #19425

Merged

Conversation

davdhacs commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing and quality

How I validated my change

Uh oh!

openshift-ci bot commented Feb 19, 2026

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

rhacs-bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Feb 19, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

sourcery-ai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

porridge commented Feb 24, 2026

Uh oh!

davdhacs commented Feb 24, 2026

Uh oh!

porridge commented Feb 26, 2026

Uh oh!

davdhacs commented Mar 2, 2026

Uh oh!

davdhacs commented Mar 3, 2026

Uh oh!

porridge commented Mar 3, 2026

Uh oh!

davdhacs commented Mar 3, 2026

Uh oh!

davdhacs commented Mar 4, 2026

Uh oh!

davdhacs commented Mar 5, 2026

Uh oh!

davdhacs commented Mar 5, 2026

Uh oh!

davdhacs commented Mar 9, 2026

Uh oh!

davdhacs commented Mar 11, 2026

Uh oh!

openshift-ci bot commented Mar 11, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

davdhacs commented Feb 19, 2026 •

edited

Loading

rhacs-bot commented Feb 19, 2026 •

edited

Loading

codecov bot commented Feb 19, 2026 •

edited

Loading