Skip to content

SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java: add timeouts#5539

Open
tomasz-tylenda-sonarsource wants to merge 2 commits intomasterfrom
chore/ttylenda/BUILD-XXXXX-optimizeRepoxBandwidth
Open

SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java: add timeouts#5539
tomasz-tylenda-sonarsource wants to merge 2 commits intomasterfrom
chore/ttylenda/BUILD-XXXXX-optimizeRepoxBandwidth

Conversation

@tomasz-tylenda-sonarsource
Copy link
Copy Markdown
Contributor

Summary

Automated CI optimization to reduce JFrog Artifactory (Repox) bandwidth usage.

Auto-generated by jfrog-optimizer plugin — requires human review before merge

Jira: BUILD-XXXXX
Anomaly detected: no-data — info
Current bandwidth: N/A GB/week (vs N/A GB/week average)
Week-over-week change: N/A
Estimated savings: ~3-5 builds/week avoided + reliability improvements

Applied Changes (safe, low-risk)

File Change Impact
6 workflows (build.yml, dogfood.yml, pr-cleanup.yml, PullRequestClosed.yml, PullRequestCreated.yml, RequestReview.yml) paths-ignore — Skip CI for documentation-only changes (.md, docs/, .github/CODEOWNERS, LICENSE) Avoids ~3-5 unnecessary builds/week
4 PR workflows (pr-cleanup.yml, PullRequestClosed.yml, PullRequestCreated.yml, RequestReview.yml) concurrency — Added cancel-in-progress to cancel redundant runs on new commits to same PR Reduces wasted compute resources
All 17 workflows timeout-minutes — Added timeouts to prevent indefinite hangs (60min for heavy builds, 30min for medium, 15min for light jobs) Improved reliability + faster failure detection

Recommended Changes (manual review needed)

1. Maven Cache Key Rotation

File: build.yml, dogfood.yml, unified-dogfooding.yml, release.yml, automated-release.yml

Issue: Cache keys embed commit hash, causing ~6 distinct keys per 5 runs with no cache reuse.

Suggestion:

- uses: actions/cache@v3
  with:
    path: ~/.m2/repository
    key: maven-${{ hashFiles('**/pom.xml') }}-${{ env.CACHE_MONTH }}
    restore-keys: maven-${{ hashFiles('**/pom.xml') }}-
  env:
    CACHE_MONTH: ${{ format('{0:yyyy-MM}', github.run_date) }}

Risk: Low (cache behavior is well-understood; may need coordination with composite actions in SonarSource/ci-github-actions)

Estimated Impact: ~30-50 GB/week


2. Volatile Matrix Gating

File: build.yml, automated-release.yml, PrepareNextIteration.yml

Issue: 3 workflows reference DEV/SNAPSHOT versions without nightly_only guard. 61 SNAPSHOT dependencies force re-resolution every build (~46 downloads per job).

Suggestion:

jobs:
  build:
    if: |
      github.event_name == 'push' ||
      github.event_name == 'schedule' ||
      contains(github.event.pull_request.labels.*.name, 'build-snapshot-deps')

Risk: Low-Medium (requires testing with nightly schedule; may need to add override label for on-demand full builds)

Estimated Impact: ~10-20 GB/week


3. JDK Caching

File: build.yml, dogfood.yml, unified-dogfooding.yml

Issue: 46 Repox JDK downloads per build job; JDK is not cached.

Suggestion:

- name: Cache JDK
  uses: actions/cache@v3
  with:
    path: ~/.java
    key: jdk-${{ matrix.java-version }}-${{ runner.os }}
    restore-keys: jdk-${{ matrix.java-version }}-

Risk: Low

Estimated Impact: ~5-10 GB/week


4. Workflow_run Sequencing

File: automated-release.yml

Issue: Runs in parallel with build.yml push trigger, missing cache warmup opportunity.

Suggestion: Switch to workflow_run trigger to wait for build cache warmup before starting release jobs.

on:
  workflow_run:
    workflows: ['build']
    types: [completed]
    branches: [master]

Risk: Low-Medium (requires testing; may introduce sequential delay but ensures cache hits)

Estimated Impact: ~5-10 GB/week

How to Verify

  1. Review the CI run on this PR — check cache hit/miss logs
  2. Merge this PR
  3. Monitor bandwidth in Repox Usage Dashboard
  4. Compare week-over-week after merge

Verification Checklist

  • CI passes on this PR
  • Cache hits visible in test/QA job logs
  • No increase in build times
  • Bandwidth decrease observed in Datadog (check 1 week post-merge)

References

Applied automated safe optimizations to reduce Repox bandwidth consumption.
See PR description for details of changes and additional recommendations.

Generated by jfrog-optimizer plugin
@tomasz-tylenda-sonarsource tomasz-tylenda-sonarsource added the automated-ci-optimization Automated CI optimization by jfrog-optimizer label Mar 27, 2026
@hashicorp-vault-sonar-prod hashicorp-vault-sonar-prod bot changed the title BUILD-XXXXX Optimize CI to reduce Repox bandwidth for sonar-java SONARJAVA-6216 BUILD-XXXXX Optimize CI to reduce Repox bandwidth for sonar-java Mar 27, 2026
@hashicorp-vault-sonar-prod
Copy link
Copy Markdown
Contributor

hashicorp-vault-sonar-prod bot commented Mar 27, 2026

SONARJAVA-6216

@tomasz-tylenda-sonarsource tomasz-tylenda-sonarsource changed the title SONARJAVA-6216 BUILD-XXXXX Optimize CI to reduce Repox bandwidth for sonar-java SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java Mar 27, 2026
@tomasz-tylenda-sonarsource
Copy link
Copy Markdown
Contributor Author

I manually undid paths-ignore - I don't fully understand its impact and feeel that it should be done in a separate PR. The remaining changes, timeouts and concurency groups look very safe, although I'm not sure if they are needed.

@tomasz-tylenda-sonarsource tomasz-tylenda-sonarsource marked this pull request as ready for review March 27, 2026 13:35
@tomasz-tylenda-sonarsource tomasz-tylenda-sonarsource changed the title SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java - add timeouts Mar 27, 2026
@tomasz-tylenda-sonarsource tomasz-tylenda-sonarsource changed the title SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java - add timeouts SONARJAVA-6216 Optimize CI to reduce Repox bandwidth for sonar-java: add timeouts Mar 27, 2026
@sonar-review-alpha
Copy link
Copy Markdown

sonar-review-alpha bot commented Mar 27, 2026

Summary

This PR applies two safe, operational improvements across GitHub Actions workflows:

  1. Timeout guards (60/30/15 min): Added timeout-minutes to all 17 workflows to prevent indefinite hangs and enable faster failure detection. Heavy builds use 60 min, medium use 30 min, light/admin tasks use 15 min.

  2. Concurrency cancellation: Added cancel-in-progress to 4 PR-event workflows (pull_request_created, pull_request_closed, request_review, pr_cleanup) to automatically cancel redundant runs when new commits are pushed.

Not included: The PR description mentions recommended changes (Maven cache rotation, SNAPSHOT gating, JDK caching, workflow_run sequencing), but those are deferred to future work. The paths-ignore optimization was also undone (see recent commit "Undo paths-ignore").

What reviewers should know

Scope is smaller than the description suggests — verify you're only reviewing timeouts + concurrency, not the larger optimization proposals mentioned in the recommendation section.

Where to verify:

  • Timeout changes are purely defensive (no behavioral change unless jobs hang; normal jobs should finish well before limits)
  • Concurrency changes only affect PR-event workflows; check one run on this PR to confirm cancel-in-progress: true takes effect

What reviewers should check:

  1. Timeout values are realistic for each job type (build jobs at 60m, lightweight tasks at 15m)
  2. Concurrency grouping logic correctly keys off PR number (prevents cross-PR cancellation)
  3. No "paths-ignore" reversion was missed — the "Undo paths-ignore" commit confirms it was intentionally removed

No cache behavior change, no build speedup expected — these are reliability/resource-waste improvements, not bandwidth reduction. The recommended optimizations in the description are future work.


  • Generate Walkthrough
  • Generate Diagram

🗣️ Give feedback

@sonarqube-next
Copy link
Copy Markdown

Copy link
Copy Markdown

@sonar-review-alpha sonar-review-alpha bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conclusion: One genuine bug to fix — the cancel-in-progress setting on RequestReview.yml can silently drop reviewer processing when multiple reviewers are added at once. The timeout additions across all workflows are straightforward and look correct.

🗣️ Give feedback

Comment on lines 5 to +8
types: ["review_requested"]

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: cancel-in-progress: true with a per-PR concurrency group is unsafe for a review_requested trigger.

GitHub fires one review_requested event per reviewer added. If two reviewers are requested in quick succession (or as a batch), both runs land in the same concurrency group (workflow-pr_number), so the second run cancels the first mid-flight. Whatever this workflow does for the first reviewer (Jira transition, notification, etc.) is silently abandoned.

cancel-in-progress is appropriate when only the latest run matters (e.g. a lint check on every push). For event-driven workflows where each event represents a distinct real-world action, it loses data.

Either drop cancel-in-progress: true, or scope the group to the specific reviewer so concurrent runs for different reviewers don't collide:

concurrency:
  group: ${{ github.workflow }}-${{ github.event.pull_request.number }}-${{ github.event.requested_reviewer.login || github.event.requested_team.slug }}
  cancel-in-progress: true
Suggested change
types: ["review_requested"]
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.ref }}
concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number }}-${{ github.event.requested_reviewer.login || github.event.requested_team.slug }}
cancel-in-progress: true
  • Mark as noise

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automated-ci-optimization Automated CI optimization by jfrog-optimizer

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant