Skip to content

ROX-33327: Image fetch request coalescing,improved image scan caching#19523

Merged
clickboo merged 2 commits intomasterfrom
boo-adm-ctrl-coalesce
Mar 25, 2026
Merged

ROX-33327: Image fetch request coalescing,improved image scan caching#19523
clickboo merged 2 commits intomasterfrom
boo-adm-ctrl-coalesce

Conversation

@clickboo
Copy link
Copy Markdown
Contributor

@clickboo clickboo commented Mar 20, 2026 •
edited
Loading

Description

This is the third in the series of planned optimizations for admission controller.

The goal of this PR is to optimize image fetch calls to sensor/central and ultimately registry scan requests, thus improving the throughput of the policy eval webhook, and its burst resilience and timeout avoidance, at scale.

Two optimizations have been added:
(1) request coalescing (singleflight) -- concurrent fetches for the same image share a single in-flight call;
(2) two-level caching -- the existing digest-keyed cache plus a new image-name-to-cache-key map so tag-only references can hit cached results without a digest. The name-to-key map can be disabled via ROX_ADMISSION_CONTROL_IMAGE_NAME_CACHE_ENABLED=false for environments where image tags are reused for different content (mutable tags).

Note to reviewers:

  1. Test rigs are available in a draft PR ROX-33343: Automated suite for adm cntrl bench testing & profiling #19427, and are generated by our 🤖 friends, with a lot of supervision and iteration.
  2. I do not intend to merge them just yet, so don't review them. Sharing to give you context on how the test results were achieved.
  3. I will continue to enhance the rig for future optimizations that are planned and merge the draft PR at a later point. Will solicit reviews then.

User-facing documentation

Testing and quality

  • the change is production ready: the change is GA, or otherwise the functionality is gated by a feature flag
  • CI results are inspected

Automated testing

  • added unit tests
  • added e2e tests
  • added regression tests
  • added compatibility tests
  • modified existing tests

How I validated my change

  1. Synthetic benchmarks available in ROX-33343: Automated suite for adm cntrl bench testing & profiling #19427
  2. Burst testing for scale and correctness (regression testing) on live Openshift 4.x infra cluster
  3. Existing groovy tests in AdmissionControllerTest.groovy for correctness and regression testing.

Results:
https://docs.google.com/document/d/13B_1X5MPcrMB8qH076S-1QS_21CKhYEjh-YyH-vfbQM/edit?tab=t.0#heading=h.y9iqi8q2pwjt

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 20, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

sourcery-ai[bot]

This comment was marked as outdated.

Sign in to view

@rhacs-bot
Copy link
Copy Markdown
Contributor

rhacs-bot commented Mar 20, 2026 •
edited
Loading

Images are ready for the commit at 2728dc7.

To use with deploy scripts, first export MAIN_IMAGE_TAG=4.11.x-405-g2728dc72ba.

@clickboo clickboo force-pushed the boo-adm-ctrl-coalesce branch from f30251a to 864b775 Compare March 20, 2026 16:07
@codecov
Copy link
Copy Markdown

codecov bot commented Mar 20, 2026 •
edited
Loading

Codecov Report

❌ Patch coverage is 30.70175% with 79 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.25%. Comparing base (bff9977) to head (2728dc7).
⚠️ Report is 33 commits behind head on master.

Files with missing lines Patch % Lines
sensor/admission-control/manager/images.go 40.69% 50 Missing and 1 partial ⚠️
sensor/admission-control/manager/manager_impl.go 0.00% 13 Missing ⚠️
...nsor/admission-control/manager/evaluate_runtime.go 0.00% 8 Missing ⚠️
...r/admission-control/manager/evaluate_deploytime.go 0.00% 5 Missing ⚠️
sensor/admission-control/manager/manager.go 0.00% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##           master   #19523      +/-   ##
==========================================
- Coverage   49.25%   49.25%   -0.01%     
==========================================
  Files        2727     2727              
  Lines      205788   205833      +45     
==========================================
+ Hits       101362   101381      +19     
- Misses      96890    96916      +26     
  Partials     7536     7536
Flag Coverage Δ
go-unit-tests 49.25% <30.70%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@clickboo clickboo force-pushed the boo-adm-ctrl-coalesce branch 4 times, most recently from 89e65c5 to 7825c8e Compare March 21, 2026 17:41
@stackrox stackrox deleted a comment from openshift-ci bot Mar 22, 2026
@clickboo clickboo changed the title ROX-33327: WIP - Coalesce image scan requests originating from admiss… ROX-33327: Image scan request coalescing, two level image scan caching in admission controller Mar 22, 2026
@clickboo clickboo changed the title ROX-33327: Image scan request coalescing, two level image scan caching in admission controller ROX-33327: Image scan request coalescing, two level image scan caching Mar 22, 2026
@clickboo clickboo changed the title ROX-33327: Image scan request coalescing, two level image scan caching ROX-33327: Image scan request coalescing, improved image scan caching Mar 22, 2026
@clickboo clickboo requested a review from charmik-redhat March 22, 2026 05:59
@clickboo clickboo changed the title ROX-33327: Image scan request coalescing, improved image scan caching ROX-33327: Image fetch request coalescing, improved image scan caching Mar 22, 2026
@clickboo clickboo changed the title ROX-33327: Image fetch request coalescing, improved image scan caching ROX-33327: Image fetch request coalescing, improved image scan caching Mar 22, 2026
@clickboo clickboo changed the title ROX-33327: Image fetch request coalescing, improved image scan caching ROX-33327: Image fetch request coalescing,improved image scan caching Mar 22, 2026
@clickboo clickboo force-pushed the boo-adm-ctrl-coalesce branch from 7825c8e to ee70d2d Compare March 22, 2026 13:21
@clickboo
Copy link
Copy Markdown
Contributor Author

clickboo commented Mar 22, 2026

/test all

@clickboo clickboo marked this pull request as ready for review March 23, 2026 13:37
@clickboo clickboo requested a review from a team as a code owner March 23, 2026 13:37
sourcery-ai[bot]

This comment was marked as off-topic.

Sign in to view

@clickboo
Copy link
Copy Markdown
Contributor Author

clickboo commented Mar 23, 2026

/test ocp-4-21-operator-e2e-tests

@clickboo
Copy link
Copy Markdown
Contributor Author

clickboo commented Mar 23, 2026

/test ocp-4-20-nongroovy-e2e-tests

dashrews78
dashrews78 reviewed Mar 23, 2026
View reviewed changes
dashrews78
dashrews78 reviewed Mar 23, 2026
View reviewed changes
charmik-redhat
charmik-redhat reviewed Mar 23, 2026
View reviewed changes
@clickboo
Copy link
Copy Markdown
Contributor Author

clickboo commented Mar 23, 2026

/test ocp-4-12-nongroovy-e2e-tests

@clickboo clickboo requested a review from a team as a code owner March 23, 2026 20:00
dashrews78
dashrews78 reviewed Mar 24, 2026
View reviewed changes
charmik-redhat
charmik-redhat approved these changes Mar 24, 2026
View reviewed changes
Copy link
Copy Markdown
Contributor

@charmik-redhat charmik-redhat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to go after @dashrews78's comment about context timeout is addressed.

@clickboo clickboo merged commit 4f8098a into master Mar 25, 2026
110 of 112 checks passed
@clickboo clickboo deleted the boo-adm-ctrl-coalesce branch March 25, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@sourcery-ai sourcery-ai[bot] sourcery-ai[bot] left review comments

@dashrews78 dashrews78 dashrews78 left review comments

@charmik-redhat charmik-redhat charmik-redhat approved these changes

Assignees

No one assigned

Labels

ai-review area/admission-controller area/sensor

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@clickboo @rhacs-bot @dashrews78 @charmik-redhat