ROX-33339: Sensor support for refresh image cache TTL message#19837
ROX-33339: Sensor support for refresh image cache TTL message#19837
Conversation
|
This change is part of the following stack: Change managed by git-spice. |
|
Skipping CI for Draft Pull Request. |
9fc3225 to
c15a13f
Compare
🚀 Build Images ReadyImages are ready for commit 39d15cc. To use with deploy scripts: export MAIN_IMAGE_TAG=4.11.x-601-g39d15cc177 |
2b11784 to
a5bd2d1
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #19837 +/- ##
==========================================
- Coverage 49.60% 49.59% -0.02%
==========================================
Files 2763 2766 +3
Lines 208331 208572 +241
==========================================
+ Hits 103338 103435 +97
- Misses 97327 97462 +135
- Partials 7666 7675 +9
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
6211fa0 to
8e833aa
Compare
a5bd2d1 to
de678cb
Compare
de678cb to
b1142f8
Compare
|
/test all |
b1142f8 to
f71c251
Compare
|
/test ocp-4-12-qa-e2e-tests |
…InvalidateImageCache
📝 WalkthroughWalkthroughThe changes introduce a new image cache TTL refresh feature for sensors by extracting Changes
Sequence Diagram(s)sequenceDiagram
participant Central
participant Pipeline as Event Pipeline
participant Reprocessor
participant Cache as Expiring Cache
Central->>Pipeline: RefreshImageCacheTTL(image_keys=[...])
activate Pipeline
Pipeline->>Pipeline: Accepts(msg) validates message type
Pipeline->>Pipeline: ProcessMessage routes to handler
Pipeline->>Reprocessor: ProcessRefreshImageCacheTTL(req)
activate Reprocessor
loop For each image_key in req
Reprocessor->>Reprocessor: cacheKeyFromImageKey(imageKey)
Reprocessor->>Cache: Touch(resolvedKey)
activate Cache
Cache->>Cache: Lock cache
Cache->>Cache: cleanNoLock() prune expired
Cache->>Cache: Lookup key exists?
alt Key exists
Cache->>Cache: Remove and re-add value
Cache-->>Reprocessor: true (TTL refreshed)
else Key not found
Cache-->>Reprocessor: false (key absent)
end
deactivate Cache
end
Reprocessor-->>Pipeline: error or nil
deactivate Reprocessor
Pipeline-->>Central: Acknowledge/respond
deactivate Pipeline
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
🧹 Nitpick comments (2)
sensor/common/reprocessor/handler_test.go (1)
32-43: This test doesn't currently prove the TTL was refreshed.Because
newTestCachehardcodes a real one-hour cache and the assertions run immediately afterProcessRefreshImageCacheTTL, this would still pass for a no-op. Consider using a controllable clock or a very short TTL and asserting that refreshed keys survive past the original expiry while untouched keys do not.Also applies to: 125-191
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sensor/common/reprocessor/handler_test.go` around lines 32 - 43, The test never verifies TTL refresh because newTestCache() creates a fixed 1-hour cache and assertions run immediately; change the tests to use a controllable/short TTL and advance time to observe expiry. Modify newTestCache to accept a TTL (or a clock) parameter and construct the expiring cache with that TTL (or pass a mock clock), then in the tests that call ProcessRefreshImageCacheTTL (and the block around lines 125-191) create the cache with a very short TTL (e.g., 10–50ms) or a fake clock, call ProcessRefreshImageCacheTTL, advance the clock (or sleep) past the original TTL for non-refreshed keys, and assert refreshed keys still exist while untouched keys have expired; refer to newTestCache and ProcessRefreshImageCacheTTL to locate the changes.sensor/common/reprocessor/handler.go (1)
95-97: Align empty-key handling with refresh path.On Line 96, invalidation appends keys unconditionally, while the refresh handler skips empty keys. Filtering here too improves behavioral consistency and avoids acting on empty cache keys.
Suggested change
keysToDelete := make([]cache.Key, 0, len(req.GetImageKeys())) for _, image := range req.GetImageKeys() { - keysToDelete = append(keysToDelete, cacheKeyFromImageKey(image)) + if key := cacheKeyFromImageKey(image); key != "" { + keysToDelete = append(keysToDelete, key) + } } h.imageCache.Remove(keysToDelete...)As per coding guidelines, "Focus on major issues impacting performance, readability, maintainability and security. Avoid nitpicks and avoid verbosity."
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@sensor/common/reprocessor/handler.go` around lines 95 - 97, The invalidation loop appends cache keys unconditionally; align it with the refresh path by skipping empty keys — call cacheKeyFromImageKey(image) and only append to keysToDelete when the returned cache key is non-empty (i.e., if cacheKey == "" continue). Update the loop that iterates req.GetImageKeys() so it filters out empty cache keys before appending to keysToDelete.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@sensor/common/reprocessor/handler_test.go`:
- Around line 32-43: The test never verifies TTL refresh because newTestCache()
creates a fixed 1-hour cache and assertions run immediately; change the tests to
use a controllable/short TTL and advance time to observe expiry. Modify
newTestCache to accept a TTL (or a clock) parameter and construct the expiring
cache with that TTL (or pass a mock clock), then in the tests that call
ProcessRefreshImageCacheTTL (and the block around lines 125-191) create the
cache with a very short TTL (e.g., 10–50ms) or a fake clock, call
ProcessRefreshImageCacheTTL, advance the clock (or sleep) past the original TTL
for non-refreshed keys, and assert refreshed keys still exist while untouched
keys have expired; refer to newTestCache and ProcessRefreshImageCacheTTL to
locate the changes.
In `@sensor/common/reprocessor/handler.go`:
- Around line 95-97: The invalidation loop appends cache keys unconditionally;
align it with the refresh path by skipping empty keys — call
cacheKeyFromImageKey(image) and only append to keysToDelete when the returned
cache key is non-empty (i.e., if cacheKey == "" continue). Update the loop that
iterates req.GetImageKeys() so it filters out empty cache keys before appending
to keysToDelete.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Central YAML (base), Organization UI (inherited)
Review profile: CHILL
Plan: Pro
Run ID: d331c9e9-1efa-420d-8a79-026e0d7fa5fc
⛔ Files ignored due to path filters (4)
generated/internalapi/central/sensor_iservice.pb.gois excluded by!**/*.pb.go,!**/generated/**generated/internalapi/central/sensor_iservice_vtproto.pb.gois excluded by!**/*.pb.go,!**/generated/**generated/internalapi/sensor/admission_control.pb.gois excluded by!**/*.pb.go,!**/generated/**generated/internalapi/sensor/admission_control_vtproto.pb.gois excluded by!**/*.pb.go,!**/generated/**
📒 Files selected for processing (17)
central/image/service/service_impl.gocentral/image/service/service_impl_test.gocentral/vulnmgmt/vulnerabilityrequest/manager/requestmgr/manager_impl.gopkg/expiringcache/cache.gopkg/expiringcache/cache_test.gopkg/expiringcache/mocks/cache.goproto/internalapi/central/sensor_iservice.protoproto/internalapi/sensor/admission_control.protosensor/admission-control/manager/manager_impl_test.gosensor/common/admissioncontroller/mocks/settings_manager.gosensor/common/admissioncontroller/settings_manager.gosensor/common/admissioncontroller/settings_manager_impl.gosensor/common/reprocessor/handler.gosensor/common/reprocessor/handler_test.gosensor/common/reprocessor/mocks/handler.gosensor/kubernetes/eventpipeline/pipeline_impl.gosensor/kubernetes/eventpipeline/pipeline_test.go
|
/test ocp-4-21-qa-e2e-tests |
Description
Adds Sensor-side processing of
RefreshImageCacheTTLmessages from Central. When Central's reprocessor determines that an image's scan data has not changed, it sendsRefreshImageCacheTTLinstead ofUpdatedImage. Sensor handles this by extending the TTL of the existing entry in its local image cache via a newTouch()method onexpiringcache.Cache, avoiding unnecessary image data transfer, proto serialization and deserialization, and deployment re-evaluation.Key changes:
RefreshImageCacheTTLproto message inMsgToSensorcarrying a batch of image keysexpiringcache.Cache.Touch(key)method that resets an entry's TTL without replacing its valueProcessRefreshImageCacheTTLhandler in Sensor's reprocessor that callsTouchfor each image keycacheKeyFromImageKeyhelper to deduplicate key-selection logic betweenProcessInvalidateImageCacheandProcessRefreshImageCacheTTLNote:
PR #19840 contains the logic to send
RefreshImageCacheTTL.User-facing documentation
Testing and quality
Automated testing
How I validated my change
Manually tested on full stack including #19840 and #19600 using the following: