ROX-30570: Add ScanSBOM API to Scanner V4#18716
ROX-30570: Add ScanSBOM API to Scanner V4#18716BradLugo wants to merge 12 commits intoblugo/ROX-30588-update-claircorefrom
Conversation
So that it is easier to differentiate between the the generation vs. scanning handler.
- HTTP handler - Pass http request reader to Scanner V4 integration - Mock conversion of vuln report to Scan SBOM response
- made 'getScannerV4SBOMIntegration' reusable from SBOM gen and scan - now pass request context down to scannerv4 client - read size exceeds limit from error directly - removed operating system variable from the sbom response type
Adds a new RPC, GetRepositoryToCPEMapping, to the Indexer service that returns the Red Hat repository-to-CPE mapping used for RHEL package vulnerability matching. This mapping will be needed by the matcher's ScanSBOM API to enrich RHEL packages with CPE information during SBOM vulnerability scanning.
dfad150 to
a394359
Compare
00f7ddd to
c595688
Compare
To prepare for the SBOMer ScanSBOM API. We'll need to pass in a RemoteIndexer, which currently lives in the service layer.
Add ScanSBOM RPC to the Matcher service that decodes an SBOM, parses its components, and matches vulnerabilities.
a394359 to
6a38701
Compare
c595688 to
d8c20d8
Compare
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## blugo/ROX-30588-update-claircore #18716 +/- ##
====================================================================
- Coverage 49.21% 49.16% -0.06%
====================================================================
Files 2658 2661 +3
Lines 199650 200062 +412
====================================================================
+ Hits 98265 98363 +98
- Misses 93955 94266 +311
- Partials 7430 7433 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
|
Images are ready for the commit at d8c20d8. To use with deploy scripts, first |
| MediaTypeSPDXJSON = "application/spdx+json" | ||
| MediaTypeSPDXText = "text/spdx+json" |
There was a problem hiding this comment.
super nit: both media types represent JSON, perhaps:
| MediaTypeSPDXJSON = "application/spdx+json" | |
| MediaTypeSPDXText = "text/spdx+json" | |
| MediaTypeSPDXAppJSON = "application/spdx+json" | |
| MediaTypeSPDXTextJSON = "text/spdx+json" |
| reg.RegisterPurlType(python.PURLType, purl.NoneNamespace, python.ParsePURL) | ||
| if repo2cpeProvider != nil { | ||
| reg.RegisterPurlType(rhel.PURLType, rhel.PURLNamespace, rhel.ParseRPMPURL, repo2CPETransformer(repo2cpeProvider)) | ||
| } else if features.SBOMScanning.Enabled() { | ||
| zlog.Warn(context.Background()).Msg("no repositoryToCPE provider configured") | ||
| } |
There was a problem hiding this comment.
What drove registering just 'pypi' + 'rpm' vs. other known purl types? (ie: maven, etc.)
| return nil | ||
| } | ||
|
|
||
| repo2cpe, err := provider.GetRepositoryToCPEMapping(ctx) |
There was a problem hiding this comment.
IIUC this would trigger a gRPC request to indexer to get the CPE mapping for every PURL found in the SBOM (assuming could be many).
For dev preview probably OK, adding a caching layer to the provider that makes the gRPC call once every <interval> (or similar) may be a quick fix.
| // Add the repository_cpes qualifier as a comma-separated list. | ||
| // Don't overwrite if it's already set. | ||
| if _, exists := qualifiersMap[rhel.PURLRepositoryCPEs]; exists { | ||
| zlog.Debug(ctx).Msgf("found extra CPEs (not recorded): %s", strings.Join(cpes, ", ")) | ||
| return nil | ||
| } |
There was a problem hiding this comment.
nit: Could move this above the repo2cpe pull/lookup to save some cycles when true.
| // repo2CPETransformer creates a TransformerFunc that adds repository CPEs to RPM PURLs. | ||
| // It looks up the repository_id qualifier and adds the repository_cpes qualifier | ||
| // with the corresponding CPEs from the provider. | ||
| func repo2CPETransformer(provider RepositoryToCPEProvider) purl.TransformerFunc { |
There was a problem hiding this comment.
nit: Consider adding some tests to cover the transformer's behavior.
6a38701 to
68728a0
Compare
|
PR needs rebase. DetailsInstructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. |
Description
SBOMerout ofmatcherImpland intomatcherServiceto allow better injection of the repository-to-cpe dependency (i.e., we need the indexer that's passed in tomatcherService).ScanSBOMAPI to the scanner v4 matcher service.ScanSBOMto the scanner v4 client.PR stack:
masterUser-facing documentation
Testing and quality
Automated testing
No automated testing. Will follow up in ROX-27690.
How I validated my change
ROX_SBOM_SCANNINGfeature flag:Matcher API
scanner v4 client
Replicate the validation from #18484: