-
Notifications
You must be signed in to change notification settings - Fork 174
ROX-12834: Introduce fake nodescan generator #3399
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
37 commits
Select commit
Hold shift + click to select a range
59bfc5d
Introduce Compliance Full Node Scan message
Maddosaurus cd92c35
Improve formatting
Maddosaurus cf590d9
Remodel message to contain the correct set of fields
Maddosaurus ef6f4b7
Add missing newline
Maddosaurus 9b63fa6
Add updated gen proto
Maddosaurus 7d09adb
Address review: Reorganize
Maddosaurus c891b99
WIP: Introduce fake message channel, update Scanner lib dep
Maddosaurus 62623dc
WIP: Build loop
Maddosaurus 2c16306
WIP: First successful message
Maddosaurus 64f315e
WIP: cleanup
Maddosaurus f4df5cb
WIP: prepare gens
Maddosaurus e165360
WIP: Prepare interface and testing
Maddosaurus 664568f
WIP: Rename package
Maddosaurus a88a5e9
WIP: How to add service
Maddosaurus aa4e23a
WIP: Reroll to existing bidi stream
Maddosaurus 5d2f17c
WIP: Back to reusing existing Sensor/Compliance bidi stream
Maddosaurus 813c14c
Introduce configurable interval for node scanning
Maddosaurus 8a28d1c
Improve log message and add error handling
Maddosaurus bc01dd7
Dev cleanup
Maddosaurus 9da5ee6
Address style issues
Maddosaurus aaa5f09
Update message comment in NodeScan struct
Maddosaurus 9f57e60
Move log message in compliance main
Maddosaurus 2406223
Review comments
Maddosaurus f4959f0
Address review, part II
Maddosaurus 69aee63
Extend fake message with more components
Maddosaurus e4028e2
Fix TODO command format so CI understands
Maddosaurus 9dcac92
Move logger to fake nodescan implementation
Maddosaurus b2e0a56
Introduce duration check for Node Scan interval
Maddosaurus 8c621b4
Rename NodeScanInterval setting env and fix linter warnings
Maddosaurus bc70585
Change to two separate channels
Maddosaurus 7c7eb2e
WIP: Address comments
Maddosaurus 940b0f6
Rework sending NodeScan messages from compliance to sensor
Maddosaurus cb9abb7
WIP: Changes by prygiels
Maddosaurus ed7e56d
Updated FakeNodeScan and slimmed message
Maddosaurus 0a0095b
Set NodeScanInterval default value visibility to private
Maddosaurus 624fb1a
Unexport nodeRescanInterval
vikin91 707ccf2
Remove duplication of initializeStream
vikin91 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| package nodescanv2 | ||
|
|
||
| import ( | ||
| timestamp "github.com/gogo/protobuf/types" | ||
| "github.com/stackrox/rox/generated/storage" | ||
| "github.com/stackrox/rox/pkg/logging" | ||
| scannerV1 "github.com/stackrox/scanner/generated/scanner/api/v1" | ||
| ) | ||
|
|
||
| var ( | ||
| log = logging.LoggerForModule() | ||
| ) | ||
|
|
||
| // FakeNodeScanner can be used to send fake messages that would be emitted by NodeScanV2 | ||
| type FakeNodeScanner struct { | ||
| } | ||
|
|
||
| // Scan returns a fake message in the same format a real NodeScanV2 would produce | ||
| func (f *FakeNodeScanner) Scan(nodeName string) (*storage.NodeScanV2, error) { | ||
| log.Infof("Generating fake scan result message...") | ||
| msg := &storage.NodeScanV2{ | ||
| NodeId: "", | ||
| NodeName: nodeName, | ||
| ScanTime: timestamp.TimestampNow(), | ||
| Components: &scannerV1.Components{ | ||
| Namespace: "Testme OS", | ||
| RhelComponents: []*scannerV1.RHELComponent{ | ||
| { | ||
| Name: "vim-minimal", | ||
| Namespace: "rhel:8", | ||
| Version: "2:7.4.629-6.el8.x86_64", | ||
| Arch: "x86_64", | ||
| Module: "FakeMod", | ||
| Cpes: []string{"cpe:/a:redhat:enterprise_linux:8::baseos"}, | ||
| AddedBy: "FakeLayer", | ||
| }, | ||
| { | ||
| Name: "libsolv", | ||
| Namespace: "rhel:8", | ||
| Version: "0.7.7-1.el8.x86_64", | ||
| Arch: "x86_64", | ||
| Module: "FakeMod", | ||
| AddedBy: "FakeLayer", | ||
| }, | ||
| }, | ||
| LanguageComponents: nil, | ||
| }, | ||
| Notes: nil, | ||
| } | ||
| return msg, nil | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,20 @@ | ||
| package nodescanv2 | ||
|
|
||
| import ( | ||
| "github.com/pkg/errors" | ||
| "github.com/stackrox/rox/generated/storage" | ||
| ) | ||
|
|
||
| // NodeScanner defines an interface for V2 NodeScanning | ||
| type NodeScanner interface { | ||
| Scan(nodeName string) (*storage.NodeScanV2, error) | ||
msugakov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
| } | ||
|
|
||
| // NodeScan is the V2 NodeScanning implementation | ||
| type NodeScan struct { | ||
| } | ||
|
|
||
| // Scan scans the current node and returns the results as storage.NodeScanV2 object | ||
| func (n *NodeScan) Scan(nodeName string) (*storage.NodeScanV2, error) { | ||
| return nil, errors.New("Not implemented") | ||
| } | ||
msugakov marked this conversation as resolved.
Show resolved
Hide resolved
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,23 @@ | ||
| package nodescanv2 | ||
|
|
||
| import ( | ||
| "testing" | ||
|
|
||
| "github.com/stackrox/rox/generated/storage" | ||
| "github.com/stretchr/testify/suite" | ||
| ) | ||
|
|
||
| func TestNodeScan(t *testing.T) { | ||
| suite.Run(t, &NodeScanSuite{}) | ||
| } | ||
|
|
||
| type NodeScanSuite struct { | ||
| suite.Suite | ||
| } | ||
|
|
||
| func (n *NodeScanSuite) TestMessageFormat() { | ||
| fns, err := (&FakeNodeScanner{}).Scan("someNode") | ||
| n.Nil(err) | ||
| n.NotNil(fns) | ||
| n.IsType(&storage.NodeScanV2{}, fns) | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| package env | ||
|
|
||
| import "time" | ||
|
|
||
| var ( | ||
| nodeRescanIntervalDefault = 4 * time.Hour | ||
| // nodeRescanInterval will set the duration for when to scan nodes for vulnerabilities (NodeScanV2) | ||
| nodeRescanInterval = registerDurationSetting("ROX_NODE_RESCAN_INTERVAL", nodeRescanIntervalDefault) | ||
| ) | ||
|
|
||
| // GetNodeRescanInterval returns NodeRescanInterval if positive, otherwise returns the default | ||
| func GetNodeRescanInterval() time.Duration { | ||
| if nodeRescanInterval.DurationSetting() <= 0 { | ||
| log.Warnf("Negative or zero duration found. Setting to %s.", nodeRescanIntervalDefault.String()) | ||
| return nodeRescanIntervalDefault | ||
| } | ||
| return nodeRescanInterval.DurationSetting() | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I read up about gRPC yesterday and got a bit more dangerous ;-) although still not very educated.
And I'm definitely here to spoil the party that started (even though I haven't used the opportunity to do that earlier). It seems this code will open a stream each time there's something to send. I think gRPC maintains a pool of connections on the client, plus streams get multiplexed over HTTP/2 (but only when that's used), so opening a stream doesn't mean establishing new connection, but I think it still has some overhead.
What is worse, I believe that we reuse the same
rpc Communicateas the ordinary Compliance does. An effect of that might be that Sensor will push its messages over that stream too, but on Compliance side there's nothing receiving and acting on them for this stream. In case Sensor pushes a message to perform Compliance checks (message TriggerRun) through this stream, it will get lost.From looking at Sensor code, it seems to me that more recent connection (actually stream) replaces previous in the map keyed by a node name. Therefore, I strongly suspect that what I described actually happens if the feature flag for node scans is turned on, i.e. compliance checks stop working properly.
I don't think it's an issue for now as the feature is gated but I think the logic should be reworked before we are ready to release.
WDYT?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
That is a very good find. We should definitely improve this before releasing - it deserves a ticket.
Applying the connection factory with shared pointer to the stream (as seen in Sesor-Central) should fix the problem that you described, but I am not saying that this is the only viable solution to it. I will open a ticket.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A thought about this:
we may close the stream immediately after sending the scan results to notify sensor that this connection should not be used anymore.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm afraid the older active connection established by regular compliance logic may not get restored in Sensor's connection map. It deserves some investigation.
I think the problem is that currently Sensor first calls Compliance to do something (after the initial handshake) whereas in our new flow Compliance will first call Sensor. I don't think gRPC bidirectional streams allow receiving and sending at the same time (i.e. full duplex). As a peer in the stream you can either receive or send (or do nothing) but not receive and send concurrently. Therefore the existing stream can only be reused if Sensor initiates scans (going back to one of your proposals Piotr). We will need a different
rpcif otherwise we want Compliance to initiate sending scan results (and Sensor ack or nack it).This may be confusing to read, probably more efficient would be to explain with showing what I mean in code.
I think, your idea about Sensors initiating scans deserves to be reconsidered.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This would be a limitation. I got the impression from Sensor<->Central that the sending and receiving works concurrently, but I didn't confirm this experimentally.
Having a separate
rpcin the proto file would be also an option (maybe even a much cleaner one), but I do not think that this is necessary. The way we initiate the stream clients gives us possibility to send without depending on whether something was received (it is not in the final PR, but we had this variant shortly in the branch).I am thinking how to design a manual test to confirm or deny our theories 🤔
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Created https://issues.redhat.com/browse/ROX-13383 to not forget about the issue we are discussing here.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It turns I was wrong about no duplex support. Looking at Sensor code, one stream is used for both sending and receiving concurrently.
stackrox/sensor/common/sensor/central_communication_impl.go
Lines 157 to 158 in adcf5c3
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
A single manual test is nothing to go by, but I was able to successfully run a compliance check with the FF turned on, so it might be the message isn't lost necessarily.
But we should definitely investigate this further! Thanks for creating the ticket @vikin91