Skip to content

refactor(chore): Decouple v1 API from storage.Cluster protobuf#19824

Open
vjwilson wants to merge 6 commits intomasterfrom
vjwilson/refactor-cluster-storage-proto
Open

refactor(chore): Decouple v1 API from storage.Cluster protobuf#19824
vjwilson wants to merge 6 commits intomasterfrom
vjwilson/refactor-cluster-storage-proto

Conversation

@vjwilson
Copy link
Copy Markdown
Contributor

@vjwilson vjwilson commented Apr 3, 2026

Introduce a dedicated v1.ClusterConfig message in cluster_service.proto that is field-number-identical to storage.Cluster, ensuring full gRPC wire and REST/JSON backward compatibility across version-skewed Central/roxctl/Sensor deployments.

  • Define v1.ClusterConfig with all 32 fields matching storage.Cluster field numbers and types; shared sub-messages and enums remain in storage package to limit blast radius
  • Add conversion layer in central/cluster/service/convert.go between storage.Cluster and v1.ClusterConfig; server-managed fields (status, health_status, etc.) are accepted in requests but ignored
  • Update service_impl.go RPCs to use v1.ClusterConfig at the API boundary while keeping storage.Cluster internally
  • Update all consumers: roxctl (sensor generate, cluster delete), config-controller client and mocks
  • Add roxctl/sensor/generate/convert.go helper for validation calls that still require storage.Cluster

Co-Authored-By: Claude Opus 4.6 noreply@anthropic.com

User-facing documentation

  • CHANGELOG.md update is not needed
  • documentation PR is not needed

Testing and quality

  • the change is production ready
  • CI results are inspected

Automated testing

  • added unit tests
  • modified existing tests

How I validated my change

  • go build ./... passes with zero errors
  • All unit tests pass for changed packages: central/cluster/service, roxctl/sensor/generate, roxctl/cluster/delete, config-controller/pkg/client
  • v1.ClusterConfig is field-number-identical to storage.Cluster, verified by inspection of the proto definition, ensuring wire compatibility

@vjwilson vjwilson requested a review from a team April 3, 2026 18:19
@vjwilson vjwilson added the do-not-merge A change which is not meant to be merged label Apr 3, 2026
@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 3, 2026

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 1 issue, and left some high level feedback:

  • There are now three separate mapping implementations between storage.Cluster and v1.ClusterConfig (central service, sensor generate, plus the implicit proto identity), which makes it easy for fields to drift; consider centralizing these conversions in a shared helper/package so new/changed fields only need to be wired in one place.
  • The notion of "server‑managed" fields (status, health_status, helm_config, etc.) is currently enforced only implicitly in convertAPIClusterToStorage; it may be safer to capture this set in a single, clearly named helper or configuration so future contributors don’t accidentally start persisting client-supplied values for those fields.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- There are now three separate mapping implementations between `storage.Cluster` and `v1.ClusterConfig` (central service, sensor generate, plus the implicit proto identity), which makes it easy for fields to drift; consider centralizing these conversions in a shared helper/package so new/changed fields only need to be wired in one place.
- The notion of "server‑managed" fields (status, health_status, helm_config, etc.) is currently enforced only implicitly in `convertAPIClusterToStorage`; it may be safer to capture this set in a single, clearly named helper or configuration so future contributors don’t accidentally start persisting client-supplied values for those fields.

## Individual Comments

### Comment 1
<location path="central/cluster/service/convert.go" line_range="11-20" />
<code_context>
+func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
</code_context>
<issue_to_address>
**suggestion:** Manual 1:1 field mapping in both conversion functions is fragile and may drift as proto definitions evolve.

Both `convertStorageClusterToAPI` and `convertAPIClusterToStorage` manually copy every field, which is easy to miss when new fields are added, contradicting the comment that all fields are copied. Please consider a more robust pattern (shared helper, codegen, or tests/compile-time checks such as round-trip `proto.Equal`) so new fields can’t be accidentally omitted.

Suggested implementation:

```golang
import (
	v1 "github.com/stackrox/rox/generated/api/v1"
	"github.com/stackrox/rox/generated/storage"
	"google.golang.org/protobuf/encoding/protojson"
)

```

```golang
 // convertStorageClusterToAPI converts a storage.Cluster to the API representation.
 // All fields are copied 1:1 via protojson. The API type is structurally identical to the storage
 // type today; they are separated to allow independent evolution in the future. Using protojson
 // reduces the risk of fields drifting as proto definitions evolve.
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
	if cluster == nil {
		return nil
	}

	out := &v1.ClusterConfig{}

	// Use protojson to perform a structural copy so newly-added fields are automatically handled
	// as long as the underlying proto definitions remain structurally compatible.
	bytes, err := protojson.Marshal(cluster)
	if err != nil {
		// In the unlikely event of an error, return an empty config rather than partially populated data.
		// Callers should treat a nil cluster input as the primary "no config" signal.
		return out
	}

	if err := protojson.Unmarshal(bytes, out); err != nil {
		return out
	}

	return out

```

To fully implement the suggestion for both directions, the same pattern should be applied to `convertAPIClusterToStorage`, replacing its manual 1:1 field assignments with a protojson-based structural copy (marshal the `*v1.ClusterConfig` to JSON, then unmarshal into `*storage.Cluster`). This will ensure that both conversion functions automatically handle newly added fields as long as the proto messages remain structurally compatible.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Comment on lines +11 to +20
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
if cluster == nil {
return nil
}
return &v1.ClusterConfig{
Id: cluster.GetId(),
Name: cluster.GetName(),
Type: cluster.GetType(),
Labels: cluster.GetLabels(),
MainImage: cluster.GetMainImage(),
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

suggestion: Manual 1:1 field mapping in both conversion functions is fragile and may drift as proto definitions evolve.

Both convertStorageClusterToAPI and convertAPIClusterToStorage manually copy every field, which is easy to miss when new fields are added, contradicting the comment that all fields are copied. Please consider a more robust pattern (shared helper, codegen, or tests/compile-time checks such as round-trip proto.Equal) so new fields can’t be accidentally omitted.

Suggested implementation:

import (
	v1 "github.com/stackrox/rox/generated/api/v1"
	"github.com/stackrox/rox/generated/storage"
	"google.golang.org/protobuf/encoding/protojson"
)
 // convertStorageClusterToAPI converts a storage.Cluster to the API representation.
 // All fields are copied 1:1 via protojson. The API type is structurally identical to the storage
 // type today; they are separated to allow independent evolution in the future. Using protojson
 // reduces the risk of fields drifting as proto definitions evolve.
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
	if cluster == nil {
		return nil
	}

	out := &v1.ClusterConfig{}

	// Use protojson to perform a structural copy so newly-added fields are automatically handled
	// as long as the underlying proto definitions remain structurally compatible.
	bytes, err := protojson.Marshal(cluster)
	if err != nil {
		// In the unlikely event of an error, return an empty config rather than partially populated data.
		// Callers should treat a nil cluster input as the primary "no config" signal.
		return out
	}

	if err := protojson.Unmarshal(bytes, out); err != nil {
		return out
	}

	return out

To fully implement the suggestion for both directions, the same pattern should be applied to convertAPIClusterToStorage, replacing its manual 1:1 field assignments with a protojson-based structural copy (marshal the *v1.ClusterConfig to JSON, then unmarshal into *storage.Cluster). This will ensure that both conversion functions automatically handle newly added fields as long as the proto messages remain structurally compatible.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented Apr 3, 2026

📝 Walkthrough

Walkthrough

Introduces a new API type v1.ClusterConfig and conversion helpers between storage Cluster and API ClusterConfig. Service handlers, client interfaces, and CLI/roxctl sensor commands are updated to use the API type; API→storage conversion omits server-managed fields.

Changes

Cohort / File(s) Summary
Proto API Definition
proto/api/v1/cluster_service.proto
Added ClusterConfig message; updated PostCluster/PutCluster request types and ClusterResponse.cluster / ClustersList.clusters to use ClusterConfig instead of storage.Cluster.
Service Layer Conversions
central/cluster/service/convert.go, central/cluster/service/convert_test.go
Added conversion helpers: convertStorageClusterToAPI, convertStorageClustersToAPI, convertAPIClusterToStorage; tests validating nil handling, field mapping, and discarding of server-managed fields.
Cluster Service Implementation
central/cluster/service/service_impl.go
Changed PostCluster/PutCluster signatures to accept *v1.ClusterConfig; handlers convert API config → storage model before persistence and convert stored clusters → API models for responses.
Config Controller Client
config-controller/pkg/client/client.go, config-controller/pkg/client/mocks/client.go, config-controller/pkg/client/client_test.go
CentralClient.ListClusters (and mocks/tests) now return []*v1.ClusterConfig instead of []*storage.Cluster; adjusted implementations and tests accordingly.
roxctl Cluster Delete
roxctl/cluster/delete/delete.go, roxctl/cluster/delete/delete_test.go
CLI delete command and tests switched to operate on *v1.ClusterConfig (removed unused storage import and adapted mocks/helpers).
roxctl Sensor Generate
roxctl/sensor/generate/convert.go, roxctl/sensor/generate/generate.go, roxctl/sensor/generate/generate_test.go, roxctl/sensor/generate/k8s.go, roxctl/sensor/generate/openshift.go, roxctl/sensor/generate/convert_test.go
Added clusterConfigToStorage conversion; sensor generation command internals now use *v1.ClusterConfig; validation calls updated to validate converted storage representation; tests updated/added.

Sequence Diagram

sequenceDiagram
    participant Client
    participant Service as Cluster Service
    participant Convert as Conversion Layer
    participant Storage as Datastore

    Client->>Service: PostCluster(v1.ClusterConfig)
    Service->>Convert: convertAPIClusterToStorage(config)
    Convert-->>Service: storage.Cluster
    Service->>Storage: AddCluster(storage.Cluster)
    Storage-->>Service: id
    Service->>Storage: GetCluster(id)
    Storage-->>Service: storage.Cluster
    Service->>Convert: convertStorageClusterToAPI(storage.Cluster)
    Convert-->>Service: v1.ClusterConfig
    Service-->>Client: ClusterResponse(v1.ClusterConfig)
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

🚥 Pre-merge checks | ✅ 1 | ❌ 2

❌ Failed checks (2 warnings)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 17.65% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Description check ⚠️ Warning The PR description provides clear objectives and validation details but omits critical sections from the template. Add the 'How I validated my change' section with detailed explanation of validation approach, and fill in the generic 'change me!' placeholder in the Description section with substantive details about the refactoring.
✅ Passed checks (1 passed)
Check name Status Explanation
Title check ✅ Passed The title 'refactor(chore): Decouple v1 API from storage.Cluster protobuf' clearly and concisely summarizes the main architectural change: decoupling the v1 API from storage.Cluster by introducing a dedicated v1.ClusterConfig message.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch vjwilson/refactor-cluster-storage-proto

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
roxctl/sensor/generate/k8s.go (1)

33-35: Keep validation and submission in lockstep.

Roxctl now validates a synthesized storage.Cluster, but later submits the original v1.ClusterConfig. That adds a second API↔storage mapping alongside the server-side converter, so any missed field in clusterConfigToStorage will make local validation disagree with what Central actually receives. Please either share the mapping or add exhaustive parity tests around the converter.

As per coding guidelines, focus on major issues impacting performance, readability, maintainability and security.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@roxctl/sensor/generate/k8s.go` around lines 33 - 35, Validation uses
clusterConfigToStorage(k8sCommand.cluster) but the code later submits the
original v1.ClusterConfig, causing possible mismatches; convert once and keep in
lockstep by validating and submitting the same storage.Cluster instance returned
by clusterConfigToStorage(k8sCommand.cluster) (or else validate the v1 type
using the exact server-side converter), and add exhaustive parity tests for
clusterConfigToStorage to ensure every field is mapped identically to the server
converter; refer to clusterValidation.ValidatePartial, clusterConfigToStorage,
and k8sCommand.cluster when making this change.
proto/api/v1/cluster_service.proto (1)

26-94: Add a descriptor-parity test to guard against future drift.

ClusterConfig is currently field-number-identical to storage.Cluster, but there is no automated check preventing this contract from diverging. A future refactoring could alter field numbers, types, or reserved ranges in one message without catching the mismatch, breaking version-skewed Central/roxctl/Sensor interop at runtime. Adding a presubmit or CI check that validates structural parity would catch such regressions early.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@proto/api/v1/cluster_service.proto` around lines 26 - 94, Add a
descriptor-parity presubmit/CI test that verifies ClusterConfig is
field-number-identical to storage.Cluster: implement a test (e.g.,
TestClusterConfigDescriptorParity) that loads the protobuf descriptors for
message names "ClusterConfig" and "storage.Cluster" and asserts equality of each
field's number, name, type, label (singular/repeated), and reserved tag ranges;
also assert parity for map fields (labels, audit_log_state) and repeated
sensor_capabilities and that reserved tags (e.g., reserved 6,8,9-12,14) match.
Hook this test into the existing proto test suite so any future changes to
ClusterConfig or storage.Cluster fail CI until explicitly reconciled.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Nitpick comments:
In `@proto/api/v1/cluster_service.proto`:
- Around line 26-94: Add a descriptor-parity presubmit/CI test that verifies
ClusterConfig is field-number-identical to storage.Cluster: implement a test
(e.g., TestClusterConfigDescriptorParity) that loads the protobuf descriptors
for message names "ClusterConfig" and "storage.Cluster" and asserts equality of
each field's number, name, type, label (singular/repeated), and reserved tag
ranges; also assert parity for map fields (labels, audit_log_state) and repeated
sensor_capabilities and that reserved tags (e.g., reserved 6,8,9-12,14) match.
Hook this test into the existing proto test suite so any future changes to
ClusterConfig or storage.Cluster fail CI until explicitly reconciled.

In `@roxctl/sensor/generate/k8s.go`:
- Around line 33-35: Validation uses clusterConfigToStorage(k8sCommand.cluster)
but the code later submits the original v1.ClusterConfig, causing possible
mismatches; convert once and keep in lockstep by validating and submitting the
same storage.Cluster instance returned by
clusterConfigToStorage(k8sCommand.cluster) (or else validate the v1 type using
the exact server-side converter), and add exhaustive parity tests for
clusterConfigToStorage to ensure every field is mapped identically to the server
converter; refer to clusterValidation.ValidatePartial, clusterConfigToStorage,
and k8sCommand.cluster when making this change.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 0cfeadff-3685-4ba7-9dce-7e5c7151ca09

📥 Commits

Reviewing files that changed from the base of the PR and between 2e6eff0 and 9f9a3e5.

⛔ Files ignored due to path filters (6)
  • generated/api/v1/cluster_service.pb.go is excluded by !**/*.pb.go, !**/generated/**
  • generated/api/v1/cluster_service.pb.gw.go is excluded by !**/*.pb.gw.go, !**/generated/**
  • generated/api/v1/cluster_service.swagger.json is excluded by !**/generated/**
  • generated/api/v1/cluster_service_grpc.pb.go is excluded by !**/*.pb.go, !**/generated/**
  • generated/api/v1/cluster_service_vtproto.pb.go is excluded by !**/*.pb.go, !**/generated/**
  • generated/internalapi/central/v1/token_service.pb.go is excluded by !**/*.pb.go, !**/generated/**
📒 Files selected for processing (13)
  • central/cluster/service/convert.go
  • central/cluster/service/service_impl.go
  • config-controller/pkg/client/client.go
  • config-controller/pkg/client/client_test.go
  • config-controller/pkg/client/mocks/client.go
  • proto/api/v1/cluster_service.proto
  • roxctl/cluster/delete/delete.go
  • roxctl/cluster/delete/delete_test.go
  • roxctl/sensor/generate/convert.go
  • roxctl/sensor/generate/generate.go
  • roxctl/sensor/generate/generate_test.go
  • roxctl/sensor/generate/k8s.go
  • roxctl/sensor/generate/openshift.go

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 3, 2026

🚀 Build Images Ready

Images are ready for commit 7a482b4. To use with deploy scripts:

export MAIN_IMAGE_TAG=4.11.x-607-g7a482b4fca

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@central/cluster/service/convert_test.go`:
- Around line 65-67: The test sets AuditLogState in the input fixture but never
verifies it in the "AllFields" test; update the TestAllFields (or the test
function that builds the input struct in convert_test.go) to assert that the
converted output contains the same AuditLogState map (keys and
storage.AuditLogFileState values). Use the same comparison style used elsewhere
in the test (e.g. require.Equal/reflect.DeepEqual/cmp) to compare the expected
AuditLogState against the actual result from the conversion function,
referencing the AuditLogState field and storage.AuditLogFileState type so future
regressions are caught.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Central YAML (base), Organization UI (inherited)

Review profile: CHILL

Plan: Pro

Run ID: 35e087e5-4f43-4841-b55c-394ed16ccd10

📥 Commits

Reviewing files that changed from the base of the PR and between 9f9a3e5 and 460e71f.

📒 Files selected for processing (2)
  • central/cluster/service/convert_test.go
  • roxctl/sensor/generate/convert_test.go

Comment on lines +65 to +67
AuditLogState: map[string]*storage.AuditLogFileState{
"node1": {CollectLogsSince: nil},
},
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot Apr 3, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

AuditLogState is populated but never asserted in the “AllFields” test.

Line 65 sets AuditLogState, but the verification block does not check it. That leaves this field unprotected against conversion regressions.

Suggested test fix
@@
 	protoassert.Equal(t, cluster.GetMostRecentSensorId(), result.GetMostRecentSensorId())
+	assert.Equal(t, cluster.GetAuditLogState(), result.GetAuditLogState())
 	assert.Equal(t, cluster.GetInitBundleId(), result.GetInitBundleId())
 	assert.Equal(t, cluster.GetSensorCapabilities(), result.GetSensorCapabilities())

Also applies to: 74-97

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@central/cluster/service/convert_test.go` around lines 65 - 67, The test sets
AuditLogState in the input fixture but never verifies it in the "AllFields"
test; update the TestAllFields (or the test function that builds the input
struct in convert_test.go) to assert that the converted output contains the same
AuditLogState map (keys and storage.AuditLogFileState values). Use the same
comparison style used elsewhere in the test (e.g.
require.Equal/reflect.DeepEqual/cmp) to compare the expected AuditLogState
against the actual result from the conversion function, referencing the
AuditLogState field and storage.AuditLogFileState type so future regressions are
caught.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re: Manual 1:1 field mapping fragility (inline comment on convert.go)

We intentionally did not adopt the suggested protojson marshal/unmarshal approach:

  1. Performance: JSON round-trips are meaningfully slower than direct field assignment, and these conversions sit in the hot path for cluster list/get API calls.
  2. Silent field propagation is a bug, not a feature: The protojson approach would automatically copy any new field added to storage.Cluster into the API response. New storage-internal fields should
    not be exposed to API clients by default — explicit field mapping forces a conscious decision about what's in the API contract.
  3. Server-managed field stripping: convertAPIClusterToStorage intentionally omits server-managed fields. A protojson round-trip would silently copy those through, defeating the purpose.

The field-drift risk is mitigated by unit tests in pkg/cluster/convert_test.go that validate all 22 fields are correctly mapped in the storage→API direction.

---This analysis and replies were generated with the assistance of Claude (Anthropic). Code changes addressing comment #1 were also made with Claude's assistance.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vjwilson, the reasoning for explicit field mapping is well-taken — the points about performance, intentional API surface control, and server-managed field stripping are all sound, and I agree the protojson approach would undermine those goals.

To clarify, this specific comment thread is narrower in scope: it's about a missing assertion in the test rather than the mapping strategy itself. Line 65 populates AuditLogState in the input fixture, but the verification block doesn't assert that the converted output carries it through. The mapping implementation may be correct, but the test as written doesn't protect against a regression in that field. The suggested fix is simply:

protoassert.Equal(t, cluster.GetMostRecentSensorId(), result.GetMostRecentSensorId())
+assert.Equal(t, cluster.GetAuditLogState(), result.GetAuditLogState())
 assert.Equal(t, cluster.GetInitBundleId(), result.GetInitBundleId())

Could you confirm whether this assertion was added as part of the Claude-assisted changes, or is it still outstanding?

@codecov
Copy link
Copy Markdown

codecov bot commented Apr 3, 2026

Codecov Report

❌ Patch coverage is 88.39286% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 49.62%. Comparing base (39d15cc) to head (7a482b4).
⚠️ Report is 21 commits behind head on master.

Files with missing lines Patch % Lines
central/cluster/service/service_impl.go 22.22% 7 Missing ⚠️
config-controller/pkg/client/client.go 0.00% 2 Missing ⚠️
roxctl/sensor/generate/generate.go 0.00% 2 Missing ⚠️
roxctl/sensor/generate/k8s.go 0.00% 1 Missing ⚠️
roxctl/sensor/generate/openshift.go 0.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master   #19824      +/-   ##
==========================================
+ Coverage   49.60%   49.62%   +0.02%     
==========================================
  Files        2766     2769       +3     
  Lines      208567   208663      +96     
==========================================
+ Hits       103454   103558     +104     
+ Misses      97436    97431       -5     
+ Partials     7677     7674       -3     
Flag Coverage Δ
go-unit-tests 49.62% <88.39%> (+0.02%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@vjwilson vjwilson force-pushed the vjwilson/refactor-cluster-storage-proto branch from d131b22 to ec37bcd Compare April 6, 2026 19:50
@vjwilson
Copy link
Copy Markdown
Contributor Author

vjwilson commented Apr 7, 2026

/test all

@vjwilson
Copy link
Copy Markdown
Contributor Author

vjwilson commented Apr 7, 2026

/test gke-nongroovy-e2e-tests

@vjwilson
Copy link
Copy Markdown
Contributor Author

vjwilson commented Apr 7, 2026

/test gke-qa-e2e-tests

@vjwilson
Copy link
Copy Markdown
Contributor Author

vjwilson commented Apr 7, 2026

/test gke-nongroovy-e2e-tests

vjwilson added 6 commits April 8, 2026 17:24
Move StorageClusterToAPIClusterConfig to pkg/cluster so the storage-to-API
conversion exists in one place. Keep API-to-storage converters local to
their packages since they have intentionally different field sets.

Also fix tests/policy_as_code_test.go and ClusterService.groovy which
still referenced storage.Cluster where v1.ClusterConfig is now expected.
@vjwilson vjwilson force-pushed the vjwilson/refactor-cluster-storage-proto branch from ec37bcd to 7a482b4 Compare April 8, 2026 21:24
@vjwilson
Copy link
Copy Markdown
Contributor Author

vjwilson commented Apr 9, 2026

Hey - I've found 1 issue, and left some high level feedback:

  • There are now three separate mapping implementations between storage.Cluster and v1.ClusterConfig (central service, sensor generate, plus the implicit proto identity), which makes it easy for fields to drift; consider centralizing these conversions in a shared helper/package so new/changed fields only need to be wired in one place.
  • The notion of "server‑managed" fields (status, health_status, helm_config, etc.) is currently enforced only implicitly in convertAPIClusterToStorage; it may be safer to capture this set in a single, clearly named helper or configuration so future contributors don’t accidentally start persisting client-supplied values for those fields.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- There are now three separate mapping implementations between `storage.Cluster` and `v1.ClusterConfig` (central service, sensor generate, plus the implicit proto identity), which makes it easy for fields to drift; consider centralizing these conversions in a shared helper/package so new/changed fields only need to be wired in one place.
- The notion of "server‑managed" fields (status, health_status, helm_config, etc.) is currently enforced only implicitly in `convertAPIClusterToStorage`; it may be safer to capture this set in a single, clearly named helper or configuration so future contributors don’t accidentally start persisting client-supplied values for those fields.

## Individual Comments

### Comment 1
<location path="central/cluster/service/convert.go" line_range="11-20" />
<code_context>
+func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
</code_context>
<issue_to_address>
**suggestion:** Manual 1:1 field mapping in both conversion functions is fragile and may drift as proto definitions evolve.

Both `convertStorageClusterToAPI` and `convertAPIClusterToStorage` manually copy every field, which is easy to miss when new fields are added, contradicting the comment that all fields are copied. Please consider a more robust pattern (shared helper, codegen, or tests/compile-time checks such as round-trip `proto.Equal`) so new fields can’t be accidentally omitted.

Suggested implementation:

```golang
import (
	v1 "github.com/stackrox/rox/generated/api/v1"
	"github.com/stackrox/rox/generated/storage"
	"google.golang.org/protobuf/encoding/protojson"
)

 // convertStorageClusterToAPI converts a storage.Cluster to the API representation.
 // All fields are copied 1:1 via protojson. The API type is structurally identical to the storage
 // type today; they are separated to allow independent evolution in the future. Using protojson
 // reduces the risk of fields drifting as proto definitions evolve.
func convertStorageClusterToAPI(cluster *storage.Cluster) *v1.ClusterConfig {
	if cluster == nil {
		return nil
	}

	out := &v1.ClusterConfig{}

	// Use protojson to perform a structural copy so newly-added fields are automatically handled
	// as long as the underlying proto definitions remain structurally compatible.
	bytes, err := protojson.Marshal(cluster)
	if err != nil {
		// In the unlikely event of an error, return an empty config rather than partially populated data.
		// Callers should treat a nil cluster input as the primary "no config" signal.
		return out
	}

	if err := protojson.Unmarshal(bytes, out); err != nil {
		return out
	}

	return out

To fully implement the suggestion for both directions, the same pattern should be applied to convertAPIClusterToStorage, replacing its manual 1:1 field assignments with a protojson-based structural copy (marshal the *v1.ClusterConfig to JSON, then unmarshal into *storage.Cluster). This will ensure that both conversion functions automatically handle newly added fields as long as the proto messages remain structurally compatible.
</issue_to_address>


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
* [X](https://twitter.com/intent/tweet?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
* [Mastodon](https://mastodon.social/share?text=I%20just%20got%20an%20instant%20code%20review%20from%20%40SourceryAI%2C%20and%20it%20was%20brilliant%21%20It%27s%20free%20for%20open%20source%20and%20has%20a%20free%20trial%20for%20private%20code.%20Check%20it%20out%20https%3A//sourcery.ai)
* [LinkedIn](https://www.linkedin.com/sharing/share-offsite/?url=https://sourcery.ai)
* [Facebook](https://www.facebook.com/sharer/sharer.php?u=https://sourcery.ai)

Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

Re: Centralizing conversions (high-level comment #1)

Addressed. The storage→API conversion has been centralized into pkg/cluster/convert.go (StorageClusterToAPIClusterConfig / StorageClustersToAPIClusterConfigs). Both central/cluster/service/convert.go and any future consumer now delegate to this shared package rather than maintaining duplicate implementations.

The API→storage converters remain intentionally local to their respective packages (central/cluster/service/ and roxctl/sensor/generate/) because they have different field sets by design:

  • Central's convertAPIClusterToStorage strips server-managed fields (status, health_status, helm_config, etc.) to prevent clients from overwriting server-authoritative state.
  • roxctl's clusterConfigToStorage includes HelmConfig because roxctl legitimately sets Helm configuration during sensor generate.

Merging these into a single shared function would require parameterizing which fields to include, adding complexity without real benefit.

Re: Server-managed fields enforcement (high-level comment #2)
The server-managed fields are documented in convertAPIClusterToStorage via both a doc comment and an inline comment listing all 7 omitted fields. We considered extracting a named constant set, but since this is the only call site where the distinction matters, it would be premature abstraction. If a second API→storage conversion path is added in the future, that would be the right time to extract a shared definition.

---This analysis and replies were generated with the assistance of Claude (Anthropic). Code changes addressing comment #1 were also made with Claude's assistance.

@vjwilson vjwilson marked this pull request as ready for review April 10, 2026 12:00
@vjwilson vjwilson requested review from a team and janisz as code owners April 10, 2026 12:00
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've found 2 issues, and left some high level feedback:

  • There are now three separate Cluster<->ClusterConfig conversion implementations (pkg/cluster, central/cluster/service, roxctl/sensor/generate); consider centralizing these or having pkg/cluster expose both directions with options (e.g., a flag to drop server-managed fields) to avoid drift when new fields are added.
  • The manual field-by-field copies in the conversion helpers are easy to miss on future proto changes; you might want to add a small safeguard (e.g., a test that reflects over both types to assert they have the same set of fields you map, or use proto.Clone plus explicit zeroing for server-managed fields) to catch omissions early.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- There are now three separate Cluster<->ClusterConfig conversion implementations (pkg/cluster, central/cluster/service, roxctl/sensor/generate); consider centralizing these or having pkg/cluster expose both directions with options (e.g., a flag to drop server-managed fields) to avoid drift when new fields are added.
- The manual field-by-field copies in the conversion helpers are easy to miss on future proto changes; you might want to add a small safeguard (e.g., a test that reflects over both types to assert they have the same set of fields you map, or use proto.Clone plus explicit zeroing for server-managed fields) to catch omissions early.

## Individual Comments

### Comment 1
<location path="central/cluster/service/convert.go" line_range="9" />
<code_context>
+	clusterutil "github.com/stackrox/rox/pkg/cluster"
+)
+
+// convertStorageClusterToAPI delegates to the shared conversion in pkg/cluster.
+var convertStorageClusterToAPI = clusterutil.StorageClusterToAPIClusterConfig
+
</code_context>
<issue_to_address>
**issue (complexity):** Consider removing the package-level conversion vars and centralizing both directions of cluster conversion in pkg/cluster so this service just calls the shared helpers directly.

You can simplify this by:

1. Removing the package-level function variables and calling the shared conversion directly.
2. Centralizing the API→storage conversion in `pkg/cluster` alongside the existing storage→API helpers, so we only have one implementation per direction.

### 1. Inline the function variables

Instead of:

```go
// convertStorageClusterToAPI delegates to the shared conversion in pkg/cluster.
var convertStorageClusterToAPI = clusterutil.StorageClusterToAPIClusterConfig

// convertStorageClustersToAPI delegates to the shared conversion in pkg/cluster.
var convertStorageClustersToAPI = clusterutil.StorageClustersToAPIClusterConfigs
```

Call the shared helpers directly where used:

```go
cfg := clusterutil.StorageClusterToAPIClusterConfig(sc)
// or
cfgs := clusterutil.StorageClustersToAPIClusterConfigs(storageClusters)
```

If you need test seams, define a small interface on the consumer side rather than package-level vars, e.g.:

```go
type ClusterConverter interface {
    ToAPI(*storage.Cluster) *v1.ClusterConfig
}

type defaultClusterConverter struct{}

func (defaultClusterConverter) ToAPI(c *storage.Cluster) *v1.ClusterConfig {
    return clusterutil.StorageClusterToAPIClusterConfig(c)
}
```

and inject a `ClusterConverter` in tests.

### 2. Move `convertAPIClusterToStorage` to `pkg/cluster`

To avoid having a third independent conversion implementation, move the inverse logic into `pkg/cluster` (or extend the existing one there) and call it from this service.

In `pkg/cluster/convert.go`:

```go
func APIClusterConfigToStorage(config *v1.ClusterConfig) *storage.Cluster {
    if config == nil {
        return nil
    }
    return &storage.Cluster{
        Id:                             config.GetId(),
        Name:                           config.GetName(),
        Type:                           config.GetType(),
        Labels:                         config.GetLabels(),
        MainImage:                      config.GetMainImage(),
        CollectorImage:                 config.GetCollectorImage(),
        CentralApiEndpoint:             config.GetCentralApiEndpoint(),
        CollectionMethod:               config.GetCollectionMethod(),
        AdmissionController:            config.GetAdmissionController(),
        AdmissionControllerUpdates:     config.GetAdmissionControllerUpdates(),
        AdmissionControllerEvents:      config.GetAdmissionControllerEvents(),
        AdmissionControllerFailOnError: config.GetAdmissionControllerFailOnError(),
        DynamicConfig:                  config.GetDynamicConfig(),
        TolerationsConfig:              config.GetTolerationsConfig(),
        SlimCollector:                  config.GetSlimCollector(),
        Priority:                       config.GetPriority(),
        ManagedBy:                      config.GetManagedBy(),
        // intentionally omit server-managed fields here
    }
}
```

Then in this service:

```go
func convertAPIClusterToStorage(config *v1.ClusterConfig) *storage.Cluster {
    return clusterutil.APIClusterConfigToStorage(config)
}
```

or even skip the local wrapper entirely and call `clusterutil.APIClusterConfigToStorage` directly.

This keeps all behavior (including the “server-managed fields are ignored” policy) in one place, reducing divergence risk and making the conversion story easier to understand.
</issue_to_address>

### Comment 2
<location path="roxctl/sensor/generate/convert.go" line_range="8" />
<code_context>
+	"github.com/stackrox/rox/generated/storage"
+)
+
+// clusterConfigToStorage converts a v1.ClusterConfig to storage.Cluster for use
+// with validation functions that accept storage.Cluster.
+func clusterConfigToStorage(config *v1.ClusterConfig) *storage.Cluster {
</code_context>
<issue_to_address>
**issue (complexity):** Consider replacing this hand-written ClusterConfig-to-Cluster converter with a shared helper in pkg/cluster and calling that here instead.

You’re re‑introducing another hand‑written `ClusterConfig -> storage.Cluster` converter, which duplicates existing logic and will be easy to forget when fields are added.

To reduce complexity and maintenance risk, centralize the mapping in `pkg/cluster` and make this file a thin wrapper.

For example, in `pkg/cluster/convert.go` add a reusable converter:

```go
// pkg/cluster/convert.go

// APIClusterConfigToStorage converts v1.ClusterConfig into storage.Cluster,
// excluding server-managed/runtime fields.
func APIClusterConfigToStorage(cfg *v1.ClusterConfig) *storage.Cluster {
	if cfg == nil {
		return nil
	}
	return &storage.Cluster{
		Id:                             cfg.GetId(),
		Name:                           cfg.GetName(),
		Type:                           cfg.GetType(),
		Labels:                         cfg.GetLabels(),
		MainImage:                      cfg.GetMainImage(),
		CollectorImage:                 cfg.GetCollectorImage(),
		CentralApiEndpoint:             cfg.GetCentralApiEndpoint(),
		CollectionMethod:               cfg.GetCollectionMethod(),
		AdmissionController:            cfg.GetAdmissionController(),
		AdmissionControllerUpdates:     cfg.GetAdmissionControllerUpdates(),
		AdmissionControllerEvents:      cfg.GetAdmissionControllerEvents(),
		AdmissionControllerFailOnError: cfg.GetAdmissionControllerFailOnError(),
		DynamicConfig:                  cfg.GetDynamicConfig(),
		TolerationsConfig:              cfg.GetTolerationsConfig(),
		SlimCollector:                  cfg.GetSlimCollector(),
		Priority:                       cfg.GetPriority(),
		ManagedBy:                      cfg.GetManagedBy(),
		HelmConfig:                     cfg.GetHelmConfig(),
	}
}
```

Then in this file, reuse that instead of maintaining another mapping:

```go
// roxctl/generate/cluster.go (or current file)

import (
    v1 "github.com/stackrox/rox/generated/api/v1"
    "github.com/stackrox/rox/generated/storage"
    clusterconvert "github.com/stackrox/rox/pkg/cluster"
)

func clusterConfigToStorage(config *v1.ClusterConfig) *storage.Cluster {
    return clusterconvert.APIClusterConfigToStorage(config)
}
```

This keeps the public behavior unchanged, but removes the duplicated field‑by‑field mapping and ensures future field additions only need to be updated in one place.
</issue_to_address>

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

clusterutil "github.com/stackrox/rox/pkg/cluster"
)

// convertStorageClusterToAPI delegates to the shared conversion in pkg/cluster.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (complexity): Consider removing the package-level conversion vars and centralizing both directions of cluster conversion in pkg/cluster so this service just calls the shared helpers directly.

You can simplify this by:

  1. Removing the package-level function variables and calling the shared conversion directly.
  2. Centralizing the API→storage conversion in pkg/cluster alongside the existing storage→API helpers, so we only have one implementation per direction.

1. Inline the function variables

Instead of:

// convertStorageClusterToAPI delegates to the shared conversion in pkg/cluster.
var convertStorageClusterToAPI = clusterutil.StorageClusterToAPIClusterConfig

// convertStorageClustersToAPI delegates to the shared conversion in pkg/cluster.
var convertStorageClustersToAPI = clusterutil.StorageClustersToAPIClusterConfigs

Call the shared helpers directly where used:

cfg := clusterutil.StorageClusterToAPIClusterConfig(sc)
// or
cfgs := clusterutil.StorageClustersToAPIClusterConfigs(storageClusters)

If you need test seams, define a small interface on the consumer side rather than package-level vars, e.g.:

type ClusterConverter interface {
    ToAPI(*storage.Cluster) *v1.ClusterConfig
}

type defaultClusterConverter struct{}

func (defaultClusterConverter) ToAPI(c *storage.Cluster) *v1.ClusterConfig {
    return clusterutil.StorageClusterToAPIClusterConfig(c)
}

and inject a ClusterConverter in tests.

2. Move convertAPIClusterToStorage to pkg/cluster

To avoid having a third independent conversion implementation, move the inverse logic into pkg/cluster (or extend the existing one there) and call it from this service.

In pkg/cluster/convert.go:

func APIClusterConfigToStorage(config *v1.ClusterConfig) *storage.Cluster {
    if config == nil {
        return nil
    }
    return &storage.Cluster{
        Id:                             config.GetId(),
        Name:                           config.GetName(),
        Type:                           config.GetType(),
        Labels:                         config.GetLabels(),
        MainImage:                      config.GetMainImage(),
        CollectorImage:                 config.GetCollectorImage(),
        CentralApiEndpoint:             config.GetCentralApiEndpoint(),
        CollectionMethod:               config.GetCollectionMethod(),
        AdmissionController:            config.GetAdmissionController(),
        AdmissionControllerUpdates:     config.GetAdmissionControllerUpdates(),
        AdmissionControllerEvents:      config.GetAdmissionControllerEvents(),
        AdmissionControllerFailOnError: config.GetAdmissionControllerFailOnError(),
        DynamicConfig:                  config.GetDynamicConfig(),
        TolerationsConfig:              config.GetTolerationsConfig(),
        SlimCollector:                  config.GetSlimCollector(),
        Priority:                       config.GetPriority(),
        ManagedBy:                      config.GetManagedBy(),
        // intentionally omit server-managed fields here
    }
}

Then in this service:

func convertAPIClusterToStorage(config *v1.ClusterConfig) *storage.Cluster {
    return clusterutil.APIClusterConfigToStorage(config)
}

or even skip the local wrapper entirely and call clusterutil.APIClusterConfigToStorage directly.

This keeps all behavior (including the “server-managed fields are ignored” policy) in one place, reducing divergence risk and making the conversion story easier to understand.

"github.com/stackrox/rox/generated/storage"
)

// clusterConfigToStorage converts a v1.ClusterConfig to storage.Cluster for use
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

issue (complexity): Consider replacing this hand-written ClusterConfig-to-Cluster converter with a shared helper in pkg/cluster and calling that here instead.

You’re re‑introducing another hand‑written ClusterConfig -> storage.Cluster converter, which duplicates existing logic and will be easy to forget when fields are added.

To reduce complexity and maintenance risk, centralize the mapping in pkg/cluster and make this file a thin wrapper.

For example, in pkg/cluster/convert.go add a reusable converter:

// pkg/cluster/convert.go

// APIClusterConfigToStorage converts v1.ClusterConfig into storage.Cluster,
// excluding server-managed/runtime fields.
func APIClusterConfigToStorage(cfg *v1.ClusterConfig) *storage.Cluster {
	if cfg == nil {
		return nil
	}
	return &storage.Cluster{
		Id:                             cfg.GetId(),
		Name:                           cfg.GetName(),
		Type:                           cfg.GetType(),
		Labels:                         cfg.GetLabels(),
		MainImage:                      cfg.GetMainImage(),
		CollectorImage:                 cfg.GetCollectorImage(),
		CentralApiEndpoint:             cfg.GetCentralApiEndpoint(),
		CollectionMethod:               cfg.GetCollectionMethod(),
		AdmissionController:            cfg.GetAdmissionController(),
		AdmissionControllerUpdates:     cfg.GetAdmissionControllerUpdates(),
		AdmissionControllerEvents:      cfg.GetAdmissionControllerEvents(),
		AdmissionControllerFailOnError: cfg.GetAdmissionControllerFailOnError(),
		DynamicConfig:                  cfg.GetDynamicConfig(),
		TolerationsConfig:              cfg.GetTolerationsConfig(),
		SlimCollector:                  cfg.GetSlimCollector(),
		Priority:                       cfg.GetPriority(),
		ManagedBy:                      cfg.GetManagedBy(),
		HelmConfig:                     cfg.GetHelmConfig(),
	}
}

Then in this file, reuse that instead of maintaining another mapping:

// roxctl/generate/cluster.go (or current file)

import (
    v1 "github.com/stackrox/rox/generated/api/v1"
    "github.com/stackrox/rox/generated/storage"
    clusterconvert "github.com/stackrox/rox/pkg/cluster"
)

func clusterConfigToStorage(config *v1.ClusterConfig) *storage.Cluster {
    return clusterconvert.APIClusterConfigToStorage(config)
}

This keeps the public behavior unchanged, but removes the duplicated field‑by‑field mapping and ensures future field additions only need to be updated in one place.

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Apr 10, 2026

@vjwilson: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/ocp-4-12-qa-e2e-tests ec37bcd link false /test ocp-4-12-qa-e2e-tests
ci/prow/ocp-4-21-qa-e2e-tests ec37bcd link false /test ocp-4-21-qa-e2e-tests
ci/prow/gke-qa-e2e-tests 7a482b4 link false /test gke-qa-e2e-tests

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant