This tracks the process of introducing versioning for all CRD related data types and CRDs itself across the SDP. In addition to cross-cutting tasks, there are still some open tasks specific to the stackable-versioned crate. The ADR on CRD versioning will be written based on learnings from all phases.

Documentation Tasks

• docs(stackable-versioned): Improve docs for module usage operator-rs#957
• docs(stackable-versioned): Improve various sections of docs operator-rs#966
• docs: Add CRD versioning ADR documentation#712

Macro Tasks

• test(stackable-versioned): Add snapshot testing operator-rs#881
• feat!: Add stackable-shared crate operator-rs#883
• feat(stackable-versioned): Add YAML serialization for merged CRD operator-rs#884
• feat(stackable-versioned): Add support for modules operator-rs#891
• feat(stackable-versioned): Add basic handling for enum variants with data operator-rs#892
• fix(stackable-versioned): Emit Kubernetes code in modules operator-rs#912
• feat(stackable-versioned)!: Pass through arguments to the kube attribute operator-rs#914
• refactor(stackable-versioned): Simplify crate overrides operator-rs#919
• refactor(stackable-versioned): Convert Override to enum operator-rs#922
• feat(stackable-versioned): Support visibility for CRD enum operator-rs#923
• chore(stackable-versioned): Remove {write,print}_merged_crd function operator-rs#924
• fix(stackable-versioned): Remove type comparison and use into() on all field conversions operator-rs#925
• fix(stackable-versioned): Check whether to skip all from impls when versioning a module operator-rs#926
• feat(stackable-versioned): Add support for multiple k8s shortname arguments operator-rs#958
• refactor(stackable-versioned): Move preserve_module into options() operator-rs#961
• feat(stackable-versioned): Add convert_with arg in changed action operator-rs#967
• feat(stackable-versioned): Add support for generics operator-rs#969
• feat(stackable-versioned): Re-emit and merge modules in versioned modules operator-rs#971
• chore: bump kube, update status type in stackable-versioned  operator-rs#982
• fix(stackable-versioned): Emit type parameter defaults operator-rs#991
• fix(stackable-webhook): Avoid aws-lc-rs crate, as it fails in nix operator-rs#1043
• feat(stackable-versioned): Generate downgrade From impls operator-rs#1033
• test(stackable-versioned): Combine snapshot and compile tests operator-rs#1041
• feat(stackable-versioned): Add kube_client to crate overrides operator-rs#1038
• chore!(stackable-webhook): Default listen address to bind to all addresses operator-rs#1045
• feat: Add flux-converter operator-rs#1035
• feat: Add CRD previewer operator-rs#1030
• refactor(stackable-versioned): Simplify attribute handling operator-rs#1053
• feat(stackable-versioned)!: Integrate with ConversionReviews operator-rs#1050
• feat(stackable-versioned)!: Add conversion tracking operator-rs#1056
• feat!: Add working conversion webhook with cert rotation operator-rs#1066
• feat!(stackable-certs): Support adding SAN entries operator-rs#1057
• fix(stackable-versioned): Validate kind, desired and current version operator-rs#1061
• fix(stackable-versioned): Emit correct match arms for version enum operator-rs#1065
• feat: Add Service on top of operators operator-templating#534
• fix(stackable-versioned): Use crate override in generated code operator-rs#1079
• feat!: Re-export stackable-certs and stackable-webhook crates operator-rs#1074

Upstream #[kube] changes

• Add deprecated argument to derive macro kube-rs/kube#1697
• feat: Add storage and served argument to derive macro kube-rs/kube#1644

Upstream darling changes

• feat: Add optional serde support for IdentString TedDriggs/darling#341

Phase One

Introduce stackable-versioned in all operator CRDs. This includes adding the macro to the CRD structs and producing the current v1alpha1 version without introducing any changes. Additionally, generate the merged CRD.

Replace the items in the task lists below with the applicable Pull Requests.

Tasks

• chore(stackable-operator)!: Version common CRDs operator-rs#968
• chore: Version CRD and AirflowClusterConfig airflow-operator#581
• chore: Use versioned CRDs commons-operator#353
• chore: Version CRD and DruidClusterConfig druid-operator#687
• edc-operator
• chore: Version CRD and HbaseClusterConfig hbase-operator#624
• chore: Version CRD and HdfsClusterConfig hdfs-operator#651
• hello-world-operator
• chore: Version CRD and HiveClusterConfig hive-operator#571
• chore: Version CRD and KafkaClusterConfig kafka-operator#826
• chore: Use versioned CRDs listener-operator#307
• chore: Version CRD and NifiClusterConfig nifi-operator#752
• chore: Version CRD opa-operator#661
• feat: Version the CRD types secret-operator#636
• chore: Version CRDs and SparkHistoryServerClusterConfig spark-k8s-operator#525
• chore: Version CRD superset-operator#597
• chore: Add CRD versioning zookeeper-operator#917
• chore: Version CRD trino-operator#711

Phase Two

Use the common versioned structs from insert operator-rs PR here.

Tasks

• Move the git-sync implementation to operator-rs airflow-operator#623
• chore: Bump stackable_operator to 0.93.0 druid-operator#725
• HBase Doesn't use common crd structs (bumped stackable-operator anyway in chore: Bump stackable_operator to 0.93.1 hbase-operator#664)
• chore: Bump stackable_operator to 0.93.1 hdfs-operator#684
• chore: Bump stackable_operator to 0.93.1 hive-operator#604
• chore: Bump stackable_operator to 0.93.1 kafka-operator#861
• Support custom NiFi components via git-sync nifi-operator#793 (replaces chore: Bump stackable_operator to 0.93.1 nifi-operator#798)
• chore: Bump stackable_operator to 0.93.1 spark-k8s-operator#572
• chore: Bump stackable_operator to 0.93.1 superset-operator#635
• chore: Bump stackable_operator to 0.93.1 trino-operator#748
• chore: Bump stackable_operator to 0.93.1 zookeeper-operator#946
• chore: Bump stackable-operator to 0.93.0 and bump deps commons-operator#357
• chore: Bump stackable-operator to 0.93.0 and bump deps listener-operator#310
• chore: Bump stackable-operator to 0.93.0 and bump deps opa-operator#727
• chore: Update changelog secret-operator#601 (replaces chore: Use versioned CRDs secret-operator#600)

Phase Three

This phase will be an end-to-end implementation of CRD versioning, meaning we do the following tasks:

• Pick an operator where we want to introduce a CRD change, consider items from List of planned breaking changes #504. The following two changes were choosen:
  • Mark samAccountName generation as non-experimental secret-operator#627
  • Mark cert-manager backend as non-experimental secret-operator#626
• Implement the webhook to enable automatic conversion between CRD versions.
• Document the changes and provide feedback on using stackable-versioned and stackable-webhook.