secureCodeBox · Weltraumschaf · May 26, 2025 · May 13, 2025 · May 13, 2025 · May 13, 2025
diff --git a/documentation/docs/getting-started/upgrading.md b/documentation/docs/getting-started/upgrading.md
@@ -34,6 +34,19 @@ The default `k8sScanScope` scope was also changed from `cluster` to `namespace`,
 
 ➡️  [Reference: #3025](https://github.com/secureCodeBox/secureCodeBox/pull/3025)
 
+### Removed Integrated Elasticsearch and Kibana Helm Charts
+
+The integrated Elasticsearch and Kibana Helm charts have been dropped from the Persistence ElasticSearch Hook. These charts were intended as a quick-start option, but since Elastic no longer provides their own Helm charts, they have been removed. The documentation has been updated with guidance on setting up an Elasticsearch cluster using the [ECK operator](https://www.elastic.co/elastic-cloud-kubernetes).
+
+➡️  [Reference: #2892](https://github.com/secureCodeBox/secureCodeBox/issues/2892)
+
+### Changed Default Elasticsearch Index
+
+The default Elasticsearch index has been updated from `scbv2` to `scb`. The inclusion of `v2` was a confusing oversight that has been outdated since the release of secureCodeBox v3.
+If you had previously ingested finding using the scbv2 index prefix you can keep using it by setting the `indexPrefix` helm value back to `scbv2` or by migrating your existing indexes to match the new naming scheme.
+
+➡️  [Reference: #2892](https://github.com/secureCodeBox/secureCodeBox/issues/2892)
+
 ## From 3.X to 4.X
 
 ### Renamed the docker images of demo-targets to include a "demo-target-" prefix

diff --git a/hooks/persistence-elastic/.helm-docs.gotmpl b/hooks/persistence-elastic/.helm-docs.gotmpl
@@ -22,22 +22,73 @@ usecase: "Publishes all Scan Findings to Elasticsearch."
 
 {{- define "extra.chartAboutSection" -}}
 ## What is "Persistence ElasticSearch" Hook about?
-The ElasticSearch persistenceProvider hook saves all findings and reports into the configured ElasticSearch index. This allows for some easy searching and visualization of the findings. To learn more about Elasticsearch visit [elastic.io].
+
+The ElasticSearch persistenceProvider hook saves all findings and reports into the configured ElasticSearch index.
+This allows for some easy searching and visualization of the findings. To learn more about Elasticsearch visit [elastic.io].
 
 Installing the Elasticsearch persistenceProvider hook will add a _ReadOnly Hook_ to your namespace.
+
+This hook supports both Elasticsearch as well as OpenSearch. The configuration is the same for both.
 {{- end }}
 
 {{- define "extra.scannerConfigurationSection" -}}{{- end }}
 
 {{- define "extra.chartConfigurationSection" -}}
 ## Additional Chart Configurations
 
+### Connecting the hook an Elasticsearch Cluster
+
+The Elastic hook requires an existing Elasticsearch instance to store findings. To set up the hook, follow these steps:
+
+1. **Deploy an Elasticsearch Cluster**:
+   You can deploy an Elasticsearch cluster using various methods, such as using the official Elastic Cloud, self-hosting, or using a managed service. For Kubernetes environments, we recommend using the [Elastic Cloud on Kubernetes (ECK)](https://www.elastic.co/elastic-cloud-kubernetes) operator.
+   See the [ECK documentation](https://www.elastic.co/docs/deploy-manage/deploy/cloud-on-k8s) for instructions on deploying Elasticsearch.
+
+2. **Create a Custom Values File**:
+   Create a new `custom-values.yaml` file with the following content to configure the hook to connect to your Elasticsearch instance:
+
+   ```yaml
+   externalElasticStack:
+     elasticsearchAddress: "https://quickstart-es-http:9200" # these are the default values for the ECK quickstart, change them to match your setup
+     kibanaAddress: "https://quickstart-kb-http:5601"
+
+   authentication:
+     userSecret: "elasticsearch-credentials-secret"
+     # Alternatively, use an API key:
+     # apiKeySecret: "elasticsearch-api-key-secret"
+   ```
+
+3. **Create Kubernetes Secrets for Elasticsearch Credentials**:
+   Use the following `kubectl` command to create a secret for Elasticsearch credentials:
+
+   ```bash
+   kubectl create secret generic elasticsearch-credentials-secret \
+     --from-literal=username=your-username \
+     --from-literal=password=your-password
+   ```
+
+   If using an API key, create the secret as follows:
+
+   ```bash
+   kubectl create secret generic elasticsearch-api-key-secret \
+     --from-literal=id=your-api-key-id \
+     --from-literal=key=your-api-key
+   ```
+
+4. **Deploy the Hook**:
+   Install the persistence-elastic chart using Helm with the custom values file:
+
+   ```bash
+   helm upgrade --install persistence-elastic oci://ghcr.io/securecodebox/helm/persistence-elastic --values custom-values.yaml
+   ```
+
+Ensure that the `userSecret` or `apiKeySecret` is created in your Kubernetes cluster with the appropriate credentials for accessing your Elasticsearch instance.
+
 ### Elasticsearch Indexing
 
 For the elasticsearch `indexSuffix` you can provide a date format pattern. We use [Luxon](https://moment.github.io/luxon/) to format the date. So checkout
 the [Luxon documentation](https://moment.github.io/luxon/docs/manual/formatting.html#table-of-tokens) to see what kind of format patterns you can use for the
 `indexSuffix`. Default pattern is `yyyy-MM-dd`
-
 {{- end }}
 
 {{- define "extra.scannerLinksSection" -}}

diff --git a/hooks/persistence-elastic/Chart.yaml b/hooks/persistence-elastic/Chart.yaml
@@ -11,7 +11,6 @@ type: application
 # version - gets automatically set to the secureCodeBox release version when the helm charts gets published
 version: v3.1.0-alpha1
 
-appVersion: 7.9.2
 kubeVersion: ">=v1.11.0-0"
 
 keywords:
@@ -26,13 +25,3 @@ sources:
 maintainers:
   - name: iteratec GmbH
     email: secureCodeBox@iteratec.com
-
-dependencies:
-  - name: elasticsearch
-    version: 8.5.1
-    repository: https://helm.elastic.co
-    condition: elasticsearch.enabled
-  - name: kibana
-    version: 8.5.1
-    repository: https://helm.elastic.co
-    condition: kibana.enabled
diff --git a/hooks/persistence-elastic/README.md b/hooks/persistence-elastic/README.md
@@ -33,10 +33,14 @@ Otherwise your changes will be reverted/overwritten automatically due to the bui
 </p>
 
 ## What is "Persistence ElasticSearch" Hook about?
-The ElasticSearch persistenceProvider hook saves all findings and reports into the configured ElasticSearch index. This allows for some easy searching and visualization of the findings. To learn more about Elasticsearch visit [elastic.io].
+
+The ElasticSearch persistenceProvider hook saves all findings and reports into the configured ElasticSearch index.
+This allows for some easy searching and visualization of the findings. To learn more about Elasticsearch visit [elastic.io].
 
 Installing the Elasticsearch persistenceProvider hook will add a _ReadOnly Hook_ to your namespace.
 
+This hook supports both Elasticsearch as well as OpenSearch. The configuration is the same for both.
+
 ## Deployment
 The persistence-elastic chart can be deployed via helm:
 
@@ -49,13 +53,56 @@ helm upgrade --install persistence-elastic oci://ghcr.io/securecodebox/helm/pers
 
 Kubernetes: `>=v1.11.0-0`
 
-| Repository | Name | Version |
-|------------|------|---------|
-| https://helm.elastic.co | elasticsearch | 8.5.1 |
-| https://helm.elastic.co | kibana | 8.5.1 |
-
 ## Additional Chart Configurations
 
+### Connecting the hook an Elasticsearch Cluster
+
+The Elastic hook requires an existing Elasticsearch instance to store findings. To set up the hook, follow these steps:
+
+1. **Deploy an Elasticsearch Cluster**:
+   You can deploy an Elasticsearch cluster using various methods, such as using the official Elastic Cloud, self-hosting, or using a managed service. For Kubernetes environments, we recommend using the [Elastic Cloud on Kubernetes (ECK)](https://www.elastic.co/elastic-cloud-kubernetes) operator.
+   See the [ECK documentation](https://www.elastic.co/docs/deploy-manage/deploy/cloud-on-k8s) for instructions on deploying Elasticsearch.
+
+2. **Create a Custom Values File**:
+   Create a new `custom-values.yaml` file with the following content to configure the hook to connect to your Elasticsearch instance:
+
+   ```yaml
+   externalElasticStack:
+     elasticsearchAddress: "https://quickstart-es-http:9200" # these are the default values for the ECK quickstart, change them to match your setup
+     kibanaAddress: "https://quickstart-kb-http:5601"
+
+   authentication:
+     userSecret: "elasticsearch-credentials-secret"
+     # Alternatively, use an API key:
+     # apiKeySecret: "elasticsearch-api-key-secret"
+   ```
+
+3. **Create Kubernetes Secrets for Elasticsearch Credentials**:
+   Use the following `kubectl` command to create a secret for Elasticsearch credentials:
+
+   ```bash
+   kubectl create secret generic elasticsearch-credentials-secret \
+     --from-literal=username=your-username \
+     --from-literal=password=your-password
+   ```
+
+   If using an API key, create the secret as follows:
+
+   ```bash
+   kubectl create secret generic elasticsearch-api-key-secret \
+     --from-literal=id=your-api-key-id \
+     --from-literal=key=your-api-key
+   ```
+
+4. **Deploy the Hook**:
+   Install the persistence-elastic chart using Helm with the custom values file:
+
+   ```bash
+   helm upgrade --install persistence-elastic oci://ghcr.io/securecodebox/helm/persistence-elastic --values custom-values.yaml
+   ```
+
+Ensure that the `userSecret` or `apiKeySecret` is created in your Kubernetes cluster with the appropriate credentials for accessing your Elasticsearch instance.
+
 ### Elasticsearch Indexing
 
 For the elasticsearch `indexSuffix` you can provide a date format pattern. We use [Luxon](https://moment.github.io/luxon/) to format the date. So checkout
@@ -73,12 +120,7 @@ the [Luxon documentation](https://moment.github.io/luxon/docs/manual/formatting.
 | dashboardImporter.enabled | bool | `true` | Enable if you want to import some example kibana dashboards for secureCodeBox findings analytics. |
 | dashboardImporter.image.repository | string | `"securecodebox/persistence-elastic-dashboard-importer"` |  |
 | dashboardImporter.image.tag | string | `nil` |  |
-| elasticsearch | object | `{"enabled":true,"minimumMasterNodes":1,"replicas":1}` | Configures the included elasticsearch subchart (see: https://github.com/elastic/helm-charts/tree/elasticsearch) |
-| elasticsearch.enabled | bool | `true` | Enable if you want to deploy an elasticsearch service. |
-| elasticsearch.minimumMasterNodes | int | `1` | The value for discovery.zen.minimum_master_nodes. Should be set to (master_eligible_nodes / 2) + 1. Ignored in Elasticsearch versions >= 7 |
-| elasticsearch.replicas | int | `1` | Kubernetes replica count for the StatefulSet (i.e. how many pods) |
 | externalElasticStack.elasticsearchAddress | string | `"https://elasticsearch.example.com"` | The URL of the elasticsearch service to persists all findings to. |
-| externalElasticStack.enabled | bool | `false` | Enable this when you already have an Elastic Stack running to which you want to send your results |
 | externalElasticStack.kibanaAddress | string | `"https://kibana.example.com"` | The URL of the kibana service used to visualize all findings. |
 | fullnameOverride | string | `""` |  |
 | hook.affinity | object | `{}` | Optional affinity settings that control how the hook job is scheduled (see: https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes-using-node-affinity/) |
@@ -94,10 +136,8 @@ the [Luxon documentation](https://moment.github.io/luxon/docs/manual/formatting.
 | hook.ttlSecondsAfterFinished | string | `nil` | Seconds after which the kubernetes job for the hook will be deleted. Requires the Kubernetes TTLAfterFinished controller: https://kubernetes.io/docs/concepts/workloads/controllers/ttlafterfinished/ |
 | imagePullSecrets | list | `[]` | Define imagePullSecrets when a private registry is used (see: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/) |
 | indexAppendNamespace | bool | `true` | Define if the name of the namespace where this hook is deployed to must be added to the index name. The namespace can be used to separate index by tenants (namespaces). |
-| indexPrefix | string | `"scbv2"` | Define a specific index prefix used for all elasticsearch indices. |
-| indexSuffix | string | `"“yyyy-MM-dd”"` | Define a specific index suffix based on date pattern (YEAR (yyyy), MONTH (yyyy-MM), WEEK (yyyy-'W'W), DATE (yyyy-MM-dd)). We use Luxon for date formatting (https://moment.github.io/luxon/docs/manual/formatting.html#table-of-tokens) |
-| kibana | object | `{"enabled":true}` | Configures included Elasticsearch subchart |
-| kibana.enabled | bool | `true` | Enable if you want to deploy an kibana service (see: https://github.com/elastic/helm-charts/tree/master/kibana) |
+| indexPrefix | string | `"scb"` | Define a specific index prefix used for all elasticsearch indices. |
+| indexSuffix | string | `"yyyy-MM-dd"` | Define a specific index suffix based on date pattern (YEAR (yyyy), MONTH (yyyy-MM), WEEK (yyyy-'W'W), DATE (yyyy-MM-dd)). We use Luxon for date formatting (https://moment.github.io/luxon/docs/manual/formatting.html#table-of-tokens) |
 | nameOverride | string | `""` |  |
 | nodeSelector | object | `{}` |  |
 | podSecurityContext | object | `{}` |  |

diff --git a/hooks/persistence-elastic/charts/elasticsearch-7.17.3.tgz b/hooks/persistence-elastic/charts/elasticsearch-7.17.3.tgz
diff --git a/hooks/persistence-elastic/charts/elasticsearch-7.17.3.tgz.license b/hooks/persistence-elastic/charts/elasticsearch-7.17.3.tgz.license
diff --git a/hooks/persistence-elastic/charts/kibana-7.17.3.tgz b/hooks/persistence-elastic/charts/kibana-7.17.3.tgz
diff --git a/hooks/persistence-elastic/charts/kibana-7.17.3.tgz.license b/hooks/persistence-elastic/charts/kibana-7.17.3.tgz.license
diff --git a/hooks/persistence-elastic/dashboard-importer/Dockerfile b/hooks/persistence-elastic/dashboard-importer/Dockerfile
@@ -2,7 +2,7 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-FROM alpine:3.11
+FROM alpine:3.21
 
 RUN apk add --no-cache curl bash
 
@@ -11,7 +11,7 @@ USER app
 
 WORKDIR /home/dashboard-importer/
 
-COPY dashboards/ ./dashboards/
-COPY import-dashboards.sh ./
+COPY --chown=root:root --chmod=755 dashboards/ ./dashboards/
+COPY --chown=root:root --chmod=755 import-dashboards.sh ./
 
 CMD [ "bash", "import-dashboards.sh" ]
diff --git a/hooks/persistence-elastic/dashboard-importer/dashboards/daily-summary.json b/hooks/persistence-elastic/dashboard-importer/dashboards/daily-summary.json
@@ -288,7 +288,7 @@
       "updated_at": "2020-10-07T06:45:25.958Z",
       "version": "WzcwLDFd",
       "attributes": {
-        "title": "scbv2_*",
+        "title": "scb_*",
         "timeFieldName": "@timestamp",
         "fields": "[{\"name\":\"@timestamp\",\"type\":\"date\",\"esTypes\":[\"date\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true},{\"name\":\"_id\",\"type\":\"string\",\"esTypes\":[\"_id\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_index\",\"type\":\"string\",\"esTypes\":[\"_index\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"_score\",\"type\":\"number\",\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_source\",\"type\":\"_source\",\"esTypes\":[\"_source\"],\"count\":0,\"scripted\":false,\"searchable\":false,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"_type\",\"type\":\"string\",\"esTypes\":[\"_type\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":false},{\"name\":\"attributes.hostname\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"attributes.hostname.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"attributes.hostname\"}}},{\"name\":\"attributes.ip_address\",\"type\":\"ip\",\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"category\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"category.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"category\"}}},{\"name\":\"description\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"description.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"description\"}}},{\"name\":\"id\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"id.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"id\"}}},{\"name\":\"location\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"location.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"location\"}}},{\"name\":\"name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"name\"}}},{\"name\":\"osi_layer\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"osi_layer.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"osi_layer\"}}},{\"name\":\"parameters\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"parameters.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"parameters\"}}},{\"name\":\"scan_id\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"scan_id.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"scan_id\"}}},{\"name\":\"scan_name\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"scan_name.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"scan_name\"}}},{\"name\":\"scan_type\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"scan_type.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"scan_type\"}}},{\"name\":\"severity\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"severity.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"severity\"}}},{\"name\":\"type\",\"type\":\"string\",\"esTypes\":[\"text\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":false,\"readFromDocValues\":false},{\"name\":\"type.keyword\",\"type\":\"string\",\"esTypes\":[\"keyword\"],\"count\":0,\"scripted\":false,\"searchable\":true,\"aggregatable\":true,\"readFromDocValues\":true,\"subType\":{\"multi\":{\"parent\":\"type\"}}}]"
       },