Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 9 additions & 9 deletions Taskfile.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -108,16 +108,16 @@ tasks:
- 'echo "Cleaning up testing environment"'
- kind delete cluster --name testing-env

minio-port-forward:
desc: "Port forward the MinIO service to access it locally"
cmds:
- 'echo "Port forwarding MinIO service to localhost:9001"'
- 'echo "You can access MinIO at http://localhost:9001"'
- 'echo "Use the credentials from the secureCodeBox operator to log in:"'
- 'echo "Access Key: $(kubectl get secret -n securecodebox-system securecodebox-operator-minio -o jsonpath="{.data.root-user}" | base64 --decode)"'
- 'echo "Secret Key: $(kubectl get secret -n securecodebox-system securecodebox-operator-minio -o jsonpath="{.data.root-password}" | base64 --decode)"'
garage-port-forward:
desc: "Port forward the Garage service to access it locally"
cmds:
- 'echo "Port forwarding Garage S3 API to localhost:3900"'
- 'echo "You can access Garage at http://localhost:3900 with any S3-compatible client"'
- 'echo "Use the credentials from the secureCodeBox operator to connect:"'
- 'echo "Access Key: $(kubectl get secret -n securecodebox-system securecodebox-operator-garage -o jsonpath="{.data.access-key}" | base64 --decode)"'
- 'echo "Secret Key: $(kubectl get secret -n securecodebox-system securecodebox-operator-garage -o jsonpath="{.data.secret-key}" | base64 --decode)"'
- 'echo "Press Ctrl+C to stop port forwarding"'
- kubectl port-forward -n securecodebox-system svc/securecodebox-operator-minio 9001:9001
- kubectl port-forward -n securecodebox-system svc/securecodebox-operator-garage 3900:3900
interactive: true

test:helm:all:
Expand Down
20 changes: 10 additions & 10 deletions bin/minio-port-forward.sh → bin/garage-port-forward.sh
Original file line number Diff line number Diff line change
Expand Up @@ -4,24 +4,24 @@
#
# SPDX-License-Identifier: Apache-2.0
#
# Convenience script to start a local minio instance, where your scanner results and findings are stored by default.
# Convenience script to start a local garage instance, where your scanner results and findings are stored by default.
# Port can be set via -p option.
# The instance runs on your localhost at your defined port (default: 9000).
# The instance runs on your localhost at your defined port (default: 3900).
#
# The script (i.e. the port forwarding) must run while you want to access the minio instance.
# The script (i.e. the port forwarding) must run while you want to access the garage instance.
# However, it does not need to run while scans are executed in order to store the results.
#
# Will also print your access key and secret key.
#
# For more information see
# https://www.securecodebox.io/docs/getting-started/installation/#accessing-the-included-minio-instance
# https://www.securecodebox.io/docs/getting-started/installation/#accessing-the-included-garage-instance

set -euo pipefail
shopt -s extglob

COLOR_EMPHASIS="\e[32m"
COLOR_RESET="\e[0m"
HOST_PORT=9000 # Default port
HOST_PORT=3900 # Default port

function print() {
if [[ $# == 0 ]]; then
Expand All @@ -42,19 +42,19 @@ if [[ "$#" -eq 2 ]]; then
print "Using host port $HOST_PORT"
fi
else
print "No port with option -p set. Using default host port 9000"
print "No port with option -p set. Using default host port 3900"
fi

print "$COLOR_EMPHASIS" "Starting minio instance on localhost:9000..\n"
print "$COLOR_EMPHASIS" "Starting garage instance on localhost:$HOST_PORT..\n"

print "Your access key: "
ACCESS_KEY=$(kubectl get secret securecodebox-operator-minio -n securecodebox-system -o=jsonpath='{.data.root-user}' |
ACCESS_KEY=$(kubectl get secret securecodebox-operator-garage -n securecodebox-system -o=jsonpath='{.data.access-key}' |
base64 --decode)
print "$COLOR_EMPHASIS" "$ACCESS_KEY"

print "Your secret key: "
SECRET_KEY=$(kubectl get secret securecodebox-operator-minio -n securecodebox-system -o=jsonpath='{.data.root-password}' |
SECRET_KEY=$(kubectl get secret securecodebox-operator-garage -n securecodebox-system -o=jsonpath='{.data.secret-key}' |
base64 --decode)
print "$COLOR_EMPHASIS" "$SECRET_KEY"

kubectl port-forward -n securecodebox-system service/securecodebox-operator-minio "$HOST_PORT":9001
kubectl port-forward -n securecodebox-system service/securecodebox-operator-garage "$HOST_PORT":3900
30 changes: 17 additions & 13 deletions documentation/docs/contributing/operator.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,34 +32,38 @@ brew install go helm

After that start the `Docker.app` and go to it's settings and start Kubernetes.

#### Minio
#### Garage

For your local development you will need a S3 compatible storage.
We would recommend to use [Minio](https://min.io/download#/) inside a docker container.
We would recommend to use [Garage](https://garagehq.deuxfleurs.fr/) inside a docker container, since it can auto-create its default bucket and credentials on startup.

```bash
docker container run \
--name minio \
-p 9000:9000 \
-p 9001:9001 \
--name garage \
-p 3900:3900 \
-p 3903:3903 \
-d \
--rm \
minio/minio \
server /data \
--console-address ":9001"
-e GARAGE_DEFAULT_ACCESS_KEY="your-garage-access-key" \
-e GARAGE_DEFAULT_SECRET_KEY="your-garage-secret-key" \
-e GARAGE_DEFAULT_BUCKET="name-of-your-bucket" \
dxflrs/garage \
server \
--single-node \
--default-bucket
```

In the Minio management GUI you will need to add a new bucket for the operator. The default credentials for your minio instance are `minioadmin:minioadmin`. You might change those. Go to the management UI at [http://localhost:9001/](http://localhost:9001/) and add a new bucket. After creating your bucket you will need to specify some environment variables to enable the operator to use the bucket. For that export these variables:
This creates the bucket and credentials for you on startup. After that you will need to specify some environment variables to enable the operator to use the bucket. For that export these variables:

```bash
export MINIO_ACCESS_KEY="your-minio-access-key"
export MINIO_SECRET_KEY="your-minio-secret-key"
export MINIO_ACCESS_KEY="your-garage-access-key"
export MINIO_SECRET_KEY="your-garage-secret-key"
export S3_BUCKET="name-of-your-bucket"
export S3_USE_SSL="false" # This ensures that the operator will connect even without HTTPS
export S3_ENDPOINT="127.0.0.1:9000"
export S3_ENDPOINT="127.0.0.1:3900"
```

You can save time by using [direnv](https://direnv.net/) to export these variables in your project. If you use direnv just add a file `.s3_credentials` with your Minio credentials.
You can save time by using [direnv](https://direnv.net/) to export these variables in your project. If you use direnv just add a file `.s3_credentials` with your Garage credentials.

### Build and Run the Operator

Expand Down
4 changes: 2 additions & 2 deletions documentation/docs/getting-started/first-scans.md
Original file line number Diff line number Diff line change
Expand Up @@ -201,10 +201,10 @@ Events: <none>
```

This gives us an overview of the results of the scan.
To view the actual findings produced by the scan you can use the download link to download the findings as JSON from MinIO/S3.
To view the actual findings produced by the scan you can use the download link to download the findings as JSON from Garage/S3.

:::note
When using the default MinIO installation, you'll need to port-forward the to be able to access the finding download link. See: [installation](/docs/getting-started/installation#accessing-the-included-minio-instance)
When using the default Garage installation, you'll need to port-forward the to be able to access the finding download link. See: [installation](/docs/getting-started/installation#accessing-the-included-garage-instance)
:::

## Next Steps
Expand Down
56 changes: 32 additions & 24 deletions documentation/docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,48 +24,56 @@ You're now ready to install your [first scan types and start your first scans](/

## Supported Kubernetes Version

The secureCodeBox supports the 4 latest Kubernetes releases (`v1.35`, `v1.34`, `v1.33` & `v1.32`). Older versions might also work but are not officially supported or tested.
The secureCodeBox supports the 4 latest Kubernetes releases (`v1.37`, `v1.36`, `v1.35` & `v1.34`). Older versions might also work but are not officially supported or tested.

## Accessing the included MinIO Instance
## Accessing the included Garage Instance

:::warning Development/Quickstart Only
The included MinIO instance is intended **only for development, testing, and quickstart purposes**. For production environments, you should use either:
The included [Garage](https://garagehq.deuxfleurs.fr/) instance is intended **only for development, testing, and quickstart purposes**. For production environments, you should use either:
- A managed S3-compatible storage service from your cloud provider (AWS S3, Google Cloud Storage, etc.)
- An externally managed MinIO instance (e.g., via the MinIO Operator)
- An externally managed Garage cluster (e.g., a properly configured multi-node deployment)

This provides better reliability, scalability, and security compared to the embedded MinIO instance.
This provides better reliability, scalability, and security compared to the embedded single-node Garage instance.
:::

The default secureCodeBox Operator includes a [MinIO](https://min.io/) instance, which acts as a local S3 filestorage API used by the secureCodeBox to store the results files of its scans. You can switch it out with a S3 compatible API provided by most cloud providers.
:::note Upgrading from MinIO
As of release 5.9.0, the bundled storage backend was migrated from `minio` to `garage`. This was necessary because MinIO stopped publishing their own Docker image, and the migration had to happen outside of a major release.
:::

The default secureCodeBox Operator includes a [Garage](https://garagehq.deuxfleurs.fr/) instance, which acts as a local S3 filestorage API used by the secureCodeBox to store the results files of its scans. You can switch it out with a S3 compatible API provided by most cloud providers.

You can access the MinIO instance included in the default installation like the following:
Garage doesn't ship a web UI, so you access it with any S3-compatible client (e.g. the [`mc`](https://garagehq.deuxfleurs.fr/documentation/connect/cli/) CLI or `aws s3`) after port forwarding its S3 API:

Port Forward MinIO UI: `kubectl port-forward -n securecodebox-system service/securecodebox-operator-minio 9000:9001`
Port Forward Garage S3 API: `kubectl port-forward -n securecodebox-system service/securecodebox-operator-garage 3900:3900`

- AccessKey (Username): `kubectl get secret securecodebox-operator-minio -n securecodebox-system -o=jsonpath='{.data.root-user}' | base64 --decode; echo`
- SecretKey (Password): `kubectl get secret securecodebox-operator-minio -n securecodebox-system -o=jsonpath='{.data.root-password}' | base64 --decode; echo`
- AccessKey: `kubectl get secret securecodebox-operator-garage -n securecodebox-system -o=jsonpath='{.data.access-key}' | base64 --decode; echo`
- SecretKey: `kubectl get secret securecodebox-operator-garage -n securecodebox-system -o=jsonpath='{.data.secret-key}' | base64 --decode; echo`

:::note
If you are using Windows, do not include the `| base64 --decode; echo` part in the Command Prompt. Instead, after running the command, manually decode the resulting base64 output using PowerShell.
:::

Then open your browser on [http://localhost:9000](http://localhost:9000) and login in with the credentials returned by the command listed above.
Then configure your S3 client to use `http://localhost:3900` as endpoint with the credentials returned by the commands listed above.

If you find yourself running these snippets regularly, you might want to check out this [helper script](https://github.com/secureCodeBox/secureCodeBox/blob/main/bin/minio-port-forward.sh)
If you find yourself running these snippets regularly, you might want to check out this [helper script](https://github.com/secureCodeBox/secureCodeBox/blob/main/bin/garage-port-forward.sh)

## Operator Configuration Options

### Using a hosted S3 Buckets as storage backend (Recommended for Production)

For production environments, it is **strongly recommended** to replace the default MinIO instance with a managed S3-compatible storage service from your cloud provider. This provides better reliability, scalability, security, and backup capabilities.
For production environments, it is **strongly recommended** to replace the default Garage instance with a managed S3-compatible storage service from your cloud provider. This provides better reliability, scalability, security, and backup capabilities.

To change out the default MinIO instance with a S3 Bucket from a cloud provider you can update the helm values to connect the operator with you S3 bucket.
To change out the default Garage instance with a S3 Bucket from a cloud provider you can update the helm values to connect the operator with you S3 bucket.

:::note Upgrading from MinIO
As of release 5.9.0, the bundled storage backend was migrated from `minio` to `garage`. This was necessary because MinIO stopped publishing their own Docker image, and the migration had to happen outside of a major release. If you already disabled the bundled instance via `minio.enabled: false` together with your own `s3:` configuration, that value is still honored, so no changes are required. New installs and any values files that still reference other `minio.*` keys should switch to the `garage.*` equivalents described below.
:::

#### AWS S3 Buckets

```yaml
minio:
# disable the local minio instance
garage:
# disable the local garage instance
enabled: false
s3:
enabled: true
Expand Down Expand Up @@ -173,7 +181,7 @@ Therefore, scans must complete within 2.4 hours to ensure results can be persist
secureCodeBox Operator values:

```yaml
minio:
garage:
enabled: false
s3:
enabled: true
Expand Down Expand Up @@ -262,7 +270,7 @@ Therefore, scans must complete within 2.4 hours to ensure results can be persist
secureCodeBox Operator values:

```yaml
minio:
garage:
enabled: false
s3:
enabled: true
Expand All @@ -279,8 +287,8 @@ Therefore, scans must complete within 2.4 hours to ensure results can be persist
#### Google Cloud Storage

```yaml
minio:
# disable the local minio instance
garage:
# disable the local garage instance
enabled: false
s3:
enabled: true
Expand Down Expand Up @@ -356,17 +364,17 @@ These charts will be installed in the `default` namespace, but you can choose th

## Troubleshooting

### MinIO Startup Problems
### Garage Startup Problems

If your secureCodeBox Operator install is failing, and you see that the operator pod seems to be working okay, but the MinIO pods started alongside it does not start up properly, your cluster probably isn't configured to have a working default [Storage Class for Persistent Volumes](https://kubernetes.io/docs/concepts/storage/storage-classes/).
If your secureCodeBox Operator install is failing, and you see that the operator pod seems to be working okay, but the Garage pod started alongside it does not start up properly, your cluster probably isn't configured to have a working default [Storage Class for Persistent Volumes](https://kubernetes.io/docs/concepts/storage/storage-classes/).

Suggested solutions:

- Use a Cloud Storage provider instead of MinIO. This has to provide a API compatible to AWS S3. Providers that we have tried and worked great include:
- Use a Cloud Storage provider instead of Garage. This has to provide a API compatible to AWS S3. Providers that we have tried and worked great include:
- AWS S3
- Google Cloud Storage
- DigitalOcean Spaces
- Configure MinIO to use a HostPath Volume. This is more work to set up and manage, but also works for local / on-prem installation.
- Configure Garage to use a HostPath Volume. This is more work to set up and manage, but also works for local / on-prem installation.

### ClusterRole & CRD Issues

Expand Down
2 changes: 1 addition & 1 deletion documentation/docs/how-tos/hooks.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ In this step-by-step tutorial, we will go through all the required stages to set

## Hooks Basics

In principle, the _secureCodeBox_ works in such a way that the _raw findings_ (the result provided by a _scanner_) are stored in an S3 storage (e.g. Minio). These stored _raw findings_ are converted by specific parsers (each scanner has its own implementation) into a uniform _secureCodeBox_-specific finding format and will be also stored in the S3 storage. All configured hooks that can read (_Read Hooks_) and write (_ReadAndWrite Hooks_) these Findings are then executed. More details can be found in [ADR-0002](/docs/architecture/architecture_decisions/adr_0002).
In principle, the _secureCodeBox_ works in such a way that the _raw findings_ (the result provided by a _scanner_) are stored in an S3 storage (e.g. Garage). These stored _raw findings_ are converted by specific parsers (each scanner has its own implementation) into a uniform _secureCodeBox_-specific finding format and will be also stored in the S3 storage. All configured hooks that can read (_Read Hooks_) and write (_ReadAndWrite Hooks_) these Findings are then executed. More details can be found in [ADR-0002](/docs/architecture/architecture_decisions/adr_0002).

The basic purpose is to transfer the findings to third-party systems. Findings can be read with a _ReadHook_ and processed further as required (e.g. saved in another system via a REST call). On the other hand, _ReadWriteHooks_ also offer the option of adding further information to Findings or modifying them.

Expand Down
2 changes: 1 addition & 1 deletion documentation/docs/how-tos/persistence-storage.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ sidebar_position: 5

Let's say you need to manage a cluster of potential security risks, such as a large number of websites or
your company's internal network. Your needs will exceed the limitations of manual findings handling or our build-in
*minio* instance quite soon. The solution comes with a so-called *persistence provider*, which allows you to not only
*garage* instance quite soon. The solution comes with a so-called *persistence provider*, which allows you to not only
store your findings over a long period of time (as the name suggests), but also classify, prioritize and analyze
them. This tutorial discusses two persistence providers, [DefectDojo](#defectdojo) and [ElasticSearch](#elasticsearch),
which can both be combined with the *secureCodeBox*.
Expand Down
2 changes: 1 addition & 1 deletion documentation/docs/how-tos/scanning-web-applications.md
Original file line number Diff line number Diff line change
Expand Up @@ -445,5 +445,5 @@ NAME TYPE STATE FIND
zap-automation-framework-juice-shop zap-automation-framework Done 14
```

If the scan's `STATE` is set to done, We can see our findings via the S3 bucket. If you've used the default installation method you can follow the [guide](/docs/getting-started/installation#accessing-the-included-minio-instance) to access the integrated Minio S3 Bucket to view the findings.
If the scan's `STATE` is set to done, We can see our findings via the S3 bucket. If you've used the default installation method you can follow the [guide](/docs/getting-started/installation#accessing-the-included-garage-instance) to access the integrated Garage S3 Bucket to view the findings.
And we're done! Have Fun Scanning :)
2 changes: 1 addition & 1 deletion hooks/persistence-defectdojo/.helm-docs.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -284,7 +284,7 @@ For development purposes, it can be useful to run this hook locally. You can do
1. Make sure you have access to a running [DefectDojo](https://github.com/DefectDojo/django-DefectDojo) instance.
2. [Run a Scan](https://www.securecodebox.io/docs/getting-started/first-scans) of your choice.
3. Supply Download Links for the Scan Results (Raw Result and Findings.json). You can access them from the
included [Minio Instance](https://www.securecodebox.io/docs/getting-started/installation/#accessing-the-included-minio-instance)
included [Garage Instance](https://www.securecodebox.io/docs/getting-started/installation/#accessing-the-included-garage-instance)
and upload them to a GitHub Gist.
4. Set the following environment variables:
- `DEFECTDOJO_URL`: URL to your DefectDojo server, e.g `DEFECTDOJO_URL="http://192.168.0.1:8080"`. (Required by _defectdojo-client-java_ lib.)
Expand Down
Loading
Loading