Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 0 additions & 1 deletion .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -418,7 +418,6 @@ jobs:
fail-fast: false
matrix:
unit:
- amass
- ffuf
- git-repo-scanner
- gitleaks
Expand Down
2 changes: 0 additions & 2 deletions .github/workflows/release-build.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -322,7 +322,6 @@ jobs:
strategy:
matrix:
parser:
- amass
- ffuf
- git-repo-scanner
- gitleaks
Expand Down Expand Up @@ -408,7 +407,6 @@ jobs:
strategy:
matrix:
scanner:
- amass
- ffuf
- kube-hunter
- ncrack
Expand Down
5 changes: 2 additions & 3 deletions .github/workflows/scb-bot.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,6 @@ jobs:
fail-fast: false
matrix:
scanner:
- amass
- ffuf
- gitleaks
- kube-hunter
Expand Down Expand Up @@ -61,7 +60,7 @@ jobs:
cmd: echo local=$(yq e .appVersion scanners/${{ matrix.scanner }}/Chart.yaml) >> $GITHUB_ENV

# Fetching scanner version API from local chart .annotations.versionApi attribute
# This would look like https://api.github.com/repos/OWASP/Amass/releases/latest
# This would look like https://api.github.com/repos/projectdiscovery/nuclei/releases/latest
- name: Fetch scanner's version API
uses: mikefarah/yq@v4.46.1
with:
Expand Down Expand Up @@ -130,7 +129,7 @@ jobs:
# Reformats the versionApi to have an HTML view of the release changelog
# sed command is divided into three parts s/api.//g; , s/\/repos//g; and s/latest//g
# "api." and "/repos" and "latest" are replaced with nothing (a.k.a removed)
# example: https://api.github.com/repos/OWASP/Amass/releases/latest --> https://github.com/OWASP/Amass/releases/
# example: https://api.github.com/repos/projectdiscovery/nuclei/releases/latest --> https://github.com/projectdiscovery/nuclei/releases/
# the next command then appends the link with the new release version as it is in Github.
run: |
changelog=$(echo ${{env.versionApi}} | sed -e 's/api.//g;s/\/repos//g;s/latest//g')
Expand Down
2 changes: 1 addition & 1 deletion documentation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ Recommended websites for free icons or images:

## Documentation Guide

Since we work with various different tools, it is even more important to keep a clean and well structured Documentation. Our website's purpose is to provide a comfortable navigation and clear overview. But since no one is going to update the documentation if it's untraceable, it is also very important to keep the documentation's location clear and as less spread as possible throughout our project. So no single-page documentation hidden in the deep structure of a remote repository! In general we keep specific documentation in the respective directory (e.g. the Amass documentation is a README.md in the Amass directory). Documentation can also be housed in this repository, if it is not specifically categorized to something, but follows the guidelines of the documentation build scripts. One key feature our documentations share in order to be put on the website is a frontmatter in each documentation. It is mandatory since at least a title (and for integrations also a description) are required. But it also can provide helpful other information very easily, for a documentation see [frontmatter]. Don't be afraid to use it for including important information, which you can't get/provide otherwise very well, but don't overuse it! For more detailed examples see the following sections.
Since we work with various different tools, it is even more important to keep a clean and well structured Documentation. Our website's purpose is to provide a comfortable navigation and clear overview. But since no one is going to update the documentation if it's untraceable, it is also very important to keep the documentation's location clear and as less spread as possible throughout our project. So no single-page documentation hidden in the deep structure of a remote repository! In general we keep specific documentation in the respective directory (e.g. the nmap documentation is a README.md in the nmap directory). Documentation can also be housed in this repository, if it is not specifically categorized to something, but follows the guidelines of the documentation build scripts. One key feature our documentations share in order to be put on the website is a frontmatter in each documentation. It is mandatory since at least a title (and for integrations also a description) are required. But it also can provide helpful other information very easily, for a documentation see [frontmatter]. Don't be afraid to use it for including important information, which you can't get/provide otherwise very well, but don't overuse it! For more detailed examples see the following sections.

### Adding a Scanner or Hook

Expand Down
21 changes: 12 additions & 9 deletions documentation/docs/contributing/test-concept/scanner-test.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,24 +42,27 @@ in the scanner directory.

Each scanner has a folder with integration tests. For the integration tests we check the results of the `scan` function. This function runs an actual SCB scan in the Kind Cluster (Through the [Scan CRD](/docs/api/crds/scan)). It expects the following parameters: Name of the scan, scanType, scanner-specific parameters for the scan and the allowed timeout.

An integration test for, for example, the amass scanner looks like this:
An integration test for, for example, the nmap scanner looks like this:

```js
test(
"amass should find at least 20 subdomains",
"nmap should identify open ports of bodgeit",
async () => {
const { count } = await scan(
"amass-scanner-dummy-scan",
"amass",
["-passive", "-noalts", "-norecursive", "-d", "owasp.org"],
const { count, categories } = await scan(
"nmap-scanner-dummy-scan",
"nmap",
["bodgeit.demo-targets.svc"],
180
);
expect(count).toBeGreaterThanOrEqual(20);
expect(count).toBe(2);
expect(categories["Host"]).toBe(1);
expect(categories["Open Port"]).toBe(1);
},
{ timeout: 6 * 60 * 1000 },
{ timeout: 3 * 60 * 1000 },
);
```
For this test to be considered successful, it has to match the expected condition. In this case, the condition is that the count of the findings is greater or equal to 20.
For this test to be considered successful, it has to match the expected condition.

### How to Run an Integration Test

To run the test it suffices to run:
Expand Down
2 changes: 1 addition & 1 deletion documentation/docs/getting-started/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -120,13 +120,13 @@ You can optionally deploy SCB scanner charts for each security scanner you want
```bash
# The following chart will be installed in the `default` namespace by you can choose the namespace of your choice by
# adding `--namespace YOURNAMESPACE` to each line
helm upgrade --install amass oci://ghcr.io/securecodebox/helm/amass
helm upgrade --install gitleaks oci://ghcr.io/securecodebox/helm/gitleaks
helm upgrade --install kube-hunter oci://ghcr.io/securecodebox/helm/kube-hunter
helm upgrade --install nikto oci://ghcr.io/securecodebox/helm/nikto
helm upgrade --install nmap oci://ghcr.io/securecodebox/helm/nmap
helm upgrade --install ssh-audit oci://ghcr.io/securecodebox/helm/ssh-audit
helm upgrade --install sslyze oci://ghcr.io/securecodebox/helm/sslyze
helm upgrade --install subfinder oci://ghcr.io/securecodebox/helm/subfinder
helm upgrade --install trivy oci://ghcr.io/securecodebox/helm/trivy
helm upgrade --install wpscan oci://ghcr.io/securecodebox/helm/wpscan
helm upgrade --install zap-automation-framework oci://ghcr.io/securecodebox/helm/zap-automation-framework
Expand Down
5 changes: 3 additions & 2 deletions documentation/docs/getting-started/upgrading.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,11 +14,12 @@ sidebar_position: 3

### Removed / Replaced ScanTypes

* `typo3scan` was removed as the scanner itself [isn't maintaned anymore](https://github.com/whoot/Typo3Scan?tab=readme-ov-file#unsupported). Most security aspects of typo3 are now hard to verify from the outside as it requires authentication (which is really good). Some typo3 security aspects (e.g. a incomplete installation) can be verified by [nuclei](https://www.securecodebox.io/docs/scanners/nuclei).
* `zap-baseline-scan` and `zap-advanced` in favor of the `zap-automation-framework`. The `zap-automation-framework` ScanTpye includes all functionalities of the removed ScanTypes and can be customized easily. The default ScanType for the AutoDiscovery has been changed to the `zap-automation-framework` as well. For migrating to the `zap-automation-framework` please refer to [migration to zap-automation framework](/docs/scanners/zap-automation-framework#migration-to-zap-automation-framework) guide.
* `amass` has been replaced with `subfinder`. Amass is still an amzing tool, but with its focus on becoming more of a standalone platform / database for attack surfaces keeping it integrated and updated in the secureCodeBox was getting harder and harder. [subfinder](https://github.com/projectdiscovery/subfinder) is a very good replacement for subdomain discovery, thats also generally quicker and produces a similar result.
* `kubeaudit` was removed as the scanner itself [isn't maintaned anymore](https://github.com/Shopify/kubeaudit?tab=readme-ov-file#-deprecation-notice-). As a replacement you can use the `trivy` with it's `k8s` scanning mode, see [trivy ScanType k8s example](https://www.securecodebox.io/docs/scanners/trivy#k8s).
* `typo3scan` was removed as the scanner itself [isn't maintaned anymore](https://github.com/whoot/Typo3Scan?tab=readme-ov-file#unsupported). Most security aspects of typo3 are now hard to verify from the outside as it requires authentication (which is really good). Some typo3 security aspects (e.g. a incomplete installation) can be verified by [nuclei](https://www.securecodebox.io/docs/scanners/nuclei).
* `doggo` was removed. Doggo was added primarily as an experimentation to be used to deduplicate duplicate scan target from cascading rules based on DNS entries. That approach hasn't worked out unfortunately. The doggo integration has been non-functional for a while (see: https://github.com/secureCodeBox/secureCodeBox/issues/2853). As an alternative, nuclei already includes some DNS record based checks, if checks for specific records are required custom nuclei rules could be used to fulfil those requirements.
* `cmseek` was removed. cmseek has seen little updates in the last years. Our secureCodeBox integration with cmseek was always pretty basic, only supporting joomla (a specfifc CMS) results, which hasn't been a big focus for us. As a replacement we recommend using nuclei which has joomla rules which will likely receive more updates in the future.
* `zap-baseline-scan` and `zap-advanced` in favor of the `zap-automation-framework`. The `zap-automation-framework` ScanTpye includes all functionalities of the removed ScanTypes and can be customized easily. The default ScanType for the AutoDiscovery has been changed to the `zap-automation-framework` as well. For migrating to the `zap-automation-framework` please refer to [migration to zap-automation framework](/docs/scanners/zap-automation-framework#migration-to-zap-automation-framework) guide.

➡️ [Reference: #2670](https://github.com/secureCodeBox/secureCodeBox/issues/2670)

Expand Down
60 changes: 30 additions & 30 deletions documentation/docs/how-tos/scope.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,19 +11,19 @@ sidebar_position: 7
## Introduction

In this step-by-step tutorial, we will go through all the required stages to set up engagement scope enforcement with secureCodeBox.
In this example, we are going to set up Amass and Nmap and run a scan with rules set-up so that we don't scan domains which are out-of-scope.
In this example, we are going to set up subfinder and Nmap and run a scan with rules set-up so that we don't scan domains which are out-of-scope.

## Setup

For the sake of the tutorial, we assume that you have your Kubernetes cluster already up and running and that we can work in your default namespace.
If not, check out the [installation](/docs/getting-started/installation/) for more information.
We also assume that you have the latest version of cascading scans installed.

We will start by installing Amass and Nmap:
We will start by installing subfinder and Nmap:

```bash
helm upgrade --install amass oci://ghcr.io/securecodebox/helm/amass
Release "amass" does not exist. Installing it now.
helm upgrade --install subfinder oci://ghcr.io/securecodebox/helm/subfinder
Release "subfinder" does not exist. Installing it now.
[...]
helm upgrade --install nmap oci://ghcr.io/securecodebox/helm/nmap
Release "nmap" does not exist. Installing it now.
Expand All @@ -33,14 +33,14 @@ Release "nmap" does not exist. Installing it now.
## Start Scan With Scope Limit

Next, we can start creating our scan definition.
Let's assume that we would like amass to scan `nmap.org` for subdomains, but we would only like to run nmap on `scanme.nmap.org`.
Let's assume that we would like subfinder to scan `nmap.org` for subdomains, but we would only like to run nmap on `scanme.nmap.org`.
In the example below, we only cascade if our `scope.cascading.securecodebox.io/domain` equals `{{attributes.name}}`.

```yaml
apiVersion: "execution.securecodebox.io/v1"
kind: Scan
metadata:
name: "amass-scan"
name: "subdomain-scan"
annotations:
scope.cascading.securecodebox.io/domain: "scanme.nmap.org"
spec:
Expand All @@ -50,7 +50,7 @@ spec:
- key: "scope.cascading.securecodebox.io/domain"
operator: "In"
values: ["{{attributes.name}}"]
scanType: "amass"
scanType: "subfinder"
parameters:
- "-d"
- "nmap.org"
Expand All @@ -64,23 +64,23 @@ Running this scan results in the following state:

```shell
$ kubectl get scans
amass-scan amass Done 65
nmap-scan-nmap-hostscan-86f2m nmap Done 6
subdomain-scan subfinder Done 65
nmap-scan-nmap-hostscan-86f2m nmap Done 6
$ kubectl get pods
cascading-scans-amass-scan-8cssx--1-hh2w6 0/1 Completed 0 2m51s
cascading-scans-nmap-scan-nmap-hostscan-86f2m-4mljv--1-7zgm2 0/1 Completed 0 2m23s
parse-amass-scan-zfhxg--1-shwkj 0/1 Completed 0 2m54s
parse-nmap-scan-nmap-hostscan-86f2m-d6jh6--1-ztgls 0/1 Completed 0 2m26s
scan-amass-scan-79h5f--1-vk7v5 0/2 Completed 0 4m22s
scan-nmap-scan-nmap-hostscan-86f2m-54pkd--1-pfpp6 0/2 Completed 0 2m48s
cascading-scans-subdomain-scan-8cssx--1-hh2w6 0/1 Completed 0 2m51s
cascading-scans-nmap-scan-nmap-hostscan-86f2m-4mljv--1-7zgm2 0/1 Completed 0 2m23s
parse-subdomain-scan-zfhxg--1-shwkj 0/1 Completed 0 2m54s
parse-nmap-scan-nmap-hostscan-86f2m-d6jh6--1-ztgls 0/1 Completed 0 2m26s
scan-subdomain-scan-79h5f--1-vk7v5 0/2 Completed 0 4m22s
scan-nmap-scan-nmap-hostscan-86f2m-54pkd--1-pfpp6 0/2 Completed 0 2m48s
```

As you can see, amass found 65 domain names, but only a single nmap scan was created.
As you can see, subfinder found 65 domain names, but only a single nmap scan was created.
In the cascading scans logs, you will see that lots of rules were not triggered as the domain was out of scope.

```shell
$ kubectl logs cascading-scans-amass-scan-8cssx--1-hh2w6
Starting hook for Scan "amass-scan"
$ kubectl logs cascading-scans-subfinder-scan-8cssx--1-hh2w6
Starting hook for Scan "subfinder-scan"
Fetched 65 findings from the file storage
Fetching CascadingScans using LabelSelector: ""
Fetched 2 CascadingRules
Expand All @@ -97,7 +97,7 @@ As an example, let's say you want to set up Nikto as a scanner.
This scanner cascades on Nmap's open port finding and uses `$.hostOrIp` to start the scan.
In some cases, nmap can return a hostname different to the original hostname, thus with scope rules we want to check for this.
Preferably, we would like to use the same rule as above.
Unfortunately, nmap gives its hostname in `attributes.hostname` instead of the defined `attributes.name` (Amass finding).
Unfortunately, nmap gives its hostname in `attributes.hostname` instead of the defined `attributes.name` (subfinder finding).
This results in the scope rule failing, and prevents Nikto from getting cascaded.

To solve this situation, you have two options:
Expand All @@ -112,7 +112,7 @@ Example:
apiVersion: "execution.securecodebox.io/v1"
kind: Scan
metadata:
name: "amass-scan"
name: "subfinder-scan"
annotations:
scope.cascading.securecodebox.io/domain: "scanme.nmap.org"
spec:
Expand All @@ -124,7 +124,7 @@ spec:
- key: "scope.cascading.securecodebox.io/domain"
operator: "In"
values: ["{{attributes.name}}"]
scanType: "amass"
scanType: "subfinder"
parameters:
- "-d"
- "nmap.org"
Expand All @@ -136,9 +136,9 @@ A more fool-proof solution is to ensure that the hostname field is available in
When deploying your scanner, you can define `scopeLimiterAliases`.

```shell
$ helm upgrade --install amass oci://ghcr.io/securecodebox/helm/amass \
$ helm upgrade --install subfinder oci://ghcr.io/securecodebox/helm/subfinder \
--set="parser.scopeLimiterAliases.hostname=\{\{attributes.name\}\}"
Release "amass" has been upgraded. Happy Helming!
Release "subfinder" has been upgraded. Happy Helming!
[...]
$ helm upgrade --install nmap oci://ghcr.io/securecodebox/helm/nmap \
--set="parser.scopeLimiterAliases.hostname=\{\{attributes.hostname\}\}"
Expand All @@ -152,11 +152,11 @@ The aliases are added to the scanner's parse definition:
apiVersion: execution.securecodebox.io/v1
kind: ParseDefinition
metadata:
name: amass-jsonl
name: subfinder-jsonl
namespace: default
spec:
env: []
image: docker.io/securecodebox/parser-amass:3.5.0
image: docker.io/securecodebox/parser-subfinder:3.16.0
imagePullPolicy: IfNotPresent
scopeLimiterAliases:
hostname: "{{attributes.name}}"
Expand All @@ -173,14 +173,14 @@ scopeLimiter:
values: ["{{$.hostname}}"]
```

Running this scan inside the cluster runs Amass, Nmap, and Nikto as expected.
Running this scan inside the cluster runs subfinder, Nmap, and Nikto as expected.

```shell
$ kubectl get scans
NAME TYPE STATE FINDINGS
amass-scan amass Done 65
nikto-scan-nmap-hostscan-rhhqz-nikto-http-ps8cl nikto Done 6
nmap-scan-nmap-hostscan-rhhqz nmap Done 6
NAME TYPE STATE FINDINGS
subdomain-scan subfinder Done 65
nikto-scan-nmap-hostscan-rhhqz-nikto-http-ps8cl nikto Done 6
nmap-scan-nmap-hostscan-rhhqz nmap Done 6
```

:::caution
Expand Down
7 changes: 0 additions & 7 deletions documentation/src/integrations.js
Original file line number Diff line number Diff line change
Expand Up @@ -76,13 +76,6 @@ export const Hooks = [
];

export const Scanners = [
{
title: "Amass",
type: "Network",
usecase: "Subdomain Enumeration Scanner",
path: "docs/scanners/amass",
imageUrl: "img/integrationIcons/Amass.svg",
},
{
title: "ffuf",
type: "Webserver",
Expand Down
2 changes: 1 addition & 1 deletion documentation/static/img/cascades.drawio.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
2 changes: 1 addition & 1 deletion hooks/cascading-scans/.helm-docs.gotmpl
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@ usecase: "Cascading Scans based declarative Rules."
The Cascading Scans Hook can be used to start additional scans based on the results of other scans.
This allows you to create powerful setups to automatically discover targets, and then trigger more specialized scans for the type of target that was discovered.

![Diagram of CascadingScans showing one amass scans for example.com finding two subdomains. These then trigger a port scan each. An identified ssh port then gets a SSH Scan and a Ncrack scan triggered. A https port gets a sslyze and a nuclei scan triggered.](https://www.securecodebox.io/img/cascades.drawio.svg)
![Diagram of CascadingScans showing one subfinder scan for example.com finding two subdomains. These then trigger a port scan each. An identified ssh port then gets a SSH Scan and a Ncrack scan triggered. A https port gets a sslyze and a nuclei scan triggered.](https://www.securecodebox.io/img/cascades.drawio.svg)

The so called `CascadingRules` consist of a `matches` section which contains one or multiple rules which are compared against `findings`.
When a `finding` matches a `rule` the `scanSpec` section will then be used to create a new scan.
Expand Down
Loading