secureCodeBox · Weltraumschaf · Oct 14, 2020 · Oct 9, 2020 · Oct 13, 2020 · Oct 13, 2020
diff --git a/docs/api/crds/cascading-rule.md b/docs/api/crds/cascading-rule.md
@@ -0,0 +1,57 @@
+---
+title: "CascadingRule"
+---
+
+## Specification (Spec)
+
+CascadingRules are Custom Resource Definitions (CRD's) used to define how scans can be started automatically based on the results of previous scans. This lets you run large exploratory scans and automatically start more in depth scans on the targets found by the initial scans.
+
+You can find a more concrete example on how this works in the [network scanning how-to](/docs/how-tos/scanning-networks).
+
+### Matches (Required)
+
+### Matches.AnyOf (Required)
+
+The `matches.anyOf` fields consists of a list of objects / hashes.
+These objects are compared using a partial deep comparison, meaning that all field of the object must exactly match the finding.
+
+If multiple anyOf rules are specified at least one must match the finding. 
+If multiple rules are matching, the CascadingRule will still only create one scan.
+
+### ScanSpec (Required)
+
+Contains the [spec of the scan](/docs/api/crds/scan#specification-spec) which is supposed to be started of the a finding matches the CascadingRule.
+
+The `scanType` and the entries in the `parameters` list  can use [mustache](https://mustache.github.io/mustache.5.html) templates to refer to fields of the finding the CascadingRule has been applied to. The finding is passed in directly into the mustache templating call, so that fields of the findings can be directly referenced. E.g. the location can be directly referred to by: `{{location}}`.
+
+For convenience a helper object has been added to the mustache call under the `$` shorthand.
+
+This helper object has the following attributes:
+
+- `$.hostOrIP` returns either the hostname (if available) or the hostname of the current finding.
+
+## Example
+
+```yaml
+apiVersion: "cascading.securecodebox.io/v1"
+kind: CascadingRule
+metadata:
+  name: "zap-http"
+  labels:
+    securecodebox.io/invasive: non-invasive
+    securecodebox.io/intensive: medium
+spec:
+  matches:
+    anyOf:
+      - category: "Open Port"
+        attributes:
+          service: http
+          state: open
+      - category: "Open Port"
+        attributes:
+          service: https
+          state: open
+  scanSpec:
+    scanType: "zap-baseline"
+    parameters: ["-t", "{{attributes.service}}://{{$.hostOrIP}}"]
+```
diff --git a/docs/api/crds/parse-definition.md b/docs/api/crds/parse-definition.md
@@ -0,0 +1,31 @@
+---
+title: "ParseDefinition"
+---
+
+ParseDefinitions are Custom Resource Definitions (CRD's) used to describe to the secureCodeBox how it can convert a raw finding report (e.g. XML report from nmap) into the generic [secureCodeBox finding format](/docs/api/finding).
+
+ParseDefinitions are generally packaged together with a [ScanType](https://docs.securecodebox.io/docs/crds/scan-type).
+A scanType will reference the **name** of a *ParseDefinition* via the [extractResults.type field](/docs/api/crds/scan-type#extractresultstype-required).
+
+## Specification (Spec)
+
+### Image (Required)
+
+`image` is the reference to the parser container image which can transform the raw scan report into findings.
+
+To see how to write parsers and package them into images, checkout the [documentation page on integrating new scanners](docs/contributing/integrating-a-scanner).
+
+### ImagePullSecrets (Optional)
+
+`imagePullSecrets` can be used to integrate private parser images.
+
+## Example
+
+```yaml
+apiVersion: execution.securecodebox.io/v1
+kind: ParseDefinition
+metadata:
+  name: zap-json
+spec:
+  image: docker.io/securecodebox/parser-zap
+```
diff --git a/docs/api/crds/scan-completion-hook.md b/docs/api/crds/scan-completion-hook.md
@@ -0,0 +1,64 @@
+---
+title: "ScanCompletionHook"
+---
+
+ScanCompletionHooks are Custom Resource Definitions (CRD's) used to define custom behavior which should be run after a scan has been completed.
-ScanCompletionHooks are Custom Resource Definitions (CRD's) used to define custom behavior which should be run after a scan has been completed.
+ScanCompletionHooks are Custom Resource Definitions (CRD's) used to define custom behavior which should execute after a scan has been completed.
-ScanCompletionHooks are Custom Resource Definitions (CRD's) used to define custom behavior which should be run after a scan has been completed.
+ScanCompletionHooks are Custom Resource Definitions (CRD's) used to define custom behavior which should execute after a scan has been completed.
+
+For more detailed explanations on how a new hook can be integrated, see the [hooks section](/docs/contributing/integrating-a-hook) in the contribution part of our docs.
+
+## Specification (Spec)
+
+### Type (Required)
+
+The `type` field can be either `ReadOnly` or `ReadAndWrite`.
+
+`ReadOnly` hooks only have read rights on the findings and the raw scan reports (e.g. XML output from nmap). These are usually used to export the findings into a external system like "Elasticsearch" or "DefectDojo" or to send out notifications to chats like "Slack". ReadOnly hooks are executed in parallel to speed up their runtime.
+
+`ReadAndWrite` hooks have the ability to update both the findings and raw scan reports. This can be used to attach additional metadata to the findings by comparing the findings to external inventory systems or APIs of cloud providers.
+
+### Image (Required)
+
+The `image` field contains a container image reference for the image supposed to run as the hook.
+
+### ImagePullSecrets (Optional)
+
+The `imagePullSecrets` field can be used to specify pull secrets used to access the hooks image from a private registry.
+
+### Env (Optional)
+
+The `env` field can be used to specify env variables and to mount secrets into containers.
+
+### ServiceAccountName (Optional)
+
+The `serviceAccountName` field can be used to specify a custom ServiceAccount to use for the Kubernetes Job running the hook.
+Should only be used if your hook needs specific RBAC Access. Otherwise the hook is run using a `scan-completion-hook` service account.
-Should only be used if your hook needs specific RBAC Access. Otherwise the hook is run using a `scan-completion-hook` service account.
+Should only be used if your hook needs specific RBAC Access. Otherwise the executed hook uses a `scan-completion-hook` service account.
-Should only be used if your hook needs specific RBAC Access. Otherwise the hook is run using a `scan-completion-hook` service account.
+Should only be used if your hook needs specific RBAC Access. Otherwise the executed hook uses a `scan-completion-hook` service account.
+
+The service account should have at least `get` rights on `scans.execution.securecodebox.io`, and `get` & `patch` on `scans.execution.securecodebox.io/status` so that the hooks can work correctly.
+
+## Example
+
+```yaml
+apiVersion: execution.securecodebox.io/v1
+kind: ScanCompletionHook
+metadata:
+  name: elastic-persistence-hook
+spec:
+  type: ReadOnly
+  image: docker.io/securecodebox/persistence-elastic:latest
+  imagePullSecrets:
+  - name: image-pull-secret
+  serviceAccountName: elastic-persistence
+  env:
+  - name: ELASTICSEARCH_ADDRESS
+    value: https://data.chase.securecodebox.io
+  - name: ELASTICSEARCH_USERNAME
+    valueFrom:
+      secretKeyRef:
+        key: username
+        name: elastic-persistence-credentials
+  - name: ELASTICSEARCH_PASSWORD
+    valueFrom:
+      secretKeyRef:
+        key: password
+        name: elastic-persistence-credentials
+```
diff --git a/docs/api/crds/scan-type.md b/docs/api/crds/scan-type.md
@@ -0,0 +1,62 @@
+---
+title: "ScanType"
+---
+
+The ScanType Custom Resource Definition (CRD) is used to define to the secureCodeBox how a specific scanner can be executed in Kubernetes. The main part of the ScanType is the [JobTemplate](#jobtemplate-required), which contains a Kubernetes Job definition which will be used to construct the scans Job.
+
+## Specification (Spec)
+
+### ExtractResults (Required)
+
+The `extractResults` field contains an object (fields of the object listed below) which describes what types of results this scanType produced and from where these should be extracted.
-The `extractResults` field contains an object (fields of the object listed below) which describes what types of results this scanType produced and from where these should be extracted.
+The `extractResults` field contains an object (fields of the object listed below) which describes what types of results this scanType produces and from where these should be extracted.
-The `extractResults` field contains an object (fields of the object listed below) which describes what types of results this scanType produced and from where these should be extracted.
+The `extractResults` field contains an object (fields of the object listed below) which describes what types of results this scanType produces and from where these should be extracted.
+
+#### ExtractResults.Type (Required)
+
+The `type` field indicates the type of the file.
+Usually a combination of the scanner name and file type. E.g. `nmap-xml`
+
+The type is used to determine which parser would be used to handle this result file.
+
+#### ExtractResults.Location (Required)
+
+The `location` field describes from where the result file can be extracted.
+Absolute path on the containers file system.
-Absolute path on the containers file system.
+Absolute path on the container's file system.
-Absolute path on the containers file system.
+Absolute path on the container's file system.
+
+Must be located in `/home/securecodebox/` so that the result is reachable by the secureCodeBox Lurcher sidecar which performs the actual extraction of the result.
+E.g. `/home/securecodebox/nmap-results.xml`
+
+### JobTemplate (Required)
+
+Template of the kubernetes job to create when running the scan.
+
+See: https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#job-v1-batch
+
+## Example
+
+```yaml
+apiVersion: "execution.securecodebox.io/v1"
+kind: ScanType
+metadata:
+  name: "zap-baseline"
+spec:
+  extractResults:
+    type: zap-json
+    location: "/home/securecodebox/zap-results.json"
+  jobTemplate:
+    spec:
+      ttlSecondsAfterFinished: 10
+      template:
+        spec:
+          restartPolicy: Never
+          containers:
+            - name: zap-baseline
+              image: owasp/zap2docker-stable:2.9.0
+              command:
+                - "zap-baseline.py"
+                # Force Zap to always return a zero exit code. k8s would otherwise try to restart zap.
+                - "-I"
+                - "-J"
+                # ZAP Baseline Script doesn't allow absolute paths...
+                # Hacky workaround: specify a relative path to the `/zap/wrk` base dir.
+                - "../../home/securecodebox/zap-results.json"
+```
diff --git a/docs/api/crds/scan.md b/docs/api/crds/scan.md
@@ -0,0 +1,69 @@
+---
+title: "Scan"
+---
+
+The Scan Custom Resource Definition (CRD) lets you define how a specific scan should be configured.
+The secureCodeBox Operator will then use this specification the execute the scan.
-The secureCodeBox Operator will then use this specification the execute the scan.
+The secureCodeBox Operator will then use this specification to execute the scan.
-The secureCodeBox Operator will then use this specification the execute the scan.
+The secureCodeBox Operator will then use this specification to execute the scan.
+
+## Specification (Spec)
+
+### ScanType (Required)
+
+The `scanType` references the **name** of a [ScanType Custom Resource](https://docs.securecodebox.io/docs/crds/scan-type).
+
+### Parameters (Required)
+
+`parameters` is a string array of command line flags which are passed to the scanner.
+
+These usually contain scanner specific configurations and target specification.
+
+### Env (Optional)
+
+`env` lets you pass in custom environnement variables to the scan container.
+This can be useful to pass in secret values like login credentials scanner require without having to define them in plain text.
+
+Env has the same api as "env" property on Kubernetes Pods. 
+
+See:
+- [Documentation](https://kubernetes.io/docs/tasks/inject-data-application/define-environment-variable-container/)
+- [API Reference](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.18/#envvar-v1-core)
+
+### Cascades (Optional)
+
+`cascades` let you start new scans based on the results of the current scan.
+
+The cascades config in the scans spec contains [Kubernetes Label Selectors](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.19/#labelselector-v1-meta) which allow you to select which allow you to select which [CascadingRule](https://docs.securecodebox.io/docs/crds/cascading-rule) are allowed to be used by the cascading logic.
+
+To use cascades you'll need to have the [CombinedScan hook](https://docs.securecodebox.io/docs/hooks/Cascading%20Scans) installed.
+
+For an example on how they can be used see the [Scanning Networks HowTo](https://docs.securecodebox.io/docs/how-tos/scanning-networks)
+
+## Example
+
+```yaml
+apiVersion: "execution.securecodebox.io/v1"
+kind: Scan
+metadata:
+  name: "nmap-scanme.nmap.org"
+spec:
+  scanType: "nmap"
+  parameters:
+    # Use nmap's service detection feature
+    - "-sV"
+    - scanme.nmap.org
+  env:
+    - name: TEST_ENV
+      valueFrom:
+        secretKeyRef:
+          key: secret-name
+          name: zap-customer-credentials
+    - name: GREETING
+      value: "Hello from the secureCodeBox :D"
+  cascades:
+    matchLabels:
+      securecodebox.io/intensive: light
+    matchExpression:
+      key: "securecodebox.io/invasive"
+      operator: In
+      values: [non-invasive, invasive]
+```
diff --git a/docs/api/crds/scheduled-scan.md b/docs/api/crds/scheduled-scan.md
@@ -0,0 +1,54 @@
+---
+title: "ScheduledScan"
+---
+
+The ScheduledScan Custom Resource Definition (CRD) lets you define a [Scan](https://docs.securecodebox.io/docs/crds/scan) which gets repeated in a specific time interval. E.g. every 24 hours or every 7 days.
+
+## Specification (Spec)
+
+### Interval (Required)
+
+The `interval` specifies the interval between two scans.
+
+Specified as a [golang duration string](https://golang.org/pkg/time/#ParseDuration).
+
+:::caution 
+The biggest duration golang time strings support is **hours**. Longer durations e.g. days / weeks need to specified as multiples of hours.
+We plan to improve this in the future, by providing a custom format which also supports days and weeks.
+:::
+
+### ScanSpec (Required)
+
+The `scanSpec` contains the specification of the scan which should be repeated.
+
+See the `spec` field of the [Scan CRD](https://docs.securecodebox.io/docs/crds/scan) for all supported attributes.
+
+### SuccessfulJobsHistoryLimit (Optional)
+
+The `successfulJobsHistoryLimit` controls how many completed scans are supposed to be kept until the oldest one will be deleted. 
+
+Defaults to 3 if not set. When set to `0`, scans will be deleted directly after their completion.
+
+:::info 
+The `successfulJobsHistoryLimit` applies only to "successful" scans.
+Failed jobs currently need to be manually deleted.
+We plan to add a `failedJobsHistoryLimit` field in a future release.
+:::
+
+## Example
+
+```yaml
+apiVersion: "execution.securecodebox.io/v1"
+kind: ScheduledScan
+metadata:
+  name: "nmap-scanme.nmap.org-daily"
+spec:
+  interval: 24h
+  scanSpec:
+    scanType: "nmap"
+    parameters:
+        # Use nmaps service detection feature
+        - "-sV"
+        - scanme.nmap.org
+  historyLimit: 3
+```
diff --git a/docs/api/finding.md b/docs/api/finding.md
@@ -0,0 +1,40 @@
+---
+title: "Finding"
+---
+
+All scanners integrated in the secureCodeBox create findings objects.
+These findings all contain the same set of fields listed in the example below.
+
+```yaml
+{
+  # Unique uuid4 for the finding
+  "id": "e18cdc5e-6b49-4346-b623-28a4e878e154",
+  # name contains a short description of the finding
+  "name": "Open mysql Port",
+  # In depth description, can span multiple paragraphs
+  "description": "Port 3306 is open using tcp protocol.",
+  # The category is often used to group finding based on their types
+  "category": "Open Port",
+  # OSI network layer the finding fits into
+  "osi_layer": "NETWORK",
+  # One of "INFORMATIONAL", "LOW", "MEDIUM", "HIGH"
+  "severity": "INFORMATIONAL",
+  # Attributes are not standardized. They differ from scanner to scanner
+  "attributes": {
+    "port": 3306,
+    "state": "open",
+    "ip_address": "198.51.100.42",
+    "mac_address": null,
+    "protocol": "tcp",
+    "hostname": "example.com",
+    "method": "table",
+    "operating_system": null,
+    "service": "mysql",
+    "serviceProduct": null,
+    "serviceVersion": null,
+    "scripts": null
+  },
+  # Full url with protocol, port, and path if existing
+  "location": "tcp://127.0.0.1:3306"
+}
+```
diff --git a/scripts/docs.build.js b/scripts/docs.build.js
@@ -120,7 +120,7 @@ async function createDocFilesFromDir(relPath, targetPath, dirNames) {
     const filePathInRepo = relPath.replace(/^githubRepo\//, "");
     const readmeWithEditUrl = matter.stringify(content, {
       ...frontmatter,
-      custom_edit_url: `https://github.com/${config.repository}/edit/master/${filePathInRepo}/${dirName}/README.md`,
+      custom_edit_url: `https://github.com/${config.repository}/edit/master/${filePathInRepo}/${dirName}/README.md.gotmpl`,
     });
 
     // Skip File if its marked as "hidden" in its frontmatter