Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
66 commits
Select commit Hold shift + click to select a range
08f4781
Adding template folder for new scanner files
SebieF Sep 28, 2021
856d56f
Adding create-new-scanner command to Makefile
SebieF Sep 28, 2021
c60ce4f
Updating Helm Docs
SebieF Sep 28, 2021
6f874f0
Adding quotes to Chart.yaml template
SebieF Sep 29, 2021
b6c290b
Merge remote-tracking branch 'origin/consistency/new-scanner-makefile…
SebieF Sep 30, 2021
2e4a5e9
Making new-scanner-scan-type.yaml consistent with existing scan type …
SebieF Sep 30, 2021
8185951
Adding empty result files and licenses
SebieF Oct 4, 2021
19228f3
Adding empty result files and licenses
SebieF Oct 4, 2021
0448940
Adding handling and testing of empty file to ssh-scan
SebieF Oct 4, 2021
fde224c
ci(ci): simplify scanner makefile build
nigthknight Oct 5, 2021
b0aae7d
ci(ci): add nuclei to matrix build
nigthknight Oct 5, 2021
dc45669
ci(ci): add ssh-scan to matrix build
nigthknight Oct 5, 2021
11c57a4
ci(ci): add sslyze to matrix build with make
nigthknight Oct 5, 2021
794ae88
ci(ci): add trivy to matrix make build
nigthknight Oct 5, 2021
b06a2f4
ci(ci): add wpscan to matrix make build
nigthknight Oct 5, 2021
d60055b
Removing unnecessary logging
SebieF Oct 5, 2021
e1cdb72
Adding handling of empty files
SebieF Oct 5, 2021
4bf953e
Adding const to findings
SebieF Oct 5, 2021
4edea4d
Adding empty file tests where missing
SebieF Oct 5, 2021
f31f834
Updating snapshot for kubeaudit juiceshop test
SebieF Oct 5, 2021
6b5036f
Merge branch 'main' into consistency/empty-files-parser-tests
SebieF Oct 5, 2021
c123074
Updating snapshot name
SebieF Oct 5, 2021
9fba067
Merge branch 'main' into consistency/new-scanner-makefile
SebieF Oct 5, 2021
0768283
Make template files consistent with current used files
SebieF Oct 5, 2021
a1a39e6
Replacing old name nmap with generic name
SebieF Oct 5, 2021
971a573
Making codeclimate happy
SebieF Oct 5, 2021
497bda8
ci(ci): disable cascading scans for sslyze
nigthknight Oct 11, 2021
f063ffc
ci(ci): add zap to matrix make build
nigthknight Oct 11, 2021
7747711
ci(ci): add angularjs-csti to matrix make build
nigthknight Oct 11, 2021
f984666
ci(ci): add gitleaks to matrix make build
nigthknight Oct 11, 2021
a2c3283
build: change parser to error on invaid findings
nigthknight Oct 11, 2021
c5b77bf
ci(ci): add kube-hunter to matrix make build
nigthknight Oct 11, 2021
bf4558b
ci(ci): add kubeaudit to matrix make build
nigthknight Oct 11, 2021
bb1dbdc
ci(ci): add ncrack to matrix make build
nigthknight Oct 11, 2021
41cc70b
fix(kube-hunter): fix makefile syntax
nigthknight Oct 11, 2021
21ab7d9
ci(ci): add nmap to matrix make build
nigthknight Oct 11, 2021
ab51315
ci(ci): add nikto to matrix make build
nigthknight Oct 11, 2021
50a8919
ci(ci): add typo3scan to matrix make build
nigthknight Oct 11, 2021
44d4501
ci(ci): add whatweb to matrix make build
nigthknight Oct 11, 2021
ae9ed6c
ci(ci): add screenshooter to matrix make build
nigthknight Oct 11, 2021
9eb0087
ci(ci): disable fail-fast for make build
nigthknight Oct 12, 2021
8217c4c
ci(ci): add debugging steps on failure
nigthknight Oct 12, 2021
2709126
build(common): default BASE_IMG_TAG to commit hash
nigthknight Oct 12, 2021
1b2f82a
ci(ci): remove unnecessary steps
nigthknight Oct 12, 2021
32eeb30
build(parser-sdk): add target to import sdk
nigthknight Oct 12, 2021
e2761b7
Merge branch 'main' into consistency/empty-files-parser-tests
SebieF Oct 12, 2021
67e7fb0
Extracting category detection to a separate function
SebieF Oct 12, 2021
e1a1968
Removing autogenerated files
SebieF Oct 12, 2021
1fd5877
Removing optional file that is not always used
SebieF Oct 12, 2021
8819b39
Fixing typo
SebieF Oct 12, 2021
adba333
Updating Helm Docs
SebieF Oct 12, 2021
b766b3f
Update ADR-0009 with decision
malexmave Oct 12, 2021
f75ae54
Merge branch 'main' into ci/switch-scanner-build
nigthknight Oct 12, 2021
2ec69ff
ci(ci): add cmseek to matrix make build
nigthknight Oct 12, 2021
7a57d32
build: add target to deploy old-joomla
nigthknight Oct 12, 2021
c8deec5
ci(ci): add git-repo-scanner to make matrix
nigthknight Oct 12, 2021
0738cfa
Merge branch 'main' into consistency/new-scanner-makefile
SebieF Oct 13, 2021
64f3964
Merge pull request #706 from secureCodeBox/consistency/new-scanner-ma…
nigthknight Oct 13, 2021
750a50d
Merge branch 'main' into consistency/empty-files-parser-tests
SebieF Oct 13, 2021
55c0dc8
Implement init container support
malexmave Oct 13, 2021
90a635d
Merge pull request #718 from secureCodeBox/doc/adr-0009
J12934 Oct 13, 2021
f0973e8
Update CRDs in correct location for Helm
malexmave Oct 13, 2021
a125bea
Merge pull request #705 from secureCodeBox/consistency/empty-files-pa…
J12934 Oct 13, 2021
c1e5321
Merge pull request #712 from secureCodeBox/ci/switch-scanner-build
J12934 Oct 13, 2021
0e94265
Merge pull request #724 from secureCodeBox/feature/init-containers
malexmave Oct 14, 2021
eefc9a5
Upgrading amass from 3.14 to 3.14.1
secureCodeBoxBot Oct 15, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
469 changes: 52 additions & 417 deletions .github/workflows/ci.yaml

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions .templates/new-scanner/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

*.tar
31 changes: 31 additions & 0 deletions .templates/new-scanner/.helm-docs.gotmpl
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
{{- /*
SPDX-FileCopyrightText: 2021 iteratec GmbH

SPDX-License-Identifier: Apache-2.0
*/ -}}

{{- define "extra.docsSection" -}}
---
title: "new-scanner"
category: "scanner"
type: "Network"
state: "released"
appVersion: "{{ template "chart.appVersion" . }}"
usecase: "NEW SCANNER USECASE"
---

ADD A [LOGO]() HERE!

{{- end }}

{{- define "extra.dockerDeploymentSection" -}}
## Supported Tags
- `latest` (represents the latest stable release build)
- tagged releases, e.g. `{{ template "chart.appVersion" . }}`
{{- end }}

{{- define "extra.chartAboutSection" -}}
## What is new-scanner?

Please write some information about the new scanner.
{{- end }}
40 changes: 40 additions & 0 deletions .templates/new-scanner/.helmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0
# Patterns to ignore when building packages.
# This supports shell glob matching, relative path matching, and
# negation (prefixed with !). Only one pattern per line.
.DS_Store
# Common VCS dirs
.git/
.gitignore
.bzr/
.bzrignore
.hg/
.hgignore
.svn/
# Common backup files
*.swp
*.bak
*.tmp
*~
# Various IDEs
.project
.idea/
*.tmproj
.vscode/
# Node.js files
node_modules/*
package.json
package-lock.json
src/*
config/*
Dockerfile
.dockerignore
*.tar
parser/*
scanner/*
integration-tests/*
examples/*
docs/*
Makefile
26 changes: 26 additions & 0 deletions .templates/new-scanner/Chart.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

apiVersion: v2
name: new-scanner
description: A Helm chart for the new-scanner security Scanner that integrates with the secureCodeBox.

type: application
# version - gets automatically set to the secureCodeBox release version when the helm charts gets published
version: v3.1.0-alpha1
appVersion: "" # Set appVersion
kubeVersion: ">=v1.11.0-0"

keywords:
- security
- new-scanner
- scanner
- secureCodeBox
home: https://docs.securecodebox.io/docs/scanners/new-scanner
icon: https://docs.securecodebox.io/img/integrationIcons/new-scanner.svg # Upload new icon
sources:
- https://github.com/secureCodeBox/secureCodeBox
maintainers:
- name: iteratec GmbH
email: secureCodeBox@iteratec.com
12 changes: 12 additions & 0 deletions .templates/new-scanner/Makefile
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#!/usr/bin/make -f
#
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0
#

include_guard = set
scanner = new-scanner
custom_scanner = set

include ../../scanners.mk
49 changes: 49 additions & 0 deletions .templates/new-scanner/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
---
title: "new-scanner"
category: "scanner"
type: "Network"
state: "released"
appVersion: ""
usecase: "NEW SCANNER USECASE"
---

ADD A [LOGO]() HERE!

<!--
SPDX-FileCopyrightText: 2021 iteratec GmbH

SPDX-License-Identifier: Apache-2.0
-->
<!--
.: IMPORTANT! :.
--------------------------
This file is generated automatically with `helm-docs` based on the following template files:
- ./.helm-docs/templates.gotmpl (general template data for all charts)
- ./chart-folder/.helm-docs.gotmpl (chart specific template data)

Please be aware of that and apply your changes only within those template files instead of this file.
Otherwise your changes will be reverted/overwritten automatically due to the build process `./.github/workflows/helm-docs.yaml`
--------------------------
-->

<p align="center">
<a href="https://opensource.org/licenses/Apache-2.0"><img alt="License Apache-2.0" src="https://img.shields.io/badge/License-Apache%202.0-blue.svg"/></a>
<a href="https://github.com/secureCodeBox/secureCodeBox/releases/latest"><img alt="GitHub release (latest SemVer)" src="https://img.shields.io/github/v/release/secureCodeBox/secureCodeBox?sort=semver"/></a>
<a href="https://owasp.org/www-project-securecodebox/"><img alt="OWASP Incubator Project" src="https://img.shields.io/badge/OWASP-Incubator%20Project-365EAA"/></a>
<a href="https://artifacthub.io/packages/search?repo=securecodebox"><img alt="Artifact HUB" src="https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/securecodebox"/></a>
<a href="https://github.com/secureCodeBox/secureCodeBox/"><img alt="GitHub Repo stars" src="https://img.shields.io/github/stars/secureCodeBox/secureCodeBox?logo=GitHub"/></a>
<a href="https://twitter.com/securecodebox"><img alt="Twitter Follower" src="https://img.shields.io/twitter/follow/securecodebox?style=flat&color=blue&logo=twitter"/></a>
</p>

## What is new-scanner?

Please write some information about the new scanner.

## Deployment
The new-scanner chart can be deployed via helm:

```bash
# Install HelmChart (use -n to configure another namespace)
helm upgrade --install new-scanner secureCodeBox/new-scanner
```

Empty file.
Empty file.
14 changes: 14 additions & 0 deletions .templates/new-scanner/examples/localhost/scan.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

apiVersion: "execution.securecodebox.io/v1"
kind: Scan
metadata:
name: "new-scanner-localhost"
spec:
scanType: "new-scanner"
parameters:
# Note: Localhost here isn't "your" localhost, but the scanner container.
# This container doesn't have any ports open...
- localhost
5 changes: 5 additions & 0 deletions .templates/new-scanner/parser/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

node_modules/
24 changes: 24 additions & 0 deletions .templates/new-scanner/parser/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

# No additional dependencies
ARG namespace
ARG baseImageTag
FROM ${namespace:-securecodebox}/parser-sdk-nodejs:${baseImageTag:-latest}
WORKDIR /home/app/parser-wrapper/parser/
COPY --chown=app:app ./parser.js ./parser.js

# Additional packages
# ARG namespace
# ARG baseImageTag
# FROM node:14-alpine as build
# RUN mkdir -p /home/app
# WORKDIR /home/app
# COPY package.json package-lock.json ./
# RUN npm ci --production
#
# FROM ${namespace:-securecodebox}/parser-sdk-nodejs:${baseImageTag:-latest}
# WORKDIR /home/app/parser-wrapper/parser/
# COPY --from=build --chown=app:app /home/app/node_modules/ ./node_modules/
# COPY --chown=app:app ./parser.js ./parser.js
2 changes: 2 additions & 0 deletions .templates/new-scanner/parser/__testFiles__/empty.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
[
]
3 changes: 3 additions & 0 deletions .templates/new-scanner/parser/__testFiles__/example.com.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
[
{"target":"http://example.com"}
]
33 changes: 33 additions & 0 deletions .templates/new-scanner/parser/parser.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
// SPDX-FileCopyrightText: 2021 iteratec GmbH
//
// SPDX-License-Identifier: Apache-2.0

async function parse(fileContent) {
const targets = parseResultFile(fileContent);
return transformToFindings(targets);
}

function transformToFindings(targets) {

const targetFindings = [];

// Code to transform the scanner results to scb findings

return [...targetFindings];
}

/**
* Parses a given new-scanner result file and extracts all targets
* @param {*} fileContent
*/
function parseResultFile(fileContent) {
let targetList = [];

for(const rawTarget of fileContent) {
// Code to transform raw target findings to usable js format
// If scanner is only able to output xml files, you have to transform them first (look at nmap parser for example)
}
return targetList;
}

module.exports.parse = parse;
39 changes: 39 additions & 0 deletions .templates/new-scanner/parser/parser.test.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
// SPDX-FileCopyrightText: 2021 iteratec GmbH
//
// SPDX-License-Identifier: Apache-2.0

const fs = require("fs");
const util = require("util");

const {
validateParser,
} = require("@securecodebox/parser-sdk-nodejs/parser-utils");

// eslint-disable-next-line security/detect-non-literal-fs-filename
const readFile = util.promisify(fs.readFile);

const { parse } = require("./parser");

test("should properly parse new-scanner json file", async () => {
const fileContent = JSON.parse(
await readFile(__dirname + "/__testFiles__/example.com.json", {
encoding: "utf8",
})
);
const findings = await parse(fileContent);
// validate findings
await expect(validateParser(findings)).resolves.toBeUndefined();
expect(findings).toMatchInlineSnapshot();
});

test("should properly parse empty json file", async () => {
const fileContent = JSON.parse(
await readFile(__dirname + "/__testFiles__/empty.json", {
encoding: "utf8",
})
);
const findings = await parse(fileContent);
// validate findings
await expect(validateParser(findings)).resolves.toBeUndefined();
expect(findings).toMatchInlineSnapshot();
});
6 changes: 6 additions & 0 deletions .templates/new-scanner/scanner/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

# Write your dockerfile for the scanner new-scanner here
# Alternatively, you can use an existing image from docker-hub
15 changes: 15 additions & 0 deletions .templates/new-scanner/templates/cascading-rules.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

# We only want to import the default cascading rules if they are enabled
{{ if .Values.cascadingRules.enabled }}
# The CascadingRules are not directly in the /templates directory as their curly bracket syntax clashes with helms templates ... :(
# We import them as raw files to avoid these clashes as escaping them is even more messy
{{ range $path, $_ := .Files.Glob "cascading-rules/*" }}
# Include File
{{ $.Files.Get $path }}
# Separate multiple files
---
{{ end }}
{{ end }}
14 changes: 14 additions & 0 deletions .templates/new-scanner/templates/new-scanner-parse-definition.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

apiVersion: "execution.securecodebox.io/v1"
kind: ParseDefinition
metadata:
name: "new-scanner-json"
spec:
image: "{{ .Values.parser.image.repository }}:{{ .Values.parser.image.tag | default .Chart.Version }}"
imagePullPolicy: {{ .Values.parser.image.pullPolicy }}
ttlSecondsAfterFinished: {{ .Values.parser.ttlSecondsAfterFinished }}
env:
{{- toYaml .Values.parser.env | nindent 4 }}
44 changes: 44 additions & 0 deletions .templates/new-scanner/templates/new-scanner-scan-type.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# SPDX-FileCopyrightText: 2021 iteratec GmbH
#
# SPDX-License-Identifier: Apache-2.0

apiVersion: "execution.securecodebox.io/v1"
kind: ScanType
metadata:
name: "new-scanner{{ .Values.scanner.nameAppend | default ""}}"
spec:
extractResults:
type: new-scanner-json
location: "/home/securecodebox/new-scanner-results.json"
jobTemplate:
spec:
{{- if .Values.scanner.ttlSecondsAfterFinished }}
ttlSecondsAfterFinished: {{ .Values.scanner.ttlSecondsAfterFinished }}
{{- end }}
backoffLimit: {{ .Values.scanner.backoffLimit }}
{{- if .Values.scanner.activeDeadlineSeconds }}
activeDeadlineSeconds: {{ .Values.scanner.activeDeadlineSeconds }}
{{- end }}
template:
spec:
restartPolicy: OnFailure
containers:
- name: new-scanner
image: "{{ .Values.scanner.image.repository }}:{{ .Values.scanner.image.tag | default .Chart.AppVersion }}"
imagePullPolicy: {{ .Values.scanner.image.pullPolicy }}
command:
- "new-scanner"
- "/home/securecodebox/new-scanner-results.json" # Replace output command
resources:
{{- toYaml .Values.scanner.resources | nindent 16 }}
securityContext:
{{- toYaml .Values.scanner.securityContext | nindent 16 }}
env:
{{- toYaml .Values.scanner.env | nindent 16 }}
volumeMounts:
{{- toYaml .Values.scanner.extraVolumeMounts | nindent 16 }}
{{- if .Values.scanner.extraContainers }}
{{- toYaml .Values.scanner.extraContainers | nindent 12 }}
{{- end }}
volumes:
{{- toYaml .Values.scanner.extraVolumes | nindent 12 }}
Loading