Skip to content

Build Scanners With New Make Targets - #712

Merged
J12934 merged 29 commits into
mainfrom
ci/switch-scanner-build
Oct 13, 2021
Merged

J12934 merged 29 commits into
mainfrom
ci/switch-scanner-build

Conversation

@nigthknight

@nigthknight nigthknight commented Oct 8, 2021 •

Copy link
Copy Markdown
Member

This PR, if applied, will

  • ci(ci): simplify scanner makefile build
  • ci(ci): add nuclei to matrix build
  • ci(ci): add ssh-scan to matrix build
  • ci(ci): add sslyze to matrix build with make
  • ci(ci): add trivy to matrix make build
  • ci(ci): add wpscan to matrix make build
  • ci(ci): add zap to matrix make build
  • ci(ci): add angular-csti-scanner to matrix make build
  • ci(ci): add cmseek to matrix make build
  • ci(ci): add git-repo-scanner to matrix make build
  • build(cmseek): add target to deploy old-joomla
  • build(scanner): enable finding format validation on helm install

I simplified the build steps for scanners by removing the docker build
and push action and use the makefile targets instead.
This also made the docker meta action unnecessar.

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
I added ssh-scan to matrix build and moved the integration tests

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
I moved sslyze to the new makefile job and moved the integration tests
as well

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight nigthknight added the ci Changes to the continuous integration setup label Oct 8, 2021
@nigthknight nigthknight self-assigned this Oct 8, 2021
@EndPositive

Copy link
Copy Markdown
Contributor

@fuhrmeistery what's the reason for overwriting the deploy-with-scanner and integration-test targets in the Makefiles for ssh-scan, trivy, etc.? These targets are inherited from scanners.mk and should work for every scanner.

Only if there are scanner specific changes required for deployment (e.g. nuclei template cache) then you should override the target.

@nigthknight

Copy link
Copy Markdown
Member Author

@fuhrmeistery what's the reason for overwriting the deploy-with-scanner and integration-test targets in the Makefiles for ssh-scan, trivy, etc.? These targets are inherited from scanners.mk and should work for every scanner.

Only if there are scanner specific changes required for deployment (e.g. nuclei template cache) then you should override the target.

@EndPositive The reason is that we want to move the integration tests for the scanners into the scanner directory to have all scanner related files in one place. The goal is to update the integration-test target in the future, but I want to avoid changing it while migrating the scanner build jobs.

@EndPositive

Copy link
Copy Markdown
Contributor

Awesome, makes sense!

we decided the general test for sslyze and the test of cascading scans
in general is sufficient

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch from edd9d0d to 39cc147 Compare October 11, 2021 06:51
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch from 39cc147 to f063ffc Compare October 11, 2021 07:39
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch from 1736b08 to 422a0df Compare October 11, 2021 08:17
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch 2 times, most recently from f66166b to 4b3d91c Compare October 11, 2021 10:33
Prior we introduced finding valitation for our expected findings format.
This commit ensures that all parsers are throwing an error on invalid
formats
Previously changed make targets adding this to a specific scanner are
reverted as well

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch from 49a8ddd to c5b77bf Compare October 11, 2021 12:22
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Like with sslyze we decided to disable the cascading test for ncrack and
rely on the test of cascading scans in general

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch from a1e566c to 44eb52d Compare October 11, 2021 13:08
I removed the old build job for third party scanners because it is not
needed anymore

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight
nigthknight force-pushed the ci/switch-scanner-build branch from 44eb52d to 44d4501 Compare October 11, 2021 13:08
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
fail-fast causes all remaining jobs of this matrix to be canceld if one
job fails. To see the impact of the change on the others it might be
better to let these finish as well.

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
I removed these steps and variable definitions because they are already
handled by make by default

Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
@nigthknight nigthknight linked an issue Oct 12, 2021 that may be closed by this pull request
8 of 9 tasks
nigthknight and others added 4 commits October 12, 2021 16:00
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@iteratec.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@protonmail.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@protonmail.com>
Signed-off-by: Yannik Fuhrmeister <yannik.fuhrmeister@protonmail.com>
@nigthknight
nigthknight marked this pull request as ready for review October 13, 2021 15:50
@nigthknight
nigthknight requested a review from J12934 October 13, 2021 15:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci Changes to the continuous integration setup

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Epic] CI Migration To use Makefile Builds

3 participants