You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If the repository is cloned to a location where the path leading to the repository contains spaces, Makefile commands can fail.
Try the following:
$ cd /tmp
$ git clone https://github.com/securecodebox/secureCodeBox "secure Code Box"
$ cd"secure Code Box"
$ make create-new-scanner NAME=test
find: /tmp/secure: No such file or directory
find: Code: No such file or directory
find: Box: No such file or directory
rm -rf /tmp/secure Code Box/scanners/test
cp -r /tmp/secure Code Box/.templates/new-scanner/ /tmp/secure Code Box/scanners/test
cp: Box/scanners/test is not a directory
Expected behavior
The Makefile should create a new scanner, instead of choking on the space and dying (or, even worse, copying stuff around randomly based on prefixes and polluting the file system of the user, or deleting their files, or...).
System (please complete the following information):
Should apply equally to Mac OS (tested), Linux (untested), Windows with WSL (tested)
Ideology
Yes, I know, spaces in folder names are considered an antipattern by some people. Still, in many situations (e.g., if you are using Windows with WSL and use your full name as username, and you drop your stuff in the "My Files" area of the system), you are pretty much guaranteed to encounter spaces. Instead of forcing developers to switch their workflows and where they store their code, we should just support this use case, if at all possible. And if we do not want to support this for whatever reason, we should ensure that the Makefile checks if there are spaces in the folder path, and refuse to perform operations that may break because of them, instead of bugging out and doing weird stuff that may or may not end up creating and deleting files in random places.
This seems to be a problem caused by the fix to #754 in commit eceb125 (edit: not caused by this directly, but probably related to it), where all quotes were removed from the Makefile. This breaks support for spaces in the paths (edit: actually, it doesn't), and it also breaks the sed call in the find -exec part of the Makefile, which will now return:
make create-new-scanner NAME=ffuf
rm -rf /tmp/secureCOdeBox/scanners/ffuf
cp -r /tmp/secureCOdeBox/.templates/new-scanner/ /tmp/secureCOdeBox/scanners/ffuf
find /tmp/secureCOdeBox/scanners/ffuf -type f ! -name tmp \
-exec sed -n s/new-scanner/ffuf/g;w /tmp/secureCOdeBox/scanners/ffuf/tmp {} \; \
-exec mv /tmp/secureCOdeBox/scanners/ffuf/tmp {} \;
find: -exec: no terminating ";" or "+"
11:13 up 3:23, 2 users, load averages: 2,10 2,20 2,11
USER TTY FROM LOGIN@ IDLE WHAT
mv /tmp/secureCOdeBox/scanners/ffuf/templates/new-scanner-parse-definition.yaml \
/tmp/secureCOdeBox/scanners/ffuf/templates/ffuf-parse-definition.yaml
mv /tmp/secureCOdeBox/scanners/ffuf/templates/new-scanner-scan-type.yaml \
/tmp/secureCOdeBox/scanners/ffuf/templates/ffuf-scan-type.yaml
Reverting the commit and adding the quotes back in does not fix the original issue, but it at least fixes the find -exec sed call.
I think we need some test cases for our Makefiles...
So, if the shell expects and can interpret a quoted string, then you should use quotes. Where the shell doesn't expect or won't correctly interpret a quoted string, you should not use quotes. (SO)
To find all NPM packages. This results in a list of paths separated by spaces. Later we loop over them with shell for loop, which treats any whitespcae for word split. First approach is to use while read which splits only on newline. But the paths in the list are splitted by spaces...
Activity
malexmave commentedon Sep 13, 2022
This seems to be a problem caused by the fix to #754 in commit eceb125 (edit: not caused by this directly, but probably related to it), where all quotes were removed from the Makefile. This
breaks support for spaces in the paths(edit: actually, it doesn't), and it also breaks thesedcall in thefind -execpart of the Makefile, which will now return:Reverting the commit and adding the quotes back in does not fix the original issue, but it at least fixes the
find -exec sedcall.I think we need some test cases for our Makefiles...
Weltraumschaf commentedon Sep 13, 2022
The important part missed by me is:
secureCodeBox#1353 Quote project path to find package list
secureCodeBox#1353 Quote paths in Makefile passed to shell
Weltraumschaf commentedon Sep 13, 2022
Houston, we have a problem!
We have this "sophsticated" find:
To find all NPM packages. This results in a list of paths separated by spaces. Later we loop over them with shell for loop, which treats any whitespcae for word split. First approach is to use
while readwhich splits only on newline. But the paths in the list are splitted by spaces...secureCodeBox#1353 Movel help target to bottom of file
secureCodeBox#1353 Fix phony targets
Weltraumschaf commentedon Sep 14, 2022
First fixes: #1357
But IMHO this is a major problem.
secureCodeBox#1353 Quote project path to find package list
secureCodeBox#1353 Quote paths in Makefile passed to shell
288 remaining items
secureCodeBox#1353 Remove quotes bc they're printed literaly in the o…
secureCodeBox#1353 Implement check for spaces in project path
secureCodeBox#1353 USe built in error function to stop on missing nam…
secureCodeBox#1353 Allso include spaces check in operator make file
secureCodeBox#1353 Allso include spaces check in auto-discovery make …
secureCodeBox#1353 Allso include spaces check in hooks make file
secureCodeBox#1353 Allso include spaces check in scanners make file
secureCodeBox#1353 Use PHONY at the target it self
secureCodeBox#1353 Add common makle checks to the scanner sub module …
secureCodeBox#1353 The ci jobs execute shells w/ /bin/sh so we can't …
secureCodeBox#1353 Show errornous path to help understand the error
secureCodeBox#1353 Exctract prerequisites check into own make include
secureCodeBox#1353 Add python to error message
secureCodeBox#1353 Use extracted prerequistes.mk instead of common.mk
secureCodeBox#1353 Use extracted include and remove duplicated code