Skip to content

Conversation

@robertbrignull
Copy link
Contributor

@robertbrignull robertbrignull commented May 26, 2020

Tightens up the rules on relative paths in the config file, and introduces new options to reference suites from the codeql bundle.

This PR is pending the decision on the exact names of the suites, and actually introducing those suites into the codeql bundle.

Merge / deployment checklist

  • Run test builds as necessary. Can be on this repository or elsewhere as needed in order to test the change - please include links to tests in other repos!
    • CodeQL using init/analyze actions
    • 3rd party tool using upload action
  • Confirm this change is backwards compatible with existing workflows.
  • Confirm the readme has been updated if necessary.

}

// The set of acceptable values for built-in suites from the codeql bundle
const builtinSuites = ['security-experimental', 'security-and-quality'] as const;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
const builtinSuites = ['security-experimental', 'security-and-quality'] as const;
const builtinSuites = ['security-extended', 'security-and-quality'] as const;

Let's go with security-extended instead, since it's not strictly experimental, but just more queries of varied quality. Thanks!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should have said the suite names in this PR were mostly just a placeholder as I thought that decision was still being made. I'll update this once we know what they'll be.

Copy link
Contributor

@chrisgavin chrisgavin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for improving the input validation. 👍 This looks good once we've settled on the right suite names.

@robertbrignull robertbrignull mentioned this pull request Jun 1, 2020
4 tasks
@robertbrignull
Copy link
Contributor Author

robertbrignull commented Jun 1, 2020

Running all the new suites in all languages at https://github.com/Anthophila/multi-language-test/runs/727232837
I checked that all queries we run that we expected to be run, and that query compilation was fast.

@robertbrignull
Copy link
Contributor Author

Note this change is technically not backwards compitable for anyone using the unadvised ./lgtm-full.qls hack. Anyone using that will get an error, however the error is hopefully fairly understandable. They will need to switch to - uses: security-and-quality.

@robertbrignull
Copy link
Contributor Author

The command line is too long.
##[error]The process 'C:\hostedtoolcache\windows\CodeQL\1.0.0\x64\codeql\codeql.cmd' failed with exit code 1

@robertbrignull
Copy link
Contributor Author

In order to avoid the command line length issue, I've changed it to create a temporary query suite file that just lists the queries. I can't find the comment or email anymore now but I think @Daverlo and @hmakholm had a discussion recently suggesting this plan for another reason, so I hope this is not a bad move to make.

Also note, the issue with the command line length on windows was already there if somebody tried to run a lot of queries. Adding this new syntax just makes it easier to trigger.

@jhutchings1
Copy link
Contributor

Thanks for putting this one together @robertbrignull ! I've tagged our friends in docs to get this on their radar too.

@hmakholm
Copy link
Contributor

hmakholm commented Jun 2, 2020

In order to avoid the command line length issue, I've changed it to create a temporary query suite file that just lists the queries.

Looks good to me.

@robertbrignull
Copy link
Contributor Author

I think this has been tested enough. I manually ran all query suites for all languages, and the integration tests pass. Right now there's not much more we can do.

@robertbrignull robertbrignull merged commit a0d60d5 into master Jun 3, 2020
@robertbrignull robertbrignull deleted the suite_syntax branch June 3, 2020 10:18
This was referenced Jun 3, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants