Skip to content

Conversation

@robertbrignull
Copy link
Contributor

@robertbrignull robertbrignull commented Nov 29, 2021

Uploads CodeQL databases using the uploads.github.com domain, if doing so is enabled/supported for that repository. We will slowly be enabling this feature and this change allows the codeql-action to follow along and use the correct upload mechanism. Should be a no-op change until we start enabling the feature.

Merge / deployment checklist

  • Confirm this change is backwards compatible with existing workflows.
  • Confirm the readme has been updated if necessary.
  • Confirm the changelog has been updated if necessary.

@robertbrignull robertbrignull requested a review from a team as a code owner November 29, 2021 12:18
@robertbrignull robertbrignull force-pushed the robertbrignull/upload_domain branch from 59ea2f8 to ad0a233 Compare November 29, 2021 12:32
Copy link
Contributor

@aeisenberg aeisenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like there's a codescanning warning that should be addressed. But otherwise seems reasonable.

repo: repositoryNwo.repo,
language,
name: `${language}-database`,
data: payload,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like this payload data needs to be sanitized. Good catch code scanning. :) Also below.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have any suggestions for how to sanitize it? Both how to make the code safer, and how to make the query not flag it.

I'm struggling to suggest anything we could reasonably do. The file contents is going straight into the request body and then I argue it's up to the server-side to only accept safe files. And soon it will do and the endpoint will only accept files that are CodeQL databases.

I've had a look at the implementation of bundleDb and there is admittedly a potentially dodgy bit where we check if the file exists already and just return. This does give a malicious actor a way to put a file in that location and have us use it. This could allow a sneaky attack where some seemingly innocuous code that's just moving a file into a directory actually causes us to upload it.

However, it's also worth remembering that all of this is effectively untrusted code. Anything that runs in the action, any other user could do too. If someone wants to upload a random file to this API method, they can do it without invoking this piece of code.

Copy link
Contributor

@aeisenberg aeisenberg Nov 29, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good points. Maybe there's nothing we need to do inside the action. Though, maybe a comment describing the fact that there is a possibility of uploading untrusted code, but that would be checked on the server side would be a good idea.

Also, would it be safer to delete the database bundle (if it exists) immediately before calling bundleDb? This would avoid the no-op behaviour you described. It wouldn't prevent all malicious uploads, but at least it would ensure that anything created by the codeql CLI is indeed a database.

Inspecting the CLI code, if the output file exists, then the command throws an exception. So, probably a good idea to delete before creating anyway.

edit On further thought, I think deleting before creating is the best thing to do since if the database already exists, then it's mostly likely an error. Perhaps, being left over from a previous job or step.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if the database already exists, then it's mostly likely an error. Perhaps, being left over from a previous job or step.

So bundleDb is called from multiple places. There's uploading a database to remote queries, and there's the debug setting of uploading a database as an actions artifact. So if we re-bundle every time it could result in duplicate work, but likely not much as the debug setting is likely to be pretty rarely used, and zipping up the database is moderately quick compared to creating the database.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've updated bundleDb to delete the file if it exists, and added some comments explaining the situation. Let me know if you're happy with this or would rather go with not deleting the file.

@robertbrignull robertbrignull force-pushed the robertbrignull/upload_domain branch from ad0a233 to 460d053 Compare December 1, 2021 12:13
@robertbrignull
Copy link
Contributor Author

There seems to be a problem with CI (which is happening on master too) but I've tested this manually and it is working and successfully uploading a database to the right place.

Copy link
Contributor

@aeisenberg aeisenberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all looks good to me. I'll rerun the cancelled jobs to see if they pass now. Looks like they were cancelled because they timed out.

@aeisenberg
Copy link
Contributor

I think the process is hanging because dotnet is waiting for user input to accept some terms and conditions or something like that.

@robertbrignull robertbrignull merged commit f44219c into main Dec 6, 2021
@robertbrignull robertbrignull deleted the robertbrignull/upload_domain branch December 6, 2021 10:24
@github-actions github-actions bot mentioned this pull request Dec 6, 2021
5 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants