Skip to content

Add retrying to the upload #13

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
robertbrignull merged 6 commits into from
May 1, 2020
Merged

Add retrying to the upload #13

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
1da651c
Add retries to the upload
robertbrignull May 1, 2020
cffc0f7
fix typo
robertbrignull May 1, 2020
e52e34b
remove change to behaviour on 500 errors
robertbrignull May 1, 2020
0c4fc16
only retry on 5xx status codes
robertbrignull May 1, 2020
a23cb1d
include status code is error message
robertbrignull May 1, 2020
129ce28
Update upload-lib.ts
robertbrignull May 1, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 48 additions & 20 deletions lib/upload-lib.js
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

76 changes: 57 additions & 19 deletions src/upload-lib.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,61 @@ export function combineSarifFiles(sarifFiles: string[]): string {
return JSON.stringify(combinedSarif);
}

// Upload the given payload.
// If the request fails then this will retry a small number of times.
async function uploadPayload(payload) {
core.info('Uploading results');

const githubToken = core.getInput('token');
const ph: auth.BearerCredentialHandler = new auth.BearerCredentialHandler(githubToken);
const client = new http.HttpClient('Code Scanning : Upload SARIF', [ph]);
const url = 'https://api.github.com/repos/' + process.env['GITHUB_REPOSITORY'] + '/code-scanning/analysis';

// Make up to 4 attempts to upload, and sleep for these
// number of seconds between each attempt.
// We don't want to backoff too much to avoid wasting action
// minutes, but just waiting a little bit could maybe help.
const backoffPeriods = [1, 5, 15];

for (let attempt = 0; attempt <= backoffPeriods.length; attempt++) {

const res: http.HttpClientResponse = await client.put(url, payload);
core.debug('response status: ' + res.message.statusCode);

const statusCode = res.message.statusCode;
if (statusCode === 202) {
core.info("Successfully uploaded results");
return;
}

const requestID = res.message.headers["x-github-request-id"];

// On any other status code that's not 5xx mark the upload as failed
if (!statusCode || statusCode < 500 || statusCode >= 600) {
core.setFailed('Upload failed (' + requestID + '): (' + statusCode + ') ' + await res.readBody());
return;
}

// On a 5xx status code we may retry the request
if (attempt < backoffPeriods.length) {
// Log the failure as a warning but don't mark the action as failed yet
core.warning('Upload attempt (' + (attempt + 1) + ' of ' + (backoffPeriods.length + 1) +
') failed (' + requestID + '). Retrying in ' + backoffPeriods[attempt] +
' seconds: (' + statusCode + ') ' + await res.readBody());
// Sleep for the backoff period
await new Promise(r => setTimeout(r, backoffPeriods[attempt] * 1000));
continue;
Copy link
Contributor

@anaarmas anaarmas May 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this continue here?

Copy link
Contributor Author

@robertbrignull robertbrignull May 1, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To get to the next iteration of the loop. Technically it's not necessary as this is the last statement of the loop in this branch, but it doesn't hurt as far as I can see.


} else {
// If the upload fails with 5xx then we assume it is a temporary problem
// and not an error that the user has caused or can fix.
// We avoid marking the job as failed to avoid breaking CI workflows.
core.error('Upload failed (' + requestID + '): (' + statusCode + ') ' + await res.readBody());
return;
}
}
}

// Uploads a single sarif file or a directory of sarif files
// depending on what the path happens to refer to.
export async function upload(input: string) {
Expand Down Expand Up @@ -112,25 +167,8 @@ async function uploadFiles(sarifFiles: string[]) {
"tool_names": toolNames,
});

core.info('Uploading results');
const githubToken = core.getInput('token');
const ph: auth.BearerCredentialHandler = new auth.BearerCredentialHandler(githubToken);
const client = new http.HttpClient('Code Scanning : Upload SARIF', [ph]);
const url = 'https://api.github.com/repos/' + process.env['GITHUB_REPOSITORY'] + '/code-scanning/analysis';
const res: http.HttpClientResponse = await client.put(url, payload);
const requestID = res.message.headers["x-github-request-id"];

core.debug('response status: ' + res.message.statusCode);
if (res.message.statusCode === 500) {
// If the upload fails with 500 then we assume it is a temporary problem
// with turbo-scan and not an error that the user has caused or can fix.
// We avoid marking the job as failed to avoid breaking CI workflows.
core.error('Upload failed (' + requestID + '): ' + await res.readBody());
} else if (res.message.statusCode !== 202) {
core.setFailed('Upload failed (' + requestID + '): ' + await res.readBody());
} else {
core.info("Successfully uploaded results");
}
// Make the upload
await uploadPayload(payload);

// Mark that we have made an upload
fs.writeFileSync(sentinelFile, '');
Expand Down