Skip to content

Retry more.#7880

Merged
sean-mcmanus merged 1 commit intomainfrom
seanmcm/retryMore
Jul 27, 2021
Merged

Retry more.#7880
sean-mcmanus merged 1 commit intomainfrom
seanmcm/retryMore

Conversation

@sean-mcmanus
Copy link
Contributor

@sean-mcmanus sean-mcmanus commented Jul 27, 2021

Add more retries. The failure issue today would fix itself if it retried more, but I'm not sure what the ideal retry amount is because the issue stopped reproing. Maybe we could check telemetry afterwards to see if the number of failures decreased.

@sean-mcmanus sean-mcmanus requested a review from a team July 27, 2021 01:57
@Colengms
Copy link
Contributor

Depending on what caused the error, retries might not help. The outage seemed to repro consistently for certain systems, and for quite a while (an hour?). If a DNS issue, a client machine might keep retrying with the cached IP it got from an earlier DNS request.

@sean-mcmanus sean-mcmanus merged commit bc4457c into main Jul 27, 2021
@sean-mcmanus sean-mcmanus deleted the seanmcm/retryMore branch July 27, 2021 02:34
@sean-mcmanus
Copy link
Contributor Author

Depending on what caused the error, retries might not help. The outage seemed to repro consistently for certain systems, and for quite a while (an hour?). If a DNS issue, a client machine might keep retrying with the cached IP it got from an earlier DNS request.

Yeah, but in this particular incident, I was seeing the failures be random, so retrying fixed it if enough retries were done.

@bobbrow
Copy link
Member

bobbrow commented Jul 27, 2021

I think it's too many retries. This is supposed to be about the developer's connection being bad, not about (rare) CDN incidents. We won't print out the "go get the offline package" message for a really long time for these folks now. This adds 2.5 minutes, right?

@sean-mcmanus
Copy link
Contributor Author

@bobbrow I reduced the max delay between retries to 15 seconds: #7883 .

@github-actions github-actions bot locked and limited conversation to collaborators Sep 10, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants