Skip to content

ci: compute commit range for rerun workflows - #27775

Closed
gkalpak wants to merge 2 commits into
angular:masterfrom
gkalpak:ci-commit-range-on-rerun
Closed

gkalpak wants to merge 2 commits into
angular:masterfrom
gkalpak:ci-commit-range-on-rerun

Conversation

@gkalpak

@gkalpak gkalpak commented Dec 20, 2018

Copy link
Copy Markdown
Member

On push builds, CircleCI provides CIRCLE_COMPARE_URL, which we use to extract the commit range for a given build. When a workflow is rerun (e.g. to recover from a flaked job), CIRCLE_COMPARE_URL is not defined, causing some jobs to fail.

This commit fixes it by retrieving the compare URL from the original workflow. It uses a slow process involving a (potentially large) number of requests to CircleCI API.
It depends on the (undocumented) fact, that the workspace_id is the same on all rerun workflows and the same as the original workflow_id.

@gkalpak gkalpak added area: build & ci Related the build and CI infrastructure of the project action: review The PR is still awaiting reviews from at least one requested reviewer target: patch This PR is targeted for the next patch release labels Dec 20, 2018
@ngbot ngbot Bot added this to the needsTriage milestone Dec 20, 2018
@gkalpak
gkalpak requested a review from IgorMinar December 20, 2018 22:56

@IgorMinar IgorMinar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gkalpak

gkalpak commented Dec 22, 2018

Copy link
Copy Markdown
Member Author

how slow is slow? a few seconds? a minute? a few minutes?

It depends on how many builds there have been (on the same branch) since the workflow you are rerunning. For example, if you are rerunning the latest workflow, it would be less than 1sec. But if you are rerunning a workflow from 10 days ago, while there have been hundreds of jobs on the same branch, it may take several seconds or minutes.

In a nutshell, we keep "walking" the completed builds backwards (via API requests) until we find the original workflow (i.e. the one that we clicked "rerun" on). So, depending on how many builds there have been since that original workflow, we need to make that many API requests.

Note that this slow path will only be used on non-PR builds and only if CIRCLE_COMPARE_URL is not defined. So, (afaict) it will only affect:

  • Rerun workflows for push builds.
  • Cron-scheduled workflows (i.e. only aio_monitoring atm), which is not a rerun workflow and that we will only make one API request (~1sec).

also could you credit the original code in https://circleci.com/orbs/registry/orb/iynere/compare-url / https://github.com/iynere/compare-url-orb

Happy to (although all they have in common is the idea of querying the builds via the CircleCI API).

On push builds, CircleCI provides `CIRCLE_COMPARE_URL`, which we use to
extract the commit range for a given build. When a workflow is rerun
(e.g. to recover from a flaked job), `CIRCLE_COMPARE_URL` is not
defined, causing some jobs to fail.

This commit fixes it by retrieving the compare URL from the original
workflow. It uses a slow process involving a (potentially large) number
of requests to CircleCI API.
It depends on the (undocumented) fact, that the `workspace_id` is the
same on all rerun workflows and the same as the original `workflow_id`.
@gkalpak
gkalpak force-pushed the ci-commit-range-on-rerun branch from 0e7b760 to a1311de Compare December 22, 2018 10:43
@gkalpak

gkalpak commented Dec 22, 2018

Copy link
Copy Markdown
Member Author

@IgorMinar, PTAL.

@IgorMinar IgorMinar added action: merge The PR is ready for merge by the caretaker and removed action: review The PR is still awaiting reviews from at least one requested reviewer labels Jan 8, 2019
@IgorMinar

Copy link
Copy Markdown
Contributor

lgtm. thanks.

and sorry that I didn't follow up on this sooner. I didn't realize that it was blocked on me.

@IgorMinar

Copy link
Copy Markdown
Contributor

@gkalpak can you please clarify one more thing: why didn't you use the "official" workaround https://circleci.com/orbs/registry/orb/iynere/compare-url and instead implemented our own? I'm just curious because you must have had a reason...

@kara kara closed this in d72ae7b Jan 8, 2019
kara pushed a commit that referenced this pull request Jan 8, 2019
On push builds, CircleCI provides `CIRCLE_COMPARE_URL`, which we use to
extract the commit range for a given build. When a workflow is rerun
(e.g. to recover from a flaked job), `CIRCLE_COMPARE_URL` is not
defined, causing some jobs to fail.

This commit fixes it by retrieving the compare URL from the original
workflow. It uses a slow process involving a (potentially large) number
of requests to CircleCI API.
It depends on the (undocumented) fact, that the `workspace_id` is the
same on all rerun workflows and the same as the original `workflow_id`.

PR Close #27775
@gkalpak

gkalpak commented Jan 8, 2019

Copy link
Copy Markdown
Member Author

why didn't you use the "official" workaround https://circleci.com/orbs/registry/orb/iynere/compare-url

In descending order of importantnce:

  1. AFAICT (by looking at the code), it would only work if the rerun workflow is the latest workflow on the branch (which is not guaranteed to be true).
  2. It is less efficient (making unnecessary CircleCI requests for builds on different branches, having to install extra dependencies, persisting files to the workspace (as a means of passing the result to the calling job), etc.).
  3. It is slightly more complicated to setup and consume than our own script.
  4. It is more complicated than needed for our usecase (e.g. handling different git providers, handling newly created branches, etc.).

@IgorMinar

IgorMinar commented Jan 9, 2019 via email

Copy link
Copy Markdown
Contributor

gkalpak added a commit to gkalpak/angular that referenced this pull request Jan 9, 2019
@gkalpak

gkalpak commented Jan 9, 2019

Copy link
Copy Markdown
Member Author

#28010

@gkalpak
gkalpak deleted the ci-commit-range-on-rerun branch January 9, 2019 12:55
AndrewKushnir pushed a commit that referenced this pull request Jan 10, 2019
AndrewKushnir pushed a commit that referenced this pull request Jan 10, 2019
AndrewKushnir pushed a commit that referenced this pull request Jan 16, 2019
On push builds, CircleCI provides `CIRCLE_COMPARE_URL`, which we use to
extract the commit range for a given build. When a workflow is rerun
(e.g. to recover from a flaked job), `CIRCLE_COMPARE_URL` is not
defined, causing some jobs to fail.

This commit fixes it by retrieving the compare URL from the original
workflow. It uses a slow process involving a (potentially large) number
of requests to CircleCI API.
It depends on the (undocumented) fact, that the `workspace_id` is the
same on all rerun workflows and the same as the original `workflow_id`.

PR Close #27775
AndrewKushnir pushed a commit that referenced this pull request Jan 16, 2019
@angular-automatic-lock-bot

Copy link
Copy Markdown

This issue has been automatically locked due to inactivity.
Please file a new issue if you are encountering a similar or related problem.

Read more about our automatic conversation locking policy.

This action has been performed automatically by a bot.

@angular-automatic-lock-bot angular-automatic-lock-bot Bot locked and limited conversation to collaborators Sep 14, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

action: merge The PR is ready for merge by the caretaker area: build & ci Related the build and CI infrastructure of the project cla: yes target: patch This PR is targeted for the next patch release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants