Skip to content

CI AutoFormat fails when CI is under load. #6272

@ShaharNaveh

Description

@ShaharNaveh

The current CI auto-format implementation waits for all jobs to be finished before proceeding, and that wait time is 1 hour.
In the case where the auto-format job gets scheduled before the rest of the other jobs, it will start it's countdown immediately, but there's a chance for the other jobs to still be in queue because some other CI jobs are running for a pushed commit/other PR, which might take more than 1 hour.

It looks somewhat like this:

gantt
    title CI Timeline
    dateformat HH:mm
    axisFormat %H:%M
    PR Auto Format: 00:15, 1h
    Task 1: 00:00, 45m
    Task 2: 00:45, 45m
    Fails if still waiting: milestone, m1, 01:15, 0m
Loading

There are several approaches that I think could work to solve this:

  1. increase waiting timeout (or remove the timeout).
  2. Make sure that the PR auto format is triggered before any CI jobs.
  3. Make the auto-formag opt-in format by triggering it via a comment on the PR, just like we have take assigning people to issues (https://github.com/RustPython/RustPython/blob/cc2e84b9fc0af690f3281cd4b02e219b4752bed7/.github/workflows/comment-commands.yml)

I'm leaning towards having the ability to trigger it on-demand. And if not, then move it to the start of the CI as having a job just waiting for 99% of it's time is a waste of a runner that could be used to speed up the CI (at the time writing this we have 6 concurrent auto-pr format running, and 16 CI jobs in queue)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions