-
-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Open
Labels
Description
Summary
This is a collection of ideas to make more efficient use of CI resources. Benefits are:
- Less cost: While we don't pay for CI, our CI donors implicitly bear the cost of runnig our jobs. We should hande the resources that we are given responsibly
- Less environmental impact
- Possibly faster cycle times
Proposed fix
This is a collection of possible ideas:
- Speed up the test suite:
1.1. Make our code faster. To be checked, but I'd guess that figure creation and drawing are the expensive parts.
1.2. Optimize test types.
- Cost of the test types:@check_figures_equal()(2 drawn figures) > @image_comparison (1 drawn figure) > just checking that proper change have been made to Axes / Artists (0 drawn figures) - e.g. check that new artists have been created, artist has the right color etc. - Revisit the test matrix. Do we need all combinations?
- Two-stage tests, e.g. if
test_*.pyfiles are modified, do a pre-run only on these files. If they break, stop. If they pass, do the full test run. - Don't run tests on draft PRs.
- Add a label
skip-testsfor manual control.
...