-
Notifications
You must be signed in to change notification settings - Fork 461
Comparing changes
Open a pull request
base repository: microsoft/VFSForGit
base: master
head repository: microsoft/VFSForGit
compare: run-functional-tests-in-parallel-test
- 10 commits
- 3 files changed
- 1 contributor
Commits on Sep 16, 2025
-
In #1866 I introduced logic to skip builds and tests when a previous workflow run already succeeded for the same commit. However, the original revision of that Pull Request tried something _even more_ elaborate, and when I dropped that elaborate logic (because it is a bit fragile), I made a mistake in the replacement logic. The `run.status` can never be `success`, it can be `completed` and _`run.conclusion`_ can be `success`. So let's test for that instead ;-) Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 7f197c4 - Browse repository at this point
Copy the full SHA 7f197c4View commit details -
Prepare for running the Functional Tests in parallel
This teaches a new command-line option to `GVFS.FunctionalTests.exe` that allows running only a slice of all tests at a time, via `--slice=<m>,<n>` where `m` is the zero-based index of the slice to run, and `n` is the total number of slices to split the tests into. The idea is to distribute the test cases evenly across the slices, so that running all slices in parallel will finish in about the same time as running all tests in a single slice. This is slightly tricky because the test cases of classes within the `EnlistmentPerFixture` namespace are known to rely on side effects of previous test cases within the same class. Therefore, all test cases of such a class must be run in the same slice. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 226ad53 - Browse repository at this point
Copy the full SHA 226ad53View commit details -
ci: run Functional Tests in parallel
Following the example of git/git, which runs the Windows tests in parallel to compensate for a very long run time, we split the Functional Tests into 10 buckets and run them in parallel. This is particularly useful in light of some flaky tests that frequently need to be re-run. That way, the cost of re-running a failed test is reduced by virtue of only having to re-run a _slice_ of the tests. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for e19bb48 - Browse repository at this point
Copy the full SHA e19bb48View commit details -
ci: do provide the job names expected by the Required PR Checks
In the GitHub UI, it is possible to specify Checks that need to be successful before a PR can be merged. These "Checks" are actually the names of jobs in the workflow. It would be much better to specify a required _workflow_, but only workflow _jobs_ can be specified. The Required Checks are currently: - Functional Tests (Debug, arm64) - Functional Tests (Debug, x86_64) - Functional Tests (Release, arm64) - Functional Tests (Release, x86_64) So let's make sure that there are job names that match the recorded names and that indicate a successful workflow run. An alternative would be to add a single job with a defined name that depends on all the other jobs, and that can compensate for the missing GitHub feature to require a whole workflow instead of individual jobs to have run successfully. But that would interfere with other PRs that are currently in flight, and will therefore have to wait its turn in a separate PR. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for b63f77d - Browse repository at this point
Copy the full SHA b63f77dView commit details -
ci: prepare for a single Required Check
GitHub has a feature called "Require status checks before merging": https://docs.github.com/en/repositories/configuring-branches-and-merges-in-your-repository/managing-protected-branches/about-protected-branches#require-status-checks-before-merging What they actually mean by "status checks" is "successful workflow job runs". And what they _really_ actually mean is "_display labels_ of successful workflow job runs". That is, workflow job runs are identified by the label they display. Example: In VFSforGit, there is currently a "required status check" that reads "Functional Test (Debug, arm64)". This is the label that is displayed for the matrix job with vector {configuration: Debug, architecture: arm64} in the "functional_test" workflow job. This is quite restrictive! In VFSforGit, specifying those four matrix job labels is a _work-around_ for the _actual_ requirement, namely that the workflow defined in `build.yaml` succeeds. It just so happens that those four matrix jobs are the leaf jobs, i.e. when they all succeed, the workflow _run_ has succeeded. And vice versa, if the workflow run failed, at least one of those four matrix jobs must have failed or not even run. Now that I multiplied the matrix jobs even further by running the Functional Tests in parallel, the _display labels_ (and the number) of the matrix jobs has changed. As a consequence, to appease the "Require status checks before merging" rule, I had to add _another_ set of matrix jobs just to guarantee that the same four matrix job labels exist. This is silly, because those four matrix jobs are not needed at all for actually testing the code. They are just there to make GitHub happy. This commit prepares to change that. It adds a new workflow job that fits the bill "if the job succeeded, the workflow run must have succeeded as a hole, and vice versa". This new job will be made the only "required status check", once this here PR has been merged, and then we can remove the silly "duplicate" matrix jobs again. Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for ed85493 - Browse repository at this point
Copy the full SHA ed85493View commit details -
Configuration menu - View commit details
-
Copy full SHA for 468d920 - Browse repository at this point
Copy the full SHA 468d920View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ee2429 - Browse repository at this point
Copy the full SHA 5ee2429View commit details -
TO-DROP: run Functional Tests on all pushes
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for 057b7a1 - Browse repository at this point
Copy the full SHA 057b7a1View commit details -
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Configuration menu - View commit details
-
Copy full SHA for f9af301 - Browse repository at this point
Copy the full SHA f9af301View commit details -
Configuration menu - View commit details
-
Copy full SHA for 539e8a8 - Browse repository at this point
Copy the full SHA 539e8a8View commit details
This comparison is taking too long to generate.
Unfortunately it looks like we can’t render this comparison for you right now. It might be too big, or there might be something weird with your repository.
You can try running this command locally to see the comparison on your machine:
git diff master...run-functional-tests-in-parallel-test