Skip to content

AppVeyor build matrix: more efficient build job split to reduce total time by another 5 minutes #7021

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TravisEz13 merged 21 commits into from
Jun 11, 2018
Merged

AppVeyor build matrix: more efficient build job split to reduce total time by another 5 minutes #7021

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
e3275e1
Put Pester Admin/Feature and XUnit tests into new matrix element to s…
bergmeister Jun 6, 2018
ad2d0c8
fix stuff
bergmeister Jun 6, 2018
f117ede
fix variable names
bergmeister Jun 6, 2018
bbfcea4
Merge branch 'master' of https://github.com/PowerShell/PowerShell int…
bergmeister Jun 7, 2018
51e8349
fix Test method calls of test results
bergmeister Jun 7, 2018
70ec034
Split only into 2 matrices: One for CI Pester tests and the the 2nd f…
bergmeister Jun 7, 2018
c0a3941
fix variable names
bergmeister Jun 7, 2018
42bb0a6
Fix test cmdlet calls of test results
bergmeister Jun 7, 2018
dac37e1
Merge branch 'master' of https://github.com/PowerShell/PowerShell int…
bergmeister Jun 7, 2018
30297a0
fail fast seems to be inconvenient in this case now
bergmeister Jun 7, 2018
f47acef
Merge branch 'AppVeyorMoreEfficientSplit' of https://github.com/bergm…
bergmeister Jun 7, 2018
6d6c6bb
rename variables as per PR review comment by @daxian-dbw and make sur…
bergmeister Jun 7, 2018
69011e3
[Feature]
bergmeister Jun 7, 2018
3db542a
Merge branch 'master' of https://github.com/PowerShell/PowerShell int…
bergmeister Jun 8, 2018
1a30a40
Fix condition to run feature tests for unelevated Pester tests as wel…
bergmeister Jun 8, 2018
cd2fc3b
[feature]
bergmeister Jun 8, 2018
85af8a7
Simplify Invoke-AppVeyorAfterTest function
bergmeister Jun 8, 2018
9282dd7
revert artificial change to simplify diff
bergmeister Jun 8, 2018
342db57
[feature]
bergmeister Jun 10, 2018
bc91d38
Merge branch 'master' of https://github.com/PowerShell/PowerShell int…
bergmeister Jun 10, 2018
4cc358a
[feature]
bergmeister Jun 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 7 additions & 11 deletions appveyor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,9 @@ environment:
DOTNET_SKIP_FIRST_TIME_EXPERIENCE: 1
matrix:
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
Purpose: BuildingAndPackaging
Purpose: UnelevatedPesterTests
- APPVEYOR_BUILD_WORKER_IMAGE: Visual Studio 2017
Purpose: BuildingAndTesting

# Stop all jobs in the matrix if any one them fail
matrix:
fast_finish: true
Purpose: ElevatedPesterTests_xUnit_Packaging

# cache version - netcoreapp.2.1-sdk.2.1.300
cache:
Expand All @@ -26,14 +22,14 @@ install:
- ps: Invoke-AppveyorInstall

build_script:
- ps: if ($env:Purpose -eq 'BuildingAndTesting'){ Invoke-AppveyorBuild }
- ps: Invoke-AppveyorBuild

test_script:
- ps: if ($env:Purpose -eq 'BuildingAndTesting'){ Invoke-AppveyorTest }
- ps: Invoke-AppveyorTest -Purpose $env:Purpose

after_test:
- ps: if ($env:Purpose -eq 'BuildingAndTesting'){ Invoke-AppVeyorAfterTest }
- ps: Invoke-AppVeyorAfterTest

# Run the finish command and delete files which should not be in the cache
# Packaging
on_finish:
- ps: if ($env:Purpose -eq 'BuildingAndPackaging'){ Invoke-AppveyorFinish }
- ps: if ($env:Purpose -eq 'ElevatedPesterTests_xUnit_Packaging'){ Invoke-AppveyorFinish }
73 changes: 37 additions & 36 deletions tools/appveyor.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -318,7 +318,10 @@ function Update-AppVeyorTestResults
function Invoke-AppVeyorTest
{
[CmdletBinding()]
param()
param(
[ValidateSet('UnelevatedPesterTests', 'ElevatedPesterTests_xUnit_Packaging')]
[string] $Purpose
)
#
# CoreCLR

Expand All @@ -333,47 +336,45 @@ function Invoke-AppVeyorTest
throw "CoreCLR pwsh.exe was not built"
}

if(-not (Test-DailyBuild))
{
# Pester doesn't allow Invoke-Pester -TagAll@('CI', 'RequireAdminOnWindows') currently
# https://github.com/pester/Pester/issues/608
# To work-around it, we exlude all categories, but 'CI' from the list
$ExcludeTag = @('Slow', 'Feature', 'Scenario')
Write-Host -Foreground Green 'Running "CI" CoreCLR tests..'
}
else
{
# Pester doesn't allow Invoke-Pester -TagAll@('CI', 'RequireAdminOnWindows') currently
# https://github.com/pester/Pester/issues/608
# To work-around it, we exlude all categories, but 'CI' from the list
$ExcludeTag = @('Slow', 'Feature', 'Scenario')
if (Test-DailyBuild) {
$ExcludeTag = @()
Write-Host -Foreground Green 'Running all CoreCLR tests..'
}
else {
Write-Host -Foreground Green 'Running "CI" CoreCLR tests..'
}

Start-PSPester -Terse -bindir $env:CoreOutput -outputFile $testResultsNonAdminFile -Unelevate -Tag @() -ExcludeTag ($ExcludeTag + @('RequireAdminOnWindows'))
Write-Host -Foreground Green 'Upload CoreCLR Non-Admin test results'
Update-AppVeyorTestResults -resultsFile $testResultsNonAdminFile

Start-PSPester -Terse -bindir $env:CoreOutput -outputFile $testResultsAdminFile -Tag @('RequireAdminOnWindows') -ExcludeTag $ExcludeTag
Write-Host -Foreground Green 'Upload CoreCLR Admin test results'
Update-AppVeyorTestResults -resultsFile $testResultsAdminFile

Start-PSxUnit -SequentialTestResultsFile $SequentialXUnitTestResultsFile -ParallelTestResultsFile $ParallelXUnitTestResultsFile
Write-Host -ForegroundColor Green 'Uploading PSxUnit test results'
Update-AppVeyorTestResults -resultsFile $SequentialXUnitTestResultsFile
Update-AppVeyorTestResults -resultsFile $ParallelXUnitTestResultsFile
if ($Purpose -eq 'UnelevatedPesterTests') {
Start-PSPester -Terse -bindir $env:CoreOutput -outputFile $testResultsNonAdminFile -Unelevate -Tag @() -ExcludeTag ($ExcludeTag + @('RequireAdminOnWindows'))
Write-Host -Foreground Green 'Upload CoreCLR Non-Admin test results'
Update-AppVeyorTestResults -resultsFile $testResultsNonAdminFile

#
# Fail the build, if tests failed
@(
$testResultsNonAdminFile,
$testResultsAdminFile
) | ForEach-Object {
Test-PSPesterResults -TestResultsFile $_
# Fail the build, if tests failed
Test-PSPesterResults -TestResultsFile $testResultsNonAdminFile
}

@(
$SequentialXUnitTestResultsFile,
$ParallelXUnitTestResultsFile
) | ForEach-Object {
Test-XUnitTestResults -TestResultsFile $_
if ($Purpose -eq 'ElevatedPesterTests_xUnit_Packaging') {
Start-PSPester -Terse -bindir $env:CoreOutput -outputFile $testResultsAdminFile -Tag @('RequireAdminOnWindows') -ExcludeTag $ExcludeTag
Write-Host -Foreground Green 'Upload CoreCLR Admin test results'
Update-AppVeyorTestResults -resultsFile $testResultsAdminFile

Start-PSxUnit -SequentialTestResultsFile $SequentialXUnitTestResultsFile -ParallelTestResultsFile $ParallelXUnitTestResultsFile
Write-Host -ForegroundColor Green 'Uploading PSxUnit test results'
Update-AppVeyorTestResults -resultsFile $SequentialXUnitTestResultsFile
Update-AppVeyorTestResults -resultsFile $ParallelXUnitTestResultsFile

# Fail the build, if tests failed
Test-PSPesterResults -TestResultsFile $testResultsAdminFile
@(
$SequentialXUnitTestResultsFile,
$ParallelXUnitTestResultsFile
) | ForEach-Object {
Test-XUnitTestResults -TestResultsFile $_
}
}

Set-BuildVariable -Name TestPassed -Value True
Expand All @@ -385,7 +386,7 @@ function Invoke-AppVeyorAfterTest
[CmdletBinding()]
param()

if(Test-DailyBuild)
if (Test-DailyBuild)
{
## Publish code coverage build, tests and OpenCover module to artifacts, so webhook has the information.
## Build webhook is called after 'after_test' phase, hence we need to do this here and not in AppveyorFinish.
Expand Down