Re-balance CI tests #11420

iSazonov · 2019-12-21T17:42:16Z

PR Summary

Re-balance tests to reduce CI times for PRs.

Better review commit by commit.

PR Context

PR Checklist

test/powershell/Host/ConsoleHost.Tests.ps1

PoshChan · 2019-12-22T07:31:10Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-windows
Tests for Get-ComputerInfo: Ensure Type returned.Verify type returned by Get-ComputerInfo

Expected the value to have type [Microsoft.PowerShell.Commands.ComputerInfo] or any of its subtypes, but got @{BiosCaption=Intel(R) Xeon(R) CPU E5-2673 v4 @ 2.30GHz} with type [PSCustomObject]. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Management\Get-ComputerInfo.Tests.ps1: line 1011 1011: $computerInfo | Should -BeOfType 'Microsoft.PowerShell.Commands.ComputerInfo'

PoshChan · 2019-12-22T15:05:48Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-windows
Basic FileSystem Provider Tests.Appx path.Can get an appx package item

Cannot find path 'TestDrive:\Calculator.exe' because it does not exist. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Management\FileSystem.Tests.ps1: line 292 292: Get-Item $pkgDir\Calculator.exe -ErrorAction Stop | Should -BeOfType [System.IO.FileInfo]

PoshChan · 2019-12-22T15:41:38Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-linux
Test-Connection.TraceRoute.returns false without error if MaxHops is exceeded during -Traceroute -Quiet

Missing an argument for parameter 'TimeoutSeconds'. Specify a parameter of type 'System.Int32' and try again. at <ScriptBlock>, /home/vsts/work/1/s/test/powershell/Modules/Microsoft.PowerShell.Management/Test-Connection.Tests.ps1: line 278 278: Test-Connection 8.8.8.8 -Traceroute -MaxHops 2 -TimeoutSeconds -Quiet | Should -BeFalse

PoshChan · 2019-12-22T15:41:39Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 2 failures in PowerShell-CI-windows
Basic FileSystem Provider Tests.Appx path.Can get an appx package item

Cannot find path 'TestDrive:\Calculator.exe' because it does not exist. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Management\FileSystem.Tests.ps1: line 292 292: Get-Item $pkgDir\Calculator.exe -ErrorAction Stop | Should -BeOfType [System.IO.FileInfo]

Test-Connection.TraceRoute.returns false without error if MaxHops is exceeded during -Traceroute -Quiet

Missing an argument for parameter 'TimeoutSeconds'. Specify a parameter of type 'System.Int32' and try again. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Management\Test-Connection.Tests.ps1: line 278 278: Test-Connection 8.8.8.8 -Traceroute -MaxHops 2 -TimeoutSeconds -Quiet | Should -BeFalse

PoshChan · 2019-12-22T15:42:20Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-macos
Test-Connection.TraceRoute.returns false without error if MaxHops is exceeded during -Traceroute -Quiet

Missing an argument for parameter 'TimeoutSeconds'. Specify a parameter of type 'System.Int32' and try again. at <ScriptBlock>, /Users/runner/runners/2.163.1/work/1/s/test/powershell/Modules/Microsoft.PowerShell.Management/Test-Connection.Tests.ps1: line 278 278: Test-Connection 8.8.8.8 -Traceroute -MaxHops 2 -TimeoutSeconds -Quiet | Should -BeFalse

PoshChan · 2019-12-22T16:15:13Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 2 failures in PowerShell-CI-windows
Basic FileSystem Provider Tests.Appx path.Can get an appx package item

Cannot find path 'TestDrive:\Calculator.exe' because it does not exist. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Management\FileSystem.Tests.ps1: line 292 292: Get-Item $pkgDir\Calculator.exe -ErrorAction Stop | Should -BeOfType [System.IO.FileInfo]

Test-Connection.TraceRoute.returns false without error if MaxHops is exceeded during -Traceroute -Quiet

Missing an argument for parameter 'TimeoutSeconds'. Specify a parameter of type 'System.Int32' and try again. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Management\Test-Connection.Tests.ps1: line 278 278: Test-Connection 8.8.8.8 -Traceroute -MaxHops 2 -TimeoutSeconds -Quiet | Should -BeFalse

PoshChan · 2019-12-22T16:15:15Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-linux
Test-Connection.TraceRoute.returns false without error if MaxHops is exceeded during -Traceroute -Quiet

Missing an argument for parameter 'TimeoutSeconds'. Specify a parameter of type 'System.Int32' and try again. at <ScriptBlock>, /home/vsts/work/1/s/test/powershell/Modules/Microsoft.PowerShell.Management/Test-Connection.Tests.ps1: line 278 278: Test-Connection 8.8.8.8 -Traceroute -MaxHops 2 -TimeoutSeconds -Quiet | Should -BeFalse

PoshChan · 2019-12-22T16:15:28Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-macos
Test-Connection.TraceRoute.returns false without error if MaxHops is exceeded during -Traceroute -Quiet

Missing an argument for parameter 'TimeoutSeconds'. Specify a parameter of type 'System.Int32' and try again. at <ScriptBlock>, /Users/runner/runners/2.163.1/work/1/s/test/powershell/Modules/Microsoft.PowerShell.Management/Test-Connection.Tests.ps1: line 278 278: Test-Connection 8.8.8.8 -Traceroute -MaxHops 2 -TimeoutSeconds -Quiet | Should -BeFalse

test/powershell/Host/ConsoleHost.Tests.ps1

PoshChan · 2019-12-22T17:58:21Z

test/powershell/Host/ConsoleHost.Tests.ps1

@iSazonov, your last commit had 1 failures in PowerShell-CI-windows
Enter-PSHostProcess tests.By CustomPipeName.Can enter, exit, and re-enter using CustomPipeName

Expected $true, because The script was able to re-enter another process and grab the pipe of '4pzbeujh.2it'., but got $false. at <ScriptBlock>, D:\a\1\s\test\powershell\Modules\Microsoft.PowerShell.Core\Enter-PSHostProcess.Tests.ps1: line 186 186: Should -BeTrue -Because "The script was able to re-enter another process and grab the pipe of '$pipeName'."

iSazonov · 2019-12-30T10:50:28Z

@SteveL-MSFT @vexx32 I reverted last commit because it cause the test fail #11420 (review) that looks very amizing.
I can not repo locally (I don't see 40 sec delay).
I concern this could be a tricky bug in our code.

TravisEz13 · 2020-01-03T21:37:22Z

Can we split this into two PRs? One for the tests fixes and refactoring and one for the rebalancing?

test/powershell/Host/ConsoleHost.Tests.ps1

iSazonov · 2020-01-04T12:45:29Z

Can we split this into two PRs? One for the tests fixes and refactoring and one for the rebalancing?

@TravisEz13 There are only two commits/files for explicit rebalancing where I replace CI tag with Feature one - Jobs and HelpSystemOnline. I don't see any value to move them to new PR. I guess you ask about first commit "Correct pwsh run" that single looks like unrelated change.
Moved to #11486

This reverts commit 9d83d30377c660fdeabbc4e18a7d3a20e81c0d48.

ghost · 2020-03-26T17:44:22Z

🎉v7.1.0-preview.1 has been released which incorporates this pull request.:tada:

Handy links:

Release Notes

iSazonov self-assigned this Dec 21, 2019

PoshChan reviewed Dec 21, 2019

View reviewed changes

test/powershell/Host/ConsoleHost.Tests.ps1 Outdated Show resolved Hide resolved

iSazonov force-pushed the test-improve-perf branch from ecce392 to 1e806eb Compare December 21, 2019 18:48

PoshChan reviewed Dec 21, 2019

View reviewed changes

test/powershell/Host/ConsoleHost.Tests.ps1 Outdated Show resolved Hide resolved

iSazonov force-pushed the test-improve-perf branch from 1e806eb to e6f48f4 Compare December 21, 2019 19:36

PoshChan reviewed Dec 22, 2019

View reviewed changes

iSazonov force-pushed the test-improve-perf branch 3 times, most recently from db545b7 to 1868802 Compare December 22, 2019 11:18

PoshChan reviewed Dec 22, 2019

View reviewed changes

iSazonov force-pushed the test-improve-perf branch from cca5a81 to c986694 Compare December 22, 2019 15:23

PoshChan reviewed Dec 22, 2019

View reviewed changes

iSazonov force-pushed the test-improve-perf branch from c986694 to 99d4b7e Compare December 22, 2019 15:56

PoshChan reviewed Dec 22, 2019

View reviewed changes

iSazonov force-pushed the test-improve-perf branch from 99d4b7e to dbe38e6 Compare December 22, 2019 16:24

PoshChan reviewed Dec 22, 2019

View reviewed changes

test/powershell/Host/ConsoleHost.Tests.ps1 Outdated Show resolved Hide resolved

PoshChan reviewed Dec 22, 2019

View reviewed changes

test/powershell/Host/ConsoleHost.Tests.ps1 Outdated Show resolved Hide resolved

PoshChan reviewed Dec 22, 2019

View reviewed changes

test/powershell/Host/ConsoleHost.Tests.ps1 Outdated Show resolved Hide resolved

iSazonov force-pushed the test-improve-perf branch from dbe38e6 to 9d83d30 Compare December 22, 2019 17:37

PoshChan reviewed Dec 22, 2019

View reviewed changes

iSazonov marked this pull request as ready for review December 30, 2019 10:50

iSazonov requested review from SteveL-MSFT, TravisEz13, adityapatwardhan and vexx32 December 30, 2019 10:51

iSazonov added the CL-Test Indicates that a PR should be marked as a test change in the Change Log label Dec 30, 2019

iSazonov added this to the 7.1.0-preview.1 milestone Dec 30, 2019

iSazonov changed the title ~~Improve test performance~~ Re-balance CI tests Dec 30, 2019

iSazonov force-pushed the test-improve-perf branch from 65f361e to 8454725 Compare January 4, 2020 06:14

PoshChan reviewed Jan 4, 2020

View reviewed changes

test/powershell/Host/ConsoleHost.Tests.ps1 Outdated Show resolved Hide resolved

iSazonov force-pushed the test-improve-perf branch from 8454725 to 4b56fee Compare January 4, 2020 08:27

iSazonov mentioned this pull request Jan 4, 2020

Run pwsh correctly in tests #11486

Merged

14 tasks

iSazonov force-pushed the test-improve-perf branch from 4b56fee to 3b60f3b Compare January 4, 2020 12:55

iSazonov added 10 commits January 4, 2020 17:56

Serialization

c8d20a3

TaskBasedAsyncPowerShellAPI

7e7d88f

DefaultCommands

eedc3f2

HelpSystemOnline

10e2387

Jobs

1a57bd7

Job

75b8c81

Get-WinEvent

28adce9

FileSystem

d6139b1

Test-Connection

f4820c9

Revert "Test-Connection"

3b60f3b

This reverts commit 9d83d30377c660fdeabbc4e18a7d3a20e81c0d48.

TravisEz13 approved these changes Jan 4, 2020

View reviewed changes

iSazonov merged commit 99cf9fd into PowerShell:master Jan 5, 2020

iSazonov deleted the test-improve-perf branch January 5, 2020 08:01

iSazonov restored the test-improve-perf branch January 8, 2020 15:23

iSazonov mentioned this pull request Jan 8, 2020

Make Test-Connection always use the default synchronization context for sending ping requests #11517

Merged

14 tasks

iSazonov deleted the test-improve-perf branch May 15, 2020 13:25

Re-balance CI tests #11420

Re-balance CI tests #11420

Uh oh!

Conversation

iSazonov commented Dec 21, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Context

PR Checklist

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iSazonov commented Dec 30, 2019

Uh oh!

TravisEz13 commented Jan 3, 2020

Uh oh!

Uh oh!

iSazonov commented Jan 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ghost commented Mar 26, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

iSazonov commented Dec 21, 2019 •

edited

Loading

iSazonov commented Jan 4, 2020 •

edited

Loading