Skip to content

Always run test with crossgen'ed assemblies in CI #5315

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
daxian-dbw merged 1 commit into from
Nov 3, 2017
Merged

Always run test with crossgen'ed assemblies in CI #5315

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 0 additions & 22 deletions test/powershell/Language/Classes/Scripting.Classes.BasicParsing.Tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,6 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
#

try {
#
# CrossGen'ed assemblies cause a hang to happen intermittently when running this test suite in Linux and macOS.
# The issue has been reported to CoreCLR team. We need to work around it for now with the following approach:
# 1. For pull request and push commit, build without '-CrossGen' and run the parsing tests
# 2. For daily build, build with '-CrossGen' but don't run the parsing tests
# In this way, we will continue to exercise these parsing tests for each CI build, and skip them for daily
# build to avoid a hang.
# Note: this change should be reverted once the 'CrossGen' issue is fixed by CoreCLR. The issue is tracked by
# https://github.com/dotnet/coreclr/issues/9745
#
$isDailyBuild = $env:TRAVIS_EVENT_TYPE -eq 'cron' -or $env:TRAVIS_EVENT_TYPE -eq 'api'
$defaultParamValues = $PSdefaultParameterValues.Clone()
$IsSkipped = (!$IsWindows -and $isDailyBuild)
$PSDefaultParameterValues["it:skip"] = $IsSkipped
$PSDefaultParameterValues["ShouldBeParseError:SkipInTravisFullBuild"] = $IsSkipped


Describe 'Positive Parse Properties Tests' -Tags "CI" {
It 'PositiveParsePropertiesTest' {
# Just a bunch of random basic things here
Expand Down Expand Up @@ -859,7 +841,3 @@ Describe 'variable analysis' -Tags "CI" {
[B]::getA().getFoo() | Should Be 'foo'
}
}

} finally {
$global:PSdefaultParameterValues = $defaultParamValues
}
22 changes: 0 additions & 22 deletions test/powershell/Language/Classes/scripting.Classes.inheritance.tests.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,24 +2,6 @@
# Copyright (c) Microsoft Corporation. All rights reserved.
#

try {
#
# CrossGen'ed assemblies cause a hang to happen intermittently when running this test suite in Linux and macOS.
# The issue has been reported to CoreCLR team. We need to work around it for now with the following approach:
# 1. For pull request and push commit, build without '-CrossGen' and run the parsing tests
# 2. For daily build, build with '-CrossGen' but don't run the parsing tests
# In this way, we will continue to exercise these parsing tests for each CI build, and skip them for daily
# build to avoid a hang.
# Note: this change should be reverted once the 'CrossGen' issue is fixed by CoreCLR. The issue is tracked by
# https://github.com/dotnet/coreclr/issues/9745
#
$isDailyBuild = $env:TRAVIS_EVENT_TYPE -eq 'cron' -or $env:TRAVIS_EVENT_TYPE -eq 'api'
$defaultParamValues = $PSdefaultParameterValues.Clone()
$IsSkipped = (!$IsWindows -and $isDailyBuild)
$PSDefaultParameterValues["it:skip"] = $IsSkipped
$PSDefaultParameterValues["ShouldBeParseError:SkipInTravisFullBuild"] = $IsSkipped


Describe 'Classes inheritance syntax' -Tags "CI" {

It 'Base types' {
Expand Down Expand Up @@ -539,7 +521,3 @@ class Derived : Base
$sb.Invoke() | Should Be 200
}
}

} finally {
$global:PSdefaultParameterValues = $defaultParamValues
}
33 changes: 1 addition & 32 deletions test/tools/Modules/HelpersLanguage/HelpersLanguage.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -65,40 +65,9 @@ function ShouldBeParseError
# This is a temporary solution after moving type creation from parse time to runtime
[switch]$SkipAndCheckRuntimeError,
# for test coverarage purpose, tests validate columnNumber or offset
[switch]$CheckColumnNumber,
# Skip this test in Travis CI nightly build
[switch]$SkipInTravisFullBuild
[switch]$CheckColumnNumber
)

#
# CrossGen'ed assemblies cause a hang to happen when running tests with this helper function in Linux and macOS.
# The issue has been reported to CoreCLR team. We need to work around it for now with the following approach:
# 1. For pull request and push commit, build without '-CrossGen' and run the parsing tests
# 2. For nightly build, build with '-CrossGen' but don't run the parsing tests
# In this way, we will continue to exercise these parsing tests for each CI build, and skip them for nightly
# build to avoid a hang.
# Note: this change should be reverted once the 'CrossGen' issue is fixed by CoreCLR. The issue is tracked by
# https://github.com/dotnet/coreclr/issues/9745
#
if ($SkipInTravisFullBuild) {
## Report that we skipped the tests and return
## be sure to report the same number of tests
## it should have the same appearance as if the tests were run
Context "Parse error expected: <<$src>>" {
if ($SkipAndCheckRuntimeError)
{
It "error should happen at parse time, not at runtime" -Skip {}
}
It "Error count" -Skip { }
foreach($expectedError in $expectedErrors)
{
It "Error Id" -Skip { }
It "Error position" -Skip { }
}
}
return
}

Context "Parse error expected: <<$src>>" {
# Test case error if this fails
$expectedErrors.Count | Should Be $expectedOffsets.Count
Expand Down
18 changes: 1 addition & 17 deletions tools/travis.ps1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,23 +173,11 @@ elseif($Stage -eq 'Build')
Write-Host -Foreground Green "Executing travis.ps1 `$isPR='$isPr' `$isFullBuild='$isFullBuild' - $commitMessage"
$output = Split-Path -Parent (Get-PSOutput -Options (New-PSOptions))

# CrossGen'ed assemblies cause a hang to happen intermittently when running powershell class
# basic parsing tests in Linux/macOS. The hang seems to happen when generating dynamic assemblies.
# This issue has been reported to CoreCLR team. We need to work around it for now because
# the Travis CI build failures caused by this is draining our builder resource and severely
# affect our daily work. The workaround is:
# 1. For pull request and push commit, build without '-CrossGen' and run the parsing tests
# 2. For nightly build, build with '-CrossGen' but don't run the parsing tests
# With this workaround, CI builds triggered by pull request and push commit will exercise
# the parsing tests with IL assemblies, while nightly builds will exercise CrossGen'ed assemblies
# without running those class parsing tests so as to avoid the hang.
# NOTE: this change should be reverted once the 'CrossGen' issue is fixed by CoreCLR. The issue
# is tracked by https://github.com/dotnet/coreclr/issues/9745
$originalProgressPreference = $ProgressPreference
$ProgressPreference = 'SilentlyContinue'
try {
## We use CrossGen build to run tests only if it's the daily build.
Start-PSBuild -CrossGen:$isDailyBuild -PSModuleRestore
Start-PSBuild -CrossGen -PSModuleRestore
}
finally{
$ProgressPreference = $originalProgressPreference
Expand Down Expand Up @@ -245,10 +233,6 @@ elseif($Stage -eq 'Build')
}

if (-not $isPr) {
# Run 'CrossGen' for push commit, so that we can generate package.
# It won't rebuild powershell, but only CrossGen the already built assemblies.
if (-not $isDailyBuild) { Start-PSBuild -CrossGen }

$packageParams = @{}
if($env:TRAVIS_BUILD_NUMBER)
{
Expand Down