Skip to content

Automation for PS package validation using Docker containers #5401

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
TravisEz13 merged 4 commits into from
Nov 13, 2017
Merged

Automation for PS package validation using Docker containers #5401

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
f4724b7
PackageValidation automation
Nov 10, 2017
2988c16
Fixed file-ending newlines and fedora26 config
Nov 10, 2017
58ba3e2
Changed to using build-args and other misc updates
Nov 11, 2017
10e54da
Ending newline in psm1
Nov 11, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions docker/tests/Templates/centos7/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
FROM centos:7

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN yum install -y \
curl \
glibc-locale-source \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN localedef --charmap=UTF-8 --inputfile=en_US $LANG

RUN curl -L -o powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm $PACKAGELOCATIONSTUB/powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm
RUN yum install -y powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
27 changes: 27 additions & 0 deletions docker/tests/Templates/debian.8/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM debian:jessie

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-utils \
ca-certificates \
curl \
apt-transport-https \
locales \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN locale-gen $LANG && update-locale

RUN curl -L -o powershell_$PSVERSIONSTUB-1.debian.8_amd64.deb $PACKAGELOCATIONSTUB/powershell_$PSVERSIONSTUB-1.debian.8_amd64.deb
RUN dpkg -i powershell_$PSVERSIONSTUB-1.debian.8_amd64.deb || :
RUN apt-get install -y -f
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
27 changes: 27 additions & 0 deletions docker/tests/Templates/debian.9/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM debian:stretch

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-utils \
ca-certificates \
curl \
apt-transport-https \
locales \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN locale-gen $LANG && update-locale

RUN curl -L -o powershell_$PSVERSIONSTUB-1.debian.9_amd64.deb $PACKAGELOCATIONSTUB/powershell_$PSVERSIONSTUB-1.debian.9_amd64.deb
RUN dpkg -i powershell_$PSVERSIONSTUB-1.debian.9_amd64.deb || :
RUN apt-get install -y -f
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
22 changes: 22 additions & 0 deletions docker/tests/Templates/fedora25/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
FROM fedora:25

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN dnf install -y \
curl \
glibc-locale-source \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN localedef --charmap=UTF-8 --inputfile=en_US $LANG

RUN curl -L -o powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm $PACKAGELOCATIONSTUB/powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm
RUN dnf install -y powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
23 changes: 23 additions & 0 deletions docker/tests/Templates/fedora26/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
FROM fedora:26

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN dnf install -y \
curl \
glibc-locale-source \
compat-openssl10 \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN localedef --charmap=UTF-8 --inputfile=en_US $LANG

RUN curl -L -o powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm $PACKAGELOCATIONSTUB/powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm
RUN dnf install -y powershell-$PSVERSIONSTUBRPM-1.rhel.7.x86_64.rpm
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
29 changes: 29 additions & 0 deletions docker/tests/Templates/kalilinux/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
FROM kalilinux/kali-linux-docker:latest

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-utils \
ca-certificates \
curl \
apt-transport-https \
locales \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN locale-gen $LANG && update-locale

RUN curl -L -o libssl1.0.0_1.0.1t-1+deb8u7_amd64.deb http://security.debian.org/debian-security/pool/updates/main/o/openssl/libssl1.0.0_1.0.1t-1+deb8u7_amd64.deb
RUN curl -L -o powershell_$PSVERSIONSTUB-1.ubuntu.16.04_amd64.deb $PACKAGELOCATIONSTUB/powershell_$PSVERSIONSTUB-1.ubuntu.16.04_amd64.deb
RUN dpkg -i libssl1.0.0_1.0.1t-1+deb8u7_amd64.deb || :
RUN dpkg -i powershell_$PSVERSIONSTUB-1.ubuntu.16.04_amd64.deb || :
RUN apt-get install -y -f
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
38 changes: 38 additions & 0 deletions docker/tests/Templates/opensuse42.2/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
FROM opensuse:42.2

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

ARG POWERSHELL_LINKFILE=/usr/bin/pwsh

# Install dependencies
RUN zypper --non-interactive update --skip-interactive \
&& zypper --non-interactive install \
glibc-locale \
glibc-i18ndata \
tar \
curl \
libunwind \
libicu \
openssl \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN localedef --charmap=UTF-8 --inputfile=en_US $LANG

RUN curl -L -o powershell-$PSVERSIONSTUB-linux-x64.tar.gz $PACKAGELOCATIONSTUB/powershell-$PSVERSIONSTUB-linux-x64.tar.gz

# Create the target folder where powershell will be placed
RUN mkdir -p /opt/microsoft/powershell/$PSVERSIONSTUB
# Expand powershell to the target folder
RUN tar zxf powershell-$PSVERSIONSTUB-linux-x64.tar.gz -C /opt/microsoft/powershell/$PSVERSIONSTUB

# Create the symbolic link that points to powershell
RUN ln -s /opt/microsoft/powershell/$PSVERSIONSTUB/pwsh $POWERSHELL_LINKFILE

RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
27 changes: 27 additions & 0 deletions docker/tests/Templates/ubuntu14.04/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM ubuntu:trusty

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-utils \
ca-certificates \
curl \
apt-transport-https \
locales \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN locale-gen $LANG && update-locale

RUN curl -L -o powershell_$PSVERSIONSTUB-1.ubuntu.14.04_amd64.deb $PACKAGELOCATIONSTUB/powershell_$PSVERSIONSTUB-1.ubuntu.14.04_amd64.deb
RUN dpkg -i powershell_$PSVERSIONSTUB-1.ubuntu.14.04_amd64.deb || :
RUN apt-get install -y -f
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
27 changes: 27 additions & 0 deletions docker/tests/Templates/ubuntu16.04/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM ubuntu:xenial

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-utils \
ca-certificates \
curl \
apt-transport-https \
locales \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN locale-gen $LANG && update-locale

RUN curl -L -o powershell_$PSVERSIONSTUB-1.ubuntu.16.04_amd64.deb $PACKAGELOCATIONSTUB/powershell_$PSVERSIONSTUB-1.ubuntu.16.04_amd64.deb
RUN dpkg -i powershell_$PSVERSIONSTUB-1.ubuntu.16.04_amd64.deb || :
RUN apt-get install -y -f
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
27 changes: 27 additions & 0 deletions docker/tests/Templates/ubuntu17.04/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM ubuntu:zesty

ARG PSVERSIONSTUB
ARG PSVERSIONSTUBRPM
ARG PACKAGELOCATIONSTUB
ARG TESTLISTSTUB

# Install dependencies
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
apt-utils \
ca-certificates \
curl \
apt-transport-https \
locales \
git

# Setup the locale
ENV LANG en_US.UTF-8
ENV LC_ALL $LANG
RUN locale-gen $LANG && update-locale

RUN curl -L -o powershell_$PSVERSIONSTUB-1.ubuntu.17.04_amd64.deb $PACKAGELOCATIONSTUB/powershell_$PSVERSIONSTUB-1.ubuntu.17.04_amd64.deb
RUN dpkg -i powershell_$PSVERSIONSTUB-1.ubuntu.17.04_amd64.deb || :
RUN apt-get install -y -f
RUN git clone --recursive https://github.com/PowerShell/PowerShell.git
RUN pwsh -c "Import-Module /PowerShell/build.psm1;exit (Invoke-Pester $TESTLISTSTUB -PassThru).FailedCount"
66 changes: 66 additions & 0 deletions docker/tests/containerTestCommon.psm1
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,3 +190,69 @@ function Get-ContainerPowerShellVersion
}
return (Get-Content -Encoding Ascii $testContext.resolvedLogPath)[0]
}

# This function is used for basic validation of PS packages during a release;
# During the process Docker files are filled out and executed with Docker build;
# During the build PS packages are downloaded onto Docker containers, installed and selected Pester tests from PowerShell Github repo are executed.
# This function must be run on a Docker host machine in 'Linux containers' mode, such as Windows 10 server with Hyper-V role installed.
function Test-PSPackage
{
param(
[string]
[Parameter(Mandatory=$true)]
$PSPackageLocation, # e.g. Azure storage
[string]
$PSVersion = "6.0.0-beta.9",
[string]
$TestList = "/PowerShell/test/powershell/Modules/PackageManagement/PackageManagement.Tests.ps1,/PowerShell/test/powershell/engine/Module"
Copy link
Author

@anmenaga anmenaga Nov 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to review default set of tests to run.

)

$PSPackageLocation = $PSPackageLocation.TrimEnd('/','\')
Write-Verbose "Ensure that PowerShell packages of version $PSVersion exist at $PSPackageLocation" -Verbose

$tempFolder = $env:Temp
if (-not $tempFolder) {$tempFolder = "~"}
$RootFolder = Join-Path $tempFolder 'PSPackageDockerValidation'
$SourceFolder = Join-Path $PSScriptRoot 'Templates'

if (Test-Path $RootFolder)
{
Remove-Item $RootFolder -Recurse -Force
}

Copy-Item -Recurse $SourceFolder $RootFolder

$versionRpmStubName = 'PSVERSIONSTUBRPM'
$versionRpmStubValue = $PSVersion -replace '-','_'
$versionStubName = 'PSVERSIONSTUB'
$versionStubValue = $PSVersion
$testlistStubName = 'TESTLISTSTUB'
$testlistStubValue = $TestList
$packageLocationStubName = 'PACKAGELOCATIONSTUB'
$packageLocationStubValue = $PSPackageLocation


$results = @{}
$returnValue = $true

# run builds sequentially, but don't block for errors so that configs after failed one can run
foreach($dir in Get-ChildItem -Path $RootFolder)
{
$buildArgs = @()
$buildArgs += "--build-arg","$versionRpmStubName=$versionRpmStubValue"
$buildArgs += "--build-arg","$versionStubName=$versionStubValue"
$buildArgs += "--build-arg","$testlistStubName=$testlistStubValue"
$buildArgs += "--build-arg","$packageLocationStubName=$packageLocationStubValue"
$buildArgs += $dir.FullName

$dockerResult = Invoke-Docker -Command 'build' -Params $buildArgs -FailureAction warning
$results.Add($dir.Name, $dockerResult)
if (-not $dockerResult) {$returnValue = $false}
}

# in the end print results for all configurations
Write-Verbose "Package validation results:" -Verbose
$results

return $returnValue
}
Copy link
Member

@adityapatwardhan adityapatwardhan Nov 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use Export-ModuleMember to specify which functions are public.

Copy link
Author

@anmenaga anmenaga Nov 11, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unless it is required, I'd rather not.
It appears that this module does not do this for other functions.