Added cmdlets to get Pester failures #2600

SteveL-MSFT · 2016-11-03T20:41:44Z

Get-PSPesterFailure will parse the Pester log from start-pspester and extract the failures as objects
Format-PSPesterFailure will format the failures similar to output from Pester (you can use directly without Get-PSPesterFailure)

msftclas · 2016-11-03T20:41:49Z

Hi @SteveL-MSFT, I'm your friendly neighborhood Microsoft Pull Request Bot (You can call me MSBOT). Thanks for your contribution!

It looks like you're a Microsoft contributor (Steve Lee (POWERSHELL)). If you're full-time, we DON'T require a Contribution License Agreement. If you are a vendor, please DO sign the electronic Contribution License Agreement. It will take 2 minutes and there's no faxing! https://cla.microsoft.com.

TTYL, MSBOT;

…er to extract the failures

iSazonov · 2016-11-05T15:40:54Z

build.psm1

+    {
+        if ($PSBoundParameters.Count -eq 0 -or $PSBoundParameters.ContainsKey("NUnitLog"))
+        {
+            $PesterFailure = Get-PSPesterFailures -NUnitLog $NUnitLog


Get-PSPesterFailures should be replaced by Get-PSPesterFailure

iSazonov · 2016-11-08T05:59:16Z

LGTM.
(Maybe add in docs how to use this?)

SteveL-MSFT · 2016-11-08T17:34:01Z

@iSazonov good suggestion, I'll add something to the existing testing guidelines

Fix spelling error.

iSazonov · 2016-11-09T07:49:53Z

docs/testing-guidelines/testing-guidelines.md

 * `Get-PSPesterFailure` will parse the NUnit test result log and return PowerShell objects for each failure so you can do additional filtering, sorting, grouping, etc...
-* `Format-PSPesterFailure` will call `Get-PSPesterFailure` if no parameters are provided and show just the failures at the console simliar to what Pester displays
+* `Format-PSPesterFailure` will call `Get-PSPesterFailure` if no parameters are provided and show just the failures at the console similar to what Pester displays



Maybe add call samples as above for Start-PSPester?

TravisEz13 · 2016-11-11T03:02:56Z

build.psm1

+        foreach ($failureTest in $failureTests)
+        {
+            $totalFailures++
+            $failure = New-Object -TypeName PSCustomObject -Property @{SuiteName=$failureSuite.Name;TestDescription=$failureTest.Description;


Perhaps the properties could be more readable

@TravisEz13 do you have a specific suggestion as it seems readable to me?

Guidance developed in the DSC Resource project is to have each name-value pair on a new line

got it, will change

added sample to using failure cmdlets

iSazonov · 2016-11-15T06:36:49Z

docs/testing-guidelines/testing-guidelines.md

+* `Format-PSPesterFailure` will call `Get-PSPesterFailure` if no parameters are provided and show just the failures at the console similar to what Pester displays
+
+```PowerShell
+Start-PSPester    # summary shows failures


This seems to be a typo and comment must be on a new line.

iSazonov · 2016-11-15T06:41:42Z

In future, it would be good to use it to reduce logs on appveyor and travis-ci to only display failed tests.

kilasuit · 2016-11-15T07:28:22Z

Only query that I would has is should these functions be added to the PowerShell repo or would it make more sense that they be added directly into Pester?

My rational behind adding to Pester is that I can see use cases for them being used outside of just testing the PowerShell Code base so to me it would make more sense for these to be added to Pester.

TravisEz13 · 2016-11-15T15:01:11Z

docs/testing-guidelines/testing-guidelines.md


 For example, to run all the Pester tests for CI (assuming you are at the root of the PowerShell repo):
-```
+```PowerShell


MINOR: lowercase powershell works more universally, but this will work for now on GitHub

@TravisEz13 as a maintainer, you can make small edits directly into this PR

I'll make this change since I'm already updating this PR

iSazonov · 2016-11-16T04:01:14Z

@kilasuit Your proposal looks very good. I think it would be better to create a Issue in Pester repo and don't block this PR.

kilasuit · 2016-11-17T02:01:17Z

@iSazonov - As much as I agree with your suggestion, I do feel that the onus to do so would lie with @SteveL-MSFT considering that this is his PR & his code etc etc

SteveL-MSFT · 2016-11-17T16:58:10Z

@kilasuit I created pester/Pester#645 as a feature request to Pester. I think this could be solved in a number of ways and this PR was specifically to solve a problem I encountered where we have a large number of tests for PowerShell (and growing!) and needed an easy way to just see the failures so I can investigate before pushing my changes

nohwnd · 2016-11-18T09:26:32Z

Is this the best way to do it?

You are at the end, looking at the nUnit report in XML. Instead you can tap into the pipeline right after Invoke-Pester. Get the results as a PSObject (-PassThru), filter test results to failures with Where, and then either use your print functions, or reuse the internal Pester print functions.

#store the results in a variable
$results = invoke-pester -passthru

#get first successful result (you'd do -ne "Success")
$result = $results.TestResult | Where {$_.Result -eq "Passed"} | Select -first 1 

#get reference to internal Pester print function
#(see output.ps1 in Pester, for other output functions)
$writePesterResult = &(Get-Module Pester) {Get-Command Write-PesterResult}

#print the result 
&$writePesterResult $result
#outputs:  [+] adds positive numbers 44ms

Or just wait for us to put it directly in Pester. 😃 I just wanted to highlight what is in my opinion a better way to approach this problem.

SteveL-MSFT · 2016-11-18T16:46:46Z

@nohwnd I think you're right, we can close this PR and wait for pester/Pester#645

msftclas added the cla-not-required label Nov 3, 2016

SteveL-MSFT added 4 commits November 3, 2016 14:51

added get-pspesterfailures and format-pspesterfailure to make it easi…

3df4081

…er to extract the failures

fixed formatting

1cf3d74

made cmdlet singular

97f9765

Merge remote-tracking branch 'origin/master' into getfailures

b793956

iSazonov reviewed Nov 5, 2016

View reviewed changes

fixed call to get-pspesterfailure

613be77

SteveL-MSFT and others added 2 commits November 8, 2016 09:41

Added to test docs new cmdlets

5e1d48a

Update testing-guidelines.md

40ce3d6

Fix spelling error.

iSazonov reviewed Nov 9, 2016

View reviewed changes

PowerShellTeam added the Review - Needed label Nov 10, 2016

TravisEz13 approved these changes Nov 11, 2016

View reviewed changes

TravisEz13 added Area-Maintainers-Build specific to affecting the build WG-Quality-Test issues in a test or in test infrastructure labels Nov 11, 2016

Update testing-guidelines.md

6795cc1

added sample to using failure cmdlets

vors closed this Nov 14, 2016

vors reopened this Nov 14, 2016

msftclas added the cla-not-required label Nov 14, 2016

iSazonov reviewed Nov 15, 2016

View reviewed changes

TravisEz13 reviewed Nov 15, 2016

View reviewed changes

SteveL-MSFT mentioned this pull request Nov 17, 2016

Feature Request: get just the failures pester/Pester#645

Closed

SteveL-MSFT closed this Nov 18, 2016

iSazonov mentioned this pull request Nov 30, 2016

modify Start-PSPester to accept -Quiet to eliminate Pester output #2795

Merged

iSazonov mentioned this pull request Aug 23, 2017

For reliability, start using pester PassThru object to verify tests passed #4644

Merged

Added cmdlets to get Pester failures #2600

Added cmdlets to get Pester failures #2600

Uh oh!

Conversation

SteveL-MSFT commented Nov 3, 2016

Uh oh!

msftclas commented Nov 3, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iSazonov commented Nov 8, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SteveL-MSFT commented Nov 8, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

TravisEz13 Nov 15, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iSazonov commented Nov 15, 2016

Uh oh!

kilasuit commented Nov 15, 2016

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iSazonov commented Nov 16, 2016

Uh oh!

kilasuit commented Nov 17, 2016

Uh oh!

SteveL-MSFT commented Nov 17, 2016

Uh oh!

nohwnd commented Nov 18, 2016 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

SteveL-MSFT commented Nov 18, 2016

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

9 participants

iSazonov commented Nov 8, 2016 •

edited

Loading

TravisEz13 Nov 15, 2016 •

edited

Loading

nohwnd commented Nov 18, 2016 •

edited

Loading