Skip to content

Conversation

@jeffbi
Copy link
Contributor

@jeffbi jeffbi commented Feb 8, 2017

No description provided.

Copy link
Collaborator

@iSazonov iSazonov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we mark these tests by Slow tag?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$j.gettype() can raise exception.
Maybe better:

$j | Should BeOfType System.Management.Automation.Job

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch 👍

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Much better. Fixed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same:

Get-Job -id $j.id | Should BeOfType System.Management.Automation.Job

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is seems we can remove $jobs.gettype().fullname | should be "System.Object[]" because next two statement make all needed checks well.

And again:

$job | Should BeOfType System.Management.Automation.Job

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why pipe? It is confused with | Should Be.
We could use:
Remove-Job -Job $j -ErrorAction SilentlyContinue

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same about pipes.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please replace -ea Stop with -ErrorAction Stop

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't like this idea - it can exhaust all CPU resources.
We could utilize something like get-content $TestDrive\jobs.txt -wait to get chunks in the job and send them to output.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I understand this. What would be adding data to jobs.txt while get-content waits? Would we use yet another background job for this?

Would injecting a short start-sleep (say .25 sec) mitigate the CPU usage concern?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perhaps it is justified in these tests.
If no I would prefer synchronous and more deterministic tests.
0.25 is 250 ms so we could use file IO to be even faster.

  1. The test write a test string to a file.
  2. "start-job" job read (Get-Content -wait) the string and send to output
  3. The test call Receive-Job (in loop up to 5 sec) to get the test string

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think @iSazonov means this:

BeforeAll {
    New-Item $TestDrive\jobs.txt -type File
    $j = Start-Job { get-content $TestDrive\jobs.txt -wait }
}

AfterAll {
    Get-Job | Remove-Job -Force
}

It "....." {
     Add-Content $TestDrive\job.txt -Value "Hello"
     Receive-Job $j | should be "Hello"
     Add-Content $TestDrive\job.txt -Value "World"
     Receive-Job $j | should be "World"
}

It "...." {
    Add-Content $TestDrive\job.txt -Value "Keep"
    Receive-Job $j -Keep | should be "Keep"
    Add-Content $TestDrive\job.txt -Value "Output"
    (Receive-Job $j) -Join "," | should be "Keep,Output"
}

I think this is better than the loop approach because 1) it's simpler to read and understand; 2) in case that receive-job really is broken, the loop might leave the test in a bad place.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@daxian-dbw Thanks! 👍
I believe we should put Receive-Job $j -Keep in waiting loop:

function WaitJobResult {
    $result = Receive-Job $j -Keep 
    $startTime = [Datetime]::Now
    while ($rerult -eq $null -or ([Datetime]::Now - $startTime) -lt 5)
    {
        Start-Sleep -Milliseconds  50
        $result = Receive-Job $j -Keep
    }
}

WaitJobResult | should be "Hello"

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seems to be some problem with comments on GitHub 😖

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The while loop can be infinite 😕

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will it? $n will only be set to 5 or 10, and unless Receive-Job never returns enough output to meet those numbers the loop should end pretty quickly.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here we test *-Job cmdlets so we cannot trust them.
We get results from Receive-Job. If Receive-Job will get the same results every call we shall get infinite loop.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quite right. Changed to avoid endless loop.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't Remove-Job stop a job?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It does. Stop-Job removed.

@PowerShellTeam PowerShellTeam added the Review - Needed The PR is being reviewed label Feb 8, 2017
@daxian-dbw daxian-dbw self-assigned this Feb 8, 2017
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lines 87 to 89 might not be necessary for this test as that behavior has already been exercised by the above test.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Removed those lines

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be changed to $results = @(Receive-Job -Keep $job)

@daxian-dbw daxian-dbw added Review - Waiting on Author and removed Review - Needed The PR is being reviewed labels Feb 8, 2017
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

   It "Start-Job produces a job object" {
       $j.gettype().fullname | should be "System.Management.Automation.PSRemotingJob"
    }
  It "Start-Job can name the job" {
       $j = Start-Job -ScriptBlock {6 * 7} -Name "My Job"
       $j.Name | should be "My Job"
   }
  It "Get-Job retrieves a job object" {
       (Get-Job -id $j.id).gettype().fullname | should be "System.Management.Automation.PSRemotingJob"
   }

This may not strictly relevant, but for the 3 simple tests above, it seems too heavy to run start-job -scriptblock { 1 + 1 }, which essentially starts a process, before each test. Can we change the BeforeEach to

        BeforeEach {
            $j = start-job -scriptblock { 1 + 1 } -Name "My Job"
        }

and group those 3 tests into one?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can certainly ad the -Name parameter in the BeforeEach block and merge the first two test together. Personally, I'd rather keep the Get-Job test separate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That sounds good too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @jeffbi #Closed

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

doesn't the AfterEach remove the job? Do you need to start a job here?

@PowerShellTeam PowerShellTeam added Review - Needed The PR is being reviewed and removed Review - Waiting on Author labels Feb 8, 2017
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you want to replace the original { start-sleep 15 } with this script block? None of the tests in this Context check the output of the job, so it's not necessary to write out anything. If you think 15 seconds is more than needed, maybe just { start-sleep 10 }?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That was done when I expected the Receive-Job tests to be in this Context block. Reverted back to Start-Sleep.

@daxian-dbw daxian-dbw added Review - Waiting on Author and removed Review - Needed The PR is being reviewed labels Feb 9, 2017
@PowerShellTeam PowerShellTeam added Review - Needed The PR is being reviewed and removed Review - Waiting on Author labels Feb 9, 2017
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we should remove the line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Darn. I thought I had. Removed now.

Copy link
Collaborator

@iSazonov iSazonov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We now have mixed capitalization for cmdlet names, parameter names, "should be","should beoftype". It is difficult to read. We should fix this and use "Should Be", "Should BeOfType".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we should remove the test because we already have a test for Wait-Job and have tests for Receive-Jobs too. (So much that the test contains errors)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test is about receiving all the output. The other tests are about receiving partial output.
I'll remove this one if you prefer.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Next test implicitly check the same but it can be modified in future so we should leave the test with corrections:

        It "Receive-Job can retrieve job results" {
            Wait-Job -Timeout 10 -id $j.id | Should Not BeNullOrEmpty
            receive-job -id $j.id | Should be 2
        }

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Usually we explicitly use a string in quotes "Completed".

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It makes no sense to make 'Should Be' for every array member.
Maybe better:

checkContent $array | Should Be $true

            function checkContent($array)
            {
                for ($i=1; $i -lt $array.Count; $i++)
                {
                    if ( $array[$i] -ne ($array[$i-1] + 1) ) { return $false}
                }
                return $true
            }

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both checkContent should be removed because this has already been checked in the previous test.

And it makes no sense to make 'Should Be' for every array member. We could use Compare-Object here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now using Compare-Object to compare the beginning of the second Receive-Job with the results of the first, and a single check to ensure that the second Receive-Job didn't miss any data after the first.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The same about string quotes .

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please remove the debug code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if ($count -eq 10000)

It should be 1000 instead, otherwise the timeout would be 50 minutes 😄

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yikes! Fixed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To be accurate, we should use the Perser style:

{ throw "qwerty" } | Should Not Throw

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comparing

PS:89> describe "abc" {
>>       it "def" { throw 'Receive-Job behaves suspiciously: Cannot receive 5 results in 5 minutes.' }
>>       it "hgi" { $true | Should be $true }
>>     }
Describing abc
 [-] def 25ms
   Receive-Job behaves suspiciously: Cannot receive 5 results in 5 minutes.
   at line: 2 in
 [+] hgi 15ms

with

PS:90> describe "abc" {
>>       it "def" { { throw 'Receive-Job behaves suspiciously: Cannot receive 5 results in 5 minutes.' } | Should Not Th
row }
>>       it "hgi" { $true | Should be $true }
>>     }
Describing abc
 [-] def 32ms
   Expected: the expression not to throw an exception. Message was {Receive-Job behaves suspiciously: Cannot receive 5 results in 5 minutes.}
       from line:2 char:20
       + ... t "def" { { throw 'Receive-Job behaves suspiciously: Cannot receive 5 ...
       +                 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   at line: 2 in
   2:       it "def" { { throw 'Receive-Job behaves suspiciously: Cannot receive 5 results in 5 minutes.' } | Should Not Throw }
 [+] hgi 17ms

The first one is more readable. And in this case we throw the exception on purpose to fail the test case, so "{ throw "qwerty" } | Should Not Throw" feels like doing extra work for the same purpose. Maybe we can just keep it as is?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, we should keep. (This exception should raise never. So a performance should not be taken into account.)
The idea was that all the results of the tests have to be caught by Should that Pester normally logs the results. But I check "Pester.xml" and see that the original throw is logged well.
Closed.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System.Management.Automation.Job is string so it would be safer to use quotation marks:

$j | Should BeOfType "System.Management.Automation.Job"

Below too.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added quotes around type names

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about $result2[0..-1] ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is to compare the whole of $result1 with the first n elements of $result2, where n is the length of $result1. Did the second Receive-Job return all of the data that the first one did?

Then it checks for any missing data by seeing if the element immediately following the first n in $result2 is the next number in sequence.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Closed.
Maybe only use -SyncWindow 0.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It seems we agreed to leave the test with improvments.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test added back, with improvements

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you don't do "| Should Be" in CheckContent now, the result from this function should be checked.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Told myself to do it, missed it. Thanks for catching.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With -SyncWindow 0, Compare-Object compares 2 lists in order. Example:

PS> $a = 1,2,3,4
PS> $b = 2,3,4,1
PS> $c = 1,2,3,4
PS> Compare-Object -ReferenceObject $a -DifferenceObject $b
PS> Compare-Object -ReferenceObject $a -DifferenceObject $b -SyncWindow 0

InputObject SideIndicator
----------- -------------
          2 =>
          1 <=
          3 =>
          2 <=
          4 =>
          3 <=
          1 =>
          4 <=

PS> Compare-Object -ReferenceObject $a -DifferenceObject $c -SyncWindow 0
PS>

You can add this parameter if you want to compare them in order.

@iSazonov
Copy link
Collaborator

iSazonov commented Feb 17, 2017

@jeffbi Please resolve the merge conflit.

Copy link
Collaborator

@iSazonov iSazonov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jeffbi Thanks for your patience!

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please swap BeforeEach and AfterEach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test don't use $j so we should move it in separate Context block without creating a job in BeforeEach.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest using the excellent function ShouldBeErrorId
(I have already prepared new PR #3161 to move existing tests to this pattern)
The new look of this test:

            { Get-Job $j -ErrorAction Stop } | ShouldBeErrorId "JobWithSpecifiedNameNotFound,Microsoft.PowerShell.Commands.GetJobCommand" 

Only insert:

Import-Module $PSScriptRoot\..\..\Common\Test.Helpers.psm1

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice! Changed to use ShouldBeErrorId

More changes per code review
Fix merge conflict
@iSazonov
Copy link
Collaborator

@jeffbi Thanks for the great work!
LGTM

We are still far from a full set of job tests. What are your plans - stop here and merge or add new tests?

@daxian-dbw
Copy link
Member

@jeffbi @iSazonov Thank you both for the great work and thoroughness! I will merge this one once Travis CI passes. If you are ready to add more new tests, feel free to open another PR.

BTW, @iSazonov, do you mind clicking "Review Changes" on the right top corner of the "Files Changed" page and select "Approve"? Thanks!

@daxian-dbw daxian-dbw merged commit a557c03 into PowerShell:master Feb 22, 2017
@joeyaiello joeyaiello mentioned this pull request Feb 28, 2017
@iSazonov iSazonov removed the Review - Needed The PR is being reviewed label Mar 27, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants