Replace Remaining HttpBin.org Tests with WebListener #5665

markekraus · 2017-12-10T19:19:23Z

PR Summary

Replaces all remaining test that rely on httpbin.org
Adds Put, Post, Patch, and Delete tests to WebListener by means of routes to Get test and modifications to the Get controller.
Adds responsephrase option to the Response test to accommodate error message tests
removed redundant GET tests from irm and iwr tests.
Fixed markdown linting errors in README.md for WebListener
closes Make Web Cmdlets Tests More Terse Friendly #5611
references Tests should not rely on external websites which lead to random failures #2504

PR Checklist

Note: Please mark anything not applicable to this PR NA.

PR has a meaningful title
- Use the present tense and imperative mood when describing your changes
Summarized changes
NA User facing Documentation needed
- NA Issue filed - Issue link:
Change is not breaking
Make sure you've added a new test if existing tests do not effectively test the code changed
- Add [feature] if the change is significant or affectes feature tests
This PR is ready to merge and is not Work in Progress.
- If the PR is work in progress, please add the prefix WIP: to the beginning of the title and remove the prefix when the PR is ready.

iSazonov · 2017-12-11T13:51:03Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

        $result = ExecuteWebCommand -command $command

-        $result.Error.ErrorDetails.Message | Should Match "\-=\[ teapot \]"
+        $result.Error.ErrorDetails.Message | Should be $query.body


Typo be -> Be.

iSazonov · 2017-12-11T13:54:00Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

+    $testMethods = @("POST", "PATCH", "PUT", "DELETE")
    $contentTypes = @("text/plain", "application/xml", "application/json")

    foreach ($contentType in $contentTypes)


Minor comment - I'd prefer TestCases here.

I have an open issue somewhere about refactoring the web cmdlet tests. There is much room for improvement.

iSazonov · 2017-12-11T14:00:15Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

-                    {
-                        $jsonContent.data | Should Match $body
-                    }
+                    $jsonContent.data | Should Match $body


Maybe 'Should Be' too?

I can try changing it. I assumed there was specific logic to use match for some good reason. if you look at the if block above they have Should Be for application/xml but match for everything else. *shrugs (same for the other block liek this)

My question was just about the "some good reason".
Right question is - should we expect that Invoke-WebRequest change the body (add/remove whitespaces)?

iSazonov · 2017-12-11T14:02:54Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

    It "Validate Invoke-WebRequest default ContentType for CustomMethod POST" {

-        $command = "Invoke-WebRequest -Uri 'http://httpbin.org/post' -CustomMethod POST -Body 'testparam=testvalue'"
+        $uri = Get-WebListenerUrl -Test 'Post'


The test header say "default ContentType" but why the test don't check a content type name?

Default content type as in -ContentType is not specified. see the line below this.

I think I misinterpreted the question. The headline says we're checking the default content type, but inside the test we're not doing it. I expect something like

$jsonContent.headers.'Content-Type' | Should Be "default-content-type-value"

I will take another look at the surrounding code and try to determine what this test was really looking for.

I'm not 100% sure what this is testing for, so i added the Headers test.
Fixed

iSazonov · 2017-12-11T14:06:20Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1


-        $result.Error.ErrorDetails.Message | Should Match "\-=\[ teapot \]"
+        $result.Error.ErrorDetails.Message | Should be $query.body
        $result.Error.Exception | Should BeOfType Microsoft.PowerShell.Commands.HttpResponseException


Minor comment - could you please put the type in quotes?

iSazonov · 2017-12-11T14:08:21Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

    It "Validate Invoke-RestMethod default ContentType for CustomMethod POST" {

-        $command = "Invoke-RestMethod -Uri 'http://httpbin.org/post' -CustomMethod POST -Body 'testparam=testvalue'"
+        $uri = Get-WebListenerUrl -Test 'Post'


The same about default ContentType.

same as above.

iSazonov · 2017-12-11T14:09:00Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1


-        $result.Error.ErrorDetails.Message | Should Match "\-=\[ teapot \]"
+        $result.Error.ErrorDetails.Message | Should Be $query.body
        $result.Error.Exception | Should BeOfType Microsoft.PowerShell.Commands.HttpResponseException


The same minor comment about quotes.

iSazonov · 2017-12-11T14:13:15Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

+            $result = Invoke-WebRequest -InFile $filePath  -Uri $uri -Method Post
            $content = $result.Content | ConvertFrom-Json
-            $content.form | Should Match "hello"
+            $content.form.hello[0] | Should Match "world"


I think we should check $content.form.hello.Count | Should Be 1 before the line.

iSazonov · 2017-12-11T14:14:05Z

test/powershell/Modules/Microsoft.PowerShell.Utility/WebCmdlets.Tests.ps1

-            $result = Invoke-RestMethod -InFile $filePath  -Uri http://httpbin.org/post -Method Post
-            $result.form | Should Match "hello"
+            $result = Invoke-RestMethod -InFile $filePath  -Uri $uri -Method Post
+            $result.form.hello[0] | Should Match "world"


iSazonov · 2017-12-11T14:15:38Z

test/tools/WebListener/Controllers/GetController.cs

+                {"method" , Request.Method}
            };
+
+            if(Request.HasFormContentType){


Typo - spaces skipped and the brace to next line.

TravisEz13 · 2017-12-12T01:11:32Z

test/tools/WebListener/README.md

 ASP.NET Core 2.0 app for testing HTTP and HTTPS Requests.

-# Run with `dotnet`
+## Run with `dotnet`


Is this file included in the markdown meta test?

It is not.

PowerShell/test/common/markdown/markdown.tests.ps1

Lines 74 to 85 in 4dc5512

$docsToTest = @(

'./*.md'

'./docs/*.md'

'./docs/building/*.md'

'./docs/cmdlet-example/*.md'

'./docs/installation/*.md'

'./docs/maintainers/README.md'

'./demos/SSHRemoting/*.md'

'./docker/*.md'

'./tools/*.md'

'./.github/CONTRIBUTING.md'

)

You you prefer that it be added?

codecov-io · 2017-12-12T15:54:33Z

Codecov Report

Merging #5665 into master will increase coverage by 18.5%.
The diff coverage is n/a.

@@            Coverage Diff             @@
##           master    #5665      +/-   ##
==========================================
+ Coverage   42.39%    60.9%   +18.5%     
==========================================
  Files         924      924              
  Lines      274195   274195              
==========================================
+ Hits       116258   166994   +50736     
+ Misses     157937   107201   -50736

Impacted Files	Coverage Δ
...on/engine/remoting/commands/PushRunspaceCommand.cs	`15.17% <0%> (+0.15%)`	⬆️
...ion/engine/remoting/fanin/WSManTransportManager.cs	`64.11% <0%> (+0.23%)`	⬆️
....Management.Automation/help/UpdatableHelpSystem.cs	`61.87% <0%> (+0.26%)`	⬆️
...Automation/engine/remoting/client/ThrottlingJob.cs	`81.16% <0%> (+0.26%)`	⬆️
...tion/engine/interpreter/ControlFlowInstructions.cs	`82.93% <0%> (+0.29%)`	⬆️
src/Microsoft.WSMan.Management/WsManHelper.cs	`41.81% <0%> (+0.38%)`	⬆️
src/System.Management.Automation/engine/Pipe.cs	`78.26% <0%> (+0.39%)`	⬆️
src/Microsoft.WSMan.Management/ConfigProvider.cs	`73.36% <0%> (+0.41%)`	⬆️
...on/engine/interpreter/CallInstruction.Generated.cs	`32.38% <0%> (+0.47%)`	⬆️
...nt.Automation/namespaces/NavigationProviderBase.cs	`72.84% <0%> (+0.52%)`	⬆️
... and 626 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 2a9c1ca...466d019. Read the comment docs.

anmenaga

AppVeyor test pass does not look complete as if something hang. @daxian-dbw can you please restart AppVeyor test run? Thank you.

iSazonov · 2017-12-13T10:03:40Z

Reopen to restart AppVeyor CI

•Replaces all remaining test that rely on httpbin.org •Adds Put, Post, Patch, and Delete tests to WebListener by means of routes to Get test and modifications to the Get controller. •Adds responsephrase option to the Response test to accommodate error message tests •removed redundant GET tests from irm and iwr tests. •Fixed markdown linting errors in README.md for WebListener

[feature] Replace Remaining HttpBin.org Tests with WebListener

70fab5c

markekraus requested review from TravisEz13, adityapatwardhan and daxian-dbw as code owners December 10, 2017 19:19

markekraus force-pushed the ReplaceHttpBin branch from 7412274 to 70fab5c Compare December 10, 2017 19:19

markekraus mentioned this pull request Dec 10, 2017

Tests should not rely on external websites which lead to random failures #2504

Closed

iSazonov reviewed Dec 11, 2017

View reviewed changes

iSazonov self-assigned this Dec 11, 2017

markekraus changed the title ~~Replace Remaining HttpBin.org Tests with WebListener~~ WIP: Replace Remaining HttpBin.org Tests with WebListener Dec 11, 2017

[Feature] Address PR Feedback

466d019

markekraus force-pushed the ReplaceHttpBin branch from 63b3559 to 466d019 Compare December 11, 2017 21:48

markekraus changed the title ~~WIP: Replace Remaining HttpBin.org Tests with WebListener~~ Replace Remaining HttpBin.org Tests with WebListener Dec 11, 2017

TravisEz13 reviewed Dec 12, 2017

View reviewed changes

TravisEz13 approved these changes Dec 12, 2017

View reviewed changes

iSazonov approved these changes Dec 12, 2017

View reviewed changes

anmenaga reviewed Dec 12, 2017

View reviewed changes

iSazonov closed this Dec 13, 2017

iSazonov reopened this Dec 13, 2017

iSazonov merged commit 19197e1 into PowerShell:master Dec 13, 2017

TravisEz13 added this to the 6.0.0-GA milestone Dec 16, 2017

TravisEz13 mentioned this pull request Dec 19, 2017

Merge 6.0.0-ga PRs #5713

Merged

markekraus deleted the ReplaceHttpBin branch January 19, 2018 18:59

	$docsToTest = @(
	'./*.md'
	'./docs/*.md'
	'./docs/building/*.md'
	'./docs/cmdlet-example/*.md'
	'./docs/installation/*.md'
	'./docs/maintainers/README.md'
	'./demos/SSHRemoting/*.md'
	'./docker/*.md'
	'./tools/*.md'
	'./.github/CONTRIBUTING.md'
	)

Replace Remaining HttpBin.org Tests with WebListener #5665

Replace Remaining HttpBin.org Tests with WebListener #5665

Uh oh!

Conversation

markekraus commented Dec 10, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR Summary

PR Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

markekraus Dec 11, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov-io commented Dec 12, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

anmenaga left a comment

markekraus commented Dec 10, 2017 •

edited

Loading

markekraus Dec 11, 2017 •

edited

Loading

codecov-io commented Dec 12, 2017 •

edited

Loading