Skip to content

Conversation

@dantraMSFT
Copy link
Contributor

@dantraMSFT dantraMSFT commented May 30, 2017

Fix #2227

Update Invoke-WebRequest and Invoke-RestMethod cmdlets to strip an Authorization header on redirect.

The FullCLR implementation uses WebRequest to perform the request which silently strips the Authorization header when a redirect occurs. The CoreCLR implementation uses HttpClient to perform the request which does not strip the authorization header. The change explicitly handles the initial redirect, removes the authorization header and submits the request to location in the response.
A new switch, PreserveAuthorizationOnRedirect, disables this handling.

See MicrosoftDocs/PowerShell-Docs#1269 for the doc change to document the new switch -PerserveAuthorizationOnRedirect.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: compatibility

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally, you should not mix formatting changes with functional changes. Have them as separate PRs to make them more readable and reviewable.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should put this in WebCmdletStrings.resx

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll remove it. Not appropriate for release.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Don't we need to support other 3xx redirect codes?

https://en.wikipedia.org/wiki/URL_redirection#HTTP_status_codes_3xx

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a race condition on Mac where stopping the httplistener and then starting a new one, the OS may not have cleaned up the actual underlying http service so the next start fails as the reservation still exists. In anycase, you should rebase against upstream/master as this script already starts and stops httplistener for all tests.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Seems like we should test multiple redirects to make sure it's stripped off the first one and subsequent redirects get handled correctly (3 may be sufficient)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should also have a test case where -PreserveAuthorizationOnRedirect is specified, but no Authorization header was supplied

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_cancelToken.Cancel(); [](start = 15, length = 23)

I am a little confused by this logic. It looks like an asynchronous call is being cancelled here. Is there a possibility of a race condition? Can you add a comment indicating what is being cancelled?

@PaulHigin
Copy link
Contributor

This seems to have a lot of unrelated changes (26 files now changed) now.

@SteveL-MSFT
Copy link
Member

@dantraMSFT looks like you accidentally pulled in other commits, you need to do a rebase -i to clean this up. Let me know if you need help with this.

@dantraMSFT dantraMSFT changed the title Automatically strip Authorization header on 302 redirects Automatically strip Authorization header on redirects Jun 7, 2017
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You need to make the corresponding doc change in PowerShell/PowerShell-Docs and mention it here to link the two

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's in progress. Will submit a PR shortly and update the description to reference it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, do we need this capability with invoke-restmethod?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't 308 also valid?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since you have this specific checks it seems you should have TestCases that go through the different status codes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

308 isn't currently supported in CoreCLR - HttpStatusCode doesn't have a define for it so there's no way for the code to see it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactored all tests to cover each redirect status code.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why isn't $response.content | convertfrom-json sufficient? this code seems unnecessary

Copy link
Contributor Author

@dantraMSFT dantraMSFT Jun 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Because I just want the json part of the raw content. The rest is extraneous for my purposes and it is also not valid json.
NOTE: $response.Content is a byte array.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at your usage below, you pass in the output of Invoke-WebRequest to this helper. This helper isn't necessary. Line 189 below can just be:

$result.Content = $result.Output.Content | ConvertFrom-Json

This is assuming the output you create in httplistener is json (and if not, perhaps it should be).

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The listener does add the output as json and I've even tried setting ContentType and ContentEncoding with no change in behavior. It appears it may be an issue on the client side. For now, I'll update the function to detect byte[] versus string and convert the byte[] to string before calling ConvertFrom-Json so it will work with arbitrary servers.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be better to have this as a -testcase where one of the parameters is $method and you can then pass Get and Post

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only a choice between GET and POST. No other methods are being used.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lines 182 and 186 below only differ by use of -Method. You should either change $UsePost to $Method or use splatting so there's only a single Invoke-WebRequest line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll update the function to use a ValidateSet and default to GET. I don't want to accept arbitrary methods since I'm explicitly testing the POST->GET logic on redirects.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a specific reason you want to add a new Describe rather than having your tests part of the existing Invoke-WebRequest tests?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I placed the tests in a new describe so I can run them in isolation while developing them. it turns out appveyor doesn't allow extra tag so I'll be removing it shortly.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should validate the value as well

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

validate the value

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will fix

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, I can't fix it without fixing ConvertTo-Json. $request.Headers is a WebHeaderCollection which is a NamedValueCollection (ICollection). ConvertTo-Json (and Select-Object for that matter) treat it as a collection and would have to special case the type.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why not validate all the headers are there except authorization?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't know explicitly what other headers 'should' be there so I don't have a source of truth to compare against what is echoed in the response. I know the authorization header should or should not be in a specific echoed response so I can verify it.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lines 182 and 186 below only differ by use of -Method. You should either change $UsePost to $Method or use splatting so there's only a single Invoke-WebRequest line.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking at your usage below, you pass in the output of Invoke-WebRequest to this helper. This helper isn't necessary. Line 189 below can just be:

$result.Content = $result.Output.Content | ConvertFrom-Json

This is assuming the output you create in httplistener is json (and if not, perhaps it should be).

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not entirely sure about the Pester syntax, but do you have to have $redirectedMethod here since you don't use it? Or does Pester complain?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

$response.Content is a byte array, not string ($result.Output.Content is this byte array). $response.RawContent has a few lines of text inserted by the underlying API followed by the json form of the original request; inserted by the httplistener. The purpose of the function is to extract the json from the RawContent and return it explicitly since it is needed to verify the incoming request.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll see if I can drop the unused $redirectedMethod params.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm remove the unused parameters.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought you were going to put these tests into an existing Describe?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, do we need this capability with invoke-restmethod?

@dantraMSFT
Copy link
Contributor Author

Please squash when merged.

Copy link
Member

@SteveL-MSFT SteveL-MSFT left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We try to avoid == with string comparisons and instead use string.Equals(a,b, StringComparison.Ordinal) or the appropriate comparison - this makes it clearer that the author considered culture and case when writing the code.

Copy link
Contributor Author

@dantraMSFT dantraMSFT Jun 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reqeusts [](start = 68, length = 8)

Spelling.

Copy link
Contributor Author

@dantraMSFT dantraMSFT Jun 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

false [](start = 76, length = 5)

Just a suggestion - we like to use named parameters when using bool constants - it makes it much easier to understand the code, so this could be GetResult(uri, stripAuthorization: false).

Copy link
Contributor Author

@dantraMSFT dantraMSFT Jun 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated.

Copy link
Contributor

@lzybkr lzybkr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There were a couple of minor issues I'd like to see addressed before I merge.

@dantraMSFT
Copy link
Contributor Author

@lzybkr Would you review my changes so I can get this merged before I go OOF.
Thanks.

@lzybkr
Copy link
Contributor

lzybkr commented Jun 15, 2017

I'll squash, but I'll ask you to write the commit message that follows our guidelines - you can just comment here or squash the changes yourself and update the PR, either way is fine.

@dantraMSFT
Copy link
Contributor Author

dantraMSFT commented Jun 15, 2017

@lzybkr Description updated. I think this is good now; can we get it merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants