-
Notifications
You must be signed in to change notification settings - Fork 8.1k
Add SslProtocol Support to WebCmdlets #5329
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add SslProtocol Support to WebCmdlets #5329
Conversation
* Adds `-SslProtocol` parameter to Web Cmdlets * Adds WebSslProtocol Enum to support limited subset of SslProtocol enum supported by HttpClientHandler * Adds TLS 1.1 and TLS 1.0 listening ports to WebListener * Adds 100% test coverage for new feature
| /// <summary> | ||
| /// The valid values for the -SslProtocol parameter for Invoke-RestMethod and Invoke-WebRequest | ||
| /// </summary> | ||
| [Flags] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is it possible to use this in other places? Then the name must be more generic.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's "public" in the way that other parameter enums are "public" but it shouldn't used in other places. users should use SecurityProtocolType or SslProtocol enums. This one is specific to the web cmdlets as HttpClientHandler doesn't support some of the protocols in those Enums and thus the web cmdlets cannot consume them. I did this Enum to avoid having write yet another paramter logic set inside the cmdlet body.
| [Flags] | ||
| public enum WebSslProtocol | ||
| { | ||
| /// <summary> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is "system default"? In .Net? Os?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It is environment specific and subject to implementation changes by CoreFX, thus the comment is not specific on purpose. It is SslProtocol.None... which is terribly named for PowerShell users as it implies no SSL/TLS would be used (that's a C# specific terminology for the 0 field on flag enums).
| } | ||
|
|
||
| // Set The SslProtocol. WebSslProtocol is used because not all SslProtocols are supported by HttpClientHandler. | ||
| // Also SslProtocols.Default is not the "default" for HttpClientHandler as SslProtocols.Ssl3 is not supported. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I believe it is better to move the comment in WebSslProtocol definition and make it more accurate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
will fix
| { | ||
| /// <summary> | ||
| /// No SSL protocol will be set and the system defaults will be used. | ||
| /// </summary> |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should be exactly Default = SslProtocols.None.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure how well that works with Flags. We have to define 0 for the default here and if CoreFX decides none is no longer 0 we lose our Default option.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see handler.SslProtocols = (SslProtocols)SslProtocol; so I believe Default = SslProtocols.None is more accurate.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
SslProtocols will always have a 0 too since it is a Flag as well. I'm weary of changing this one. Wehn I was researching I saw multiple asks for the SslProtocols to be cleaned up an None removed. it seems to be universally detested as a field name that is better defined as SystemDefault. Looking at the implementation of SecurityProtocolType they, too, erred on the side of using 0.
| { | ||
| 'Tls11' { $Uri.Port = $runningListener.Tls11Port } | ||
| 'Tls' { $Uri.Port = $runningListener.TlsPort } | ||
| default { $Uri.Port = $runningListener.HttpsPort } |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kestrel default protocol is really Tls12? If so we need comments here and in docs.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not Kesterl, but WebListener has Tls2 as the only protocol for HTTPS port. this is "default" in the sense that it is the original HTTPS port added when we launched WebListener.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see. Please add comments here and in README.md
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
ah.. I did have it in the readme.. it missed the commit. I'll add it
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed.
|
|
||
| // Set The SslProtocol. WebSslProtocol is used because not all SslProtocols are supported by HttpClientHandler. | ||
| // Also SslProtocols.Default is not the "default" for HttpClientHandler as SslProtocols.Ssl3 is not supported. | ||
| // Set The SslProtocol. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please remove obvious comment.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fixed
test/tools/WebListener/README.md
Outdated
| * The Server Certificate Password | ||
| * The TCP Port to bind on for HTTP | ||
| * The TCP Port to bind on for HTTPS | ||
| * The TCP Port to bind on for HTTPS using TLS 2.0 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What is real protocol we use? Tls12?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
oh whoops.. yes 1.2. fixed in all instances.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please clarify for me : is Tls 1.3 == Tls 2.0 today?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Kind of... as I understand it a minor version iteration of 1.3 is 2.0. They should have versioned 1.3 as 2.0 as it was a significant change from 1.2, 1.1, and 1.0. In any case, neither 1.3 or 2.0 is supported by HttpClientHandler at this time. My hope is that when they decide on a new Enum for all of this, they at least pick field names that make sense instead of None being "whatever default the system uses" and Default being "one version that is not widely supported by APIs and another that is out of date".
* TLSO 2.0 -> TLS 1.2 * remove onbvious comment
| /// <summary> | ||
| /// Gets or sets the TLS/SSL protocol used by the Web Cmdlet | ||
| /// </summary> | ||
| [Parameter] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I should ask: maybe use WebSslProtocol for parameter name?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the Web is kind of redundant in the parameter name. I was just continuing with the convention of prefixing Web in front of Enums used for web cmdlet parameter options. Technically, it is setting the SslProtocol, we are just limiting the options that the user can use to those it actually supports.
iSazonov
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM with one concern about Default = 0 vs Default = SslProtocols.None.
| $result.headers.Host | Should Be $params.Uri.Authority | ||
| } | ||
|
|
||
| It "Verifies Invoke-WebRequest -SslProtocol -SslProtocol <IntendedProtocol> fails on a <ActualProtocol> only connection" -TestCases @( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should have some tests were intended is multiple protocols. that should be allowed since you used a flags enum.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Added.
|
@TravisEz13 It looks like macOS does not support multiple protocols being set at once. This is not surprising as the entire SSL/TLS/Certificate implementation in CoreFX is flakey on macOS where some things work sometimes, not at all, or only partially. I'm going to logic gate the multi-protocol test cases so they don't run on macOS. $handler = [System.Net.Http.HttpClientHandler]::new()
$handler.SslProtocols = [System.Security.Authentication.SslProtocols]::Tls -bor [System.Security.Authentication.SslProtocols]::Tls11
$HttpClient = [System.Net.Http.HttpClient]::New($handler)
$Response = $HttpClient.GetAsync("https://httpbin.org/user-agent").GetAwaiter().GetResult()result |
|
Hmm. It looks like the $url = Get-WebListenerUrl -Test Get -Https -SslProtocol Tls12
$handler = [System.Net.Http.HttpClientHandler]::new()
$handler.ServerCertificateCustomValidationCallback = [System.Net.Http.HttpClientHandler]::DangerousAcceptAnyServerCertificateValidator
$handler.SslProtocols = [System.Security.Authentication.SslProtocols]::Tls -bor [System.Security.Authentication.SslProtocols]::Tls12
$HttpClient = [System.Net.Http.HttpClient]::New($handler)
$Response = $HttpClient.GetAsync("$url").GetAwaiter().GetResult()I'll logic this one out of being run on both linux and macOS. When I get some time I will open an issue with CoreFX. |
Possible CoreFX bug where `Tls, Tls12` does not honor the Tls12 on a Tls12 endpoint.
|
In previous years, we had a simple logic when we started with a highest protocol and fallback at a lower. |
|
@iSazonov Your recommendation is to switch it to a Non-Flags Enum? My plan was to not document the possibility of multiple protocols and leave as an exercise for advanced users. They don't show up in intellisense and unless someone advanced enough in the l;language took a good look at the Enum, they wouldn't really know. Multiple protocols work 100% on windows , 99% on linux and 0% on macOS. The linux one is a bug and macOS curl is a known trouble spot in CoreFX that they have been steadily improving. I'd rather leave it in and let advanced users give it a try than take the option away. If there is a concern, it could be documented that "setting multiple protocols is possible, though not supported on all platforms". |
|
Thinking about security I believe we lost nothing if remove |
|
@iSazonov you are right in that there is no security loss. But, lets say a user has a contractual agreement to use TLS 1.1 at minimum, they could supply So no, security loss, but, we do make it harder for the user. |
|
Since POODLE attacks TLS fallbacks was disabled. Currently only client moderated fallbacks is allowed by means of TLS_FALLBACK_SCSV fake cipher suite. |
|
@iSazonov I'm not exactly sure what is being done, but I know for a fact from the tests in This PR, that if you supply |
|
This design is similar to most browsers. I don't think we need to remove fallback. The user can of course specify just one protocol. We are giving the user the control. |
|
The design - yes, most of browsers has check boxes to select some protocols. But its behavior is secure - all of browsers disable fallback since POODLE discovered. Currently client and server must support TLS_FALLBACK_SCSV to do moderated fallback. If PowerShell and .Net Core allow to select some protocols and don't support TLS_FALLBACK_SCSV - it is security hole, an user using PowerShell can be redirected to a fake (Azure) server. |
|
@iSazonov I do not follow your logic. User has a requirement to NOT use TLS 1.0. The user supplies
I'm not sure how POODLE is even a problem since SSL 3.0 is not even supported in CoreFX by the When |
|
@markekraus Currently not only TLS->SSL fallback is security hole, TLS high level -> TLS low level also is security hole and is under the prohibition. If PowerShell is trying to connect with TLS 1.2 and server reject the connection, PowerShell shouldn't fallback to TLS 1.1 in classic way (only by supporting TLS_FALLBACK_SCSV). Also if today CoreFX don't implement the same behavior for all platforms it is better for script portability to be limited to one protocol. |
|
@iSazonov I doesn't matter. if the user doesn't want TLS 1.0, or TLS 1,1, they specify |
Then we should rip out all of our security features for macOS since they are all partially or not supported or only work under the right circumstances. or, we should completely rip out these cmdlets from macOS because the entire CoreFX HttpClient implementation on macOS is riddled with issues. When we go RTM I am expecting macOS users to flood with web cmdlets issue. CoreFX is just weak there. This kind of thing just needs to be documented and to a degree it will be expected by users. Python, PHP, and other cross-platform languages have plenty of their own cross-platform implementation quirks. Something being either not supported on one platform or partially supported on another never stops other languages from implementing enhancements for a majority of platforms. Again, multiple protocols works 100% on Windows, 99% on Linux (the one instance that doesn't work appears to be a bug in CoreFX) and not at all on macOS. I don't think throwing the feature out just because macOS and CoreFX have issues with libcurl is worth it, when all windows platforms, and almost all Linux platforms will support it. If we start down that path we will never get to implement anything that is an improvement for most platforms just because one platform happens to be fragile. If this only worked on Windows, then maybe I would agree it's not worth implementing. CoreFX will have better support for these things in macOS in the future. Their issues are littered with HttpClient implementation issues on macOS. They know it's a weak point and we should acknowledge it as one for ourselves as well. |
If I were a user of PowerShell on MacOs, I'd feel cheated if I had cmdlets that didn't work. |
I'd rather have a web cmdlets that at least works for the most part with basic web calls than none at all. Even if certain features aren't all supported on my platform, I'd still want it. The point is that partial support is better than none at all. |
|
If we need to remove a disable the cmdlets for macOS file another issue. For the fallback issue, I think fallback is a required feature based on past experience. If we need a more restricted fallback, I think we would need support for .NET to support that. |
|
I'm leaving a note here for those who view this discussion in the future and for clarity to avoid panic: The issues I have seen with For the most part, simple HTTP calls work to a high rate of success. However, they will randomly fail even in our tests in this project for macOS where they are not failing for the same tests in Windows and Linux. Some of that maybe Travis CI related. However, since I acquired a macOS machine to test with I have seen similar stability issues on my own machine. Again, I don't know how much of that is just bad luck, my lack of experience with macOS in general, or possible stability issues in CoreFX. I cannot reliably reproduce the issues I find, so it is impossible at my skill level to track down a specific root cause. As such, I can only say, without solid evidence, that I have general distrust for the stability of It is clear, however, that many of the features relating to security and encryption are not fully implemented CoreFX on macOS. They work at the basic level depending on how the environment is configured. Sometimes they do not work at all, again, depending on how the environment is configured. However, I do not feel it is prudent to cripple these features for all platforms simply because a single platform cannot full implement them. |
|
@markekraus Clould you please open tracking Issue? It wouldd be usefull for documentating the limitations and tracking CoreFX. |
|
That would be very good if users were given this description (broken cmdlets) in the documentation (release notes). If you open the issue anyone can complete it at any time. |
closes #2662
This feature adds the ability to restrict the SSL/TLS protocol used when making the web request. In 5.1 the user could make use of .NET API's to enforce this on the Web Cmdlets. With the move to
HttpClientin PowerShell Core, those APIs have no impact. The user still has requirements to ensure specific protocols are used.The public enum
WebSslProtocolis added as a wrapper to the underlyingSslProtocolsenum. Neither it norSecurityProtocolTypecan be used becauseSsl3andSsl2are not supported byHttpClientHandler.SslProtocols. While it may not be intuitive to a PowerShell user to use-boror"Tls, Tls11"to set multiple options, the general use case for this will be a single protocol.-SslProtocolparameter to Web CmdletsWebSslProtocolEnum to support limited subset of SslProtocol enum supported byHttpClientHandlerI spoke with @SteveL-MSFT about getting this approved before RC release.