Won't this add even more time to executing pwsh, you now have to wait for 2 processes to initialise and while it's minute it's still a problem today with just pwsh.exe itself.

It will add some time, but should be minimal impact as the shim is in native code, but there is a cost to starting a 2nd process, but I think overall it'll be minimal compared to the current limitations of pwsh startup with .NET

I think @awakecoding's comment has a good point. He suggests that, if we decide to go with the shim approach, then the bootstrapper pwsh can actually host CoreCLR runtime, and calls into pwsh.dll via its entry point. In this way, we don't need to spin up 2 processes and handle STD io redirections.

@daxian-dbw it's actually even better than that: you can literally use the hostfxr APIs to call into the pwsh.exe command-line entry point... within the current process. You literally load pwsh.exe and launch it within the PowerShell native host process without a subprocess! When using this specific mode of launching with a PowerShell native host, you can only do it as a one-shot thing where you basically just call it once and let it return, but that's precisely what we want here.

Thank @awakecoding, I just saw the run_pwsh_lib and run_pwsh_app in your project. What's the benefit of loading and launching pwsh.exe comparing to loading Microsoft.PowerShell.ConsoleHost.dll and calling into the entry point function Start(string[] args, int argc)? The former will load extra libraries (pwsh.exe and pwsh.dll).

Thank @awakecoding, I just saw the run_pwsh_lib and run_pwsh_app in your project. What's the benefit of loading and launching pwsh.exe comparing to loading Microsoft.PowerShell.ConsoleHost.dll and calling into the entry point function Start(string[] args, int argc)? The former will load extra libraries (pwsh.exe and pwsh.dll).

There is currently no way to load less than what pwsh.exe needs, and even if there was a way to do it (lazy loading of DLLs, etc) then the proper way would still be with a .NET native host process, so we're back to my technique again.

Also, how would you load ConsoleHost.dll from a process that isn't .NET, and without creating a subprocess? Chances are you would need to use the hostfxr APIs again to do this cleanly, and then you'd be limited to the command-line interface.

The PowerShell native host approach is the only one that avoids creating a subprocess and solves the problem very efficiently. Think of it as just replacing pwsh.exe out of the full PowerShell installation, except it doesn't rely on .NET at all, making it the perfect shim.

Also, how would you load ConsoleHost.dll from a process that isn't .NET, and without creating a subprocess? Chances are you would need to use the hostfxr APIs again to do this cleanly, and then you'd be limited to the command-line interface.

The pwrshplugin used by PowerShell for WSMan remoting loads System.Management.Automation.dll directly from native process to create a function pointer, see code here.

We also have an ancient native powershell host written in C++, and it used to load ConsoleHost.dll directly from native process, though it hasn't been used for a long time. See the code here.

We also have the DSC native agent that loads CoreCLR and calls into managed code form native code. So, I think it's doable, only that we need to figure out how to get the Trusted Assembly List (TPA) populated by using hostfx.

This is a similar approach that shouldn't change much, the sample you linked to appears to be using COM Interop to load PowerShell. While I believe it is still possible to use COM Interop on Windows with PowerShell 7, I do recall trying it on Linux only to realize the COM exports were missing on non-Windows.

The current project is only for Windows, but it wouldn't hurt to use the recommended hosting APIs, and that would mean the hostfxr APIs. Maybe the only thing we can conclude at this point is to use the hostfxr APIs to make a .NET host, and then move on to how we'd load and call PowerShell using it.

Agreed. Regardless of the implementation details, I believe we should make the bootstrapper a .NET host, if we want the bootstrapper a shim that always runs.

I think having the pwsh bootstrapper be a .NET host is interesting. If it's simply hosting the consolehost and passes all arguments to it to evaluate, then I would agree that is a fine solution.

You would also need to ensure this works in an interactive situation where just doing pwsh will spawn the process that inherits the console handles itself rather than a redirected stdio.

It also means if you do something like pwsh -c "my command" from WinPS it will not wait until until it is finished because the parent (bootstrap) process has ended.

good points, will make these explicit

Another issue is how do you get the exit code to bubble itself back up to the caller. You essentially need to either have the bootstrap process continue to run until pwsh is finished and have that set it's own exit code to that of pwsh's.

The suggestion of launching a subprocess pwsh.exe from the shim pwsh.exe has a lot of potential issues, but if we make the pwsh.exe shim a PowerShell native host process, we can actually make it behave exactly like the original without even spawning a subprocess. It would literally be the same as pwsh.exe but without PowerShell, and the ability to load the .NET runtime + launch PowerShell entirely from disk. I've done this before, just not with the full "let's find and download PowerShell on the fly" part.

Yea this does sound like a better solution, I'm not too well versed in hosting the CLR side but a colleague mentioned it should be possible for the pwsh shim to do the bootstrapping and then just host the CLR in the same process once that step is done. You save the hit of spawning yet another process and don't need to worry about IO redirection and other things like handle inheritance and the like.

I think the shim has to stay resident and manage the pwsh lifecycle. I have many scripts that depend on calling pwsh.exe synchronously.

Shouldn't this really be part of Windows/Microsoft Update? I dislike the idea of pwsh trying to do even more work at startup and potentially change the ground that I'm working on by automatically updating. I feel like the bootstrap script if this goes ahead should just install the initial version and updates are managed through WU.

I think the original reason not to just use WU was because it wasn't supported on Nanoserver, but since we've decided Nano is out of scope, we should revisit this

As of today, I think WU will work only if the installation is done through MSI and has the "use windows update" checked. How would it work for non-admin scenarios? I believe there will be many users using the non-admin installation.

@jborean93 Agreed. I'm not comfortable with the idea that my set of build nodes will update to a newer version of PowerShell merely by running pwsh as part of a build. I prefer to be "in control" of when that happens.

Use winget as fallback.

winget is not available on Windows Server and Windows Server support is required

@SteveL-MSFT Since you say that Nano Server is out of scope, does that mean that PowerShell could be delivered via MSI after all (rather than Squirrel mentioned above)? IIRC, all SKUs of Windows other than Nano Server support Windows Installer.

This is another reason for doing updates through WU, it's something already understood by admins and offers the ability for orgs to cache the update files on their own servers for distribution rather than having to find another way to get that data from the net.

Updating from 7.x to 7.x+1 can potentially break existing automation workload. Here are 2 real-word issue reported after the users moved to 7.2 from 7.1:

1. The reader's MaxDepth of 64 has been exceeded PowerShell#16457 -- PowerCLI module was broken in 7.2 because of a breaking change in Newtonsoft.Json that was made to prevent a DoS vulnerability.
2. Multiple Ambiguous Error under Windows 11 but not Windows 10 PowerShell#16506 -- method resolution failed for a working script because new overloads were added to System.Text.Json.JsonSerializer.Deserialize(...) that causes a conflict in method resolution.

Even if PowerShell itself can be backward regression free (best case scenario), the assemblies shipped with PowerShell can break user's existing workload, and there is nothing PowerShell can do to prevent that from happening. So, I think we should never automatically update from 7.x to 7.x+1. That should always be an explicit decision made by the user.

It's a matter of opt-in or opt-out. Secure by default means you have to opt-out of automatic updates. I would expect folks who rely on automation to be experts vs folks who simply want to use a shell

I would expect folks who rely on automation to be experts vs folks who simply want to use a shell

I'm not necessarily arguing for or against automatic updates, but FWIW in my experience the opposite is true. At least on the Windows side, there's a much larger amount of less experienced folks who can cobble a script together than folks who are even a little comfortable in a shell.

Feels like this would essentially circumvent software installation policies established by the enterprise and - in a worst case scenario - might become yet another hole enterprise will now need to be closing.

how would one control aspects of such installation? E.g. things like path, whether it is single/multi user install, logging and verbosity

There is another issue that comes to mind: what about software that first checks for the presence of pwsh.exe, and then calls "pwsh.exe --version" to detect the installed PowerShell version? Not only the shim would fool most software into thinking it's already installed, but the "pwsh.exe --version" version call would trigger the actual installation.

Yep, this is the same problem the Python for Windows folks created. They have a python.exe alias that build scripts detected as "Python is installed". Terrible dev experience.

In this case, PS7 should just be considered a Windows feature-on-demand (even though it doesn't use that mechanism specifically because a literal Feature-on-Demand requires the 10 year support lifecycle) and would install via Windows Update channel (this is in the RFC). So for all intents and purposes, pwsh.exe is in-box regardless if it needs to bootstrap itself initially. pwsh -version would work as though it was installed.

appears to me it would break a range of scenarios where users formed PS command lines and used explicit process start and wait (ShellExecute, _wpopen, Process.Start, etc.)

I think these problems are solvable although it might require the installed pwsh to be first in the path so that the bootstrapper is no longer called unless called explicitly after PS7 is installed

it might require the installed pwsh to be first in the path so that the bootstrapper is no longer called unless called explicitly after PS7 is installed

Yes please!

I'd like to add some specifics here.

Windows PowerShell is often used as a wrapper around poorly-written application installers. Software distribution systems, such as Configuration Manager or Microsoft Endpoint Manager (Intune), call powershell.exe and pass it the wrapper script. Termination of the powershell.exe process signals the caller (ConfigMgr, Intune, etc.) that the installation is complete, which triggers evaluation of application detection rules. If powershell.exe were to spawn another copy of itself and exit, the caller would run detection rules while the real installation was still in progress, and it would report failure to the user (e.g., in Software Center or Company Portal). If PowerShell (pwsh.exe) spawns another process and exits, then it cannot serve as a replacement for Windows PowerShell in this scenario.

That may not be a terribly compelling argument, since such uses would rarely require features beyond what Windows PowerShell provides; administrators could just keep using Windows PowerShell for that purpose. The same pattern is commonly used for other functionality, though, where the improved features of PowerShell may be highly useful and desirable. PowerShell is used as a gap-filler for functionality that is missing from such endpoint management products: If FunctionalityX is not provided by the system, an administrator can write a custom solution in PowerShell to implement the needed functionality and deploy it to devices as an "application" that can be installed and uninstalled. Again, if pwsh.exe were to launch a separate process and then exit, this scenario would be broken and render PowerShell unusable for this purpose.

ps1 registration/integration makes sense as feature of PowerShell installer; not something for bootstrapper to be solving.

"and not from Explorer" - few questions:

why blocking a launch from shell?

and in doing so, why special treatment to explorer?
if I am using another shell - e.g. Total Commander, what should be different about it?

Two other aspects come into my mind:

I am using self elevating PowerShell scripts.
How would this affect the secondary instance that's elevated.

For Windows Explorer please consider an implementation that's compatible to have PowerShell 7 when using the address line entering PowerShell
This currently launcher a PowerShell in the directory currently Windows Explorer opened. (same as cmd does)

What if my script has no TUI/GUI and running via Explorer blindly is an expected/valid scenario? This constraint seems orthogonal to the bootstrapper story.

The mention about explorer.exe is to prevent folks from downloading a .ps1 and accidentally executing it by double clicking it in Explorer. In this case, to support default execution by other processes (so they can just expect a foo.ps1 to work as foo.cmd does today), we would need to associate .ps1 with pwsh but special case the explorer.exe use case. Software today have to wrap .ps1 within a .cmd which makes them not write .ps1 and write batch files instead

This auto-update is so scary.

I maintain few dozen of psm1 modules - several hundred of exported functions total, and I would not jump to a different version of PowerShell that had not been scrutinized by thorough testing before.

Imagine - one day, because of some apparently small change somewhere in ConvertTo-Json function behavior - all CI pipelines fall apart, just because somebody maintaining the server didn't read the fine print.

I feel this area of the proposal needs a bit more scrutiny.

Having my automation engine upgrade outside of my patch cycle is kind of scary. I love the work the team has done to add PS7 to WU/MU. I believe that established process is the better method for upgrading versions.

Statically linking the CRT will prevent servicing should the need arise (e.g. security). Instead, I would look into the hybrid CRT solution. https://github.com/microsoft/WindowsAppSDK/blob/main/docs/Coding-Guidelines/HybridCRT.md

You could alternatively investigate using AppInstaller. Or at least evaluate it and include the "why not" in this section.

Isn't that MSIX specific? MSIX is not supported on Windows Server

@SteveL-MSFT MSIX is supported on Windows Server. You might be thinking of the AppInstaller app (which is separate from the .appinstaller manifest).

Will the .ps1 extension be added to the PATHEXT environment variable? I feel if the goal is to give PS scripts a first-class treatment on Windows similar to .bat and .cmd scripts, it'd make sense to give the ability to invoke them without having to specify the extension every time.

yes, that is the intent so that foo.ps1 could just be executed as foo

This may be out-of-scope, but please consider the Hosted App Model.
PowerShell/PowerShell#15758