Skip to content

Conversation

@SteveL-MSFT
Copy link
Member

@SteveL-MSFT SteveL-MSFT commented Nov 13, 2021

PR Summary

Some native unix command will shell out to run something (like ssh) and uses the bash built-in exec to spawn a new process that replaces the current one. This fails when PowerShell is the default shell as exec is not a valid command.
This is affecting some known scripts like copy-ssh-id or some subcommands of AzCLI.

Since Replace is not a valid verb, I decided to use Switch. Also, the other "Process" cmdlets are in the ManagementModule, but this is not related to those cmdlets and is really a Core cmdlet expected to be part of a Unix shell.

This PR adds a new Switch-Process cmdlet aliased to exec that calls execv() function to provide similar behavior as POSIX shells. This is not intended to have parity with the exec built-in function in POSIX shells (like how file descriptors are handled), but should cover most cases. This cmdlet only shows up for non-Windows systems.

We should consider backporting to 7.2 (as LTS release) for this reason, but should wait on sufficient user feedback in 7.3.

This may conflict with a user defined function called exec on non-Windows systems, but this seems necessary and uncommon. @PowerShell/powershell-committee had previously decided this was acceptable.

PR Context

Fix #4691

PR Checklist

@iSazonov
Copy link
Collaborator

I wonder MSFT team denies its own policy of not increasing Engine :-) Why don't put the short function in AzCLI with a helper module to collect a feedback. Then we could put the function in Utility module for 7.3.

@SteveL-MSFT
Copy link
Member Author

@iSazonov it's not just AzCLI, that's just the most recent example. I've hit this myself a few times and it's really hard to figure out what's going on as an end user as the native command doesn't expect exec not to work so you get incorrect error messages. In this case, I think it's a core cmdlet and expected of a shell on Unix systems.

@iSazonov
Copy link
Collaborator

I think it's a core cmdlet and expected of a shell on Unix systems.

@SteveL-MSFT I see the point and I approve the intention but I'm afraid we may fall back into a long commit series of adaptations, especially if there is an intention to backport.
I mean it would be nice to see "we have received positive feedbacks for our helper module from Azure users in a few months" before we include this in Engine.

@iSazonov
Copy link
Collaborator

What do we know popular scripts using the exec which we could test with the solution? I see only two references in the related issue.

@daxian-dbw
Copy link
Member

daxian-dbw commented Nov 13, 2021

Why not do the real system call execv on Unix platforms? We already have some code for that in src\PowerShell\Program.cs.

Also, why making this visible on Windows too? I know why you exposed it on Windows after seeing the tests. I doubt its usefulness on Windows 😕

@iSazonov
Copy link
Collaborator

Why not do the real system call execv on Unix platforms? We already have some code for that in src\PowerShell\Program.cs.

We call sh -l -c 'exec pwsh "$@"' there. But we could call an exec here.

@SteveL-MSFT
Copy link
Member Author

I'm going to re-work this PR so that exec calls execv api and is, of course, limited to just non-Windows.

fcharlie added a commit to baulk/baulk that referenced this pull request Nov 17, 2021
exec is a built-in function
See: PowerShell/PowerShell#16462
@SteveL-MSFT SteveL-MSFT changed the title Add exec function for bash compatibility Add exec cmdlet for bash compatibility Nov 20, 2021
@JamesWTruher
Copy link
Collaborator

It occurs to me that there might be some insight that the dotnet folks may have on this. I'm thinking especially of open file handles (which are preserved through the exec ) - this could mean that some clean up that happens when a managed app exits won't happen. I'm fairly certain that exec will not look like a standard process end to dotnet, but not absolutely sure.

@JamesWTruher
Copy link
Collaborator

Also, since this is a PowerShell view of exec I would consider adding a parameter -Environment as a hashtable which would enable the use of execve

@JamesWTruher
Copy link
Collaborator

The actual implementation should be in our native library and we should be calling the API from that library rather than directly having the system call.

@JamesWTruher
Copy link
Collaborator

We also need to figure out what to do about the exec code in src/powershell/Program.cs shouldn't there be a single call here rather than a PInvoke here and one there?

@SteveL-MSFT
Copy link
Member Author

SteveL-MSFT commented Nov 30, 2021

@JamesWTruher over time, we probably want to get rid of our native library and move them all to P/Invoke or .NET API calls. This will make it easier for us to support new architectures and also support MSIL builds. As for the separate execv calls, we already have this pattern where we don't necessarily want different assemblies calling into other assemblies internal methods for the purpose of P/Invoke calls. In this case, the ConsoleHost code is specific to ConsoleHost for purpose of working as a login shell and this is something that could be used in EditorServices, for example.

I've also asked .NET team to review this.

@iSazonov
Copy link
Collaborator

I've also asked .NET team to review this.

They introduced new GeneratedDllImport attribute and it would be great if they reviewed PowerShell Native repo too - can we rid of the PowerShell Native now (after moving to .Net 7) or no?
Also .Net has System.Native.dll - have they plan to rid of it or could we benefit from it to replace PowerShell Native?

@SteveL-MSFT
Copy link
Member Author

@iSazonov I thought I had opened an issue previously on moving away from PowerShell-Native, but I created this issue PowerShell/PowerShell-Native#76. I haven't looked to see what the effort would look like based on new .NET APIs that weren't available when PS Core 6 first needed this library.

@JamesWTruher
Copy link
Collaborator

with regard to eliminating the native library, there may be difficulties with some of the APIs. I know when I did the UnixStat work I needed to handle all the platform differences in returning the stat structure. The stat structure is different on Mac, Linux, and ARM - this would likely be problematic - i solved it in the native library by returning a CommonStat structure which smoothed over the rough edges of these differences.

@iSazonov
Copy link
Collaborator

iSazonov commented Dec 4, 2021

The stat structure is different on Mac, Linux, and ARM - this would likely be problematic - i solved it in the native library by returning a CommonStat structure which smoothed over the rough edges of these differences.

It is great request for .Net team. I see their dll import generator works today for simple types but structs are under question.
Then historically we do condition compiling (with #if) because absence .Net API. As soon as this API appears, we remove these conditions.
GeneratedDllImport is intermediate solution. It allow to move workarounds and mappings to managed code like if Platform.IsMac -> p/invoke GetMacStat() -> remap Mac Stat struct to generic Stat struct.

@pull-request-quantifier-deprecated

This PR has 127 quantified lines of changes. In general, a change size of upto 200 lines is ideal for the best PR experience!


Quantification details

Label      : Medium
Size       : +125 -2
Percentile : 45.4%

Total files changed: 6

Change summary by file extension:
.cs : +75 -1
.resx : +6 -0
.ps1 : +44 -1

Change counts above are quantified counts, based on the PullRequestQuantifier customizations.

Why proper sizing of changes matters

Optimal pull request sizes drive a better predictable PR flow as they strike a
balance between between PR complexity and PR review overhead. PRs within the
optimal size (typical small, or medium sized PRs) mean:

  • Fast and predictable releases to production:
    • Optimal size changes are more likely to be reviewed faster with fewer
      iterations.
    • Similarity in low PR complexity drives similar review times.
  • Review quality is likely higher as complexity is lower:
    • Bugs are more likely to be detected.
    • Code inconsistencies are more likely to be detetcted.
  • Knowledge sharing is improved within the participants:
    • Small portions can be assimilated better.
  • Better engineering practices are exercised:
    • Solving big problems by dividing them in well contained, smaller problems.
    • Exercising separation of concerns within the code changes.

What can I do to optimize my changes

  • Use the PullRequestQuantifier to quantify your PR accurately
    • Create a context profile for your repo using the context generator
    • Exclude files that are not necessary to be reviewed or do not increase the review complexity. Example: Autogenerated code, docs, project IDE setting files, binaries, etc. Check out the Excluded section from your prquantifier.yaml context profile.
    • Understand your typical change complexity, drive towards the desired complexity by adjusting the label mapping in your prquantifier.yaml context profile.
    • Only use the labels that matter to you, see context specification to customize your prquantifier.yaml context profile.
  • Change your engineering behaviors
    • For PRs that fall outside of the desired spectrum, review the details and check if:
      • Your PR could be split in smaller, self-contained PRs instead
      • Your PR only solves one particular issue. (For example, don't refactor and code new features in the same PR).

How to interpret the change counts in git diff output

  • One line was added: +1 -0
  • One line was deleted: +0 -1
  • One line was modified: +1 -1 (git diff doesn't know about modified, it will
    interpret that line like one addition plus one deletion)
  • Change percentiles: Change characteristics (addition, deletion, modification)
    of this PR in relation to all other PRs within the repository.


Was this comment helpful? 👍  :ok_hand:  :thumbsdown: (Email)
Customize PullRequestQuantifier for this repository.

Copy link
Member

@daxian-dbw daxian-dbw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@daxian-dbw daxian-dbw added CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log and removed Review - Needed The PR is being reviewed labels Jan 4, 2022
@daxian-dbw daxian-dbw merged commit a9b14a1 into PowerShell:master Jan 4, 2022
@SteveL-MSFT SteveL-MSFT deleted the exec branch January 4, 2022 22:29
TrapGodBrim pushed a commit to TrapGodBrim/PowerShell that referenced this pull request Jan 19, 2022
@ghost
Copy link

ghost commented Feb 24, 2022

🎉v7.3.0-preview.2 has been released which incorporates this pull request.:tada:

Handy links:

@TravisEz13
Copy link
Member

@PowerShell/powershell-maintainers
Although it is potentially unstable, it has had some bake time, and the functionality is behind and experimental feature.

Let's give this some bake time to verify that it is stable.

@TravisEz13
Copy link
Member

I have seen customers that implement exec as a cmdlet which this behavior could break. The maintainers are inclined not to backport this unless we have a compelling argument to backport to the last LTS.

@TravisEz13 TravisEz13 added the Review - Committee The PR/Issue needs a review from the PowerShell Committee label Mar 29, 2022
@SteveL-MSFT SteveL-MSFT added Committee-Reviewed PS-Committee has reviewed this and made a decision and removed Review - Committee The PR/Issue needs a review from the PowerShell Committee labels Apr 27, 2022
@SteveL-MSFT
Copy link
Member Author

@PowerShell/powershell-committee reviewed this. Given that this is an experimental feature in 7.3 and the risk of impact of an existing exec command, we do not recommend backporting this to 7.2 pending customer feedback that it blocks their adoption of 7.2 in the future.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CL-General Indicates that a PR should be marked as a general cmdlet change in the Change Log Committee-Reviewed PS-Committee has reviewed this and made a decision Medium

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ssh-copy-id fails if powershell is default shell

6 participants