Skip to content

pwsh (Mac) does not work with "expect" correctly (unusual escape characters printed) #18874

Closed

Description

@christopherdwelton

Prerequisites

Steps to reproduce

pwsh on Mac is showing odd output when used with /usr/bin/expect
I first noticed this while using pwsh with "unbuffer", which is a shell wrapper for expect

I managed to repro with just:
bash>$ expect -c "spawn pwsh -c [command]; expect eof"
where [command] is another compiled program rather than a cmdlet

For instance:
bash>$ expect -c "spawn pwsh -c Write-Output 'PowerShell 7.3.1'; expect eof"

Works fine, and has the following output:

spawn pwsh -c Write-Output 'PowerShell 7.3.1'`
PowerShell 7.3.1

On the other hand,
bash>$ expect -c "spawn pwsh -c pwsh --version; expect eof"

which should have the same output (and does when not run from expect), outputs this:

spawn pwsh -c pwsh --version
^[[31;1R^[[31;9Rbash>$ ;1R;9R

Note that some of the garbage comes out after the prompt
Also note, this is using expect version 5.45. In addition, I rebuilt expect from brew to be sure, and it was still the same.
No other shells have an issue like this with expect.

This started when I had written a simple python script that writes to stdout and stderr on a clock, and was using it to see when a program's stdout would be buffered, and when it would think it was printing to a terminal and come out in real time. (This was how this all started, as long scripts that buffer their stdout but not stderr can have stack traces that come out dozens if not hundreds of lines from the corresponding output, which is annoying)

Here is the program, in python 3.10.7

import os
import sys
import time

i: int = 0

while i < 4:
    sys.stdout.write(f'Stdout mark {i}{os.linesep}')
    sys.stderr.write(f'Stderr mark {i}{os.linesep}')
    time.sleep(1)
    i += 1

when launched from:
pwsh -c python3 clock.py
The output is as expected:

Stdout mark 0
Stderr mark 0
Stdout mark 1
Stderr mark 1
Stdout mark 2
Stderr mark 2
Stdout mark 3
Stderr mark 3

when launched from:
unbuffer pwsh -c python3 clock.py
(unbuffer being a wrapper for expect) the command hangs for a few seconds, then the output is as follows:

^[[32;1R^[[32;9RStdout mark 0
Stderr mark 0
Stdout mark 1
Stderr mark 1
Stdout mark 2
Stderr mark 2
Stdout mark 3
Stderr mark 3
bash>$ ;1R;9R

Once again, note that some of the garbage is printed into the terminal after prompt is written.

Expected behavior

bash>$ expect -c "spawn pwsh -c pwsh --version; expect eof"
spawn pwsh -c pwsh --version
PowerShell 7.3.1

Actual behavior

bash>$ expect -c "spawn pwsh -c pwsh --version; expect eof"
spawn pwsh -c pwsh --version
^[[32;1R^[[32;9Rbash>$ ;1R;9R

Error details

No response

Environment data

Name                           Value
----                           -----
PSVersion                      7.3.1
PSEdition                      Core
GitCommitId                    7.3.1
OS                             Darwin 21.6.0 Darwin Kernel Version 21.6.0: Wed Aug 10 14:28:35 PDT 2022; root:xnu-8020.141.5~2/RELEASE_ARM64_T8101
Platform                       Unix
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

Activity

  1. SteveL-MSFT commented on Jan 3, 2023

    @SteveL-MSFT
    Member

    That specific escape sequence is for scrolling margins. Looking through the PS7 code base, it's not explicitly emitted by PS7. I couldn't find it in the dotnet/runtime code base either. This repros with PS7.0 so it's not something new that's been introduced. Couldn't figure out yet where it's coming from.

  2. JamesWTruher commented on Jun 14, 2023

    @JamesWTruher
    Collaborator

    Is it possible to try this with a very simple C# application (I'm not sure if this is a dotnet thing or a pwsh thing). Ok - it's not a plain dotnet thing, my simple c# hello world didn't have this issue.

  3. added
    Needs-InvestigationThe behavior reported in the issue is unexpected and needs further investigation.
    and removed
    Needs-TriageThe issue is new and needs to be triaged by a work group.
    on Jun 14, 2023
  4. JamesWTruher commented on Jun 14, 2023

    @JamesWTruher
    Collaborator

    the WG discussed and determined that there is a problem when PS starts a new process. It attempts to determine whether screen scraping is available (via GetCursorPosition) which results in an escape sequence being sent to the tty. This escape sequence generates a response which is not captured by expect or pwsh and it tumbles to the input (which is why when you press enter, you attempt to invoke what was returned). We believe that we know how to correct this.

  5. added
    Issue-BugIssue has been identified as a bug in the product
    and removed
    Needs-InvestigationThe behavior reported in the issue is unexpected and needs further investigation.
    on Jun 14, 2023
  6. ghost added
    In-PRIndicates that a PR is out for the issue
    on Jun 19, 2023
  7. ghost removed
    In-PRIndicates that a PR is out for the issue
    on Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions