Skip to content

Data passed through a pipe will be parsed differently in PowerShell 5.1 vs current release #17523

@lclutz

Description

@lclutz

Prerequisites

Steps to reproduce

When outputting UTF-8 from a native application PowerShell pipes fail to parse the data correctly even if a BOM is present.
In contrast this works in PowerShell 5.1 - that's why I'm reporting this as a bug.

The following python script will produce UTF-8 encoded output with a BOM.

#!/usr/bin/env python3
import sys
sys.stdout.buffer.write("äöüß αβγδ\n".encode("utf_8_sig"))

Powershell 5.1 is able to recognise the output is UTF-8 encoded if the BOM is present. PowerShell 7.2.4 does not.

Note that both shells were started with the -noprofile option to ensure default configuration.

Expected behavior

PS> python .\test.py | echo
äöüß αβγδ

Actual behavior

PS > python .\test.py | echo
´╗┐├ñ├Â├╝├ƒ ╬▒╬▓╬│╬┤

Error details

No response

Environment data

PS> $PSVersionTable

Name                           Value
----                           -----
PSVersion                      7.2.4
PSEdition                      Core
GitCommitId                    7.2.4
OS                             Microsoft Windows 10.0.19044
Platform                       Win32NT
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0…}
PSRemotingProtocolVersion      2.3
SerializationVersion           1.1.0.1
WSManStackVersion              3.0

Visuals

No response

Metadata

Metadata

Assignees

Labels

Needs-TriageThe issue is new and needs to be triaged by a work group.Resolution-No ActivityIssue has had no activity for 6 months or moreWG-Enginecore PowerShell engine, interpreter, and runtime

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions