Skip to content

Add ABI-compatible Rust native implementations and side-by-side CI - #118

Open
Steve Lee (SteveL-MSFT) wants to merge 10 commits into
masterfrom
stevel-msft-native-api-assessment
Open

Steve Lee (SteveL-MSFT) wants to merge 10 commits into
masterfrom
stevel-msft-native-api-assessment

Conversation

@SteveL-MSFT

@SteveL-MSFT Steve Lee (SteveL-MSFT) commented Nov 11, 2026 •

Copy link
Copy Markdown
Member

Motivation

Provide an additive Rust implementation of the native APIs while preserving compatibility for existing callers. Since callers can use .NET 11 or newer, document managed migration paths without removing the existing native entry points.

Approach

  • Implement all 30 Unix exports with the original C ABI, including native boolean widths, stat/time layouts, host-libc string allocation, and errno behavior. Document intentional fixes to unsafe failure paths.
  • Port the Windows launcher, CoreCLR/.NET Framework hosting, and shell integration. Leave the WSMan plugin and all original C++ sources/builds unchanged.
  • Package separate Linux x64/arm64 glibc and musl shared libraries. Musl is deliberately host-linked, not embedded as a second libc, to preserve allocator and errno compatibility with the calling process.
  • Add GitHub Actions that build C++ and Rust side-by-side into artifacts/cpp/<rid>/ and artifacts/rust/<rid>/. Enforce workspace-wide pedantic Clippy with warnings as errors, and add Pester v5 packaged-binary tests with NUnit XML reports retained on failure.
  • Statically link the MSVC CRT on x86/x64 and verify packaged launcher imports. Guard null COM variants before creating owning wrappers, with ownership regression tests.

Windows compatibility

CoreCLR retains the original three-argument UnmanagedPSEntry.Start delegate contract. CI uses checksum-verified PowerShell 6.0.5 payloads to exercise it on both architectures. This end-of-life payload is a compatibility fixture, not a deployment recommendation; modern payloads with overloaded Start entry points do not satisfy that legacy contract.

The Framework host retains COM-visible managed activation and IDispatch invocation. Windows PowerShell 5.1 is intentionally unsupported, as requested: its console-host assembly disables COM visibility. Framework integration tests instead compile and activate a COM-visible C# fixture, verifying argument marshaling and positive/negative exit codes without changing registry configuration or adding reflection-based hosting.

Validation

CI run 42950325986 passes all eight matrix jobs:

  • Linux x64/arm64: C++/Rust side-by-side builds, pedantic Clippy, glibc/musl runtime tests, original-header ABI clients, and ELF linkage checks.
  • macOS arm64/x64: C++/Rust builds, pedantic Clippy, native tests, original-header ABI clients, and packaged-library Pester tests.
  • Windows x86/x64, CoreCLR/Framework: C++ baseline builds, Rust build/lint/tests, static CRT import checks, and packaged-launcher Pester tests. CoreCLR integration verifies real managed startup, architecture, Unicode/empty/spaced/quoted/trailing-backslash arguments, and signed exit codes; Framework integration verifies the preserved COM hosting contract with the managed fixture.

CodeQL, C++ analysis, CLA, and all other reported PR checks also pass. All three review threads are addressed and resolved.

Local validation includes 19 Rust tests and 44 Pester tests on macOS arm64, cross-target pedantic Clippy, Linux/macOS cross-builds, original-header ABI clients on macOS arm64 and x64/Rosetta, actionlint, and PowerShell parsing.

Pester covers musl x64 using a musl PowerShell host; musl arm64 instead runs native Rust and original-header C++ ABI tests because PowerShell 7.7.0 has no published standalone musl-arm64 host.

Port the Unix native APIs and Windows launcher/hosting code to Rust while preserving the original C ABI and leaving the C++ WSMan plugin unchanged. Document .NET 11 migration paths and package separate GNU and host-linked musl libraries for Linux x64 and arm64.

Build C++ and Rust side-by-side in GitHub Actions, enforce pedantic Clippy, and add Pester v5 packaged-binary tests with NUnit reports.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A legal null COM variant can trigger a crash, packaged MSVC binaries depend on an unbundled VC runtime, and CoreCLR tests do not cover successful managed execution.

3 open issues
What changed in this PR

Adds Rust implementations alongside the existing native C++ code, preserving the Unix C ABI and providing Windows launcher and hosting support while leaving the WSMan plugin unchanged.

Changes:

  • Implements all 30 Unix exports and Windows CoreCLR/.NET Framework hosting.
  • Adds side-by-side builds, architecture/libc-specific packaging, and native and Pester validation.
  • Documents compatibility contracts, deployment requirements, and managed migration options.
File Description
rust/​windows/​tests/​launcher-smoke.ps1 Checks launcher resources and startup.
rust/​windows/​src/​win.rs Provides Windows API bindings and resource wrappers.
rust/​windows/​src/​shell.rs Implements shortcuts and taskbar integration.
rust/​windows/​src/​policy.rs Defines parsing and hosting policies with tests.
rust/​windows/​src/​main.rs Adds the launcher entry point.
rust/​windows/​src/​lib.rs Exposes Windows hosting modules.
rust/​windows/​src/​launcher.rs Processes arguments and invokes managed hosts.
rust/​windows/​src/​hosting.rs Implements CLR hosting and COM Automation.
rust/​windows/​src/​common.rs Handles registry, configuration, and XML discovery.
rust/​windows/​README.md Documents Windows builds and deployment contracts.
rust/​windows/​Cargo.toml Defines launcher features and Windows dependencies.
rust/​windows/​build.rs Embeds launcher resources using Windows SDK tools.
rust/​tests/​WindowsLauncher.Tests.ps1 Tests packaged Windows launchers.
rust/​tests/​NativeMethods.cs Defines managed Unix ABI test bindings.
rust/​tests/​NativeLibrary.Tests.ps1 Tests packaged Unix exports and behavior.
rust/​tests/​Invoke-Tests.ps1 Runs Pester and writes NUnit reports.
rust/​README.md Documents compatibility, builds, migration, and validation.
rust/​libpsl-native/​tests/​native.rs Tests Unix API behavior and layouts.
rust/​libpsl-native/​tests/​abi.ps1 Checks managed string, boolean, and errno compatibility.
rust/​libpsl-native/​tests/​abi.cpp Validates ABI compatibility using original headers.
rust/​libpsl-native/​src/​time.rs Implements native time conversion and clock setting.
rust/​libpsl-native/​src/​process.rs Implements process creation, identity, and control.
rust/​libpsl-native/​src/​logging.rs Implements syslog and macOS unified logging.
rust/​libpsl-native/​src/​lib.rs Exposes Unix APIs and shared ABI helpers.
rust/​libpsl-native/​src/​filesystem.rs Implements filesystem and metadata exports.
rust/​libpsl-native/​src/​accounts.rs Implements account and process-owner lookups.
rust/​libpsl-native/​Cargo.toml Defines the Unix library crate.
rust/​Cargo.toml Configures workspace lint and release policies.
rust/​Cargo.lock Locks Rust dependencies.
rust/​build-linux.py Builds and packages four Linux variants.
rust/​.gitignore Excludes Rust build outputs.
rust/​.cargo/​config.toml Configures Linux linkage and shared-library naming.
README.md Introduces the additive Rust implementation.
.gitignore Excludes repository-level artifacts.
.github/​workflows/​rust-native.yml Adds cross-platform side-by-side builds and validation.

Review effort: Balanced

Comment thread .github/workflows/rust-native.yml Resolved
Comment thread rust/windows/src/hosting.rs Resolved
Comment thread rust/windows/src/hosting.rs Resolved
Isolate Pester dependencies, avoid host-mounted account assumptions in Alpine, and reject null COM variants before constructing an owner. Link the Windows CRT statically and verify PE imports. Exercise successful CoreCLR hosting with checksum-verified matching legacy payloads on x86 and x64.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Install make for the C++ musl baseline. Bind negative exit codes explicitly in the legacy managed test script and decode managed diagnostics as UTF-8. Unwrap Framework handles through the documented IObjectHandle interface and report failed COM interface IDs.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Match the original host's VT_DISPATCH ownership contract and cover AddRef/Release balancing with a regression test.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Keep Windows PowerShell 5.1 unsupported as requested. Exercise real Framework activation, Automation argument marshaling, and signed exit codes without depending on its incompatible non-COM-visible entry point.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

BOM-prefixed configuration files can prevent startup, and redirected CoreCLR stdout can mix UTF-16LE diagnostics with byte-oriented command output.

No open issues 🎉

2 previously missed issues

In code that hasn't changed since last review

Medium severity Strip UTF-8 BOM before parsing the first config setting

rust/​windows/​src/​policy.rs:120

A UTF-8 BOM prevents the first configuration setting from being recognized: str::trim leaves U+FEFF intact, so a file beginning with PSHOMEDIR= after the BOM returns None here. HostEnvironment::discover then silently ignores both configured paths and searches beside the executable, preventing startup when the payload is elsewhere. Strip an initial BOM before parsing, and add a BOM-prefixed configuration regression test.

Medium severity Keep CoreCLR stdout byte-oriented to avoid mixed encodings

rust/​windows/​src/​win.rs:295

CoreCLR native stdout now uses UTF-16LE when redirected, unlike the original CssMainEntry.cpp output through wprintf. With -Verbose -Command ..., the success message can therefore append NUL bytes to the managed command's byte-oriented output, producing a mixed-encoding stream. Keep UTF-16 for legacy resource diagnostics, but use a separate byte-oriented output path for CoreCLR native stdout and update the help/debug/verbose output tests accordingly.

3 resolved since last review

Review effort: Balanced

Preserve legacy UTF-16 resource diagnostics while writing native CoreCLR stdout as UTF-8. Add packaged-launcher regressions for external BOM-prefixed configuration and redirected help, debug prompts, and mixed native/managed verbose output.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Nov 11, 2026 •

Copy link
Copy Markdown
Member Author

Fixed both previously missed issues in 701c6a4:

  • Strip an initial UTF-8 BOM before parsing configuration, without stripping BOM characters inside path values. Added portable parser coverage and x86/x64 packaged-launcher tests that load an external payload from BOM-prefixed configuration in either setting order.
  • Keep native CoreCLR stdout byte-oriented UTF-8 for help, debug prompts, and verbose messages. Legacy UTF-16 resource diagnostics remain unchanged. Added redirected help/debug checks and a real managed command test that verifies exact Unicode output followed by the verbose success message, with no embedded NULs.

All eight matrix jobs pass: https://github.com/PowerShell/PowerShell-Native/actions/runs/42953093912

GitHub
Contribute to PowerShell/PowerShell-Native development by creating an account on GitHub.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Public hosting APIs allow safe Rust code to trigger invalid frees and out-of-bounds vtable calls, leaving critical memory-safety issues unresolved.

No open issues 🎉

4 previously missed issues

In code that hasn't changed since last review

High severity Make variant tag and payload fields private

rust/​windows/​src/​hosting.rs:176

The public tag and payload can be overwritten by entirely safe Rust code after AutomationVariant::default(). For example, setting kind to VT_BSTR and value to a non-BSTR pointer makes Drop pass that pointer to VariantClear, causing an invalid free. Keep these fields private so the safe constructors preserve the ownership invariant; any raw-variant construction or mutation should require an explicit unsafe contract.

High severity Restrict invoke to IDispatch interfaces

rust/​windows/​src/​hosting.rs:338

query returns the same publicly invokable type for every interface IID, but invoke unconditionally calls IDispatch slots 5 and 6. A safe caller can query IID_IUnknown on an instance and then call invoke on the returned three-slot interface, causing an out-of-bounds vtable call rather than a COM error. Keep this low-level helper private (its current callers are all internal), or return interface-specific wrappers that only expose invoke for IDispatch.

Medium severity Follow symlinks when enumerating DLL files

rust/​windows/​src/​common.rs:250

DirEntry::file_type() does not follow symbolic links, so this excludes DLL symlinks that the original FindFirstFile enumeration included. A payload with a symlinked System.Private.CoreLib.dll or Microsoft.PowerShell.ConsoleHost.dll can load CoreCLR but then fail initialization or delegate creation because the assembly is missing from the TPA list. Follow links when checking whether the candidate is a file, while retaining the filename exclusions.

Medium severity Use an unsupported version to test native argument rejection

rust/​windows/​tests/​launcher-smoke.ps1:38

The Framework smoke path still expects the registered PowerShell console host to execute exit 37. On a normal Windows installation this selects Windows PowerShell 5.1, which this port explicitly does not support because its entry point is not COM-visible, so the documented smoke command fails even when the launcher is built correctly. Pass @('-Version', 'invalid') instead and update the assertion at lines 58–59 to expect BAD_COMMAND_LINE (-196608) and report native argument-rejection failures.

Review effort: Balanced

Make Automation storage and arbitrary COM queries private, add compile-fail API regressions, follow DLL symlinks, and exercise payload-independent launcher smoke checks.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Nov 11, 2026 •

Copy link
Copy Markdown
Member Author

Addressed all four newly reported findings in 439f1be:

  • Made the owning Automation variant tag/payload and underlying union private, preserving native layout and alignment. Safe callers can no longer forge values that VariantClear would free.
  • Made arbitrary COM interface queries private. Publicly returned objects remain dispatch interfaces, so safe callers cannot query IUnknown and invoke IDispatch-only slots. Added compile-fail regressions for both API boundaries.
  • Follow DLL symbolic links during TPA enumeration while retaining filename exclusions. Added real CoreCLR startup coverage with symlinked System.Private.CoreLib.dll and Microsoft.PowerShell.ConsoleHost.dll.
  • Changed the Framework smoke script to reject -Version invalid with exact BAD_COMMAND_LINE (-196608), without requiring Windows PowerShell 5.1. Both packaged configurations now execute the documented smoke script through Pester.

All eight native CI jobs and all PR checks passed at this commit: https://github.com/PowerShell/PowerShell-Native/actions/runs/42955460413

GitHub
Contribute to PowerShell/PowerShell-Native development by creating an account on GitHub.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Handwritten COM vtables and Windows shell integration are not exercised by headless CI, requiring Windows platform-owner review and desktop validation to establish compatibility.

No open issues 🎉

Review effort: Balanced

Prevent concurrent native test forks from crashing in macOS child-side libSystem initialization before exec. Keep other tests parallel and report child wait status before asserting SIGINT output.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@SteveL-MSFT

Steve Lee (SteveL-MSFT) commented Nov 11, 2026 •

Copy link
Copy Markdown
Member Author

Fixed the intermittent macOS ARM64 test failure in fc065b9.

The empty SIGINT-test output was a child-process crash before exec, not a SIGINT assertion race. Local stress runs also reproduced failures in the other process tests; the crash report identified macOS libSystem/libnotify child-side fork initialization (os_once_t is corrupt).

Added a macOS-only mutex around test fork calls, leaving the rest of the suite parallel and production behavior unchanged. The SIGINT test now reaps the child and checks its termination status before checking output, so a future process crash reports the actual wait status.

The parallel native suite passed 200 consecutive local runs after the fix. All eight native CI jobs, including macOS ARM64, and all PR checks now pass: https://github.com/PowerShell/PowerShell-Native/actions/runs/43122250368

GitHub
Contribute to PowerShell/PowerShell-Native development by creating an account on GitHub.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants