Skip to content

Add documentation about how to create libpsl and psrp.windows packages #5278

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
daxian-dbw merged 4 commits into from
Nov 1, 2017
Merged

Add documentation about how to create libpsl and psrp.windows packages #5278

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
201dff9
Add document about creating libpsl and psrp.windows nuget packages
daxian-dbw Oct 31, 2017
734bc71
Fix spelling
daxian-dbw Oct 31, 2017
43b0e70
Minor update
daxian-dbw Oct 31, 2017
0560c8f
Fix markdown test failure
daxian-dbw Oct 31, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .spelling
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -490,8 +490,12 @@ nanoserver-insider-powershell
#region docs/building/internals.md Overrides
- docs/building/internals.md
Catalog
flavor
libpsl
MSBuild
plugin
powershell-unix
psrp.windows
src
#endregion

Expand Down
117 changes: 112 additions & 5 deletions docs/building/internals.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,11 +44,10 @@ dotnet run

Running the program does everything else:

- for each project, given a `resources` folder
- creates a `gen` folder
- for each `*.resx` file
- fills in a strongly typed C# class
- writes it out to the corresponding `*.cs` file
- For each project, given a `resources` folder, create a `gen` folder.
- For each `*.resx` file from the `resources` folder,
fill in a strongly typed C# class,
and write it out to the corresponding `*.cs` file in the `gen` folder.

These files are *not* automatically updated on each build,
as the project lacks the ability to detect changes.
Expand Down Expand Up @@ -97,3 +96,111 @@ which generates a source file `CorePsTypeCatalog.cs` for the `Microsoft.PowerShe
The error `The name 'InitializeTypeCatalog' does not exist in the current context`
indicates that the `CorePsTypeCatalog.cs` source file does not exist,
so follow the steps to generate it.

## Native Components

On Windows, PowerShell Core depends on the WinRM plugin `pwrshplugin.dll` to enable remoting over WinRM.
On Linux/macOS, PowerShell Core depends on the binary `libpsl-native.so/libpsl-native.dylib` to provide some necessary supports.

Building those native components requires setting up additional dependencies,
which could be a burden to those who don't seek to make changes to the native components.
At the meantime, the native component code seldom changes,
so it doesn't make sense to always build them with `Start-PSBuild`.
Therefore, we decided to wrap the native components into NuGet packages,
so that we only need to build them once when changes are made,
and then reuse the produced binaries for many builds subsequently.

The NuGet package for `pwrshplugin.dll` is `psrp.windows`,
and the NuGet package for `libpsl-native` is `libpsl`.

### psrp.windows

To build `pwrshplugin.dll`, you need to install Visual Studio 2015 and run `Start-PSBootstrap -BuildWindowsNative` to install the prerequisites.
Then run `Start-BuildNativeWindowsBinaries` to build the binary.
For example, the following builds the release flavor of the binary targeting x64 architecture.

```powershell
Start-BuildNativeWindowsBinaries -Configuration Release -Arch x64
```

After that, the binary `pwrshplugin.dll` and its PDB file will be placed under 'src/powershell-win-core'.
The script file `Install-PowerShellRemoting.ps1` will also be placed in the same folder,
which is supposed to be used to set up remoting configurations for PowerShell Core.

To create a new NuGet package for `pwrshplugin.dll`, first you need to get the `psrp.windows.nuspec` from an existing `psrp.windows` package.
You can find it at `~/.nuget/packages/psrp.windows` on your windows machine if you have recently built PowerShell on it.
Or you can download the existing package from [powershell-core feed](https://powershell.myget.org/feed/powershell-core/package/nuget/psrp.windows).
Once you get `psrp.windows.nuspec`, copy it to an empty folder.

Then you need to build `pwrshplugin.dll` targeting both `win-x64` and `win-x86` on Windows 10.
After building successfully, copy the produced files to the same folder,
and create the same layout of files as in the existing package.
The layout of files should look like this:

```none
+---contentFiles
| \---any
| \---any
| Install-PowerShellRemoting.ps1
|
\---runtimes
+---win-x64
| \---native
| pwrshplugin.dll
| pwrshplugin.pdb
|
\---win-x86
\---native
pwrshplugin.dll
pwrshplugin.pdb
```

Lastly, run `nuget pack .` from within the folder. Note that you may need the latest `nuget.exe`.

### libpsl

For `linux-arm`, you need to run `Start-PSBootstrap -BuildLinuxArm` to install additional prerequisites to build `libpsl-native`.
Note that currently you can build `linux-arm` only on a Ubuntu machine.

For `linux-x64` and macOS, the initial run of `Start-PSBootstrap` would be enough -- no additional prerequisite required.

After making sure the prerequisites are met, run `Start-BuildNativeUnixBinaries` to build the binary:

```powershell
## Build targeting linux-x64 or macOS
Start-BuildNativeUnixBinaries

## Build targeting linux-arm
Start-BuildNativeUnixBinaries -BuildLinuxArm
```

After the build succeeds, the binary `libpsl-native.so` (`libpsl-native.dylib` on macOS) will be placed under `src/powershell-unix`.

To create a new NuGet package for `libpsl-native`, first you need to get the `libpsl.nuspec` from an existing `libpsl` package.
You can find it at `~/.nuget/packages/libpsl` on your Linux or macOS machine if you have recently built PowerShell on it.
Or you can download the existing package from [powershell-core feed](https://powershell.myget.org/feed/powershell-core/package/nuget/libpsl).
Once you get `psrp.windows.nuspec`, copy it to an empty folder on your Windows machine.

Then you need to build three binaries of `libpsl-native` targeting `linux-x64`, `linux-arm` and `osx` respectively.
**Please note that, in order for the `linux-x64` binary `libpsl-native.so` to be portable to all other Linux distributions,
the `linux-x64` binary needs to be built on CentOS 7**
(.NET Core Linux native binaries are also built on CentOS 7 to ensure that they don't depend on newer `glibc`).

After building successfully, copy those three binaries to the same folder,
and create the same layout of files as in the existing package.
The layout of files should look like this:

```none
└── runtimes
├── linux-arm
│   └── native
│   └── libpsl-native.so
├── linux-x64
│   └── native
│   └── libpsl-native.so
└── osx
└── native
└── libpsl-native.dylib
```

Lastly, run `nuget pack .` from within the folder. Note that you may need the latest `nuget.exe`.
106 changes: 13 additions & 93 deletions docs/building/linux.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,39 +1,25 @@
Build PowerShell on Linux
=========================
# Build PowerShell on Linux

This guide will walk you through building PowerShell on Linux.
We'll start by showing how to set up your environment from scratch.

Environment
===========
## Environment

These instructions are written assuming the Ubuntu 14.04 LTS, since that's the distro the team uses.
The build module works on a best-effort basis for other distributions.

Git Setup
---------
### Git Setup

Using Git requires it to be set up correctly;
refer to the [Working with the PowerShell Repository](../git/README.md),
[README](../../README.md), and [Contributing Guidelines](../../.github/CONTRIBUTING.md).

**This guide assumes that you have recursively cloned the PowerShell repository and `cd`ed into it.**

Toolchain Setup
---------------
### Toolchain Setup

We use the [.NET Command-Line Interface][dotnet-cli] (`dotnet`) to build the managed components,
and [CMake][] to build the native components.
Install the following packages for the toolchain:

- `dotnet`: Must be installed from the `Start-PSBootstrap` module as described below.
- `cmake`
- `make`
- `g++`

Unfortunately, the `apt-get` feed for `dotnet` has been deprecated,
and the latest version is only distributed in the form of three separate packages,
which require manual dependency resolution.

Installing the toolchain is as easy as running `Start-PSBootstrap` in PowerShell.
Of course, this requires a self-hosted copy of PowerShell on Linux.
Expand All @@ -44,12 +30,12 @@ The `./tools/download.sh` script will also install the PowerShell package.
In Bash:

```sh
./tools/download.sh
./tools/install-powershell.sh

powershell
pwsh
```

You should now be in a `powershell` console host that is installed separately from any development copy you're about to build.
You should now be in a PowerShell console host that is installed.
Just import our module, bootstrap the dependencies, and build!

In PowerShell:
Expand All @@ -64,94 +50,28 @@ The `Start-PSBootstrap` function does the following:
- Adds the LLVM package feed
- Installs our dependencies combined with the dependencies of the .NET CLI toolchain via `apt-get`
- Uninstalls any prior versions of .NET CLI
- Downloads and installs the latest .NET Core SDK 1.0.1 to `~/.dotnet`
- Downloads and installs the .NET Core SDK 2.0.0 to `~/.dotnet`

If you want to use `dotnet` outside of `Start-PSBuild`, add `~/.dotnet` to your `PATH` environment variable.

[dotnet-cli]: https://github.com/dotnet/cli
[CMake]: https://cmake.org/cmake/help/v2.8.12/cmake.html

.NET CLI
--------

If you have any problems installing `dotnet`, please see their [documentation][cli-docs].

The version of .NET CLI is very important; the version we are currently using is `1.0.1`.

Previous installations of DNX, `dnvm`, or older installations of .NET CLI can cause odd failures when running.
Please check your version and uninstall prior any prior versions.

[cli-docs]: https://www.microsoft.com/net/core

Build using our module
======================
## Build using our module

We maintain a [PowerShell module](../../build.psm1) with the function `Start-PSBuild` to build PowerShell.
Since this is PowerShell code, it requires self-hosting.
If you have followed the toolchain setup section above, you should have `powershell` installed.

> If you cannot or do not want to self-host, `Start-PSBuild` is just a
> convenience; you can execute each step of the build process yourself
> in Bash; see [Build manually](#build-manually) below.
If you have followed the toolchain setup section above, you should have PowerShell Core installed.

```powershell
Import-Module ./build.psm1
Start-PSBuild
```

Congratulations! If everything went right, PowerShell is now built.
The `Start-PSBuild` script will output the location of the executable:

`./src/powershell-unix/bin/Linux/netcoreapp1.1/ubuntu.14.04-x64/powershell`.
`./src/powershell-unix/bin/Linux/netcoreapp2.0/linux-x64/publish/pwsh`.

You should now be running the `powershell` that you just built, if your run the above executable.
You should now be running the PowerShell Core that you just built, if your run the above executable.
You can run our cross-platform Pester tests with `Start-PSPester`, and our xUnit tests with `Start-PSxUnit`.

Build manually
==============

The following goes into detail about what `Start-PSBuild` does.

There are two preliminary steps that apply to all operating systems,
the [ResGen](internals.md#resgen) and [type catalog generation](internals.md#type-catalog),
documented in [internals of build process](internals.md#preliminary-steps).

Build the native library
------------------------

The `libpsl-native.so` library consists of native functions that `CorePsPlatform.cs` P/Invokes.

```sh
pushd src/libpsl-native
cmake -DCMAKE_BUILD_TYPE=Debug .
make -j
make test
popd
```

This library will be emitted in the `src/powershell-unix` project,
where `dotnet` consumes it as "content" and thus automatically deploys it.

Build the managed projects
--------------------------

The `powershell` project is the .NET Core PowerShell host.
It is the top level project, so `dotnet build` transitively builds all its dependencies, and emits a `powershell` executable.
The `--configuration Linux` flag is necessary to ensure that the preprocessor definition `LINUX` is defined (see [issue #673][]).

```sh
dotnet restore
cd src/powershell-unix
dotnet build --configuration Linux
```

The executable will be in `./bin/[configuration]/[framework]/[rid]/publish/[binary name]`,
where our configuration is `Linux`, framework is `netcoreapp2.0`,
runtime identifier is `linux-x64`, and binary name is `powershell`.
The function `Get-PSOutput` will return the path to the executable;
thus you can execute the development copy via `& (Get-PSOutput)`.

For deploying PowerShell, `dotnet publish` will emit a `publish` directory that contains a flat list of every dependency required for
PowerShell.
This can be copied to, for example, `/usr/local/share/powershell` or packaged.

[issue #673]: https://github.com/PowerShell/PowerShell/issues/673
8 changes: 3 additions & 5 deletions docs/building/macos.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ building on macOS is almost identical.
You will want [Homebrew](http://brew.sh/), the missing package manager for macOS.
Once installed, follow the same instructions to download and
install a self-hosted copy of PowerShell on your macOS machine,
and use`Start-PSBootstrap` to install the dependencies.
and use `Start-PSBootstrap` to install the dependencies.

The `Start-PSBootstrap` function does the following:

Expand All @@ -34,9 +34,7 @@ We cannot do this for you in the build module due to #[847][].

## Build using our module

Instead of installing the Ubuntu package of PowerShell,
download the `pkg` from our GitHub releases page using your browser, complete the wizard,
start a `powershell` session, and use `Start-PSBuild` from the module.
Start a PowerShell session by running `pwsh`, and then use `Start-PSBuild` from the module.

After building, PowerShell will be at `./src/powershell-unix/bin/Linux/netcoreapp2.0/osx.10.12-x64/publish/powershell`.
Note that configuration is still `Linux` because it would be silly to make yet another separate configuration when it's used solely to work-around a CLI issue.
Note that configuration is still `Linux`.
Loading