Skip to content

Commit 532044f

Browse files
daxian-dbwdaxian-dbw
authored
Add documentation about how to create libpsl and psrp.windows packages (#5278)
- Add document about creating libpsl and psrp.windows nuget packages - Clean up and update the existing building docs.
1 parent 9c75cea commit 532044f

6 files changed

Lines changed: 149 additions & 133 deletions

File tree

‎.spelling‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -490,8 +490,12 @@ nanoserver-insider-powershell
490490
#region docs/building/internals.md Overrides
491491
- docs/building/internals.md
492492
Catalog
493+
flavor
494+
libpsl
493495
MSBuild
496+
plugin
494497
powershell-unix
498+
psrp.windows
495499
src
496500
#endregion
497501

‎docs/building/internals.md‎

Lines changed: 112 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -44,11 +44,10 @@ dotnet run
4444

4545
Running the program does everything else:
4646

47-
- for each project, given a `resources` folder
48-
- creates a `gen` folder
49-
- for each `*.resx` file
50-
- fills in a strongly typed C# class
51-
- writes it out to the corresponding `*.cs` file
47+
- For each project, given a `resources` folder, create a `gen` folder.
48+
- For each `*.resx` file from the `resources` folder,
49+
fill in a strongly typed C# class,
50+
and write it out to the corresponding `*.cs` file in the `gen` folder.
5251

5352
These files are *not* automatically updated on each build,
5453
as the project lacks the ability to detect changes.
@@ -97,3 +96,111 @@ which generates a source file `CorePsTypeCatalog.cs` for the `Microsoft.PowerShe
9796
The error `The name 'InitializeTypeCatalog' does not exist in the current context`
9897
indicates that the `CorePsTypeCatalog.cs` source file does not exist,
9998
so follow the steps to generate it.
99+
100+
## Native Components
101+
102+
On Windows, PowerShell Core depends on the WinRM plugin `pwrshplugin.dll` to enable remoting over WinRM.
103+
On Linux/macOS, PowerShell Core depends on the binary `libpsl-native.so/libpsl-native.dylib` to provide some necessary supports.
104+
105+
Building those native components requires setting up additional dependencies,
106+
which could be a burden to those who don't seek to make changes to the native components.
107+
At the meantime, the native component code seldom changes,
108+
so it doesn't make sense to always build them with `Start-PSBuild`.
109+
Therefore, we decided to wrap the native components into NuGet packages,
110+
so that we only need to build them once when changes are made,
111+
and then reuse the produced binaries for many builds subsequently.
112+
113+
The NuGet package for `pwrshplugin.dll` is `psrp.windows`,
114+
and the NuGet package for `libpsl-native` is `libpsl`.
115+
116+
### psrp.windows
117+
118+
To build `pwrshplugin.dll`, you need to install Visual Studio 2015 and run `Start-PSBootstrap -BuildWindowsNative` to install the prerequisites.
119+
Then run `Start-BuildNativeWindowsBinaries` to build the binary.
120+
For example, the following builds the release flavor of the binary targeting x64 architecture.
121+
122+
```powershell
123+
Start-BuildNativeWindowsBinaries -Configuration Release -Arch x64
124+
```
125+
126+
After that, the binary `pwrshplugin.dll` and its PDB file will be placed under 'src/powershell-win-core'.
127+
The script file `Install-PowerShellRemoting.ps1` will also be placed in the same folder,
128+
which is supposed to be used to set up remoting configurations for PowerShell Core.
129+
130+
To create a new NuGet package for `pwrshplugin.dll`, first you need to get the `psrp.windows.nuspec` from an existing `psrp.windows` package.
131+
You can find it at `~/.nuget/packages/psrp.windows` on your windows machine if you have recently built PowerShell on it.
132+
Or you can download the existing package from [powershell-core feed](https://powershell.myget.org/feed/powershell-core/package/nuget/psrp.windows).
133+
Once you get `psrp.windows.nuspec`, copy it to an empty folder.
134+
135+
Then you need to build `pwrshplugin.dll` targeting both `win-x64` and `win-x86` on Windows 10.
136+
After building successfully, copy the produced files to the same folder,
137+
and create the same layout of files as in the existing package.
138+
The layout of files should look like this:
139+
140+
```none
141+
+---contentFiles
142+
| \---any
143+
| \---any
144+
| Install-PowerShellRemoting.ps1
145+
|
146+
\---runtimes
147+
+---win-x64
148+
| \---native
149+
| pwrshplugin.dll
150+
| pwrshplugin.pdb
151+
|
152+
\---win-x86
153+
\---native
154+
pwrshplugin.dll
155+
pwrshplugin.pdb
156+
```
157+
158+
Lastly, run `nuget pack .` from within the folder. Note that you may need the latest `nuget.exe`.
159+
160+
### libpsl
161+
162+
For `linux-arm`, you need to run `Start-PSBootstrap -BuildLinuxArm` to install additional prerequisites to build `libpsl-native`.
163+
Note that currently you can build `linux-arm` only on a Ubuntu machine.
164+
165+
For `linux-x64` and macOS, the initial run of `Start-PSBootstrap` would be enough -- no additional prerequisite required.
166+
167+
After making sure the prerequisites are met, run `Start-BuildNativeUnixBinaries` to build the binary:
168+
169+
```powershell
170+
## Build targeting linux-x64 or macOS
171+
Start-BuildNativeUnixBinaries
172+
173+
## Build targeting linux-arm
174+
Start-BuildNativeUnixBinaries -BuildLinuxArm
175+
```
176+
177+
After the build succeeds, the binary `libpsl-native.so` (`libpsl-native.dylib` on macOS) will be placed under `src/powershell-unix`.
178+
179+
To create a new NuGet package for `libpsl-native`, first you need to get the `libpsl.nuspec` from an existing `libpsl` package.
180+
You can find it at `~/.nuget/packages/libpsl` on your Linux or macOS machine if you have recently built PowerShell on it.
181+
Or you can download the existing package from [powershell-core feed](https://powershell.myget.org/feed/powershell-core/package/nuget/libpsl).
182+
Once you get `psrp.windows.nuspec`, copy it to an empty folder on your Windows machine.
183+
184+
Then you need to build three binaries of `libpsl-native` targeting `linux-x64`, `linux-arm` and `osx` respectively.
185+
**Please note that, in order for the `linux-x64` binary `libpsl-native.so` to be portable to all other Linux distributions,
186+
the `linux-x64` binary needs to be built on CentOS 7**
187+
(.NET Core Linux native binaries are also built on CentOS 7 to ensure that they don't depend on newer `glibc`).
188+
189+
After building successfully, copy those three binaries to the same folder,
190+
and create the same layout of files as in the existing package.
191+
The layout of files should look like this:
192+
193+
```none
194+
└── runtimes
195+
├── linux-arm
196+
│   └── native
197+
│   └── libpsl-native.so
198+
├── linux-x64
199+
│   └── native
200+
│   └── libpsl-native.so
201+
└── osx
202+
└── native
203+
└── libpsl-native.dylib
204+
```
205+
206+
Lastly, run `nuget pack .` from within the folder. Note that you may need the latest `nuget.exe`.

‎docs/building/linux.md‎

Lines changed: 13 additions & 93 deletions
Original file line numberDiff line numberDiff line change
@@ -1,39 +1,25 @@
1-
Build PowerShell on Linux
2-
=========================
1+
# Build PowerShell on Linux
32

43
This guide will walk you through building PowerShell on Linux.
54
We'll start by showing how to set up your environment from scratch.
65

7-
Environment
8-
===========
6+
## Environment
97

108
These instructions are written assuming the Ubuntu 14.04 LTS, since that's the distro the team uses.
119
The build module works on a best-effort basis for other distributions.
1210

13-
Git Setup
14-
---------
11+
### Git Setup
1512

1613
Using Git requires it to be set up correctly;
1714
refer to the [Working with the PowerShell Repository](../git/README.md),
1815
[README](../../README.md), and [Contributing Guidelines](../../.github/CONTRIBUTING.md).
1916

2017
**This guide assumes that you have recursively cloned the PowerShell repository and `cd`ed into it.**
2118

22-
Toolchain Setup
23-
---------------
19+
### Toolchain Setup
2420

2521
We use the [.NET Command-Line Interface][dotnet-cli] (`dotnet`) to build the managed components,
2622
and [CMake][] to build the native components.
27-
Install the following packages for the toolchain:
28-
29-
- `dotnet`: Must be installed from the `Start-PSBootstrap` module as described below.
30-
- `cmake`
31-
- `make`
32-
- `g++`
33-
34-
Unfortunately, the `apt-get` feed for `dotnet` has been deprecated,
35-
and the latest version is only distributed in the form of three separate packages,
36-
which require manual dependency resolution.
3723

3824
Installing the toolchain is as easy as running `Start-PSBootstrap` in PowerShell.
3925
Of course, this requires a self-hosted copy of PowerShell on Linux.
@@ -44,12 +30,12 @@ The `./tools/download.sh` script will also install the PowerShell package.
4430
In Bash:
4531

4632
```sh
47-
./tools/download.sh
33+
./tools/install-powershell.sh
4834

49-
powershell
35+
pwsh
5036
```
5137

52-
You should now be in a `powershell` console host that is installed separately from any development copy you're about to build.
38+
You should now be in a PowerShell console host that is installed.
5339
Just import our module, bootstrap the dependencies, and build!
5440

5541
In PowerShell:
@@ -64,94 +50,28 @@ The `Start-PSBootstrap` function does the following:
6450
- Adds the LLVM package feed
6551
- Installs our dependencies combined with the dependencies of the .NET CLI toolchain via `apt-get`
6652
- Uninstalls any prior versions of .NET CLI
67-
- Downloads and installs the latest .NET Core SDK 1.0.1 to `~/.dotnet`
53+
- Downloads and installs the .NET Core SDK 2.0.0 to `~/.dotnet`
6854

6955
If you want to use `dotnet` outside of `Start-PSBuild`, add `~/.dotnet` to your `PATH` environment variable.
7056

7157
[dotnet-cli]: https://github.com/dotnet/cli
7258
[CMake]: https://cmake.org/cmake/help/v2.8.12/cmake.html
7359

74-
.NET CLI
75-
--------
76-
77-
If you have any problems installing `dotnet`, please see their [documentation][cli-docs].
78-
79-
The version of .NET CLI is very important; the version we are currently using is `1.0.1`.
80-
81-
Previous installations of DNX, `dnvm`, or older installations of .NET CLI can cause odd failures when running.
82-
Please check your version and uninstall prior any prior versions.
83-
84-
[cli-docs]: https://www.microsoft.com/net/core
85-
86-
Build using our module
87-
======================
60+
## Build using our module
8861

8962
We maintain a [PowerShell module](../../build.psm1) with the function `Start-PSBuild` to build PowerShell.
9063
Since this is PowerShell code, it requires self-hosting.
91-
If you have followed the toolchain setup section above, you should have `powershell` installed.
92-
93-
> If you cannot or do not want to self-host, `Start-PSBuild` is just a
94-
> convenience; you can execute each step of the build process yourself
95-
> in Bash; see [Build manually](#build-manually) below.
64+
If you have followed the toolchain setup section above, you should have PowerShell Core installed.
9665

9766
```powershell
9867
Import-Module ./build.psm1
9968
Start-PSBuild
10069
```
70+
10171
Congratulations! If everything went right, PowerShell is now built.
10272
The `Start-PSBuild` script will output the location of the executable:
10373

104-
`./src/powershell-unix/bin/Linux/netcoreapp1.1/ubuntu.14.04-x64/powershell`.
74+
`./src/powershell-unix/bin/Linux/netcoreapp2.0/linux-x64/publish/pwsh`.
10575

106-
You should now be running the `powershell` that you just built, if your run the above executable.
76+
You should now be running the PowerShell Core that you just built, if your run the above executable.
10777
You can run our cross-platform Pester tests with `Start-PSPester`, and our xUnit tests with `Start-PSxUnit`.
108-
109-
Build manually
110-
==============
111-
112-
The following goes into detail about what `Start-PSBuild` does.
113-
114-
There are two preliminary steps that apply to all operating systems,
115-
the [ResGen](internals.md#resgen) and [type catalog generation](internals.md#type-catalog),
116-
documented in [internals of build process](internals.md#preliminary-steps).
117-
118-
Build the native library
119-
------------------------
120-
121-
The `libpsl-native.so` library consists of native functions that `CorePsPlatform.cs` P/Invokes.
122-
123-
```sh
124-
pushd src/libpsl-native
125-
cmake -DCMAKE_BUILD_TYPE=Debug .
126-
make -j
127-
make test
128-
popd
129-
```
130-
131-
This library will be emitted in the `src/powershell-unix` project,
132-
where `dotnet` consumes it as "content" and thus automatically deploys it.
133-
134-
Build the managed projects
135-
--------------------------
136-
137-
The `powershell` project is the .NET Core PowerShell host.
138-
It is the top level project, so `dotnet build` transitively builds all its dependencies, and emits a `powershell` executable.
139-
The `--configuration Linux` flag is necessary to ensure that the preprocessor definition `LINUX` is defined (see [issue #673][]).
140-
141-
```sh
142-
dotnet restore
143-
cd src/powershell-unix
144-
dotnet build --configuration Linux
145-
```
146-
147-
The executable will be in `./bin/[configuration]/[framework]/[rid]/publish/[binary name]`,
148-
where our configuration is `Linux`, framework is `netcoreapp2.0`,
149-
runtime identifier is `linux-x64`, and binary name is `powershell`.
150-
The function `Get-PSOutput` will return the path to the executable;
151-
thus you can execute the development copy via `& (Get-PSOutput)`.
152-
153-
For deploying PowerShell, `dotnet publish` will emit a `publish` directory that contains a flat list of every dependency required for
154-
PowerShell.
155-
This can be copied to, for example, `/usr/local/share/powershell` or packaged.
156-
157-
[issue #673]: https://github.com/PowerShell/PowerShell/issues/673

‎docs/building/macos.md‎

Lines changed: 3 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ building on macOS is almost identical.
1010
You will want [Homebrew](http://brew.sh/), the missing package manager for macOS.
1111
Once installed, follow the same instructions to download and
1212
install a self-hosted copy of PowerShell on your macOS machine,
13-
and use`Start-PSBootstrap` to install the dependencies.
13+
and use `Start-PSBootstrap` to install the dependencies.
1414

1515
The `Start-PSBootstrap` function does the following:
1616

@@ -34,9 +34,7 @@ We cannot do this for you in the build module due to #[847][].
3434

3535
## Build using our module
3636

37-
Instead of installing the Ubuntu package of PowerShell,
38-
download the `pkg` from our GitHub releases page using your browser, complete the wizard,
39-
start a `powershell` session, and use `Start-PSBuild` from the module.
37+
Start a PowerShell session by running `pwsh`, and then use `Start-PSBuild` from the module.
4038

4139
After building, PowerShell will be at `./src/powershell-unix/bin/Linux/netcoreapp2.0/osx.10.12-x64/publish/powershell`.
42-
Note that configuration is still `Linux` because it would be silly to make yet another separate configuration when it's used solely to work-around a CLI issue.
40+
Note that configuration is still `Linux`.

0 commit comments

Comments
 (0)