[PHP] Recompile PHP.wasm Node JSPI with MAIN_MODULE set to 2 - #3244

I think what caused unreachable errors in Asyncify was directly related to -Drecv=wasm_recv.

Xdebug globals list includes _getpid but -Wl, --wrap=getpid will create a __wrap_getpid instead.
Redis globals list includes _usleep which doesn't exist either, but there's a wrap for usleep too.

The following issue occurred in JSPI [ due to -fwasm-exceptions ] :

stderr | src/test/php-part-1.spec.ts > PHP 8.5 > proc_open() > cat: stdin=pipe, stdout=file, stderr=file, file_get_contents
Failed loading /internal/shared/extensions/xdebug.so:  
could not load dynamic lib: /internal/shared/extensions/xdebug.so
LinkError: WebAssembly.Instance(): Import #412 "env" "__c_longjmp": tag import requires a WebAssembly.Tag

By printing the symbols from Xdebug I found this line :

 (import "env" "__c_longjmp" (tag (param i32)))

I added this simple workaround to see what would happen :

/root/replace.sh 's/wasm_shutdown:\s_wasm_shutdown/wasm_shutdown: _wasm_shutdown,\n  \/** \@export *\/ __c_longjmp: new WebAssembly.Tag({ parameters: ["i32"] })/g' /root/output/php.js; \

Which added a new WebAssembly.Tag inside the wasmImports object, and it ran successfully.

I will find a better way to add programmatically these necessary tags inside the wasmImports object later. Maybe some other extensions will need to instantiate tags in env too.

🎉

🎉 🎉 🎉 🎉 🎉 🎉 🎉

It looks like this pull request is ready for review.

Here’s a summary of how I implemented this pull request :

Adding MAIN_MODULE=2 will eliminate dead code normally. In consequence, side modules that need something from main will crash.

And that is what happened first, when running the php-dynamic-loading.spec.ts file :

Failed loading /internal/shared/extensions/xdebug.so:  
could not load dynamic lib: /internal/shared/extensions/xdebug.so
LinkError: WebAssembly.Instance(): Import #202 "env" "__stack_pointer": imported mutable global must be a WebAssembly.Global object

Actually, Emscripten's documentation says this about MAIN_MODULE :

A main module is a file compiled in a way that allows us to link it to a side module at runtime.

1: Normal main module.

2: DCE’d main module. We eliminate dead code normally. If a side module needs something from main, it is up to you to make sure it is kept alive.

So I needed to keep the __stack_pointer symbol from Xdebug alive.

While running a script to list all the functions of xdebug :

import fs from "fs";

const wasm = fs.readFileSync("packages/php-wasm/node-builds/8-4/jspi/extensions/xdebug/xdebug.so");

const module = new WebAssembly.Module(wasm);

// const imports = WebAssembly.Module.imports(module);

const imports = WebAssembly.Module.imports(module).filter(element => element.kind === "global");


fs.writeFileSync('imports-xdebug.json', JSON.stringify(imports, null, 2));

I found this :

{
  "module": "env",
  "name": "__stack_pointer",
  "kind": "global"
},

Which could mean, every global imports should be kept alive by PHP.wasm... But how?

My first attempt was to add the xdebug.so file directly inside php Dockerfile and build the Wasm file with it.

The compilation was successful, but this added the entire function lists to the PHP.js file. Which is not the optimization we needed with MAIN_MODULE=2. Our goal is to both optimize .wasm and .js.

My second attempt was to add __stack_pointer to the list of EXPORTED_FUNCTIONS which will keep the function alive during Dead Code Elimination.

It also compiled but running php-dynamic-loading.spec.ts again returned :

Failed loading /internal/shared/extensions/xdebug.so:
could not load dynamic lib: /internal/shared/extensions/xdebug.so
Error: bad export type for 'core_globals': undefined (undefined)

Which in the functions list matches this :

{
  "module": "GOT.mem",
  "name": "core_globals",
  "kind": "global"
},

I had to create a list of required EXPORTED_FUNCTIONS for Xdebug. But only with "env" and "GOT.mem" globals.

With a lot of back and forth, it turned out two lists were necessary :

1. Globals
2. Exports

Globals are the ones that are in "env" with "func" and "globals" and are NOT "invokes"
Exports are the ones that are in "GOT.mem" or "GOT.func" with "globals"

I added two new steps in my xdebug Dockerfile that will print these symbols inside two dedicated files inside the dedicated xdebug extension directory.

# List globals symbols [ env func and globals without invoke_ and libc++(_Z) prefixed ones ]
RUN	if [ "$JSPI" = "no" ]; then \
		/root/emsdk/upstream/bin/wasm-dis /root/xdebug/modules/xdebug.so \
		| sed -n 's/^[[:space:]]*(import "env" "\([^"]*\)"[[:space:]]*(\(func\|global\).*/\1/p' \
		| sed '/^invoke_/d' \
		| sed '/^_Z/d' \
		| sed 's/^/_/' \
		| sort \
		> /root/xdebug/modules/xdebug.globals.list; \
	fi;

# List exports symbols [ GOT.mem and GOT.func globals ]
RUN	if [ "$JSPI" = "no" ]; then \
		/root/emsdk/upstream/bin/wasm-dis /root/xdebug/modules/xdebug.so \
		| sed -n 's/^[[:space:]]*(import "\(GOT\.mem\|GOT\.func\)" "\([^"]*\)"[[:space:]]*(global.*/\2/p' \
		| sort \
		> /root/xdebug/modules/xdebug.exports.list; \
	fi;

Generating this :

packages/php-wasm/node-builds/8-5/jspi/extensions/xdebug/xdebug.globals.list : 211 items ( 4 kb )
packages/php-wasm/node-builds/8-5/jspi/extensions/xdebug/xdebug.exports.list : 209 items ( 5 kb )

The main problem happened here : what about "wasm_recv".

wasm_recv is a substitution of recv that is only needed for Xdebug, but has been implemented in phpwasm-emscripten-library.js.

Because of dead code elimination, if a method is not used in the main module, it is not listed in the wasmImports object. Even if the symbol is needed by side modules. Every missing element in wasmImports returns a resolved function not found crash.

After lots of attempts, the only way to inject wasm_recv to wasmImports was to inject it manually with /root/replace.sh in the PHP Dockerfile. I am currently not happy with that, but it works.

/root/replace.sh 's/wasm_setsockopt:\s*_wasm_setsockopt,/wasm_setsockopt: _wasm_setsockopt,\n  \/** \@export *\/ wasm_recv: _wasm_recv,/g' /root/output/php.js; \

Then :

Something quite interesting happened :

     ✓ XDebug (5) 480ms
       ✓ does not load dynamically by default
       ✓ supports dynamic loading
       ✓ has its own ini file and entries
       ✓ mounts current working directory
       ✓ communicates with a DBGP client

VS in trunk

     ✓ XDebug (5) 990ms
       ✓ does not load dynamically by default 398ms
       ✓ supports dynamic loading
       ✓ has its own ini file and entries
       ✓ mounts current working directory
       ✓ communicates with a DBGP client

Nice.

I then focused on Intl. And it was another level here.

packages/php-wasm/node-builds/8-5/jspi/extensions/intl/intl.globals.list : 219 items ( 4 kb )
packages/php-wasm/node-builds/8-5/jspi/extensions/intl/intl.exports.list : 5170 items ( 243 kb )

Pretty much the same except some symbols I had to add manually,

+ __ZNSt3__211__call_onceERVmPvPFvS2_E
+ __ZNSt3__218condition_variable10notify_allEv
+ __ZNSt3__25mutex4lockEv
+ __ZNSt3__25mutex6unlockEv

This seems to be related to Intl and ICU using libc++ symbols and some of these symbols need to be executed by PHP.
I tried to add the complete list of symbols but some made the PHP.wasm build crash while others made the JS glue crash with resolved is not a function.

These four elements made the tests pass.

Xdebug and Redis needed to be a little bit improved by forcing them to use functions from the main module instead of new ones. So :

setsockopt became wasm_setsockopt
recv became wasm_recv
getpid became __wrap_getpid
usleep became __wrap_usleep

On the other hand, as Memcached used recv and setsockopt inside libmemcached-install.a, I couldn't simply replace
recv with wasm_recv.

The only way I found was to create two wrappers in a shims file memcached/shims/sockets.c:

#include <sys/types.h>
#include <sys/socket.h>

extern ssize_t wasm_recv(int, void*, size_t, int);
extern int wasm_setsockopt(int, int, int, const void*, socklen_t);

ssize_t recv(int s, void* b, size_t l, int f) {
	return wasm_recv(s, b, l, f);
}
int setsockopt(int s, int lvl, int opt, const void* v, socklen_t n) {
	return wasm_setsockopt(s, lvl, opt, v, n);
}

And inject it inside PHP_NEW_EXTENSION() in the /php-memcached/config.m4 :

/root/replace.sh 's|PHP_NEW_EXTENSION\(memcached,|PHP_NEW_EXTENSION\(memcached, wasm_socket_shims.c|' /root/php-memcached/config.m4; \

Now that I got these symbol files from every side modules, I needed to find a way to inject them in PHP.wasm for compilation.

I focused on the PHP Dockerfile and created two new temporary files :

touch /root/.JS_ABI_EXPORTS && \
touch /root/.WASM_ABI_EXPORTS

JS_ABI_EXPORTS is a list of globals symbols that will be part of the EXPORTED_FUNCTIONS list.
WASM_ABI_EXPORTS is a list of exports that will end up being part of -Wl,@/root/.wasm-exports \ with the use of these two loops :

RUN set -eux; \
	sort -u /root/.JS_ABI_EXPORTS | \
	grep -v '^[[:space:]]*$' | \
	sed 's/^/  "/; s/$/",/' | \
	sed '$ s/,$//' | \
	{ echo '['; cat; echo ']'; } \
	> /root/.exported-functions

RUN set -eux; \
	sort -u /root/.WASM_ABI_EXPORTS | \
	grep -v '^[[:space:]]*$' | \
	sed 's/^/-Wl,--export=/' \
	> /root/.wasm-exports

After that, the different PHP versions compiled successfully.

Something important to mention : every symbol list files are necessary, as they are different from one version to another. It is still a bit messy I admit.

After tha, all the PHP versions compiled and the tests passed successfully 🎉

N.B.: I also made some minor changes :

• Cleaned the use of recv and setsockopt in the main module.
• Removed ICU_CFLAGS artifact in main module.
• Isolated the exported_functions list inside a dedicated file. Which I would like to also do for asyncify lists. Later.
• I had to replace _free with _wasm_free in phpwasm-emscripten-library.
• I forgot to add the libicuio library inside the Intl side module.
• I reordered some if statements to keep consistency between the different side module Dockerfiles.
• I removed the unnecessary PHP version number directory inside the extensions directory since it was already inside the PHP version number directory.

@brandonpayton After the multi-worker rebase. Some things broke :

• addOnInit was not found
• _builtin_fd_close and _builtin_fcntl64 were also not found.

So I first added addOnInit inside PHPWASM_deps next to allocateUTF8OnStack :

-  $PHPWASM__deps: ['$allocateUTF8OnStack'], 
+ $PHPWASM__deps: ['$allocateUTF8OnStack', '$addOnInit'],

And injected the other ones in wasmImports :

- wasmImports,
+ wasmImports: Object.assign({}, wasmImports,
+    typeof _builtin_fd_close === 'function' ? { builtin_fd_close: _builtin_fd_close } : {},
+    typeof _builtin_fcntl64 === 'function' ? { builtin_fcntl64: _builtin_fcntl64 } : {}
+ ),

Without that, the dead code elimination would eliminate these functions.

I also :

• moved the list commands from the side modules to the main module.
• deleted the *.globals.list and *.exports.list files I created before.
• created a dedicated exported-functions.list file for the 4 symbols needed in intl

So, the main logic of this pull request resides in these lines :

# List global and exports symbols
COPY ./compile/php/exported-functions.list /root/exported-functions.list
RUN cat /root/exported-functions.list >> /root/.JS_ABI_EXPORTS

COPY ./${EMSCRIPTEN_ENVIRONMENT}-builds/ /root/${EMSCRIPTEN_ENVIRONMENT}-builds
COPY ./compile/shared/ /root/shared/
RUN if [ "$WITH_JSPI" = "yes" ]; then \
		set -eux; \
		PHP_VERSION_DIR="$(echo "$PHP_VERSION" | cut -d. -f1,2 | tr . -)"; \
		EXTENSIONS_DIR_PATH="/root/${EMSCRIPTEN_ENVIRONMENT}-builds/${PHP_VERSION_DIR}/jspi/extensions"; \
		for path in "$EXTENSIONS_DIR_PATH"/*; do \
			extension="${path##*/}"; \
			# List globals symbols [ env func and globals without invoke_ and libc++(_Z) prefixed ones ]
			/root/emsdk/upstream/bin/wasm-dis "$EXTENSIONS_DIR_PATH/${extension}/${extension}.so" \
			| sed -n 's/^[[:space:]]*(import "env" "\([^"]*\)"[[:space:]]*(\(func\|global\).*/\1/p' \
			| sed '/^invoke_/d' \
			| sed '/^_Z/d' \
			| sed 's/^/_/' \
			| sort \
			>> /root/.JS_ABI_EXPORTS; \
			# List exports symbols [ GOT.mem and GOT.func globals ]
			/root/emsdk/upstream/bin/wasm-dis "$EXTENSIONS_DIR_PATH/${extension}/${extension}.so" \
			| sed -n 's/^[[:space:]]*(import "\(GOT\.mem\|GOT\.func\)" "\([^"]*\)"[[:space:]]*(global.*/\2/p' \
			| sort \
			>> /root/.WASM_ABI_EXPORTS; \
			# Inject extension-specific exported functions
			if [ -f /root/shared/${extension}/exported-functions.list ]; then \
				cat /root/shared/${extension}/exported-functions.list >> /root/.JS_ABI_EXPORTS; \
			fi; \
		done \
	fi;

RUN set -eux; \
	sort -u /root/.JS_ABI_EXPORTS | \
	grep -v '^[[:space:]]*$' | \
	sed 's/^/  "/; s/$/",/' | \
	sed '$ s/,$//' | \
	{ echo '['; cat; echo ']'; } \
	> /root/.exported-functions

RUN set -eux; \
	sort -u /root/.WASM_ABI_EXPORTS | \
	grep -v '^[[:space:]]*$' | \
	sed 's/^/-Wl,--export=/' \
	> /root/.wasm-exports

And injected the other ones in wasmImports :

- wasmImports,
+ wasmImports: Object.assign({}, wasmImports,
+    typeof _builtin_fd_close === 'function' ? { builtin_fd_close: _builtin_fd_close } : {},
+    typeof _builtin_fcntl64 === 'function' ? { builtin_fcntl64: _builtin_fcntl64 } : {}
+ ),

Without that, the dead code elimination would eliminate these functions.

It's strange those were eliminated as dead code since we declare them as deps like:

	__syscall_fcntl64__deps: [
		...LibraryManager.library.__syscall_fcntl64__deps,
		'builtin_fcntl64',
	],

I wonder if it is a bug or expected behavior for some reason.