SINGLE_FILE_BINARY_ENCODE: report a non-UTF-8 read clearly under ASSERTIONS by piandtheradians-glitch · Pull Request #28066 · emscripten-core/emscripten

piandtheradians-glitch · 2026-10-24T03:23:28Z

With -sSINGLE_FILE and the default SINGLE_FILE_BINARY_ENCODE=1, a .js that is read in an encoding other than UTF-8 (for example from a page with no <meta charset> loaded from file://) mangles the embedded Wasm binary, and fails with an error about the Wasm rather than the encoding, such as CompileError: WebAssembly.instantiate(): unknown type form: 85.

As discussed in #28048, under ASSERTIONS the embedded binary's length is now checked once before decoding, by a separate checkBinaryLength() in runtime_common.js. Read as UTF-8, each byte of the binary is one character of the string, so a different length means the file was read in another encoding. It then aborts with:

The embedded Wasm binary is 16265 characters long, but should be 14895: this file was not read as UTF-8. See the documentation on SINGLE_FILE_BINARY_ENCODE for ways to address this issue

Builds without ASSERTIONS are unchanged (byte-identical output checked for -O0 -sASSERTIONS=0, -O2 and -sMINIMAL_RUNTIME -O2).

Tests:

other.test_single_file_not_utf8: re-encodes the output as Latin-1 (what a Windows-1252 read amounts to) and expects the message instead of a CompileError. On main it fails with the CompileError.
browser.test_single_file_not_utf8: loads the build from a windows-1252 page. On main it fails with the CompileError; with this change it reports the message (checked in headless Chromium).
other.test_binary_encode and browser.test_binary_encode (and their _strict variants), which load src/binaryDecode.js without the preprocessor: unchanged, and passing.

Also run locally on the current head, with the emsdk version in test/emsdk_version.txt: all 43 other.test_single_file* tests (including the two d8 shell tests, with V8 15.8 from jsvu), all 6 browser.test_single_file* tests, and the codesize.test_codesize_hello*, codesize.test_minimal_runtime_code_size*, codesize.test_unoptimized_code_size and codesize.test_no_closure_code_size tests (expectations unchanged). Checked by hand with -O2, --closure=1, -sMINIMAL_RUNTIME and -sMODULARIZE -sEXPORT_ES6.

Fixes #28048

Disclosure: researched and written by Claude (Claude Code) at my direction, and reviewed by me. The tests were run as described above.

…RTIONS With `-sSINGLE_FILE` and the default `SINGLE_FILE_BINARY_ENCODE=1`, a `.js` that is read in an encoding other than UTF-8 (for example from a page with no `<meta charset>` loaded from `file://`) mangles the embedded Wasm binary, and fails with an error about the Wasm rather than the encoding, such as `CompileError: WebAssembly.instantiate(): unknown type form: 85`. As discussed in emscripten-core#28048, under `ASSERTIONS` the expected length of the embedded binary is now passed to `binaryDecode()`, which checks it once before decoding. Read as UTF-8, each byte of the binary is one character of the string, so a different length means the file was read in another encoding. It then aborts with: > The embedded Wasm binary is 16265 characters long, but should be 14895: this file was not read as UTF-8 (SINGLE_FILE_BINARY_ENCODE). Serve it with charset=utf-8, add <meta charset="utf-8"> to the page, or build with -sSINGLE_FILE_BINARY_ENCODE=0 Builds without `ASSERTIONS` are unchanged (byte-identical output checked for `-O0 -sASSERTIONS=0`, `-O2` and `-sMINIMAL_RUNTIME -O2`). Tests: - `other.test_single_file_not_utf8`: re-encodes the output as Latin-1 (what a Windows-1252 read amounts to) and expects the message instead of a `CompileError`. On `main` it fails with the `CompileError`. - `browser.test_single_file_not_utf8`: loads the build from a `windows-1252` page. On `main` it fails with the `CompileError`; with this change it reports the message (checked in headless Chromium). - `other.test_single_file_instantiate_failure`: updated for the new `binaryDecode` signature under `ASSERTIONS`. Also run locally, with the emsdk version in `test/emsdk_version.txt`: `other.test_single_file*` (all pass except the two `d8` shell tests, as no `d8` here), the browser `test_single_file*` tests, and the code size tests touching `SINGLE_FILE` (`test_codesize_hello_single_file`, `test_minimal_runtime_code_size*`, `test_unoptimized_code_size`, `test_binary_encode_is_smaller_than_base64_encode`). Checked by hand with `-O2`, `--closure=1`, `-sMINIMAL_RUNTIME` and `-sMODULARIZE -sEXPORT_ES6`. Fixes emscripten-core#28048

gemini-code-assist

Code Review

This pull request introduces validation for the UTF-8 encoding of embedded WebAssembly binaries in SINGLE_FILE builds when ASSERTIONS are enabled. It passes the expected binary size to binaryDecode and asserts that the decoded string length matches, adding tests to verify this behavior. Feedback suggests using strict equality (===) instead of loose equality (==) in the assertion to follow JavaScript best practices.

sbc100 · 2026-10-25T17:26:44Z

+  // differ, the file was not read as UTF-8 (for example a page without
+  // <meta charset="utf-8"> loading it from file://), and the bytes are wrong.
+  // Say so here, rather than let it fail as a confusing wasm CompileError.
+  assert(bin.length === expectedLength, `The embedded Wasm binary is ${bin.length} characters long, but should be ${expectedLength}: this file was not read as UTF-8 (SINGLE_FILE_BINARY_ENCODE). Serve it with charset=utf-8, add <meta charset="utf-8"> to the page, or build with -sSINGLE_FILE_BINARY_ENCODE=0`);


Should we just say something like: "See documentation on SINGLE_FILE_BINARY_ENCODE for ways to address this issue"

Done in 691cf6e: the message now ends "this file was not read as UTF-8. See the documentation on SINGLE_FILE_BINARY_ENCODE for ways to address this issue". (The check itself moved out of binaryDecode.js into checkBinaryLength() in runtime_common.js in ced8159; more on that under your last comment.)

sbc100 · 2026-10-25T17:27:10Z

+  // Each byte of the binary is one character of the string, so if the lengths
+  // differ, the file was not read as UTF-8 (for example a page without
+  // <meta charset="utf-8"> loading it from file://), and the bytes are wrong.
+  // Say so here, rather than let it fail as a confusing wasm CompileError.


This comment seems unnecessary.

Removed in 691cf6e.

sbc100 · 2026-10-25T17:28:22Z

 function findWasmBinary() {
 #if SINGLE_FILE && SINGLE_FILE_BINARY_ENCODE && !WASM2JS
+#if ASSERTIONS
+  return binaryDecode("<<< WASM_BINARY_DATA >>>", "<<< WASM_BINARY_SIZE >>>");


Presumably there should not be quotes around the second argument here?

If we were so simplify this patch so that this second argument is always passed, would that argument be removed by closure compiler / terser? i.e. could we get away with just the if ASSERTIONS in the function body, and not here?

Yes, it does get removed. I measured it with hello world and -sSINGLE_FILE, comparing the current branch (#if ASSERTIONS at the call sites) with the size always passed to a one-parameter binaryDecode():

Build #if at call sites Size always passed

-O0 57855 bytes 57836 bytes (, 4943 kept)

-O2 7753 7753, argument removed

-Oz --closure=1 3768 3768, argument removed

-sMINIMAL_RUNTIME -Oz --closure=1 2005 2005, argument removed

One detail: because the optimizer removes the placeholder before link.py substitutes it, do_replace for WASM_BINARY_SIZE would have to tolerate it being absent.

The catch is the #if ASSERTIONS in the function body. test_binary_encode and test_binary_encode_strict (in both other and browser) load src/binaryDecode.js raw, without the preprocessor, so the #if is a syntax error there. That was the CI failure on b2ca763. So for now the branch keeps binaryDecode.js as it is on main and does the check in checkBinaryLength() in runtime_common.js.

If you prefer your simpler shape, I'd suggest:

// preamble.js (and the same in postamble_minimal.js) -#if ASSERTIONS - return binaryDecode(checkBinaryLength("<<< WASM_BINARY_DATA >>>", "<<< WASM_BINARY_SIZE >>>")); -#else - return binaryDecode("<<< WASM_BINARY_DATA >>>"); -#endif + return binaryDecode("<<< WASM_BINARY_DATA >>>", "<<< WASM_BINARY_SIZE >>>");

// binaryDecode.js (checkBinaryLength() removed from runtime_common.js) -function binaryDecode(bin) { +function binaryDecode(bin, expectedLength) { +#if ASSERTIONS + assert(bin.length === expectedLength, `... See the documentation on SINGLE_FILE_BINARY_ENCODE ...`); +#endif

# link.py: always substitute, and tolerate the placeholder already being optimized away - if settings.ASSERTIONS: - js = do_replace(js, '"<<< WASM_BINARY_SIZE >>>"', str(os.path.getsize(wasm_target))) + js = js.replace('"<<< WASM_BINARY_SIZE >>>"', str(os.path.getsize(wasm_target)))

The two binary_encode tests (other and browser) would also run binaryDecode.js through tools/preprocessor.mjs before loading it.

Generally speaking I prefer the simpler shape, but I'd like to go with your preference here. I'm happy to make that change in this PR, or to leave the branch as it is. Either way, I can also follow up separately with anything else you'd like around this.

sbc100 · 2026-10-25T17:29:17Z

    if settings.SINGLE_FILE_BINARY_ENCODE:
      js = do_replace(js, '"<<< WASM_BINARY_DATA >>>"', binary_encode(wasm_target))
+      if settings.ASSERTIONS:
+        js = do_replace(js, '"<<< WASM_BINARY_SIZE >>>"', str(os.path.getsize(wasm_target)))


Do we need the inner quotes here?

Whichever shape we go with (the current one, or the simpler one discussed in the other thread), my agent reports that they're required, in both places (this also answers the question on preamble.js). link.py fills the placeholders into the final JS after acorn, terser and closure have run, so until then the placeholder has to be valid JS, like the "<<< WASM_BINARY_DATA >>>" next to it. Without the quotes, -O2 fails in acorn-optimizer.mjs with SyntaxError: Unexpected token. Only -O0 builds, because nothing parses it first. Including the quotes in the do_replace pattern means they go away along with the placeholder, which leaves a bare number.

Awaiting your response.

`test_binary_encode` and `test_binary_encode_strict` (other and browser) load `src/binaryDecode.js` as plain JS, without the preprocessor, so the `#if ASSERTIONS` added to it was a syntax error there ("Private field '#if' must be declared in an enclosing class"). `binaryDecode()` is back as it was on `main`; the length check is now a separate `checkBinaryLength()` in `runtime_common.js` (also `@noinline`, for the same reason as `binaryDecode()`), called on the string before decoding under `ASSERTIONS`. Run locally: `other.test_binary_encode`, `other.test_binary_encode_strict` and `other.test_single_file*` (45 tests) pass; `test_binary_encode` fails on the previous commit and passes with this one.

Point at the SINGLE_FILE_BINARY_ENCODE documentation instead of listing the fixes in the message, and remove the explanatory comment, as suggested in review.

gemini-code-assist Bot reviewed Oct 24, 2026

View reviewed changes

Comment thread src/binaryDecode.js Outdated

Use strict equality in the binaryDecode length assert

b2ca763

sbc100 reviewed Oct 25, 2026

View reviewed changes

piandtheradians-glitch added 2 commits October 25, 2026 11:31

Shorten the binary length assert message and drop its comment

691cf6e

Point at the SINGLE_FILE_BINARY_ENCODE documentation instead of listing the fixes in the message, and remove the explanatory comment, as suggested in review.

Navigation Menu

Repository navigation

SINGLE_FILE_BINARY_ENCODE: report a non-UTF-8 read clearly under ASSERTIONS - #28066

SINGLE_FILE_BINARY_ENCODE: report a non-UTF-8 read clearly under ASSERTIONS#28066
piandtheradians-glitch wants to merge 4 commits into
emscripten-core:mainfrom
piandtheradians-glitch:single-file-utf8-check

piandtheradians-glitch commented Oct 24, 2026 •

edited

Loading

Uh oh!

gemini-code-assist Bot left a comment

Uh oh!

Uh oh!

sbc100 Oct 25, 2026

Uh oh!

piandtheradians-glitch Oct 25, 2026

Uh oh!

sbc100 Oct 25, 2026

Uh oh!

piandtheradians-glitch Oct 25, 2026

Uh oh!

sbc100 Oct 25, 2026

Uh oh!

sbc100 Oct 25, 2026

Uh oh!

piandtheradians-glitch Oct 25, 2026

Uh oh!

sbc100 Oct 25, 2026

Uh oh!

piandtheradians-glitch Oct 25, 2026 •

edited

Loading

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Build	`#if` at call sites	Size always passed
`-O0`	57855 bytes	57836 bytes (`, 4943` kept)
`-O2`	7753	7753, argument removed
`-Oz --closure=1`	3768	3768, argument removed
`-sMINIMAL_RUNTIME -Oz --closure=1`	2005	2005, argument removed

Conversation

piandtheradians-glitch commented Oct 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

piandtheradians-glitch Oct 25, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

piandtheradians-glitch commented Oct 24, 2026 •

edited

Loading

piandtheradians-glitch Oct 25, 2026 •

edited

Loading