Skip to content

Cut a preview string where a character ends, not where the limit does - #305

Merged
diolektor merged 1 commit into
mainfrom
fix/shared-preview-multibyte-panic
Aug 15, 2026
Merged

Cut a preview string where a character ends, not where the limit does#305
diolektor merged 1 commit into
mainfrom
fix/shared-preview-multibyte-panic

Conversation

@diolektor

Copy link
Copy Markdown
Contributor

What was wrong

GET /__ox_shared/preview?id=N truncates a long string value at SHARED_PREVIEW_STRING_LIMIT bytes (256 out of the box) and cut the string at exactly that byte — a valid place to cut only while every character is one byte wide. When the cut byte fell inside a UTF-8 sequence, the slice panicked with byte index 256 is not a char boundary.

Nothing wraps a plugin internal route in catch_unwind (PluginManager::handle_internal_route, handle_internal_request), and the internal server serves each connection inside a tokio::spawn. So the panic killed the connection task: the client got a closed connection with no status and no body, indistinguishable from the server being gone, and the only trace was a panic in the log. The process itself survived — the build is panic = unwind.

Reachable through Shared\Mutex and Shared\Once, the only two primitives whose debug_snapshot() can yield a string; Counter, Atomic, Channel, Map and Pool return a number and Flag returns a bool, so none of them could reach it. A PHP string that is not valid UTF-8 never reaches the string arm either — the portbuf decoder routes it to SharedValue::Bytes, which previews as a byte count.

Who actually hit it

Worth stating precisely, because the obvious intuition is wrong. Whether the cut lands mid-character is arithmetic between the limit and the character width: for a string of characters w bytes wide, byte L is a boundary exactly when L % w == 0. At the default limit of 256:

Script Bytes/char 256 % w Result
ASCII 1 0 never split
Cyrillic, Greek 2 0 never split
CJK, , Devanagari 3 1 always split
Emoji 4 0 never split

So pure Cyrillic and pure emoji values never panicked at the default limit, three-byte scripts always did, and mixed text was a coin flip. A different limit moves the whole table (255 splits Cyrillic, for instance). This matters for anyone writing a reproducer: one built on Cyrillic or emoji passes even on the unfixed build.

The fix

Walk the limit back to the nearest character boundary before slicing (while cut > 0 && !s.is_char_boundary(cut) { cut -= 1 } — the stable equivalent of floor_char_boundary, which is not available on the project's MSRV of 1.91.1). The branch is only entered when s.len() > str_limit, so the starting index is always inside the string and the walk terminates at 0 at worst.

The limit stays a byte budget, which is what the documentation already described: the prefix simply lands up to 3 bytes under it. ASCII output is byte-for-byte unchanged.

Tests

Three cases, added next to the existing observability tests:

  • preview_truncates_multibyte_string_at_char_boundary — the regression. 100 × (300 bytes, byte 256 mid-sequence), expecting exactly 85 characters / 255 bytes. A three-byte character is used deliberately, per the table above.
  • preview_string_limit_stays_a_byte_budget — ASCII control on 300 bytes expecting a 256-byte prefix. This is not decoration: it is what rules out a chars().take(limit / 3) implementation, which would satisfy the regression test and quietly turn the limit into a character count.
  • preview_zero_limit_emits_empty_prefix — pins the lower bound of the walk-back.

Before the change, the regression test fails with byte index 256 is not a char boundary; it is inside '€' (bytes 255..258); the two controls pass on both sides by design.

Docs

  • The preview endpoint reference now states that truncation rounds down to a character boundary.
  • Both configuration tables name the limit as bytes; the previous wording left open whether it counted bytes or characters.
  • The configuration table also named the wrong endpoint for SHARED_PREVIEW_STRING_LIMIT — it said /entry?id=…, but that limit is read only by the preview handler. /entry?id=… applies the array limit alone, which is presumably how the wrong name got copied down the table.
  • llms-full.txt regenerated for the docs edits.

Verification

  • cargo fmt -- --check
  • cargo clippy --no-default-features -- -D warnings
  • cargo clippy --no-default-features --features plugin-shared --all-targets -- -D warnings
  • cargo test --no-default-features — 27 suites, no failures
  • cargo test --no-default-features --features plugin-shared -- --test-threads=1 — 1346 lib tests passed, 0 failed
  • scripts/gen-llms-txt.sh --check

No Docker or integration-suite run: the change touches no SAPI, worker, routing or PHP-gated code, and render_preview is a private synchronous function with no registry or FFI involvement.

Fix:
  - `/__ox_shared/preview` truncated a long string value at the raw `SHARED_PREVIEW_STRING_LIMIT` byte, which is a valid place to cut only while every character is one byte wide. A value past the limit whose cut byte fell inside a UTF-8 sequence panicked the handler instead of answering it: nothing wraps a plugin internal route in `catch_unwind`, so the connection task died and the client got a closed connection with no status and no body, indistinguishable from the server being gone. The limit is now walked back to the nearest character boundary. It stays a byte budget — the prefix lands at most 3 bytes under it — and ASCII values render exactly as before. Only `Shared\Mutex` and `Shared\Once` snapshot string values; every other primitive previews a number or a bool and could never reach this.
  - Named the right endpoint for `SHARED_PREVIEW_STRING_LIMIT` in the configuration table. It said `/entry?id=…`, but that limit is read only by the preview handler; `/entry?id=…` applies the array limit alone, which is presumably how the wrong name got copied down the table.

Tests:
  - Three cases around the truncation. The regression uses a three-byte character deliberately: the default limit of 256 divides by 2 and by 4, so pure Cyrillic and pure emoji never split there, and a reproducer written with either passes on the unfixed build. Two controls hold the semantics the regression alone would not — an ASCII string proves the limit is still counted in bytes rather than characters, which rules out a `chars().take(limit/3)` implementation that would satisfy the regression test, and a zero limit pins the lower bound of the walk-back.

Docs:
  - Said that preview truncation rounds down to a character boundary, and named the limit as bytes in both configuration tables; the previous wording left open whether it counted bytes or characters.

1346 unit tests pass with `--features plugin-shared`, 3 of them new.
@diolektor
diolektor merged commit 6c82674 into main Aug 15, 2026
7 checks passed
@diolektor
diolektor deleted the fix/shared-preview-multibyte-panic branch August 15, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant