Cut a preview string where a character ends, not where the limit does - #305
Merged
Conversation
Fix: - `/__ox_shared/preview` truncated a long string value at the raw `SHARED_PREVIEW_STRING_LIMIT` byte, which is a valid place to cut only while every character is one byte wide. A value past the limit whose cut byte fell inside a UTF-8 sequence panicked the handler instead of answering it: nothing wraps a plugin internal route in `catch_unwind`, so the connection task died and the client got a closed connection with no status and no body, indistinguishable from the server being gone. The limit is now walked back to the nearest character boundary. It stays a byte budget — the prefix lands at most 3 bytes under it — and ASCII values render exactly as before. Only `Shared\Mutex` and `Shared\Once` snapshot string values; every other primitive previews a number or a bool and could never reach this. - Named the right endpoint for `SHARED_PREVIEW_STRING_LIMIT` in the configuration table. It said `/entry?id=…`, but that limit is read only by the preview handler; `/entry?id=…` applies the array limit alone, which is presumably how the wrong name got copied down the table. Tests: - Three cases around the truncation. The regression uses a three-byte character deliberately: the default limit of 256 divides by 2 and by 4, so pure Cyrillic and pure emoji never split there, and a reproducer written with either passes on the unfixed build. Two controls hold the semantics the regression alone would not — an ASCII string proves the limit is still counted in bytes rather than characters, which rules out a `chars().take(limit/3)` implementation that would satisfy the regression test, and a zero limit pins the lower bound of the walk-back. Docs: - Said that preview truncation rounds down to a character boundary, and named the limit as bytes in both configuration tables; the previous wording left open whether it counted bytes or characters. 1346 unit tests pass with `--features plugin-shared`, 3 of them new.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What was wrong
GET /__ox_shared/preview?id=Ntruncates a long string value atSHARED_PREVIEW_STRING_LIMITbytes (256 out of the box) and cut the string at exactly that byte — a valid place to cut only while every character is one byte wide. When the cut byte fell inside a UTF-8 sequence, the slice panicked withbyte index 256 is not a char boundary.Nothing wraps a plugin internal route in
catch_unwind(PluginManager::handle_internal_route,handle_internal_request), and the internal server serves each connection inside atokio::spawn. So the panic killed the connection task: the client got a closed connection with no status and no body, indistinguishable from the server being gone, and the only trace was a panic in the log. The process itself survived — the build ispanic = unwind.Reachable through
Shared\MutexandShared\Once, the only two primitives whosedebug_snapshot()can yield a string; Counter, Atomic, Channel, Map and Pool return a number and Flag returns a bool, so none of them could reach it. A PHP string that is not valid UTF-8 never reaches the string arm either — the portbuf decoder routes it toSharedValue::Bytes, which previews as a byte count.Who actually hit it
Worth stating precisely, because the obvious intuition is wrong. Whether the cut lands mid-character is arithmetic between the limit and the character width: for a string of characters
wbytes wide, byteLis a boundary exactly whenL % w == 0. At the default limit of 256:256 % w€, DevanagariSo pure Cyrillic and pure emoji values never panicked at the default limit, three-byte scripts always did, and mixed text was a coin flip. A different limit moves the whole table (255 splits Cyrillic, for instance). This matters for anyone writing a reproducer: one built on Cyrillic or emoji passes even on the unfixed build.
The fix
Walk the limit back to the nearest character boundary before slicing (
while cut > 0 && !s.is_char_boundary(cut) { cut -= 1 }— the stable equivalent offloor_char_boundary, which is not available on the project's MSRV of 1.91.1). The branch is only entered whens.len() > str_limit, so the starting index is always inside the string and the walk terminates at 0 at worst.The limit stays a byte budget, which is what the documentation already described: the prefix simply lands up to 3 bytes under it. ASCII output is byte-for-byte unchanged.
Tests
Three cases, added next to the existing observability tests:
preview_truncates_multibyte_string_at_char_boundary— the regression. 100 ×€(300 bytes, byte 256 mid-sequence), expecting exactly 85 characters / 255 bytes. A three-byte character is used deliberately, per the table above.preview_string_limit_stays_a_byte_budget— ASCII control on 300 bytes expecting a 256-byte prefix. This is not decoration: it is what rules out achars().take(limit / 3)implementation, which would satisfy the regression test and quietly turn the limit into a character count.preview_zero_limit_emits_empty_prefix— pins the lower bound of the walk-back.Before the change, the regression test fails with
byte index 256 is not a char boundary; it is inside '€' (bytes 255..258); the two controls pass on both sides by design.Docs
SHARED_PREVIEW_STRING_LIMIT— it said/entry?id=…, but that limit is read only by the preview handler./entry?id=…applies the array limit alone, which is presumably how the wrong name got copied down the table.llms-full.txtregenerated for the docs edits.Verification
cargo fmt -- --checkcargo clippy --no-default-features -- -D warningscargo clippy --no-default-features --features plugin-shared --all-targets -- -D warningscargo test --no-default-features— 27 suites, no failurescargo test --no-default-features --features plugin-shared -- --test-threads=1— 1346 lib tests passed, 0 failedscripts/gen-llms-txt.sh --checkNo Docker or integration-suite run: the change touches no SAPI, worker, routing or PHP-gated code, and
render_previewis a private synchronous function with no registry or FFI involvement.