feat(site): add smooth streaming text engine for LLM responses#22503
Merged
feat(site): add smooth streaming text engine for LLM responses#22503
Conversation
Ports the jitter-buffer approach from coder/mux to smooth out bursty LLM token delivery into steady character-by-character reveal. SmoothText.ts contains: - SmoothTextEngine: pure state machine with adaptive-rate budget model (72-420 cps, frame-rate invariant, max 120 char visual lag) - useSmoothStreamingText: React hook bridging the engine via RAF loop with single-integer state, grapheme-safe slicing, and self-terminating animation - STREAM_SMOOTHING constants for tuning SmoothText.test.ts covers all 8 engine behaviors including frame-rate invariance across 60Hz/240Hz displays.
Adds SmoothedResponse component that wraps <Response> with the useSmoothStreamingText hook during live streaming. Historical messages bypass smoothing and render through <Response> directly. The integration is minimal: renderBlockList's 'response' case now branches on isStreaming to pick SmoothedResponse vs plain Response.
- biome import ordering (alphabetical) - biome-ignore for intentional useEffect deps (fullText, streamKey trigger RAF re-arm and stream reset) - minor formatting adjustments
UseSmoothStreamingTextOptions and UseSmoothStreamingTextResult are only consumed internally by useSmoothStreamingText. They can be re-exported later if external consumers need them.
… SmoothText (#22509) ## Summary Move RAF loop ownership and subscriber management into `SmoothTextEngine`. The hook now uses `useMemo` (keyed on `streamKey`) and `useSyncExternalStore` instead of two `useEffect`s, `useState`, and six `useRef`s. ## What changed ### `SmoothTextEngine` gained: - `listeners: Set<() => void>` — subscriber tracking - `rafId` / `previousTimestamp` — moved from the hook's refs into the engine - `subscribe(listener)` — adds a listener, returns unsubscribe (`useSyncExternalStore` contract) - `dispose()` — stops loop, clears listeners - `startLoop()` / `stopLoop()` — private RAF lifecycle - `frame` — the RAF callback, previously a ref-stored closure in the hook - `notify()` — calls all listeners - `update()` now calls `startLoop()`/`stopLoop()`/`notify()` as side effects of state changes - `reset()` now calls `stopLoop()` ### `useSmoothStreamingText` removed: - `useState` (visibleLength) - Both `useEffect`s (re-arm + lifecycle) - 6 refs: `engineRef`, `previousStreamKeyRef`, `visibleLengthRef`, `rafIdRef`, `previousTimestampRef`, `frameRef` - Both `biome-ignore` suppressions ### `useSmoothStreamingText` replaced with: - `useMemo(() => new SmoothTextEngine(), [streamKey])` — engine creation + reset on key change - `useSyncExternalStore(engine.subscribe, () => engine.visibleLength)` — reactive snapshot Everything else (constants, `clamp`, `getAdaptiveRate`, `tick()` internals, grapheme segmenter, `sliceAtGraphemeBoundary`, types, return value shape) is identical. ## Testing Existing `SmoothTextEngine` unit tests pass without modification.
DanielleMaywood
approved these changes
Mar 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
LLM responses currently stream in bulk chunks — multiple
message_partevents arrive per WebSocket frame, get batched into a singlestartTransitionstate update, and render as a visual jump. This looks janky compared to smooth character-by-character reveal.Solution
Port the jitter-buffer approach from coder/mux into a single self-contained file:
SmoothText.ts.What's in the file
STREAM_SMOOTHINGconstantsSmoothTextEngineclassuseSmoothStreamingTexthookrequestAnimationFrameloop, singleuseState<number>, grapheme-safe slicingHow the engine works
Intl.Segmenter(with codepoint fallback) to never split emoji mid-animationIntegration
To wire this up, wrap the
<Response>component inConversationTimeline.tsxwith the hook:Tests
8 engine tests covering: steady reveal, adaptive acceleration, max lag cap, immediate flush on stream end, bypass mode, content shrink, sub-char budget gating, and frame-rate invariance.