Skip to content

feat(muya): progressively mount large documents (#4887) - #4946

Open
Renakoni wants to merge 10 commits into
marktext:developfrom
Renakoni:feat/progressive-mount
Open

Renakoni wants to merge 10 commits into
marktext:developfrom
Renakoni:feat/progressive-mount

Conversation

@Renakoni

@Renakoni Renakoni commented Jul 16, 2026 •

Copy link
Copy Markdown
Contributor

Progressively mount documents with many top-level blocks in #4887, stacked on #4942. The complete logical state is available synchronously; DOM construction starts with a prefix and continues in scheduled chunks. This reduces initial DOM work without claiming to resolve every large-document freeze.

Updated through 37bd8816, on revised #4942 (21418d13). The shared state ownership and History changes live in #4942; the additional commits here implement state-based TOC, progressive mounting, and lifecycle/consumer fixes.

Behavior and contracts

  • TOC, reference definitions, and Markdown/state serialization read complete JSON state. DOM readers see only mounted blocks. TOC entries carry a top-level index for on-demand mounting before scrolling, retaining develop's heading scroll-position calculation.
  • Flush queued edits and re-read state before appending from the mounted child count. Pre-mount OT targets and their required successor from pre-dispatch state; suspend on-demand mounting during pick/drop to avoid applying operations twice.
  • Deep-clone mounted slices because blocks retain mutable metadata. Budget counting uses an iterative traversal and stops once the prefix budget is reached.
  • Replacement and destruction invalidate pending work. Synchronous flush/progress/completion callbacks can invalidate requests or finish the same document without duplicate continuation timers. Detached hosts remain supported.
  • Cursor restoration stops before restoring old source/history if a callback invalidates its temporary document. Frontier navigation distinguishes a successor, real EOF (null), and cancellation (false): paragraph arrows, code Shift+Enter, and table Enter/ArrowDown abandon canceled requests rather than appending to a replacement/destroyed document. Tab and Delete safely stop on cancellation as well.
  • Search, footnotes, and end-of-document consumers may finish mounting synchronously. Container-background clicks are inert while a tail is pending.
  • muya-mount-progress / muya-mount-complete describe progressive mounts. Small/empty documents mount synchronously without these events; canceled mounts do not emit completion. Public ensureMountedThrough returns false when its request is invalidated.

Limits

The 4 ms check runs between top-level blocks; it is not a task-duration or interaction-latency guarantee. A giant list, table, paragraph, or code block remains atomic. Parsing, browser layout, and synchronous full-tree consumers can still block. Container-internal virtualization is outside this PR.

The original 1.8 s / 9.6 s figures are historical results from an older revision, not current measurements of first paint/input readiness. A new single-sample local comparison mounted 512 of 3000 independent paragraphs initially, while a 3000-item list and 1000-row table still mounted synchronously as single blocks. This illustrates scope, not a universal responsiveness guarantee.

Validation

At 1418941b, the full unit run passed 1934 tests; two existing tests exceeded 5 seconds during concurrent builds, then passed with unchanged timeouts in a single-worker rerun (14 tests across their files). All 1347 specification tests, product TypeScript, scoped lint, circular check, and Muya/desktop builds passed. Existing lint warnings remain.

26 Chrome tests passed, covering clipboard, export, search, 10k-paragraph completion, and pending-tail consumers. After the cursor fix, all three progressive-consumer browser tests and 16 Electron integration tests passed again (source mode, tabs/history, TOC, and anchor scrolling).

The final review reproduced a missing combination: frontier Enter + replacement/destruction during a mount callback. Three tests first failed, demonstrating unwanted paragraph insertion. 37bd8816 fixes this and adds 20 regression cases across seven exits, including same-document reentrant completion. On that final revision, 131 targeted tests passed across 10 files (mount/cursor/table navigation, true EOF behavior, and arrows); TypeScript and scoped ESLint also passed, with the existing arrow-handler complexity warning. The earlier full-suite/build/browser totals belong to 1418941b, not a new full run of the final small patch.

@Renakoni
Renakoni force-pushed the feat/progressive-mount branch 6 times, most recently from 3e191e8 to 7df61bc Compare July 16, 2026 11:08
@Renakoni
Renakoni force-pushed the feat/progressive-mount branch 2 times, most recently from f026dad to a66b1df Compare July 16, 2026 23:04
@Renakoni

Renakoni commented Jul 16, 2026 •

Copy link
Copy Markdown
Contributor Author

A short wrap-up now that the fix has settled (head a66b1df6, squashed back to the two logical commits).

What ultimately changed. Beyond the headline (weight-budgeted synchronous prefix + time-budgeted background chunks, with the pending tail read live from jsonState rather than a snapshot), the durable work turned out to be the consumer contracts:

  • whole-document readers (TOC, serialization, search, the footnote map) read the state or complete the mount explicitly — and skip the work when no consumer is installed;
  • everything that touches the tail materializes partially (ops via ensureMountedForOperation, cursor restore, queryBlock, TOC clicks) — never a full flush;
  • every "am I at the document end?" gesture — paragraph ArrowDown/ArrowRight, table Enter/ArrowDown/Tab, code-block Shift+Enter, Delete-forward merge — resolves through one shared resolver that advances strictly from the mounted boundary (children.length), and the container click is simply inert while a tail is pending;
  • history drains the rAF batch before touching its stacks; headingCopyLink resolves block path and state at the same revision.

Why it took seven rounds. The chunked mount itself was round one; everything after was one consequence surfacing at a time: once the mounted tree stops being the whole document, every scattered place that equated "no next block" with "end of document" becomes subtly wrong — and those assumptions lived in arrow/enter/tab/delete handlers, click handling, footnotes, and history timing. The rounds converged from per-site patches to the shared resolver only after the whole class was mapped, and the final round caught the last coordinate bug in that resolver (receiver-relative instead of frontier-relative advancement). Each finding is now a regression test — 31 in the chunked-mount spec — and the full suite is green at 1478/1478.

niyo-oyin added a commit to niyo-oyin/marktext-ja that referenced this pull request Jul 29, 2026
- PR marktext#4946 (includes its base PR marktext#4942): removes the quadratic costs
  that froze large documents and mounts them progressively — a 2.1 MB
  document goes from an indefinite freeze to interactive in seconds.
  Applied verbatim; upstream review addressed 19 findings with 31
  regression tests, all passing here.
- PR marktext#4931 (marktext#4926): deleting at an offset inside a surrogate pair left
  a lone surrogate that crashed text-unicode ("Invalid offset - splits
  unicode bytes"). The content.ts hunk conflicted with our Phase 1
  isComposed guards and was integrated by hand; the new cursor snap
  additionally runs only when no IME composition is open, since moving
  the DOM selection mid-composition would detach the IME anchor.
- PR marktext#4913 (marktext#4899): Shift+Tab outdent with trailing sibling blocks
  corrupted the state tree (paragraphs appended directly into a list),
  crashing the next composed op. muya hunks only — the PR's legacy
  muyajs hunks target the removed engine.
- PR marktext#4911 skipped: the equivalent detached-block guard already exists
  in @muyajs/core's language selector (marktext#4654), verified in source.
- PR marktext#4930 (marktext#4921): CodeMirror instance wrapped in a Vue proxy broke
  source-code mode; markRaw fix applied cleanly.

muya suite 1515/1515, tsc/eslint/madge clean.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ef4yFuXFbyKNTazkwUtygY
niyo-oyin added a commit to niyo-oyin/marktext-ja that referenced this pull request Jul 29, 2026
- PR marktext#4946 (includes its base PR marktext#4942): removes the quadratic costs
  that froze large documents and mounts them progressively — a 2.1 MB
  document goes from an indefinite freeze to interactive in seconds.
  Applied verbatim; upstream review addressed 19 findings with 31
  regression tests, all passing here.
- PR marktext#4931 (marktext#4926): deleting at an offset inside a surrogate pair left
  a lone surrogate that crashed text-unicode ("Invalid offset - splits
  unicode bytes"). The content.ts hunk conflicted with our Phase 1
  isComposed guards and was integrated by hand; the new cursor snap
  additionally runs only when no IME composition is open, since moving
  the DOM selection mid-composition would detach the IME anchor.
- PR marktext#4913 (marktext#4899): Shift+Tab outdent with trailing sibling blocks
  corrupted the state tree (paragraphs appended directly into a list),
  crashing the next composed op. muya hunks only — the PR's legacy
  muyajs hunks target the removed engine.
- PR marktext#4911 skipped: the equivalent detached-block guard already exists
  in @muyajs/core's language selector (marktext#4654), verified in source.
- PR marktext#4930 (marktext#4921): CodeMirror instance wrapped in a Vue proxy broke
  source-code mode; markRaw fix applied cleanly.

muya suite 1515/1515, tsc/eslint/madge clean.
@Renakoni
Renakoni force-pushed the feat/progressive-mount branch from a66b1df to 37bd881 Compare September 19, 2026 05:03
Resolve the lexBlock conflict by keeping LinearMarked with texMathDollars, and update the two PR tests to the renamed option.
Bring in develop and the conflict resolution from PR marktext#4942, preserving LinearMarked and the renamed texMathDollars option.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant