Skip to content

Port upstream 4946: progressively mount large documents#161

Open
Renakoni wants to merge 4 commits into
mainfrom
sync/upstream-4946-progressive-mount
Open

Port upstream 4946: progressively mount large documents#161
Renakoni wants to merge 4 commits into
mainfrom
sync/upstream-4946-progressive-mount

Conversation

@Renakoni

@Renakoni Renakoni commented Jul 16, 2026

Copy link
Copy Markdown
Owner

Ports upstream marktext#4946 (progressive mount, the second half of the marktext#4887 large-file freeze fix) into the vendored Muya, plus the one app-side consumer adaptation. Companion to #159 (the algorithmic half); the two are independent to merge.

What it does

Opening a large document still spends its time synchronously building every block's DOM before anything paints. ScrollPage now mounts an over-viewport prefix synchronously (budgeted by state weight, 512 nodes) and appends the rest in scheduled, time-budgeted chunks (4 ms JS budget per task): the document is visible and editable at once and the remainder mounts in the background. On phone-class CPUs this matters more than anywhere else — the upstream desktop measurement went from 25 s blocking to interactive in 9.6 s on a 2.1 MB document, and mobile parse/mount costs are a multiple of that.

The contracts (as reviewed upstream, seven Codex rounds, plus this repo’s parallel review)

  1. TOCgetTOC collects from the JSON state (complete before any block mounts); ITocItem carries the heading's top-level index; slugs key off state nodes via WeakMap, headingCopyLink resolves the same node.
  2. Offset-based cursor restore mounts partially through the sentinel's block (ensureMountedThrough), never a full flush.
  3. Structural edits during the mount window need no special casing — the pending tail is not a snapshot; every chunk drains pending ops (jsonState.flush()) and resumes at the live child count.
  4. Ops targeting the unmounted tail apply exactly onceupdateContents pre-mounts every touched top-level block plus its successor from the pre-dispatch state, and on-demand materialization is suspended during the pick/drop walk (a residual miss falls back to the existing wholesale rebuild).
  5. Mounted blocks never share state objects_appendFromState clones each slice before constructing blocks.
  6. A replaced document cannot inherit the old selectionsetContent clears the selection and active block so focus() cannot deep-mount to a stale caret.
  7. History drains the rAF op batch before the stack check and before muting json-change — a same-frame edit enters the undo stack instead of being silently dropped; redo() with a pending edit invalidates quietly instead of popping an emptied stack; a first unflushed edit is undoable immediately.
  8. LifecycleMuya.destroy()/updateState cancel an in-flight mount explicitly; detached hosts mount to completion.
  9. Search / end-of-document consumers — search completes the mount before walking; lastContentInDescendant forces the remainder; markdown serialization reads the state and is unaffected. Hosts can track muya-mount-progress / muya-mount-complete.
  10. The mount frontier never masquerades as the document end — for every frontier exit. ArrowDown / boundary ArrowRight in a paragraph, Enter / ArrowDown in a last table row, Shift+Enter out of a code block, forward-Tab out of a last table cell, and Delete-forward paragraph merge all resolve their successor through one shared pending-aware resolver (TreeNode.resolveNextContentInContext). The resolver advances from the mounted boundary itself — it asks for the first UNMOUNTED index (children.length) each round, never the receiver's own index + 1: the context walk has already scanned every mounted successor (including mounted empty containers), and ensureMountedThrough early-returns for any already-mounted index, so a receiver-relative request at the frontier would misread the pending tail as EOF. It keeps extending until a content appears or the mount stops progressing. Only a genuine document end falls through to the original behavior.
  11. Clicking the container background is inert while a tail is pending. The area below the frontier is NOT blank — the document logically continues there, so appending would land mid-document. Flushing on the click is no better: the container also catches taps in the gutter and between blocks (an ordinary tap would freeze a large file), and after the forced reflow the real bottom moves below the click point anyway. Background chunks finish shortly; end-of-document clicks then behave normally.
  12. The footnote tool sees the whole document — and only pays for it when installed. The map collection completes the mount first (a tail definition must not be duplicated by Create), and hosts with no muya-footnote-tool listener (new EventCenter.hasListeners) skip the preparation entirely. This host installs no FootnoteTool, so a footnote tap on Android now costs nothing.
  13. headingCopyLink resolves path and state at the same revision. The block path reflects same-frame tree mutations immediately while rawState updates on the rAF flush; _activate() drains the batch before indexing, so a click landing in that one-frame window cannot slug the wrong node.

App-side adaptations

Two app features resolve top-level blocks by index against the mounted DOM; both now materialize their target first (a no-op once mounted):

  • Document outline — the sheet already lists every heading (the TOC is state-collected), but its slug → element resolution maps the TOC index onto the mounted top-level heading elements; a tail heading during the mount window would resolve to nothing. selectHeading calls editor.ensureMountedThrough(item.index) before resolving, mirroring the upstream desktop TOC click.
  • Resume position — the "continue where you left off" card resolves its stored anchor as children[topBlockIndex]; beyond the prefix the offer would be silently discarded (and large documents are exactly where resume matters). Per this repo's review: the automatic open-time probe must never materialize blocks (a stored near-end index would synchronously deep-mount the document and reintroduce the freeze), so the offer validates a not-yet-mounted index against the logical top-level count (jsonState.rawState.length) — the hash match already proved the document identical, and a pending-tail target is far past the min-distance threshold by construction. Materialization happens only on the user's tap (activateResume). Genuinely out-of-range indices still discard.

caretFollow needs nothing: it follows the active block, which is always mounted. Serialization, autosave, and share flows read the JSON state and are unaffected.

Verification

  • Muya: the ported chunkedMount.spec.ts (31 contract tests — every review finding is a repro, incl. a below-frontier click inert while pending and appending after completion; a gutter click never flushing; frontier navigation for paragraph ArrowDown, table Enter/ArrowDown, and code-block Shift+Enter; resolution continuing across an empty container (unmounted AND already-mounted at the frontier), table Tab and Delete-forward merge reaching the pending successor; the footnote map covering a tail definition and skipping without a listener; copy-link slugging correctly under an unflushed structural edit) and the setCursorByOffset partial-mount repro pass; app-gate suite 1475/1475; tsc --noEmit clean.
  • App: outline test — a heading beyond the mounted prefix is materialized (ensureMountedThrough(2)) and resolved correctly. Resume probe tests (red-green verified against the previous code): a pending-tail target is offered without any ensureMountedThrough call; an index beyond the logical document discards without mounting; activation materializes through the target. documentOutline + resumePosition suites 44/44.
  • Root unit tests 581/581 (incl. the vendor-sync contract); pnpm typecheck, pnpm lint, pnpm build clean; node_modules/@muyajs/core resynced.
  • Full Playwright mobile-WebView e2e suite (first commit): 171 passed, 0 failed. After the review fixes, the directly affected mobile-editor-resume spec re-run in full: 14/14, including the previously flaky image spec, now clean.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant