Repository navigation
Test(editor): a parse that ran out of time reads the rest of the note again - #940
Merged
Merged
Conversation
… again After 2.69.1, the note grammar's habit of reading text again looked like waste worth removing. noteMarkdown's outer tree always spans the whole note, so when a state update's 20 ms parse slice runs out, CodeMirror drops the older tree past the point the parse reached, and the parse that resumes reads that text again instead of reusing its blocks. Reusing those blocks was tried, and it is not waste. Lezer's markdown reuse can keep a block whose meaning changed with text outside it: an indented code block whose indent was deleted along with the frontmatter, a paragraph whose end moved when a table row under it broke. The grammar happens to read again exactly where that bites. Checked against a fresh parse 50,000 times over 100,000 random edits, the grammar as it is was stale 3 times, with that reuse 29 times, and plain markdown() 205 times. The time it would save is small: a typed character parses in about 4 ms in an 800 KB note, inside its slice, so a cut needs a slow machine or a pause first, and reading the rest after one costs what opening the note costs, in background slices. So the parser does not change. The header of cm-markdown-language.ts says why both full reads (after a cut, and when the body start moves) stay, and that syntaxTree(state).length is always the whole note, so code that needs to know how far the parse got asks ensureSyntaxTree or syntaxTreeAvailable. A new test pins the read after a cut, next to the one that pins the full parse when the closing fence is typed; both fail on the reusing version.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
After 2.69.1, the note grammar's habit of reading text again looked like waste worth removing. It is not, so this PR changes no code: it records why, and pins it with a test.
What was tried
noteMarkdown's outer tree always spans the whole note. When a state update's 20 ms parse slice runs out, CodeMirror drops the older tree past the point the parse reached, and the parse that resumes reads that text again instead of reusing its blocks. When the body start moves (the closing frontmatter fence typed or broken), parseMixed loses the old mount and the body is parsed in full.A custom mount that reused the older trees in both cases brought block reuse after a cut from 0 of 390 to 389 of 390, as plain markdown does. A lockstep fuzz then checked every incremental tree against a fresh parse (2,000 seeds of 25 random edits each, two generators, mounts and code fence overlays compared):
markdown()The causes are in Lezer's markdown reuse. A fragment that starts at the start of the parse kept an indented code block whose indent had been deleted along with the frontmatter. A paragraph directly above a table stays split from the table's lines when the table's delimiter row breaks; that one also happens on an ordinary edit, today and with plain markdown, and heals on the next edit nearby. Reading again after a cut and after a moved body start is what keeps the grammar out of the extra cases.
The time the reuse would save is small. A typed character parses in about 4 ms in an 800 KB note, well inside its slice, so a cut needs a slow machine or a pause first; reading the rest after one costs what opening the note costs (6, 21 and 84 ms at 50, 200 and 800 KB), in background slices.
What changes
cm-markdown-language.tssays why both full reads stay, and thatsyntaxTree(state).lengthis always the whole note, so code that needs to know how far the parse got asksensureSyntaxTreeorsyntaxTreeAvailable(asheadingAboveintable-to-database.tsdoes since 2.69.1).No behavior change, and not a release: this can ride with the next desktop release.
How to test locally
cd packages/app-core && npx vitest run src/lib/cm-markdown-language.test.ts: 18 tests pass.npx tsc --noEmit -p tsconfig.jsoninpackages/app-core: clean.