Skip to content

Summarize a dictionary or a MAT-file from a scan the caller already has - #30

Merged
ww-mw merged 2 commits into
mainfrom
summarize-from-scan
Sep 16, 2026
Merged

ww-mw merged 2 commits into
mainfrom
summarize-from-scan

Conversation

@ww-mw

@ww-mw ww-mw commented Sep 16, 2026

Copy link
Copy Markdown
Member

summarizeSlddScan and summarizeMatScan — the analogue of summarizeParsedModel
for the other two file kinds. Each takes a scanSldd / scanMat result instead of
bytes and returns the FileSummaries for that one file.

Why

A caller that has already scanned a data file could not get its summary without
scanning it again. data-explorer-vscode's cheap tier is that caller: it wants the
raw refs a scan carries and the summary, out of one read. The second scan measured
46% of that tier's per-dictionary CPU — 14.6 ms of 31.5 ms on a 20,000-entry
dictionary — spent deriving a names list it was already holding.

What changes here

summarizeFiles's sldd and mat branches now route through the new functions, as its
model branch already routes through summarizeParsedModel. It still scans the bytes
itself — it takes bytes, not scans — and only stops open-coding the summary. So the
two ways to reach one summary now share an implementation rather than agreeing.

That weakens the obvious test, and the new sweep says so: an equality between the two
paths over every committed .sldd and .mat cannot fail on shared code. So every
swept file is also compared against a summary derived from the full readers
(readSlddContent, parseMat). Verified by mutation — without that second assertion,
dropping filter(Boolean) from the summary left the whole sweep green.

The constructed cases cover what no fixture has: references in both spellings and
mixed case, unnamed entries around a named one, the basename key rule, a dictionary
whose fast path refuses and whose full read recovers, a truncated MAT, and two files
no reader can read at all.

Verification

  • npm run typecheck clean
  • npm test — 159 files / 4741 tests passing (was 158 / 4676 on main)
  • npm run build leaves dist/ byte-identical to what is committed here, and
    dist/index.d.ts declares both new exports (the prebuilt dist/ ships because
    git-dependency installs skip prepare)

`summarizeSlddScan` and `summarizeMatScan`, the exact analogue of
`summarizeParsedModel` for the other two file kinds: a `FileSummaries`
holding the one file, built from a `scanSldd`/`scanMat` result instead of
from bytes.

A caller that has already scanned a data file could not get its summary
without scanning it again. data-explorer-vscode's cheap tier is that
caller — it wants both the raw `refs` a scan carries and the summary out
of one read — and the second scan measured 46% of that tier's
per-dictionary CPU: 14.6 ms of 31.5 ms on a 20,000-entry dictionary,
spent deriving a `names` list it was already holding.

`summarizeFiles`'s sldd and mat branches now route through the new
functions, as its model branch already routes through
`summarizeParsedModel`. It still scans the bytes itself — it takes bytes,
not scans — and only stops open-coding the summary, so the two ways to
reach one summary share an implementation rather than agreeing.

Which weakens the obvious test, and the new sweep says so: an equality
between the two paths over every committed `.sldd` and `.mat` cannot fail
on the shared code, so every swept file is ALSO compared against a
summary derived from the full readers `readSlddContent` and `parseMat`.
Verified by mutation — without that second assertion, dropping
`filter(Boolean)` from the summary left the whole sweep green. The
constructed cases cover what no fixture has: references in both
spellings and mixed case, unnamed entries around a named one, the
basename key rule, a dictionary whose fast path refuses and whose full
read recovers, a truncated MAT, and the two files no reader can read at
all.
@ww-mw
ww-mw merged commit b6fc4e1 into main Sep 16, 2026
1 check passed
@ww-mw
ww-mw deleted the summarize-from-scan branch September 16, 2026 18:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant