Skip to content

fix(js): a minified bundle no longer takes hours to extract - #204

Merged
taylorg009 merged 3 commits into
mainfrom
fix-minified-js-hang
Oct 16, 2026
Merged

taylorg009 merged 3 commits into
mainfrom
fix-minified-js-hang

Conversation

@taylorg009

@taylorg009 taylorg009 commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Found by the ContextBench run (#201). Indexing element-hq/element-web@c9d9c421 never finished: the build ran 2 h 9 min before it was killed. A per-file scan pinned it on docs/lib/mermaid.min.js (2,935,907 bytes, lines up to 278,302 characters). A daemon opened on any repository that vendors a bundle like this would hang the same way.

Cause (gdb samples, release build)

The time was in the local-value analysis behind HTTP-client detection (http_call_handler -> analyze_client_call -> collect_url_template / options_method -> local_values).

  • Scope walk per read. local_values walked the variable's whole scope for every read, via visit_named_descendants.
  • Block rescan per lookup. local_binding rescanned the enclosing block's statements on every lookup.
  • Quadratic child access. Both walked children with ts_node_named_child(node, i), which costs O(i).

In a minified bundle the scope is most of the file and reads number in the thousands.

Fix (src/engine/http_consumers.cpp)

  • ExtractionMemo. A per-parse memo (language_config.hpp; extractor.cpp gives each parse a fresh one, because it holds that parse's tree nodes) indexes, once:

    • each scope's writes, by the name each write can affect, in document order;
    • each block's first declarator of each name.

    local_values and local_binding read these indexes. The per-write logic is unchanged.

  • Cursor walks. visit_named_descendants and the block scan use a tree cursor instead of indexed child lookups.

  • One pattern traversal. binds_pattern is now a lookup over pattern_names, the same traversal the write index uses.

Evidence

Before After
mermaid.min.js alone killed after 60 s, 5 min and 2 h 17-28 s
Synthetic, N = 500 1.74 s 0.26 s
Synthetic, N = 1500 14.2 s 0.23 s
New scale test (N = 3000) 62.9 s, fails 0.15 s, passes

Synthetic N means N let locals, N unrelated writes and N fetch(local) calls in one function.

  • Same output: the original code never finished mermaid.min.js, so its graph is compared across the steps of this fix instead: the first step (memo only, 51.5 s) and the final code produce identical graphs (8,318 nodes, 180,429 links, node ids and links compared in full). Against the original code, equality is shown on the synthetic inputs. The synthetic outputs are link-for-link equal, with all N calls resolved to /api/v1/items/<i>.
  • Tests: full ctest (release, mars) 99/100. The one failure is cgraph_file_watcher_test, the known mars-only failure.
  • Not addressed here: the remaining 17-28 s on that file is spread across ordinary pipeline stages (community layout, exports, call resolution). Whether minified bundles should be indexed at all is a separate policy question.

🤖 Generated with Claude Code

Indexing element-web's docs/lib/mermaid.min.js (2.9 MB, lines up to 278k
chars) never finished: one build ran 2 h before it was killed. local_values
walked the variable's whole scope for every read, and local_binding rescanned
its block, both through ts_node_named_child(node, i), which is O(i); in a
minified bundle the scope is most of the file and reads are thousands.

- A per-parse ExtractionMemo indexes each scope's writes by the name they can
  affect and each block's declarations by name, once.
- Child walks use a tree cursor (linear) instead of indexed child lookups.
- binds_pattern is a lookup over pattern_names, the one traversal the write
  index also uses.

mermaid.min.js: killed after 2 h before, 17-28 s now, identical graph (8,318
nodes, 180,429 links). The new scale test (3,000 locals, writes and calls in
one function) takes 62.9 s on the old code and 0.15 s now.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

PR brief: 0 agent commits · run the full suite

Warning

Run the full suite. The graph cannot vouch for this diff, so every test file is selected. Nothing is skipped; this is the safe default, not a failure.

8 files have no graph node (config, asset, or unextracted).

If these files cannot change which tests should run, add a pattern for them to ignore. If they feed tests (fixtures, data), this outcome is correct.

bench/billed-cost/results  ████████████████████████  7
bench/billed-cost          ███  1
All 8 unmapped files
bench/billed-cost/results/multi-cgraph.json
bench/billed-cost/results/multi-cgraph_directed.json
bench/billed-cost/results/multi-gold-recall.json
bench/billed-cost/results/multi-hook-live-cgraph.json
bench/billed-cost/results/multi-hook-live-gold-recall.json
bench/billed-cost/results/multi-read-hook-cgraph.json
bench/billed-cost/results/multi-read-hook-gold-recall.json
bench/billed-cost/tasks-multi.json

Computed at a225121 against base 43453f3.

Summary PR #204
Intent no checkpoints on this branch (6 commits)
Since push 7e1057d 1 new commit, -5 changed files, -49 tests reached

What each commit did

Commit Intent Files Reaches Ran before push
52b7746 fix(js): a minified bundle no longer takes hours to extract no checkpoint 4 files
7e1057d test(client): wait for the lone repository's graph before a… no checkpoint 1 file
1403ab0 ci: cold-install timing on clean macOS/Linux runners + Home… no checkpoint 5 files
51105f8 test(client): wait for the lone repository's graph before a… no checkpoint
c106d73 bench: multi-file billed-cost A/B — agents still don't call… no checkpoint 11 files
a225121 style(js): put visit_named_descendants' comment back above … no checkpoint 1 file

Claims checked

no checkpoint on this branch makes a claim the graph can check

blastline 0.13.0. Selection is advisory unless your workflow gates on it. How selection works. Intent: 0 of 6 commits. Not checked: symbol changes: no --change-context given.

taylor009 and others added 2 commits October 15, 2026 14:56
…hook

Same fix as #200 (identical hunk), so this branch's CI stops hitting the
'lone repository: null' race.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@taylorg009
taylorg009 merged commit a41425e into main Oct 16, 2026
10 checks passed
taylor009 pushed a commit that referenced this pull request Oct 17, 2026
…uote

- RESULTS.md finding 4: the minified-bundle hang is fixed on main by #204;
  numbers stay pinned to 43453f3, no 500/500 rerun yet.
- Quote Appendix H exactly ("averaged over evaluated tasks").
- Per-source rows for every K live in results.json, not results.md.
- run.sh: steps 4 and 5 use $PY like the other steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
taylor009 added a commit that referenced this pull request Oct 17, 2026
… item 2: fails, 0.100 vs 0.5) (#201)

* feat(bench): ContextBench Lite file precision/recall bench for CGraph (build plan item 2)

Adds bench/contextbench/: a pinned, one-command run that scores CGraph's
retrieved files against ContextBench's human-annotated gold contexts with
ContextBench's own file-level metric (EuniAI/ContextBench @ 1436c28, HF data
@ c285579, all 500 Lite instances). Retrieval reuses the ARB adapter; the
snapshot lifecycle is extracted from bench/arb/cgraph_eval.py into
served_snapshot so both benches share it (ARB output unchanged).

Result on bin-v0.9.4-1-g43453f3: default-context file precision 0.100
(95% CI 0.089-0.112), best top-K precision 0.217 at K=2. The ContextBench
half of item 2's gate (>= 0.5) fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* docs(bench): note #204 fixed the element-web hang; exact Appendix H quote

- RESULTS.md finding 4: the minified-bundle hang is fixed on main by #204;
  numbers stay pinned to 43453f3, no 500/500 rerun yet.
- Quote Appendix H exactly ("averaged over evaluated tasks").
- Per-source rows for every K live in results.json, not results.md.
- run.sh: steps 4 and 5 use $PY like the other steps.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Taylor Gagne <taylor@baremetalcloud.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
taylorg009 added a commit that referenced this pull request Oct 17, 2026
Launcher release.json (four assets, SHA-256 from the release, checked
against the downloaded files), plugin version, npm package.json,
server.json, the Homebrew formula and the README install steps, written by
scripts/pin_launcher_release.py bin-v0.9.5 plus the README bump.
bin-v0.9.5 is main c974bee: minified bundles no longer take hours to
extract (#204), daemon start at a git repository root (#199), the short
skill body (#196) and the CGR-18/19/20 fixes (#194, #206, #207).

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants