Skip to content

Reduce parseSMT memory usage and improve performance#48

Merged
p2004a merged 2 commits intobeyond-all-reason:masterfrom
burnhamrobertp:perf/parsesmt-direct-blit
May 3, 2026
Merged

Reduce parseSMT memory usage and improve performance#48
p2004a merged 2 commits intobeyond-all-reason:masterfrom
burnhamrobertp:perf/parsesmt-direct-blit

Conversation

@burnhamrobertp
Copy link
Copy Markdown
Contributor

Related to #19

Work done

Rewrites tile assembly in parseSMT to blit decoded tiles directly into a pre-allocated output buffer instead of cloning, collecting, and concatenating them through multiple intermediate arrays.

The old code had three layers of intermediate buffers all in memory at once:

  1. refTiles[][] — decoded reference tiles split into row-buffers
  2. tiles[][] — full clone of every tile via Buffer.from() per row (500k+ tiles on large maps)
  3. tileStrips[] — horizontal strips joined via Buffer.concat()
  4. Final Buffer.concat(tileStrips) to produce the image

The new code decodes reference tiles as flat RGBA buffers, pre-allocates a single output buffer, and copies each tile's rows directly into position with Buffer.copy(). No cloning, no strips, no concat. Also eliminates Array.shift() on large arrays (O(n) per call, making the old strip assembly O(n²)).

Removes the now-unused cloneTile and joinTilesHorizontally methods.

Also wraps the parseSkybox call site in try/catch to match how parseResources handles errors — a missing or invalid skybox no longer crashes the entire parse.

Results

Benchmarked against three maps at mipmap 4 and 16 (3 runs each, averaged):

Map Mipmap Before After Speedup
hooked (small) 4 1,231ms 958ms 1.3x
kolmogorov (medium) 4 15,009ms 4,885ms 3.1x
proving_grounds (large) 4 41,326ms 5,869ms 7.0x
hooked (small) 16 4,289ms 1,058ms 4.1x
kolmogorov (medium) 16 17,438ms 6,605ms 2.6x
proving_grounds (large) 16 38,657ms 10,356ms 3.7x

Full regression across all 223 BAR maps at mipmap 4: all pass, all texture/height/metal/dry RGBA buffers hash identically before and after.

AI / LLM usage statement

GitHub Copilot CLI (Claude Opus 4.6) used to assist in codebase analysis, implementation, benchmarking, and regression testing. All code was reviewed, directed, and verified by the contributor.

Copy link
Copy Markdown
Collaborator

@p2004a p2004a left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome, thank you!

@p2004a p2004a merged commit 4cf9c69 into beyond-all-reason:master May 3, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants