Skip to content

Add an experimental opt in tar based implementation of the local cache#1065

Open
christiango wants to merge 8 commits intomasterfrom
christiango/tar-based-cache
Open

Add an experimental opt in tar based implementation of the local cache#1065
christiango wants to merge 8 commits intomasterfrom
christiango/tar-based-cache

Conversation

@christiango
Copy link
Member

@christiango christiango commented Mar 23, 2026

We're seeing that for operations that are fast (such as transpiling with SWC) lage caching overhead is extremely high from copying the files over to the cache. This implements a local cache that uses tar for storing the cache. Here is the performance delta for this implementation from an internal repo

Platform Cold cache on branch (avg) Cold cache master (avg) Speedup
Windows (NTFS) 19.31s 34.78s 1.8x
M1 Mac (APFS) 17.53s 42.07s 2.4x
Codespaces (Linux) 14.0s 20.4s 1.5x
WSL (ext4) 13.49s 15.60s 1.2x

And for reference here is the overhead before and after compared to lage --no-cache on that same repo

Platform No cache (avg) Cold cache on branch (avg) Cold cache on master (avg) Tar vs no-cache Master vs no-cache
Windows (NTFS) 14.25s 19.31s 34.78s 1.4x slower 2.4x slower
M1 Mac (APFS) 14.41s 17.53s 42.07s 1.2x slower 2.9x slower
Codespaces (Linux) 13.2s 14.0s 20.4s 1.1x slower 1.5x slower
WSL (ext4) 12.34s 13.49s 15.60s 1.1x slower 1.3x slower

Key takeaway: Master's file-copy cache adds 1.3x–2.9x overhead vs no cache at all. The tar cache reduces that overhead to just 1.1x–1.4x, making cold-cache builds nearly as fast as uncached builds.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant