Repository navigation
fix(temp): remove compile build paths and sweep the shared temp directory - #233
Merged
Merged
Conversation
…tory Every distinct compile left its build directory behind: the compile worker's build-output/<sketchHash> was never evicted, and main-thread compiles without --build-path made arduino-cli keep one directory per random sketch path. The periodic sweeper scanned process.cwd()/temp, so finished sandbox runs renamed to <sharedTemp>/unosim-temp/<id>.cleanup were never removed either. - ArduinoCompiler uses one build directory per compile (the caller's or its own sibling of the sketch directory) and removes it after the compile, also on failure and cache hits; the HEX stays in the caches. - sweepTempArtifacts removes old .cleanup items in the legacy and the shared sandbox temp directory and stale worker build-output entries. - SandboxRunner, compile worker and sweeper share one path definition. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Root cause (verified)
Two independent leaks made the shared temp bind mount (
${PWD}/tempin Compose) grow without bound:--build-path <workerBuildDir>/build-output/<sketchHash>and never removed it. The LRU eviction only covershex-cache,binariesand the core cache.--build-path. arduino-cli then keeps one directory per (random UUID) sketch path in its own cache. On the dev machine this had accumulated 62,167 UnoSim build directories (9.3 GB).<sharedTemp>/unosim-temp/<id>.cleanupfor deferred removal.server/index.tsonly scanned the top level ofprocess.cwd()/temp. In Compose that is/app/temp, not the bind mount, and in no configuration is it theunosim-tempsubdirectory. The.cleanupdirectories were never removed; locally 59 were still present hours later.Proof on the old code: all four tests in
compile-build-path-cleanup.test.tswere red. For example, 20 distinct compiles in the worker layout left 20build-outputdirectories, and a compile without an explicit build path passed none to arduino-cli.Fix
ArduinoCompileruses one build directory per compile: the caller's or its own sibling of the sketch directory. It removes that directory infinally, also on failure, unsafe-include rejection and cache hits. The useful artifact, the HEX, stays inhex-cache/binariesas before. The main-thread path now also finds its HEX; it was previously alwaysundefinedthere.server/services/temp-artifact-sweep.ts:sandboxTempDir(),workerBuildRoot()), used bySandboxRunner, the compile worker and the sweeper;sweepTempArtifacts()removes entries older than 5 minutes:.cleanup/.cleanup.jsonitems in the legacy directory and inunosim-temp, plus staleworker_*/build-output/*entries, so leftovers from before this fix go away too;server/index.tsruns that sweep every minute, as before.INSTALL_SERVER.md.Tests
tests/server/services/compile-build-path-cleanup.test.ts: 20 distinct compiles in the worker layout, failed compile, cache hit, compile without an explicit build path. Each leaves no build directory, and the HEX cache is kept.tests/server/services/temp-artifact-sweep.test.ts: 25 finished runs, renamed by the realFilesystemHelper, are removed once old. Fresh.cleanupand active run directories are kept. Stalebuild-outputentries are removed and in-progress ones kept. Also covers the legacy directory and missing directories.arduino-cli1.5.1 compile without an explicit build path succeeds with the sibling build path, returns the HEX and adds nothing to arduino-cli's sketch cache.Existing tests are unchanged.
npm run check,npm run test:unit,npm run check:docs, Knip,git diff --check, the full./run-tests.sh(incl. toolchain, Docker, E2E, build, audit, Sonar) and the pre-push gate are green. The first pipeline run caught that the worker thread needs relative.tsimports;temp-artifact-sweep.tsfollows that convention, andcompilation-worker-pool-startup.test.tscovers it.🤖 Generated with Claude Code