Skip to content

fix(temp): remove compile build paths and sweep the shared temp directory - #233

Merged
ttbombadil merged 1 commit into
mainfrom
fix/temp-artifact-growth
Oct 9, 2026
Merged

ttbombadil merged 1 commit into
mainfrom
fix/temp-artifact-growth

Conversation

@ttbombadil

Copy link
Copy Markdown
Collaborator

Root cause (verified)

Two independent leaks made the shared temp bind mount (${PWD}/temp in Compose) grow without bound:

  1. Compile build directories.
    • The compile worker passes --build-path <workerBuildDir>/build-output/<sketchHash> and never removed it. The LRU eviction only covers hex-cache, binaries and the core cache.
    • Main-thread compiles (local mode, worker fallback) passed no --build-path. arduino-cli then keeps one directory per (random UUID) sketch path in its own cache. On the dev machine this had accumulated 62,167 UnoSim build directories (9.3 GB).
  2. Sandbox sketch directories.
    • A finished run renames its directory to <sharedTemp>/unosim-temp/<id>.cleanup for deferred removal.
    • The periodic sweeper in server/index.ts only scanned the top level of process.cwd()/temp. In Compose that is /app/temp, not the bind mount, and in no configuration is it the unosim-temp subdirectory. The .cleanup directories were never removed; locally 59 were still present hours later.

Proof on the old code: all four tests in compile-build-path-cleanup.test.ts were red. For example, 20 distinct compiles in the worker layout left 20 build-output directories, and a compile without an explicit build path passed none to arduino-cli.

Fix

  • Build paths: ArduinoCompiler uses one build directory per compile: the caller's or its own sibling of the sketch directory. It removes that directory in finally, also on failure, unsafe-include rejection and cache hits. The useful artifact, the HEX, stays in hex-cache/binaries as before. The main-thread path now also finds its HEX; it was previously always undefined there.
  • New server/services/temp-artifact-sweep.ts:
    • single definition of the shared paths (sandboxTempDir(), workerBuildRoot()), used by SandboxRunner, the compile worker and the sweeper;
    • sweepTempArtifacts() removes entries older than 5 minutes: .cleanup/.cleanup.json items in the legacy directory and in unosim-temp, plus stale worker_*/build-output/* entries, so leftovers from before this fix go away too;
    • active sketch directories, recent items and caches are not touched.
  • server/index.ts runs that sweep every minute, as before.
  • Docs: INSTALL_SERVER.md.

Tests

  • tests/server/services/compile-build-path-cleanup.test.ts: 20 distinct compiles in the worker layout, failed compile, cache hit, compile without an explicit build path. Each leaves no build directory, and the HEX cache is kept.
  • tests/server/services/temp-artifact-sweep.test.ts: 25 finished runs, renamed by the real FilesystemHelper, are removed once old. Fresh .cleanup and active run directories are kept. Stale build-output entries are removed and in-progress ones kept. Also covers the legacy directory and missing directories.
  • Real toolchain check: an arduino-cli 1.5.1 compile without an explicit build path succeeds with the sibling build path, returns the HEX and adds nothing to arduino-cli's sketch cache.

Existing tests are unchanged. npm run check, npm run test:unit, npm run check:docs, Knip, git diff --check, the full ./run-tests.sh (incl. toolchain, Docker, E2E, build, audit, Sonar) and the pre-push gate are green. The first pipeline run caught that the worker thread needs relative .ts imports; temp-artifact-sweep.ts follows that convention, and compilation-worker-pool-startup.test.ts covers it.

🤖 Generated with Claude Code

…tory

Every distinct compile left its build directory behind: the compile worker's
build-output/<sketchHash> was never evicted, and main-thread compiles without
--build-path made arduino-cli keep one directory per random sketch path. The
periodic sweeper scanned process.cwd()/temp, so finished sandbox runs renamed
to <sharedTemp>/unosim-temp/<id>.cleanup were never removed either.

- ArduinoCompiler uses one build directory per compile (the caller's or its
  own sibling of the sketch directory) and removes it after the compile,
  also on failure and cache hits; the HEX stays in the caches.
- sweepTempArtifacts removes old .cleanup items in the legacy and the shared
  sandbox temp directory and stale worker build-output entries.
- SandboxRunner, compile worker and sweeper share one path definition.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ttbombadil
ttbombadil merged commit 7a009ed into main Oct 9, 2026
6 checks passed
@ttbombadil
ttbombadil deleted the fix/temp-artifact-growth branch October 9, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant