From 3f633ad35fd15725c78cf985999faa9ae51fc2ff Mon Sep 17 00:00:00 2001 From: Francesco Novy Date: Thu, 9 Jul 2026 10:58:58 +0200 Subject: [PATCH 1/3] test(node-integration-tests): Free up CI disk space to prevent ENOSPC failures The full Node integration suite pulls several database docker images (mssql alone is ~1.5GB) and installs per-suite node_modules concurrently. On the default ubuntu-24.04 runner (~14GB free) this peak can exhaust the disk and fail unrelated suites with `ENOSPC: no space left on device`. Because the flaky-test detector files one issue per failing test, a single disk-full run spawns a batch of unrelated "flaky" issues that are really infra noise. Reclaim the large preinstalled toolchains this job never uses (Android SDK, .NET, GHC/ghcup, CodeQL) right after checkout to give the run ~20GB of headroom, and bracket it with `df` so a future squeeze is visible in the logs. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/build.yml | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index f3aa097087de..d34d57362315 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -810,6 +810,16 @@ jobs: uses: actions/checkout@v7 with: ref: ${{ env.HEAD_COMMIT }} + - name: Free up disk space + # The full suite pulls several DB docker images (mssql alone is ~1.5GB) and installs + # per-suite node_modules concurrently, which can exhaust the ~14GB free on the default + # runner and fail unrelated suites with ENOSPC. Reclaim the large preinstalled toolchains + # this job never uses to give the run headroom. `df` brackets it so a future squeeze is + # visible in the logs. + run: | + df -h / + sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc /usr/local/.ghcup /opt/hostedtoolcache/CodeQL + df -h / - name: Set up Node uses: actions/setup-node@v6 with: From 155e00ea9a8739986f95f1b30d5163f55c33c318 Mon Sep 17 00:00:00 2001 From: Francesco Novy Date: Thu, 9 Jul 2026 12:27:02 +0200 Subject: [PATCH 2/3] Only free disk space when the runner is actually low MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Most hosted runners now report ~90GB free, where the integration suite's docker images and per-suite installs fit comfortably, so the unconditional ~30-60s toolchain cleanup was pure overhead. Gate it behind a `df` check so it only runs when free space drops below 40GB — protecting the constrained runners that actually hit ENOSPC while keeping it a no-op on healthy ones. Co-Authored-By: Claude Opus 4.8 (1M context) --- .github/workflows/build.yml | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index d34d57362315..c8b088486da4 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -810,16 +810,23 @@ jobs: uses: actions/checkout@v7 with: ref: ${{ env.HEAD_COMMIT }} - - name: Free up disk space + - name: Free up disk space if low # The full suite pulls several DB docker images (mssql alone is ~1.5GB) and installs - # per-suite node_modules concurrently, which can exhaust the ~14GB free on the default - # runner and fail unrelated suites with ENOSPC. Reclaim the large preinstalled toolchains - # this job never uses to give the run headroom. `df` brackets it so a future squeeze is - # visible in the logs. + # per-suite node_modules concurrently. Most hosted runners now have ~90GB free, where that + # fits comfortably, but part of the fleet still ships with far less free space and those + # runners fail unrelated suites with ENOSPC. Only reclaim the large preinstalled toolchains + # (a slow ~30-60s `rm` of the Android SDK etc.) when free space is actually low, so healthy + # runners skip it and pay nothing. `df` is logged either way so a future squeeze is visible. run: | df -h / - sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc /usr/local/.ghcup /opt/hostedtoolcache/CodeQL - df -h / + avail_kb=$(df -k --output=avail / | tail -1) + if [ "$avail_kb" -lt $((40 * 1024 * 1024)) ]; then + echo "Low disk space (<40GB free) — reclaiming unused toolchains" + sudo rm -rf /usr/local/lib/android /usr/share/dotnet /opt/ghc /usr/local/.ghcup /opt/hostedtoolcache/CodeQL + df -h / + else + echo "Sufficient disk space — skipping cleanup" + fi - name: Set up Node uses: actions/setup-node@v6 with: From 9a6d187319f9b39be8eec063500c40e41fba00b0 Mon Sep 17 00:00:00 2001 From: Francesco Novy Date: Thu, 9 Jul 2026 12:35:20 +0200 Subject: [PATCH 3/3] better comment --- .github/workflows/build.yml | 8 ++------ 1 file changed, 2 insertions(+), 6 deletions(-) diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index c8b088486da4..7086341742f9 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -811,12 +811,8 @@ jobs: with: ref: ${{ env.HEAD_COMMIT }} - name: Free up disk space if low - # The full suite pulls several DB docker images (mssql alone is ~1.5GB) and installs - # per-suite node_modules concurrently. Most hosted runners now have ~90GB free, where that - # fits comfortably, but part of the fleet still ships with far less free space and those - # runners fail unrelated suites with ENOSPC. Only reclaim the large preinstalled toolchains - # (a slow ~30-60s `rm` of the Android SDK etc.) when free space is actually low, so healthy - # runners skip it and pay nothing. `df` is logged either way so a future squeeze is visible. + # The full suite pulls several DB docker images (mssql alone is ~1.5GB) + # Available disk space is not consistent, if we detect low space this cleans up some unused toolchains run: | df -h / avail_kb=$(df -k --output=avail / | tail -1)