Repository navigation
feat(daemon): reclaim private-seat scratch and merged-PR worktrees via a leased idle pass - #14776
loom-fleet-dispatch[bot] wants to merge 2 commits into
Conversation
…a a leased idle pass Add the periodic private-workspace seat pass (#14678): a leased maintenance verb (account lease, container proven running and idle, no prepare step) that removes ended step scratch (cache/<role>-<n>.* and top-level *-tmp / <role>-<n>.* dirs past scratchReclaim.maxAgeHours) and reaps in-volume linked worktrees only when clean, fully published, unlocked, and confirmed merged at that exact tip past the reaper grace. Dirty, unpublished, open-PR, closed-unmerged and unreadable worktrees are preserved with reasons through reaper.pass spans (new class 'seat'); the host reaper's private-workspace ownership skip is test-pinned. Scheduled from eager-reclaim tier 1; reuses scratchReclaim.* and gracePeriodSecs (no new flags). Ownership split vs preparation-time reaping (#14483) recorded in defaults/docs/private-session-dispatch.md. Closes #14678
|
PR #14776's |
|
Judge: code review of |
|
✅ Approved! Reviewed at Spec reconciliationRe-read #14678's current body (the 2026-11-19 and 2026-11-20
What I verifiedDestructive-path audit (the thing that matters here). Removal is gated, in order, by: canonical-path identity against the clone root (the main checkout is decremented out of
Preserve wordings land in real telemetry categories. Traced each new wording through Scratch shape gates. Walked Entry point matches the AC. Test quality. TDD claim: the line reads Test ExecutionPR test plan (scoped
Live verification: unverified, and that is explicitThe PR states plainly that the issue's manual step could not be performed, and no fixture here claims live behavior — the honest path. I checked the two triggers in the Judge "Live Verification and the Circular-Fixture Smell" rule and neither fires: the diff drives no browser, scraper or foreign DOM/HTML parser (trigger A), and #14678's body reports unbounded disk growth, not a silent-omission failure — it contains none of the silent-omission fragments (trigger B). So the blocking evidence requirement does not attach, and I am not stamping No circular fixture: the one place a fabricated payload could hide is the PR-state side, and it is injected through the The live gap is real and worth naming: this pass's first production run will be its first contact with a 40 GB, 72-clone seat volume, and the in-container verb only exists in session images built from this code — existing seat containers will answer the protocol error and be skipped (handled: it maps to a per-seat skip reason, not a pass failure), so the pass no-ops on the fleet until containers are recycled. The PR's own plan — the first below-floor pass post-merge produces the Non-blocking: filed #14777The seat sub-pass runs synchronously inside the dispatch loop's tick ( This is not a defect in this PR: #14678's curated "Affected Files" names Nit (not worth a round-trip)
Worktree noteThe builder worktree Clean, careful work on a genuinely dangerous surface — the gate ordering, the merged-only classifier's parity with its sibling, and the fixture that makes cleanliness the sole discriminator are all exactly right. |
|
PR #14776's |
|
Champion: Holding for Human Merge Judge-approved and passing the mechanical safety criteria (CI green, mergeable, fresh verdict), but not auto-merging:
Next steps — this hold stays in force until one of these happens; Champion re-reading the same diff will not clear it:
Keeping Automated by Champion role |
Summary
Add the periodic private-workspace seat pass (remaining Builder increment of #14678; the seat-volume telemetry AC landed earlier via #14693): a leased, idle-only maintenance verb that reclaims inside
loom-codex-workspace-<seat>volumes — ended step scratch and clean, published, merged-past-grace linked worktrees — which no host-side pass could reach.Changes
private-workspace seat-reclaimin-volume verb (private_workspace/seat_reclaim.rs): removescache/<role>-<n>.*and top-level*-tmp/<role>-<n>.*step dirs pastscratchReclaim.maxAgeHours(future-dated mtimes never reclaimable), and reaps a linked worktree only when clean, fully published, unlocked, in-volume, non-detached, and its branch has a confirmed merged PR at that exact tip pastgracePeriodSecs. No quarantine, no backup, plaingit worktree remove; every other worktree is preserved with a reason.lifecycle.rs+docker.rs): enumerates seats, takes each account's exclusiveLease::acquire, requires the container running and idle (docker::idle), and execs the verb — mirroringrun_job's admission but with noprepare_target, so a dirty seat still gets scratch reclaim and preserved-dirty reporting. Busy/stopped/non-idle seats are skipped with a reason, never failing the pass.PrAtHead/pr_at_head(worktree_ops/clean_owner.rs): same request and row strictness asfinished_pr_at_head, but distinguishing open / closed-unmerged / merged-at-tip (+merged_atfor the grace gate) — the seat pass preserves what preparation-time force-removal (fix: let recover list and clear dirty leftover worktrees that block a private-clone seat #14483) may remove.eager_reclaim.rs) with a host-wide cooldown; honors the existingscratchReclaim.*surface andgracePeriodSecs— no new flags.reaper.passspans (classseat, seat account name as the repo label, preserved reasons and bytes);reaper.passschema row indefaults/docs/telemetry-schema.mdgains theseatclass. Ownership split vs preparation (fix: let recover list and clear dirty leftover worktrees that block a private-clone seat #14483) recorded indefaults/docs/private-session-dispatch.md("Who reaps what: preparation vs the periodic seat pass").worktree_reaperkeeps an unrelated hostissue-Ncheckout a private job owns.Acceptance Criteria Verification
maxAgeHours; live steps never touchedseat_reclaim::tests::scratch_reclaim_removes_ended_steps_and_keeps_live_and_future_dated_ones(ended vs live vs future-dated vs non-step vs reserved roots); liveness is structural — the verb runs only under the seat's account lease with the container proven idlereap_merged_worktreesgates in order (locked / outside volume / detached / unpublished / dirty / open / closed-unmerged / grace / unreadable); removal is plaingit worktree remove— noRemoveWithQuarantine/RemoveWithBackuproute exists in the moduleonly_the_eligible_checkout_is_removed; negative check: disabling only the cleanliness gate makes it fail (dirty sibling removed), restored to greenworktree_reaper::tests::test_private_workspace_owned_host_worktree_is_never_reaped;private_route.rsunchangedprepare_target; dirty seat still reclaims scratchlifecycle.rs::seat_reclaim(acquire → inspect → validate → lease.recover → running+idle);tests::seat_reclaim_exec_drives_the_verb_without_any_preparationasserts argv contains noprepareand no token leak;a_dirty_seat_still_reclaims_its_scratchdefaults/docs/private-session-dispatch.md; preparation behavior documented separatelyreaper.passreclaimed bytes + preserved reasonsobservability::ops::reaper::tests::a_seat_pass_span_names_the_seat_and_its_class(classseat, seat name label, reason categories, no host path); schema row updatedloom.host.codex_workspace_volume.*); not re-done herescratchReclaim.*config; no new flagsrun_forreadsread_scratch_reclaim_config+worktree_reapergrace; the verb's flags are host→container plumbing params, not operator configTest Plan
Scoped runs in the issue worktree, all green:
cargo test -p loom-daemon --lib -- seat_reclaim eager_reclaim observability::ops::reaper private_workspace_owned_host_worktree(45 passed), CI's exact clippy line forloom-daemon+loom-api--all-targets(clean after oneobfuscated_if_elsefix),cargo fmt --check,cargo check --workspace, all re-verified after merging currentorigin/main. Full suite is CI's gate.TDD: partial — module and tests originate from a prior interrupted attempt in this same worktree (recovered, reviewed, completed: docs, schema row, clippy fix); the key fixture's fail-first was verified here by disabling the cleanliness gate alone and confirming
only_the_eligible_checkout_is_removedfails because the dirty sibling is removed.Live verification NOT performed. The issue Test Plan's manual step (run the pass on a worker with a real seat volume and confirm bytes freed +
reaper.passreasons) requires a live seat container, unavailable in this worktree; no offline fixture claims live behavior. The pass is scheduled from eager-reclaim tier 1 on every fleet worker, so the first below-floor (or 15m-tick) pass post-merge produces the livereaper.pass/seatspans and log lines to confirm.Closes #14678
loom dashboard