Skip to content

B5: Sandbox lifecycle — generation CAS, exec lease, Draining and first-class Lost #548

Closed as not planned

Description

@mengmengmengqiang

Parent: #545 (index). This issue owns the B5 Sandbox lifecycle slice. It is not the daemon control-slot work in #547.

Summary

#545 B5 is 0/10 done and 2/10 partial. Today sandbox.close is an op-intent that stops and deletes the Sandbox with no Draining state, no CAS on a generation and no exec lease. Three failure modes follow:

  • a reclaimer that decided "idle" can kill an exec between the decision and the kill, losing uncommitted Workspace data;
  • a crashed exec leaves busy set with no lease deadline, so "never kill while an exec is active" becomes "never reclaim", leaking capacity and cost;
  • a provider-side disappearance leaves the record Ready/Busy forever, and the stale base revision can then be used by a snapshot push.

The fix is a small, explicit protocol, not a new subsystem.

Scope (from #545 B5, plus the state enum in B0)

  • Draining is entered by CAS on generation.
  • an active exec holds an exec lease deadline.
  • close sequence: mark Draining → check lease/generation → close or abort → record Closed / Lost.
  • no kill while a lease is live; lease expiry moves to Lost / Unresolved, never a blind kill.
  • close is idempotent; a stale generation aborts.
  • failures are Unresolved; no unbounded retry.
  • Lost is a first-class state.
  • last activity uses a monotonic clock.
  • reclamation has reclaim.enabled, --dry-run and a freeze / kill switch, and records the reason for every close.
  • reclamation is keyed by machine / backend, never by isolation class.
  • Creating / Provisioning / Starting / Ready / Busy / Draining / Closed / Lost / Failed / Unresolved are explicit (from [B] Sandbox compute plane: machines, backends, observability and lifecycle #545 B0-140).

Protocol

Ready --CAS(generation)--> Draining --active exec/mount == 0, or lease expiry--> Closed
                                   \--lease expiry / provider gone--> Lost
  • The Sandbox record carries generation, active_exec_leases[], lease_deadline, last_activity_monotonic and close_reason.
  • close(generation) is idempotent: a second close with the same generation is a no-op; a stale generation is refused, not retried.
  • A live exec lease blocks the close. Lease expiry is an explicit Lost / Unresolved transition; never a blind kill.
  • Every close records its reason and the evidence that decided it (idle deadline, lease expiry, operator, provider gone).
  • Reclamation runs under reclaim.enabled; --dry-run and --freeze are mandatory operational brakes.

Non-goals

Acceptance

  • A concurrent exec started between the reclamation decision and the kill is never killed.
  • Lost is reachable and observable from sandbox get; a lease expiry produces Lost / Unresolved, not a retry.
  • A stale-generation close aborts without side effects.
  • Close is idempotent under replay.
  • reclaim.enabled=false, --dry-run and --freeze each stop the destructive path.
  • State-machine tests cover every transition; one fault-injection test covers a panic or exit during close.

Dependencies

  • No D2 / D6 / D12 dependency; can start now.
  • The B6 issue depends on the generation and Lost / Unresolved semantics defined here.
  • The explicit state enum also unblocks the B0/B8 read models.

Related

Residuals consolidated from #422 and #423

  • Explicitly cover an abandoned/orphaned mounted Workspace that retains a Sandbox after its application process is gone.
  • Lease expiry or caller death must not authorize discarding uncommitted Workspace state; preserve exact ownership and uncertainty.
  • Cover stopped-container lifecycle and cleanup after known owner release, not only a standalone reclamation helper.

The older reports #422 and #423 are being consolidated here/#549. This is not a claim that this issue's branch has been integrated or qualified on the target.

Activity

  1. mengmengmengqiang commented on Nov 4, 2026

    @mengmengmengqiang
    ContributorAuthor

    Maintainer: this sub-issue was created without labels because the available token has no triage permission. Requested labels: v2.0.0, bug. Please add them (and adjust if the repo's triage rules disagree).

  2. mengmengmengqiang commented on Nov 4, 2026

    @mengmengmengqiang
    ContributorAuthor

    All #548 checkboxes are now covered by the B5 branches and tests.

    Evidence:

    • agent-b5 / agent-b548 / agent-b548b.
    • control sandbox_lifecycle 8/8, api sandbox_lifecycle 9/9, sandbox_records 3/3, typed_records 7/7, home_layout_3 3/3.
    • Explicit SandboxState enum: Creating, Provisioning, Starting, Ready, Busy, Draining, Closed, Lost, Failed, Unresolved; one facts→state mapping; get/list serialization; old/absent records migrate.

    Qualification (not a checkbox): the Docker E2E was not run because the verification sandbox has no Docker socket/CLI. sandbox_family fails at project.init on overlayfs (SQLite lock), before any B5 path. failed has no live control producer today; it is reachable from an explicit state: "failed" record.

  3. mengmengmengqiang commented on Nov 5, 2026

    @mengmengmengqiang
    ContributorAuthor

    All checkboxes are covered by the verified branches/PRs; evidence is recorded in the issue evidence comments and the subagent reports.

  4. mengmengmengqiang commented on Nov 5, 2026

    @mengmengmengqiang
    ContributorAuthor

    Reopened: the delivery gate is code on the target branch + PR merged + CI green + traceable evidence. This issue was closed on branch/test evidence only; its PR/branch is not merged to layerfs_v2_0_0 yet.

  5. added this to the LayerFS v2.0.0 milestone on Nov 6, 2026
  6. added theissue type on Nov 6, 2026
  7. 19 remaining items

  8. added a commit that references this issue on Nov 7, 2026
  9. yifanxuaaa commented on Nov 7, 2026

    @yifanxuaaa
    Contributor

    Closed by the owner (2026-11-07). PR #566 is shown as merged into layerfs_v2_0_0, but it was recorded as history only (a git merge -s ours in #601): mengmengmengqiang's commits and credit are kept, but none of its code is on the branch. The control layer split (layerfs-project#101) replaced this area, and the port was cancelled, so this is not delivered. The exec-lease and cancel part continues in Ephemeral-AI-Lab/layerfs-project#113.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

new featureA new user-facing capability

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions