Skip to content

Plan / Interface: per-session Kata VM isolation for Kubernetes execution - #2052

Open
tgrunnagle wants to merge 4 commits into
mainfrom
plan/kubernetes-session-vm-isolation
Open

tgrunnagle wants to merge 4 commits into
mainfrom
plan/kubernetes-session-vm-isolation

Conversation

@tgrunnagle

Copy link
Copy Markdown
Contributor

Stage: Plan / Interface

This PR adds the acceptance plan docs/acceptance/kubernetes-session-vm-isolation.md and ADR 0373. It follows the native Kubernetes execution plan and ADR 0364. Merging this PR approves the contract. It contains no implementation.

Relates to the native Kubernetes execution work (#1579, #1614).

Plan summary

Work classification: Architectural. It changes the execution isolation boundary from a shared node kernel to one VM per environment. It also changes the retained-environment lifecycle and removes every path for executing commands on the mecak8s host. The delivery path is Split.

The plan has five scenarios:

  1. A Kata VM per environment. The qualified handler is kata-clh. Each environment's executor gets a distinct guest kernel and boot_id. The controller verifies runtimeClassName and spec.overhead, and fails closed when the sandbox handler is unavailable.
  2. No execution in the mecak8s Pod. With --execution-enabled:
    • mecak8s builds no host CommandRunner on any placement path.
    • An explicitly set --shell is rejected at startup.
    • Agent definitions that carry hooks: are refused at the single hook-runner construction point. This also covers any session-scoped definitions added later.
  3. Last-reference stop. When an environment's last reference is removed, the controller retires it through the existing lifecycle (type: RetireEnvironment). The VM stops and the PVC is kept. The stop is admitted from not-Ready, never-provisioned and post-operation states, while FenceUnknown and active run claims are still respected. The reference RPCs keep their existing classifications.
  4. The parent plan's five pending journeys, run under Kata: TUI OIDC coding, Alice/Bob isolation, reconnect and restart, cancellation and authority loss, and capacity retirement.
  5. CI lanes. A new Kind+Kata lane task e2e:k8s:execution:kata and CI job execution-kata-e2e run on ubuntu-24.04 with KVM and fail closed without /dev/kvm. The existing runc lanes remain the macOS path.

Unchanged contracts: the ExecutionProviderService protobuf, the engine and Harness APIs, tool schemas, the profile schema, chart values and the CRD schema.

Out of scope: GitHub/Git credentials, the coding workload image, hostname egress, warm pools and resumable stopped environments.

Human decisions recorded

All decisions are checked in the plan:

  • A true VM per session.
  • A follow-up plan rather than an amendment.
  • Kind+Kata on Linux/KVM, with macOS covered by runc.
  • No API changes.
  • The VM stops on session delete.
  • Hook-bearing definitions are refused.
  • All five journeys are completed.
  • The listed scope exclusions.
  • kata-clh as the hypervisor.
  • The stop reuses Retired.
  • An explicit --shell is rejected.

Review notes

  • Cost of reusing Retired. Every deleted session keeps a capacity slot and a PVC until an operator calls DeleteRetiredEnvironment.
  • ADR 0364 supersession. ADR 0364 gains a partial "Superseded by" pointer for its §6 last-reference retention rule.
  • Advisory adversarial review. One pass ran, and its findings are folded in:
    • non-Ready stop admission;
    • CRD enum reuse;
    • reference replay during a stop;
    • capacity journey semantics;
    • a structural, not sampled, host-execution proof;
    • Calico enforcement in the Kata lane;
    • session-scoped agent definitions;
    • the complete §6 supersession.

Checks

  • check-acceptance-plan.sh: passed.
  • check-acceptance-plan-test.sh: passed.
  • task docs: passed (0 broken links, 0 unreachable).

🤖 Generated with Claude Code

tgrunnagle and others added 4 commits October 1, 2026 14:50
Add the Plan / Interface contract and ADR 0373 for the follow-up to native
Kubernetes execution: one Kata (kata-clh) VM per execution environment, no
host command execution in the mecak8s Pod while execution is enabled, VM stop
with retained PVC when the last reference is removed, the parent plan's five
pending journeys under Kata, and a Kind+Kata KVM CI lane. The provider gRPC,
engine, Harness, and tool contracts are unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant