Run unmodified task images: guest injected as an OCI image volume, plus second-runtime layers

[Tomer Glottmann (tomergee)]

What

Run any digest-pinned OCI image as an ate-env environment without rebuilding it. The task image stays as published; the ate-env-guest is mounted into it as a read-only OCI image volume at /ate and started from there. Two commits:

1. Task images (3f943a8)

• CreateEnvironment gains an image field. ate-env-api takes the request's template (default default-template) as the base, derives a template named <base>-<12 hex of the digest> next to it on first use, reuses it afterwards, and reports it in the response. Concurrent first creates on the same image both succeed.
• ate-env manifest template --task-image writes such a template up front; --workspace passes -workspace to the guest.
• CLI ate-env create --image, Python client.create(image=).
• Guard rails: unpinned image, derived name over 63 characters, or a base command that is not the guest binary are InvalidArgument before anything is created; missing base or a derived name held by a template running another image are FailedPrecondition.
• The derivation lives in internal/apiservice/imagetemplate.go and accepts either kind of base (guest as image, or already injected).

2. A second runtime as a layer (a76974d)

• ate-env-guest gains -sidecar (start and supervise an extra runtime from a layer: no shell, output logged with a prefix, restart with backoff 1s→30s) and -sidecar-readyz (URLs the guest's /readyz, Substrate's wakeup probe, waits for once).
• ate-env manifest template gains --layer name=image@sha256:...=/mount, --sidecar, --sidecar-readyz. Layers survive derivation, so create --image on a layered base inherits the second runtime.

This is step 1 of the runtime-injection plan: the mechanism every injected runtime needs. The intended first foreign runtime is OpenSandbox's execd, which is built to be injected this way.

Docs

• docs/task-images/README.md: how-to, both flows, what an image needs, errors, cleanup.
• docs/task-images/RUNTIMES.md: second runtime as a layer, OpenSandbox specifics, reachability, limits.
• docs/task-images/DESIGN.md: derivation rules, naming and idempotency, failure semantics, security, alternatives, future work.
• examples/runtime-layers: runnable demo with a static busybox as the stand-in runtime.

Testing

• go test ./...: 11 packages, including derivation from both base kinds, naming limits, every rejection, server create/reuse/race/failure paths against the fake control plane, manifest layers and sidecars, and the guest's sidecar supervisor (restart with backoff, stop on shutdown, readiness gate once per URL).
• clients/python unit tests: 59 pass. tests/e2e/test_full_stack.py gains two cheap negative tests and a create-from-image test gated on ATE_ENV_TASK_IMAGE that checks the guest comes from the volume and not the image, the rootfs is the image's, files round-trip, and a second environment reuses the derived template.
• On a Kubernetes cluster running Substrate, fresh two-worker ate-env built from this branch:
  • ate-env create --image of an unmodified python:3.12-slim: created in 1 s, derived template present, python3 --version answered from the image's own rootfs with the guest under /ate, deleted; 44 s end to end on a cold worker.
  • examples/runtime-layers/demo.sh: layered template, environment serving in 3 s, the second runtime answered from inside, then a debian:bookworm-slim environment created on the same base with --image did the same; 23 s total.
  • A first attempt with the dynamically linked busybox:1.37 failed on the debian image with a glibc version error, exactly as the supervisor and fail-fast readiness are meant to surface it. The docs now state the self-contained-layer rule and that signature.
• hack/verify/boilerplate.sh passes. Go stubs via buf generate with protoc-gen-go v1.36.11; Python stubs via grpcio-tools 1.61.3, matching the committed generator and keeping the license headers.

Known gaps, by design

• The task image is not checked for existence at create time; a bad reference shows as the actor failing to start.
• Derived templates and their golden snapshots are not garbage collected.
• A second runtime is reachable from processes inside the environment now. Outside clients reach only port 80: the router forwards there unless a CONNECT authority names another port, and ate-env-api proxies only the guest. An endpoint lookup that names a runtime's port is the next piece and belongs with the OpenSandbox backend.
• Pre-existing and unchanged: CreateEnvironment resolves the template in the environment's atespace and ignores template.atespace.

Relation to other PRs

• Independent of integrations/nemo-gym: NeMo Gym sandbox provider built on the Python client #69 (NeMo Gym provider); they merge cleanly in either order. Once both land, the provider can pass spec.image straight through instead of its image_templates map.
• Overlaps Redesign guest ProcessService and FileSystemService #65 (regenerates the same Python stubs, edits fakes.py and test_full_stack.py) and Enable settings to use warm cache #67 (cmd/ate-env/manifest.go). If those land first I will rebase; the conflicts are mechanical.