feat(runtime): support isolated multi-judge execution in Docker and OpenSandbox #308

Context and motivation

Follow-up to #246 and the independent multi-judge implementation merged in #290.

The initial implementation runs named judges over one evaluated-Agent execution, with independent verdicts and workspace copies. Its optional JudgeSnapshotProvider / JudgeSnapshot capability is currently implemented only by NoneRuntime. Multi-judge evaluations in Docker and OpenSandbox therefore fail with an unsupported-isolation error before the evaluated Agent starts.

This limitation applies to multi-judge snapshot/fork support, not to all single-judge execution. We should extend the capability to Docker and OpenSandbox while preserving the independence established by #290.

Required environment and isolation contract

Judges must run in an environment consistent with the evaluated Agent, while each judge receives an independent isolated copy of the completed evaluation state. Environment consistency does not mean sharing the Agent's mutable workspace or one container/sandbox among all judges.

• Preserve the runtime type and the applicable image/template, OS, toolchain, runtime environment configuration, and network/egress policy. Avoid silently downloading an isolated Agent workspace to the host and running judges under none.
• All members observe the same frozen post-Agent workspace, transcript, generated outputs, and execution metadata. The evaluated Agent must not be rerun to produce inputs for each judge.
• Create a separate writable runtime/workspace for each judge. A judge's file changes must not modify the frozen source, original Agent workspace, or any sibling judge. If containers/sandboxes share physical storage, it must satisfy this isolation contract rather than exposing a shared writable mount.
• Keep Agent-judge model/provider credentials and judge-only Skills governed by their existing explicit configuration. Environment consistency does not require using the same model or copying all Agent credentials into every judge.
• Explicitly define the handling of dependencies installed during setup or Agent execution, including files outside the workspace. Starting another instance of the same base image alone is insufficient if required tools or dependencies were added later. Use an appropriate snapshot mechanism or a documented, reproducible environment reconstruction; do not rerun the evaluated Agent or silently replay arbitrary side-effecting setup commands.
• Define the boundary for running processes, local services, mounted volumes, and external service state. If relevant state cannot be captured or reconstructed consistently, report a clear unsupported/error result instead of silently scoring a different environment. Do not claim network or external-service isolation merely because workspace files are copied.

Implementation direction

Implement the existing optional JudgeSnapshotProvider / JudgeSnapshot contract in runtime adapters. Keep evaluator dispatch capability-based rather than adding Docker/OpenSandbox concrete-type switches.

CaptureJudgeSnapshot must freeze the relevant completed state once. Fork must return a ready-to-use isolated runtime and a cleanup function. Choose and document backend-specific capture/reconstruction mechanisms after verifying what each backend actually supports; this issue does not assume either backend exposes a native snapshot API.

Retain member input-path remapping, per-judge material/artifact namespaces, independent timeouts, all_required aggregation, and explicit ERROR/SKIP outcomes on capture/fork failures. A member failure must not prevent subsequent members from running unless the case is cancelled or its deadline is exhausted.

Acceptance criteria

• Docker and OpenSandbox both implement multi-judge isolation through the existing runtime capability contract; none and legacy singular-judge behavior remain compatible.
• Each judge uses the evaluated Agent's runtime type and applicable image/template, toolchain, environment, and egress policy; there is no implicit host-runtime fallback.
• One Agent execution produces a single frozen input state, and every judge receives an independent writable runtime/workspace derived from it.
• A script judge's mutation is invisible to the next semantic judge, the original Agent workspace, and the frozen snapshot, including relevant attachment/generated-file references.
• Runtime-installed dependencies and required state outside the workspace are preserved or reconstructed according to a documented contract; unavailable state is rejected explicitly. Verification goes beyond launching the same base image.
• Capture/fork/setup failures, timeout, cancellation, cleanup, and partial execution preserve ordered per-member diagnostics and ERROR/SKIP artifacts without leaking temporary resources.
• Existing gate semantics, independent verdicts, aggregation policy, and namespaced reports/artifacts remain unchanged.
• Real Docker and OpenSandbox validation covers a positive script-plus-semantic run and a controlled script FAIL / semantic PASS / aggregate FAIL run, with mutation-isolation and artifact readback evidence. Mock/unit tests alone do not establish backend support.
• Regression coverage verifies environment/toolchain/egress consistency and a dependency created during setup or Agent execution, alongside runtime lifecycle/error paths.
• English/Chinese documentation states supported runtime versions/capabilities, reproduction steps, resource costs, and process/volume/external-state limitations.

Out of scope

Weighted/numerical scoring, dependent judge pipelines, durable replay/regrading, and universal isolation of external services remain separate extensions. This issue extends runtime coverage without changing the independent multi-judge semantics delivered by #246 / #290.