Phase B follow-up: adversarial probes, doc repair, trust scoping

docs/ISOLATION_MODEL.md

@@ -19,6 +19,8 @@ marked enforced; everything else is an explicit gap with an owner.
   cross-lab trust; the OSS verification contract is
   `or_audit.eval.attestation`.
 
+## What is enforced
+
 1. **Oracle routing**: labels travel only to the verifier context, never in
    agent payloads (`runner.py`, `plugins.py`). A protocol-conformant agent
    cannot receive labels through the harness API.
@@ -30,6 +32,11 @@ marked enforced; everything else is an explicit gap with an owner.
    stops accidental inheritance only: same-UID code can still read parent
    state through OS channels (e.g. `/proc/$PPID/environ` on Linux), so it
    is not a credential boundary against hostile code. HOME points at a
+   fresh empty directory removed on close.
+4. **Timeouts and cleanup**: bounded requests, kill on expiry, pipe cleanup.
+   Evidence transfer is one bounded value per request with a deadline
+   covering the whole read, so dribbled output cannot bypass the timeout.
+
 ## Explicitly not enforced on the local path (gaps, not bugs)
 
 These hold for `local` subprocess execution (T0). The `container` backend
@@ -44,10 +51,18 @@ forgery and task-author rows hold everywhere until B5/governance land.
   evaluate untrusted code that must not reach the network outside it.
 - **Resources**: no CPU/RAM/GPU limits on local plugin children. The
   container backend applies memory/CPU/pids caps.
-- **Result forgery**: job heads are unkeyed digests; local files are
-  re-stampable by anyone holding them. Holds on every backend until B5
-  hosted attestation lands.
 - **Task-author trust**: sandboxing an agent never validates a dishonest task
+  verifier. Benchmark tasks need review/governance, not just isolation.
+- **Semantic output policing**: transfer is bounded by size, deadline, and
+  protocol shape, but prediction payload keys are not validated against
+  interface output slugs (slug spelling differs from payload keys today).
+  Semantic output contracts belong to workstream E; enforcing them here
+  would be arbitrary.
+- **B5 execution**: the OSS attestation contract (`or_audit.eval.attestation`)
+  is implemented and tested, but no executor mints or stores stamps yet —
+  minting lives in the private cloud tree, which has no attestation
+  endpoint. The operator secret must never be distributed: verification
+  happens at hosted ingestion, not in labs. Blocked on the cloud owner.
 
 ## Acceptance for B (reminder)
 

src/or_audit/eval/attestation.py

@@ -4,8 +4,11 @@
 trust needs the executor to stamp what it observed with a key the submitter
 does not hold. This module defines the stamp and its verification; minting
 happens in the hosted executor (private cloud tree), which holds the
-operator secret. Nothing here mints without a secret, and nothing verifies
-without the same one.
+operator secret. The secret is never distributed: verification happens at
+hosted ingestion, never in evaluating labs — anyone holding the secret could
+mint, so HMAC here is an operator-held seal, not a public verification key.
+An asymmetric upgrade is the documented path if verification ever needs to
+leave the operator boundary.
 """
 
 from __future__ import annotations

tests/test_eval_plugin_host.py

@@ -543,3 +543,66 @@ def refuses(extra: dict[str, object], pattern: str) -> None:
     refuses({"container_pids_limit": "-1"}, r"outside 16\.\.4096")
     refuses({"container_memory": "999g"}, r"outside 64m\.\.16g")
     refuses({"container_cpus": "0"}, r"outside 0\.1\.\.16")
+
+
+_ADVERSARIAL_SPY = """
+import os
+import socket
+from pathlib import Path
+from typing import Any
+
+class Predictor:
+    def predict(self, item: dict[str, Any]) -> dict[str, Any]:
+        del item
+        report: dict[str, Any] = {"uid": os.getuid(), "pkg": sorted(os.listdir("/pkg"))}
+        try:
+            open("/pkg/_probe_write", "w").write("x")
+            report["pkg_writable"] = True
+        except OSError:
+            report["pkg_writable"] = False
+        try:
+            socket.create_connection(("8.8.8.8", 53), timeout=3).close()
+            report["egress"] = True
+        except OSError:
+            report["egress"] = False
+        return report
+
+def load_predictor(*, root: Path, weights_path: Path) -> Predictor:
+    del root, weights_path
+    return Predictor()
+"""
+
+
+def test_container_adversarial_probes_fail(tmp_path: Path) -> None:
+    """Untrusted-code probes against the container backend: non-root UID, a
+    read-only package mount with no sibling leakage, and no egress. Needs
+    docker and SURGEVAL_TEST_PLUGIN_IMAGE=image@digest; skips otherwise."""
+    import shutil
+
+    from or_audit.eval.contracts import RuntimeDescriptor, RuntimeKind
+
+    image = os.environ.get("SURGEVAL_TEST_PLUGIN_IMAGE", "")
+    if not image or shutil.which("docker") is None:
+        pytest.skip("needs docker and SURGEVAL_TEST_PLUGIN_IMAGE=image@digest")
+    ref, _, digest = image.partition("@")
+    assert digest, "SURGEVAL_TEST_PLUGIN_IMAGE must be digest-pinned"
+    plugin = tmp_path / "plugin"
+    plugin.mkdir()
+    (plugin / "spy.py").write_text(_ADVERSARIAL_SPY, encoding="utf-8")
+    (plugin / "weights.json").write_text("{}", encoding="utf-8")
+    (tmp_path / "canary.txt").write_text("oracle-labels", encoding="utf-8")
+    runtime = load_predictor_runtime(
+        plugin,
+        "spy.py:load_predictor",
+        "weights.json",
+        runtime=RuntimeDescriptor(kind=RuntimeKind.CONTAINER, image=ref, image_digest=digest),
+    )
+    assert isinstance(runtime, SubprocessPredictorRuntime)
+    try:
+        report = runtime.predict({})
+    finally:
+        runtime.close()
+    assert report["uid"] == 65534
+    assert report["pkg_writable"] is False
+    assert report["egress"] is False
+    assert "canary.txt" not in report["pkg"]