Repository navigation
Verify laboratory observations and executable-adapter gates - #257
Conversation
Require authenticated VM or container isolation, canary, and reset observations before verification. Derive case outcomes from signed adapter observations and require completed worker authorization, isolation, scope, and egress gates for executable adapters. Journal dispatch intent and correlate unknown attempts with worker audit request IDs. Checks: focused laboratory and skill tests passed; make check passed; staged diff check and gitleaks staged scan passed. Recovery: revert this commit to restore prior harness behavior. Operator-owned laboratory state and journal records require separate review and cleanup; no live operator receipt was available for this offline validation. Refs #190
…bservations # Conflicts: # docs/THREAT_MODEL.md
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba9a7acfa6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| canary_found = case_observation.canary_token_detected | ||
| blocked = case_observation.control_blocked | ||
| cleaned = result.cleanup_status in ("completed", "not_required") |
There was a problem hiding this comment.
When a remote run reports cleanup_status as partial or failed—statuses the worker uses when effects remain unresolved—this only makes the case result failed; the environment remains verified. A subsequent execute_case therefore passes the status and isolation gates and can run against a contaminated laboratory, corrupting case results or compounding residual effects. Invalidate the environment and require a verified reset whenever cleanup is incomplete.
Useful? React with 👍 / 👎.
| "UPDATE cases SET state = 'pending', result_id = ?, payload_json = ? " | ||
| "WHERE attempt_id = ? AND state = 'dispatching'", | ||
| (result_id, json.dumps({"failure_result": failure_result}, sort_keys=True), attempt_id), |
There was a problem hiding this comment.
After dispatch succeeds, this update replaces the entire intent payload with only failure_result, discarding dispatch_request_id, authorization_id, and plan_digest. If the controller restarts before classification, the in-memory RemoteAuthorizedRun is lost and the journal converts this pending record to failed without the identifiers needed to correlate the execution with worker audit evidence. Merge the failure result into the existing payload instead of overwriting the dispatch metadata.
AGENTS.md reference: AGENTS.md:L189-L194
Useful? React with 👍 / 👎.
| environment.isolation["verification_status"] = "failed" | ||
| environment.isolation["verification_notes"] = f"Unsupported isolation type: {iso_type}" | ||
| environment.transition_to("failed") | ||
| expected_type = {"container": "container_unprivileged", "vm": "vm_hypervisor"}.get(environment.environment_type) |
There was a problem hiding this comment.
For every schema-valid process_sandbox environment using process_isolated, this lookup returns None, so verification always marks the environment failed. The parent implementation accepted process_isolated, and both the current public schema and TestedMatrix.supported_runtimes still advertise process_sandbox; existing contracts can therefore register successfully but can never verify. Add the missing type pair or remove it through a coordinated migration.
AGENTS.md reference: AGENTS.md:L132-L134
Useful? React with 👍 / 👎.
| if lower_bound is not None and observed_at <= lower_bound: | ||
| raise LaboratoryGateError("laboratory receipt predates the operation") |
There was a problem hiding this comment.
Receipt timestamps accept values without fractional seconds, but requested_at records microseconds and this strict comparison rejects a legitimately post-challenge receipt whenever its signer emits second-resolution RFC 3339 within the same wall-clock second. classify_case applies the same strict comparison against result.finished_at, so live observation and reset integrations can fail intermittently despite a valid nonce and signature. Normalize timestamp precision or use a bounded causal tolerance while retaining the nonce and freshness checks.
Useful? React with 👍 / 👎.
Summary
cryptographydependency only during signed receipt verification so the standard-library operator CLI can start before contributor dependencies are installed. Verification fails closed when the package is absent.Validation and review
python -Sregression cases.make checkpassed in the issue worktree (All COPS checks passed).git diff --checkpassed.Evidence limits
Validation used inert local fixtures. No live operator VM/container receipts were available, so this PR does not claim observed isolation or a live reset. The operator-provisioned signer, worker, and inventory remain trust boundaries, documented in the updated threat model and laboratory guidance.
Human review is requested for exact head
b476937cf850aad655ccbee826d374425e16af83, including the authorization and scope gates, signed observation handling, outcome classification, recovery journal, negative cases, and documentation.Closes #190