Milestone 5: add executable behavioral scenarios

.github/workflows/f7las-ci.yml

@@ -62,8 +62,12 @@ jobs:
       run: python scripts/validate-contracts.py
     - name: Validate canonical OPA policy
       run: opa check --strict config/policies/canonical-workflow.rego
+    - name: Run behavioral scenarios
+      run: 'pytest -q tests/test_behavioral_scenarios.py
+
+        '
     - name: Run unit tests
-      run: 'pytest -q
+      run: 'pytest -q --ignore=tests/test_behavioral_scenarios.py
 
         '
     - name: Run golden dataset evaluation

ROADMAP.md

@@ -14,7 +14,7 @@ The target is an **Executable Reference Implementation**: one offline, determini
 2. **Canonical data contracts** — define deterministic request, context, plan, proposed action, approval, decision, result, and audit records.
 3. **Canonical Python + OPA path** — one offline, fail-closed Layers 1–7 workflow is implemented for the bounded synthetic action.
 4. **Approval binding** — synthetic approval is bound to the exact request/action digests, scope, complete policy reference, authority, and expiry in the canonical path.
-5. **Behavioral scenarios** — test permitted, denied, malformed, unauthorized, unavailable, tampered, recovery, and other required paths.
+5. **Behavioral scenarios** — an executable canonical matrix tests permitted, denied, malformed, unauthorized, unavailable, timed-out, tampered, expired, obligation, and recovery paths.
 6. **Evidence and replay** — correlate records, detect tampering, and reproduce deterministic outcomes.
 7. **Supply chain and CI** — pin dependencies and actions; add integrity, vulnerability, secret, and SBOM checks.
 8. **Documentation reconciliation** — execute every documented command in a clean environment.

examples/canonical-workflow/README.md

@@ -42,3 +42,21 @@ If OPA is missing, times out, rejects the policy, or returns malformed output,
 the workflow emits a denial and `not_executed` result. It never falls back to an
 allow decision. The evidence file is preserved and the CLI returns exit status
 `3` for a denied or otherwise unexecuted action.
+
+## Behavioral scenarios
+
+The executable scenario manifest at `tests/behavioral_scenarios.json` covers
+permit, policy denial, malformed input and PDP output, unauthorized identity,
+PDP unavailability and timeout, post-decision tampering, missing obligations,
+approval expiry, and recovery after a transient PDP outage. Accepted attempts
+must emit schema-valid, cross-record-valid evidence with no reported side
+effects. Admission refusals must emit no workflow records.
+
+Run the matrix with the pinned OPA CLI available:
+
+```bash
+OPA_BIN=opa pytest -q tests/test_behavioral_scenarios.py
+```
+
+These are deterministic reference-workflow scenarios, not claims of production
+fault injection, infrastructure recovery, or OS/container isolation.

tests/behavioral_scenarios.json

@@ -0,0 +1,195 @@
+{
+  "scenario_set_version": "1.0.0",
+  "scenarios": [
+    {
+      "id": "BHV-001",
+      "category": "permitted",
+      "title": "Approved read-only action succeeds",
+      "driver": "permitted",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "permit",
+          "decision_reason": "approved-synthetic-read",
+          "execution": "succeeded",
+          "execution_reason": null,
+          "audit": "success",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-002",
+      "category": "denied",
+      "title": "OPA policy denial prevents execution",
+      "driver": "policy_denied",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "deny",
+          "decision_reason": "policy-denied",
+          "execution": "not_executed",
+          "execution_reason": "policy-denied",
+          "audit": "denied",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-003",
+      "category": "malformed",
+      "title": "Malformed workflow interface is refused",
+      "driver": "malformed_input",
+      "expected_attempts": [
+        {
+          "admission": "refused",
+          "refusal_reason": "malformed-input",
+          "cli_exit": 2,
+          "records_emitted": 0
+        }
+      ]
+    },
+    {
+      "id": "BHV-004",
+      "category": "unauthorized",
+      "title": "Unauthorized actor is refused",
+      "driver": "unauthorized_actor",
+      "expected_attempts": [
+        {
+          "admission": "refused",
+          "refusal_reason": "unauthorized-identity",
+          "cli_exit": 2,
+          "records_emitted": 0
+        }
+      ]
+    },
+    {
+      "id": "BHV-005",
+      "category": "unavailable",
+      "title": "Unavailable PDP fails closed with evidence",
+      "driver": "pdp_unavailable",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "deny",
+          "decision_reason": "pdp-unavailable",
+          "execution": "not_executed",
+          "execution_reason": "pdp-unavailable",
+          "audit": "denied",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-006",
+      "category": "unavailable",
+      "title": "Timed-out PDP fails closed with evidence",
+      "driver": "pdp_timeout",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "deny",
+          "decision_reason": "pdp-unavailable",
+          "execution": "not_executed",
+          "execution_reason": "pdp-unavailable",
+          "audit": "denied",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-007",
+      "category": "malformed",
+      "title": "Malformed PDP response fails closed with evidence",
+      "driver": "pdp_malformed_response",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "deny",
+          "decision_reason": "pdp-invalid-response",
+          "execution": "not_executed",
+          "execution_reason": "pdp-invalid-response",
+          "audit": "denied",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-008",
+      "category": "tampered",
+      "title": "Post-decision action tampering is blocked by the executor",
+      "driver": "tampered_action",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "permit",
+          "decision_reason": "approved-synthetic-read",
+          "execution": "not_executed",
+          "execution_reason": "executor-binding-mismatch",
+          "audit": "not_executed",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-009",
+      "category": "obligation",
+      "title": "Permit missing a required obligation is not executed",
+      "driver": "missing_obligation",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "permit",
+          "decision_reason": "mocked-permit",
+          "execution": "not_executed",
+          "execution_reason": "unfulfilled-policy-obligation",
+          "audit": "not_executed",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-010",
+      "category": "expired",
+      "title": "Approval at its expiration boundary is not executed",
+      "driver": "expired_approval",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "permit",
+          "decision_reason": "approved-synthetic-read",
+          "execution": "not_executed",
+          "execution_reason": "executor-approval-invalid-at-execution",
+          "audit": "not_executed",
+          "side_effects": []
+        }
+      ]
+    },
+    {
+      "id": "BHV-011",
+      "category": "recovery",
+      "title": "Workflow recovers after a transient PDP outage",
+      "driver": "recovery_after_pdp_unavailable",
+      "expected_attempts": [
+        {
+          "admission": "accepted",
+          "decision": "deny",
+          "decision_reason": "pdp-unavailable",
+          "execution": "not_executed",
+          "execution_reason": "pdp-unavailable",
+          "audit": "denied",
+          "side_effects": []
+        },
+        {
+          "admission": "accepted",
+          "decision": "permit",
+          "decision_reason": "approved-synthetic-read",
+          "execution": "succeeded",
+          "execution_reason": null,
+          "audit": "success",
+          "side_effects": []
+        }
+      ]
+    }
+  ]
+}