diff --git a/CLAUDE.md b/CLAUDE.md index e2ebdf6..697e409 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -78,18 +78,18 @@ These are non-negotiable. Each ties back to a schema validator, a wrapper, or a ### 3.1 Evidence integrity - **Never write to `/evidence/`.** It is a read-only microsandbox mount with `noexec` on data partitions; the host also `chattr +i`s evidence files. Any tool wrapper that writes to evidence is a bug, not a feature. -- **Hash on entry, re-hash periodically.** Every evidence file gets a SHA-256 at `case_init` recorded in the `EvidenceManifest`. The runtime re-hashes every 10 super-steps (`verdict/runtime/evidence_recheck.py`). Mismatch raises `HashMismatchError` and halts the case. +- **Hash on entry, re-hash periodically.** Every evidence file gets a SHA-256 at `case_init` recorded in the `EvidenceManifest`. The runtime re-hashes every 10 super-steps (`src/verdict/runtime/evidence_recheck.py`). Mismatch raises `HashMismatchError` and halts the case. - **Per-invocation hash.** Every tool call records `invocation_hash = blake3(tool_name + tool_version + args + evidence_hash)` in its `ToolOutput` and ledger entry. - **Per-output-file hash.** `LedgerEntry.output_files_sha256: dict[str, str]` records SHA-256 of every file the tool emits. NIST SP 800-86 §5.1.2 / §5.1.4 compliance. ### 3.2 Multi-artifact corroboration - `Finding.artifact_paths` and `Finding.artifact_classes` both have `min_length=2`. Single-artifact execution claims are forensically unsound and the validator rejects them. -- **Execution-class MITRE techniques** — T1059, T1106, T1204, T1218, T1543, T1547 — require **≥2 distinct `ArtifactClass` values** (not just two paths in the same class). Validator: `Finding._execution_requires_two_classes`. +- **Execution-class MITRE techniques** — T1059, T1106, T1204, T1218, T1543, T1547 — require **≥2 distinct `ArtifactClass` values** (not just two paths in the same class). Validator: `Finding._forensic_corroboration`. ### 3.3 Tier-1 caveat acknowledgment -`Finding.caveats_acknowledged: list[CaveatID]` is enforced at the schema layer. Cite the artifact, acknowledge the caveat. Caveat triggers are keyed by `Finding.artifact_classes` membership unless otherwise noted; `LOGON_TYPE_3_VS_10` is the named exception (triggered by `EVTX_4624` artifact_class AND the `EvtxRecord.LogonType` field equaling 3 or 10). The seven Tier-1 caveats (encoded in `verdict/schemas/caveat_id.py` and `verdict/prompts/examiner_caveats.md`): +`Finding.caveats_acknowledged: list[CaveatID]` is enforced at the schema layer. Cite the artifact, acknowledge the caveat. Caveat triggers are keyed by `Finding.artifact_classes` membership unless otherwise noted; `LOGON_TYPE_3_VS_10` is the named exception (triggered by `EVTX_4624` artifact_class AND the `EvtxRecord.LogonType` field equaling 3 or 10). The seven Tier-1 caveats (encoded in `src/verdict/schemas/caveat_id.py` and `src/verdict/planning/prompts/examiner_caveats.md`): | CaveatID | Trigger | |----------|---------| @@ -153,7 +153,7 @@ Every new dependency must be **MIT or Apache-2.0** unless explicitly approved in - API keys, OAuth tokens, and bearer tokens **never enter a microVM**. This includes `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`, GitHub tokens, Langfuse keys, and any bearer token used by host-side AI agents. They are injected via TSI on host egress only; tcpdump-verifiable. - HMAC ledger key is TPM-backed (`/dev/tpmrm0`) when available, else gpg-encrypted at `~/.verdict/key.gpg` with passphrase prompted at gateway init. -- Ledger redaction strips `authorization`, `auth_user`, `api_key` **before** the entry is hashed and HMAC-signed (`verdict/ledger/redaction.py`). +- Ledger redaction strips `authorization`, `auth_user`, `api_key` **before** the entry is hashed and HMAC-signed (`src/verdict/ledger/redaction.py`). - Anthropic OAuth tokens (Claude Code interactive auth) are not redistributable per Anthropic's commercial terms — do not commit, do not bake into images. ### 3.10 No mocks, no stubs, no placeholders — full-stack real @@ -183,7 +183,7 @@ If you find yourself reaching for a mock to make a test fast or hermetic, you ar ## 4. Architecture at a glance -9-node LangGraph state machine: **planner → planner_critique (CoVe) → comprehension_gate → executor_fanout (n=4; each branch composed of DenyRuleWrapper / ToolExecutor / LedgerEmitter) → pivot (≤15) → quorum → replan (≤3) → unverifiable_finalize → finalize**. +8-node LangGraph state machine: **planner → planner_critique (CoVe) → comprehension_gate → executor_fanout (n=4; each branch composed of DenyRuleWrapper / ToolExecutor / LedgerEmitter) → pivot (≤15) → quorum → replan (≤3) → finalize**. (`unverifiable_finalize_node` is a helper called by `replan_node` when the budget is exhausted — it is not a registered graph node; `build_graph()` registers 8 nodes.) Three operational modes, auto-detected at `case_init` and **locked**: @@ -210,7 +210,7 @@ Verdict/ ├── CLAUDE.md README.md CONTRIBUTING.md SECURITY.md LICENSE .env.example ├── docs/ │ ├── ARCHITECTURE.md BUILD_PLAN.md DEVPOST_COMPLIANCE.md DOCS_ACCURACY_REPORT.md -│ └── spec/ ← frozen audit archive (01..05 + README) +│ └── spec/ ← frozen audit archive (01..04 + README) ├── downloads/ ← SIFT OVA, evidence samples (gitignored) └── protocol-sift/ ← upstream submodule ``` @@ -229,7 +229,7 @@ scripts/ .github/workflows/ packer/ ## 7. Forensic doctrine (one-paragraph summaries) -The SANS-canonical knowledge an agent must internalise. Encoded in `verdict/playbooks/`, `verdict/knowledge/`, `verdict/prompts/` — never duplicated in narrative code comments. Full discipline (with rationale, validators, schema field references): **`docs/ARCHITECTURE.md` §4**. +The SANS-canonical knowledge an agent must internalise. Encoded in `src/verdict/playbooks/`, `src/verdict/knowledge/`, `src/verdict/planning/prompts/` — never duplicated in narrative code comments. Full discipline (with rationale, validators, schema field references): **`docs/ARCHITECTURE.md` §4**. - **Canonical first moves.** Memory → `windows.info`. Disk → `image_hash_verify` → `mmls` → `fsstat`. Triage zip → registry hives first. - **DKOM / T1014.** `set(psscan_pids) - set(pslist_pids)` non-empty → emit T1014 hypothesis. First-class playbook rule (v4.6 F4), not a prompt suggestion. @@ -241,10 +241,10 @@ The SANS-canonical knowledge an agent must internalise. Encoded in `verdict/play ## 8. Verifier strategies (one-line each) -`verdict/verification/strategy.py` — `VerifierStrategy` Protocol; quorum dispatches per locked mode. +`src/verdict/verification/strategy.py` — `VerifierStrategy` Protocol; quorum dispatches per locked mode. - `CloudSelfConsistency` — n=3 with three blake3-keyed seeds at `temp=0.7` (v4.6 F1; **never** temp=0, that collapses to n=1). ≥ 2-of-3 → `VETTED_CLOUD`. -- `AirGapCrossEngine` — Qwen3 + GLM-4.5-Air both execute. Jaccard ≥ 0.80 on `artifact_paths` AND identical `mitre_technique` → `VETTED_AIRGAP`. +- `AirGapCrossEngine` — Qwen3 + GLM-4.5-Air both execute. Jaccard ≥ 0.80 on `parsed_artifacts` AND identical `mitre_technique` → `VETTED_AIRGAP`. - `DualLaneCrossEngine` — cloud + both locals. Cloud agrees with ≥1 local AND locals agree → `VETTED_DUAL`. - `UniversalSelfConsistency` (Chen 2023) — judge of last resort before declaring `CONTESTED`. @@ -254,7 +254,7 @@ The SANS-canonical knowledge an agent must internalise. Encoded in `verdict/play ## 9. Ledger discipline -`verdict/ledger/writer.py` + `chain.py`. The ledger is the chain-of-custody artifact a SANS judge will scrutinise. +`src/verdict/ledger/writer.py`. The ledger is the chain-of-custody artifact a SANS judge will scrutinise. - JSONL append-only at `cases//ledger.jsonl`. `prev_entry_hash` chains entries; `verdict validate ` walks them. - Three-tier IDs on every entry: `case_id` → `langfuse_trace_id` → `langgraph_checkpoint_id`. @@ -275,7 +275,7 @@ Event types: `case_init`, `tool_call`, `finding`, `approval`, `rejection`, `mode ```bash # Microsandbox -curl -sSL https://get.microsandbox.dev | sh +curl -fsSL https://install.microsandbox.dev | sh # Bootstrap (three-credential-path detection) bash scripts/install.sh @@ -316,7 +316,7 @@ verdict health ```bash # Schema/playbook/knowledge gates (W1) -uv run --directory services/agent pytest tests/schemas/ -v +uv run pytest tests/schemas/ -v uv run pytest tests/playbooks/ -v uv run pytest tests/knowledge/ -v diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e65835d..42f3381 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -6,7 +6,7 @@ New-contributor setup guide. If you've already done the SANS Find Evil! 2026 tea **Default branch:** `main` **License:** MIT **Deadline gate:** 2026-06-15 22:45 CDT (team-internal target: 2026-06-14 EOD = ~28 h buffer) -**Recommended platform:** SANS **SIFT Workstation VM** (canonical — all forensic tools, Microsandbox, SGLang, evidence mounts work out-of-box) **or** any modern **Linux box** (Ubuntu 22.04+ / Debian 12+ / Fedora 39+ / Arch). **macOS host is acceptable** for schema/planner/MCP/unit-test work that doesn't shell out to forensic tools — but the moment your task touches `verdict/sandboxes/`, `verdict/tools/vol3/`, or anything that runs in Microsandbox, switch into the SIFT VM. **Windows host is not supported** for development; use WSL2 + Ubuntu or pull the SIFT VM (it runs under Hyper-V / VMware / VirtualBox). +**Recommended platform:** SANS **SIFT Workstation VM** (canonical — all forensic tools, Microsandbox, SGLang, evidence mounts work out-of-box) **or** any modern **Linux box** (Ubuntu 22.04+ / Debian 12+ / Fedora 39+ / Arch). **macOS host is acceptable** for schema/planner/MCP/unit-test work that doesn't shell out to forensic tools — but the moment your task touches `src/verdict/sandboxes/`, `src/verdict/tools/`, or anything that runs in Microsandbox, switch into the SIFT VM. **Windows host is not supported** for development; use WSL2 + Ubuntu or pull the SIFT VM (it runs under Hyper-V / VMware / VirtualBox). Authority chain when docs disagree: Devpost rules → `DEVPOST_COMPLIANCE.md` → `ARCHITECTURE.md` → `BUILD_PLAN.md` → this file. Code + lockfiles win over docs (per `CLAUDE.md`); update the doc, don't roll back the code, unless the code is wrong. @@ -118,13 +118,13 @@ Idempotent. Installs uv + Python 3.11, Rust 1.88, Node 20 + pnpm, and Microsandb **Where to develop.** Three supported configurations, in order of preference: -1. **Inside the SIFT VM** (canonical, recommended for everyone). All forensic tools, Microsandbox, SGLang, and the evidence mounts resolve here without setup. **Required** for any work touching the executor branches, the Microsandbox layer, evidence I/O, or anything under `verdict/sandboxes/`, `verdict/tools/vol3/`, or `services/mcp/src/tools/`. Pull the OVA from `downloads/README.md`; snapshot it as `clean-install` before installing anything. +1. **Inside the SIFT VM** (canonical, recommended for everyone). All forensic tools, Microsandbox, SGLang, and the evidence mounts resolve here without setup. **Required** for any work touching the executor branches, the Microsandbox layer, evidence I/O, or anything under `src/verdict/sandboxes/`, `src/verdict/tools/`. Pull the OVA from `downloads/README.md`; snapshot it as `clean-install` before installing anything. 2. **A modern Linux box** (Ubuntu 22.04+ / Debian 12+ / Fedora 39+ / Arch). Acceptable for the full stack as long as you can install Microsandbox (Linux-only) and run the SIFT toolchain (`apt install sleuthkit volatility ...`). Faster I/O than the VM. Take a `pre-verdict` snapshot of your home before installing forensic tools — they leave state. 3. **macOS host with the SIFT VM as a runtime target** (faster edit loop, smaller surface). Use VS Code Remote-SSH or `Develop on a Container` into the VM. Acceptable for schema, planner, MCP gateway, and unit-test work that doesn't shell out to forensic tools or Microsandbox. The moment your task touches a Microsandbox path or a forensic CLI, switch into the VM. **Windows host is not supported** for development. Either use WSL2 + Ubuntu (treat as configuration #2) or run the SIFT VM under Hyper-V / VMware / VirtualBox (configuration #1). Native Windows breaks Microsandbox (libkrun is Linux-only), pre-commit hooks (path/exec semantics), and several SIFT tools. -If your work touches `services/mcp/src/tools/`, `services/agent_mcp/`, or anything under a Microsandbox path, you **must** run integration tests inside the SIFT VM (or your Linux box with Microsandbox installed) before opening a PR. macOS-only test runs do not satisfy the gate. +If your work touches `src/verdict/tools/`, `src/verdict/sandboxes/`, or anything under a Microsandbox path, you **must** run integration tests inside the SIFT VM (or your Linux box with Microsandbox installed) before opening a PR. macOS-only test runs do not satisfy the gate. --- @@ -271,7 +271,7 @@ If the run fails before producing a Finding, you have a P0 environment problem. Authority order (escalate left-to-right): 1. **NotebookLM Q&A:** https://notebooklm.google.com/notebook/f0957a60-6fb2-452b-93d4-ecd73ba47779?authuser=1 — chief location for "how does X work?" -2. **`docs/` and the audit history in `archive/`** — most "why did we choose X?" questions are answered in `archive/03-audit-v4.5.md`. +2. **`docs/` and the audit history in `docs/spec/`** — most "why did we choose X?" questions are answered in `docs/spec/03-audit-v4.5.md`. 3. **Team chat** — PUG / Beaver / Haley / KP. Use the appropriate thread; don't DM PUG for things the team should see. 4. **Devpost platform issues only:** https://help.devpost.com/ diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 58786a8..a2b9c21 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -96,7 +96,7 @@ START │ │ parsed_success_criteria_hash. Mismatch │ │ → clarify sub-state (re-prompts within │ │ the same node, not a separate top-level -│ │ node — total node count stays 9). +│ │ node — total node count stays 8). └────────┬────────┘ ▼ (fanout — 4 parallel branches) ┌─────────────────┐ @@ -192,49 +192,46 @@ The differentiator vs. competitors. Schema validators reject sloppy findings bef class Finding(BaseModel): artifact_paths: list[Path] = Field(min_length=2) artifact_classes: list[ArtifactClass] = Field(min_length=2) - caveats_acknowledged: list[CaveatID] = [] + caveats_acknowledged: list[CaveatID] = Field(default_factory=list) mitre_technique: str | None # validated against ^T\d{4}(\.\d{3})?$ @model_validator(mode="after") - def _execution_claims_need_two_classes(self): - is_exec = any( - self.mitre_technique and self.mitre_technique.startswith(p) - for p in ("T1059", "T1106", "T1204", "T1218", "T1543", "T1547") + def _forensic_corroboration(self): + is_exec = self.mitre_technique and self.mitre_technique.startswith( + ("T1059", "T1106", "T1204", "T1218", "T1543", "T1547") ) if is_exec and len(set(self.artifact_classes)) < 2: - raise ValueError(f"execution claim needs ≥2 distinct artifact classes") - return self - - @model_validator(mode="after") - def _amcache_caveat_required(self): - if ArtifactClass.AMCACHE in self.artifact_classes: - if CaveatID.AMCACHE_LASTMODIFIED_NOT_EXEC not in self.caveats_acknowledged: - raise ValueError("Finding cites Amcache without LastModified caveat") + raise ValueError("execution claims require two distinct artifact classes") + acknowledged = set(self.caveats_acknowledged) + for artifact_class, required_caveat in AVAILABLE_CAVEAT_TRIGGERS.items(): + if artifact_class in self.artifact_classes and required_caveat not in acknowledged: + raise ValueError(f"{required_caveat.value} must be acknowledged") return self ``` ### CaveatID — Tier-1 examiner caveats from `CLAUDE.md` §3.3 ```python -class CaveatID(str, Enum): - AMCACHE_LASTMODIFIED_NOT_EXEC = "amcache_lastmodified_neq_execution" - SHIMCACHE_ORDER_CHANGED_WIN81 = "shimcache_order_lru_pre81_insertion_post81" - PREFETCH_SSD_DISABLED = "prefetch_disabled_on_ssd_or_gpo" - MFT_SI_STOMPABLE = "mft_si_timestomp_use_fn" - USNJRNL_WRAPS = "usnjrnl_wraps_treat_gaps_carefully" - LOGON_TYPE_3_VS_10 = "evtx_4624_type3_network_neq_type10_rdp" - SYSMON_PROCESSGUID_OVER_PID = "sysmon_processguid_correlation_key_not_pid" +class CaveatID(StrEnum): + AMCACHE_LASTMODIFIED_NOT_EXEC = "AMCACHE_LASTMODIFIED_NOT_EXEC" + SHIMCACHE_ORDER_CHANGED_WIN81 = "SHIMCACHE_ORDER_CHANGED_WIN81" + PREFETCH_SSD_DISABLED = "PREFETCH_SSD_DISABLED" + MFT_SI_STOMPABLE = "MFT_SI_STOMPABLE" + USNJRNL_WRAPS = "USNJRNL_WRAPS" + LOGON_TYPE_3_VS_10 = "LOGON_TYPE_3_VS_10" + SYSMON_PROCESSGUID_OVER_PID = "SYSMON_PROCESSGUID_OVER_PID" ``` -Loaded into every executor system prompt via `verdict/planning/prompts/examiner_caveats.md`. +Loaded into every executor system prompt via `src/verdict/planning/prompts/examiner_caveats.md`. ### ArtifactClass — multi-source corroboration vocabulary ```python -class ArtifactClass(str, Enum): +class ArtifactClass(StrEnum): PREFETCH = "prefetch" AMCACHE = "amcache" SHIMCACHE = "shimcache" + EVTX_4624 = "evtx_4624" EVTX_4688 = "evtx_4688" SYSMON_1 = "sysmon_1" NETWORK = "network" @@ -242,6 +239,7 @@ class ArtifactClass(str, Enum): TASK_SCHEDULER = "task_scheduler" WMI_SUBSCRIPTION = "wmi_subscription" MFT = "mft" + USNJRNL = "usnjrnl" PROCESS_MEMORY = "process_memory" YARA_HIT = "yara_hit" SIGMA_HIT = "sigma_hit" @@ -249,7 +247,7 @@ class ArtifactClass(str, Enum): ### Playbooks — SANS canonical tool sequencing -Three YAMLs in `verdict/playbooks/` (memory.yml / disk.yml / triage.yml) encode the SANS-canonical sequencing summarized in `CLAUDE.md` §7 and this document's forensic doctrine. Loaded into planner system prompt at case_init based on detected evidence type. +Three YAMLs in `src/verdict/playbooks/` (memory.yml / disk.yml / triage.yml) encode the SANS-canonical sequencing summarized in `CLAUDE.md` §7 and this document's forensic doctrine. Loaded into planner system prompt at case_init based on detected evidence type. `memory.yml` example rule (DKOM detection): ```yaml @@ -263,7 +261,7 @@ This is one of the architecture's clearest moats — DKOM/T1014 detection auto-f ### Hunt Evil baseline -`verdict/knowledge/hunt_evil.yml` keyed by process name with expected parent / path / signing / instance count for 8 canonical Windows processes (svchost, lsass, csrss, winlogon, services, wininit, explorer, smss). `ProcessBaselineAnomaly` Hypothesis subtype maps to `T1036.005` (Match Legitimate Name or Location). Catches `scvhost.exe` with parent `cmd.exe` automatically. +`src/verdict/knowledge/hunt_evil.yml` keyed by process name with expected parent / path / signing / instance count for 8 canonical Windows processes (svchost, lsass, csrss, winlogon, services, wininit, explorer, smss). `ProcessBaselineAnomaly` Hypothesis subtype maps to `T1036.005` (Match Legitimate Name or Location). Catches `scvhost.exe` with parent `cmd.exe` automatically. --- @@ -530,5 +528,5 @@ These were raised during the v4.4 research and v4.5 system-design review. They'r | v4.4 agentic + DFIR research findings (raw) | `docs/spec/02-audit-v4.4.md` | | v4.6 schema patches (raw spec) | `docs/spec/04-spec-plan-v4.6.md` | | Project-wide build conventions | `../CLAUDE.md` | -| Tier-1 examiner caveat source | `../CLAUDE.md` §3.3 and planned `../verdict/planning/prompts/examiner_caveats.md` | -| Tool sequencing playbook source | `../verdict/playbooks/*.yml` and `docs/ARCHITECTURE.md` §4 | +| Tier-1 examiner caveat source | `../CLAUDE.md` §3.3 and `../src/verdict/planning/prompts/examiner_caveats.md` | +| Tool sequencing playbook source | `../src/verdict/playbooks/*.yml` and `docs/ARCHITECTURE.md` §4 |