Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 12 additions & 12 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,18 +78,18 @@ These are non-negotiable. Each ties back to a schema validator, a wrapper, or a
### 3.1 Evidence integrity

- **Never write to `/evidence/`.** It is a read-only microsandbox mount with `noexec` on data partitions; the host also `chattr +i`s evidence files. Any tool wrapper that writes to evidence is a bug, not a feature.
- **Hash on entry, re-hash periodically.** Every evidence file gets a SHA-256 at `case_init` recorded in the `EvidenceManifest`. The runtime re-hashes every 10 super-steps (`verdict/runtime/evidence_recheck.py`). Mismatch raises `HashMismatchError` and halts the case.
- **Hash on entry, re-hash periodically.** Every evidence file gets a SHA-256 at `case_init` recorded in the `EvidenceManifest`. The runtime re-hashes every 10 super-steps (`src/verdict/runtime/evidence_recheck.py`). Mismatch raises `HashMismatchError` and halts the case.
- **Per-invocation hash.** Every tool call records `invocation_hash = blake3(tool_name + tool_version + args + evidence_hash)` in its `ToolOutput` and ledger entry.
- **Per-output-file hash.** `LedgerEntry.output_files_sha256: dict[str, str]` records SHA-256 of every file the tool emits. NIST SP 800-86 §5.1.2 / §5.1.4 compliance.

### 3.2 Multi-artifact corroboration

- `Finding.artifact_paths` and `Finding.artifact_classes` both have `min_length=2`. Single-artifact execution claims are forensically unsound and the validator rejects them.
- **Execution-class MITRE techniques** — T1059, T1106, T1204, T1218, T1543, T1547 — require **≥2 distinct `ArtifactClass` values** (not just two paths in the same class). Validator: `Finding._execution_requires_two_classes`.
- **Execution-class MITRE techniques** — T1059, T1106, T1204, T1218, T1543, T1547 — require **≥2 distinct `ArtifactClass` values** (not just two paths in the same class). Validator: `Finding._forensic_corroboration`.

### 3.3 Tier-1 caveat acknowledgment

`Finding.caveats_acknowledged: list[CaveatID]` is enforced at the schema layer. Cite the artifact, acknowledge the caveat. Caveat triggers are keyed by `Finding.artifact_classes` membership unless otherwise noted; `LOGON_TYPE_3_VS_10` is the named exception (triggered by `EVTX_4624` artifact_class AND the `EvtxRecord.LogonType` field equaling 3 or 10). The seven Tier-1 caveats (encoded in `verdict/schemas/caveat_id.py` and `verdict/prompts/examiner_caveats.md`):
`Finding.caveats_acknowledged: list[CaveatID]` is enforced at the schema layer. Cite the artifact, acknowledge the caveat. Caveat triggers are keyed by `Finding.artifact_classes` membership unless otherwise noted; `LOGON_TYPE_3_VS_10` is the named exception (triggered by `EVTX_4624` artifact_class AND the `EvtxRecord.LogonType` field equaling 3 or 10). The seven Tier-1 caveats (encoded in `src/verdict/schemas/caveat_id.py` and `src/verdict/planning/prompts/examiner_caveats.md`):

| CaveatID | Trigger |
|----------|---------|
Expand Down Expand Up @@ -153,7 +153,7 @@ Every new dependency must be **MIT or Apache-2.0** unless explicitly approved in

- API keys, OAuth tokens, and bearer tokens **never enter a microVM**. This includes `ANTHROPIC_API_KEY`, `OPENROUTER_API_KEY`, GitHub tokens, Langfuse keys, and any bearer token used by host-side AI agents. They are injected via TSI on host egress only; tcpdump-verifiable.
- HMAC ledger key is TPM-backed (`/dev/tpmrm0`) when available, else gpg-encrypted at `~/.verdict/key.gpg` with passphrase prompted at gateway init.
- Ledger redaction strips `authorization`, `auth_user`, `api_key` **before** the entry is hashed and HMAC-signed (`verdict/ledger/redaction.py`).
- Ledger redaction strips `authorization`, `auth_user`, `api_key` **before** the entry is hashed and HMAC-signed (`src/verdict/ledger/redaction.py`).
- Anthropic OAuth tokens (Claude Code interactive auth) are not redistributable per Anthropic's commercial terms — do not commit, do not bake into images.

### 3.10 No mocks, no stubs, no placeholders — full-stack real
Expand Down Expand Up @@ -183,7 +183,7 @@ If you find yourself reaching for a mock to make a test fast or hermetic, you ar

## 4. Architecture at a glance

9-node LangGraph state machine: **planner → planner_critique (CoVe) → comprehension_gate → executor_fanout (n=4; each branch composed of DenyRuleWrapper / ToolExecutor / LedgerEmitter) → pivot (≤15) → quorum → replan (≤3) → unverifiable_finalize → finalize**.
8-node LangGraph state machine: **planner → planner_critique (CoVe) → comprehension_gate → executor_fanout (n=4; each branch composed of DenyRuleWrapper / ToolExecutor / LedgerEmitter) → pivot (≤15) → quorum → replan (≤3) → finalize**. (`unverifiable_finalize_node` is a helper called by `replan_node` when the budget is exhausted — it is not a registered graph node; `build_graph()` registers 8 nodes.)

Three operational modes, auto-detected at `case_init` and **locked**:

Expand All @@ -210,7 +210,7 @@ Verdict/
├── CLAUDE.md README.md CONTRIBUTING.md SECURITY.md LICENSE .env.example
├── docs/
│ ├── ARCHITECTURE.md BUILD_PLAN.md DEVPOST_COMPLIANCE.md DOCS_ACCURACY_REPORT.md
│ └── spec/ ← frozen audit archive (01..05 + README)
│ └── spec/ ← frozen audit archive (01..04 + README)
├── downloads/ ← SIFT OVA, evidence samples (gitignored)
└── protocol-sift/ ← upstream submodule
```
Expand All @@ -229,7 +229,7 @@ scripts/ .github/workflows/ packer/

## 7. Forensic doctrine (one-paragraph summaries)

The SANS-canonical knowledge an agent must internalise. Encoded in `verdict/playbooks/`, `verdict/knowledge/`, `verdict/prompts/` — never duplicated in narrative code comments. Full discipline (with rationale, validators, schema field references): **`docs/ARCHITECTURE.md` §4**.
The SANS-canonical knowledge an agent must internalise. Encoded in `src/verdict/playbooks/`, `src/verdict/knowledge/`, `src/verdict/planning/prompts/` — never duplicated in narrative code comments. Full discipline (with rationale, validators, schema field references): **`docs/ARCHITECTURE.md` §4**.

- **Canonical first moves.** Memory → `windows.info`. Disk → `image_hash_verify` → `mmls` → `fsstat`. Triage zip → registry hives first.
- **DKOM / T1014.** `set(psscan_pids) - set(pslist_pids)` non-empty → emit T1014 hypothesis. First-class playbook rule (v4.6 F4), not a prompt suggestion.
Expand All @@ -241,10 +241,10 @@ The SANS-canonical knowledge an agent must internalise. Encoded in `verdict/play

## 8. Verifier strategies (one-line each)

`verdict/verification/strategy.py` — `VerifierStrategy` Protocol; quorum dispatches per locked mode.
`src/verdict/verification/strategy.py` — `VerifierStrategy` Protocol; quorum dispatches per locked mode.

- `CloudSelfConsistency` — n=3 with three blake3-keyed seeds at `temp=0.7` (v4.6 F1; **never** temp=0, that collapses to n=1). ≥ 2-of-3 → `VETTED_CLOUD`.
- `AirGapCrossEngine` — Qwen3 + GLM-4.5-Air both execute. Jaccard ≥ 0.80 on `artifact_paths` AND identical `mitre_technique` → `VETTED_AIRGAP`.
- `AirGapCrossEngine` — Qwen3 + GLM-4.5-Air both execute. Jaccard ≥ 0.80 on `parsed_artifacts` AND identical `mitre_technique` → `VETTED_AIRGAP`.
- `DualLaneCrossEngine` — cloud + both locals. Cloud agrees with ≥1 local AND locals agree → `VETTED_DUAL`.
- `UniversalSelfConsistency` (Chen 2023) — judge of last resort before declaring `CONTESTED`.

Expand All @@ -254,7 +254,7 @@ The SANS-canonical knowledge an agent must internalise. Encoded in `verdict/play

## 9. Ledger discipline

`verdict/ledger/writer.py` + `chain.py`. The ledger is the chain-of-custody artifact a SANS judge will scrutinise.
`src/verdict/ledger/writer.py`. The ledger is the chain-of-custody artifact a SANS judge will scrutinise.

- JSONL append-only at `cases/<id>/ledger.jsonl`. `prev_entry_hash` chains entries; `verdict validate <case_id>` walks them.
- Three-tier IDs on every entry: `case_id` → `langfuse_trace_id` → `langgraph_checkpoint_id`.
Expand All @@ -275,7 +275,7 @@ Event types: `case_init`, `tool_call`, `finding`, `approval`, `rejection`, `mode

```bash
# Microsandbox
curl -sSL https://get.microsandbox.dev | sh
curl -fsSL https://install.microsandbox.dev | sh

# Bootstrap (three-credential-path detection)
bash scripts/install.sh
Expand Down Expand Up @@ -316,7 +316,7 @@ verdict health

```bash
# Schema/playbook/knowledge gates (W1)
uv run --directory services/agent pytest tests/schemas/ -v
uv run pytest tests/schemas/ -v
uv run pytest tests/playbooks/ -v
uv run pytest tests/knowledge/ -v

Expand Down
8 changes: 4 additions & 4 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ New-contributor setup guide. If you've already done the SANS Find Evil! 2026 tea
**Default branch:** `main`
**License:** MIT
**Deadline gate:** 2026-06-15 22:45 CDT (team-internal target: 2026-06-14 EOD = ~28 h buffer)
**Recommended platform:** SANS **SIFT Workstation VM** (canonical — all forensic tools, Microsandbox, SGLang, evidence mounts work out-of-box) **or** any modern **Linux box** (Ubuntu 22.04+ / Debian 12+ / Fedora 39+ / Arch). **macOS host is acceptable** for schema/planner/MCP/unit-test work that doesn't shell out to forensic tools — but the moment your task touches `verdict/sandboxes/`, `verdict/tools/vol3/`, or anything that runs in Microsandbox, switch into the SIFT VM. **Windows host is not supported** for development; use WSL2 + Ubuntu or pull the SIFT VM (it runs under Hyper-V / VMware / VirtualBox).
**Recommended platform:** SANS **SIFT Workstation VM** (canonical — all forensic tools, Microsandbox, SGLang, evidence mounts work out-of-box) **or** any modern **Linux box** (Ubuntu 22.04+ / Debian 12+ / Fedora 39+ / Arch). **macOS host is acceptable** for schema/planner/MCP/unit-test work that doesn't shell out to forensic tools — but the moment your task touches `src/verdict/sandboxes/`, `src/verdict/tools/`, or anything that runs in Microsandbox, switch into the SIFT VM. **Windows host is not supported** for development; use WSL2 + Ubuntu or pull the SIFT VM (it runs under Hyper-V / VMware / VirtualBox).

Authority chain when docs disagree: Devpost rules → `DEVPOST_COMPLIANCE.md` → `ARCHITECTURE.md` → `BUILD_PLAN.md` → this file. Code + lockfiles win over docs (per `CLAUDE.md`); update the doc, don't roll back the code, unless the code is wrong.

Expand Down Expand Up @@ -118,13 +118,13 @@ Idempotent. Installs uv + Python 3.11, Rust 1.88, Node 20 + pnpm, and Microsandb

**Where to develop.** Three supported configurations, in order of preference:

1. **Inside the SIFT VM** (canonical, recommended for everyone). All forensic tools, Microsandbox, SGLang, and the evidence mounts resolve here without setup. **Required** for any work touching the executor branches, the Microsandbox layer, evidence I/O, or anything under `verdict/sandboxes/`, `verdict/tools/vol3/`, or `services/mcp/src/tools/`. Pull the OVA from `downloads/README.md`; snapshot it as `clean-install` before installing anything.
1. **Inside the SIFT VM** (canonical, recommended for everyone). All forensic tools, Microsandbox, SGLang, and the evidence mounts resolve here without setup. **Required** for any work touching the executor branches, the Microsandbox layer, evidence I/O, or anything under `src/verdict/sandboxes/`, `src/verdict/tools/`. Pull the OVA from `downloads/README.md`; snapshot it as `clean-install` before installing anything.
2. **A modern Linux box** (Ubuntu 22.04+ / Debian 12+ / Fedora 39+ / Arch). Acceptable for the full stack as long as you can install Microsandbox (Linux-only) and run the SIFT toolchain (`apt install sleuthkit volatility ...`). Faster I/O than the VM. Take a `pre-verdict` snapshot of your home before installing forensic tools — they leave state.
3. **macOS host with the SIFT VM as a runtime target** (faster edit loop, smaller surface). Use VS Code Remote-SSH or `Develop on a Container` into the VM. Acceptable for schema, planner, MCP gateway, and unit-test work that doesn't shell out to forensic tools or Microsandbox. The moment your task touches a Microsandbox path or a forensic CLI, switch into the VM.

**Windows host is not supported** for development. Either use WSL2 + Ubuntu (treat as configuration #2) or run the SIFT VM under Hyper-V / VMware / VirtualBox (configuration #1). Native Windows breaks Microsandbox (libkrun is Linux-only), pre-commit hooks (path/exec semantics), and several SIFT tools.

If your work touches `services/mcp/src/tools/`, `services/agent_mcp/`, or anything under a Microsandbox path, you **must** run integration tests inside the SIFT VM (or your Linux box with Microsandbox installed) before opening a PR. macOS-only test runs do not satisfy the gate.
If your work touches `src/verdict/tools/`, `src/verdict/sandboxes/`, or anything under a Microsandbox path, you **must** run integration tests inside the SIFT VM (or your Linux box with Microsandbox installed) before opening a PR. macOS-only test runs do not satisfy the gate.

---

Expand Down Expand Up @@ -271,7 +271,7 @@ If the run fails before producing a Finding, you have a P0 environment problem.
Authority order (escalate left-to-right):

1. **NotebookLM Q&A:** https://notebooklm.google.com/notebook/f0957a60-6fb2-452b-93d4-ecd73ba47779?authuser=1 — chief location for "how does X work?"
2. **`docs/` and the audit history in `archive/`** — most "why did we choose X?" questions are answered in `archive/03-audit-v4.5.md`.
2. **`docs/` and the audit history in `docs/spec/`** — most "why did we choose X?" questions are answered in `docs/spec/03-audit-v4.5.md`.
3. **Team chat** — PUG / Beaver / Haley / KP. Use the appropriate thread; don't DM PUG for things the team should see.
4. **Devpost platform issues only:** https://help.devpost.com/

Expand Down
54 changes: 26 additions & 28 deletions docs/ARCHITECTURE.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ START
│ │ parsed_success_criteria_hash. Mismatch
│ │ → clarify sub-state (re-prompts within
│ │ the same node, not a separate top-level
│ │ node — total node count stays 9).
│ │ node — total node count stays 8).
└────────┬────────┘
▼ (fanout — 4 parallel branches)
┌─────────────────┐
Expand Down Expand Up @@ -192,64 +192,62 @@ The differentiator vs. competitors. Schema validators reject sloppy findings bef
class Finding(BaseModel):
artifact_paths: list[Path] = Field(min_length=2)
artifact_classes: list[ArtifactClass] = Field(min_length=2)
caveats_acknowledged: list[CaveatID] = []
caveats_acknowledged: list[CaveatID] = Field(default_factory=list)
mitre_technique: str | None # validated against ^T\d{4}(\.\d{3})?$

@model_validator(mode="after")
def _execution_claims_need_two_classes(self):
is_exec = any(
self.mitre_technique and self.mitre_technique.startswith(p)
for p in ("T1059", "T1106", "T1204", "T1218", "T1543", "T1547")
def _forensic_corroboration(self):
is_exec = self.mitre_technique and self.mitre_technique.startswith(
("T1059", "T1106", "T1204", "T1218", "T1543", "T1547")
)
if is_exec and len(set(self.artifact_classes)) < 2:
raise ValueError(f"execution claim needs ≥2 distinct artifact classes")
return self

@model_validator(mode="after")
def _amcache_caveat_required(self):
if ArtifactClass.AMCACHE in self.artifact_classes:
if CaveatID.AMCACHE_LASTMODIFIED_NOT_EXEC not in self.caveats_acknowledged:
raise ValueError("Finding cites Amcache without LastModified caveat")
raise ValueError("execution claims require two distinct artifact classes")
acknowledged = set(self.caveats_acknowledged)
for artifact_class, required_caveat in AVAILABLE_CAVEAT_TRIGGERS.items():
if artifact_class in self.artifact_classes and required_caveat not in acknowledged:
raise ValueError(f"{required_caveat.value} must be acknowledged")
return self
```

### CaveatID — Tier-1 examiner caveats from `CLAUDE.md` §3.3

```python
class CaveatID(str, Enum):
AMCACHE_LASTMODIFIED_NOT_EXEC = "amcache_lastmodified_neq_execution"
SHIMCACHE_ORDER_CHANGED_WIN81 = "shimcache_order_lru_pre81_insertion_post81"
PREFETCH_SSD_DISABLED = "prefetch_disabled_on_ssd_or_gpo"
MFT_SI_STOMPABLE = "mft_si_timestomp_use_fn"
USNJRNL_WRAPS = "usnjrnl_wraps_treat_gaps_carefully"
LOGON_TYPE_3_VS_10 = "evtx_4624_type3_network_neq_type10_rdp"
SYSMON_PROCESSGUID_OVER_PID = "sysmon_processguid_correlation_key_not_pid"
class CaveatID(StrEnum):
AMCACHE_LASTMODIFIED_NOT_EXEC = "AMCACHE_LASTMODIFIED_NOT_EXEC"
SHIMCACHE_ORDER_CHANGED_WIN81 = "SHIMCACHE_ORDER_CHANGED_WIN81"
PREFETCH_SSD_DISABLED = "PREFETCH_SSD_DISABLED"
MFT_SI_STOMPABLE = "MFT_SI_STOMPABLE"
USNJRNL_WRAPS = "USNJRNL_WRAPS"
LOGON_TYPE_3_VS_10 = "LOGON_TYPE_3_VS_10"
SYSMON_PROCESSGUID_OVER_PID = "SYSMON_PROCESSGUID_OVER_PID"
```

Loaded into every executor system prompt via `verdict/planning/prompts/examiner_caveats.md`.
Loaded into every executor system prompt via `src/verdict/planning/prompts/examiner_caveats.md`.

### ArtifactClass — multi-source corroboration vocabulary

```python
class ArtifactClass(str, Enum):
class ArtifactClass(StrEnum):
PREFETCH = "prefetch"
AMCACHE = "amcache"
SHIMCACHE = "shimcache"
EVTX_4624 = "evtx_4624"
EVTX_4688 = "evtx_4688"
SYSMON_1 = "sysmon_1"
NETWORK = "network"
REGISTRY_RUN = "registry_run"
TASK_SCHEDULER = "task_scheduler"
WMI_SUBSCRIPTION = "wmi_subscription"
MFT = "mft"
USNJRNL = "usnjrnl"
PROCESS_MEMORY = "process_memory"
YARA_HIT = "yara_hit"
SIGMA_HIT = "sigma_hit"
```

### Playbooks — SANS canonical tool sequencing

Three YAMLs in `verdict/playbooks/` (memory.yml / disk.yml / triage.yml) encode the SANS-canonical sequencing summarized in `CLAUDE.md` §7 and this document's forensic doctrine. Loaded into planner system prompt at case_init based on detected evidence type.
Three YAMLs in `src/verdict/playbooks/` (memory.yml / disk.yml / triage.yml) encode the SANS-canonical sequencing summarized in `CLAUDE.md` §7 and this document's forensic doctrine. Loaded into planner system prompt at case_init based on detected evidence type.

`memory.yml` example rule (DKOM detection):
```yaml
Expand All @@ -263,7 +261,7 @@ This is one of the architecture's clearest moats — DKOM/T1014 detection auto-f

### Hunt Evil baseline

`verdict/knowledge/hunt_evil.yml` keyed by process name with expected parent / path / signing / instance count for 8 canonical Windows processes (svchost, lsass, csrss, winlogon, services, wininit, explorer, smss). `ProcessBaselineAnomaly` Hypothesis subtype maps to `T1036.005` (Match Legitimate Name or Location). Catches `scvhost.exe` with parent `cmd.exe` automatically.
`src/verdict/knowledge/hunt_evil.yml` keyed by process name with expected parent / path / signing / instance count for 8 canonical Windows processes (svchost, lsass, csrss, winlogon, services, wininit, explorer, smss). `ProcessBaselineAnomaly` Hypothesis subtype maps to `T1036.005` (Match Legitimate Name or Location). Catches `scvhost.exe` with parent `cmd.exe` automatically.

---

Expand Down Expand Up @@ -530,5 +528,5 @@ These were raised during the v4.4 research and v4.5 system-design review. They'r
| v4.4 agentic + DFIR research findings (raw) | `docs/spec/02-audit-v4.4.md` |
| v4.6 schema patches (raw spec) | `docs/spec/04-spec-plan-v4.6.md` |
| Project-wide build conventions | `../CLAUDE.md` |
| Tier-1 examiner caveat source | `../CLAUDE.md` §3.3 and planned `../verdict/planning/prompts/examiner_caveats.md` |
| Tool sequencing playbook source | `../verdict/playbooks/*.yml` and `docs/ARCHITECTURE.md` §4 |
| Tier-1 examiner caveat source | `../CLAUDE.md` §3.3 and `../src/verdict/planning/prompts/examiner_caveats.md` |
| Tool sequencing playbook source | `../src/verdict/playbooks/*.yml` and `docs/ARCHITECTURE.md` §4 |
Loading