[d3vhex]

Why

Measuring ATT&CK coverage by tactic instead of counting rules gave an
uncomfortable answer:

collection      0
exfiltration    0
impact          1

Every rule we shipped watched an intruder arriving, gaining privilege and
staying. Nothing watched them leaving with anything. A host could be swept,
archived and uploaded and the console would stay quiet — while the coverage
page reported 37 rules and 38 techniques, which reads as thorough.

Six rules for the end of the chain

Rule	Tactic	Techniques
Local data collected into an archive in a staging directory	collection	T1560.001, T1074.001
Clipboard and screen captured	collection	T1113, T1115
File uploaded to a web service from a script host	exfiltration	T1567.002, T1048.003
Data staged in an archive (Linux)	collection	T1560.001, T1074.001
File pushed off the host over a non-interactive channel	exfiltration	T1048, T1567.002
Recovery and protection services stopped	impact	T1489, T1562.001

collection 3, exfiltration 2, impact 2; 43 rules, 47 techniques.

Each rule keys on a pairing rather than a tool, because the tools are
ordinary. rar a is a backup job; rar a -hp into %TEMP% is not.
Invoke-RestMethod is how our own automation talks to our own APIs;
-Method Post -InFile to transfer.sh is not.

One branch was written and then deliberately deleted: scp/rsync in the
Linux exfiltration rule fired on rsync -az --delete /srv/data/ backup-01:/srv/data/, which is a scheduled backup. A rule that pages on
every backup gets muted, and a muted rule still counts as coverage — worse
than not shipping it.

The invariant that found eleven older gaps

There was a test asserting every Linux rule is exercised by a corpus case.
There was no Windows equivalent. Adding one found 13 unexercised Windows
rules, 11 of them pre-existing — rules that had never fired against
anything, and nothing could say so.

CORPUS_FIELDS had no Image mapping, so a rule keyed on the child process
image (win_office_spawns_shell) could not be expressed by any corpus case.
It was unexercisable by construction.

The corpus grew 19 → 35 cases, plus six evidence samples. Writing them found
real detection bugs in rules both new and old:

• net.exe stop did not match a rule looking for net stop
• cat matched inside ncat , so a netcat reverse shell fired the
  exfiltration rule
• tar matched inside Start-Sleep, so a clipboard loop fired the archive
  rule
• ' -F ' collided with certutil -f under case-insensitive matching

The AI triage layer was matching text the agent does not produce

ai/criteria.py verifies the criterion the model claims, so a supported
criterion forces CRITICAL. Three of its markers could never have fired:

"the audit log was cleared"      (EID 1102)
"a service was installed"        (EID 7045)

Those are message templates. Windows stores the fields and renders the
sentence when something asks; the collector joins StringInserts with " | "
and never asks — 1102 arrives as four fields naming the account, 7045 as
service name, image path, type, start type, account. No prose in either. The
markers matched hand-written corpus lines and nothing else, and both events
have their own Sigma rule keyed on the event id, so one layer covered it while
the other reported coverage it did not have.

Added the eid= forms alongside the phrases, plus markers for the gaps the
new corpus cases exposed: ntdsutil ifm (the supported way to extract the AD
database never names ntds.dit), wevtutil cl, amsiInitFailed,
advfirewall state off, certutil -urlcache.

Removed one. currentversion\run + \appdata\ was still there — the same
alternative the comment above it says was removed, written longer. It matched
constructed:t1547-run-key, which the corpus grades SUSPICIOUS, and a
forced CRITICAL made that case unscoreable: no answer from the model could
win. Two things hid it — the test used a sample with CurrentVersion elided
from the key path, so the first marker was absent and the assertion passed for
the wrong reason, and the benign-corpus test only looks at NOT_CRITICAL
rows.

The coverage test now distinguishes a gap from a decision

test_the_markers_cover_the_constructed_attacks asserted every corpus
positive matches a criterion. That made sense when the corpus was eight cases
written beside the markers. It is now also what proves every Sigma rule fires,
so it grows with every rule — and the assertion had come to read "every new
rule needs a marker beside it".

That is the wrong direction. A marker forces CRITICAL and skips the confidence
gate. net user /add is a real technique and also what the helpdesk does all
morning; net group "Domain Admins" is the first thing an administrator types
and the first thing an intruder types. Those belong to the rules, which can
score and be tuned.

Replaced with three tests and a written-down list of the eight cases the
criteria deliberately do not cover, each with its reason:

• everything not on the list must match a marker
• everything on the list must still exist in the corpus, and must not
  match — a marker widening until it covers a case somebody argued out is how
  appdata got back in
• everything on the list must fire a Sigma rule, so "the rules handle it"
  is checked rather than assumed

Plus test_no_case_below_critical_is_forced_to_critical, which covers the
SUSPICIOUS band both existing benign tests skipped.

Testing

2518 passed, 5 skipped. The five are PTY tests on a POSIX host and one
correlation shape Sigma cannot express.