From cf4ee0814833b8ab9b321a58c0ae339595fe6119 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 12 Oct 2026 19:32:37 +0000 Subject: [PATCH 1/2] skill: describe rook 0.1.11 Rook-Skill-Coverage: [{"id":"sync-orphans","action":"edit","reason":"Added a note to scenarios.md that sync stops before uploading when a scenario names a feature the agent lacks, lists the fix, and must succeed before an upstream run.","paths":["skill-installer/skills/references/scenarios.md"]},{"id":"registered-session","action":"edit","reason":"Added an errors bullet saying such commands refuse before running or charging when the session is unregistered, and should be reported as no run and no spend.","paths":["skill-installer/skills/references/headless-contract.md"]},{"id":"session-transcript","action":"edit","reason":"Extended the home-state paragraph to say transcript.jsonl records raw input and output of every role call, including refused, failed and interrupted calls.","paths":["skill-installer/skills/references/headless-contract.md"]},{"id":"client-headers","action":"no-edit","reason":"Request headers and debug client display are internal transport details. The skill neither uses nor describes them for driving or interpreting runs.","paths":[]},{"id":"owasp-mapping","action":"edit","reason":"Added to scenarios.md that red-team scenarios carry OWASP LLM, ASI and MITRE ATLAS references, and that generate writes and names unrunnable scenarios with a reason instead of dropping them.","paths":["skill-installer/skills/references/scenarios.md"]},{"id":"redteam-reports","action":"edit","reason":"Added to verdicts.md that a grounded partial compromise counts as Fail and that reports keep refusals, interrupted runs and coverage gaps for the summary.","paths":["skill-installer/skills/references/verdicts.md"]},{"id":"cli-text","action":"no-edit","reason":"SKILL.md and ci.md already teach sync before run, and troubleshooting.md already routes the update remedy through rook update. The guide and help wording changes alter no command, flag or result the skill relies on.","paths":["skill-installer/skills/SKILL.md","skill-installer/skills/references/ci.md","skill-installer/skills/references/troubleshooting.md"]},{"id":"logout-note","action":"no-edit","reason":"The skill never runs rook logout. ci.md already explains that LT_USERNAME and LT_ACCESS_KEY sign commands in ahead of any stored sign-in.","paths":["skill-installer/skills/references/ci.md"]},{"id":"consent-logo","action":"no-edit","reason":"The logo on the consent page and the one-time OAuth client re-registration are cosmetic. They do not change the login commands or remedies the skill documents.","paths":[]},{"id":"boot-banner","action":"no-edit","reason":"The TUI boot banner mascot is purely visual. The skill drives the CLI headlessly and does not describe the banner.","paths":[]}] --- .agents/skills/rook/references/headless-contract.md | 7 ++++++- .agents/skills/rook/references/scenarios.md | 9 +++++++++ .agents/skills/rook/references/verdicts.md | 4 +++- .claude/skills/rook/references/headless-contract.md | 7 ++++++- .claude/skills/rook/references/scenarios.md | 9 +++++++++ .claude/skills/rook/references/verdicts.md | 4 +++- skill-installer/skills/references/headless-contract.md | 7 ++++++- skill-installer/skills/references/scenarios.md | 9 +++++++++ skill-installer/skills/references/verdicts.md | 4 +++- 9 files changed, 54 insertions(+), 6 deletions(-) diff --git a/.agents/skills/rook/references/headless-contract.md b/.agents/skills/rook/references/headless-contract.md index cb636f6..887d010 100644 --- a/.agents/skills/rook/references/headless-contract.md +++ b/.agents/skills/rook/references/headless-contract.md @@ -48,6 +48,9 @@ Use `error`, or `reason` when present. Translate remedy tokens with - `halted: true` can retain a report and exit 0. Present it as incomplete. - Gate refusals on `report`, `status` and `ask`, and parser errors, may leave stdout empty. Use the stderr diagnostic rather than inventing an error object. +- A command that calls the model controller or opens an upstream job refuses + before anything runs or is charged when its session could not be registered. + Report it as no run and no spend, using the diagnostic it returns. ## Grants and accounting @@ -87,4 +90,6 @@ assuming either form. Under its `agents//` directory: | `runs//scenarios//` | `snapshot.yaml`, `request.json`, `response.json`, artifacts and `verdict.yaml` | Home state at `~/.testmuai/rook/` includes credentials, grants and session -transcripts. Keep it outside commits and shared evidence. +transcripts. Each session's `transcript.jsonl` records the raw input and output +of every role call, including refused, failed and interrupted calls. Keep it +outside commits and shared evidence. diff --git a/.agents/skills/rook/references/scenarios.md b/.agents/skills/rook/references/scenarios.md index cd34421..ba01ecc 100644 --- a/.agents/skills/rook/references/scenarios.md +++ b/.agents/skills/rook/references/scenarios.md @@ -17,6 +17,8 @@ them, though existing scenario files that use them still load and run. Every scenario also carries free-form tags; `run --tag ` selects on them. +Red-team scenarios also carry OWASP LLM, OWASP Agentic (ASI) and MITRE ATLAS +references. ## Generating @@ -38,6 +40,11 @@ different feature, `generate` warns and saves that scenario excluded; review it in `rook scenarios list --json` and bring it back with `rook scenarios include ` if it belongs in runs. +`rook sync` stops before uploading anything when a scenario names a feature its +agent does not carry, and lists each one with its agent, the missing feature and +a fix. Apply the listed fix and sync again; do not run against upstream until +sync succeeds. + ## Listing and curating ```bash @@ -56,6 +63,8 @@ multiple turns without an observed conversation handle, or token-economy grading without reported usage, is skipped by `run` and counts toward the assurance gap, not toward Fail. File and image requirements are not rejected solely by their input kind: inspect what the profile script can actually send. +`generate` writes a scenario that cannot run and names it with a reason instead +of dropping it; report those as coverage gaps. ## Scoping a run diff --git a/.agents/skills/rook/references/verdicts.md b/.agents/skills/rook/references/verdicts.md index 11fd55f..448aa8f 100644 --- a/.agents/skills/rook/references/verdicts.md +++ b/.agents/skills/rook/references/verdicts.md @@ -15,7 +15,9 @@ criterion details, read `runs//scenarios//verdict.yaml`. Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran -has no verdict; do not turn its absence into a failure. +has no verdict; do not turn its absence into a failure. A grounded partial +compromise counts as Fail. Reports keep refusals, interrupted runs and coverage +gaps; carry them into the summary rather than dropping them. ## Pass rate diff --git a/.claude/skills/rook/references/headless-contract.md b/.claude/skills/rook/references/headless-contract.md index cb636f6..887d010 100644 --- a/.claude/skills/rook/references/headless-contract.md +++ b/.claude/skills/rook/references/headless-contract.md @@ -48,6 +48,9 @@ Use `error`, or `reason` when present. Translate remedy tokens with - `halted: true` can retain a report and exit 0. Present it as incomplete. - Gate refusals on `report`, `status` and `ask`, and parser errors, may leave stdout empty. Use the stderr diagnostic rather than inventing an error object. +- A command that calls the model controller or opens an upstream job refuses + before anything runs or is charged when its session could not be registered. + Report it as no run and no spend, using the diagnostic it returns. ## Grants and accounting @@ -87,4 +90,6 @@ assuming either form. Under its `agents//` directory: | `runs//scenarios//` | `snapshot.yaml`, `request.json`, `response.json`, artifacts and `verdict.yaml` | Home state at `~/.testmuai/rook/` includes credentials, grants and session -transcripts. Keep it outside commits and shared evidence. +transcripts. Each session's `transcript.jsonl` records the raw input and output +of every role call, including refused, failed and interrupted calls. Keep it +outside commits and shared evidence. diff --git a/.claude/skills/rook/references/scenarios.md b/.claude/skills/rook/references/scenarios.md index cd34421..ba01ecc 100644 --- a/.claude/skills/rook/references/scenarios.md +++ b/.claude/skills/rook/references/scenarios.md @@ -17,6 +17,8 @@ them, though existing scenario files that use them still load and run. Every scenario also carries free-form tags; `run --tag ` selects on them. +Red-team scenarios also carry OWASP LLM, OWASP Agentic (ASI) and MITRE ATLAS +references. ## Generating @@ -38,6 +40,11 @@ different feature, `generate` warns and saves that scenario excluded; review it in `rook scenarios list --json` and bring it back with `rook scenarios include ` if it belongs in runs. +`rook sync` stops before uploading anything when a scenario names a feature its +agent does not carry, and lists each one with its agent, the missing feature and +a fix. Apply the listed fix and sync again; do not run against upstream until +sync succeeds. + ## Listing and curating ```bash @@ -56,6 +63,8 @@ multiple turns without an observed conversation handle, or token-economy grading without reported usage, is skipped by `run` and counts toward the assurance gap, not toward Fail. File and image requirements are not rejected solely by their input kind: inspect what the profile script can actually send. +`generate` writes a scenario that cannot run and names it with a reason instead +of dropping it; report those as coverage gaps. ## Scoping a run diff --git a/.claude/skills/rook/references/verdicts.md b/.claude/skills/rook/references/verdicts.md index 11fd55f..448aa8f 100644 --- a/.claude/skills/rook/references/verdicts.md +++ b/.claude/skills/rook/references/verdicts.md @@ -15,7 +15,9 @@ criterion details, read `runs//scenarios//verdict.yaml`. Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran -has no verdict; do not turn its absence into a failure. +has no verdict; do not turn its absence into a failure. A grounded partial +compromise counts as Fail. Reports keep refusals, interrupted runs and coverage +gaps; carry them into the summary rather than dropping them. ## Pass rate diff --git a/skill-installer/skills/references/headless-contract.md b/skill-installer/skills/references/headless-contract.md index cb636f6..887d010 100644 --- a/skill-installer/skills/references/headless-contract.md +++ b/skill-installer/skills/references/headless-contract.md @@ -48,6 +48,9 @@ Use `error`, or `reason` when present. Translate remedy tokens with - `halted: true` can retain a report and exit 0. Present it as incomplete. - Gate refusals on `report`, `status` and `ask`, and parser errors, may leave stdout empty. Use the stderr diagnostic rather than inventing an error object. +- A command that calls the model controller or opens an upstream job refuses + before anything runs or is charged when its session could not be registered. + Report it as no run and no spend, using the diagnostic it returns. ## Grants and accounting @@ -87,4 +90,6 @@ assuming either form. Under its `agents//` directory: | `runs//scenarios//` | `snapshot.yaml`, `request.json`, `response.json`, artifacts and `verdict.yaml` | Home state at `~/.testmuai/rook/` includes credentials, grants and session -transcripts. Keep it outside commits and shared evidence. +transcripts. Each session's `transcript.jsonl` records the raw input and output +of every role call, including refused, failed and interrupted calls. Keep it +outside commits and shared evidence. diff --git a/skill-installer/skills/references/scenarios.md b/skill-installer/skills/references/scenarios.md index cd34421..ba01ecc 100644 --- a/skill-installer/skills/references/scenarios.md +++ b/skill-installer/skills/references/scenarios.md @@ -17,6 +17,8 @@ them, though existing scenario files that use them still load and run. Every scenario also carries free-form tags; `run --tag ` selects on them. +Red-team scenarios also carry OWASP LLM, OWASP Agentic (ASI) and MITRE ATLAS +references. ## Generating @@ -38,6 +40,11 @@ different feature, `generate` warns and saves that scenario excluded; review it in `rook scenarios list --json` and bring it back with `rook scenarios include ` if it belongs in runs. +`rook sync` stops before uploading anything when a scenario names a feature its +agent does not carry, and lists each one with its agent, the missing feature and +a fix. Apply the listed fix and sync again; do not run against upstream until +sync succeeds. + ## Listing and curating ```bash @@ -56,6 +63,8 @@ multiple turns without an observed conversation handle, or token-economy grading without reported usage, is skipped by `run` and counts toward the assurance gap, not toward Fail. File and image requirements are not rejected solely by their input kind: inspect what the profile script can actually send. +`generate` writes a scenario that cannot run and names it with a reason instead +of dropping it; report those as coverage gaps. ## Scoping a run diff --git a/skill-installer/skills/references/verdicts.md b/skill-installer/skills/references/verdicts.md index 11fd55f..448aa8f 100644 --- a/skill-installer/skills/references/verdicts.md +++ b/skill-installer/skills/references/verdicts.md @@ -15,7 +15,9 @@ criterion details, read `runs//scenarios//verdict.yaml`. Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran -has no verdict; do not turn its absence into a failure. +has no verdict; do not turn its absence into a failure. A grounded partial +compromise counts as Fail. Reports keep refusals, interrupted runs and coverage +gaps; carry them into the summary rather than dropping them. ## Pass rate From 4ce9689daad54762e4f409ebce044a44a9605913 Mon Sep 17 00:00:00 2001 From: samyakLambda Date: Tue, 13 Oct 2026 15:31:37 +0530 Subject: [PATCH 2/2] skill: name kept-unrunnable scenarios, the per-pattern cap and red-team hooks - generate keeps unrunnable scenarios with executable: false and a skip_reason prefix; coverage-gaps.yaml lists patterns with no scenario. - Red-team patterns that need fixture seeding, a collect hook or a fresh session are skipped until the profile has that hook. - run takes at most two scenarios per attack pattern unless --only. - report --json can carry coverage_gaps, redteam_coverage and summary_claims; an older stored Partial reads as Unable to Verify. - "In 0.1.6" notes that still hold now say "Since 0.1.6". Co-Authored-By: Claude Opus 5.5 (1M context) --- .../skills/rook/references/headless-contract.md | 2 +- .agents/skills/rook/references/scenarios.md | 16 ++++++++++++++-- .agents/skills/rook/references/verdicts.md | 7 +++++-- .../skills/rook/references/headless-contract.md | 2 +- .claude/skills/rook/references/scenarios.md | 16 ++++++++++++++-- .claude/skills/rook/references/verdicts.md | 7 +++++-- .../skills/references/headless-contract.md | 2 +- skill-installer/skills/references/scenarios.md | 16 ++++++++++++++-- skill-installer/skills/references/verdicts.md | 7 +++++-- 9 files changed, 60 insertions(+), 15 deletions(-) diff --git a/.agents/skills/rook/references/headless-contract.md b/.agents/skills/rook/references/headless-contract.md index 887d010..03cd955 100644 --- a/.agents/skills/rook/references/headless-contract.md +++ b/.agents/skills/rook/references/headless-contract.md @@ -68,7 +68,7 @@ still use the network, change state or invoke the target. ## On disk -In 0.1.6, `.testmuai/rook/settings.json` keeps a project selection per +Since 0.1.6, `.testmuai/rook/settings.json` keeps a project selection per environment in `selections[environment]`. After a switch, rook uses the new environment's selection, which may be empty even when another environment has one. Run `rook doctor` to see the active environment and diff --git a/.agents/skills/rook/references/scenarios.md b/.agents/skills/rook/references/scenarios.md index ba01ecc..8756ca4 100644 --- a/.agents/skills/rook/references/scenarios.md +++ b/.agents/skills/rook/references/scenarios.md @@ -63,8 +63,16 @@ multiple turns without an observed conversation handle, or token-economy grading without reported usage, is skipped by `run` and counts toward the assurance gap, not toward Fail. File and image requirements are not rejected solely by their input kind: inspect what the profile script can actually send. -`generate` writes a scenario that cannot run and names it with a reason instead -of dropping it; report those as coverage gaps. +`generate` keeps a scenario it cannot make runnable instead of dropping it: +`executable: false` with a `skip_reason` starting `coverage gap:`, +`invalid scenario:` or `invalid red-team scenario:`. Attack patterns with no +scenario for a feature are listed in the agent's `coverage-gaps.yaml`. Report +both as gaps, not as failures. + +Some attack patterns are skipped until the profile has the hook they need, +and the skip reason names it: an `open` hook that seeds the fixture in +`$ROOK_STATE_DIR/attack-fixture.json`, a `collect` hook that reports the +observed control and outcome, or `open` and `close` hooks for a fresh session. ## Scoping a run @@ -79,6 +87,10 @@ rook run --run --phases collect,judge --json # collect evidence that la rook run --resume --json # carry finished work into a new run ``` +Without `--only`, a run selects at most two scenarios per attack pattern; the +rest are skipped as `pattern already has 2 candidates in this run`. Name +them with `--only` to run more. + `--only` takes one comma-separated argument; repeating the flag keeps only the last occurrence, and a space-separated second id is read as instruction text. Each scenario runs once; there is no repeat count. diff --git a/.agents/skills/rook/references/verdicts.md b/.agents/skills/rook/references/verdicts.md index 448aa8f..567610a 100644 --- a/.agents/skills/rook/references/verdicts.md +++ b/.agents/skills/rook/references/verdicts.md @@ -17,11 +17,14 @@ Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran has no verdict; do not turn its absence into a failure. A grounded partial compromise counts as Fail. Reports keep refusals, interrupted runs and coverage -gaps; carry them into the summary rather than dropping them. +gaps; carry them into the summary rather than dropping them. `report --json` +can add `coverage_gaps`, `redteam_coverage` (OWASP coverage of the scenario +inventory) and `summary_claims`; an unmapped control is unverified, not passed. +A **Partial** stored by an older run reads as Unable to Verify. ## Pass rate -In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. +Since 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. `totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the pass-rate denominator as a recorded result, but remains a separate verdict; never describe it as Fail. If every recorded result is Unable to Verify, diff --git a/.claude/skills/rook/references/headless-contract.md b/.claude/skills/rook/references/headless-contract.md index 887d010..03cd955 100644 --- a/.claude/skills/rook/references/headless-contract.md +++ b/.claude/skills/rook/references/headless-contract.md @@ -68,7 +68,7 @@ still use the network, change state or invoke the target. ## On disk -In 0.1.6, `.testmuai/rook/settings.json` keeps a project selection per +Since 0.1.6, `.testmuai/rook/settings.json` keeps a project selection per environment in `selections[environment]`. After a switch, rook uses the new environment's selection, which may be empty even when another environment has one. Run `rook doctor` to see the active environment and diff --git a/.claude/skills/rook/references/scenarios.md b/.claude/skills/rook/references/scenarios.md index ba01ecc..8756ca4 100644 --- a/.claude/skills/rook/references/scenarios.md +++ b/.claude/skills/rook/references/scenarios.md @@ -63,8 +63,16 @@ multiple turns without an observed conversation handle, or token-economy grading without reported usage, is skipped by `run` and counts toward the assurance gap, not toward Fail. File and image requirements are not rejected solely by their input kind: inspect what the profile script can actually send. -`generate` writes a scenario that cannot run and names it with a reason instead -of dropping it; report those as coverage gaps. +`generate` keeps a scenario it cannot make runnable instead of dropping it: +`executable: false` with a `skip_reason` starting `coverage gap:`, +`invalid scenario:` or `invalid red-team scenario:`. Attack patterns with no +scenario for a feature are listed in the agent's `coverage-gaps.yaml`. Report +both as gaps, not as failures. + +Some attack patterns are skipped until the profile has the hook they need, +and the skip reason names it: an `open` hook that seeds the fixture in +`$ROOK_STATE_DIR/attack-fixture.json`, a `collect` hook that reports the +observed control and outcome, or `open` and `close` hooks for a fresh session. ## Scoping a run @@ -79,6 +87,10 @@ rook run --run --phases collect,judge --json # collect evidence that la rook run --resume --json # carry finished work into a new run ``` +Without `--only`, a run selects at most two scenarios per attack pattern; the +rest are skipped as `pattern already has 2 candidates in this run`. Name +them with `--only` to run more. + `--only` takes one comma-separated argument; repeating the flag keeps only the last occurrence, and a space-separated second id is read as instruction text. Each scenario runs once; there is no repeat count. diff --git a/.claude/skills/rook/references/verdicts.md b/.claude/skills/rook/references/verdicts.md index 448aa8f..567610a 100644 --- a/.claude/skills/rook/references/verdicts.md +++ b/.claude/skills/rook/references/verdicts.md @@ -17,11 +17,14 @@ Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran has no verdict; do not turn its absence into a failure. A grounded partial compromise counts as Fail. Reports keep refusals, interrupted runs and coverage -gaps; carry them into the summary rather than dropping them. +gaps; carry them into the summary rather than dropping them. `report --json` +can add `coverage_gaps`, `redteam_coverage` (OWASP coverage of the scenario +inventory) and `summary_claims`; an unmapped control is unverified, not passed. +A **Partial** stored by an older run reads as Unable to Verify. ## Pass rate -In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. +Since 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. `totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the pass-rate denominator as a recorded result, but remains a separate verdict; never describe it as Fail. If every recorded result is Unable to Verify, diff --git a/skill-installer/skills/references/headless-contract.md b/skill-installer/skills/references/headless-contract.md index 887d010..03cd955 100644 --- a/skill-installer/skills/references/headless-contract.md +++ b/skill-installer/skills/references/headless-contract.md @@ -68,7 +68,7 @@ still use the network, change state or invoke the target. ## On disk -In 0.1.6, `.testmuai/rook/settings.json` keeps a project selection per +Since 0.1.6, `.testmuai/rook/settings.json` keeps a project selection per environment in `selections[environment]`. After a switch, rook uses the new environment's selection, which may be empty even when another environment has one. Run `rook doctor` to see the active environment and diff --git a/skill-installer/skills/references/scenarios.md b/skill-installer/skills/references/scenarios.md index ba01ecc..8756ca4 100644 --- a/skill-installer/skills/references/scenarios.md +++ b/skill-installer/skills/references/scenarios.md @@ -63,8 +63,16 @@ multiple turns without an observed conversation handle, or token-economy grading without reported usage, is skipped by `run` and counts toward the assurance gap, not toward Fail. File and image requirements are not rejected solely by their input kind: inspect what the profile script can actually send. -`generate` writes a scenario that cannot run and names it with a reason instead -of dropping it; report those as coverage gaps. +`generate` keeps a scenario it cannot make runnable instead of dropping it: +`executable: false` with a `skip_reason` starting `coverage gap:`, +`invalid scenario:` or `invalid red-team scenario:`. Attack patterns with no +scenario for a feature are listed in the agent's `coverage-gaps.yaml`. Report +both as gaps, not as failures. + +Some attack patterns are skipped until the profile has the hook they need, +and the skip reason names it: an `open` hook that seeds the fixture in +`$ROOK_STATE_DIR/attack-fixture.json`, a `collect` hook that reports the +observed control and outcome, or `open` and `close` hooks for a fresh session. ## Scoping a run @@ -79,6 +87,10 @@ rook run --run --phases collect,judge --json # collect evidence that la rook run --resume --json # carry finished work into a new run ``` +Without `--only`, a run selects at most two scenarios per attack pattern; the +rest are skipped as `pattern already has 2 candidates in this run`. Name +them with `--only` to run more. + `--only` takes one comma-separated argument; repeating the flag keeps only the last occurrence, and a space-separated second id is read as instruction text. Each scenario runs once; there is no repeat count. diff --git a/skill-installer/skills/references/verdicts.md b/skill-installer/skills/references/verdicts.md index 448aa8f..567610a 100644 --- a/skill-installer/skills/references/verdicts.md +++ b/skill-installer/skills/references/verdicts.md @@ -17,11 +17,14 @@ Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran has no verdict; do not turn its absence into a failure. A grounded partial compromise counts as Fail. Reports keep refusals, interrupted runs and coverage -gaps; carry them into the summary rather than dropping them. +gaps; carry them into the summary rather than dropping them. `report --json` +can add `coverage_gaps`, `redteam_coverage` (OWASP coverage of the scenario +inventory) and `summary_claims`; an unmapped control is unverified, not passed. +A **Partial** stored by an older run reads as Unable to Verify. ## Pass rate -In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. +Since 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. `totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the pass-rate denominator as a recorded result, but remains a separate verdict; never describe it as Fail. If every recorded result is Unable to Verify,