From 913005973c259a8a81e98c445848276397a66023 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" Date: Mon, 5 Oct 2026 12:14:19 +0000 Subject: [PATCH 1/4] skill: describe rook 0.1.6 --- .agents/skills/rook/references/troubleshooting.md | 4 ++-- .claude/skills/rook/references/troubleshooting.md | 4 ++-- skill-installer/skills/references/troubleshooting.md | 4 ++-- 3 files changed, 6 insertions(+), 6 deletions(-) diff --git a/.agents/skills/rook/references/troubleshooting.md b/.agents/skills/rook/references/troubleshooting.md index e1221f1..58fdbfb 100644 --- a/.agents/skills/rook/references/troubleshooting.md +++ b/.agents/skills/rook/references/troubleshooting.md @@ -10,8 +10,8 @@ rook status --json ``` `doctor` prints version, node, workspace, environment, controller and api -reachability, identity, auth state, project, mode (`headless` or `tui`), -tty, and state. +reachability, identity, auth state, project, mode (`headless` or `tui`) +and tty. ## Failure documents diff --git a/.claude/skills/rook/references/troubleshooting.md b/.claude/skills/rook/references/troubleshooting.md index e1221f1..58fdbfb 100644 --- a/.claude/skills/rook/references/troubleshooting.md +++ b/.claude/skills/rook/references/troubleshooting.md @@ -10,8 +10,8 @@ rook status --json ``` `doctor` prints version, node, workspace, environment, controller and api -reachability, identity, auth state, project, mode (`headless` or `tui`), -tty, and state. +reachability, identity, auth state, project, mode (`headless` or `tui`) +and tty. ## Failure documents diff --git a/skill-installer/skills/references/troubleshooting.md b/skill-installer/skills/references/troubleshooting.md index e1221f1..58fdbfb 100644 --- a/skill-installer/skills/references/troubleshooting.md +++ b/skill-installer/skills/references/troubleshooting.md @@ -10,8 +10,8 @@ rook status --json ``` `doctor` prints version, node, workspace, environment, controller and api -reachability, identity, auth state, project, mode (`headless` or `tui`), -tty, and state. +reachability, identity, auth state, project, mode (`headless` or `tui`) +and tty. ## Failure documents From 3f7dc0962946cccf3fafdd802dc43120a20cac48 Mon Sep 17 00:00:00 2001 From: samyakLambda Date: Tue, 6 Oct 2026 15:53:46 +0530 Subject: [PATCH 2/4] skill: cover JSON checks and result semantics in rook 0.1.6 --- .../rook/references/headless-contract.md | 8 +++++++- .../skills/rook/references/troubleshooting.md | 18 ++++++++++-------- .agents/skills/rook/references/verdicts.md | 9 +++++++++ .../rook/references/headless-contract.md | 8 +++++++- .../skills/rook/references/troubleshooting.md | 18 ++++++++++-------- .claude/skills/rook/references/verdicts.md | 9 +++++++++ .../skills/references/headless-contract.md | 8 +++++++- .../skills/references/troubleshooting.md | 18 ++++++++++-------- skill-installer/skills/references/verdicts.md | 9 +++++++++ 9 files changed, 78 insertions(+), 27 deletions(-) diff --git a/.agents/skills/rook/references/headless-contract.md b/.agents/skills/rook/references/headless-contract.md index e45508f..60de0ab 100644 --- a/.agents/skills/rook/references/headless-contract.md +++ b/.agents/skills/rook/references/headless-contract.md @@ -65,7 +65,13 @@ still use the network, change state or invoke the target. ## On disk -`.testmuai/rook/settings.json` selects the project. New project directories +In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for +`local`, `stage` and `prod` in `selections[environment]`. Switching environments +uses that environment's selection; it can leave no project selected even when +another environment has one. Check `rook status --json` in the active environment +and select the intended project there before running project commands. + +New project directories usually use `.testmuai/rook/projects/--/`; existing `` directories are reused. Find the directory by its project ID rather than assuming either form. Under its `agents//` directory: diff --git a/.agents/skills/rook/references/troubleshooting.md b/.agents/skills/rook/references/troubleshooting.md index 58fdbfb..06240da 100644 --- a/.agents/skills/rook/references/troubleshooting.md +++ b/.agents/skills/rook/references/troubleshooting.md @@ -59,14 +59,16 @@ are about something rook cannot see (a tool call the agent made off-record, an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the -target omitted evidence. Two limitations observed in 0.1.1 can hide evidence already saved: -an explicitly empty execute `calls` array is omitted from the hook record, and -`json_path` checks do not decode a JSON string in `raw_response` before traversing -it. A recorded `calls: []` or a field visible inside that string therefore does -not guarantee a conclusive check. Report the limitation and retain the verdict; -do not invent calls or change scenario criteria just to obtain a pass. On another -release, verify whether the limitation still applies before attributing the gap -to it. +target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array could +be omitted from the hook record, and `json_path` checks did not decode a JSON +string in `raw_response` before traversing it. Verify the installed release +before attributing a gap to either historical limitation. + +The JSON-string decoding limitation is fixed in 0.1.6: deterministic `json_path` +checks can traverse valid JSON saved as a string in `raw_response`. Invalid or +oversized JSON can still be uncheckable; inspect the recorded reason and retain +Unable to Verify when the path cannot be resolved. Do not invent evidence or +change scenario criteria just to obtain a pass. ## A verdict you disagree with diff --git a/.agents/skills/rook/references/verdicts.md b/.agents/skills/rook/references/verdicts.md index 97267d6..89f1111 100644 --- a/.agents/skills/rook/references/verdicts.md +++ b/.agents/skills/rook/references/verdicts.md @@ -17,6 +17,15 @@ Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran has no verdict; do not turn its absence into a failure. +## Pass rate + +In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. +`totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the +pass-rate denominator as a recorded result, but remains a separate verdict; +never describe it as Fail. An all-Unverifiable result has pass rate `0`; no +recorded Pass, Fail or Unable to Verify results yields `null`, not zero. +Unjudged, not-run and unrunnable work does not enter this denominator. + ## Presenting results Use a table like this, populated from the records: diff --git a/.claude/skills/rook/references/headless-contract.md b/.claude/skills/rook/references/headless-contract.md index e45508f..60de0ab 100644 --- a/.claude/skills/rook/references/headless-contract.md +++ b/.claude/skills/rook/references/headless-contract.md @@ -65,7 +65,13 @@ still use the network, change state or invoke the target. ## On disk -`.testmuai/rook/settings.json` selects the project. New project directories +In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for +`local`, `stage` and `prod` in `selections[environment]`. Switching environments +uses that environment's selection; it can leave no project selected even when +another environment has one. Check `rook status --json` in the active environment +and select the intended project there before running project commands. + +New project directories usually use `.testmuai/rook/projects/--/`; existing `` directories are reused. Find the directory by its project ID rather than assuming either form. Under its `agents//` directory: diff --git a/.claude/skills/rook/references/troubleshooting.md b/.claude/skills/rook/references/troubleshooting.md index 58fdbfb..06240da 100644 --- a/.claude/skills/rook/references/troubleshooting.md +++ b/.claude/skills/rook/references/troubleshooting.md @@ -59,14 +59,16 @@ are about something rook cannot see (a tool call the agent made off-record, an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the -target omitted evidence. Two limitations observed in 0.1.1 can hide evidence already saved: -an explicitly empty execute `calls` array is omitted from the hook record, and -`json_path` checks do not decode a JSON string in `raw_response` before traversing -it. A recorded `calls: []` or a field visible inside that string therefore does -not guarantee a conclusive check. Report the limitation and retain the verdict; -do not invent calls or change scenario criteria just to obtain a pass. On another -release, verify whether the limitation still applies before attributing the gap -to it. +target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array could +be omitted from the hook record, and `json_path` checks did not decode a JSON +string in `raw_response` before traversing it. Verify the installed release +before attributing a gap to either historical limitation. + +The JSON-string decoding limitation is fixed in 0.1.6: deterministic `json_path` +checks can traverse valid JSON saved as a string in `raw_response`. Invalid or +oversized JSON can still be uncheckable; inspect the recorded reason and retain +Unable to Verify when the path cannot be resolved. Do not invent evidence or +change scenario criteria just to obtain a pass. ## A verdict you disagree with diff --git a/.claude/skills/rook/references/verdicts.md b/.claude/skills/rook/references/verdicts.md index 97267d6..89f1111 100644 --- a/.claude/skills/rook/references/verdicts.md +++ b/.claude/skills/rook/references/verdicts.md @@ -17,6 +17,15 @@ Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran has no verdict; do not turn its absence into a failure. +## Pass rate + +In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. +`totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the +pass-rate denominator as a recorded result, but remains a separate verdict; +never describe it as Fail. An all-Unverifiable result has pass rate `0`; no +recorded Pass, Fail or Unable to Verify results yields `null`, not zero. +Unjudged, not-run and unrunnable work does not enter this denominator. + ## Presenting results Use a table like this, populated from the records: diff --git a/skill-installer/skills/references/headless-contract.md b/skill-installer/skills/references/headless-contract.md index e45508f..60de0ab 100644 --- a/skill-installer/skills/references/headless-contract.md +++ b/skill-installer/skills/references/headless-contract.md @@ -65,7 +65,13 @@ still use the network, change state or invoke the target. ## On disk -`.testmuai/rook/settings.json` selects the project. New project directories +In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for +`local`, `stage` and `prod` in `selections[environment]`. Switching environments +uses that environment's selection; it can leave no project selected even when +another environment has one. Check `rook status --json` in the active environment +and select the intended project there before running project commands. + +New project directories usually use `.testmuai/rook/projects/--/`; existing `` directories are reused. Find the directory by its project ID rather than assuming either form. Under its `agents//` directory: diff --git a/skill-installer/skills/references/troubleshooting.md b/skill-installer/skills/references/troubleshooting.md index 58fdbfb..06240da 100644 --- a/skill-installer/skills/references/troubleshooting.md +++ b/skill-installer/skills/references/troubleshooting.md @@ -59,14 +59,16 @@ are about something rook cannot see (a tool call the agent made off-record, an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the -target omitted evidence. Two limitations observed in 0.1.1 can hide evidence already saved: -an explicitly empty execute `calls` array is omitted from the hook record, and -`json_path` checks do not decode a JSON string in `raw_response` before traversing -it. A recorded `calls: []` or a field visible inside that string therefore does -not guarantee a conclusive check. Report the limitation and retain the verdict; -do not invent calls or change scenario criteria just to obtain a pass. On another -release, verify whether the limitation still applies before attributing the gap -to it. +target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array could +be omitted from the hook record, and `json_path` checks did not decode a JSON +string in `raw_response` before traversing it. Verify the installed release +before attributing a gap to either historical limitation. + +The JSON-string decoding limitation is fixed in 0.1.6: deterministic `json_path` +checks can traverse valid JSON saved as a string in `raw_response`. Invalid or +oversized JSON can still be uncheckable; inspect the recorded reason and retain +Unable to Verify when the path cannot be resolved. Do not invent evidence or +change scenario criteria just to obtain a pass. ## A verdict you disagree with diff --git a/skill-installer/skills/references/verdicts.md b/skill-installer/skills/references/verdicts.md index 97267d6..89f1111 100644 --- a/skill-installer/skills/references/verdicts.md +++ b/skill-installer/skills/references/verdicts.md @@ -17,6 +17,15 @@ Statuses are **Pass**, **Fail**, and **Unable to Verify**. Unverifiable reasons: `agent_never_ran`, `not_observable`, or `undecidable`. A scenario that never ran has no verdict; do not turn its absence into a failure. +## Pass rate + +In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. +`totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the +pass-rate denominator as a recorded result, but remains a separate verdict; +never describe it as Fail. An all-Unverifiable result has pass rate `0`; no +recorded Pass, Fail or Unable to Verify results yields `null`, not zero. +Unjudged, not-run and unrunnable work does not enter this denominator. + ## Presenting results Use a table like this, populated from the records: From bedf3522810f08531509bd9ce73ca951234e85f7 Mon Sep 17 00:00:00 2001 From: samyakLambda Date: Tue, 6 Oct 2026 15:59:02 +0530 Subject: [PATCH 3/4] skill: clarify preserved evidence and environment selection guidance --- .../rook/references/headless-contract.md | 8 +++---- .../skills/rook/references/troubleshooting.md | 21 ++++++++++--------- .../rook/references/headless-contract.md | 8 +++---- .../skills/rook/references/troubleshooting.md | 21 ++++++++++--------- .../skills/references/headless-contract.md | 8 +++---- .../skills/references/troubleshooting.md | 21 ++++++++++--------- 6 files changed, 45 insertions(+), 42 deletions(-) diff --git a/.agents/skills/rook/references/headless-contract.md b/.agents/skills/rook/references/headless-contract.md index 60de0ab..2ad6c5a 100644 --- a/.agents/skills/rook/references/headless-contract.md +++ b/.agents/skills/rook/references/headless-contract.md @@ -68,11 +68,11 @@ still use the network, change state or invoke the target. In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for `local`, `stage` and `prod` in `selections[environment]`. Switching environments uses that environment's selection; it can leave no project selected even when -another environment has one. Check `rook status --json` in the active environment -and select the intended project there before running project commands. +another environment has one. Use `rook doctor` to see the active environment, +then `rook status --json` to check its selected project. Use `rook project use ` +to select the intended project there before running project commands. -New project directories -usually use `.testmuai/rook/projects/--/`; existing +New project directories usually use `.testmuai/rook/projects/--/`; existing `` directories are reused. Find the directory by its project ID rather than assuming either form. Under its `agents//` directory: diff --git a/.agents/skills/rook/references/troubleshooting.md b/.agents/skills/rook/references/troubleshooting.md index 06240da..9f4c512 100644 --- a/.agents/skills/rook/references/troubleshooting.md +++ b/.agents/skills/rook/references/troubleshooting.md @@ -59,16 +59,17 @@ are about something rook cannot see (a tool call the agent made off-record, an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the -target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array could -be omitted from the hook record, and `json_path` checks did not decode a JSON -string in `raw_response` before traversing it. Verify the installed release -before attributing a gap to either historical limitation. - -The JSON-string decoding limitation is fixed in 0.1.6: deterministic `json_path` -checks can traverse valid JSON saved as a string in `raw_response`. Invalid or -oversized JSON can still be uncheckable; inspect the recorded reason and retain -Unable to Verify when the path cannot be resolved. Do not invent evidence or -change scenario criteria just to obtain a pass. +target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array +could be omitted from the hook record. Check whether this limitation still +applies to the installed release: a recorded `calls: []` does not guarantee a +conclusive check. Report the limitation and retain the recorded verdict; do not +invent calls or change scenario criteria just to obtain a pass. + +The JSON-string decoding limitation observed in 0.1.1 is fixed in 0.1.6: +deterministic `json_path` checks can traverse valid JSON saved as a string in +`raw_response`. Invalid JSON or a raw JSON string larger than the 8 MiB decode +limit remains uncheckable. Inspect the criterion's recorded `achieved` explanation +and retain its verdict; do not invent evidence to fill an unresolved path. ## A verdict you disagree with diff --git a/.claude/skills/rook/references/headless-contract.md b/.claude/skills/rook/references/headless-contract.md index 60de0ab..2ad6c5a 100644 --- a/.claude/skills/rook/references/headless-contract.md +++ b/.claude/skills/rook/references/headless-contract.md @@ -68,11 +68,11 @@ still use the network, change state or invoke the target. In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for `local`, `stage` and `prod` in `selections[environment]`. Switching environments uses that environment's selection; it can leave no project selected even when -another environment has one. Check `rook status --json` in the active environment -and select the intended project there before running project commands. +another environment has one. Use `rook doctor` to see the active environment, +then `rook status --json` to check its selected project. Use `rook project use ` +to select the intended project there before running project commands. -New project directories -usually use `.testmuai/rook/projects/--/`; existing +New project directories usually use `.testmuai/rook/projects/--/`; existing `` directories are reused. Find the directory by its project ID rather than assuming either form. Under its `agents//` directory: diff --git a/.claude/skills/rook/references/troubleshooting.md b/.claude/skills/rook/references/troubleshooting.md index 06240da..9f4c512 100644 --- a/.claude/skills/rook/references/troubleshooting.md +++ b/.claude/skills/rook/references/troubleshooting.md @@ -59,16 +59,17 @@ are about something rook cannot see (a tool call the agent made off-record, an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the -target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array could -be omitted from the hook record, and `json_path` checks did not decode a JSON -string in `raw_response` before traversing it. Verify the installed release -before attributing a gap to either historical limitation. - -The JSON-string decoding limitation is fixed in 0.1.6: deterministic `json_path` -checks can traverse valid JSON saved as a string in `raw_response`. Invalid or -oversized JSON can still be uncheckable; inspect the recorded reason and retain -Unable to Verify when the path cannot be resolved. Do not invent evidence or -change scenario criteria just to obtain a pass. +target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array +could be omitted from the hook record. Check whether this limitation still +applies to the installed release: a recorded `calls: []` does not guarantee a +conclusive check. Report the limitation and retain the recorded verdict; do not +invent calls or change scenario criteria just to obtain a pass. + +The JSON-string decoding limitation observed in 0.1.1 is fixed in 0.1.6: +deterministic `json_path` checks can traverse valid JSON saved as a string in +`raw_response`. Invalid JSON or a raw JSON string larger than the 8 MiB decode +limit remains uncheckable. Inspect the criterion's recorded `achieved` explanation +and retain its verdict; do not invent evidence to fill an unresolved path. ## A verdict you disagree with diff --git a/skill-installer/skills/references/headless-contract.md b/skill-installer/skills/references/headless-contract.md index 60de0ab..2ad6c5a 100644 --- a/skill-installer/skills/references/headless-contract.md +++ b/skill-installer/skills/references/headless-contract.md @@ -68,11 +68,11 @@ still use the network, change state or invoke the target. In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for `local`, `stage` and `prod` in `selections[environment]`. Switching environments uses that environment's selection; it can leave no project selected even when -another environment has one. Check `rook status --json` in the active environment -and select the intended project there before running project commands. +another environment has one. Use `rook doctor` to see the active environment, +then `rook status --json` to check its selected project. Use `rook project use ` +to select the intended project there before running project commands. -New project directories -usually use `.testmuai/rook/projects/--/`; existing +New project directories usually use `.testmuai/rook/projects/--/`; existing `` directories are reused. Find the directory by its project ID rather than assuming either form. Under its `agents//` directory: diff --git a/skill-installer/skills/references/troubleshooting.md b/skill-installer/skills/references/troubleshooting.md index 06240da..9f4c512 100644 --- a/skill-installer/skills/references/troubleshooting.md +++ b/skill-installer/skills/references/troubleshooting.md @@ -59,16 +59,17 @@ are about something rook cannot see (a tool call the agent made off-record, an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the -target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array could -be omitted from the hook record, and `json_path` checks did not decode a JSON -string in `raw_response` before traversing it. Verify the installed release -before attributing a gap to either historical limitation. - -The JSON-string decoding limitation is fixed in 0.1.6: deterministic `json_path` -checks can traverse valid JSON saved as a string in `raw_response`. Invalid or -oversized JSON can still be uncheckable; inspect the recorded reason and retain -Unable to Verify when the path cannot be resolved. Do not invent evidence or -change scenario criteria just to obtain a pass. +target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array +could be omitted from the hook record. Check whether this limitation still +applies to the installed release: a recorded `calls: []` does not guarantee a +conclusive check. Report the limitation and retain the recorded verdict; do not +invent calls or change scenario criteria just to obtain a pass. + +The JSON-string decoding limitation observed in 0.1.1 is fixed in 0.1.6: +deterministic `json_path` checks can traverse valid JSON saved as a string in +`raw_response`. Invalid JSON or a raw JSON string larger than the 8 MiB decode +limit remains uncheckable. Inspect the criterion's recorded `achieved` explanation +and retain its verdict; do not invent evidence to fill an unresolved path. ## A verdict you disagree with From 8121fdac38d7381b69c4b71a53b66266594bd309 Mon Sep 17 00:00:00 2001 From: samyakLambda Date: Tue, 6 Oct 2026 16:02:19 +0530 Subject: [PATCH 4/4] skill: make recorded verdict and selection guidance explicit --- .../rook/references/headless-contract.md | 19 ++++++++++--------- .../skills/rook/references/troubleshooting.md | 15 ++++++++------- .agents/skills/rook/references/verdicts.md | 5 +++-- .../rook/references/headless-contract.md | 19 ++++++++++--------- .../skills/rook/references/troubleshooting.md | 15 ++++++++------- .claude/skills/rook/references/verdicts.md | 5 +++-- .../skills/references/headless-contract.md | 19 ++++++++++--------- .../skills/references/troubleshooting.md | 15 ++++++++------- skill-installer/skills/references/verdicts.md | 5 +++-- 9 files changed, 63 insertions(+), 54 deletions(-) diff --git a/.agents/skills/rook/references/headless-contract.md b/.agents/skills/rook/references/headless-contract.md index 2ad6c5a..0323f6c 100644 --- a/.agents/skills/rook/references/headless-contract.md +++ b/.agents/skills/rook/references/headless-contract.md @@ -66,15 +66,16 @@ still use the network, change state or invoke the target. ## On disk In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for -`local`, `stage` and `prod` in `selections[environment]`. Switching environments -uses that environment's selection; it can leave no project selected even when -another environment has one. Use `rook doctor` to see the active environment, -then `rook status --json` to check its selected project. Use `rook project use ` -to select the intended project there before running project commands. - -New project directories usually use `.testmuai/rook/projects/--/`; existing -`` directories are reused. Find the directory by its project ID -rather than assuming either form. Under its `agents//` directory: +`local`, `stage` and `prod` in `selections[environment]`. After a switch, rook +uses the new environment's selection, which may be empty even when another +environment has one. Run `rook doctor` to see the active environment and +`rook status --json` to read its `project_id`; if it is missing or wrong, run +`rook project use `. + +New project directories usually use +`.testmuai/rook/projects/--/`; existing `` +directories are reused. Find the directory by its project ID rather than +assuming either form. Under its `agents//` directory: | Path | Contents | | --- | --- | diff --git a/.agents/skills/rook/references/troubleshooting.md b/.agents/skills/rook/references/troubleshooting.md index 9f4c512..47397fd 100644 --- a/.agents/skills/rook/references/troubleshooting.md +++ b/.agents/skills/rook/references/troubleshooting.md @@ -60,16 +60,17 @@ an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array -could be omitted from the hook record. Check whether this limitation still -applies to the installed release: a recorded `calls: []` does not guarantee a -conclusive check. Report the limitation and retain the recorded verdict; do not -invent calls or change scenario criteria just to obtain a pass. +could be omitted from the hook record. It is not established whether later +releases still do this. If the saved raw response contains `calls: []` but the +hook record has no `calls`, report that gap and retain the recorded verdict; +do not invent calls or change scenario criteria just to obtain a pass. The JSON-string decoding limitation observed in 0.1.1 is fixed in 0.1.6: deterministic `json_path` checks can traverse valid JSON saved as a string in -`raw_response`. Invalid JSON or a raw JSON string larger than the 8 MiB decode -limit remains uncheckable. Inspect the criterion's recorded `achieved` explanation -and retain its verdict; do not invent evidence to fill an unresolved path. +`raw_response`. Invalid JSON, a raw string over the 8 MiB decode limit, or a +path that does not resolve yields Unable to Verify; the criterion's `achieved` +field explains why. Report that explanation and retain the recorded verdict; +do not invent evidence to fill the path. ## A verdict you disagree with diff --git a/.agents/skills/rook/references/verdicts.md b/.agents/skills/rook/references/verdicts.md index 89f1111..11fd55f 100644 --- a/.agents/skills/rook/references/verdicts.md +++ b/.agents/skills/rook/references/verdicts.md @@ -22,8 +22,9 @@ has no verdict; do not turn its absence into a failure. In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. `totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the pass-rate denominator as a recorded result, but remains a separate verdict; -never describe it as Fail. An all-Unverifiable result has pass rate `0`; no -recorded Pass, Fail or Unable to Verify results yields `null`, not zero. +never describe it as Fail. If every recorded result is Unable to Verify, +`pass_rate` is `0`; if there are no recorded Pass, Fail or Unable to Verify +results, it is `null`, not zero. Unjudged, not-run and unrunnable work does not enter this denominator. ## Presenting results diff --git a/.claude/skills/rook/references/headless-contract.md b/.claude/skills/rook/references/headless-contract.md index 2ad6c5a..0323f6c 100644 --- a/.claude/skills/rook/references/headless-contract.md +++ b/.claude/skills/rook/references/headless-contract.md @@ -66,15 +66,16 @@ still use the network, change state or invoke the target. ## On disk In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for -`local`, `stage` and `prod` in `selections[environment]`. Switching environments -uses that environment's selection; it can leave no project selected even when -another environment has one. Use `rook doctor` to see the active environment, -then `rook status --json` to check its selected project. Use `rook project use ` -to select the intended project there before running project commands. - -New project directories usually use `.testmuai/rook/projects/--/`; existing -`` directories are reused. Find the directory by its project ID -rather than assuming either form. Under its `agents//` directory: +`local`, `stage` and `prod` in `selections[environment]`. After a switch, rook +uses the new environment's selection, which may be empty even when another +environment has one. Run `rook doctor` to see the active environment and +`rook status --json` to read its `project_id`; if it is missing or wrong, run +`rook project use `. + +New project directories usually use +`.testmuai/rook/projects/--/`; existing `` +directories are reused. Find the directory by its project ID rather than +assuming either form. Under its `agents//` directory: | Path | Contents | | --- | --- | diff --git a/.claude/skills/rook/references/troubleshooting.md b/.claude/skills/rook/references/troubleshooting.md index 9f4c512..47397fd 100644 --- a/.claude/skills/rook/references/troubleshooting.md +++ b/.claude/skills/rook/references/troubleshooting.md @@ -60,16 +60,17 @@ an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array -could be omitted from the hook record. Check whether this limitation still -applies to the installed release: a recorded `calls: []` does not guarantee a -conclusive check. Report the limitation and retain the recorded verdict; do not -invent calls or change scenario criteria just to obtain a pass. +could be omitted from the hook record. It is not established whether later +releases still do this. If the saved raw response contains `calls: []` but the +hook record has no `calls`, report that gap and retain the recorded verdict; +do not invent calls or change scenario criteria just to obtain a pass. The JSON-string decoding limitation observed in 0.1.1 is fixed in 0.1.6: deterministic `json_path` checks can traverse valid JSON saved as a string in -`raw_response`. Invalid JSON or a raw JSON string larger than the 8 MiB decode -limit remains uncheckable. Inspect the criterion's recorded `achieved` explanation -and retain its verdict; do not invent evidence to fill an unresolved path. +`raw_response`. Invalid JSON, a raw string over the 8 MiB decode limit, or a +path that does not resolve yields Unable to Verify; the criterion's `achieved` +field explains why. Report that explanation and retain the recorded verdict; +do not invent evidence to fill the path. ## A verdict you disagree with diff --git a/.claude/skills/rook/references/verdicts.md b/.claude/skills/rook/references/verdicts.md index 89f1111..11fd55f 100644 --- a/.claude/skills/rook/references/verdicts.md +++ b/.claude/skills/rook/references/verdicts.md @@ -22,8 +22,9 @@ has no verdict; do not turn its absence into a failure. In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. `totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the pass-rate denominator as a recorded result, but remains a separate verdict; -never describe it as Fail. An all-Unverifiable result has pass rate `0`; no -recorded Pass, Fail or Unable to Verify results yields `null`, not zero. +never describe it as Fail. If every recorded result is Unable to Verify, +`pass_rate` is `0`; if there are no recorded Pass, Fail or Unable to Verify +results, it is `null`, not zero. Unjudged, not-run and unrunnable work does not enter this denominator. ## Presenting results diff --git a/skill-installer/skills/references/headless-contract.md b/skill-installer/skills/references/headless-contract.md index 2ad6c5a..0323f6c 100644 --- a/skill-installer/skills/references/headless-contract.md +++ b/skill-installer/skills/references/headless-contract.md @@ -66,15 +66,16 @@ still use the network, change state or invoke the target. ## On disk In 0.1.6, `.testmuai/rook/settings.json` keeps project selections separately for -`local`, `stage` and `prod` in `selections[environment]`. Switching environments -uses that environment's selection; it can leave no project selected even when -another environment has one. Use `rook doctor` to see the active environment, -then `rook status --json` to check its selected project. Use `rook project use ` -to select the intended project there before running project commands. - -New project directories usually use `.testmuai/rook/projects/--/`; existing -`` directories are reused. Find the directory by its project ID -rather than assuming either form. Under its `agents//` directory: +`local`, `stage` and `prod` in `selections[environment]`. After a switch, rook +uses the new environment's selection, which may be empty even when another +environment has one. Run `rook doctor` to see the active environment and +`rook status --json` to read its `project_id`; if it is missing or wrong, run +`rook project use `. + +New project directories usually use +`.testmuai/rook/projects/--/`; existing `` +directories are reused. Find the directory by its project ID rather than +assuming either form. Under its `agents//` directory: | Path | Contents | | --- | --- | diff --git a/skill-installer/skills/references/troubleshooting.md b/skill-installer/skills/references/troubleshooting.md index 9f4c512..47397fd 100644 --- a/skill-installer/skills/references/troubleshooting.md +++ b/skill-installer/skills/references/troubleshooting.md @@ -60,16 +60,17 @@ an image); that is a gap to report, not a failure to fix. Inspect the saved raw response and hook records before concluding that the target omitted evidence. In 0.1.1, an explicitly empty execute `calls` array -could be omitted from the hook record. Check whether this limitation still -applies to the installed release: a recorded `calls: []` does not guarantee a -conclusive check. Report the limitation and retain the recorded verdict; do not -invent calls or change scenario criteria just to obtain a pass. +could be omitted from the hook record. It is not established whether later +releases still do this. If the saved raw response contains `calls: []` but the +hook record has no `calls`, report that gap and retain the recorded verdict; +do not invent calls or change scenario criteria just to obtain a pass. The JSON-string decoding limitation observed in 0.1.1 is fixed in 0.1.6: deterministic `json_path` checks can traverse valid JSON saved as a string in -`raw_response`. Invalid JSON or a raw JSON string larger than the 8 MiB decode -limit remains uncheckable. Inspect the criterion's recorded `achieved` explanation -and retain its verdict; do not invent evidence to fill an unresolved path. +`raw_response`. Invalid JSON, a raw string over the 8 MiB decode limit, or a +path that does not resolve yields Unable to Verify; the criterion's `achieved` +field explains why. Report that explanation and retain the recorded verdict; +do not invent evidence to fill the path. ## A verdict you disagree with diff --git a/skill-installer/skills/references/verdicts.md b/skill-installer/skills/references/verdicts.md index 89f1111..11fd55f 100644 --- a/skill-installer/skills/references/verdicts.md +++ b/skill-installer/skills/references/verdicts.md @@ -22,8 +22,9 @@ has no verdict; do not turn its absence into a failure. In 0.1.6, `totals.pass_rate` is `Pass / (Pass + Fail + Unable to Verify)`. `totals.decided` still means `Pass + Fail`. Unable to Verify contributes to the pass-rate denominator as a recorded result, but remains a separate verdict; -never describe it as Fail. An all-Unverifiable result has pass rate `0`; no -recorded Pass, Fail or Unable to Verify results yields `null`, not zero. +never describe it as Fail. If every recorded result is Unable to Verify, +`pass_rate` is `0`; if there are no recorded Pass, Fail or Unable to Verify +results, it is `null`, not zero. Unjudged, not-run and unrunnable work does not enter this denominator. ## Presenting results