Skip to content

docs(rook): 0.1.11 polish, real-run captures (Attack intent, Coverage gaps, OWASP table), skill 0.1.11 - #3694

Merged
vipulkv merged 3 commits into
LambdaTest:stagefrom
samyakLambda:docs/rook-polish-0.1.11
Oct 13, 2026
Merged

vipulkv merged 3 commits into
LambdaTest:stagefrom
samyakLambda:docs/rook-polish-0.1.11

Conversation

@samyakLambda

@samyakLambda samyakLambda commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Third commit (22cd7575): screenshots recaptured on 0.1.11, QA findings

From Mayur's QA pass:

  • Screenshots: I replayed the Quickstart exactly as written on the published 0.1.11 against production, and recaptured all 15 TUI walkthrough images from that session. All 26 /help <command> screens and the /doctor screen were recaptured too. Local paths are shortened to ~/rook-samples/… and ~/.testmuai/rook/…, and the account name is replaced. Image sizes, captions and alt texts are updated to match, for example SC-001 instead of SC-002, and the triage profile instead of the old CommerceCare demo.
  • Scenario path (Scenarios): the path is now the full .testmuai/rook/projects/<project-id>/agents/<agent-id>/scenarios/, with a link to the Workspace Files tree.
  • /docs (CLI Reference): the page said /docs opens the public repository. In 0.1.11 (and 0.1.7) it opens the documentation site, so the docs are what changed, not the command.
  • OWASP Agentic codes: ASI01–ASI10 are the official codes, published by the OWASP GenAI Security Project in December 2025. The Scenarios page now links to that list.

Found in the replay:

  • Generated call checks (Quickstart): generate wrote expect: called_with with empty arguments: {} for every tool call. A run then reported the T-1043 scenario as Unable to Verify, even though all four criteria passed. The Quickstart now says to review the calls list and fill in the expected arguments; with that change the scenario passes. This is a product bug and will be filed in Jira.

Docs test: 44/44. Static-md fences: 0 errors.


Second commit (260ab763): real 0.1.11 run, plus #3693

This PR now also replaces #3693. One promotion covers both.

Captured from a signed-in run of the published @testmuai/rook@0.1.11 against production, using the Quickstart's triage-service sample:

  • Get-started screenshot: now the signed-in session with the project chooser, which is the state the original showed. Captions and sizes are updated on Overview, Quickstart and Installation.
  • Scenarios: a local-viewer capture of the new Attack intent panel (OWASP ASI01/LLM01, MITRE ATLAS AML.T0051.001, objective, delivery, target control).
  • Web UI: a capture of the new Coverage gaps panel on a run.
  • Results and Evidence: an excerpt of the report's OWASP red-team coverage table, with what each column counts (checked against the 0.1.11 source).

Corrections found in that run:

  • Skip reasons: a non-runnable scenario's skip_reason starts with coverage gap:, invalid scenario: or invalid red-team scenario: only when Rook's own validation wrote it. The scenario checker writes plain sentences (for example, a precondition the profile can't establish).
  • Attacks from plain generate: a plain generate can plan no attacks for a small agent; this run planned 9 functional scenarios and no attacks. The page now says to ask with --class adversarial and to check the coverage gaps.
  • Skill version (from docs(rook): the Rook skill package is now 0.1.11 #3693): the Coding Agents page now says the CLI and the skill package are both at 0.1.11.

Docs test: 44/44. Static-md fences: 0 errors.


Summary

Follow-up polish after #3675/#3690. No new features are documented.

  • Structured output (CLI Reference): lists every command that writes one JSON document with --json. Previously only plan, status, run and report were listed; this adds scenarios list, project --workspace, mcp and ask, checked against the 0.1.11 CLI. CI/CD links to that list.
  • Installation:
    • The npm upgrade is now npm install -g @testmuai/rook@latest. It used to be npm update -g; the new command matches the PowerShell section and the CLI's own update fallback.
    • Drops two stale "0.1.7" notes (Windows ARM64, Node.js engine).
    • "Next Step" moves to the end of the page, after the UI section.
  • Quickstart: the PowerShell npm line is now unpinned, like the Homebrew, shell and bash npm lines on that page. The installation page keeps its pinned Windows example, and the docs test checks it there.
  • Client pages: the 7 manual skill-copy recipes now pin 439480f2, the commit published as @testmuai/rook-skill@0.1.11 (Describe rook 0.1.11 in the agent skill rook#52). They pinned f199cb99, from September 14, and 7 skill files have changed since.
  • Small fixes:
    • dateModified → 2026-10-13 on the 13 pages changed for 0.1.11;
    • Quickstart step 2 heading case (its anchor is explicit, so it doesn't change);
    • removes the Web UI "Searching local agents by description is coming soon" note. In 0.1.11, search matches name and ID only, and nothing tracks the description search.

Left as is:

  • "Artefacts" (bold) matches the viewer's UI labels, and prose already uses "artifacts".
  • The CLI Reference h4 headings mix title and sentence case; changing them would move their anchors.
  • The Overview title belongs to the SEO wording.

Verification

  • node --test scripts/rook-docs.test.cjs: 44/44. The quickstart PowerShell pin assertion moved to the installation page.
  • generate-static-md + check-static-md-fences: 0 errors. Generated static/docs is not included.

🤖 Generated with Claude Code

samyakLambda and others added 2 commits October 13, 2026 16:29
- CLI Reference lists every command that writes one JSON document with
  --json (adds scenarios list, project --workspace, mcp, ask); CI/CD links
  to it.
- Installation: npm upgrade uses `npm install -g @testmuai/rook@latest`,
  matching PowerShell and the CLI's own fallback; drop stale 0.1.7 notes;
  Next Step moves to the end of the page.
- Quickstart: PowerShell install is unpinned like the other install lines.
- Client skill-copy recipes pin the 0.1.11 skill commit (439480f2) instead
  of the September commit (f199cb99).
- dateModified -> 2026-10-13 on pages changed for 0.1.11; quickstart step 2
  heading case; remove the web UI "coming soon" search note.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
From a signed-in 0.1.11 run of the Quickstart's triage-service sample:
- get-started screenshot is now the signed-in session with the project
  chooser (captions and sizes updated on Overview, Quickstart, Install);
- local viewer captures of the Attack intent panel (Scenarios) and the
  Coverage gaps panel (Web UI);
- an excerpt of the report's OWASP coverage table, with what each
  column counts (Results and Evidence).

Corrections found in that run:
- skip_reason prefixes come only from Rook's own validation; the
  checker writes plain sentences (Scenarios, Troubleshooting);
- a plain generate can plan no attacks; ask with --class adversarial.

Also folds in LambdaTest#3693: the skill package is now 0.1.11 (Coding Agents).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@samyakLambda samyakLambda changed the title docs(rook): polish after the 0.1.11 update docs(rook): 0.1.11 polish, real-run captures (Attack intent, Coverage gaps, OWASP table), skill 0.1.11 Oct 13, 2026
…ings

- Replay the Quickstart as written on Rook 0.1.11 and recapture its 15
  walkthrough images, plus all 26 /help screens and /doctor.
- Quickstart: document reviewing generated call checks; a called_with
  check with empty arguments reports Unable to Verify.
- Scenarios: full on-disk path under projects/<project-id>/; link the
  OWASP Top 10 for Agentic Applications.
- CLI Reference: /docs opens the documentation site, not the repository.
- Update captions and alt texts for the new captures.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@vipulkv
vipulkv merged commit d77b1de into LambdaTest:stage Oct 13, 2026
1 check passed
surishubham pushed a commit that referenced this pull request Oct 14, 2026
… QA fixes

Applies the #3694 changes (merged to stage as
d77b1de) unchanged: recaptured TUI walkthrough, /help and /doctor screenshots
on Rook 0.1.11, Attack intent and Coverage gaps captures, the OWASP coverage
table, skill 0.1.11, and the QA fixes (scenario path, /docs, OWASP Agentic
link, Quickstart calls arguments).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
surishubham added a commit that referenced this pull request Oct 14, 2026
…estmucom

docs(rook): promote #3694 to testmuCom — 0.1.11 screenshots and QA fixes
surishubham pushed a commit that referenced this pull request Oct 14, 2026
… QA fixes

Applies the #3694 changes (merged to stage as
d77b1de) unchanged: recaptured TUI walkthrough, /help and /doctor screenshots
on Rook 0.1.11, Attack intent and Coverage gaps captures, the OWASP coverage
table, skill 0.1.11, and the QA fixes (scenario path, /docs, OWASP Agentic
link, Quickstart calls arguments).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
surishubham added a commit that referenced this pull request Oct 14, 2026
…t-docs

docs(rook): promote #3694 to lt-docs — 0.1.11 screenshots and QA fixes
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants