[ishanaggarwal04]

Adds samples/ops-assistant: one ops agent (orders, incidents, logs, service status, reports) that returns the same answer in six response formats. Each format gets its own Rook profile and scenario, so the sample shows how profiles adapt to each response shape.

Format	Endpoint	Profile
JSON	POST /v1/chat	json
Plain text	POST /v1/chat/text	text
Async job	POST /v1/chat/async + GET /v1/jobs/{id}	async
NDJSON	POST /v1/chat/ndjson	ndjson
SSE	POST /v1/chat/sse	sse
WebSocket	WS /v1/chat/ws	ws

What's included

• Agent: FastAPI front door + LLM tool-calling loop (agent/server.py), two stdio MCP servers (mcp/), a mock backend (mock-api/), a Streamlit UI (ui/) and an endpoint selfcheck (scripts/selfcheck.mjs).
• rook/*.txt: the curl / wscat inputs used to create each profile with rook profile add --from.
• .mcp.json declaring data-server. The text endpoint can't hand tool output to the judge, so a text scenario's "doesn't invent details" check comes back Unable to Verify. Declaring verification_requires on data-server.summarize_incident lets the judge query the tool itself.
• A row for the sample in samples/README.md.

The Rook workspace (.testmuai/) is not committed. The README describes the profiles, the manual edit to the text profile, and the hand edits made to the generated scenarios.

Verified

• scripts/selfcheck.mjs: 7/7 endpoint checks pass.
• Rook runs against this agent: all six format scenarios pass. The text scenario passes 4/4 with the judge querying data-server.

Notes for reviewers

• Needs a Z.ai GLM key (GLM-5.3-Flash, through the Anthropic-format API). The setup commands in the README are PowerShell.

🤖 Generated with Claude Code

[vipulkv]

Continuous Review — Round 1 — ⚠️ INCOMPLETE

Head: 4b5458e Reviewer panel: 2 CI: pass=4
only 1/2 reviewers responded (need ≥2); abstained: claude=ABSTAIN_ERROR
Outside contributor (fork ishanaggarwal04/rook-public, author ishanaggarwal04 is CONTRIBUTOR): reviewed by the diff-only panel (claude, glm) — this PR's code was not checked out.

Per-reviewer summary

Reviewer	Verdict	CRIT	HIGH	MED	LOW	Duration
claude	ABSTAIN_ERROR	0	0	0	0	1s
glm	PASS	0	0	2	2	216s

Findings

glm — 2 finding(s)

[MED] Selfcheck never reads the X-Tool-Calls header the text profile depends on

• File: samples/ops-assistant/scripts/selfcheck.mjs:109
• Scenario: The /v1/chat/text case checks status, content-type and body text, then returns toolCalls: null; the harness then marks summarize_incident as '(inferred)' so coverage passes. The X-Tool-Calls header is the load-bearing mechanism for the text Rook profile (README: 'Script: scripts/text.mjs reads that header and returns it as calls'; profile declares capabilities: calls: true), yet nothing in the sample validates it. If the agent stopped sending the header or sent it malformed (e.g. a serialization regression in tool_calls_header), selfcheck still prints 7/7 passed, and the failure only surfaces later as lost tool-call checks in a paid Rook run. The case also cannot distinguish a hallucinated answer (model never called the tool) from a grounded one — exactly the 'doesn't invent incident details' behavior SC-003 exists to catch.

[MED] done.output omits text streamed in earlier tool rounds, so joined deltas and done disagree

• File: samples/ops-assistant/agent/server.py:139
• Scenario: run() yields every text content_block_delta from every LLM round, but result.output is set only on the final no-tool round (line 144, result.output = ''.join(text) after the break). If the model emits any narration alongside a tool_use in an intermediate round ("Let me look up A-100…"), NDJSON/SSE/WS clients that join deltas receive text that done.output omits. selfcheck asserts text === events.at(-1).output (scripts/selfcheck.mjs:57), so the PR's '7/7 endpoint checks pass' verification is one chatty model response away from failing on all three streaming endpoints, and the Streamlit transcript (built from deltas via st.write_stream) diverges from the done event Rook's scripts consume.

---

Auto-posted by continuous-review-agent v1.3 (findings now include a suggested fix / code snippet). Contact @vipulkv for feedback.

[vipulkv]

Continuous Review — Round 2 — ⚠️ INCOMPLETE

Head: 4b5458e Reviewer panel: 2 CI: pass=4
only 1/2 reviewers responded (need ≥2); abstained: claude=ABSTAIN_ERROR
Outside contributor (fork ishanaggarwal04/rook-public, author ishanaggarwal04 is CONTRIBUTOR): reviewed by the diff-only panel (claude, glm) — this PR's code was not checked out.

Per-reviewer summary

Reviewer	Verdict	CRIT	HIGH	MED	LOW	Duration
claude	ABSTAIN_ERROR	0	0	0	0	1s
glm	PASS	0	0	2	0	141s

Findings

glm — 2 finding(s)

[MED] done.output omits text streamed in earlier tool rounds, so joined deltas and done disagree

• File: samples/ops-assistant/agent/server.py:143
• Scenario: The model emits text ('Let me check that order…') in the same round as a tool_use: delta events are yielded for every round (server.py:138) but result.output joins only the final round's text (server.py:143, since text resets at server.py:129). The selfcheck's own check text === events.at(-1).output (scripts/selfcheck.mjs:57) then fails spuriously, and the same request returns different answers by format — Rook's ndjson/sse/ws scripts report done.output while ui/app.py's stream_answer renders the joined deltas including the dropped preamble.

[MED] Text-endpoint case never reads the X-Tool-Calls header the text Rook profile depends on

• File: samples/ops-assistant/scripts/selfcheck.mjs:113
• Scenario: The X-Tool-Calls header is the documented mechanism that makes the text profile's tool checks verifiable (server sends it via tool_calls_header/chat_text; Rook's scripts/text.mjs parses it). The selfcheck's text case reads only the body and returns toolCalls: null, then the harness marks summarize_incident '(inferred)' from the test's own expectation — its comment 'Text responses carry no tool list' is false for this server. If the header is dropped or emits invalid JSON, selfcheck still passes while the text profile silently loses its calls capability.

---

Auto-posted by continuous-review-agent v1.3 (findings now include a suggested fix / code snippet). Contact @vipulkv for feedback.