feat: evidence-driven review hub, agent review chains, and process layer v2

CODEBASE_DOCUMENTATION.md

@@ -122,9 +122,29 @@ server/githubCloneWorktreeService.js - GitHub import flow for Quick Work (`owner
 server/portRegistry.js             - Port assignment + live service scanner (`/api/ports/scan`)
 ├─ Windows scan path: uses hidden `netstat`/`tasklist` probes so packaged Tauri builds do not flash console windows when Ports/Dashboard panels refresh
 └─ UI metadata: labels orchestrator-assigned ports, known dev servers, and custom user labels
+server/commanderManager.js         - Holds N Commander instances keyed by id (primary 'commander' = unchanged single-Commander behavior); list/spawn/remove; per-instance PTY + cwd
+server/utils/shellSafety.js        - Allowlist validators for values interpolated into shell commands (model/reasoning/flags) — guards custom-agent + server-launch command construction
 server/commanderService.js         - Top-level Commander PTY (Claude/Codex) + launch buffering
 ├─ Packaged CWD: uses `ORCHESTRATOR_DATA_DIR/commander` so desktop users can edit `CLAUDE.md` / `AGENTS.md` safely
 └─ First-run seed: copies the packaged `docs/COMMANDER_CLAUDE.md` into the Commander data directory when missing
+server/evidenceService.js          - Per-task evidence collection (review-readiness proof)
+├─ Sources: fenced ```agent-evidence JSON blocks in PR body/comments/reviews, `.agent-evidence.json` in the worktree, direct API
+├─ Merge: later sources win per scalar section; reviews/media/data accumulate de-duped; PR diff stats computed server-side
+├─ Media: `GET /api/process/evidence/:id/media/:idx` streams only from the server-recorded worktree root (extension whitelist, traversal rejected)
+└─ Protocol agents follow: docs/agents/EVIDENCE_PROTOCOL.md (compact snippet auto-appended to batch launch prompts via server/evidencePromptSnippet.js)
+server/reviewWorkflowService.js    - Data-driven multi-agent review chains (config/review-workflows.json + ~/.agent-workspace/review-workflows.json override)
+├─ Stages: role + agentId + model + effort per stage; riskDefaults pick a chain per task risk
+├─ Runner: spawns each reviewer into an idle worktree, polls GitHub review verdicts, records outcomes into evidence.reviews[]
+└─ Run state persists on the task record (`reviewWorkflow`) — restarts resume polling; stalls on timeout, blocks on needs_fix
+server/agentSpawnHelper.js         - Shared worktree-locate + one-shot agent launch (used by PR review automation, review workflows)
+├─ Agent-agnostic: launch flags + init delay resolve from the agentManager registry (any registered agent id works)
+└─ Two-write submit: prompt text, then `\r` separately (a single "text\r" chunk is treated as a bracketed paste by agent CLIs)
+server/agentManager.js             - Agent registry: built-ins (claude/codex) + custom CLI agents merged from `~/.agent-workspace/custom-agents.json`
+├─ Custom agents (Gemini/OpenCode/Grok/aider/...) are pure config: modes, flags, defaultFlags, per-agent `modelFlag`/`reasoningFlag` CLI syntax, initDelayMs — see config/custom-agents.example.json
+└─ Registered agents surface automatically in /api/agents (agent picker UI), batch launches, and review-workflow stages
+server/visibilityPresetService.js  - One-click UI Mode presets (simple ↔ power/process) rewriting ui.visibility (`POST /api/user-settings/visibility-preset`)
+server/contextSwitchTelemetryService.js - Local-only context-switch JSONL log + summary (Context Tax estimator; `~/.agent-workspace/telemetry/context-switches.jsonl`)
+server/serverLaunchCommandResolver.js - Data-driven dev-server launch: cascaded `serverCommand` template + {{gameMode}}/{{commonFlags}} substitution (replaces hardcoded `hytopia start`)
 scripts/tauri/prepare-backend-resources.js - Tauri backend packager
 ├─ Bundles: server/client/config/templates/scripts + optional Node runtime into `src-tauri/resources/backend`
 ├─ Commander instructions: copies `docs/COMMANDER_CLAUDE.md` into `resources/backend/{COMMANDER_CLAUDE.md,CLAUDE.md,AGENTS.md}` for desktop builds
@@ -291,7 +311,19 @@ client/styles/projects-board.css   - Projects Board modal styling
 client/plugin-host.js              - Client plugin runtime for UI slots/actions
 ├─ Loads: `/api/plugins/client-surface` slot actions with cache/refresh support
 ├─ Exposes: `window.orchestratorPluginHost`
-└─ Supports actions: open_url, open_route, copy_text, commander_action
+├─ Supports actions: open_url, open_route, copy_text, commander_action, post_route (local route + optional prompted input)
+└─ Rendered slots: `commander.tools` (Commander panel strip), `dashboard.telemetry.actions` (telemetry overlay)
+
+client/queue-evidence.js           - Evidence card in Queue detail (badges: tests/app-ran/reviews/media/data/diff; media lightbox; refresh)
+client/queue-workflow.js           - Review-workflow block in Queue detail (chain picker by risk, run/skip/cancel, stage chips)
+client/visibility-preset.js        - Settings → UI Mode preset switch (simple ↔ power)
+client/plugins-admin.js            - Settings → Plugins list (loaded + failed) + reload
+client/context-telemetry.js        - Fire-and-forget context-switch tracking hooks (window.ContextTelemetry)
+client/styles/queue-evidence.css   - Evidence card + workflow chips + cache-cold chip styling
+
+plugins/youtube-transcript/        - Example plugin: YouTube URL → yt-dlp subtitles → plain-text transcript in ~/Downloads/transcripts (commander.tools button + `youtube-transcript-transcribe` command)
+config/review-workflows.json       - Named review chains (standard/hardened/full-gate), role prompt focuses, riskDefaults
+docs/agents/EVIDENCE_PROTOCOL.md   - How agents report evidence (schema, channels, reviewer obligations, handoff notes)
 ```
 
 ### Tabbed Workspace System (NEW)
@@ -643,6 +675,16 @@ GET /api/policy/templates                                    - Built-in team gov
 POST /api/policy/bundles/export                              - Export policy bundle (template/current/custom) for sharing
 POST /api/policy/bundles/import                              - Apply policy bundle (replace/merge) into global settings
 GET /api/audit/export?signed=1                               - Signed audit export (HMAC-SHA256; requires signing enabled + secret)
+POST /api/process/evidence/:id/refresh                        - Re-collect task evidence from PR comments + worktree file
+PUT /api/process/evidence/:id                                 - Directly set/merge task evidence
+GET /api/process/evidence/:id/media/:idx                      - Stream an evidence screenshot/video (path-validated)
+GET /api/process/review-workflows                             - Review chain catalog (workflows/roles/riskDefaults)
+POST /api/process/review-workflows/:id/{start,advance,cancel} - Run/skip-stage/cancel a review chain for a PR task
+POST /api/process/telemetry/context-switch                    - Record a local context-switch event
+GET /api/process/telemetry/context-switches?hours=24          - Context-switch summary (count, est. refocus cost, top pairs)
+GET /api/user-settings/visibility-presets                     - List UI Mode presets + current
+POST /api/user-settings/visibility-preset                     - Apply a UI Mode preset (simple|power)
+
 GET /api/agent-providers                                      - List registered agent providers and capabilities
 GET /api/agent-providers/:providerId/sessions                 - List provider sessions from SessionManager
 POST /api/agent-providers/:providerId/resume-plan             - Build provider-specific resume command/config plan

PLANS/2026-01-20/ROLLING_LOG.md

@@ -681,3 +681,15 @@ Purpose: keep a terse but complete log of what changed, why, and where to resume
 ### Docs: remove old Q-tier terminology (merged)
 - Process docs now refer to tiers as `T1–T4` and interactive pressure as `T1+T2`.
 - PR: https://github.com/web3dev1337/claude-orchestrator/pull/219 (merged 2026-01-25)
+
+## 2026-07-15 — Evidence + Review Workflows + Process Layer v2 (branch feature/review-inbox-and-tier-workflows)
+
+Research synthesis from `optimal-agent-orcestration-system` + 6-scout code audit → PLANS/2026-07-15/EVIDENCE_REVIEW_WORKFLOWS_PLAN.md. Shipped in one PR:
+- Evidence system: task-record `evidence` field, evidenceService (PR fenced blocks + worktree file + diff stats + safe media serving), Queue evidence card, docs/agents/EVIDENCE_PROTOCOL.md + auto-injected prompt snippet
+- Review workflows: config/review-workflows.json chains (standard/hardened/full-gate), reviewWorkflowService stage runner (per-role agent/model/effort, GitHub verdict polling, evidence.reviews recording), Queue workflow block
+- Fixed: reviewer/batch spawn config was invalid (never launched); commander slash-commands (/clear) sent as paste chunk; plugin POST routes had no body parser; server launch hardcoded `hytopia start`
+- UI Mode visibility presets (simple ↔ power) + Settings section; Review Hub button un-hidden
+- Prompt-cache freshness: fresh-window fixer + cache-cold chip
+- Local context-switch telemetry + dashboard surfacing
+- Plugins made real: youtube-transcript example, post_route action, commander.tools slot, Settings admin, README slot docs
+- Multi-commander: feasibility note (PLANS/2026-07-15/MULTI_COMMANDER_FEASIBILITY.md), follow-up PR

PLANS/2026-07-15/EVIDENCE_REVIEW_WORKFLOWS_PLAN.md

@@ -0,0 +1,100 @@
+# Evidence + Review Workflows + Process Layer v2 (2026-07-15)
+
+Synthesis of the `optimal-agent-orcestration-system` research (tier system, P-A-R math, review-chain math, risk-based verification) + a 6-scout code audit of this repo. This is the implementation plan for branch `feature/review-inbox-and-tier-workflows`.
+
+## What already exists (don't rebuild)
+
+The Jan-Feb 2026 process layer is mature and mostly HIDDEN, not missing:
+- `taskRecordService` (tier/risk/pFail/verify/deps/review outcomes/timers) + Queue panel (`showQueuePanel`, app.js:26380+) + Review Console + conveyors + 42 `queue-*` commands + `processStatusService` (WIP + B/W/Q/X + per-tier caps + launch gating) + `processAdvisorService` + `processTelemetryService` + `prReviewAutomationService` (single-role reviewer spawn) + prompt artifacts + dependency graph.
+- Hidden via `ui.visibility` defaults (commits 8abc8aa2..858a2bcc, Feb-Mar 2026). The queue header button is additionally hidden by a HARDCODED `style="display:none"` in index.html (~line 81) despite `header.queue: true`.
+- Two abandoned branches (PR #804 open/conflicting; #806 stacked into it) built `latestReview*` persistence + terminal review buttons. docs/REVIEW_SYSTEM_DOCUMENTATION.md documents that UNMERGED code. Decision: adopt field naming ideas, implement fresh (single-review model doesn't fit chains; 5 months of drift; untested).
+
+## Confirmed bugs found (fix in this PR)
+
+1. `prReviewAutomationService._spawnReviewerForPr` calls `startAgentWithConfig(sessionId, {provider, skipPermissions:true, mode:'fresh'})` but the API requires `{agentId, mode, flags:['skipPermissions']}` → validation always fails → **auto-reviewer spawn has never worked**. Correct pattern lives in `batchLaunchService.js:181` (also: prompt then `\r` as separate writes, not `prompt+'\n'`).
+2. `pr-review-automation` socket event has zero client listeners (dead telemetry).
+3. Queue button: visibility flag true but inline `display:none` wins.
+
+## Research → design constants (from FINAL_ARTICLE et al.)
+
+- Review chains: p_chain = Πp_i (30% → 9% → 2.7%); sweet spot 2-3 reviewers; more only for high-risk/security. Chains raise fan-out capacity ~60%.
+- Risk: `impact = 0.25*live + 0.20*users + 0.20*(1-rollback_ease) + 0.20*breaks_other + 0.15*money`; `p_fail = 0.30*complexity + 0.25*testsPenalty + 0.20*novel + 0.15*(chain?0.3:1.0) + 0.10*(1-specQuality)`; `risk = 0.6*impact + 0.4*p_fail`; bands: <0.2 AUTO_MERGE / 0.2-0.4 QUICK_CHECK / 0.4-0.6 BASIC_VERIFY / >0.6 FULL_REVIEW.
+- Low-testability domains (games/UI): tests are weak evidence (p_auto_catch≈30%) → screenshots/app-ran proof must be FIRST-CLASS evidence.
+- Context tax: 5-15 min/switch; batch by repo/type. ρ ≤ 0.85. Caps: WIP≤5, T1≤1/T2≤2/T3≤5/T4≤1 (already in processStatusService).
+- Cache: >~1h old prompt = cold cache → reprompt in FRESH window with handoff notes (ledger pattern).
+- All research %s are priors, not measurements — telemetry exists to calibrate them.
+
+## New feature 1 — Evidence system (centerpiece)
+
+The 7 things a human needs at a glance per finished task: tests ran+passed · app actually ran · review-chain verdicts+fixes · screenshots/video · data/balance proof · diff stats · standards used.
+
+**Sources, merged by new `server/evidenceService.js`:**
+1. Fenced ```agent-evidence JSON blocks in PR body + PR comments (primary; travels with the PR, cross-machine, reviewers append their own blocks as comments).
+2. `.agent-evidence.json` + `.agent-evidence/` media dir in the worktree (local supplement; primary for worktree/session tasks with no PR yet).
+3. Direct API: `PUT /api/process/evidence/:taskId`.
+Server-computed: `diffStats` aggregated from `pullRequestService` per-file additions/deletions (never trust agent-supplied numbers for PRs).
+
+**Task record field `evidence`** (normalized in taskRecordService, pattern: normalizeReviewChecklist):
+```json
+{ "schema": 1, "updatedAt": "ISO", "summary": "...",
+  "tests": {"ran":true,"command":"npm test","passed":47,"failed":0,"output":"tail","at":"ISO"},
+  "appRun": {"ran":true,"method":"puppeteer|server-smoke|studio|manual","url":"","notes":"","at":"ISO"},
+  "media": [{"type":"image","path":".agent-evidence/feature.png","caption":""}],
+  "data": [{"metric":"dps","before":120,"after":90,"note":"autoplay 3 runs"}],
+  "reviews": [{"role":"security","agentId":"codex","model":"gpt-5.5","verdict":"approved","summary":"","findings":2,"fixed":2,"at":"ISO"}],
+  "standards": ["CLAUDE.md"],
+  "handoff": {"notes":"for successor agent"},
+  "diffStats": {"files":12,"additions":340,"deletions":80} }
+```
+
+**UI**: evidence card in Queue `renderDetail` via new `client/queue-evidence.js` — badge row (🧪 47✅ · ▶️ ran · 🛡️✅ · 📸3 · 📊2 · 12 files +340/−80) + expandable sections + media lightbox (`GET /api/process/evidence/:taskId/media/:idx`, path-validated streaming). Evidence completeness indicator drives review-readiness.
+
+**Protocol**: `docs/agents/EVIDENCE_PROTOCOL.md` — how agents self-report (JSON schema + fenced-block examples + media conventions + handoff notes + how an implementer agent can run its OWN review chain). Referenced/injected by launch prompts (batchLaunch prefix, workflow templates).
+
+## New feature 2 — Review workflows (data-driven chains)
+
+`config/review-workflows.json`: named workflows (stages[] with role/agentId/model/effort/promptTemplate), role prompt templates ({{prNumber}} {{owner}} {{repo}} {{standards}} …), riskDefaults (low→standard 1-stage, high→hardened 2-stage, critical→full-gate 3-stage). New `server/reviewWorkflowService.js`: sequential stage runner on top of the fixed spawn machinery — spawn stage reviewer → detect its GitHub review + agent-evidence comment → record into `evidence.reviews[]` → next stage → done → notify. Queue detail gets workflow picker + "Run review workflow" + per-stage status chips. Per-role model: claude `--model <m>` (extend buildClaudeCommand), codex `-m <m> -c model_reasoning_effort=<e>`. Research note: reviewer model strength should scale with risk (cheap reviewer for low-risk, strong for security/high-risk) — encode in the default config, keep data-driven.
+
+## New feature 3 — Review Hub surfacing + visibility presets
+
+- Remove hardcoded display:none from queue button; label "Review".
+- `ui.visibilityPreset`: `simple` (today's defaults) | `power` (workflow modes, tier filters, PRs, review route, activity, diff, process banner, dashboard process cards, commander controls ON). Server: preset maps + `POST /api/user-settings/visibility-preset`. Settings panel: preset switch section (finally a UI for this — none exists today).
+
+## New feature 4 — Cache freshness + fresh-window reprompt
+
+promptAge from `promptSentAt`; >55 min → queue detail + reprompt actions warn "cache cold — use fresh window"; "Reprompt (fresh)" action spawns fresh-mode session seeded with `evidence.handoff.notes` + prompt artifact.
+
+## New feature 5 — Context-switch telemetry (local-only)
+
+`server/contextSwitchTelemetryService.js` → JSONL `~/.orchestrator/telemetry/context-switches.jsonl`; `POST /api/process/telemetry/context-switch` + summary endpoint (switches/day, est. cost via 10-min default, top thrash pairs); client emit on focus-worktree / workspace switch / workflow-mode change / review start-end; surfaced in dashboard Process section + advisor rule. Whole-computer monitoring = future note only.
+
+## New feature 6 — Commander fixes
+
+`/clear` slash-command passthrough fix + second-commander feasibility (pending scout-commander report; implement smallest sound fix).
+
+## New feature 7 — Plugins made real
+
+Example plugin `plugins/youtube-transcript/` (yt-dlp subtitle fetch route + registered command; graceful "install yt-dlp" recommendation when missing); `commander.tools` slot renderer in commander panel free real estate; plugin admin list + reload in Settings; document `client.slots` in plugins/README.md (currently undocumented).
+
+## New feature 8 — Play buttons revival (stretch)
+
+Uncomment/modernize Start Server block (app.js:4537-4555) behind existing `terminal.serverLaunchMenu`/`startServerDev` flags; wire `getDynamicLaunchOptions()` (cascaded gameModes/commonFlags); implement `{{gameMode}}`/`{{commonFlags}}` substitution the docs promise but code never had.
+
+## Commit plan (each pushed; priority order if interrupted)
+
+1. docs: this plan + ai-memory update
+2. fix: reviewer spawn config bug + two-step prompt (+tests)
+3. feat: evidence field in task records (+tests)
+4. feat: evidenceService + APIs + diffStats (+tests)
+5. feat: queue evidence card UI
+6. feat: review workflows config + service + queue actions (+tests)
+7. docs: EVIDENCE_PROTOCOL.md + prompt injection
+8. feat: visibility presets + review hub surfacing
+9. feat: cache-freshness + fresh reprompt
+10. feat: context-switch telemetry
+11. fix/feat: commander (/clear, layout, maybe multi)
+12. feat: plugins (example + slot renderer + admin + docs)
+13. feat: play buttons (stretch)
+14. docs: CODEBASE_DOCUMENTATION.md + PR
+
+Out of scope (documented, future): whole-computer context monitor; Bayesian per-bucket p tracking (telemetry fields land now, math later); triage 3-bucket Trello pipeline (spec exists, big); multi-commander full implementation if seam is large; heavy queue renames (Feb-21 report's full consolidation).