From ceebba284181fa4f8ee1406da0fe60302e3b7b0e Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 09:25:01 +1000 Subject: [PATCH 01/69] docs: design for autonomous supervisor, voice layer, and cross-repo Atlas Investigates three linked asks: a top-level orchestrator that watches the agent fleet, voice as the primary interface, and a cross-repo knowledge map with per-audience access scoping. Key findings: - Rules-in-a-loop / LLM-on-event keeps continuous supervision at ~0 tokens - Driving the agent CLI in a PTY bills the subscription, not the API, so autonomous Claude does not require API credits - OpenClaw and Hermes Agent are the wrong shape: neither models worktrees, tiers, queue or review state, which is the entire value here - Atlas quality must be per-topic, not per-repo: a rough prototype can still hold the best example of one thing --- ...OMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md | 205 ++++++++++++++++++ 1 file changed, 205 insertions(+) create mode 100644 PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md diff --git a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md new file mode 100644 index 00000000..ebb4ca44 --- /dev/null +++ b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md @@ -0,0 +1,205 @@ +# Autonomous Orchestrator, Voice, and the Repo Atlas (2026-07-26) + +Three asks, investigated together because they are the same product: + +1. **A top-level orchestrator** that checks in on agents, stops them getting stuck, and proactively picks up work. +2. **Voice** as the primary way to talk to it. +3. **A cross-repo map** — `CODEBASE_DOCUMENTATION.md` but for *every* repo, with per-teammate access scoping, so agents get breadcrumbs ("how did we do data compression?") without a filesystem safari. + +--- + +## 0. The one architectural decision that makes all of this affordable + +> **Do not run an LLM in a loop. Run *rules* in a loop and wake an LLM on an event.** + +This is the answer to "I'm worried about how I'd run it and what it costs." + +A supervisor that polls 16 sessions every 30s and asks a model "is this stuck?" burns money forever and produces nothing when nothing is happening. Instead: + +| Layer | Runs | Cost | +|---|---|---| +| **Sensors** — PTY tail, status detector, git state, PR state, task records | every tick (30s) | **0 tokens.** Pure JS, already in-process. | +| **Rules** — a data-driven condition table (`config/supervisor-rules.json`) | every tick | **0 tokens.** Regex + timers. | +| **Actions** — nudge text, approve prompt, `gh pr create`, notify, speak | on match | **0 tokens.** PTY writes + shell. | +| **Judgement** — "what should we do about this?", free-form voice, briefings | on escalation only, debounced | Cheap. Local Ollama first, Haiku second. | +| **Agentic work** — actually fixing the thing | on your say-so | **Subscription, not API.** | + +That last row deserves emphasis, because the premise in the ask is worth correcting: + +**You do not need Anthropic API credits to have an autonomous Claude.** OpenClaw/Hermes-style assistants call the *API*, so they cost per token. This orchestrator does something different — it drives the **Claude Code CLI inside a PTY**, which bills against your Max subscription like any interactive session. The supervisor writing `next\r` into a stuck session is exactly as expensive as you typing it. That is already how `pagerService` works today. So the autonomy budget is: **~$0 for the watching, subscription for the doing.** + +The only genuinely metered pieces are the optional judgement calls (intent haiku, voice fallback parsing). Those are already Ollama-first in `voiceCommandService`, and stay that way. + +### Where it runs + +On your machine, inside the orchestrator server process. Not a VPS, not a container, not a hosted service. Reasons: + +- The state it needs to supervise (PTYs, worktrees, session buffers, `~/.agent-workspace`) is *here*. A remote brain would have to proxy all of it back. +- The actions it takes (write to PTY, run `gh`, create worktrees) are local. +- Your agent CLIs are authenticated here. Auth doesn't travel. +- It's already a long-lived process you keep running. + +If you later want to *reach* it from your phone, that's a thin remote-control surface over the existing HTTP API (the mobile/LAN path already exists in `scripts/mobile/start-mobile.sh`) — not a second brain. + +--- + +## 1. Build vs. buy: OpenClaw / Hermes Agent + +Both are real and both are good, and neither is the shape of this problem. + +**[OpenClaw](https://openclaw.ai/)** (Peter Steinberger; was ClawdBot → Moltbot → OpenClaw after Anthropic raised a naming concern) is a self-hosted **multi-channel personal assistant** — you message it on WhatsApp/Telegram/Discord/iMessage and it does things on your machine. Enormous adoption. Its centre of gravity is *reach*: getting an agent into the chat app you already have open. + +**[Hermes Agent](https://contabo.com/blog/hermes-agent-vs-openclaw-paperclip-and-the-best-open-source-ai-agents-in-2026/)** (Nous Research) is a model-agnostic **headless agent runtime** meant to sit on a VPS and pair with any provider. Its centre of gravity is *server-side footprint and scriptability* — smaller surface area, less setup noise than OpenClaw. + +Neither knows what a worktree is, what tier a task is, that PR #1022 is waiting on evidence, or that `work3` has been sitting on a permission prompt for four minutes. That domain model — **sessions, worktrees, tiers, queue, review inbox, evidence, task records** — is the entire value here, and it already exists in this repo. Rebuilding it inside OpenClaw would be most of a rewrite; bolting OpenClaw on top would give you a chat interface to an orchestrator that still can't see anything. + +**Verdict: build the supervisor here.** Steal the good idea from OpenClaw — *reach* — later, as a notification/remote-control channel (Discord bridge already half-exists in `discordIntegrationService`), not as the brain. + +--- + +## 2. What "top-level orchestrator" actually means + +Today the fleet is **pull-based**: you look at the grid, you notice something is stuck, you fix it. Every existing mechanism is either blind or manual: + +- `pagerService` — nudges on a fixed interval regardless of whether anything is wrong. Blind. +- `schedulerService` — runs commands on a clock. Blind. +- `processAdvisorService` — computes good advice, but only when a human opens the panel. Passive. +- `statusDetector` — knows busy/waiting/idle per session, but nothing consumes it to *act*. + +The missing piece is a **push-based supervisor**: something that continuously classifies each session's condition and takes graduated action. Shipped in this branch as `server/supervisorService.js`. + +### Conditions detected + +| Condition | Signal | Default action | +|---|---|---| +| `awaiting-permission` | prompt pattern in PTY tail, held > threshold | notify → (autopilot) auto-accept safe prompts | +| `stalled` | status `busy` but no new output for N min | nudge (one-shot pager ping) | +| `idle-finished` | idle + clean tree + nothing to do | notify — free capacity, offer next queue item | +| `unpushed-work` | idle + commits ahead of origin | nudge "commit/push and open a PR" | +| `pushed-no-pr` | branch on origin, no open PR | nudge → (autopilot) `gh pr create` | +| `pr-awaiting-review` | open PR with evidence | route to review inbox / start review chain | +| `limit-reached` | usage-limit banner in tail | schedule resume at the parsed reset time | +| `error-loop` | same error line N times | escalate to human. Never auto-act. | +| `crashed` | agent exited, shell prompt returned | notify → (autopilot) relaunch with resume | + +### The escalation ladder + +Every condition resolves to one rung, and the rung is capped by a global autonomy level: + +``` +observe → notify → nudge → act → escalate(human) + ↑ ↑ + autonomy: assist always available +``` + +- `off` — nothing runs. +- `observe` — findings recorded and visible; zero side effects. **Default.** Run it for a week and read the log before you let it touch anything. +- `assist` — may notify, speak, and nudge (text into a session). Cannot run commands. +- `autopilot` — may also take listed `act` steps. + +Hard invariants regardless of level: +- **Never** auto-act on anything matching the scheduler's blocked-command patterns (merge, approve, stop-session, remove-worktree, destroy). +- **Never** auto-answer a permission prompt whose command isn't on the allowlist. +- Every action is appended to `~/.agent-workspace/logs/supervisor-audit.jsonl` with the finding that caused it. +- Per-session cooldowns; a finding that re-fires does not re-act. + +### Why this is the Iron Man bit + +The Jarvis experience isn't a nicer chat box — it's that **the assistant noticed first**. "Sir, `work3` has been waiting on a file-write permission for four minutes, and `zoo-game/work1` pushed eleven minutes ago without opening a PR." That is entirely a sensors-and-rules problem, and it is now solved with zero tokens. + +--- + +## 3. Voice + +Already present: `whisperService` (local STT: whisper.cpp / openai-whisper) and `voiceCommandService` (rule-based intent parse → `commandRegistry`, Ollama/Haiku fuzzy fallback). Two things were missing, both shipped here: + +**Speech out** — `server/speechService.js`, pluggable and degrading gracefully: +`browser` (Web Speech API — zero install, the default) → `piper` (local neural, best offline quality) → `say` (macOS) → PowerShell SAPI (Windows) → `espeak-ng`. +The browser backend matters: it means voice output works on a fresh clone with nothing installed, which is the difference between a feature people use and a feature people mean to set up. + +**Free-form routing** — previously an utterance that matched no pattern was a dead end. Now anything unmatched is forwarded to the Commander agent as a prompt. That single change converts voice from a *command remote* into a *conversation*, because the fallback is a full agent with the whole API surface rather than an error beep. + +Plus: `briefing` (spoken fleet summary assembled from supervisor findings + advisor output) and optional spoken announcements when the supervisor escalates. + +--- + +## 4. The Repo Atlas + +### The problem, stated precisely + +232 GitHub repos. 28 cloned locally. When starting anything new, the useful instinct is *"go see how we did X in Y"* — but that requires you to remember that Y exists and did X well. So you tell the agent, and if you forget, the knowledge is simply lost. Meanwhile an agent asked to find it cold burns thousands of tokens grepping a filesystem that doesn't even contain most of the repos. + +`CODEBASE_DOCUMENTATION.md` solved this *inside* one repo. The Atlas is the same idea one level up. + +### Three properties that make it work + +**1. Cloned-ness is irrelevant.** An entry describes a repo whether or not it's on this disk. `drain-the-lake` can be a first-class breadcrumb with a clone hint attached. This is the whole point — the map must cover the territory, not the local cache of it. + +**2. Quality is a first-class field, per-topic.** The ask is explicit and correct: Epic Survivors and HyFire2 are early work but *fully functioning*, and Drain the Lake is a rough prototype that might still have the best testing setup you've written. So quality is not a repo-level star rating; it's **per highlight**: + +```jsonc +"highlights": [ + { "topic": "testing", "quality": 5, "paths": ["tests/"], "notes": "best harness we have" }, + { "topic": "worldgen", "quality": 2, "notes": "prototype spaghetti — read for ideas, not patterns" } +], +"avoid": [ { "topic": "ui", "reason": "hand-rolled, superseded by roblox-game-kit" } ] +``` + +A repo can be simultaneously "don't copy this" and "copy exactly this one thing", which is the truth about real codebases and something a flat rating cannot express. + +**3. Sharing is subtractive and per-audience.** You own repos your team can't see; teammates differ from each other. So the master atlas lives locally and *compiles down* to audience bundles: + +``` +~/.agent-workspace/atlas/atlas.json # master — private, never shared + │ compile --audience core-team + ├─► atlas.core-team.json # entries visible to core-team + ├─► atlas.contractors.json # a strict subset + └─► atlas.public.json # public repos only +``` + +Each entry carries `visibility` (`private|team|public`) and `groups: [...]`. A bundle contains an entry only if the audience is in its groups. Two escape hatches: +- `redact: ["notes","paths"]` — list the repo as existing, hide the internals. Useful for "yes we have a payments service, no you can't see how." +- Group-scoped highlight overrides — the same repo can expose different highlights to different audiences. + +**Stated plainly, because it matters:** the bundle is *metadata distribution*, not access control. GitHub permissions are the enforcement. Anything in a bundle should be treated as readable by everyone in that audience. The compiler's job is to make it impossible to leak by accident, not to make leaking cryptographically hard. + +### Where entries come from + +Layer 1 — **in-repo manifest**, `.repo-atlas.json`, committed. The repo describes itself; it travels with the code; the agent working in that repo maintains it (same discipline as `CODEBASE_DOCUMENTATION.md`). + +Layer 2 — **central registry**, `~/.agent-workspace/atlas/registry.json`. Curated entries for repos with no manifest (forks, references, archived, never-cloned). This is where you write "drain-the-lake: rough, but the testing is worth reading." + +Layer 3 — **auto-discovery**, zero-effort baseline. Scan `~/GitHub` for git repos + `gh repo list` for the rest; infer kind, language, activity, fork/archive status. Produces a draft you curate rather than a blank page. Curated fields always win over inferred ones. + +### How an agent uses it + +The token-efficiency argument is the point, so the primary interface is a **digest**, not a search: + +``` +$ atlas digest --topics +roblox/luau box2d-luau(physics:5, testing:5) roblox-game-kit(mechanics:4) sabot-fps(fps-net:3) +hytopia zoo-game(data-compression:5, worldgen:4) hyfire2(matchmaking:3 ⚠ old) +monogame/c# epic-survivors(save-system:4 ⚠ old) beat-em-up-engine(input:4) +``` + +Paste that into a prompt (or a `CLAUDE.md`) and the agent *has the map* — it never needs to search to know that `box2d-luau` is where the good tests live. Then `atlas show box2d-luau` for detail and `atlas find testing --min-quality 4` when it needs to look sideways. + +Surfaces shipped: standalone CLI (`scripts/atlas.js`, no server required — symlink into `~/.claude/scripts/`), orchestrator REST API (`/api/atlas/*`), and an agent skill so any Claude/Codex session can query it without being told how. + +### Which repos this belongs in + +- **This repo (agent-workspace)** — the engine: schema, service, CLI, API, compiler. Public and open-source, so it ships with *zero* personal data; an example manifest only. +- **`~/.claude` (ai-claude-standards)** — the CLI symlink + skill, so every agent on the machine can query the atlas whether or not the orchestrator is running. +- **Your data** — `~/.agent-workspace/atlas/`, local, gitignored by construction. +- **Team distribution** — compiled bundles into whichever shared repo that audience already has access to (`agents-*` repos per the existing bootstrap/sync scripts). + +--- + +## 5. Order of work + +1. **Repo Atlas** — self-contained, immediately useful, no runtime risk. *(this branch)* +2. **Supervisor** in `observe` mode — accumulate findings, prove the rules are right before granting any autonomy. *(this branch)* +3. **Voice out + free-form routing** — makes 1 and 2 conversational. *(this branch)* +4. Promote supervisor to `assist` after reading a week of findings. *(you, later)* +5. Reach: Discord/mobile push for escalations. *(follow-up)* +6. Atlas write-back: agents propose highlights from work they just did, you approve. *(follow-up — this is what makes it stay current instead of rotting)* From d5b2bc5326d7715ee7f377b70d3128bb7898ca8d Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 09:39:07 +1000 Subject: [PATCH 02/69] feat(atlas): cross-repo knowledge map with per-audience sharing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The problem: hundreds of repos, a fraction cloned locally, and no way for an agent to know that a scruffy prototype holds the best test harness you own. Grepping ~/GitHub finds only what happens to be on disk and cannot express quality. The Repo Atlas is CODEBASE_DOCUMENTATION.md one level up: - Layered entries — discovery (disk + gh repo list) < in-repo .repo-atlas.json < your registry override. Your opinion always wins. - Cloned-ness is irrelevant. A repo that only exists on GitHub is a first-class breadcrumb with a clone hint attached. - Quality is per-topic, not per-repo, so 'don't copy this, except exactly this one thing' is expressible — which is the truth about real codebases. - Sharing is subtractive: entries compile down into audience bundles. private never leaves the machine and overrides group membership; team needs a group match; public goes everywhere. Local paths are always stripped, and per-group redaction can list a repo while hiding its internals. - 'atlas digest' emits a terse map to paste into a prompt, so an agent knows where to look without spending tokens finding out. Surfaces: standalone CLI (no server needed), /api/atlas/* routes, agent skill. 42 new unit tests; 652 total green. --- .repo-atlas.json | 50 ++++ CODEBASE_DOCUMENTATION.md | 32 +++ config/repo-atlas-topics.json | 34 +++ config/repo-atlas.example.json | 55 +++++ package.json | 1 + scripts/atlas.js | 343 +++++++++++++++++++++++++++ server/atlas/atlasCompiler.js | 133 +++++++++++ server/atlas/atlasDiscovery.js | 305 ++++++++++++++++++++++++ server/atlas/atlasQuery.js | 237 ++++++++++++++++++ server/atlas/atlasSchema.js | 286 ++++++++++++++++++++++ server/atlas/atlasStore.js | 193 +++++++++++++++ server/index.js | 10 + server/repoAtlasService.js | 291 +++++++++++++++++++++++ server/routes/atlasRoutes.js | 149 ++++++++++++ skills/public/repo-atlas/SKILL.md | 102 ++++++++ tests/unit/repoAtlasCompiler.test.js | 89 +++++++ tests/unit/repoAtlasQuery.test.js | 116 +++++++++ tests/unit/repoAtlasSchema.test.js | 97 ++++++++ tests/unit/repoAtlasService.test.js | 148 ++++++++++++ 19 files changed, 2671 insertions(+) create mode 100644 .repo-atlas.json create mode 100644 config/repo-atlas-topics.json create mode 100644 config/repo-atlas.example.json create mode 100755 scripts/atlas.js create mode 100644 server/atlas/atlasCompiler.js create mode 100644 server/atlas/atlasDiscovery.js create mode 100644 server/atlas/atlasQuery.js create mode 100644 server/atlas/atlasSchema.js create mode 100644 server/atlas/atlasStore.js create mode 100644 server/repoAtlasService.js create mode 100644 server/routes/atlasRoutes.js create mode 100644 skills/public/repo-atlas/SKILL.md create mode 100644 tests/unit/repoAtlasCompiler.test.js create mode 100644 tests/unit/repoAtlasQuery.test.js create mode 100644 tests/unit/repoAtlasSchema.test.js create mode 100644 tests/unit/repoAtlasService.test.js diff --git a/.repo-atlas.json b/.repo-atlas.json new file mode 100644 index 00000000..59a4f0cc --- /dev/null +++ b/.repo-atlas.json @@ -0,0 +1,50 @@ +{ + "id": "agent-workspace", + "name": "Agent Workspace", + "summary": "Multi-workspace orchestrator for running many CLI coding agents in parallel across repos and git worktrees. Express + Socket.IO backend, web client, Tauri desktop app.", + "kind": "tool", + "platforms": ["node"], + "languages": ["JavaScript"], + "dimension": "n/a", + "tags": ["orchestration", "agents", "worktrees"], + "status": "active", + "maturity": "production", + "visibility": "public", + "groups": [], + "highlights": [ + { + "topic": "agent-workflow", + "quality": 5, + "paths": ["server/agentManager.js", "server/agentSpawnHelper.js", "config/custom-agents.example.json"], + "notes": "Agent-agnostic launch registry — new CLI agents are pure config (flags, model/effort syntax, init delay), no code changes." + }, + { + "topic": "tooling", + "quality": 5, + "paths": ["server/repoAtlasService.js", "server/atlas/", "scripts/atlas.js"], + "notes": "The Repo Atlas itself: layered discovery/manifest/registry model with audience-scoped bundle compilation." + }, + { + "topic": "testing", + "quality": 4, + "paths": ["tests/unit/", "jest.config.js"], + "notes": "700+ fast unit tests over services; Playwright e2e runs on a dedicated port via scripts/run-e2e-safe.js." + }, + { + "topic": "ci", + "quality": 4, + "paths": ["scripts/release/", "scripts/tauri/run-tauri-build.js"], + "notes": "Cross-platform Tauri packaging with version-drift guards and bundle filename verification before release upload." + }, + { + "topic": "security", + "quality": 4, + "paths": ["server/utils/shellSafety.js", "server/policyService.js", "server/networkSecurityPolicy.js"], + "notes": "Allowlist validators for anything interpolated into a shell command, plus role policy and loopback-by-default bind rules." + } + ], + "avoid": [ + { "topic": "architecture", "reason": "server/index.js is a ~9k-line route god-file; follow server/routes/ and the service modules instead" } + ], + "seeAlso": [] +} diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index a29c5bff..e585f7e9 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -125,6 +125,23 @@ server/portRegistry.js - Port assignment + live service scanner (`/a server/commanderService.js - Top-level Commander PTY (Claude/Codex) + launch buffering ├─ Packaged CWD: uses `ORCHESTRATOR_DATA_DIR/commander` so desktop users can edit `CLAUDE.md` / `AGENTS.md` safely └─ First-run seed: copies the packaged `docs/COMMANDER_CLAUDE.md` into the Commander data directory when missing +server/repoAtlasService.js - Repo Atlas facade: one queryable map of every repo you own, cloned or not +├─ Layers (later wins): discovery (disk + `gh repo list`) < in-repo `.repo-atlas.json` manifest < `~/.agent-workspace/atlas/registry.json` (your override) +├─ Query: `find(topic)` ranked by per-topic quality 1-5, `digest()` compact paste-into-a-prompt map, `search()`, `topics()` +├─ Curation: `addHighlight()` / `addAvoid()` persist into the registry — quality is scored per topic, so a rough repo can still be the best example of one thing +└─ Sharing: `compile(audience)` emits audience-scoped bundles — `private` never leaves the machine, `team` needs a group match, `public` goes everywhere +server/atlas/atlasSchema.js - Entry normalization, layered merge, topic-alias folding (`config/repo-atlas-topics.json`), validation +server/atlas/atlasDiscovery.js - Local git scan (worktree siblings collapse into one project entry) + `gh repo list` + source merge +server/atlas/atlasStore.js - Persistence under `~/.agent-workspace/atlas/` (registry, discovery cache, compiled bundles) + in-repo manifest read/write +server/atlas/atlasQuery.js - Filters, topic lookup, topic index, digest rendering, entry description +server/atlas/atlasCompiler.js - Audience bundle compilation: visibility/group decisions, per-audience field redaction, always strips local-only fields +server/routes/atlasRoutes.js - Express router for `/api/atlas/*` (reads policy-`read`; curation and compile policy-`write`) +scripts/atlas.js - Standalone `atlas` CLI — runs without the server (`npm run atlas -- `) +config/repo-atlas-topics.json - Canonical topic vocabulary + aliases +config/repo-atlas.example.json - Annotated manifest example +.repo-atlas.json - This repo's own manifest +skills/public/repo-atlas/SKILL.md - Agent skill: query prior art instead of grepping the filesystem +tests/unit/repoAtlasSchema.test.js, repoAtlasQuery.test.js, repoAtlasCompiler.test.js, repoAtlasService.test.js - Atlas coverage (merge precedence, quality floors, sharing decisions, redaction) scripts/tauri/prepare-backend-resources.js - Tauri backend packager ├─ Bundles: server/client/config/templates/scripts + optional Node runtime into `src-tauri/resources/backend` ├─ Commander instructions: copies `docs/COMMANDER_CLAUDE.md` into `resources/backend/{COMMANDER_CLAUDE.md,CLAUDE.md,AGENTS.md}` for desktop builds @@ -648,6 +665,21 @@ GET /api/agent-providers/:providerId/sessions - List provider se POST /api/agent-providers/:providerId/resume-plan - Build provider-specific resume command/config plan GET /api/agent-providers/:providerId/history/search - Provider-scoped history search (conversation index source-aware) GET /api/agent-providers/:providerId/history/:id - Provider-scoped transcript retrieval + +GET /api/atlas/status - Repo Atlas health: where data lives, entry/highlight counts, discovery freshness +GET /api/atlas/entries?kind=&platform=&group=&query=&minQuality= - Filtered repo list +GET /api/atlas/entries/:id - One repo, merged across all layers, plus a rendered description +GET /api/atlas/find?topic=&minQuality= - "Who did this well?" — the primary query, ranked by per-topic quality +GET /api/atlas/topics - Topics in use and which repos hold them +GET /api/atlas/digest?groupBy=platform|kind|status&max= - Compact map intended for pasting into an agent prompt +GET /api/atlas/doctor - Validation report (errors + curation gaps) +POST /api/atlas/refresh - Re-run discovery (local scan + `gh repo list`) +PUT /api/atlas/entries/:id - Override entry fields in the registry +DELETE /api/atlas/entries/:id - Drop a registry override +POST /api/atlas/entries/:id/highlights - Record "this repo is good at X" (topic + quality + paths + notes) +POST /api/atlas/entries/:id/avoid - Record "do not copy X from this repo" +GET|POST /api/atlas/audiences - List/define sharing audiences +POST /api/atlas/compile - Compile an audience bundle (`dryRun: true` returns decisions without writing) ``` ### WebSocket Events diff --git a/config/repo-atlas-topics.json b/config/repo-atlas-topics.json new file mode 100644 index 00000000..2476df2f --- /dev/null +++ b/config/repo-atlas-topics.json @@ -0,0 +1,34 @@ +{ + "$comment": "Canonical topic vocabulary for the Repo Atlas. Topics are normalized through `aliases` so 'net', 'multiplayer' and 'networking' all resolve to one bucket. Unknown topics are kept as-is (kebab-cased) so the vocabulary never blocks you from recording something.", + "schemaVersion": 1, + "topics": [ + { "id": "networking", "label": "Networking / multiplayer", "aliases": ["net", "multiplayer", "netcode", "replication", "rpc", "sync"] }, + { "id": "data-persistence", "label": "Save data / persistence", "aliases": ["saves", "save-system", "datastore", "persistence", "storage", "profile-store"] }, + { "id": "data-compression", "label": "Data compression / packing", "aliases": ["compression", "bitpacking", "serialization", "serialisation"] }, + { "id": "anti-cheat", "label": "Anti-cheat / exploit protection", "aliases": ["cheat-protection", "exploit-protection", "server-authority", "hardening"] }, + { "id": "view-models", "label": "View models / first-person rigs", "aliases": ["viewmodel", "fp-arms", "first-person-view"] }, + { "id": "animation", "label": "Animation", "aliases": ["anims", "rigging", "skeletal"] }, + { "id": "physics", "label": "Physics", "aliases": ["collision", "rigidbody", "simulation"] }, + { "id": "worldgen", "label": "World / level generation", "aliases": ["procgen", "levelgen", "terrain", "wfc", "map-generation"] }, + { "id": "vfx", "label": "Visual effects", "aliases": ["particles", "shaders", "post-processing"] }, + { "id": "audio", "label": "Audio", "aliases": ["sound", "music", "sfx"] }, + { "id": "ui", "label": "UI / HUD", "aliases": ["hud", "menus", "interface", "frontend-ui"] }, + { "id": "input", "label": "Input handling", "aliases": ["controls", "controller", "keybinds"] }, + { "id": "ai-behaviour", "label": "Game AI / behaviour", "aliases": ["npc", "pathfinding", "behaviour-tree", "enemy-ai"] }, + { "id": "economy", "label": "Economy / progression balance", "aliases": ["progression", "balance", "monetization", "shop"] }, + { "id": "matchmaking", "label": "Matchmaking / lobbies", "aliases": ["lobby", "sessions", "party"] }, + { "id": "testing", "label": "Automated testing", "aliases": ["tests", "test-harness", "unit-tests", "e2e", "ci-tests"] }, + { "id": "ci", "label": "CI / release pipeline", "aliases": ["github-actions", "pipeline", "release", "packaging", "build"] }, + { "id": "performance", "label": "Performance work", "aliases": ["perf", "optimization", "optimisation", "profiling"] }, + { "id": "architecture", "label": "Overall architecture", "aliases": ["structure", "patterns", "module-layout"] }, + { "id": "tooling", "label": "Dev tooling / scripts", "aliases": ["scripts", "devtools", "cli"] }, + { "id": "agent-workflow", "label": "AI agent workflow / prompts", "aliases": ["prompts", "claude-md", "agent-setup", "skills"] }, + { "id": "api-integration", "label": "Third-party API integration", "aliases": ["integration", "webhooks", "sdk"] }, + { "id": "auth", "label": "Auth / accounts", "aliases": ["authentication", "login", "oauth", "sessions-auth"] }, + { "id": "database", "label": "Database / schema", "aliases": ["db", "sql", "migrations", "schema"] }, + { "id": "docs", "label": "Documentation quality", "aliases": ["documentation", "readme", "codebase-docs"] }, + { "id": "security", "label": "Security", "aliases": ["secrets", "sandboxing", "permissions"] }, + { "id": "graphics", "label": "Rendering / graphics", "aliases": ["render", "renderer", "voxel", "mesh"] }, + { "id": "mobile", "label": "Mobile / touch support", "aliases": ["touch", "responsive-mobile", "ios", "android"] } + ] +} diff --git a/config/repo-atlas.example.json b/config/repo-atlas.example.json new file mode 100644 index 00000000..78a8f381 --- /dev/null +++ b/config/repo-atlas.example.json @@ -0,0 +1,55 @@ +{ + "$comment": "Annotated example of a .repo-atlas.json manifest. Copy this into a repo root (or generate one with `node scripts/atlas.js init`), delete the $comment keys, and commit it. Every field is optional — discovery fills in what you leave out.", + + "id": "zoo-game", + "name": "Zoo Game", + "summary": "Multiplayer zoo tycoon built on Hytopia. Real players, real economy.", + + "$comment_classification": "kind: game|library|tool|website|service|reference|writing|experiment|infra|other", + "kind": "game", + "platforms": ["hytopia"], + "languages": ["TypeScript"], + "dimension": "3d", + "tags": ["multiplayer", "tycoon"], + + "$comment_state": "status: active|paused|prototype|archived|abandoned — maturity: production|beta|prototype|experiment", + "status": "active", + "maturity": "production", + + "$comment_sharing": "visibility: public (in every bundle) | team (only audiences listed in groups) | private (never leaves this machine, overrides groups)", + "visibility": "team", + "groups": ["core-team"], + + "$comment_highlights": "The point of the atlas: what is this repo worth reading FOR? quality is 1-5 per topic, so a rough repo can still be a 5 at one thing. Run `atlas topics --vocabulary` for the canonical topic list; unknown topics are kept as-is.", + "highlights": [ + { + "topic": "data-compression", + "quality": 5, + "paths": ["src/data/packSave.ts", "src/data/schema.ts"], + "notes": "Bitpacked player saves — 12x smaller than the JSON we started with. Copy this approach." + }, + { + "topic": "networking", + "quality": 3, + "paths": ["src/net/"], + "notes": "Works, but chatty. Fine as a reference for the handshake, not for the update loop." + } + ], + + "$comment_avoid": "Explicit do-not-copy markers. These hide the repo from `atlas find ` so nobody is sent somewhere you already regret.", + "avoid": [ + { "topic": "ui", "reason": "hand-rolled HUD, superseded by the shared component kit" } + ], + + "seeAlso": ["hyfire2", "hytopia-client-tracker"], + + "$comment_redaction": "Fields stripped from every shared bundle. Redactable: summary, notes, paths, highlights, avoid, seeAlso, tags. The repo still appears in the bundle — it is just opaque.", + "redact": [], + + "$comment_group_overrides": "Per-audience adjustments. Here contractors see that the repo exists and its quality scores, but not the notes or file paths.", + "groupOverrides": { + "contractors": { + "redact": ["notes", "paths"] + } + } +} diff --git a/package.json b/package.json index b3f26743..84d6bd32 100644 --- a/package.json +++ b/package.json @@ -21,6 +21,7 @@ "dev:web": "concurrently \"npm run dev:server\" \"npm run dev:client\"", "dev:web:safe": "ORCHESTRATOR_PORT=4001 CLIENT_PORT=4100 npm run dev:web", "site:preview": "node scripts/preview-site.js", + "atlas": "node scripts/atlas.js", "site:legal": "node scripts/render-legal-pages.js", "test": "npm run test:unit && npm run test:e2e", "test:unit": "jest", diff --git a/scripts/atlas.js b/scripts/atlas.js new file mode 100755 index 00000000..8b027f8f --- /dev/null +++ b/scripts/atlas.js @@ -0,0 +1,343 @@ +#!/usr/bin/env node +/** + * atlas — query and curate the Repo Atlas from anywhere. + * + * Runs standalone: no orchestrator server, no network unless you ask it to scan + * GitHub. Symlink it onto your PATH so every agent session can use it: + * ln -s /scripts/atlas.js ~/.local/bin/atlas + */ + +const path = require('path'); + +const RepoAtlasService = require('../server/repoAtlasService'); +const { KINDS, STATUSES, MATURITIES, VISIBILITIES, listCanonicalTopics } = require('../server/atlas/atlasSchema'); +const { formatDecisions } = require('../server/atlas/atlasCompiler'); + +const atlas = RepoAtlasService.getInstance(); + +function parseArgs(argv) { + const positionals = []; + const flags = {}; + + for (let i = 0; i < argv.length; i += 1) { + const token = argv[i]; + if (!token.startsWith('--')) { + positionals.push(token); + continue; + } + const [rawKey, inlineValue] = token.slice(2).split('='); + const key = rawKey.replace(/-([a-z])/g, (_, c) => c.toUpperCase()); + if (inlineValue !== undefined) { + flags[key] = inlineValue; + continue; + } + const next = argv[i + 1]; + if (next === undefined || next.startsWith('--')) { + flags[key] = true; + continue; + } + flags[key] = next; + i += 1; + } + + return { positionals, flags }; +} + +const listFlag = (value) => String(value === true ? '' : value || '') + .split(',') + .map((v) => v.trim()) + .filter(Boolean); + +const out = (text) => process.stdout.write(`${text}\n`); +const fail = (message) => { + process.stderr.write(`atlas: ${message}\n`); + process.exitCode = 1; +}; + +function printJson(value) { + out(JSON.stringify(value, null, 2)); +} + +const commands = { + async scan(_positionals, flags) { + out('Scanning… (local git repos + gh repo list)'); + const result = await atlas.refresh({ + scanGitHub: flags.noGithub !== true && flags.github !== 'false', + owner: flags.owner === true ? '' : String(flags.owner || ''), + limit: Number(flags.limit) || 300 + }); + out(`roots ${result.roots.join(', ')}`); + out(`local ${result.localCount}`); + out(`github ${result.githubAvailable ? result.githubCount : 'unavailable (is `gh` installed and authed?)'}`); + out(`total ${result.totalCount} repos on the map`); + out(''); + out('Next: `atlas doctor` to see what needs curating, `atlas note --topic X --quality N` to record what a repo is good at.'); + }, + + status() { + const status = atlas.getStatus(); + out(`atlas dir ${status.atlasDir}`); + out(`registry ${status.registryPath}`); + out(`scan roots ${status.scanRoots.join(', ')}`); + out(`repos ${status.entryCount} (${status.clonedCount} cloned locally)`); + out(`highlights ${status.highlightCount}`); + out(`audiences ${status.audiences.join(', ') || 'none configured'}`); + if (!status.discovery) { + out('discovery never run — start with `atlas scan`'); + } else { + out(`discovery ${status.discovery.generatedAt}${status.discovery.stale ? ' (stale — rerun `atlas scan`)' : ''}`); + } + }, + + list(_positionals, flags) { + const entries = atlas.search({ + kind: flags.kind === true ? '' : flags.kind, + platform: flags.platform === true ? '' : flags.platform, + group: flags.group === true ? '' : flags.group, + status: flags.status === true ? '' : flags.status, + language: flags.language === true ? '' : flags.language, + query: flags.query === true ? '' : flags.query, + minQuality: flags.minQuality, + includeForks: flags.noForks !== true, + includeArchived: flags.noArchived !== true + }); + + if (flags.json) return printJson(entries); + if (!entries.length) return out('No repos matched.'); + + for (const entry of entries) { + const marks = [ + entry.cloned ? 'local' : 'remote', + entry.isFork ? 'fork' : '', + entry.archived ? 'archived' : '' + ].filter(Boolean).join('/'); + const highlights = (entry.highlights || []).map((h) => `${h.topic}:${h.quality ?? '?'}`).join(' '); + out(`${entry.id.padEnd(34)} ${String(entry.kind || '').padEnd(10)} ${marks.padEnd(16)} ${highlights}`); + } + out(''); + out(`${entries.length} repos`); + return undefined; + }, + + show(positionals, flags) { + const id = positionals[0]; + if (!id) return fail('usage: atlas show '); + const entry = atlas.getEntry(id); + if (!entry) return fail(`no repo "${id}" on the map (try \`atlas list --query ${id}\`)`); + return flags.json ? printJson(entry) : out(atlas.describe(id)); + }, + + find(positionals, flags) { + const topic = positionals[0]; + if (!topic) return fail('usage: atlas find [--min-quality N]'); + + const hits = atlas.find(topic, { + minQuality: flags.minQuality, + includeAvoided: flags.includeAvoided === true + }); + if (flags.json) return printJson(hits); + if (!hits.length) { + out(`Nothing recorded for "${topic}".`); + out('Known topics: ' + atlas.topics().map((t) => t.topic).join(', ')); + return undefined; + } + + for (const hit of hits) { + const where = hit.cloned ? hit.localPath : (hit.remoteUrl || 'not cloned'); + out(`${String(hit.quality ?? '?')}/5 ${hit.id}${hit.stale ? ' ⚠old' : ''}`); + if (hit.notes) out(` ${hit.notes}`); + if (hit.paths?.length) out(` paths: ${hit.paths.join(', ')}`); + if (hit.caveat) out(` caveat: ${hit.caveat}`); + out(` ${where}`); + } + return undefined; + }, + + topics(_positionals, flags) { + if (flags.vocabulary) { + for (const topic of listCanonicalTopics()) out(`${topic.id.padEnd(20)} ${topic.label}`); + return undefined; + } + const index = atlas.topics(); + if (flags.json) return printJson(index); + if (!index.length) return out('No topics recorded yet. Use `atlas note --topic X --quality N`.'); + for (const row of index) { + out(`${row.topic.padEnd(20)} best ${row.best}/5 ${row.repos.join(', ')}`); + } + return undefined; + }, + + digest(_positionals, flags) { + const text = atlas.digest({ + groupBy: flags.groupBy === true ? 'platform' : (flags.groupBy || 'platform'), + maxPerBucket: Number(flags.max) || 8, + onlyWithHighlights: flags.all !== true + }); + out(text || 'Nothing to digest yet — record some highlights with `atlas note`.'); + }, + + note(positionals, flags) { + const id = positionals[0]; + if (!id || !flags.topic) return fail('usage: atlas note --topic [--quality 1-5] [--paths a,b] [--notes "..."]'); + const saved = atlas.addHighlight(id, { + topic: flags.topic, + quality: flags.quality === undefined ? null : Number(flags.quality), + paths: listFlag(flags.paths), + notes: flags.notes === true ? '' : String(flags.notes || '') + }); + return out(`Recorded: ${saved.id} → ${(saved.highlights || []).map((h) => `${h.topic}:${h.quality ?? '?'}`).join(', ')}`); + }, + + avoid(positionals, flags) { + const id = positionals[0]; + if (!id || !flags.topic) return fail('usage: atlas avoid --topic --reason "..."'); + const saved = atlas.addAvoid(id, { topic: flags.topic, reason: flags.reason === true ? '' : flags.reason }); + return out(`Marked do-not-copy: ${saved.id} → ${(saved.avoid || []).map((a) => a.topic).join(', ')}`); + }, + + set(positionals, flags) { + const id = positionals[0]; + if (!id) return fail('usage: atlas set [--visibility public|team|private] [--groups a,b] [--kind ...] [--status ...] [--summary "..."]'); + + const patch = {}; + if (flags.visibility) patch.visibility = flags.visibility; + if (flags.groups !== undefined) patch.groups = listFlag(flags.groups); + if (flags.kind) patch.kind = flags.kind; + if (flags.status) patch.status = flags.status; + if (flags.maturity) patch.maturity = flags.maturity; + if (flags.dimension) patch.dimension = flags.dimension; + if (flags.summary) patch.summary = String(flags.summary); + if (flags.platforms !== undefined) patch.platforms = listFlag(flags.platforms); + if (flags.tags !== undefined) patch.tags = listFlag(flags.tags); + if (flags.redact !== undefined) patch.redact = listFlag(flags.redact); + if (flags.quality !== undefined) patch.quality = Number(flags.quality); + + if (!Object.keys(patch).length) return fail('nothing to set'); + const saved = atlas.setEntry(id, patch); + return out(`Updated ${saved.id}: ${Object.keys(patch).join(', ')}`); + }, + + audience(positionals, flags) { + const action = positionals[0] || 'list'; + if (action === 'list') { + const audiences = atlas.listAudiences(); + if (!audiences.length) return out('No audiences yet. Add one: `atlas audience add core-team --label "Core team"`'); + for (const audience of audiences) { + out(`${audience.id.padEnd(20)} ${audience.label}${audience.outputPath ? ` → ${audience.outputPath}` : ''}`); + } + return undefined; + } + if (action === 'add') { + const id = positionals[1]; + if (!id) return fail('usage: atlas audience add [--label "..."] [--out ]'); + atlas.setAudience({ + id, + label: flags.label === true ? '' : String(flags.label || ''), + description: flags.description === true ? '' : String(flags.description || ''), + outputPath: flags.out === true ? '' : String(flags.out || '') + }); + return out(`Audience "${id}" saved. Tag repos into it with \`atlas set --visibility team --groups ${id}\`.`); + } + return fail(`unknown audience action "${action}"`); + }, + + compile(positionals, flags) { + const audience = positionals[0]; + if (!audience) return fail('usage: atlas compile [--dry-run] [--explain]'); + + const result = atlas.compile(audience, { write: flags.dryRun !== true }); + out(`${audience}: ${result.counts.included} shared / ${result.counts.excluded} withheld / ${result.counts.redacted} partly redacted`); + if (flags.explain || flags.dryRun) { + out(''); + out(formatDecisions(result.decisions, { onlyExcluded: flags.onlyExcluded === true })); + } + if (flags.dryRun) { + out(''); + out('Dry run — nothing written.'); + } else { + out(''); + for (const file of result.written) out(`wrote ${file}`); + } + return undefined; + }, + + doctor() { + const report = atlas.validate(); + out(`${report.entryCount} repos, ${report.curatedCount} curated, ${report.withHighlights} with highlights`); + + if (report.errors.length) { + out(''); + out('Errors:'); + for (const row of report.errors) out(` ${row.id}: ${row.errors.join('; ')}`); + } + if (report.warnings.length) { + out(''); + out(`Warnings (${report.warnings.length}):`); + for (const row of report.warnings.slice(0, 20)) out(` ${row.id}: ${row.warnings.join('; ')}`); + if (report.warnings.length > 20) out(` … and ${report.warnings.length - 20} more`); + } + if (!report.errors.length && !report.warnings.length) out('Everything checks out.'); + }, + + init(positionals, flags) { + const target = path.resolve(positionals[0] || process.cwd()); + const result = atlas.initManifest(target, { + visibility: flags.visibility === true ? 'private' : (flags.visibility || 'private'), + groups: listFlag(flags.groups) + }); + out(`Wrote ${result.path}`); + out('Fill in `summary`, `highlights` (topic + quality + paths) and `visibility`, then commit it.'); + }, + + help() { + out(`atlas — the map of every repo you own + + atlas scan [--no-github] [--owner X] rebuild the map from disk + GitHub + atlas status where things live, how fresh they are + atlas list [filters] [--json] list repos + atlas show [--json] everything known about one repo + atlas find [--min-quality N] who did this well? (the main query) + atlas topics [--vocabulary] topics in use / canonical vocabulary + atlas digest [--group-by kind] [--max N] compact map to paste into a prompt + + atlas note --topic X [--quality N] [--paths a,b] [--notes "..."] + atlas avoid --topic X --reason "..." + atlas set [--visibility ...] [--groups a,b] [--kind ...] [--summary "..."] + + atlas audience list | add [--label "..."] [--out ] + atlas compile [--dry-run] [--explain] + atlas doctor + atlas init [path] [--visibility ...] [--groups a,b] + +filters: --kind ${KINDS.join('|')} + --status ${STATUSES.join('|')} + --platform

--group --language --query + --min-quality N --no-forks --no-archived +values: maturity ${MATURITIES.join('|')} visibility ${VISIBILITIES.join('|')} + +Sharing model: entries are private by default. \`visibility: public\` goes in every +bundle, \`team\` goes only to audiences listed in its groups, \`private\` never leaves +this machine. Compiled bundles are metadata distribution — GitHub permissions are +still the real access control.`); + } +}; + +async function main() { + const [, , rawCommand, ...rest] = process.argv; + const command = rawCommand || 'help'; + const handler = commands[command]; + + if (!handler) { + fail(`unknown command "${command}" — try \`atlas help\``); + return; + } + + const { positionals, flags } = parseArgs(rest); + try { + await handler(positionals, flags); + } catch (error) { + fail(error?.message || String(error)); + } +} + +main(); diff --git a/server/atlas/atlasCompiler.js b/server/atlas/atlasCompiler.js new file mode 100644 index 00000000..90af4680 --- /dev/null +++ b/server/atlas/atlasCompiler.js @@ -0,0 +1,133 @@ +const { SCHEMA_VERSION, LOCAL_ONLY_FIELDS, kebab } = require('./atlasSchema'); + +const PUBLIC_AUDIENCE = 'public'; + +/** + * Decide whether one entry belongs in one audience's bundle. + * + * `private` is a hard kill switch — it wins over any group membership, so the + * safe thing to do when unsure about an entry is mark it private and move on. + */ +function decide(entry, audience) { + const audienceId = kebab(audience); + const visibility = entry?.visibility || 'private'; + const groups = entry?.groups || []; + + if (visibility === 'private') { + return { include: false, reason: 'visibility: private (never shared)' }; + } + if (visibility === 'public') { + return { include: true, reason: 'visibility: public' }; + } + if (audienceId === PUBLIC_AUDIENCE) { + return { include: false, reason: 'public bundle takes public entries only' }; + } + if (groups.includes(audienceId)) { + return { include: true, reason: `group match: ${audienceId}` }; + } + return { include: false, reason: `not in groups [${groups.join(', ') || 'none'}]` }; +} + +function stripFields(entry, fields) { + const next = { ...entry }; + for (const field of fields) { + if (field === 'paths') { + next.highlights = (next.highlights || []).map((h) => ({ ...h, paths: [] })); + continue; + } + if (field === 'notes') { + next.highlights = (next.highlights || []).map((h) => ({ ...h, notes: '' })); + next.avoid = (next.avoid || []).map((a) => ({ ...a, reason: '' })); + continue; + } + if (Array.isArray(next[field])) next[field] = []; + else if (typeof next[field] === 'string') next[field] = ''; + else delete next[field]; + } + return next; +} + +function redactForAudience(entry, audience) { + const audienceId = kebab(audience); + const override = entry?.groupOverrides?.[audienceId] || {}; + const redactions = [...new Set([...(entry.redact || []), ...(override.redact || [])])]; + + let next = { ...entry }; + if (override.summary !== undefined) next.summary = override.summary; + if (override.highlights !== undefined) next.highlights = override.highlights; + + if (redactions.length) next = stripFields(next, redactions); + + for (const field of LOCAL_ONLY_FIELDS) delete next[field]; + delete next.groupOverrides; + delete next.redact; + delete next.sources; + + return { entry: next, redactions }; +} + +/** + * Produce the artifact you actually hand to a teammate. + * + * This is metadata distribution, not access control — GitHub permissions remain + * the enforcement boundary. The compiler's job is to make accidental oversharing + * structurally hard, and to show its working via `decisions`. + */ +function compileBundle(entries, { audience, label = '', description = '' } = {}) { + const audienceId = kebab(audience); + if (!audienceId) throw new Error('compileBundle requires an audience'); + + const decisions = []; + const included = []; + + for (const entry of entries) { + const verdict = decide(entry, audienceId); + if (!verdict.include) { + decisions.push({ id: entry.id, included: false, reason: verdict.reason, redactions: [] }); + continue; + } + const { entry: redacted, redactions } = redactForAudience(entry, audienceId); + included.push(redacted); + decisions.push({ id: entry.id, included: true, reason: verdict.reason, redactions }); + } + + included.sort((a, b) => a.id.localeCompare(b.id)); + + return { + bundle: { + schemaVersion: SCHEMA_VERSION, + audience: audienceId, + label: label || audienceId, + description, + generatedAt: new Date().toISOString(), + entryCount: included.length, + entries: included + }, + decisions, + counts: { + total: entries.length, + included: included.length, + excluded: decisions.filter((d) => !d.included).length, + redacted: decisions.filter((d) => d.included && d.redactions.length).length + } + }; +} + +function formatDecisions(decisions, { onlyExcluded = false } = {}) { + const rows = onlyExcluded ? decisions.filter((d) => !d.included) : decisions; + return rows + .map((d) => { + const mark = d.included ? '+' : '-'; + const redaction = d.redactions.length ? ` (redacted: ${d.redactions.join(', ')})` : ''; + return `${mark} ${d.id.padEnd(32)} ${d.reason}${redaction}`; + }) + .join('\n'); +} + +module.exports = { + PUBLIC_AUDIENCE, + decide, + redactForAudience, + compileBundle, + formatDecisions +}; diff --git a/server/atlas/atlasDiscovery.js b/server/atlas/atlasDiscovery.js new file mode 100644 index 00000000..5c6c1230 --- /dev/null +++ b/server/atlas/atlasDiscovery.js @@ -0,0 +1,305 @@ +const fs = require('fs'); +const path = require('path'); +const os = require('os'); +const { execFile } = require('child_process'); + +const { kebab } = require('./atlasSchema'); + +const SKIP_DIRECTORIES = new Set([ + 'node_modules', '.git', '.svn', '.hg', 'dist', 'build', 'target', 'coverage', + 'vendor', '.next', '.nuxt', '.cache', '__pycache__', '.venv', 'venv', 'bin', 'obj' +]); + +// Path segments that classify a repo without having to look inside it. +const CATEGORY_KINDS = { + games: 'game', + game: 'game', + websites: 'website', + website: 'website', + web: 'website', + tools: 'tool', + tooling: 'tool', + automation: 'tool', + libraries: 'library', + libs: 'library', + writing: 'writing', + docs: 'reference', + experiments: 'experiment', + labs: 'experiment' +}; + +const PLATFORM_SEGMENTS = [ + 'roblox', 'hytopia', 'monogame', 'unity', 'godot', 'threejs', 'unreal', + 'phaser', 'pixi', 'love2d', 'bevy', 'react', 'nextjs', 'rails', 'django' +]; + +const REFERENCE_SEGMENTS = new Set(['_ref', '_refs', 'reference', '.reference', 'references', 'examples', 'third-party']); + +const EXTENSION_LANGUAGES = { + '.ts': 'TypeScript', '.tsx': 'TypeScript', '.js': 'JavaScript', '.jsx': 'JavaScript', + '.lua': 'Lua', '.luau': 'Luau', '.cs': 'C#', '.cpp': 'C++', '.cc': 'C++', '.c': 'C', + '.rs': 'Rust', '.py': 'Python', '.rb': 'Ruby', '.go': 'Go', '.java': 'Java', + '.kt': 'Kotlin', '.swift': 'Swift', '.sh': 'Shell', '.html': 'HTML', '.css': 'CSS', + '.md': 'Markdown', '.glsl': 'GLSL', '.wgsl': 'WGSL' +}; + +const execFileAsync = (command, args, options = {}) => new Promise((resolve) => { + execFile(command, args, { timeout: 15_000, windowsHide: true, maxBuffer: 8 * 1024 * 1024, ...options }, (error, stdout) => { + resolve(error ? null : String(stdout || '').trim()); + }); +}); + +function isGitRepoRoot(dirPath) { + const gitPath = path.join(dirPath, '.git'); + try { + // A worktree checkout has `.git` as a file pointing at the real gitdir. + return fs.existsSync(gitPath); + } catch { + return false; + } +} + +function parseOwnerRepo(remoteUrl) { + const match = String(remoteUrl || '').trim().match(/github\.com[/:]([^/]+)\/([^/]+?)(?:\.git)?$/i); + if (!match) return null; + return { owner: match[1], repo: match[2], nameWithOwner: `${match[1]}/${match[2]}` }; +} + +/** + * `~/GitHub/games/roblox/sabot-fps/master` describes a repo whose real + * identity is `sabot-fps` — the worktree folder is an implementation detail. + */ +function resolveProjectRoot(repoDir) { + const base = path.basename(repoDir); + if (/^work\d+$/i.test(base) || base.toLowerCase() === 'master' || base.toLowerCase() === 'main') { + return { projectRoot: path.dirname(repoDir), worktreeLayout: true }; + } + return { projectRoot: repoDir, worktreeLayout: false }; +} + +function inferFromPath(projectRoot, roots) { + const root = roots.find((r) => projectRoot.startsWith(r)); + const relative = root ? path.relative(root, projectRoot) : path.basename(projectRoot); + const segments = relative.split(path.sep).filter(Boolean).map((s) => s.toLowerCase()); + const contextSegments = segments.slice(0, -1); + + let kind = null; + const platforms = []; + let isReference = false; + + for (const segment of contextSegments) { + if (!kind && CATEGORY_KINDS[segment]) kind = CATEGORY_KINDS[segment]; + if (PLATFORM_SEGMENTS.includes(segment) && !platforms.includes(segment)) platforms.push(segment); + if (REFERENCE_SEGMENTS.has(segment)) isReference = true; + } + + return { kind: isReference ? 'reference' : kind, platforms, categoryPath: segments.slice(0, -1).join('/') }; +} + +function censusLanguages(repoDir, { maxFiles = 400 } = {}) { + const counts = new Map(); + let seen = 0; + const queue = [{ dir: repoDir, depth: 0 }]; + + while (queue.length && seen < maxFiles) { + const { dir, depth } = queue.shift(); + if (depth > 3) continue; + let dirents = []; + try { + dirents = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + continue; + } + for (const dirent of dirents) { + if (seen >= maxFiles) break; + if (dirent.isDirectory()) { + if (SKIP_DIRECTORIES.has(dirent.name) || dirent.name.startsWith('.')) continue; + queue.push({ dir: path.join(dir, dirent.name), depth: depth + 1 }); + continue; + } + const language = EXTENSION_LANGUAGES[path.extname(dirent.name).toLowerCase()]; + if (!language || language === 'Markdown') continue; + counts.set(language, (counts.get(language) || 0) + 1); + seen += 1; + } + } + + return [...counts.entries()] + .sort((a, b) => b[1] - a[1]) + .slice(0, 3) + .map(([language]) => language); +} + +async function readGitFacts(repoDir) { + const [remoteUrl, lastCommit] = await Promise.all([ + execFileAsync('git', ['-C', repoDir, 'remote', 'get-url', 'origin']), + execFileAsync('git', ['-C', repoDir, 'log', '-1', '--format=%cI']) + ]); + return { remoteUrl: remoteUrl || '', lastActivity: lastCommit || null }; +} + +function walkForRepos(root, maxDepth) { + const found = []; + if (!fs.existsSync(root)) return found; + + const queue = [{ dir: root, depth: 0 }]; + while (queue.length) { + const { dir, depth } = queue.shift(); + if (isGitRepoRoot(dir)) { + found.push(dir); + // Do not descend into a repo — nested worktrees are siblings, not children. + continue; + } + if (depth >= maxDepth) continue; + + let dirents = []; + try { + dirents = fs.readdirSync(dir, { withFileTypes: true }); + } catch { + continue; + } + for (const dirent of dirents) { + if (!dirent.isDirectory()) continue; + if (SKIP_DIRECTORIES.has(dirent.name)) continue; + queue.push({ dir: path.join(dir, dirent.name), depth: depth + 1 }); + } + } + return found; +} + +/** + * Discover repos on disk. Worktree siblings collapse into a single entry keyed + * by the project root, so `master/` and `work1..work8` never appear as nine repos. + */ +async function scanLocalRepos({ roots, maxDepth = 6, languageCensus = true } = {}) { + const searchRoots = (Array.isArray(roots) && roots.length ? roots : [path.join(os.homedir(), 'GitHub')]) + .map((r) => path.resolve(r)); + + const byProject = new Map(); + + for (const root of searchRoots) { + for (const repoDir of walkForRepos(root, maxDepth)) { + const { projectRoot, worktreeLayout } = resolveProjectRoot(repoDir); + const existing = byProject.get(projectRoot); + if (existing) { + // Prefer `master`/`main` as the representative checkout. + const base = path.basename(repoDir).toLowerCase(); + if (base === 'master' || base === 'main') existing.repoDir = repoDir; + continue; + } + byProject.set(projectRoot, { projectRoot, repoDir, worktreeLayout, searchRoot: root }); + } + } + + const entries = []; + for (const { projectRoot, repoDir, worktreeLayout } of byProject.values()) { + const { remoteUrl, lastActivity } = await readGitFacts(repoDir); + const parsed = parseOwnerRepo(remoteUrl); + const inferred = inferFromPath(projectRoot, searchRoots); + + entries.push({ + __source: 'discovery', + id: kebab(parsed?.repo || path.basename(projectRoot)), + name: path.basename(projectRoot), + repo: parsed?.nameWithOwner || '', + owner: parsed?.owner || '', + kind: inferred.kind || undefined, + platforms: inferred.platforms, + languages: languageCensus ? censusLanguages(repoDir) : [], + tags: inferred.categoryPath ? [kebab(inferred.categoryPath)] : [], + localPath: projectRoot, + cloned: true, + worktreeLayout, + remoteUrl, + lastActivity, + lastScannedAt: new Date().toISOString() + }); + } + + return entries.sort((a, b) => a.id.localeCompare(b.id)); +} + +/** + * Repos that exist on GitHub but are not on this disk still belong on the map — + * that is the whole point of an atlas rather than a directory listing. + */ +async function listGitHubRepos({ limit = 300, owner = '' } = {}) { + const args = ['repo', 'list']; + if (owner) args.push(owner); + args.push('--limit', String(limit), '--json', 'nameWithOwner,name,description,visibility,primaryLanguage,updatedAt,isFork,isArchived,url'); + + const stdout = await execFileAsync('gh', args); + if (!stdout) return { available: false, entries: [] }; + + let rows = []; + try { + rows = JSON.parse(stdout); + } catch { + return { available: false, entries: [] }; + } + + const entries = rows.map((row) => ({ + __source: 'discovery', + id: kebab(row?.name), + name: String(row?.name || ''), + repo: String(row?.nameWithOwner || ''), + owner: String(row?.nameWithOwner || '').split('/')[0] || '', + summary: String(row?.description || ''), + visibility: String(row?.visibility || '').toLowerCase() === 'public' ? 'public' : 'private', + languages: row?.primaryLanguage?.name ? [row.primaryLanguage.name] : [], + isFork: row?.isFork === true, + archived: row?.isArchived === true, + status: row?.isArchived === true ? 'archived' : undefined, + kind: row?.isFork === true ? 'reference' : undefined, + remoteUrl: String(row?.url || ''), + lastActivity: row?.updatedAt || null, + cloned: false, + lastScannedAt: new Date().toISOString() + })).filter((entry) => entry.id); + + return { available: true, entries }; +} + +/** + * Merge the two discovery sources by repo identity. Local wins on paths and + * language detail; GitHub wins on visibility, fork/archive state, and description. + */ +function mergeDiscovery(localEntries = [], githubEntries = []) { + const byKey = new Map(); + const keyFor = (entry) => (entry.repo ? entry.repo.toLowerCase() : `id:${entry.id}`); + + for (const entry of githubEntries) byKey.set(keyFor(entry), { ...entry }); + + for (const entry of localEntries) { + const key = keyFor(entry); + const existing = byKey.get(key) || byKey.get(`id:${entry.id}`); + if (!existing) { + byKey.set(key, { ...entry }); + continue; + } + byKey.set(key, { + ...existing, + ...entry, + summary: entry.summary || existing.summary, + visibility: existing.visibility || entry.visibility, + isFork: existing.isFork ?? entry.isFork, + archived: existing.archived ?? entry.archived, + status: existing.status || entry.status, + languages: entry.languages?.length ? entry.languages : existing.languages, + lastActivity: existing.lastActivity || entry.lastActivity, + cloned: true + }); + } + + return [...byKey.values()].sort((a, b) => a.id.localeCompare(b.id)); +} + +module.exports = { + scanLocalRepos, + listGitHubRepos, + mergeDiscovery, + parseOwnerRepo, + resolveProjectRoot, + inferFromPath, + censusLanguages +}; diff --git a/server/atlas/atlasQuery.js b/server/atlas/atlasQuery.js new file mode 100644 index 00000000..fabf2daf --- /dev/null +++ b/server/atlas/atlasQuery.js @@ -0,0 +1,237 @@ +const { normalizeTopic, kebab } = require('./atlasSchema'); + +const STALE_AFTER_DAYS = 365; + +// `null`, `''` and `undefined` all mean "no floor" — Number() turns the first +// two into 0, which would silently drop every uncurated repo. +function qualityFloor(value) { + if (value === null || value === undefined || value === '' || value === true) return null; + const num = Number(value); + return Number.isFinite(num) ? num : null; +} + +function isStale(entry) { + const ms = Date.parse(String(entry?.lastActivity || '')); + if (!Number.isFinite(ms)) return false; + return (Date.now() - ms) / 86_400_000 > STALE_AFTER_DAYS; +} + +function matchesText(entry, query) { + if (!query) return true; + const needle = String(query).trim().toLowerCase(); + if (!needle) return true; + const haystack = [ + entry.id, entry.name, entry.repo, entry.summary, + ...(entry.tags || []), ...(entry.platforms || []), ...(entry.languages || []), + ...(entry.highlights || []).flatMap((h) => [h.topic, h.notes]) + ].join(' ').toLowerCase(); + return haystack.includes(needle); +} + +function filterEntries(entries, filters = {}) { + const { + kind = '', platform = '', group = '', status = '', language = '', + query = '', includeForks = true, includeArchived = true, minQuality = null + } = filters; + + const wantPlatform = kebab(platform); + const wantGroup = kebab(group); + const wantLanguage = String(language || '').trim().toLowerCase(); + const wantKind = String(kind || '').trim().toLowerCase(); + const wantStatus = String(status || '').trim().toLowerCase(); + const floor = qualityFloor(minQuality); + + return entries.filter((entry) => { + if (wantKind && entry.kind !== wantKind) return false; + if (wantStatus && entry.status !== wantStatus) return false; + if (wantPlatform && !(entry.platforms || []).includes(wantPlatform)) return false; + if (wantGroup && !(entry.groups || []).includes(wantGroup)) return false; + if (wantLanguage && !(entry.languages || []).some((l) => l.toLowerCase() === wantLanguage)) return false; + if (!includeForks && entry.isFork) return false; + if (!includeArchived && (entry.archived || entry.status === 'archived')) return false; + if (floor !== null) { + const best = bestQuality(entry); + if (best === null || best < floor) return false; + } + return matchesText(entry, query); + }); +} + +function bestQuality(entry) { + const scores = (entry.highlights || []).map((h) => h.quality).filter((q) => Number.isFinite(q)); + if (entry.quality) scores.push(entry.quality); + if (!scores.length) return null; + return Math.max(...scores); +} + +/** + * Topic lookup is the load-bearing query: "who did X well?". Results are ranked + * by highlight quality, then recency, so the top answer is the one worth reading. + */ +function findByTopic(entries, topic, { minQuality = null, includeAvoided = false } = {}) { + const wanted = normalizeTopic(topic); + if (!wanted) return []; + const floor = qualityFloor(minQuality); + + const hits = []; + for (const entry of entries) { + const avoided = (entry.avoid || []).find((a) => a.topic === wanted); + if (avoided && !includeAvoided) continue; + + const highlight = (entry.highlights || []).find((h) => h.topic === wanted); + if (!highlight) continue; + if (floor !== null && (highlight.quality === null || highlight.quality < floor)) continue; + + hits.push({ + id: entry.id, + name: entry.name, + repo: entry.repo, + cloned: entry.cloned === true, + localPath: entry.localPath || null, + remoteUrl: entry.remoteUrl || '', + status: entry.status, + maturity: entry.maturity, + stale: isStale(entry), + topic: wanted, + quality: highlight.quality, + paths: highlight.paths, + notes: highlight.notes, + caveat: avoided ? avoided.reason : '' + }); + } + + return hits.sort((a, b) => { + const byQuality = (b.quality || 0) - (a.quality || 0); + if (byQuality) return byQuality; + return Number(a.stale) - Number(b.stale); + }); +} + +function topicIndex(entries) { + const index = new Map(); + for (const entry of entries) { + for (const highlight of entry.highlights || []) { + const bucket = index.get(highlight.topic) || []; + bucket.push({ id: entry.id, quality: highlight.quality }); + index.set(highlight.topic, bucket); + } + } + return [...index.entries()] + .map(([topic, repos]) => ({ + topic, + count: repos.length, + best: repos.reduce((max, r) => Math.max(max, r.quality || 0), 0), + repos: repos.sort((a, b) => (b.quality || 0) - (a.quality || 0)).map((r) => r.id) + })) + .sort((a, b) => b.count - a.count || a.topic.localeCompare(b.topic)); +} + +function bucketKey(entry, groupBy) { + if (groupBy === 'kind') return entry.kind || 'other'; + if (groupBy === 'status') return entry.status || 'active'; + const platform = (entry.platforms || [])[0]; + if (platform) return platform; + return entry.kind || 'other'; +} + +function describeEntryInline(entry) { + const highlights = (entry.highlights || []) + .slice() + .sort((a, b) => (b.quality || 0) - (a.quality || 0)) + .slice(0, 3) + .map((h) => `${h.topic}:${h.quality ?? '?'}`) + .join(', '); + + const flags = []; + if (isStale(entry) || entry.status === 'archived') flags.push('⚠old'); + if (!entry.cloned) flags.push('remote'); + if (entry.isFork) flags.push('fork'); + + const suffix = flags.length ? ` ${flags.join('/')}` : ''; + return highlights ? `${entry.id}(${highlights}${suffix})` : `${entry.id}(${flags.join('/') || '—'})`; +} + +/** + * The digest is the thing you paste into a prompt. It trades completeness for + * token cost on purpose: enough for an agent to know where to look, no more. + */ +function buildDigest(entries, { groupBy = 'platform', maxPerBucket = 8, onlyWithHighlights = true } = {}) { + const source = onlyWithHighlights ? entries.filter((e) => (e.highlights || []).length) : entries; + const buckets = new Map(); + + for (const entry of source) { + const key = bucketKey(entry, groupBy); + const bucket = buckets.get(key) || []; + bucket.push(entry); + buckets.set(key, bucket); + } + + const lines = []; + const sortedBuckets = [...buckets.entries()].sort((a, b) => b[1].length - a[1].length || a[0].localeCompare(b[0])); + const width = Math.min(16, Math.max(...sortedBuckets.map(([key]) => key.length), 8)); + + for (const [key, bucketEntries] of sortedBuckets) { + const ranked = bucketEntries + .slice() + .sort((a, b) => (bestQuality(b) || 0) - (bestQuality(a) || 0)) + .slice(0, maxPerBucket) + .map(describeEntryInline); + const omitted = bucketEntries.length - ranked.length; + const tail = omitted > 0 ? ` +${omitted} more` : ''; + lines.push(`${key.padEnd(width)} ${ranked.join(' ')}${tail}`); + } + + return lines.join('\n'); +} + +function describeEntry(entry) { + const lines = []; + lines.push(`# ${entry.name || entry.id}${entry.repo ? ` (${entry.repo})` : ''}`); + if (entry.summary) lines.push(entry.summary); + lines.push(''); + + const facts = [ + ['kind', entry.kind], + ['status', entry.status], + ['maturity', entry.maturity], + ['platforms', (entry.platforms || []).join(', ')], + ['languages', (entry.languages || []).join(', ')], + ['visibility', entry.visibility], + ['groups', (entry.groups || []).join(', ')], + ['last activity', entry.lastActivity ? entry.lastActivity.slice(0, 10) : ''], + ['local path', entry.cloned ? entry.localPath : `not cloned${entry.remoteUrl ? ` — ${entry.remoteUrl}` : ''}`], + ['sources', (entry.sources || []).join(' < ')] + ].filter(([, value]) => value); + + for (const [label, value] of facts) lines.push(`${label.padEnd(14)} ${value}`); + + if ((entry.highlights || []).length) { + lines.push('', 'Worth reading:'); + for (const highlight of entry.highlights) { + const paths = highlight.paths?.length ? ` [${highlight.paths.join(', ')}]` : ''; + lines.push(` ${String(highlight.quality ?? '?')}/5 ${highlight.topic}${paths}`); + if (highlight.notes) lines.push(` ${highlight.notes}`); + } + } + + if ((entry.avoid || []).length) { + lines.push('', 'Do not copy:'); + for (const avoid of entry.avoid) lines.push(` ${avoid.topic} — ${avoid.reason || 'no reason recorded'}`); + } + + if ((entry.seeAlso || []).length) lines.push('', `See also: ${entry.seeAlso.join(', ')}`); + + return lines.join('\n'); +} + +module.exports = { + STALE_AFTER_DAYS, + isStale, + bestQuality, + filterEntries, + findByTopic, + topicIndex, + buildDigest, + describeEntry, + describeEntryInline +}; diff --git a/server/atlas/atlasSchema.js b/server/atlas/atlasSchema.js new file mode 100644 index 00000000..08d63c5f --- /dev/null +++ b/server/atlas/atlasSchema.js @@ -0,0 +1,286 @@ +const fs = require('fs'); +const path = require('path'); + +const SCHEMA_VERSION = 1; + +const KINDS = ['game', 'library', 'tool', 'website', 'service', 'reference', 'writing', 'experiment', 'infra', 'other']; +const STATUSES = ['active', 'paused', 'prototype', 'archived', 'abandoned']; +const MATURITIES = ['production', 'beta', 'prototype', 'experiment']; +const VISIBILITIES = ['public', 'team', 'private']; +const DIMENSIONS = ['2d', '3d', 'mixed', 'n/a']; +const REDACTABLE_FIELDS = ['summary', 'notes', 'paths', 'highlights', 'avoid', 'seeAlso', 'tags']; + +// Never leave this machine in a compiled bundle: absolute paths expose the +// local user/folder layout and say nothing useful to anyone else. +const LOCAL_ONLY_FIELDS = ['localPath', 'cloned', 'worktreeLayout', 'lastScannedAt']; + +const TOPICS_CONFIG_PATH = path.join(__dirname, '..', '..', 'config', 'repo-atlas-topics.json'); + +let topicIndexCache = null; + +function loadTopicIndex() { + if (topicIndexCache) return topicIndexCache; + + const index = { canonical: new Map(), labels: new Map() }; + try { + const raw = JSON.parse(fs.readFileSync(TOPICS_CONFIG_PATH, 'utf8')); + for (const topic of Array.isArray(raw?.topics) ? raw.topics : []) { + const id = kebab(topic?.id); + if (!id) continue; + index.canonical.set(id, id); + index.labels.set(id, String(topic?.label || id)); + for (const alias of Array.isArray(topic?.aliases) ? topic.aliases : []) { + const key = kebab(alias); + if (key) index.canonical.set(key, id); + } + } + } catch { + // A missing or broken vocabulary must never stop the atlas from working. + } + + topicIndexCache = index; + return index; +} + +function resetTopicIndexCache() { + topicIndexCache = null; +} + +function kebab(value) { + return String(value || '') + .trim() + .toLowerCase() + .replace(/[\s_/]+/g, '-') + .replace(/[^a-z0-9.-]/g, '') + .replace(/-+/g, '-') + .replace(/^-|-$/g, ''); +} + +function normalizeTopic(value) { + const key = kebab(value); + if (!key) return ''; + return loadTopicIndex().canonical.get(key) || key; +} + +function topicLabel(topicId) { + return loadTopicIndex().labels.get(topicId) || topicId; +} + +function listCanonicalTopics() { + const index = loadTopicIndex(); + return [...index.labels.entries()].map(([id, label]) => ({ id, label })); +} + +function oneOf(value, allowed, fallback) { + const key = String(value || '').trim().toLowerCase(); + return allowed.includes(key) ? key : fallback; +} + +function stringList(value) { + const source = Array.isArray(value) ? value : String(value || '').split(','); + const out = []; + for (const item of source) { + const text = String(item || '').trim(); + if (text && !out.includes(text)) out.push(text); + } + return out; +} + +function slugList(value) { + const out = []; + for (const item of stringList(value)) { + const slug = kebab(item); + if (slug && !out.includes(slug)) out.push(slug); + } + return out; +} + +function qualityScore(value) { + const num = Number(value); + if (!Number.isFinite(num)) return null; + return Math.min(5, Math.max(1, Math.round(num))); +} + +function isoDate(value) { + const ms = Date.parse(String(value || '')); + return Number.isFinite(ms) ? new Date(ms).toISOString() : null; +} + +function normalizeHighlight(raw) { + const topic = normalizeTopic(raw?.topic || raw?.name); + if (!topic) return null; + return { + topic, + quality: qualityScore(raw?.quality), + paths: stringList(raw?.paths || raw?.path), + notes: String(raw?.notes || raw?.note || '').trim() + }; +} + +function normalizeAvoid(raw) { + const topic = normalizeTopic(raw?.topic || raw?.name); + if (!topic) return null; + return { topic, reason: String(raw?.reason || raw?.notes || '').trim() }; +} + +/** + * One entry per topic, last write wins. Later entries are corrections — if you + * say testing is a 5 and then a 2, you mean 2. + */ +function normalizeList(value, normalizer) { + const byTopic = new Map(); + for (const item of Array.isArray(value) ? value : []) { + const normalized = normalizer(item); + if (normalized) byTopic.set(normalized.topic, normalized); + } + return [...byTopic.values()]; +} + +function normalizeGroupOverrides(value) { + const out = {}; + if (!value || typeof value !== 'object') return out; + for (const [group, override] of Object.entries(value)) { + const key = kebab(group); + if (!key || !override || typeof override !== 'object') continue; + const entry = {}; + if (override.redact !== undefined) { + entry.redact = slugList(override.redact).filter((f) => REDACTABLE_FIELDS.includes(f)); + } + if (override.summary !== undefined) entry.summary = String(override.summary || '').trim(); + if (override.highlights !== undefined) entry.highlights = normalizeList(override.highlights, normalizeHighlight); + if (Object.keys(entry).length) out[key] = entry; + } + return out; +} + +/** + * Normalize any partial atlas entry into the canonical shape. + * `strict: false` (the default) keeps only the keys the caller actually + * supplied, which is what makes layered merging meaningful. + */ +function normalizeEntry(raw = {}, { strict = false } = {}) { + const has = (key) => raw && Object.prototype.hasOwnProperty.call(raw, key); + const entry = {}; + + const id = kebab(raw?.id || raw?.name || raw?.repo); + if (id) entry.id = id; + + if (strict || has('name')) entry.name = String(raw?.name || raw?.id || '').trim(); + if (strict || has('repo') || has('nameWithOwner')) { + entry.repo = String(raw?.repo || raw?.nameWithOwner || '').trim(); + } + if (strict || has('owner')) entry.owner = String(raw?.owner || '').trim(); + if (strict || has('summary')) entry.summary = String(raw?.summary || raw?.description || '').trim(); + if (strict || has('description')) entry.summary = entry.summary || String(raw?.description || '').trim(); + + if (strict || has('kind')) entry.kind = oneOf(raw?.kind, KINDS, 'other'); + if (strict || has('status')) entry.status = oneOf(raw?.status, STATUSES, 'active'); + if (strict || has('maturity')) entry.maturity = oneOf(raw?.maturity, MATURITIES, 'prototype'); + if (strict || has('visibility')) entry.visibility = oneOf(raw?.visibility, VISIBILITIES, 'private'); + if (strict || has('dimension')) entry.dimension = oneOf(raw?.dimension, DIMENSIONS, 'n/a'); + + if (strict || has('platforms') || has('platform')) entry.platforms = slugList(raw?.platforms || raw?.platform); + if (strict || has('languages') || has('language')) entry.languages = stringList(raw?.languages || raw?.language); + if (strict || has('tags')) entry.tags = slugList(raw?.tags); + if (strict || has('groups') || has('group')) entry.groups = slugList(raw?.groups || raw?.group); + if (strict || has('seeAlso')) entry.seeAlso = slugList(raw?.seeAlso); + + if (strict || has('quality')) entry.quality = qualityScore(raw?.quality); + if (strict || has('highlights')) entry.highlights = normalizeList(raw?.highlights, normalizeHighlight); + if (has('highlightsAdd')) entry.highlightsAdd = normalizeList(raw?.highlightsAdd, normalizeHighlight); + if (strict || has('avoid')) entry.avoid = normalizeList(raw?.avoid, normalizeAvoid); + + if (strict || has('redact')) { + entry.redact = slugList(raw?.redact).filter((field) => REDACTABLE_FIELDS.includes(field)); + } + if (strict || has('groupOverrides')) entry.groupOverrides = normalizeGroupOverrides(raw?.groupOverrides); + + if (strict || has('localPath')) entry.localPath = String(raw?.localPath || '').trim() || null; + if (strict || has('cloned')) entry.cloned = raw?.cloned === true; + if (strict || has('remoteUrl')) entry.remoteUrl = String(raw?.remoteUrl || '').trim(); + if (strict || has('isFork')) entry.isFork = raw?.isFork === true; + if (strict || has('archived')) entry.archived = raw?.archived === true; + if (strict || has('worktreeLayout')) entry.worktreeLayout = raw?.worktreeLayout === true; + if (strict || has('lastActivity')) entry.lastActivity = isoDate(raw?.lastActivity); + if (strict || has('lastScannedAt')) entry.lastScannedAt = isoDate(raw?.lastScannedAt); + + if (strict && !entry.name) entry.name = entry.id || ''; + + return entry; +} + +/** + * Layer entries lowest-precedence first. Present keys win; absent keys are + * left alone, so a registry override only has to state what it disagrees with. + */ +function mergeEntries(...layers) { + const merged = {}; + const sources = []; + + for (const layer of layers) { + if (!layer) continue; + const { __source: source, highlightsAdd, ...fields } = layer; + if (source && !sources.includes(source)) sources.push(source); + + for (const [key, value] of Object.entries(fields)) { + if (value === undefined) continue; + if (value === null && merged[key] !== undefined && merged[key] !== null) continue; + if (Array.isArray(value) && value.length === 0 && Array.isArray(merged[key]) && merged[key].length) continue; + if (typeof value === 'string' && value === '' && merged[key]) continue; + merged[key] = value; + } + + // Layers may arrive raw (a manifest read straight off disk), so additions + // are normalized here rather than trusting the caller to have done it. + const additions = normalizeList(highlightsAdd, normalizeHighlight); + if (additions.length) { + const existing = Array.isArray(merged.highlights) ? merged.highlights : []; + merged.highlights = normalizeList([...existing, ...additions], normalizeHighlight); + } + } + + const normalized = normalizeEntry(merged, { strict: true }); + normalized.sources = sources; + return normalized; +} + +function validateEntry(entry) { + const errors = []; + const warnings = []; + + if (!entry?.id) errors.push('missing id'); + if (!VISIBILITIES.includes(entry?.visibility)) errors.push(`invalid visibility "${entry?.visibility}"`); + if (entry?.visibility === 'team' && !(entry?.groups || []).length) { + warnings.push('visibility "team" with no groups — this entry lands in no bundle'); + } + if (!entry?.summary) warnings.push('no summary — agents get very little from this entry'); + if (!(entry?.highlights || []).length) warnings.push('no highlights — nothing for an agent to be pointed at'); + + for (const highlight of entry?.highlights || []) { + if (highlight.quality === null) warnings.push(`highlight "${highlight.topic}" has no quality score`); + } + for (const field of entry?.redact || []) { + if (!REDACTABLE_FIELDS.includes(field)) errors.push(`cannot redact unknown field "${field}"`); + } + + return { id: entry?.id || '(unknown)', ok: errors.length === 0, errors, warnings }; +} + +module.exports = { + SCHEMA_VERSION, + KINDS, + STATUSES, + MATURITIES, + VISIBILITIES, + DIMENSIONS, + REDACTABLE_FIELDS, + LOCAL_ONLY_FIELDS, + kebab, + normalizeTopic, + topicLabel, + listCanonicalTopics, + resetTopicIndexCache, + normalizeEntry, + mergeEntries, + validateEntry +}; diff --git a/server/atlas/atlasStore.js b/server/atlas/atlasStore.js new file mode 100644 index 00000000..cf271c88 --- /dev/null +++ b/server/atlas/atlasStore.js @@ -0,0 +1,193 @@ +const fs = require('fs'); +const path = require('path'); + +const { getAgentWorkspaceDir } = require('../utils/pathUtils'); +const { SCHEMA_VERSION, kebab, normalizeEntry } = require('./atlasSchema'); + +const MANIFEST_FILENAME = '.repo-atlas.json'; +const MANIFEST_SEARCH_SUBDIRS = ['', 'master', 'main']; +const DISCOVERY_CACHE_TTL_MS = 12 * 60 * 60 * 1000; + +function atlasDir() { + const override = String(process.env.AGENT_WORKSPACE_ATLAS_DIR || '').trim(); + return override ? path.resolve(override) : path.join(getAgentWorkspaceDir(), 'atlas'); +} + +function registryPath() { + return path.join(atlasDir(), 'registry.json'); +} + +function discoveryCachePath() { + return path.join(atlasDir(), 'discovery.json'); +} + +function bundlesDir() { + return path.join(atlasDir(), 'bundles'); +} + +function ensureDir(dirPath) { + fs.mkdirSync(dirPath, { recursive: true }); + return dirPath; +} + +function readJson(filePath, fallback = null) { + try { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); + } catch { + return fallback; + } +} + +function writeJson(filePath, value) { + ensureDir(path.dirname(filePath)); + fs.writeFileSync(filePath, `${JSON.stringify(value, null, 2)}\n`, 'utf8'); + return filePath; +} + +function emptyRegistry() { + return { + schemaVersion: SCHEMA_VERSION, + scanRoots: [], + audiences: [], + defaults: { visibility: 'private', groups: [] }, + entries: {} + }; +} + +function normalizeAudience(raw) { + const id = kebab(raw?.id || raw); + if (!id) return null; + return { + id, + label: String(raw?.label || id).trim(), + description: String(raw?.description || '').trim(), + // Where compiled bundles for this audience should be copied, if anywhere. + outputPath: String(raw?.outputPath || '').trim() + }; +} + +function loadRegistry() { + const raw = readJson(registryPath(), null); + if (!raw) return emptyRegistry(); + + const registry = emptyRegistry(); + registry.scanRoots = Array.isArray(raw.scanRoots) ? raw.scanRoots.map(String).filter(Boolean) : []; + registry.audiences = (Array.isArray(raw.audiences) ? raw.audiences : []).map(normalizeAudience).filter(Boolean); + registry.defaults = { + visibility: String(raw?.defaults?.visibility || 'private'), + groups: Array.isArray(raw?.defaults?.groups) ? raw.defaults.groups.map(kebab).filter(Boolean) : [] + }; + + const entries = raw.entries && typeof raw.entries === 'object' ? raw.entries : {}; + for (const [key, value] of Object.entries(entries)) { + const id = kebab(value?.id || key); + if (!id) continue; + registry.entries[id] = { ...normalizeEntry({ ...value, id }), id }; + } + + return registry; +} + +function saveRegistry(registry) { + const next = { + schemaVersion: SCHEMA_VERSION, + scanRoots: registry?.scanRoots || [], + audiences: (registry?.audiences || []).map(normalizeAudience).filter(Boolean), + defaults: registry?.defaults || { visibility: 'private', groups: [] }, + entries: registry?.entries || {} + }; + return writeJson(registryPath(), next); +} + +function upsertRegistryEntry(id, patch) { + const registry = loadRegistry(); + const key = kebab(id); + if (!key) throw new Error('An atlas entry needs an id'); + const existing = registry.entries[key] || { id: key }; + registry.entries[key] = { ...existing, ...normalizeEntry({ ...patch, id: key }), id: key }; + saveRegistry(registry); + return registry.entries[key]; +} + +function removeRegistryEntry(id) { + const registry = loadRegistry(); + const key = kebab(id); + const existed = Boolean(registry.entries[key]); + delete registry.entries[key]; + saveRegistry(registry); + return existed; +} + +function loadDiscoveryCache({ maxAgeMs = DISCOVERY_CACHE_TTL_MS } = {}) { + const cached = readJson(discoveryCachePath(), null); + if (!cached || !Array.isArray(cached.entries)) return null; + const generatedMs = Date.parse(String(cached.generatedAt || '')); + if (!Number.isFinite(generatedMs)) return null; + if (Date.now() - generatedMs > maxAgeMs) return { ...cached, stale: true }; + return { ...cached, stale: false }; +} + +function saveDiscoveryCache(entries, meta = {}) { + return writeJson(discoveryCachePath(), { + schemaVersion: SCHEMA_VERSION, + generatedAt: new Date().toISOString(), + ...meta, + entries + }); +} + +function manifestPathFor(projectRoot) { + for (const subdir of MANIFEST_SEARCH_SUBDIRS) { + const candidate = path.join(projectRoot, subdir, MANIFEST_FILENAME); + if (fs.existsSync(candidate)) return candidate; + } + return null; +} + +/** + * Read the in-repo manifest for a discovered repo. A repo describing itself is + * always more current than anything cached centrally, so this is a live read. + */ +function loadManifest(projectRoot) { + if (!projectRoot) return null; + const filePath = manifestPathFor(projectRoot); + if (!filePath) return null; + const raw = readJson(filePath, null); + if (!raw) return null; + return { ...normalizeEntry(raw), __source: 'manifest', __manifestPath: filePath }; +} + +function writeManifest(projectRoot, entry) { + const target = fs.existsSync(path.join(projectRoot, 'master')) + ? path.join(projectRoot, 'master') + : projectRoot; + return writeJson(path.join(target, MANIFEST_FILENAME), entry); +} + +function saveBundle(audienceId, bundle, outputPath = '') { + const written = [writeJson(path.join(bundlesDir(), `atlas.${kebab(audienceId)}.json`), bundle)]; + if (outputPath) written.push(writeJson(path.resolve(outputPath), bundle)); + return written; +} + +module.exports = { + MANIFEST_FILENAME, + DISCOVERY_CACHE_TTL_MS, + atlasDir, + registryPath, + discoveryCachePath, + bundlesDir, + emptyRegistry, + loadRegistry, + saveRegistry, + upsertRegistryEntry, + removeRegistryEntry, + loadDiscoveryCache, + saveDiscoveryCache, + manifestPathFor, + loadManifest, + writeManifest, + saveBundle, + readJson, + writeJson +}; diff --git a/server/index.js b/server/index.js index 347b7a62..2916f4fa 100644 --- a/server/index.js +++ b/server/index.js @@ -110,6 +110,8 @@ const { ProjectTypeService } = require('./projectTypeService'); const { ContinuityService } = require('./continuityService'); const { QuickLinksService } = require('./quickLinksService'); const { RecommendationsService } = require('./recommendationsService'); +const { RepoAtlasService } = require('./repoAtlasService'); +const { createAtlasRoutes } = require('./routes/atlasRoutes'); const { ProductLauncherService } = require('./productLauncherService'); const { CommanderService } = require('./commanderService'); const { ConversationService } = require('./conversationService'); @@ -349,6 +351,7 @@ greenfieldService.setProjectTypeService(projectTypeService); const continuityService = ContinuityService.getInstance(); const quickLinksService = QuickLinksService.getInstance(); const recommendationsService = RecommendationsService.getInstance(); +const repoAtlasService = RepoAtlasService.getInstance({ logger }); const activityFeed = ActivityFeedService.getInstance(); activityFeed.setIO(io); activityFeed.track('server.started', { port: Number(process.env.ORCHESTRATOR_PORT || 9460) }); @@ -1336,6 +1339,13 @@ app.get('/health', (req, res) => { }); }); +app.use('/api/atlas', createAtlasRoutes({ + repoAtlasService, + logger, + requireRead: requirePolicyAction('read'), + requireWrite: requirePolicyAction('write') +})); + app.get('/api/app-info', (req, res) => { res.json(readAppInfo()); }); diff --git a/server/repoAtlasService.js b/server/repoAtlasService.js new file mode 100644 index 00000000..1ed0675e --- /dev/null +++ b/server/repoAtlasService.js @@ -0,0 +1,291 @@ +const os = require('os'); +const path = require('path'); + +const schema = require('./atlas/atlasSchema'); +const store = require('./atlas/atlasStore'); +const discovery = require('./atlas/atlasDiscovery'); +const query = require('./atlas/atlasQuery'); +const compiler = require('./atlas/atlasCompiler'); +const { getProjectsRoot, getLegacyProjectsRoot } = require('./utils/pathUtils'); + +const ATLAS_CACHE_MS = 60_000; + +function defaultScanRoots() { + const roots = [getLegacyProjectsRoot(), getProjectsRoot(), path.join(os.homedir(), 'GitHub')]; + return [...new Set(roots.map((r) => path.resolve(r)))]; +} + +/** + * The Repo Atlas: one queryable map of every repo you own, cloned or not. + * + * Entries are layered lowest-precedence first — + * discovery (what the machine can see) + * < manifest (what the repo says about itself, `.repo-atlas.json`) + * < registry (what you say about it, and you always win). + */ +class RepoAtlasService { + constructor({ logger = console } = {}) { + this.logger = logger; + this.cache = null; + this.cachedAt = 0; + } + + static getInstance(options = {}) { + if (!RepoAtlasService.instance) { + RepoAtlasService.instance = new RepoAtlasService(options); + } + return RepoAtlasService.instance; + } + + invalidate() { + this.cache = null; + this.cachedAt = 0; + } + + getScanRoots() { + const registry = store.loadRegistry(); + const configured = (registry.scanRoots || []).map((r) => path.resolve(r.replace(/^~(?=$|\/)/, os.homedir()))); + return configured.length ? configured : defaultScanRoots(); + } + + /** + * Re-run discovery against disk and GitHub, then cache it. Discovery is the + * slow layer, so everything else reads from this snapshot. + */ + async refresh({ scanLocal = true, scanGitHub = true, limit = 300, owner = '' } = {}) { + const roots = this.getScanRoots(); + const localEntries = scanLocal ? await discovery.scanLocalRepos({ roots }) : []; + const github = scanGitHub ? await discovery.listGitHubRepos({ limit, owner }) : { available: false, entries: [] }; + const merged = discovery.mergeDiscovery(localEntries, github.entries); + + store.saveDiscoveryCache(merged, { + roots, + localCount: localEntries.length, + githubCount: github.entries.length, + githubAvailable: github.available + }); + this.invalidate(); + + return { + roots, + localCount: localEntries.length, + githubCount: github.entries.length, + githubAvailable: github.available, + totalCount: merged.length + }; + } + + loadLayers() { + const cached = store.loadDiscoveryCache({ maxAgeMs: Number.MAX_SAFE_INTEGER }); + const discovered = cached?.entries || []; + const registry = store.loadRegistry(); + + const byId = new Map(); + for (const entry of discovered) { + if (!entry?.id) continue; + byId.set(entry.id, { discovery: { ...entry, __source: 'discovery' } }); + } + + for (const entry of discovered) { + if (!entry?.id || !entry.localPath) continue; + const manifest = store.loadManifest(entry.localPath); + if (manifest) byId.get(entry.id).manifest = manifest; + } + + for (const [id, entry] of Object.entries(registry.entries || {})) { + const slot = byId.get(id) || {}; + slot.registry = { ...entry, __source: 'registry' }; + byId.set(id, slot); + } + + return { byId, registry, discoveryMeta: cached }; + } + + getEntries({ force = false } = {}) { + if (!force && this.cache && Date.now() - this.cachedAt < ATLAS_CACHE_MS) return this.cache; + + const { byId, registry } = this.loadLayers(); + const entries = []; + + for (const [id, layers] of byId.entries()) { + const merged = schema.mergeEntries( + { id, visibility: registry.defaults?.visibility, groups: registry.defaults?.groups, __source: 'defaults' }, + layers.discovery, + layers.manifest, + layers.registry + ); + merged.id = id; + merged.sources = (merged.sources || []).filter((s) => s !== 'defaults'); + entries.push(merged); + } + + entries.sort((a, b) => a.id.localeCompare(b.id)); + this.cache = entries; + this.cachedAt = Date.now(); + return entries; + } + + getEntry(id, options = {}) { + const key = schema.kebab(id); + return this.getEntries(options).find((entry) => entry.id === key) || null; + } + + search(filters = {}) { + return query.filterEntries(this.getEntries(), filters); + } + + find(topic, options = {}) { + return query.findByTopic(this.getEntries(), topic, options); + } + + topics() { + return query.topicIndex(this.getEntries()); + } + + digest(options = {}) { + return query.buildDigest(this.getEntries(), options); + } + + describe(id) { + const entry = this.getEntry(id); + return entry ? query.describeEntry(entry) : null; + } + + /** + * The single highest-value curation action: "remember that this repo did this + * thing well." Everything else about an entry can stay auto-discovered. + */ + addHighlight(id, { topic, quality = null, paths = [], notes = '' } = {}) { + const normalizedTopic = schema.normalizeTopic(topic); + if (!normalizedTopic) throw new Error('addHighlight requires a topic'); + + const registry = store.loadRegistry(); + const key = schema.kebab(id); + const existing = registry.entries[key] || { id: key }; + const highlights = (existing.highlights || []).filter((h) => h.topic !== normalizedTopic); + highlights.push({ + topic: normalizedTopic, + quality: quality === null ? null : Number(quality), + paths: Array.isArray(paths) ? paths : String(paths || '').split(',').map((p) => p.trim()).filter(Boolean), + notes: String(notes || '') + }); + + const saved = store.upsertRegistryEntry(key, { ...existing, highlights }); + this.invalidate(); + return saved; + } + + addAvoid(id, { topic, reason = '' } = {}) { + const normalizedTopic = schema.normalizeTopic(topic); + if (!normalizedTopic) throw new Error('addAvoid requires a topic'); + + const registry = store.loadRegistry(); + const key = schema.kebab(id); + const existing = registry.entries[key] || { id: key }; + const avoid = (existing.avoid || []).filter((a) => a.topic !== normalizedTopic); + avoid.push({ topic: normalizedTopic, reason: String(reason || '') }); + + const saved = store.upsertRegistryEntry(key, { ...existing, avoid }); + this.invalidate(); + return saved; + } + + setEntry(id, patch = {}) { + const saved = store.upsertRegistryEntry(id, patch); + this.invalidate(); + return saved; + } + + removeEntry(id) { + const removed = store.removeRegistryEntry(id); + this.invalidate(); + return removed; + } + + listAudiences() { + return store.loadRegistry().audiences || []; + } + + setAudience({ id, label = '', description = '', outputPath = '' } = {}) { + const registry = store.loadRegistry(); + const key = schema.kebab(id); + if (!key) throw new Error('An audience needs an id'); + const audiences = (registry.audiences || []).filter((a) => a.id !== key); + audiences.push({ id: key, label: label || key, description, outputPath }); + registry.audiences = audiences; + store.saveRegistry(registry); + this.invalidate(); + return audiences; + } + + compile(audience, { write = true } = {}) { + const meta = this.listAudiences().find((a) => a.id === schema.kebab(audience)) || {}; + const result = compiler.compileBundle(this.getEntries(), { + audience, + label: meta.label, + description: meta.description + }); + result.written = write ? store.saveBundle(audience, result.bundle, meta.outputPath) : []; + return result; + } + + validate() { + const entries = this.getEntries(); + const reports = entries.map((entry) => schema.validateEntry(entry)); + return { + entryCount: entries.length, + curatedCount: entries.filter((e) => (e.sources || []).some((s) => s === 'registry' || s === 'manifest')).length, + withHighlights: entries.filter((e) => (e.highlights || []).length).length, + errors: reports.filter((r) => !r.ok), + warnings: reports.filter((r) => r.ok && r.warnings.length) + }; + } + + initManifest(projectRoot, seed = {}) { + const resolved = path.resolve(projectRoot); + const existing = this.getEntries().find((entry) => entry.localPath === resolved); + const draft = schema.normalizeEntry({ + id: seed.id || existing?.id || path.basename(resolved), + name: seed.name || existing?.name || path.basename(resolved), + summary: seed.summary || existing?.summary || '', + kind: seed.kind || existing?.kind || 'other', + platforms: seed.platforms || existing?.platforms || [], + languages: seed.languages || existing?.languages || [], + status: seed.status || existing?.status || 'active', + maturity: seed.maturity || existing?.maturity || 'prototype', + visibility: seed.visibility || 'private', + groups: seed.groups || [], + highlights: seed.highlights || [], + avoid: seed.avoid || [] + }, { strict: true }); + + delete draft.sources; + return { path: store.writeManifest(resolved, draft), entry: draft }; + } + + getStatus() { + const meta = store.loadDiscoveryCache({ maxAgeMs: store.DISCOVERY_CACHE_TTL_MS }); + const entries = this.getEntries(); + return { + atlasDir: store.atlasDir(), + registryPath: store.registryPath(), + scanRoots: this.getScanRoots(), + entryCount: entries.length, + clonedCount: entries.filter((e) => e.cloned).length, + highlightCount: entries.reduce((sum, e) => sum + (e.highlights || []).length, 0), + audiences: this.listAudiences().map((a) => a.id), + discovery: meta + ? { generatedAt: meta.generatedAt, stale: meta.stale, githubAvailable: meta.githubAvailable !== false } + : null + }; + } +} + +module.exports = RepoAtlasService; +module.exports.RepoAtlasService = RepoAtlasService; +module.exports.schema = schema; +module.exports.store = store; +module.exports.discovery = discovery; +module.exports.query = query; +module.exports.compiler = compiler; +module.exports.defaultScanRoots = defaultScanRoots; diff --git a/server/routes/atlasRoutes.js b/server/routes/atlasRoutes.js new file mode 100644 index 00000000..321f6e7b --- /dev/null +++ b/server/routes/atlasRoutes.js @@ -0,0 +1,149 @@ +const express = require('express'); + +const passthrough = (req, res, next) => next(); + +const asList = (value) => String(value === undefined || value === null ? '' : value) + .split(',') + .map((item) => item.trim()) + .filter(Boolean); + +/** + * REST surface for the Repo Atlas. Reads are policy-`read`, curation is + * policy-`write`, and compiling a shareable bundle is treated as a write + * because it produces an artifact that leaves this machine. + */ +function createAtlasRoutes({ repoAtlasService, logger = console, requireRead = passthrough, requireWrite = passthrough } = {}) { + const router = express.Router(); + + const handle = (label, handler) => async (req, res) => { + try { + await handler(req, res); + } catch (error) { + logger.error(`Atlas: ${label} failed`, { error: error.message, stack: error.stack }); + res.status(400).json({ ok: false, error: error.message }); + } + }; + + router.get('/status', requireRead, handle('status', (req, res) => { + res.json({ ok: true, status: repoAtlasService.getStatus() }); + })); + + router.get('/entries', requireRead, handle('list entries', (req, res) => { + const entries = repoAtlasService.search({ + kind: req.query.kind, + platform: req.query.platform, + group: req.query.group, + status: req.query.status, + language: req.query.language, + query: req.query.query || req.query.q, + minQuality: req.query.minQuality, + includeForks: req.query.includeForks !== 'false', + includeArchived: req.query.includeArchived !== 'false' + }); + res.json({ ok: true, count: entries.length, entries }); + })); + + router.get('/entries/:id', requireRead, handle('get entry', (req, res) => { + const entry = repoAtlasService.getEntry(req.params.id); + if (!entry) return res.status(404).json({ ok: false, error: `No atlas entry "${req.params.id}"` }); + return res.json({ ok: true, entry, description: repoAtlasService.describe(req.params.id) }); + })); + + router.get('/find', requireRead, handle('find', (req, res) => { + const topic = req.query.topic || req.query.q; + if (!topic) return res.status(400).json({ ok: false, error: 'topic is required' }); + const hits = repoAtlasService.find(topic, { + minQuality: req.query.minQuality, + includeAvoided: req.query.includeAvoided === 'true' + }); + return res.json({ ok: true, topic, count: hits.length, hits }); + })); + + router.get('/topics', requireRead, handle('topics', (req, res) => { + res.json({ ok: true, topics: repoAtlasService.topics() }); + })); + + router.get('/digest', requireRead, handle('digest', (req, res) => { + const digest = repoAtlasService.digest({ + groupBy: req.query.groupBy || 'platform', + maxPerBucket: Number(req.query.max) || 8, + onlyWithHighlights: req.query.all !== 'true' + }); + res.json({ ok: true, digest }); + })); + + router.get('/doctor', requireRead, handle('doctor', (req, res) => { + res.json({ ok: true, report: repoAtlasService.validate() }); + })); + + router.post('/refresh', requireWrite, handle('refresh', async (req, res) => { + const result = await repoAtlasService.refresh({ + scanLocal: req.body?.scanLocal !== false, + scanGitHub: req.body?.scanGitHub !== false, + owner: req.body?.owner || '', + limit: Number(req.body?.limit) || 300 + }); + res.json({ ok: true, ...result }); + })); + + router.put('/entries/:id', requireWrite, handle('update entry', (req, res) => { + const entry = repoAtlasService.setEntry(req.params.id, req.body || {}); + res.json({ ok: true, entry }); + })); + + router.delete('/entries/:id', requireWrite, handle('delete entry', (req, res) => { + res.json({ ok: true, removed: repoAtlasService.removeEntry(req.params.id) }); + })); + + router.post('/entries/:id/highlights', requireWrite, handle('add highlight', (req, res) => { + const entry = repoAtlasService.addHighlight(req.params.id, { + topic: req.body?.topic, + quality: req.body?.quality, + paths: Array.isArray(req.body?.paths) ? req.body.paths : asList(req.body?.paths), + notes: req.body?.notes || '' + }); + res.json({ ok: true, entry }); + })); + + router.post('/entries/:id/avoid', requireWrite, handle('add avoid', (req, res) => { + const entry = repoAtlasService.addAvoid(req.params.id, { + topic: req.body?.topic, + reason: req.body?.reason || '' + }); + res.json({ ok: true, entry }); + })); + + router.get('/audiences', requireRead, handle('list audiences', (req, res) => { + res.json({ ok: true, audiences: repoAtlasService.listAudiences() }); + })); + + router.post('/audiences', requireWrite, handle('save audience', (req, res) => { + const audiences = repoAtlasService.setAudience({ + id: req.body?.id, + label: req.body?.label || '', + description: req.body?.description || '', + outputPath: req.body?.outputPath || '' + }); + res.json({ ok: true, audiences }); + })); + + router.post('/compile', requireWrite, handle('compile bundle', (req, res) => { + const audience = req.body?.audience; + if (!audience) return res.status(400).json({ ok: false, error: 'audience is required' }); + const dryRun = req.body?.dryRun === true; + const result = repoAtlasService.compile(audience, { write: !dryRun }); + return res.json({ + ok: true, + audience, + dryRun, + counts: result.counts, + decisions: result.decisions, + written: result.written, + bundle: dryRun ? result.bundle : undefined + }); + })); + + return router; +} + +module.exports = { createAtlasRoutes }; diff --git a/skills/public/repo-atlas/SKILL.md b/skills/public/repo-atlas/SKILL.md new file mode 100644 index 00000000..5def9db9 --- /dev/null +++ b/skills/public/repo-atlas/SKILL.md @@ -0,0 +1,102 @@ +--- +name: repo-atlas +description: Query the Repo Atlas — a map of every repo the user owns, cloned or not, with per-topic quality scores. Use whenever you need prior art ("how did we do networking/save data/testing before?"), when starting a new project and want to reuse an existing approach, when the user says "I remember doing this somewhere", or when you would otherwise grep the filesystem looking for a repo. Also use to record what a repo turned out to be good at. +allowed-tools: Bash, Read +--- + +# Repo Atlas + +One queryable map of every repo the user owns — including repos that are **not cloned on this machine**. Ask it before searching the filesystem. + +## Why this exists + +There are hundreds of repos. Grepping `~/GitHub` finds only the fraction that happen to be cloned, costs thousands of tokens, and cannot tell you that a scruffy prototype has the best test harness in the collection. The atlas answers both "where is it?" and "is it worth copying?". + +## The one command that matters + +```bash +atlas find # who did this well, and where in the repo +``` + +Example: +```bash +$ atlas find data-compression +5/5 zoo-game + bitpacked player save — 12x smaller than the JSON we started with + paths: src/data/packSave.ts + /home/ab/GitHub/games/hytopia/zoo-game +``` + +Results are ranked by quality (1–5, recorded **per topic**), and repos the user has explicitly marked do-not-copy for that topic are excluded. `⚠old` means untouched for over a year — still readable, just check it against current conventions. + +## Reading the map + +```bash +atlas digest # compact whole-map overview — cheap, paste-able +atlas topics # every topic anyone has recorded, and who has it +atlas show # everything known about one repo +atlas list --platform roblox --no-forks +atlas find testing --min-quality 4 +``` + +`atlas digest` is the right first call when you want orientation rather than an answer. It is deliberately terse: + +``` +roblox box2d-luau(physics:5, testing:5) drain-the-lake(testing:4 ⚠old) +hytopia zoo-game(data-compression:5, worldgen:4) +``` + +## Not cloned? Still useful + +An entry with `remote` instead of a local path exists only on GitHub. That is fine — read it without cloning: + +```bash +gh api repos///contents/ --jq '.content' | base64 -d +gh repo clone / /tmp/ -- --depth 1 # if you need the whole thing +``` + +Never clone into the user's `~/GitHub` tree to "just take a look" — use `/tmp`. + +## Recording what you learn + +When you finish work that produced something genuinely reusable — or discover that a repo's approach to something is excellent or awful — write it down. This is what keeps the map alive. + +```bash +atlas note --topic --quality 1-5 --paths a/b.ts,c/ --notes "why it is worth copying" +atlas avoid --topic --reason "why nobody should copy this" +``` + +Guidance on scores: **5** = copy this exactly; **4** = solid, adapt it; **3** = works, read for ideas; **2** = only if nothing better; **1** = cautionary example. Score the *topic*, not the repo — a prototype can be a 5 at one thing and a 2 at everything else. + +Use `atlas topics --vocabulary` for canonical topic names. Aliases fold automatically (`multiplayer` → `networking`, `tests` → `testing`), and unrecognized topics are kept rather than dropped. + +## Describing a repo from inside it + +If you are working in a repo with no `.repo-atlas.json`, create one and commit it: + +```bash +atlas init . # seeds from what discovery already knows +``` + +Then fill in `summary`, `highlights`, and `visibility`. Treat it like `CODEBASE_DOCUMENTATION.md`: update it when the repo gains or loses something worth pointing at. + +## Sharing (be careful here) + +Entries are **private by default**. Compiled bundles are what get shared with teammates: + +```bash +atlas audience list +atlas compile --dry-run --explain # always dry-run first +``` + +- `visibility: public` — in every bundle. +- `visibility: team` — only for audiences named in its `groups`. +- `visibility: private` — never shared, overrides groups. + +Never change a repo's `visibility` or `groups` on the user's behalf. Bundles are metadata distribution, not access control — GitHub permissions are the real boundary. + +## Setup + +If `atlas` is not on PATH, run it directly: `node /scripts/atlas.js `. +If it reports no repos, the map has never been built: `atlas scan`. +The orchestrator exposes the same data at `GET /api/atlas/find?topic=...`, `/api/atlas/digest`, `/api/atlas/entries`. diff --git a/tests/unit/repoAtlasCompiler.test.js b/tests/unit/repoAtlasCompiler.test.js new file mode 100644 index 00000000..35e711a3 --- /dev/null +++ b/tests/unit/repoAtlasCompiler.test.js @@ -0,0 +1,89 @@ +const { compileBundle, decide, redactForAudience } = require('../../server/atlas/atlasCompiler'); +const { normalizeEntry } = require('../../server/atlas/atlasSchema'); + +const entry = (overrides) => normalizeEntry({ + id: 'sample', + name: 'Sample', + summary: 'A repo', + visibility: 'private', + highlights: [{ topic: 'testing', quality: 5, paths: ['tests/'], notes: 'good harness' }], + localPath: '/home/someone/GitHub/sample', + cloned: true, + ...overrides +}, { strict: true }); + +describe('atlasCompiler', () => { + test('private entries never reach a bundle, even with a matching group', () => { + const verdict = decide(entry({ visibility: 'private', groups: ['core-team'] }), 'core-team'); + expect(verdict.include).toBe(false); + expect(verdict.reason).toMatch(/never shared/); + }); + + test('public entries reach every audience', () => { + expect(decide(entry({ visibility: 'public' }), 'contractors').include).toBe(true); + expect(decide(entry({ visibility: 'public' }), 'public').include).toBe(true); + }); + + test('team entries reach only the audiences listed in their groups', () => { + const teamEntry = entry({ visibility: 'team', groups: ['core-team'] }); + expect(decide(teamEntry, 'core-team').include).toBe(true); + expect(decide(teamEntry, 'contractors').include).toBe(false); + }); + + test('the public bundle refuses team entries outright', () => { + const teamEntry = entry({ visibility: 'team', groups: ['public'] }); + expect(decide(teamEntry, 'public').include).toBe(false); + }); + + test('local-only fields are always stripped from shared entries', () => { + const { entry: shared } = redactForAudience(entry({ visibility: 'public' }), 'core-team'); + expect(shared.localPath).toBeUndefined(); + expect(shared.cloned).toBeUndefined(); + expect(shared.groupOverrides).toBeUndefined(); + expect(shared.redact).toBeUndefined(); + }); + + test('redact strips notes and paths but keeps the repo listed', () => { + const { entry: shared, redactions } = redactForAudience( + entry({ visibility: 'public', redact: ['notes', 'paths'] }), + 'core-team' + ); + expect(redactions).toEqual(['notes', 'paths']); + expect(shared.id).toBe('sample'); + expect(shared.highlights[0].quality).toBe(5); + expect(shared.highlights[0].notes).toBe(''); + expect(shared.highlights[0].paths).toEqual([]); + }); + + test('groupOverrides redact for one audience without affecting another', () => { + const source = entry({ + visibility: 'team', + groups: ['core-team', 'contractors'], + groupOverrides: { contractors: { redact: ['notes', 'paths'] } } + }); + + const core = redactForAudience(source, 'core-team').entry; + const contractors = redactForAudience(source, 'contractors').entry; + + expect(core.highlights[0].notes).toBe('good harness'); + expect(contractors.highlights[0].notes).toBe(''); + }); + + test('compileBundle reports what it shared, withheld and redacted', () => { + const result = compileBundle([ + entry({ id: 'open', visibility: 'public' }), + entry({ id: 'shared', visibility: 'team', groups: ['core-team'], redact: ['paths'] }), + entry({ id: 'secret', visibility: 'private' }), + entry({ id: 'other-team', visibility: 'team', groups: ['contractors'] }) + ], { audience: 'core-team' }); + + expect(result.counts).toEqual({ total: 4, included: 2, excluded: 2, redacted: 1 }); + expect(result.bundle.entries.map((e) => e.id)).toEqual(['open', 'shared']); + expect(result.bundle.audience).toBe('core-team'); + expect(JSON.stringify(result.bundle)).not.toContain('/home/someone'); + }); + + test('compileBundle refuses to run without an audience', () => { + expect(() => compileBundle([], {})).toThrow(/audience/); + }); +}); diff --git a/tests/unit/repoAtlasQuery.test.js b/tests/unit/repoAtlasQuery.test.js new file mode 100644 index 00000000..5805c4e4 --- /dev/null +++ b/tests/unit/repoAtlasQuery.test.js @@ -0,0 +1,116 @@ +const { filterEntries, findByTopic, topicIndex, buildDigest } = require('../../server/atlas/atlasQuery'); +const { normalizeEntry } = require('../../server/atlas/atlasSchema'); + +const DAY_MS = 86_400_000; +const daysAgo = (days) => new Date(Date.now() - days * DAY_MS).toISOString(); + +const entries = [ + normalizeEntry({ + id: 'box2d-luau', + kind: 'library', + platforms: ['roblox'], + languages: ['Luau'], + cloned: true, + localPath: '/repos/box2d-luau', + lastActivity: daysAgo(2), + highlights: [ + { topic: 'physics', quality: 5, paths: ['src/'], notes: 'faithful port' }, + { topic: 'tests', quality: 5, notes: 'best harness we have' } + ] + }, { strict: true }), + normalizeEntry({ + id: 'drain-the-lake', + kind: 'game', + platforms: ['roblox'], + cloned: true, + lastActivity: daysAgo(500), + highlights: [{ topic: 'testing', quality: 3, notes: 'rough but useful' }], + avoid: [{ topic: 'architecture', reason: 'prototype spaghetti' }] + }, { strict: true }), + normalizeEntry({ + id: 'epic-survivors', + kind: 'game', + platforms: ['monogame'], + languages: ['C#'], + isFork: false, + archived: true, + status: 'archived', + lastActivity: daysAgo(700), + highlights: [{ topic: 'save-system', quality: 4 }] + }, { strict: true }), + normalizeEntry({ + id: 'some-fork', + kind: 'reference', + isFork: true, + cloned: false + }, { strict: true }) +]; + +describe('atlasQuery', () => { + test('an absent quality filter does not hide uncurated repos', () => { + expect(filterEntries(entries, {}).length).toBe(4); + expect(filterEntries(entries, { minQuality: null }).length).toBe(4); + expect(filterEntries(entries, { minQuality: '' }).length).toBe(4); + }); + + test('minQuality filters on the best highlight a repo has', () => { + const ids = filterEntries(entries, { minQuality: 5 }).map((e) => e.id); + expect(ids).toEqual(['box2d-luau']); + }); + + test('filters compose across kind, platform and fork state', () => { + expect(filterEntries(entries, { platform: 'roblox' }).map((e) => e.id)) + .toEqual(['box2d-luau', 'drain-the-lake']); + expect(filterEntries(entries, { includeForks: false }).map((e) => e.id)).not.toContain('some-fork'); + expect(filterEntries(entries, { includeArchived: false }).map((e) => e.id)).not.toContain('epic-survivors'); + }); + + test('text search reaches highlight notes, not just names', () => { + expect(filterEntries(entries, { query: 'best harness' }).map((e) => e.id)).toEqual(['box2d-luau']); + }); + + test('findByTopic ranks by quality and resolves topic aliases', () => { + const hits = findByTopic(entries, 'unit-tests'); + expect(hits.map((h) => h.id)).toEqual(['box2d-luau', 'drain-the-lake']); + expect(hits[0].quality).toBe(5); + }); + + test('findByTopic marks long-untouched repos as stale', () => { + const hits = findByTopic(entries, 'testing'); + expect(hits.find((h) => h.id === 'drain-the-lake').stale).toBe(true); + expect(hits.find((h) => h.id === 'box2d-luau').stale).toBe(false); + }); + + test('findByTopic honours a quality floor', () => { + expect(findByTopic(entries, 'testing', { minQuality: 4 }).map((h) => h.id)).toEqual(['box2d-luau']); + }); + + test('an avoid entry hides that repo for that topic only', () => { + expect(findByTopic(entries, 'architecture')).toEqual([]); + expect(findByTopic(entries, 'testing').map((h) => h.id)).toContain('drain-the-lake'); + }); + + test('topicIndex summarizes who has what', () => { + const index = topicIndex(entries); + const testing = index.find((row) => row.topic === 'testing'); + expect(testing.count).toBe(2); + expect(testing.repos[0]).toBe('box2d-luau'); + }); + + test('digest groups by platform and flags stale repos', () => { + const digest = buildDigest(entries, { groupBy: 'platform' }); + expect(digest).toMatch(/roblox/); + expect(digest).toMatch(/box2d-luau\(physics:5, testing:5\)/); + expect(digest).toMatch(/drain-the-lake\(testing:3 ⚠old\)/); + expect(digest).not.toMatch(/some-fork/); + }); + + test('digest truncates long buckets rather than growing without limit', () => { + const many = Array.from({ length: 12 }, (_, i) => normalizeEntry({ + id: `repo-${i}`, + platforms: ['roblox'], + highlights: [{ topic: 'ui', quality: 3 }] + }, { strict: true })); + expect(buildDigest(many, { maxPerBucket: 4 })).toMatch(/\+8 more/); + }); +}); diff --git a/tests/unit/repoAtlasSchema.test.js b/tests/unit/repoAtlasSchema.test.js new file mode 100644 index 00000000..443678f1 --- /dev/null +++ b/tests/unit/repoAtlasSchema.test.js @@ -0,0 +1,97 @@ +const { + normalizeEntry, + mergeEntries, + normalizeTopic, + validateEntry, + kebab +} = require('../../server/atlas/atlasSchema'); + +describe('atlasSchema', () => { + test('normalizeTopic folds aliases onto the canonical vocabulary', () => { + expect(normalizeTopic('multiplayer')).toBe('networking'); + expect(normalizeTopic('Net')).toBe('networking'); + expect(normalizeTopic('save-system')).toBe('data-persistence'); + expect(normalizeTopic('tests')).toBe('testing'); + }); + + test('normalizeTopic keeps unknown topics rather than dropping them', () => { + expect(normalizeTopic('Weather Simulation')).toBe('weather-simulation'); + }); + + test('kebab strips punctuation and collapses separators', () => { + expect(kebab(' My Cool_Repo / v2 ')).toBe('my-cool-repo-v2'); + }); + + test('normalizeEntry keeps only supplied keys unless strict', () => { + const partial = normalizeEntry({ id: 'zoo-game', quality: 9 }); + expect(partial).toEqual({ id: 'zoo-game', quality: 5 }); + + const strict = normalizeEntry({ id: 'zoo-game' }, { strict: true }); + expect(strict.visibility).toBe('private'); + expect(strict.highlights).toEqual([]); + }); + + test('normalizeEntry clamps quality and normalizes highlight topics', () => { + const entry = normalizeEntry({ + id: 'x', + highlights: [{ topic: 'Netcode', quality: 0, paths: 'src/net, src/rpc' }] + }); + expect(entry.highlights).toEqual([ + { topic: 'networking', quality: 1, paths: ['src/net', 'src/rpc'], notes: '' } + ]); + }); + + test('normalizeEntry only accepts redactions for known fields', () => { + const entry = normalizeEntry({ id: 'x', redact: ['notes', 'secrets', 'paths'] }); + expect(entry.redact).toEqual(['notes', 'paths']); + }); + + test('mergeEntries lets later layers win per field without wiping earlier ones', () => { + const merged = mergeEntries( + { __source: 'discovery', id: 'zoo-game', name: 'zoo-game', kind: 'game', languages: ['TypeScript'] }, + { __source: 'manifest', summary: 'Multiplayer zoo tycoon', highlights: [{ topic: 'networking', quality: 3 }] }, + { __source: 'registry', visibility: 'team', groups: ['core-team'] } + ); + + expect(merged.kind).toBe('game'); + expect(merged.languages).toEqual(['TypeScript']); + expect(merged.summary).toBe('Multiplayer zoo tycoon'); + expect(merged.visibility).toBe('team'); + expect(merged.groups).toEqual(['core-team']); + expect(merged.sources).toEqual(['discovery', 'manifest', 'registry']); + }); + + test('mergeEntries does not let an empty later layer erase real data', () => { + const merged = mergeEntries( + { __source: 'discovery', id: 'x', summary: 'Real summary', languages: ['Luau'] }, + { __source: 'registry', summary: '', languages: [] } + ); + expect(merged.summary).toBe('Real summary'); + expect(merged.languages).toEqual(['Luau']); + }); + + test('highlightsAdd appends to the inherited list instead of replacing it', () => { + const merged = mergeEntries( + { __source: 'manifest', id: 'x', highlights: [{ topic: 'testing', quality: 4 }] }, + { __source: 'registry', highlightsAdd: [{ topic: 'physics', quality: 5 }] } + ); + expect(merged.highlights.map((h) => h.topic).sort()).toEqual(['physics', 'testing']); + }); + + test('a later highlight for the same topic overrides the earlier score', () => { + const merged = mergeEntries( + { __source: 'manifest', id: 'x', highlights: [{ topic: 'testing', quality: 4 }] }, + { __source: 'registry', highlightsAdd: [{ topic: 'tests', quality: 1, notes: 'actually rotted' }] } + ); + expect(merged.highlights).toEqual([ + { topic: 'testing', quality: 1, paths: [], notes: 'actually rotted' } + ]); + }); + + test('validateEntry flags team visibility with no groups as a dead-end share', () => { + const entry = normalizeEntry({ id: 'x', visibility: 'team', summary: 's', highlights: [{ topic: 'ui', quality: 3 }] }, { strict: true }); + const report = validateEntry(entry); + expect(report.ok).toBe(true); + expect(report.warnings.join(' ')).toMatch(/lands in no bundle/); + }); +}); diff --git a/tests/unit/repoAtlasService.test.js b/tests/unit/repoAtlasService.test.js new file mode 100644 index 00000000..8eef96d1 --- /dev/null +++ b/tests/unit/repoAtlasService.test.js @@ -0,0 +1,148 @@ +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const RepoAtlasService = require('../../server/repoAtlasService'); +const store = require('../../server/atlas/atlasStore'); + +describe('RepoAtlasService', () => { + let tmpDir; + let repoDir; + let atlas; + + beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-test-')); + repoDir = path.join(tmpDir, 'repos', 'zoo-game'); + fs.mkdirSync(repoDir, { recursive: true }); + process.env.AGENT_WORKSPACE_ATLAS_DIR = path.join(tmpDir, 'atlas'); + + atlas = new RepoAtlasService(); + store.saveDiscoveryCache([{ + __source: 'discovery', + id: 'zoo-game', + name: 'zoo-game', + repo: 'owner/zoo-game', + kind: 'game', + languages: ['TypeScript'], + localPath: repoDir, + cloned: true, + lastActivity: new Date().toISOString() + }]); + }); + + afterEach(() => { + delete process.env.AGENT_WORKSPACE_ATLAS_DIR; + fs.rmSync(tmpDir, { recursive: true, force: true }); + }); + + test('discovery alone produces a usable entry', () => { + const entry = atlas.getEntry('zoo-game'); + expect(entry.kind).toBe('game'); + expect(entry.visibility).toBe('private'); + expect(entry.sources).toEqual(['discovery']); + }); + + test('an in-repo manifest layers over discovery', () => { + fs.writeFileSync(path.join(repoDir, '.repo-atlas.json'), JSON.stringify({ + id: 'zoo-game', + summary: 'Multiplayer zoo tycoon', + highlights: [{ topic: 'data-compression', quality: 5, notes: 'bitpacked saves' }] + })); + atlas.invalidate(); + + const entry = atlas.getEntry('zoo-game'); + expect(entry.summary).toBe('Multiplayer zoo tycoon'); + expect(entry.kind).toBe('game'); + expect(entry.highlights[0].topic).toBe('data-compression'); + expect(entry.sources).toEqual(['discovery', 'manifest']); + }); + + test('the registry overrides the manifest — your opinion wins', () => { + fs.writeFileSync(path.join(repoDir, '.repo-atlas.json'), JSON.stringify({ + id: 'zoo-game', + summary: 'From the repo', + maturity: 'production' + })); + atlas.setEntry('zoo-game', { summary: 'From you', maturity: 'prototype' }); + + const entry = atlas.getEntry('zoo-game'); + expect(entry.summary).toBe('From you'); + expect(entry.maturity).toBe('prototype'); + expect(entry.sources).toEqual(['discovery', 'manifest', 'registry']); + }); + + test('addHighlight persists and replaces the same topic', () => { + atlas.addHighlight('zoo-game', { topic: 'multiplayer', quality: 3, notes: 'chatty' }); + atlas.addHighlight('zoo-game', { topic: 'networking', quality: 5, paths: ['src/net'], notes: 'rewritten' }); + + const entry = new RepoAtlasService().getEntry('zoo-game'); + expect(entry.highlights).toEqual([ + { topic: 'networking', quality: 5, paths: ['src/net'], notes: 'rewritten' } + ]); + }); + + test('addAvoid records a do-not-copy note without removing the repo', () => { + atlas.addAvoid('zoo-game', { topic: 'ui', reason: 'hand-rolled' }); + const entry = atlas.getEntry('zoo-game'); + expect(entry.avoid).toEqual([{ topic: 'ui', reason: 'hand-rolled' }]); + expect(atlas.find('ui')).toEqual([]); + }); + + test('entries can be recorded for repos that were never cloned', () => { + atlas.setEntry('never-cloned', { + name: 'never-cloned', + summary: 'lives only on GitHub', + remoteUrl: 'https://github.com/owner/never-cloned' + }); + atlas.addHighlight('never-cloned', { topic: 'auth', quality: 4 }); + + const hits = atlas.find('auth'); + expect(hits).toHaveLength(1); + expect(hits[0].cloned).toBe(false); + expect(hits[0].remoteUrl).toBe('https://github.com/owner/never-cloned'); + }); + + test('compile writes an audience bundle and withholds private entries', () => { + atlas.setAudience({ id: 'core-team', label: 'Core team' }); + atlas.setEntry('zoo-game', { visibility: 'team', groups: ['core-team'] }); + atlas.setEntry('secret-thing', { name: 'secret', visibility: 'private' }); + + const result = atlas.compile('core-team'); + expect(result.counts.included).toBe(1); + expect(result.bundle.entries[0].id).toBe('zoo-game'); + + const written = JSON.parse(fs.readFileSync(result.written[0], 'utf8')); + expect(written.entries.map((e) => e.id)).toEqual(['zoo-game']); + expect(JSON.stringify(written)).not.toContain(repoDir); + }); + + test('compile --dry-run writes nothing', () => { + atlas.setEntry('zoo-game', { visibility: 'public' }); + const result = atlas.compile('core-team', { write: false }); + expect(result.written).toEqual([]); + expect(fs.existsSync(store.bundlesDir())).toBe(false); + }); + + test('initManifest seeds a manifest from what is already known', () => { + const { path: manifestPath, entry } = atlas.initManifest(repoDir); + expect(fs.existsSync(manifestPath)).toBe(true); + expect(entry.id).toBe('zoo-game'); + expect(entry.kind).toBe('game'); + expect(entry.visibility).toBe('private'); + }); + + test('refresh with both scanners disabled clears the map instead of hanging', async () => { + const result = await atlas.refresh({ scanLocal: false, scanGitHub: false }); + expect(result.totalCount).toBe(0); + expect(atlas.getEntries()).toEqual([]); + }); + + test('getStatus reports where data lives and how much is curated', () => { + atlas.addHighlight('zoo-game', { topic: 'testing', quality: 4 }); + const status = atlas.getStatus(); + expect(status.entryCount).toBe(1); + expect(status.clonedCount).toBe(1); + expect(status.highlightCount).toBe(1); + expect(status.registryPath).toContain('registry.json'); + }); +}); From 89edc70eea3910df292880bc3811b6fa094b3bac Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 09:52:47 +1000 Subject: [PATCH 03/69] feat: fleet supervisor + speech output + free-form voice routing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three pieces of the same thing: an assistant that notices first, and that you can talk to. SUPERVISOR Nothing in the orchestrator was push-based. pagerService nudges on a fixed interval whether or not anything is wrong; schedulerService runs on a clock; processAdvisorService computes good advice but only when a human opens the panel. You had to notice a stuck agent yourself. The supervisor closes that loop without spending tokens to do it. Sensors and rules run every tick — PTY tail, session status, how long a buffer has been quiet, git ahead/dirty for quiet sessions only — and a model is never called in the loop, only on escalation. That is what makes it affordable to leave running permanently. Findings climb observe -> notify -> nudge -> act, capped by an autonomy level: - observe is the shipped default and has zero side effects, so the rules can be judged from a week of findings before being trusted with anything - no shipped condition reaches 'act' (asserted in tests) - act handlers are named functions, so a rule file cannot inject shell - auto-answering a permission prompt fails closed: any deny-pattern match, or no allow-pattern match at all, escalates to a human instead of guessing - per-finding cooldowns, and every action lands in supervisor-audit.jsonl Detects: permission prompts left hanging, usage limits, error loops, stalls, exited agents, unpushed and uncommitted work, idle capacity. SPEECH OUT Backends degrade: browser speech synthesis by default (nothing to install, so it works on a fresh clone), with piper/say/SAPI/espeak preferred when present. Text is sanitized to printable ASCII with no shell metacharacters before it can reach a command line, length-capped, and de-duplicated. FREE-FORM VOICE Previously an utterance matching no pattern was a dead end, which is what makes a voice interface feel like a remote control. Unmatched speech is now handed to the Commander agent, so the fallback for 'I didn't understand' is a full agent with the whole orchestrator API rather than an error beep. 709 unit tests green. --- CODEBASE_DOCUMENTATION.md | 37 +++ client/app.js | 4 + client/index.html | 1 + client/speech-output.js | 73 ++++++ config/supervisor-rules.json | 143 +++++++++++ docs/COMMANDER_CLAUDE.md | 71 ++++++ server/index.js | 52 ++++ server/routes/speechRoutes.js | 60 +++++ server/routes/supervisorRoutes.js | 64 +++++ server/speechService.js | 226 +++++++++++++++++ server/supervisor/supervisorActions.js | 189 ++++++++++++++ server/supervisor/supervisorRules.js | 204 +++++++++++++++ server/supervisor/supervisorSignals.js | 208 ++++++++++++++++ server/supervisorService.js | 329 +++++++++++++++++++++++++ server/voiceCommandService.js | 57 ++++- tests/unit/speechService.test.js | 94 +++++++ tests/unit/supervisorActions.test.js | 177 +++++++++++++ tests/unit/supervisorRules.test.js | 113 +++++++++ tests/unit/supervisorService.test.js | 205 +++++++++++++++ tests/unit/voiceCommandService.test.js | 55 +++++ 20 files changed, 2359 insertions(+), 3 deletions(-) create mode 100644 client/speech-output.js create mode 100644 config/supervisor-rules.json create mode 100644 server/routes/speechRoutes.js create mode 100644 server/routes/supervisorRoutes.js create mode 100644 server/speechService.js create mode 100644 server/supervisor/supervisorActions.js create mode 100644 server/supervisor/supervisorRules.js create mode 100644 server/supervisor/supervisorSignals.js create mode 100644 server/supervisorService.js create mode 100644 tests/unit/speechService.test.js create mode 100644 tests/unit/supervisorActions.test.js create mode 100644 tests/unit/supervisorRules.test.js create mode 100644 tests/unit/supervisorService.test.js diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index e585f7e9..ec9af242 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -130,6 +130,28 @@ server/repoAtlasService.js - Repo Atlas facade: one queryable map of eve ├─ Query: `find(topic)` ranked by per-topic quality 1-5, `digest()` compact paste-into-a-prompt map, `search()`, `topics()` ├─ Curation: `addHighlight()` / `addAvoid()` persist into the registry — quality is scored per topic, so a rough repo can still be the best example of one thing └─ Sharing: `compile(audience)` emits audience-scoped bundles — `private` never leaves the machine, `team` needs a group match, `public` goes everywhere +server/supervisorService.js - Fleet supervisor: rule-driven watchdog over every agent session +├─ Loop: rules run on a tick (default 30s) from zero-token signals; no model is called in the loop, only on escalation +├─ Ladder: observe → notify → nudge → act, capped by autonomy (`off` | `observe` (default) | `assist` | `autopilot`) +├─ Safety: shipped conditions never reach `act`; act handlers are named functions, so rules cannot inject shell +├─ Audit: every action appended to `~/.agent-workspace/logs/supervisor-audit.jsonl` with the finding that caused it +└─ `getBriefing()` renders the spoken/at-a-glance "what needs you now" summary +server/supervisor/supervisorSignals.js - Per-session signal collection (PTY tail, quiet-time tracker, repeated-line detection, git ahead/dirty for quiet sessions only) +server/supervisor/supervisorRules.js - Condition table loader/matcher (`config/supervisor-rules.json`, override `~/.agent-workspace/supervisor-rules.json`) + autonomy ceilings +server/supervisor/supervisorActions.js - Ladder executor: notify/nudge/act, two-write submit, fail-closed permission-prompt classification +server/routes/supervisorRoutes.js - Express router for `/api/supervisor/*` +config/supervisor-rules.json - Shipped condition table (autonomy `observe`, permission allow/deny patterns, act-handler allowlist) +tests/unit/supervisorRules.test.js, supervisorActions.test.js, supervisorService.test.js - Supervisor coverage (matching, autonomy ceilings, cooldowns, fail-closed approvals, audit) + +server/speechService.js - Speech output with degrading backends +├─ Default `browser` backend emits a `speech-speak` socket event — works on a fresh clone with nothing installed +├─ Local backends preferred when present: piper (piped straight to paplay/aplay), macOS `say`, Windows SAPI, espeak-ng +└─ Sanitizes to printable ASCII with no shell metacharacters, caps length, suppresses back-to-back repeats +server/routes/speechRoutes.js - `/api/speech/*` (say, backend, enabled, spoken fleet briefing) +client/speech-output.js - Web Speech API listener for the browser backend (`window.SpeechOutput`) +server/voiceCommandService.js - (existing) rule/LLM voice parsing, now with `setCommanderForwarder()`: unmatched speech is handed to the Commander agent instead of dead-ending +tests/unit/speechService.test.js - Sanitization, repeat suppression, backend resolution + server/atlas/atlasSchema.js - Entry normalization, layered merge, topic-alias folding (`config/repo-atlas-topics.json`), validation server/atlas/atlasDiscovery.js - Local git scan (worktree siblings collapse into one project entry) + `gh repo list` + source merge server/atlas/atlasStore.js - Persistence under `~/.agent-workspace/atlas/` (registry, discovery cache, compiled bundles) + in-repo manifest read/write @@ -483,6 +505,7 @@ git-change: {branch, status, commits} - Git repository changes notification: {type, message, level} - System notifications workspace-changed: {workspaceId, sessions} - Workspace switch completed workspace-list: {workspaces} - Available workspaces update +speech-speak: {text, priority, at} - Say this out loud (browser speech backend) ``` ### Client → Server Events @@ -680,6 +703,20 @@ POST /api/atlas/entries/:id/highlights - Record "this rep POST /api/atlas/entries/:id/avoid - Record "do not copy X from this repo" GET|POST /api/atlas/audiences - List/define sharing audiences POST /api/atlas/compile - Compile an audience bundle (`dryRun: true` returns decisions without writing) + +GET /api/supervisor/status - Loop state: running, autonomy, tick rate, armed conditions +GET /api/supervisor/findings?severity=&sessionId=&limit= - Recent findings +GET /api/supervisor/briefing - "What needs you now", with a spoken rendering +POST /api/supervisor/tick - Force one pass (`dryRun: true` evaluates without acting) +POST /api/supervisor/start | /stop - Control the loop +POST /api/supervisor/autonomy - Set autonomy (`off` | `observe` | `assist` | `autopilot`) +POST /api/supervisor/reload-rules - Re-read the condition table from disk + +GET /api/speech/status - Enabled state, resolved backend, available backends, listeners +POST /api/speech/say - Speak text (`priority: high` interrupts, `force` skips repeat suppression) +POST /api/speech/backend - Choose a backend +POST /api/speech/enabled - Mute/unmute +POST /api/speech/briefing - Speak the supervisor briefing ``` ### WebSocket Events diff --git a/client/app.js b/client/app.js index 2496c39c..6a6b93ec 100644 --- a/client/app.js +++ b/client/app.js @@ -1377,6 +1377,10 @@ class ClaudeOrchestrator { this.socket = io(serverUrl, socketOptions); console.log(`Socket connecting to ${serverUrl}...`); + // Let listeners that load independently of app.js (speech output) bind. + window.socket = this.socket; + document.dispatchEvent(new CustomEvent('orchestrator-socket-ready', { detail: { socket: this.socket } })); + // Connection events this.socket.on('connect', () => { console.log('Connected to server'); diff --git a/client/index.html b/client/index.html index f7ee998e..572417ae 100644 --- a/client/index.html +++ b/client/index.html @@ -548,6 +548,7 @@

Notifications

+ diff --git a/client/speech-output.js b/client/speech-output.js new file mode 100644 index 00000000..6e536dc1 --- /dev/null +++ b/client/speech-output.js @@ -0,0 +1,73 @@ +/** + * Browser speech output. + * + * The zero-install half of the voice layer: the server emits `speech-speak` and + * the page says it with the Web Speech API. This is why speech works on a fresh + * clone with nothing set up — local backends (piper/say/SAPI) take over on the + * server side when they exist, and this simply stops receiving events. + */ +(function initSpeechOutput() { + const synth = window.speechSynthesis; + + const state = { + enabled: localStorage.getItem('speechOutputEnabled') !== 'false', + voiceName: localStorage.getItem('speechOutputVoice') || '', + rate: Number(localStorage.getItem('speechOutputRate')) || 1.05 + }; + + function pickVoice() { + if (!synth) return null; + const voices = synth.getVoices(); + if (!voices.length) return null; + if (state.voiceName) { + const chosen = voices.find((voice) => voice.name === state.voiceName); + if (chosen) return chosen; + } + return voices.find((voice) => /en[-_]/i.test(voice.lang)) || voices[0]; + } + + function speak(text, { priority = 'normal' } = {}) { + if (!synth || !state.enabled) return false; + const clean = String(text || '').trim(); + if (!clean) return false; + + // A critical announcement should not queue behind a status readout. + if (priority === 'high' && synth.speaking) synth.cancel(); + + const utterance = new SpeechSynthesisUtterance(clean); + const voice = pickVoice(); + if (voice) utterance.voice = voice; + utterance.rate = state.rate; + synth.speak(utterance); + return true; + } + + function attach(socket) { + if (!socket || socket.__speechOutputAttached) return; + socket.__speechOutputAttached = true; + socket.on('speech-speak', (payload) => speak(payload?.text, { priority: payload?.priority })); + } + + window.SpeechOutput = { + speak, + attach, + isSupported: Boolean(synth), + isEnabled: () => state.enabled, + setEnabled(enabled) { + state.enabled = enabled !== false; + localStorage.setItem('speechOutputEnabled', String(state.enabled)); + if (!state.enabled && synth?.speaking) synth.cancel(); + return state.enabled; + }, + setVoice(name) { + state.voiceName = String(name || ''); + localStorage.setItem('speechOutputVoice', state.voiceName); + return state.voiceName; + }, + listVoices: () => (synth ? synth.getVoices().map((voice) => ({ name: voice.name, lang: voice.lang })) : []) + }; + + // The socket may connect before or after this file runs, so try both. + if (window.socket) attach(window.socket); + document.addEventListener('orchestrator-socket-ready', (event) => attach(event.detail?.socket || window.socket)); +})(); diff --git a/config/supervisor-rules.json b/config/supervisor-rules.json new file mode 100644 index 00000000..6fda6f86 --- /dev/null +++ b/config/supervisor-rules.json @@ -0,0 +1,143 @@ +{ + "$comment": "Condition table for the fleet supervisor. Evaluated every tick against zero-token signals (PTY tail, status, quiet time, git state). Override per-machine with ~/.agent-workspace/supervisor-rules.json.", + "schemaVersion": 1, + + "$comment_autonomy": "off = nothing runs. observe = findings recorded, no side effects (start here). assist = may notify and nudge. autopilot = may also run built-in act handlers.", + "autonomy": "observe", + "tickSeconds": 30, + "maxFindingsRetained": 500, + + "safety": { + "$comment": "Act handlers are named functions in server/supervisor/supervisorActions.js. Rules cannot inject shell commands — they can only select a handler from this list.", + "allowedActHandlers": ["answer-permission", "relaunch-agent", "open-pull-request"], + "$comment_permission": "An auto-answered permission prompt must match one of these AND none of the deny patterns. Anything else escalates to you.", + "permissionAllowPatterns": [ + "\\b(Read|Glob|Grep|NotebookRead|WebFetch|WebSearch)\\b", + "\\bgit (status|diff|log|show|branch|fetch)\\b", + "\\b(npm|pnpm|yarn) (test|run test|run lint|run typecheck)\\b" + ], + "permissionDenyPatterns": [ + "\\brm\\b", "\\bsudo\\b", "\\bgit (push|reset|clean|checkout)\\b", + "\\bgh (pr merge|release|repo delete)\\b", "\\bDELETE\\b", "\\bDROP\\b", + "\\bcurl\\b.*\\|", "\\bchmod\\b", "\\bkill\\b", "> */" + ] + }, + + "conditions": [ + { + "id": "awaiting-permission", + "label": "Waiting on a permission prompt", + "severity": "warn", + "rung": "notify", + "cooldownSeconds": 600, + "when": { + "status": ["waiting"], + "minQuietSeconds": 120, + "tailMatches": [ + "Do you want to (proceed|make this edit|create)", + "❯\\s*1\\.\\s*Yes", + "Allow .* to run", + "\\[y/N\\]" + ] + }, + "advice": "The agent has been sitting on a permission prompt. Answer it or let autopilot approve read-only prompts." + }, + { + "id": "usage-limit-reached", + "label": "Usage limit reached", + "severity": "critical", + "rung": "notify", + "cooldownSeconds": 3600, + "when": { + "tailMatches": [ + "\\d+-hour limit reached", + "limit reached ∙ resets", + "You've reached your usage limit", + "rate.?limit(ed)? .*(retry|reset)" + ] + }, + "advice": "This session is blocked until the window resets. Nothing to do but wait — resume it after the reset time in the banner." + }, + { + "id": "error-loop", + "label": "Repeating the same error", + "severity": "critical", + "rung": "notify", + "cooldownSeconds": 1800, + "when": { + "status": ["busy", "idle"], + "repeatedTailLine": 4, + "tailMatches": ["(?i)\\b(error|failed|exception|traceback|cannot find)\\b"] + }, + "advice": "The same error line keeps coming back — the agent is looping. This one needs a human, never automation." + }, + { + "id": "stalled", + "label": "Busy but silent", + "severity": "warn", + "rung": "nudge", + "cooldownSeconds": 900, + "nudgeText": "status? if you are blocked, say what on and stop.", + "when": { + "status": ["busy"], + "minQuietSeconds": 900, + "tailNotMatches": ["limit reached", "Do you want to (proceed|make this edit)"] + }, + "advice": "Marked busy but nothing has come out for 15 minutes." + }, + { + "id": "agent-exited", + "label": "Agent exited, shell left behind", + "severity": "warn", + "rung": "notify", + "cooldownSeconds": 900, + "when": { + "status": ["idle"], + "agentPresent": false, + "minQuietSeconds": 300 + }, + "advice": "The agent CLI is gone and the terminal dropped back to a shell. Relaunch it or reuse the worktree." + }, + { + "id": "unpushed-work", + "label": "Finished with unpushed commits", + "severity": "info", + "rung": "nudge", + "cooldownSeconds": 1800, + "nudgeText": "You have local commits that are not pushed. Push the branch and open a PR.", + "when": { + "status": ["idle"], + "minQuietSeconds": 300, + "git": { "aheadMin": 1 } + }, + "advice": "Work is done on disk but has not left the machine." + }, + { + "id": "uncommitted-work", + "label": "Idle with uncommitted changes", + "severity": "info", + "rung": "nudge", + "cooldownSeconds": 3600, + "nudgeText": "You have uncommitted changes. Commit them with a descriptive message, or say why they should not be committed.", + "when": { + "status": ["idle"], + "minQuietSeconds": 900, + "git": { "dirty": true, "aheadMax": 0 } + }, + "advice": "Edits are sitting in the working tree with nothing recorded." + }, + { + "id": "idle-capacity", + "label": "Idle worktree, nothing in flight", + "severity": "info", + "rung": "observe", + "cooldownSeconds": 3600, + "when": { + "status": ["idle"], + "minQuietSeconds": 1800, + "git": { "dirty": false, "aheadMax": 0 } + }, + "advice": "Free capacity — this worktree could take the next queue item." + } + ] +} diff --git a/docs/COMMANDER_CLAUDE.md b/docs/COMMANDER_CLAUDE.md index 8baf466c..2fe045b3 100644 --- a/docs/COMMANDER_CLAUDE.md +++ b/docs/COMMANDER_CLAUDE.md @@ -118,6 +118,77 @@ curl -sS "$BASE_URL/api/commander/execute" \ --- +## Supervisor (the fleet watchdog) + +A rule-driven loop classifies every agent session every 30s from zero-token signals (PTY tail, status, quiet time, git state) and climbs an escalation ladder capped by an autonomy level. Ask it what needs attention instead of reading 16 terminals yourself. + +```bash +# What needs a human right now — start here +curl -sS "$BASE_URL/api/supervisor/briefing" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Everything recorded recently (filter by severity or session) +curl -sS "$BASE_URL/api/supervisor/findings?severity=critical" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Loop config: autonomy level, tick rate, which conditions are armed +curl -sS "$BASE_URL/api/supervisor/status" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Force a pass now (dryRun reports findings without acting on them) +curl -sS -X POST "$BASE_URL/api/supervisor/tick" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"dryRun": true}' | jq +``` + +**Autonomy levels** — `off` (nothing runs) | `observe` (default: record only, zero side effects) | `assist` (may notify and type nudges into sessions) | `autopilot` (may also run allowlisted act handlers). + +```bash +curl -sS -X POST "$BASE_URL/api/supervisor/autonomy" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"level": "assist"}' +``` + +**Never raise the autonomy level on your own.** That is the user's decision, and `observe` is deliberately the shipped default so the rules can be judged before they are trusted. Rules live in `config/supervisor-rules.json`, overridable at `~/.agent-workspace/supervisor-rules.json`; every action is appended to `~/.agent-workspace/logs/supervisor-audit.jsonl`. + +## Speech + +```bash +# Say something out loud +curl -sS -X POST "$BASE_URL/api/speech/say" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"text": "Work three is waiting on permission."}' + +# Speak the fleet briefing +curl -sS -X POST "$BASE_URL/api/speech/briefing" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" -d '{}' | jq + +curl -sS "$BASE_URL/api/speech/status" -H "X-Auth-Token: $AUTH_TOKEN" | jq +``` + +Default backend is the browser's own synthesis (nothing to install); piper/`say`/SAPI/espeak take over when present. Keep spoken text to one or two short sentences — it is read aloud, not displayed. + +## Repo Atlas (cross-repo prior art) + +The map of every repo the user owns, cloned or not, with per-topic quality scores. Query it before searching the filesystem for "how did we do X before". + +```bash +# The main query: who did this well? +curl -sS "$BASE_URL/api/atlas/find?topic=data-compression" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Compact map worth pasting into a prompt +curl -sS "$BASE_URL/api/atlas/digest" -H "X-Auth-Token: $AUTH_TOKEN" | jq -r .digest + +curl -sS "$BASE_URL/api/atlas/entries/zoo-game" -H "X-Auth-Token: $AUTH_TOKEN" | jq -r .description +curl -sS "$BASE_URL/api/atlas/topics" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Record what a repo turned out to be good at +curl -sS -X POST "$BASE_URL/api/atlas/entries/zoo-game/highlights" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"topic": "data-compression", "quality": 5, "paths": ["src/data/"], "notes": "bitpacked saves"}' +``` + +Also available as a CLI anywhere: `node scripts/atlas.js find `. + +**Do not change a repo's `visibility` or `groups`, and do not compile sharing bundles, without being asked.** Those decide what leaves the machine. + ## Session Control ```bash diff --git a/server/index.js b/server/index.js index 2916f4fa..a4ec61ac 100644 --- a/server/index.js +++ b/server/index.js @@ -112,6 +112,10 @@ const { QuickLinksService } = require('./quickLinksService'); const { RecommendationsService } = require('./recommendationsService'); const { RepoAtlasService } = require('./repoAtlasService'); const { createAtlasRoutes } = require('./routes/atlasRoutes'); +const { SupervisorService } = require('./supervisorService'); +const { createSupervisorRoutes } = require('./routes/supervisorRoutes'); +const { SpeechService } = require('./speechService'); +const { createSpeechRoutes } = require('./routes/speechRoutes'); const { ProductLauncherService } = require('./productLauncherService'); const { CommanderService } = require('./commanderService'); const { ConversationService } = require('./conversationService'); @@ -352,6 +356,9 @@ const continuityService = ContinuityService.getInstance(); const quickLinksService = QuickLinksService.getInstance(); const recommendationsService = RecommendationsService.getInstance(); const repoAtlasService = RepoAtlasService.getInstance({ logger }); +const speechService = SpeechService.getInstance({ logger }); +speechService.setIO(io); +const supervisorService = SupervisorService.getInstance({ logger }); const activityFeed = ActivityFeedService.getInstance(); activityFeed.setIO(io); activityFeed.track('server.started', { port: Number(process.env.ORCHESTRATOR_PORT || 9460) }); @@ -426,6 +433,36 @@ threadService.init({ workspaceManager, sessionManager }); intentHaikuService.setSessionManager(sessionManager); serviceStackRuntimeService.init({ workspaceManager, sessionManager, configPromoterService, io }); auditExportService.init({ activityFeed, schedulerService, userSettingsService }); +supervisorService.init({ + sessionManager, + gitHelper, + agentManager, + sessionRecoveryService, + taskRecordService, + activityFeed, + notificationService, + speechService +}); + +// Shipped default autonomy is `observe`: findings accumulate, nothing is +// touched. That is safe to leave running, and reading a week of it is how you +// decide whether to grant this thing any real autonomy. +if (String(process.env.SUPERVISOR_AUTOSTART || 'true').toLowerCase() !== 'false') { + const started = supervisorService.start(); + logger.info('Supervisor', started); +} + +// Speech that no rule matched is still useful: hand the raw words to the +// active Commander so the fallback is an agent, not an error. +voiceCommandService.setCommanderForwarder(async (transcript) => { + if (!commanderService?.sendInput) return false; + // Two writes: agent CLIs treat "text\r" in one chunk as a bracketed paste. + const wrote = commanderService.sendInput(transcript); + if (wrote === false) return false; + await new Promise((resolve) => setTimeout(resolve, 300)); + commanderService.sendInput('\r'); + return true; +}); const loadPlugins = async () => { const status = await pluginLoaderService.loadAll({ @@ -1346,6 +1383,21 @@ app.use('/api/atlas', createAtlasRoutes({ requireWrite: requirePolicyAction('write') })); +app.use('/api/supervisor', createSupervisorRoutes({ + supervisorService, + logger, + requireRead: requirePolicyAction('read'), + requireWrite: requirePolicyAction('write') +})); + +app.use('/api/speech', createSpeechRoutes({ + speechService, + supervisorService, + logger, + requireRead: requirePolicyAction('read'), + requireWrite: requirePolicyAction('write') +})); + app.get('/api/app-info', (req, res) => { res.json(readAppInfo()); }); diff --git a/server/routes/speechRoutes.js b/server/routes/speechRoutes.js new file mode 100644 index 00000000..226eee99 --- /dev/null +++ b/server/routes/speechRoutes.js @@ -0,0 +1,60 @@ +const express = require('express'); + +const passthrough = (req, res, next) => next(); + +/** + * Speech output + the spoken fleet briefing. + * + * Speaking is a `write` action: it makes the machine do something in the room. + */ +function createSpeechRoutes({ speechService, supervisorService, logger = console, requireRead = passthrough, requireWrite = passthrough } = {}) { + const router = express.Router(); + + const handle = (label, handler) => async (req, res) => { + try { + await handler(req, res); + } catch (error) { + logger.error(`Speech: ${label} failed`, { error: error.message, stack: error.stack }); + res.status(400).json({ ok: false, error: error.message }); + } + }; + + router.get('/status', requireRead, handle('status', (req, res) => { + res.json({ ok: true, status: speechService.getStatus() }); + })); + + router.post('/say', requireWrite, handle('say', (req, res) => { + const text = String(req.body?.text || '').trim(); + if (!text) return res.status(400).json({ ok: false, error: 'text is required' }); + return res.json({ + ok: true, + result: speechService.speak(text, { + priority: req.body?.priority || 'normal', + force: req.body?.force === true + }) + }); + })); + + router.post('/backend', requireWrite, handle('set backend', (req, res) => { + const backend = speechService.setBackend(String(req.body?.backend || '').trim().toLowerCase()); + res.json({ ok: true, backend, status: speechService.getStatus() }); + })); + + router.post('/enabled', requireWrite, handle('toggle', (req, res) => { + res.json({ ok: true, enabled: speechService.setEnabled(req.body?.enabled !== false) }); + })); + + /** + * "What's happening?" — the fleet summary, optionally spoken aloud. + */ + router.post('/briefing', requireWrite, handle('briefing', (req, res) => { + if (!supervisorService) return res.status(503).json({ ok: false, error: 'supervisor is not available' }); + const briefing = supervisorService.getBriefing({ limit: req.body?.limit }); + const spoken = req.body?.speak === false ? null : speechService.speak(briefing.spoken, { priority: 'high', force: true }); + return res.json({ ok: true, briefing, spoken }); + })); + + return router; +} + +module.exports = { createSpeechRoutes }; diff --git a/server/routes/supervisorRoutes.js b/server/routes/supervisorRoutes.js new file mode 100644 index 00000000..918f2386 --- /dev/null +++ b/server/routes/supervisorRoutes.js @@ -0,0 +1,64 @@ +const express = require('express'); + +const passthrough = (req, res, next) => next(); + +/** + * REST surface for the fleet supervisor. Reads are policy-`read`; anything that + * changes how much autonomy the loop has, or makes it act, is policy-`write`. + */ +function createSupervisorRoutes({ supervisorService, logger = console, requireRead = passthrough, requireWrite = passthrough } = {}) { + const router = express.Router(); + + const handle = (label, handler) => async (req, res) => { + try { + await handler(req, res); + } catch (error) { + logger.error(`Supervisor: ${label} failed`, { error: error.message, stack: error.stack }); + res.status(400).json({ ok: false, error: error.message }); + } + }; + + router.get('/status', requireRead, handle('status', (req, res) => { + res.json({ ok: true, status: supervisorService.getStatus() }); + })); + + router.get('/findings', requireRead, handle('findings', (req, res) => { + const findings = supervisorService.getFindings({ + limit: req.query.limit, + severity: req.query.severity, + sessionId: req.query.sessionId + }); + res.json({ ok: true, count: findings.length, findings }); + })); + + router.get('/briefing', requireRead, handle('briefing', (req, res) => { + res.json({ ok: true, briefing: supervisorService.getBriefing({ limit: req.query.limit }) }); + })); + + router.post('/tick', requireWrite, handle('tick', async (req, res) => { + const result = await supervisorService.tick({ dryRun: req.body?.dryRun === true }); + res.json({ ok: !result.error, ...result }); + })); + + router.post('/start', requireWrite, handle('start', (req, res) => { + res.json({ ok: true, ...supervisorService.start() }); + })); + + router.post('/stop', requireWrite, handle('stop', (req, res) => { + res.json({ ok: true, ...supervisorService.stop() }); + })); + + router.post('/autonomy', requireWrite, handle('set autonomy', (req, res) => { + const level = supervisorService.setAutonomy(String(req.body?.level || '').trim().toLowerCase()); + res.json({ ok: true, autonomy: level, status: supervisorService.getStatus() }); + })); + + router.post('/reload-rules', requireWrite, handle('reload rules', (req, res) => { + const rules = supervisorService.reloadRules(); + res.json({ ok: true, source: rules.source, conditionCount: rules.conditions.length, autonomy: rules.autonomy }); + })); + + return router; +} + +module.exports = { createSupervisorRoutes }; diff --git a/server/speechService.js b/server/speechService.js new file mode 100644 index 00000000..e800f1de --- /dev/null +++ b/server/speechService.js @@ -0,0 +1,226 @@ +const os = require('os'); +const { spawn, spawnSync } = require('child_process'); + +const { augmentProcessEnv, getHiddenProcessOptions } = require('./utils/processUtils'); + +const MAX_SPOKEN_CHARS = 400; +const REPEAT_WINDOW_MS = 30_000; +const HISTORY_LIMIT = 50; + +/** + * Anything spoken aloud is short, plain, and free of shell metacharacters. + * Terminal output is full of escape sequences and punctuation that no + * synthesizer should try to read and no command line should ever receive. + */ +function sanitizeForSpeech(text) { + return String(text || '') + .replace(/\x1b\[[0-9;?]*[ -/]*[@-~]/g, ' ') + .replace(/[`$\\|&;<>(){}[\]]/g, ' ') + .replace(/[^\x20-\x7EÀ-ɏ]/g, ' ') + .replace(/\s+/g, ' ') + .trim() + .slice(0, MAX_SPOKEN_CHARS); +} + +function commandExists(command) { + try { + const probe = process.platform === 'win32' ? 'where.exe' : 'which'; + return spawnSync(probe, [command], { stdio: 'ignore', windowsHide: true }).status === 0; + } catch { + return false; + } +} + +/** + * Speech output for the orchestrator. + * + * The default backend is the browser's own speech synthesis, reached by emitting + * a socket event — it needs nothing installed, which is the difference between a + * feature people use and a feature people mean to set up. Local backends take + * over when they exist and are preferred. + */ +class SpeechService { + constructor({ logger = console } = {}) { + this.logger = logger; + this.io = null; + this.enabled = String(process.env.SPEECH_ENABLED || 'true').toLowerCase() !== 'false'; + this.preferredBackend = String(process.env.SPEECH_BACKEND || '').trim().toLowerCase(); + this.piperModel = String(process.env.PIPER_MODEL || '').trim(); + this.voice = String(process.env.SPEECH_VOICE || '').trim(); + this.history = []; + this.lastSpokenAt = new Map(); + this.backendCache = null; + } + + static getInstance(options = {}) { + if (!SpeechService.instance) { + SpeechService.instance = new SpeechService(options); + } + return SpeechService.instance; + } + + setIO(io) { + this.io = io; + return this; + } + + setEnabled(enabled) { + this.enabled = enabled !== false; + return this.enabled; + } + + detectBackends({ force = false } = {}) { + if (this.backendCache && !force) return this.backendCache; + + const backends = [ + { id: 'browser', label: 'Browser speech synthesis', available: true, local: false }, + { id: 'piper', label: 'Piper (local neural TTS)', available: commandExists('piper') && Boolean(this.piperModel), local: true }, + { id: 'say', label: 'macOS say', available: os.platform() === 'darwin' && commandExists('say'), local: true }, + { id: 'sapi', label: 'Windows SAPI', available: os.platform() === 'win32', local: true }, + { id: 'espeak', label: 'espeak-ng', available: commandExists('espeak-ng') || commandExists('espeak'), local: true } + ]; + + this.backendCache = backends; + return backends; + } + + resolveBackend() { + const backends = this.detectBackends(); + if (this.preferredBackend) { + const preferred = backends.find((b) => b.id === this.preferredBackend); + if (preferred?.available) return preferred.id; + } + // Local synthesis is better than round-tripping through a browser tab that + // may not be open, so it wins whenever it is actually installed. + const local = backends.find((b) => b.local && b.available); + return local ? local.id : 'browser'; + } + + setBackend(backendId) { + const backend = this.detectBackends({ force: true }).find((b) => b.id === backendId); + if (!backend) throw new Error(`Unknown speech backend "${backendId}"`); + if (!backend.available) throw new Error(`Speech backend "${backendId}" is not available on this machine`); + this.preferredBackend = backendId; + return backendId; + } + + isRepeat(text) { + const last = this.lastSpokenAt.get(text); + return Boolean(last && Date.now() - last < REPEAT_WINDOW_MS); + } + + record(entry) { + this.history.unshift(entry); + if (this.history.length > HISTORY_LIMIT) this.history.length = HISTORY_LIMIT; + return entry; + } + + speakViaBrowser(text, priority) { + if (!this.io) return { spoken: false, reason: 'no socket connection to a client' }; + this.io.emit('speech-speak', { text, priority, at: new Date().toISOString() }); + return { spoken: true }; + } + + spawnQuiet(command, args) { + try { + const child = spawn(command, args, { + ...getHiddenProcessOptions({ stdio: 'ignore', detached: false }), + env: augmentProcessEnv(process.env) + }); + child.on('error', (error) => this.logger.warn?.('Speech backend failed', { command, error: error.message })); + child.unref?.(); + return { spoken: true }; + } catch (error) { + return { spoken: false, reason: error.message }; + } + } + + /** + * Piper emits raw PCM on stdout, so it is piped straight into a player + * rather than through a shell — the text never touches a command line. + */ + speakViaPiper(text) { + const player = ['paplay', 'aplay'].find((candidate) => commandExists(candidate)); + if (!player) return { spoken: false, reason: 'piper is installed but no audio player was found' }; + + const playerArgs = player === 'aplay' + ? ['-q', '-r', '22050', '-f', 'S16_LE', '-t', 'raw', '-'] + : ['--raw', '--rate=22050', '--format=s16le', '--channels=1']; + + try { + const env = augmentProcessEnv(process.env); + const piper = spawn('piper', ['--model', this.piperModel, '--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); + const playback = spawn(player, playerArgs, { stdio: ['pipe', 'ignore', 'ignore'], env }); + + piper.on('error', (error) => this.logger.warn?.('Piper failed', { error: error.message })); + playback.on('error', (error) => this.logger.warn?.('Audio playback failed', { player, error: error.message })); + + piper.stdout.pipe(playback.stdin); + piper.stdin.end(`${text}\n`); + return { spoken: true }; + } catch (error) { + return { spoken: false, reason: error.message }; + } + } + + speakLocally(backendId, text) { + if (backendId === 'say') { + return this.spawnQuiet('say', this.voice ? ['-v', this.voice, text] : [text]); + } + if (backendId === 'espeak') { + const binary = commandExists('espeak-ng') ? 'espeak-ng' : 'espeak'; + return this.spawnQuiet(binary, [text]); + } + if (backendId === 'piper') return this.speakViaPiper(text); + if (backendId === 'sapi') { + // Text is already sanitized to printable ASCII with no shell metacharacters; + // single quotes are doubled because PowerShell escapes them that way. + const escaped = text.replace(/'/g, "''"); + return this.spawnQuiet('powershell.exe', [ + '-NoProfile', '-NonInteractive', '-Command', + `Add-Type -AssemblyName System.Speech; (New-Object System.Speech.Synthesis.SpeechSynthesizer).Speak('${escaped}')` + ]); + } + return { spoken: false, reason: `no local handler for "${backendId}"` }; + } + + /** + * Say something. Never throws — speech failing must not take down whatever + * was trying to talk. + */ + speak(rawText, { priority = 'normal', force = false } = {}) { + const text = sanitizeForSpeech(rawText); + if (!text) return { spoken: false, reason: 'nothing to say' }; + if (!this.enabled) return this.record({ text, at: new Date().toISOString(), spoken: false, reason: 'speech disabled' }); + if (!force && this.isRepeat(text)) { + return this.record({ text, at: new Date().toISOString(), spoken: false, reason: 'just said that' }); + } + + const backend = this.resolveBackend(); + let result; + try { + result = backend === 'browser' ? this.speakViaBrowser(text, priority) : this.speakLocally(backend, text); + } catch (error) { + result = { spoken: false, reason: error.message }; + } + + if (result.spoken) this.lastSpokenAt.set(text, Date.now()); + return this.record({ text, backend, priority, at: new Date().toISOString(), ...result }); + } + + getStatus() { + return { + enabled: this.enabled, + backend: this.resolveBackend(), + preferredBackend: this.preferredBackend || null, + backends: this.detectBackends(), + // The browser backend only actually makes noise if a page is listening. + connectedClients: Number(this.io?.engine?.clientsCount ?? 0), + recent: this.history.slice(0, 10) + }; + } +} + +module.exports = SpeechService; +module.exports.SpeechService = SpeechService; +module.exports.sanitizeForSpeech = sanitizeForSpeech; diff --git a/server/supervisor/supervisorActions.js b/server/supervisor/supervisorActions.js new file mode 100644 index 00000000..1fcdc08c --- /dev/null +++ b/server/supervisor/supervisorActions.js @@ -0,0 +1,189 @@ +const SUBMIT_DELAY_MS = 400; + +const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); + +/** + * Agent CLIs treat a single "text\r" chunk as a bracketed paste — the text + * lands in the composer but is never submitted. Text and Enter must be + * separate writes, which is the same rule the spawn and pager paths follow. + */ +async function submitText(sessionManager, sessionId, text, { delayMs = SUBMIT_DELAY_MS } = {}) { + const wrote = sessionManager?.writeToSession?.(sessionId, text); + if (!wrote) return false; + await sleep(delayMs); + return Boolean(sessionManager?.writeToSession?.(sessionId, '\r')); +} + +/** + * Decide whether a pending permission prompt is safe to approve without a human. + * + * Fails closed on every ambiguity: no allow-pattern match, any deny-pattern + * match, or an unreadable prompt all mean "ask the human". + */ +function classifyPermissionPrompt(tail, safety) { + const text = String(tail || ''); + const window = text.slice(-1200); + + for (const pattern of safety?.permissionDenyPatterns || []) { + if (pattern.test(window)) { + return { safe: false, reason: `matched deny pattern ${pattern}` }; + } + } + + const allowed = (safety?.permissionAllowPatterns || []).find((pattern) => pattern.test(window)); + if (!allowed) { + return { safe: false, reason: 'no allow pattern matched — treating as unknown' }; + } + + return { safe: true, reason: `matched allow pattern ${allowed}` }; +} + +function buildActHandlers({ sessionManager, gitHelper, agentManager, logger = console }) { + return { + /** + * Approve a permission prompt only when it is unambiguously read-only. + */ + 'answer-permission': async ({ finding, signal, rules }) => { + const verdict = classifyPermissionPrompt(signal?.tail, rules?.safety); + if (!verdict.safe) { + return { performed: false, escalate: true, detail: `refused to auto-answer: ${verdict.reason}` }; + } + const submitted = await submitText(sessionManager, finding.sessionId, '1'); + return { performed: submitted, detail: submitted ? `approved (${verdict.reason})` : 'write failed' }; + }, + + /** + * Bring a dead agent terminal back up in the same worktree. + */ + 'relaunch-agent': async ({ finding, signal }) => { + const agentId = signal?.agent || (signal?.type === 'codex' ? 'codex' : 'claude'); + let command = ''; + try { + command = agentManager?.buildCommand?.(agentId, 'resume') || ''; + } catch { + command = ''; + } + if (!command) { + return { performed: false, escalate: true, detail: `no resume command registered for agent "${agentId}"` }; + } + const submitted = await submitText(sessionManager, finding.sessionId, command); + return { performed: submitted, detail: submitted ? `relaunched ${agentId}` : 'write failed' }; + }, + + /** + * Outward-facing: creates a real PR. No shipped rule selects this handler — + * it exists for opt-in autopilot configurations. + */ + 'open-pull-request': async ({ finding, signal }) => { + if (!signal?.cwd || !signal?.branch) { + return { performed: false, escalate: true, detail: 'no worktree path or branch to open a PR from' }; + } + try { + await gitHelper?.execGh?.(['pr', 'create', '--fill', '--head', signal.branch], { cwd: signal.cwd, timeout: 30_000 }); + return { performed: true, detail: `opened a PR for ${signal.branch}` }; + } catch (error) { + logger.warn('Supervisor could not open a pull request', { sessionId: finding.sessionId, error: error.message }); + return { performed: false, escalate: true, detail: `gh pr create failed: ${error.message}` }; + } + } + }; +} + +/** + * Executes one finding at its effective rung. + * + * Rungs are cumulative in intent but not in effect: a `nudge` notifies and + * types, an `act` notifies and runs its handler. `observe` deliberately does + * nothing outward, which is what makes it safe to leave running for a week. + */ +function createExecutor({ + sessionManager, + gitHelper, + agentManager, + activityFeed, + notificationService, + speechService, + logger = console +} = {}) { + const actHandlers = buildActHandlers({ sessionManager, gitHelper, agentManager, logger }); + + const announce = (finding, detail) => { + try { + activityFeed?.track?.('supervisor.finding', { + sessionId: finding.sessionId, + conditionId: finding.conditionId, + severity: finding.severity, + label: finding.label, + rung: finding.rung, + detail + }); + } catch (error) { + logger.warn('Supervisor could not record activity', { error: error.message }); + } + + try { + notificationService?.notify?.( + finding.sessionId, + finding.severity === 'critical' ? 'error' : 'warning', + `${finding.label} — ${finding.worktreeId || finding.sessionId}`, + { conditionId: finding.conditionId, advice: finding.advice } + ); + } catch (error) { + logger.warn('Supervisor could not send a notification', { error: error.message }); + } + + if (finding.severity === 'critical') { + try { + speechService?.speak?.(`${finding.label} on ${finding.worktreeId || finding.sessionId}`, { priority: 'high' }); + } catch (error) { + logger.warn('Supervisor could not speak', { error: error.message }); + } + } + }; + + return async function execute({ finding, signal, rules }) { + if (!finding.rung || finding.rung === 'observe') { + return { ...finding, performed: false, outcome: 'observed' }; + } + + announce(finding, finding.advice); + + if (finding.rung === 'notify') { + return { ...finding, performed: true, outcome: 'notified' }; + } + + if (finding.rung === 'nudge') { + const text = finding.nudgeText; + if (!text) return { ...finding, performed: false, outcome: 'nudge-skipped', detail: 'no nudge text configured' }; + const submitted = await submitText(sessionManager, finding.sessionId, text); + return { ...finding, performed: submitted, outcome: submitted ? 'nudged' : 'nudge-failed' }; + } + + const handlerId = finding.actHandler; + if (!handlerId) { + return { ...finding, performed: false, outcome: 'act-skipped', detail: 'no act handler configured' }; + } + if (!(rules?.safety?.allowedActHandlers || []).includes(handlerId)) { + return { ...finding, performed: false, outcome: 'act-blocked', detail: `handler "${handlerId}" is not in allowedActHandlers` }; + } + const handler = actHandlers[handlerId]; + if (!handler) { + return { ...finding, performed: false, outcome: 'act-blocked', detail: `unknown handler "${handlerId}"` }; + } + + const result = await handler({ finding, signal, rules }); + if (!result.performed && result.escalate) { + announce({ ...finding, severity: 'critical' }, result.detail); + return { ...finding, performed: false, outcome: 'escalated', detail: result.detail }; + } + return { ...finding, performed: result.performed, outcome: result.performed ? 'acted' : 'act-failed', detail: result.detail }; + }; +} + +module.exports = { + SUBMIT_DELAY_MS, + submitText, + classifyPermissionPrompt, + buildActHandlers, + createExecutor +}; diff --git a/server/supervisor/supervisorRules.js b/server/supervisor/supervisorRules.js new file mode 100644 index 00000000..f88afede --- /dev/null +++ b/server/supervisor/supervisorRules.js @@ -0,0 +1,204 @@ +const fs = require('fs'); +const path = require('path'); + +const { getAgentWorkspaceDir } = require('../utils/pathUtils'); + +const RUNGS = ['observe', 'notify', 'nudge', 'act']; +const SEVERITIES = ['info', 'warn', 'critical']; +const AUTONOMY_LEVELS = ['off', 'observe', 'assist', 'autopilot']; + +// How far each autonomy level is allowed to climb the ladder. +const AUTONOMY_CEILING = { + off: null, + observe: 'observe', + assist: 'nudge', + autopilot: 'act' +}; + +const DEFAULT_RULES_PATH = path.join(__dirname, '..', '..', 'config', 'supervisor-rules.json'); + +function overrideRulesPath() { + return path.join(getAgentWorkspaceDir(), 'supervisor-rules.json'); +} + +function compilePatterns(patterns) { + const out = []; + for (const pattern of Array.isArray(patterns) ? patterns : []) { + try { + out.push(new RegExp(String(pattern))); + } catch { + // A bad pattern must not take the whole supervisor down with it. + } + } + return out; +} + +function normalizeCondition(raw) { + const id = String(raw?.id || '').trim(); + if (!id) return null; + + const when = raw?.when || {}; + const rung = RUNGS.includes(raw?.rung) ? raw.rung : 'observe'; + + return { + id, + label: String(raw?.label || id), + severity: SEVERITIES.includes(raw?.severity) ? raw.severity : 'info', + rung, + cooldownSeconds: Math.max(0, Number(raw?.cooldownSeconds) || 0), + advice: String(raw?.advice || ''), + nudgeText: String(raw?.nudgeText || '').trim(), + actHandler: String(raw?.actHandler || '').trim(), + when: { + status: (Array.isArray(when.status) ? when.status : []).map((s) => String(s).toLowerCase()), + types: (Array.isArray(when.types) ? when.types : []).map((s) => String(s).toLowerCase()), + tailMatches: compilePatterns(when.tailMatches), + tailNotMatches: compilePatterns(when.tailNotMatches), + minQuietSeconds: Number.isFinite(Number(when.minQuietSeconds)) ? Number(when.minQuietSeconds) : null, + maxQuietSeconds: Number.isFinite(Number(when.maxQuietSeconds)) ? Number(when.maxQuietSeconds) : null, + repeatedTailLine: Number.isFinite(Number(when.repeatedTailLine)) ? Number(when.repeatedTailLine) : null, + agentPresent: typeof when.agentPresent === 'boolean' ? when.agentPresent : null, + tiers: (Array.isArray(when.tiers) ? when.tiers : []).map(Number).filter(Number.isFinite), + git: when.git && typeof when.git === 'object' ? { + dirty: typeof when.git.dirty === 'boolean' ? when.git.dirty : null, + aheadMin: Number.isFinite(Number(when.git.aheadMin)) ? Number(when.git.aheadMin) : null, + aheadMax: Number.isFinite(Number(when.git.aheadMax)) ? Number(when.git.aheadMax) : null, + hasUpstream: typeof when.git.hasUpstream === 'boolean' ? when.git.hasUpstream : null + } : null + } + }; +} + +function readJson(filePath) { + try { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); + } catch { + return null; + } +} + +/** + * Load the condition table. A machine-local override replaces the shipped + * defaults wholesale — merging rule arrays produces surprises nobody wants + * from something that types into terminals. + */ +function loadRules({ rulesPath = null } = {}) { + const override = rulesPath || overrideRulesPath(); + const raw = readJson(override) || readJson(DEFAULT_RULES_PATH) || {}; + const source = readJson(override) ? override : DEFAULT_RULES_PATH; + + const safety = raw.safety || {}; + return { + source, + autonomy: AUTONOMY_LEVELS.includes(raw.autonomy) ? raw.autonomy : 'observe', + tickSeconds: Math.max(5, Number(raw.tickSeconds) || 30), + maxFindingsRetained: Math.max(20, Number(raw.maxFindingsRetained) || 500), + safety: { + allowedActHandlers: Array.isArray(safety.allowedActHandlers) ? safety.allowedActHandlers.map(String) : [], + permissionAllowPatterns: compilePatterns(safety.permissionAllowPatterns), + permissionDenyPatterns: compilePatterns(safety.permissionDenyPatterns) + }, + conditions: (Array.isArray(raw.conditions) ? raw.conditions : []).map(normalizeCondition).filter(Boolean) + }; +} + +function gitMatches(rule, git) { + if (!rule) return true; + // A rule that asks about git cannot fire on a session we have no git read for. + if (!git) return false; + if (rule.dirty !== null && Boolean(git.dirty) !== rule.dirty) return false; + if (rule.hasUpstream !== null && Boolean(git.hasUpstream) !== rule.hasUpstream) return false; + if (rule.aheadMin !== null && !(Number(git.ahead || 0) >= rule.aheadMin)) return false; + if (rule.aheadMax !== null && !(Number(git.ahead || 0) <= rule.aheadMax)) return false; + return true; +} + +function matches(condition, signal) { + const when = condition.when; + + if (when.status.length && !when.status.includes(signal.status)) return false; + if (when.types.length && !when.types.includes(signal.type)) return false; + if (when.agentPresent !== null && Boolean(signal.agentPresent) !== when.agentPresent) return false; + if (when.tiers.length && !when.tiers.includes(Number(signal.tier))) return false; + if (when.minQuietSeconds !== null && signal.quietSeconds < when.minQuietSeconds) return false; + if (when.maxQuietSeconds !== null && signal.quietSeconds > when.maxQuietSeconds) return false; + if (when.repeatedTailLine !== null && Number(signal.repeatedLineCount || 0) < when.repeatedTailLine) return false; + + if (when.tailMatches.length && !when.tailMatches.some((re) => re.test(signal.tail))) return false; + if (when.tailNotMatches.length && when.tailNotMatches.some((re) => re.test(signal.tail))) return false; + + if (!gitMatches(when.git, signal.git)) return false; + + return true; +} + +function rungIndex(rung) { + const index = RUNGS.indexOf(rung); + return index === -1 ? 0 : index; +} + +/** + * The rung a finding may actually reach, given how much autonomy it has been + * granted. Findings above the ceiling are recorded, not acted on. + */ +function effectiveRung(conditionRung, autonomy) { + const ceiling = AUTONOMY_CEILING[autonomy]; + if (!ceiling) return null; + return rungIndex(conditionRung) <= rungIndex(ceiling) ? conditionRung : ceiling; +} + +function buildFinding(condition, signal, autonomy) { + return { + id: `${signal.sessionId}:${condition.id}`, + conditionId: condition.id, + label: condition.label, + severity: condition.severity, + sessionId: signal.sessionId, + worktreeId: signal.worktreeId, + repositoryName: signal.repositoryName, + branch: signal.branch, + tier: signal.tier, + status: signal.status, + quietSeconds: signal.quietSeconds, + advice: condition.advice, + requestedRung: condition.rung, + rung: effectiveRung(condition.rung, autonomy), + suppressedByAutonomy: rungIndex(condition.rung) > rungIndex(AUTONOMY_CEILING[autonomy] || 'observe'), + nudgeText: condition.nudgeText, + actHandler: condition.actHandler, + evidence: signal.lastLine, + detectedAt: new Date().toISOString() + }; +} + +/** + * First matching condition wins per session — the table is ordered by urgency, + * so a session at its usage limit is not also reported as merely "stalled". + */ +function evaluate(signals, rules) { + const findings = []; + if (rules.autonomy === 'off') return findings; + + for (const signal of signals) { + for (const condition of rules.conditions) { + if (!matches(condition, signal)) continue; + findings.push(buildFinding(condition, signal, rules.autonomy)); + break; + } + } + return findings; +} + +module.exports = { + RUNGS, + SEVERITIES, + AUTONOMY_LEVELS, + AUTONOMY_CEILING, + DEFAULT_RULES_PATH, + overrideRulesPath, + loadRules, + normalizeCondition, + matches, + effectiveRung, + evaluate +}; diff --git a/server/supervisor/supervisorSignals.js b/server/supervisor/supervisorSignals.js new file mode 100644 index 00000000..a223c5c3 --- /dev/null +++ b/server/supervisor/supervisorSignals.js @@ -0,0 +1,208 @@ +const TAIL_CHARS = 4000; +const SUPERVISED_TYPES = new Set(['claude', 'codex']); + +function stripControlSequences(text) { + return String(text || '') + .replace(/\x1b\[[0-9;?]*[ -/]*[@-~]/g, '') + .replace(/\x1b\][^\x07]*(?:\x07|\x1b\\)/g, '') + .replace(/\x1b[()][A-Za-z0-9]/g, ''); +} + +function lastNonEmptyLines(text, count) { + const lines = String(text || '').split('\n'); + const out = []; + for (let i = lines.length - 1; i >= 0 && out.length < count; i -= 1) { + const line = lines[i].replace(/\r/g, '').trim(); + if (line) out.push(line); + } + return out; +} + +/** + * How many times the most-repeated line appears in the tail. A high count is + * the cheapest reliable "this agent is looping" signal there is. + */ +function maxLineRepeat(text, { window = 40, minLength = 12 } = {}) { + const counts = new Map(); + let max = 0; + for (const line of lastNonEmptyLines(text, window)) { + if (line.length < minLength) continue; + const next = (counts.get(line) || 0) + 1; + counts.set(line, next); + if (next > max) max = next; + } + return max; +} + +/** + * Tracks per-session buffer growth so "quiet for N seconds" is measurable + * without touching the PTY or asking the agent anything. + */ +class QuietTracker { + constructor({ now = () => Date.now() } = {}) { + this.now = now; + this.state = new Map(); + } + + observe(sessionId, bufferLength) { + const at = this.now(); + const previous = this.state.get(sessionId); + + if (!previous) { + this.state.set(sessionId, { bufferLength, lastGrowthAt: at, firstSeenAt: at }); + return 0; + } + if (bufferLength !== previous.bufferLength) { + previous.bufferLength = bufferLength; + previous.lastGrowthAt = at; + return 0; + } + return Math.max(0, Math.round((at - previous.lastGrowthAt) / 1000)); + } + + forget(sessionId) { + this.state.delete(sessionId); + } + + prune(liveSessionIds) { + const live = new Set(liveSessionIds); + for (const id of [...this.state.keys()]) { + if (!live.has(id)) this.state.delete(id); + } + } +} + +function listSupervisedSessions(sessionManager) { + const out = []; + const addMap = (map) => { + if (!(map instanceof Map)) return; + for (const [id, session] of map.entries()) { + if (!id || !session) continue; + if (!SUPERVISED_TYPES.has(String(session.type || '').toLowerCase())) continue; + if (out.some((existing) => existing.id === id)) continue; + out.push({ id, session }); + } + }; + + addMap(sessionManager?.sessions); + const byWorkspace = sessionManager?.workspaceSessionMaps; + if (byWorkspace instanceof Map) { + for (const map of byWorkspace.values()) addMap(map); + } + return out; +} + +async function countCommits(gitHelper, cwd, range) { + try { + const { stdout } = await gitHelper.execGit(['rev-list', '--count', range], { cwd, timeout: 5000 }); + const count = Number(String(stdout || '').trim()); + return Number.isFinite(count) ? count : null; + } catch { + return null; + } +} + +/** + * How much work has not left this machine. + * + * With an upstream that is `@{upstream}..HEAD`. Without one — a branch that was + * never pushed at all — every commit since the default branch counts, which is + * the case that most deserves a nudge. + */ +async function countUnpushedCommits(gitHelper, cwd) { + const againstUpstream = await countCommits(gitHelper, cwd, '@{upstream}..HEAD'); + if (againstUpstream !== null) return { ahead: againstUpstream, hasUpstream: true }; + + const defaultBranch = await gitHelper.getDefaultBranch?.(cwd).catch(() => null); + if (!defaultBranch) return { ahead: null, hasUpstream: false }; + + const againstDefault = await countCommits(gitHelper, cwd, `${defaultBranch}..HEAD`); + return { ahead: againstDefault, hasUpstream: false }; +} + +/** + * Git state is the expensive signal, so it is only collected for sessions that + * have gone quiet — a busy agent's working tree is a moving target anyway. + */ +async function collectGitState(gitHelper, cwd) { + if (!gitHelper || !cwd) return null; + try { + const status = await gitHelper.getStatus(cwd); + if (!status) return null; + const { ahead, hasUpstream } = await countUnpushedCommits(gitHelper, cwd); + return { + dirty: status.clean === false, + changedFiles: Number(status.total || 0), + ahead: ahead === null ? 0 : ahead, + aheadKnown: ahead !== null, + hasUpstream + }; + } catch { + return null; + } +} + +async function gatherSignals({ + sessionManager, + gitHelper, + sessionRecoveryService, + taskRecordService, + quietTracker, + gitQuietThresholdSeconds = 120 +} = {}) { + const supervised = listSupervisedSessions(sessionManager); + quietTracker?.prune(supervised.map(({ id }) => id)); + + const signals = []; + for (const { id, session } of supervised) { + const buffer = String(session.buffer || ''); + const quietSeconds = quietTracker ? quietTracker.observe(id, buffer.length) : 0; + const tail = stripControlSequences(buffer.slice(-TAIL_CHARS)); + + const workspaceId = String(session.workspace || '').trim(); + const recovery = workspaceId ? sessionRecoveryService?.getSession?.(workspaceId, id) : null; + const agentPresent = recovery ? recovery.lastAgentActive !== false : true; + + const cwd = sessionManager?.getSessionCwd?.(session) || recovery?.lastCwd || null; + const git = quietSeconds >= gitQuietThresholdSeconds + ? await collectGitState(gitHelper, cwd) + : null; + + const record = taskRecordService?.get?.(`session:${id}`) || null; + + signals.push({ + sessionId: id, + type: String(session.type || '').toLowerCase(), + status: String(session.status || 'idle').toLowerCase(), + agent: recovery?.lastAgent || (session.type === 'codex' ? 'codex' : null), + agentPresent, + workspaceId, + worktreeId: session.worktreeId || null, + repositoryName: session.repositoryName || null, + branch: session.branch || null, + cwd, + quietSeconds, + tail, + lastLine: lastNonEmptyLines(tail, 1)[0] || '', + repeatedLineCount: maxLineRepeat(tail), + git, + tier: Number(record?.tier) || null, + ticketTitle: record?.ticketTitle || null + }); + } + + return signals; +} + +module.exports = { + TAIL_CHARS, + SUPERVISED_TYPES, + QuietTracker, + stripControlSequences, + lastNonEmptyLines, + maxLineRepeat, + listSupervisedSessions, + countUnpushedCommits, + collectGitState, + gatherSignals +}; diff --git a/server/supervisorService.js b/server/supervisorService.js new file mode 100644 index 00000000..50958c2c --- /dev/null +++ b/server/supervisorService.js @@ -0,0 +1,329 @@ +const fs = require('fs'); +const path = require('path'); + +const { getAgentWorkspaceDir } = require('./utils/pathUtils'); +const { QuietTracker, gatherSignals } = require('./supervisor/supervisorSignals'); +const rulesModule = require('./supervisor/supervisorRules'); +const { createExecutor } = require('./supervisor/supervisorActions'); + +const AUDIT_FILENAME = 'supervisor-audit.jsonl'; + +/** + * The fleet supervisor. + * + * Watches every agent session on a fixed tick using signals that cost nothing — + * PTY tail, status, how long a buffer has been quiet, git state — matches them + * against a data-driven condition table, and climbs an escalation ladder capped + * by the configured autonomy level. + * + * No model is called in the loop. Judgement is only invoked on escalation, which + * is what makes continuous supervision affordable to leave running. + */ +class SupervisorService { + constructor({ logger = console } = {}) { + this.logger = logger; + this.sessionManager = null; + this.gitHelper = null; + this.agentManager = null; + this.sessionRecoveryService = null; + this.taskRecordService = null; + this.activityFeed = null; + this.notificationService = null; + this.speechService = null; + + this.rules = rulesModule.loadRules(); + this.quietTracker = new QuietTracker(); + this.executor = null; + this.timer = null; + this.running = false; + this.ticking = false; + + this.findings = []; + this.lastTickAt = null; + this.lastTickDurationMs = null; + this.tickCount = 0; + this.cooldowns = new Map(); + } + + static getInstance(options = {}) { + if (!SupervisorService.instance) { + SupervisorService.instance = new SupervisorService(options); + } + return SupervisorService.instance; + } + + init({ + sessionManager, gitHelper, agentManager, sessionRecoveryService, + taskRecordService, activityFeed, notificationService, speechService + } = {}) { + this.sessionManager = sessionManager || this.sessionManager; + this.gitHelper = gitHelper || this.gitHelper; + this.agentManager = agentManager || this.agentManager; + this.sessionRecoveryService = sessionRecoveryService || this.sessionRecoveryService; + this.taskRecordService = taskRecordService || this.taskRecordService; + this.activityFeed = activityFeed || this.activityFeed; + this.notificationService = notificationService || this.notificationService; + this.speechService = speechService || this.speechService; + + this.executor = createExecutor({ + sessionManager: this.sessionManager, + gitHelper: this.gitHelper, + agentManager: this.agentManager, + activityFeed: this.activityFeed, + notificationService: this.notificationService, + speechService: this.speechService, + logger: this.logger + }); + + return this; + } + + auditPath() { + const logsDir = path.join(getAgentWorkspaceDir(), 'logs'); + try { + fs.mkdirSync(logsDir, { recursive: true }); + } catch { + // Losing the audit trail must not stop supervision. + } + return path.join(logsDir, AUDIT_FILENAME); + } + + appendAudit(row) { + try { + fs.appendFileSync(this.auditPath(), `${JSON.stringify({ at: new Date().toISOString(), ...row })}\n`, 'utf8'); + } catch (error) { + this.logger.warn?.('Supervisor could not write its audit log', { error: error.message }); + } + } + + reloadRules({ rulesPath = null } = {}) { + this.rules = rulesModule.loadRules({ rulesPath }); + if (this.running) this.restartTimer(); + return this.rules; + } + + setAutonomy(level) { + if (!rulesModule.AUTONOMY_LEVELS.includes(level)) { + throw new Error(`Unknown autonomy level "${level}" (expected ${rulesModule.AUTONOMY_LEVELS.join('|')})`); + } + const previous = this.rules.autonomy; + this.rules.autonomy = level; + this.appendAudit({ event: 'autonomy-changed', from: previous, to: level }); + return level; + } + + isCoolingDown(finding, condition) { + const cooldownMs = Math.max(0, Number(condition?.cooldownSeconds || 0)) * 1000; + if (!cooldownMs) return false; + const last = this.cooldowns.get(finding.id); + return Boolean(last && Date.now() - last < cooldownMs); + } + + markActed(finding) { + this.cooldowns.set(finding.id, Date.now()); + } + + recordFinding(entry) { + this.findings.unshift(entry); + const cap = this.rules.maxFindingsRetained; + if (this.findings.length > cap) this.findings.length = cap; + } + + /** + * One pass over the fleet. Safe to call by hand — `POST /api/supervisor/tick` + * runs exactly this, which is how you validate rule changes without waiting. + */ + async tick({ dryRun = false } = {}) { + if (this.ticking) return { skipped: 'already ticking' }; + this.ticking = true; + const startedAt = Date.now(); + + try { + const signals = await gatherSignals({ + sessionManager: this.sessionManager, + gitHelper: this.gitHelper, + sessionRecoveryService: this.sessionRecoveryService, + taskRecordService: this.taskRecordService, + quietTracker: this.quietTracker + }); + + const findings = rulesModule.evaluate(signals, this.rules); + const signalsById = new Map(signals.map((signal) => [signal.sessionId, signal])); + const conditionsById = new Map(this.rules.conditions.map((condition) => [condition.id, condition])); + const results = []; + + for (const finding of findings) { + const condition = conditionsById.get(finding.conditionId); + if (this.isCoolingDown(finding, condition)) { + results.push({ ...finding, performed: false, outcome: 'cooling-down' }); + continue; + } + + if (dryRun) { + results.push({ ...finding, performed: false, outcome: 'dry-run' }); + continue; + } + + const executed = await this.executor({ + finding, + signal: signalsById.get(finding.sessionId), + rules: this.rules + }); + + if (executed.outcome !== 'observed') { + this.markActed(finding); + this.appendAudit({ + event: 'finding', + id: finding.id, + conditionId: finding.conditionId, + sessionId: finding.sessionId, + severity: finding.severity, + requestedRung: finding.requestedRung, + rung: finding.rung, + outcome: executed.outcome, + detail: executed.detail || finding.advice + }); + } + + this.recordFinding(executed); + results.push(executed); + } + + this.lastTickAt = new Date().toISOString(); + this.lastTickDurationMs = Date.now() - startedAt; + this.tickCount += 1; + + return { + at: this.lastTickAt, + durationMs: this.lastTickDurationMs, + autonomy: this.rules.autonomy, + sessionsWatched: signals.length, + findings: results + }; + } catch (error) { + this.logger.error?.('Supervisor tick failed', { error: error.message, stack: error.stack }); + return { error: error.message }; + } finally { + this.ticking = false; + } + } + + restartTimer() { + if (this.timer) clearInterval(this.timer); + this.timer = setInterval(() => { + this.tick().catch((error) => this.logger.error?.('Supervisor tick threw', { error: error.message })); + }, this.rules.tickSeconds * 1000); + if (typeof this.timer.unref === 'function') this.timer.unref(); + } + + start() { + if (this.running) return { running: true, alreadyRunning: true }; + if (this.rules.autonomy === 'off') return { running: false, reason: 'autonomy is off' }; + this.running = true; + this.restartTimer(); + this.appendAudit({ event: 'started', autonomy: this.rules.autonomy, tickSeconds: this.rules.tickSeconds }); + return { running: true, tickSeconds: this.rules.tickSeconds, autonomy: this.rules.autonomy }; + } + + stop() { + if (this.timer) clearInterval(this.timer); + this.timer = null; + const wasRunning = this.running; + this.running = false; + if (wasRunning) this.appendAudit({ event: 'stopped' }); + return { running: false, wasRunning }; + } + + getFindings({ limit = 100, severity = '', sessionId = '' } = {}) { + const wantSeverity = String(severity || '').toLowerCase(); + const wantSession = String(sessionId || '').trim(); + return this.findings + .filter((finding) => (!wantSeverity || finding.severity === wantSeverity)) + .filter((finding) => (!wantSession || finding.sessionId === wantSession)) + .slice(0, Math.max(1, Number(limit) || 100)); + } + + /** + * A one-line-per-item read of what needs a human right now. This is what the + * voice briefing speaks and what the Commander asks for. + */ + getBriefing({ limit = 6 } = {}) { + const bySession = new Map(); + for (const finding of this.findings) { + if (!bySession.has(finding.id)) bySession.set(finding.id, finding); + } + + const rank = { critical: 0, warn: 1, info: 2 }; + const items = [...bySession.values()] + .sort((a, b) => (rank[a.severity] ?? 3) - (rank[b.severity] ?? 3) || String(b.detectedAt).localeCompare(String(a.detectedAt))) + .slice(0, Math.max(1, Number(limit) || 6)) + .map((finding) => ({ + severity: finding.severity, + sessionId: finding.sessionId, + where: finding.worktreeId || finding.sessionId, + label: finding.label, + advice: finding.advice, + outcome: finding.outcome, + detectedAt: finding.detectedAt + })); + + const counts = { critical: 0, warn: 0, info: 0 }; + for (const finding of bySession.values()) counts[finding.severity] = (counts[finding.severity] || 0) + 1; + + return { + autonomy: this.rules.autonomy, + running: this.running, + lastTickAt: this.lastTickAt, + counts, + items, + spoken: this.renderSpokenBriefing(items, counts) + }; + } + + renderSpokenBriefing(items, counts) { + if (!items.length) { + return this.running + ? 'Nothing needs you. The fleet is quiet.' + : 'The supervisor is not running.'; + } + + const headline = counts.critical + ? `${counts.critical} thing${counts.critical === 1 ? '' : 's'} need${counts.critical === 1 ? 's' : ''} you now.` + : `${items.length} thing${items.length === 1 ? '' : 's'} to look at.`; + + const details = items + .slice(0, 3) + .map((item) => `${item.where}: ${item.label.toLowerCase()}`) + .join('. '); + + return `${headline} ${details}.`; + } + + getStatus() { + return { + running: this.running, + autonomy: this.rules.autonomy, + tickSeconds: this.rules.tickSeconds, + rulesSource: this.rules.source, + conditionCount: this.rules.conditions.length, + lastTickAt: this.lastTickAt, + lastTickDurationMs: this.lastTickDurationMs, + tickCount: this.tickCount, + findingCount: this.findings.length, + auditPath: this.auditPath(), + autonomyLevels: rulesModule.AUTONOMY_LEVELS, + conditions: this.rules.conditions.map((condition) => ({ + id: condition.id, + label: condition.label, + severity: condition.severity, + rung: condition.rung, + effectiveRung: rulesModule.effectiveRung(condition.rung, this.rules.autonomy), + cooldownSeconds: condition.cooldownSeconds + })) + }; + } +} + +module.exports = SupervisorService; +module.exports.SupervisorService = SupervisorService; +module.exports.rules = rulesModule; diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 7606ab44..04412de8 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -25,6 +25,10 @@ class VoiceCommandService { this.claudeModel = process.env.CLAUDE_VOICE_MODEL || 'claude-3-haiku-20240307'; this.useClaude = false; + // Set by the server once a Commander instance exists; unmatched speech is + // handed to it rather than discarded. + this.commanderForwarder = null; + // Current context (set by orchestrator) this.context = { currentWorkspace: null, @@ -1541,13 +1545,60 @@ JSON:`; } /** - * Parse and execute in one call + * Where unrecognized speech goes. + * + * Without this, anything outside the pattern table is a dead end — which is + * what makes a voice interface feel like a remote control instead of an + * assistant. The forwarder hands the raw utterance to the Commander agent, + * so the fallback for "I didn't match that" is a full agent with the entire + * orchestrator API rather than an error beep. */ - async processVoiceCommand(transcript) { + setCommanderForwarder(forwarder) { + this.commanderForwarder = typeof forwarder === 'function' ? forwarder : null; + return Boolean(this.commanderForwarder); + } + + hasCommanderForwarder() { + return Boolean(this.commanderForwarder); + } + + async forwardToCommander(transcript) { + if (!this.commanderForwarder) { + return { forwarded: false, reason: 'no Commander is running to forward to' }; + } + try { + const result = await this.commanderForwarder(transcript); + return { forwarded: result !== false, result }; + } catch (error) { + return { forwarded: false, reason: error.message }; + } + } + + /** + * Parse and execute in one call. + * + * `forwardUnmatched` defaults on for spoken input: if no rule and no LLM can + * turn the utterance into a command, the words themselves are still useful. + */ + async processVoiceCommand(transcript, { forwardUnmatched = true } = {}) { const parsed = await this.parseCommand(transcript); if (!parsed.success) { - return parsed; + if (!forwardUnmatched || !this.commanderForwarder) return parsed; + + const forward = await this.forwardToCommander(transcript); + if (!forward.forwarded) return { ...parsed, forward }; + + return { + success: true, + method: 'commander', + command: null, + params: {}, + transcript, + executed: true, + forwardedToCommander: true, + result: forward.result + }; } const result = await this.executeCommand(parsed.command, parsed.params); diff --git a/tests/unit/speechService.test.js b/tests/unit/speechService.test.js new file mode 100644 index 00000000..a1a58633 --- /dev/null +++ b/tests/unit/speechService.test.js @@ -0,0 +1,94 @@ +const { SpeechService, sanitizeForSpeech } = require('../../server/speechService'); + +function browserSpeech() { + const emitted = []; + const service = new SpeechService({ logger: { warn: () => {} } }); + service.preferredBackend = 'browser'; + service.setIO({ emit: (event, payload) => emitted.push({ event, payload }) }); + return { service, emitted }; +} + +describe('sanitizeForSpeech', () => { + test('strips ANSI escapes so terminal output is readable aloud', () => { + expect(sanitizeForSpeech('\x1b[32mBuild passed\x1b[0m')).toBe('Build passed'); + }); + + test('removes shell metacharacters — spoken text can reach a command line', () => { + const cleaned = sanitizeForSpeech('run `rm -rf /` && echo $(whoami); cat '); + expect(cleaned).not.toMatch(/[`$\\|&;<>(){}[\]]/); + }); + + test('caps length so nothing monologues', () => { + expect(sanitizeForSpeech('word '.repeat(500)).length).toBeLessThanOrEqual(400); + }); + + test('collapses whitespace and returns empty for nothing useful', () => { + expect(sanitizeForSpeech(' hello \n\n world ')).toBe('hello world'); + expect(sanitizeForSpeech('\x00\x01')).toBe(''); + }); +}); + +describe('SpeechService', () => { + test('the browser backend needs nothing installed', () => { + const { service, emitted } = browserSpeech(); + const result = service.speak('Work one is waiting on permission'); + + expect(result.spoken).toBe(true); + expect(emitted[0].event).toBe('speech-speak'); + expect(emitted[0].payload.text).toBe('Work one is waiting on permission'); + }); + + test('the same message is not repeated back to back', () => { + const { service, emitted } = browserSpeech(); + service.speak('same thing'); + const second = service.speak('same thing'); + + expect(second.spoken).toBe(false); + expect(second.reason).toMatch(/just said that/); + expect(emitted).toHaveLength(1); + }); + + test('force overrides the repeat guard', () => { + const { service, emitted } = browserSpeech(); + service.speak('same thing'); + expect(service.speak('same thing', { force: true }).spoken).toBe(true); + expect(emitted).toHaveLength(2); + }); + + test('with no client connected it reports why rather than throwing', () => { + const service = new SpeechService({ logger: { warn: () => {} } }); + service.preferredBackend = 'browser'; + const result = service.speak('anyone there'); + expect(result.spoken).toBe(false); + expect(result.reason).toMatch(/no socket/); + }); + + test('disabling speech silences it without error', () => { + const { service, emitted } = browserSpeech(); + service.setEnabled(false); + expect(service.speak('quiet please').spoken).toBe(false); + expect(emitted).toEqual([]); + }); + + test('empty input is a no-op', () => { + const { service } = browserSpeech(); + expect(service.speak(' ').spoken).toBe(false); + }); + + test('an unavailable backend cannot be selected', () => { + const service = new SpeechService(); + expect(() => service.setBackend('nonexistent')).toThrow(/Unknown speech backend/); + }); + + test('status reports the resolved backend, listeners and recent utterances', () => { + const service = new SpeechService({ logger: { warn: () => {} } }); + service.preferredBackend = 'browser'; + service.setIO({ emit: () => {}, engine: { clientsCount: 2 } }); + service.speak('one'); + + const status = service.getStatus(); + expect(status.backend).toBe('browser'); + expect(status.connectedClients).toBe(2); + expect(status.recent[0].text).toBe('one'); + }); +}); diff --git a/tests/unit/supervisorActions.test.js b/tests/unit/supervisorActions.test.js new file mode 100644 index 00000000..f9cf0950 --- /dev/null +++ b/tests/unit/supervisorActions.test.js @@ -0,0 +1,177 @@ +const { + classifyPermissionPrompt, + submitText, + createExecutor +} = require('../../server/supervisor/supervisorActions'); +const { loadRules, DEFAULT_RULES_PATH } = require('../../server/supervisor/supervisorRules'); + +const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + +const finding = (overrides = {}) => ({ + id: 'work1-claude:stalled', + conditionId: 'stalled', + label: 'Busy but silent', + severity: 'warn', + sessionId: 'work1-claude', + worktreeId: 'work1', + rung: 'notify', + requestedRung: 'notify', + advice: 'nothing for 15 minutes', + nudgeText: '', + actHandler: '', + ...overrides +}); + +function fakeSessionManager() { + const writes = []; + return { + writes, + writeToSession(sessionId, data) { + writes.push({ sessionId, data }); + return true; + } + }; +} + +describe('supervisorActions', () => { + describe('permission classification', () => { + test('approves an unambiguously read-only prompt', () => { + const verdict = classifyPermissionPrompt('Do you want to proceed? Read(src/index.js)', rules.safety); + expect(verdict.safe).toBe(true); + }); + + test('refuses anything matching a deny pattern, even alongside an allow match', () => { + const verdict = classifyPermissionPrompt('Read(x) then Bash(rm -rf build)', rules.safety); + expect(verdict.safe).toBe(false); + expect(verdict.reason).toMatch(/deny pattern/); + }); + + test('refuses a prompt it does not recognize rather than guessing', () => { + const verdict = classifyPermissionPrompt('Do you want to proceed? SomeNewTool(x)', rules.safety); + expect(verdict.safe).toBe(false); + expect(verdict.reason).toMatch(/no allow pattern/); + }); + + test('refuses git push even though other git commands are allowed', () => { + expect(classifyPermissionPrompt('Bash(git push origin main)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Bash(git status)', rules.safety).safe).toBe(true); + }); + + test('an empty prompt is not safe', () => { + expect(classifyPermissionPrompt('', rules.safety).safe).toBe(false); + }); + }); + + describe('submitText', () => { + test('writes the text and the Enter separately', async () => { + const sessionManager = fakeSessionManager(); + await submitText(sessionManager, 'work1-claude', 'status?', { delayMs: 1 }); + expect(sessionManager.writes).toEqual([ + { sessionId: 'work1-claude', data: 'status?' }, + { sessionId: 'work1-claude', data: '\r' } + ]); + }); + + test('does not send Enter when the first write fails', async () => { + const sessionManager = { writeToSession: () => false }; + expect(await submitText(sessionManager, 'x', 'hi', { delayMs: 1 })).toBe(false); + }); + }); + + describe('executor', () => { + test('observe does nothing outward', async () => { + const sessionManager = fakeSessionManager(); + const tracked = []; + const execute = createExecutor({ sessionManager, activityFeed: { track: (k, d) => tracked.push([k, d]) } }); + + const result = await execute({ finding: finding({ rung: 'observe' }), signal: {}, rules }); + expect(result.outcome).toBe('observed'); + expect(sessionManager.writes).toEqual([]); + expect(tracked).toEqual([]); + }); + + test('notify records and alerts but never types', async () => { + const sessionManager = fakeSessionManager(); + const notified = []; + const execute = createExecutor({ + sessionManager, + activityFeed: { track: () => {} }, + notificationService: { notify: (...args) => notified.push(args) } + }); + + const result = await execute({ finding: finding({ rung: 'notify' }), signal: {}, rules }); + expect(result.outcome).toBe('notified'); + expect(sessionManager.writes).toEqual([]); + expect(notified).toHaveLength(1); + }); + + test('nudge types the configured text into the session', async () => { + const sessionManager = fakeSessionManager(); + const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); + + const result = await execute({ + finding: finding({ rung: 'nudge', nudgeText: 'status?' }), + signal: {}, + rules + }); + expect(result.outcome).toBe('nudged'); + expect(sessionManager.writes.map((w) => w.data)).toEqual(['status?', '\r']); + }); + + test('a handler outside allowedActHandlers is blocked', async () => { + const sessionManager = fakeSessionManager(); + const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); + + const result = await execute({ + finding: finding({ rung: 'act', actHandler: 'delete-everything' }), + signal: {}, + rules + }); + expect(result.outcome).toBe('act-blocked'); + expect(sessionManager.writes).toEqual([]); + }); + + test('auto-answering a risky permission prompt escalates instead', async () => { + const sessionManager = fakeSessionManager(); + const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); + + const result = await execute({ + finding: finding({ rung: 'act', actHandler: 'answer-permission' }), + signal: { tail: 'Do you want to proceed? Bash(sudo rm -rf /)' }, + rules + }); + expect(result.outcome).toBe('escalated'); + expect(sessionManager.writes).toEqual([]); + }); + + test('auto-answering a read-only permission prompt approves it', async () => { + const sessionManager = fakeSessionManager(); + const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); + + const result = await execute({ + finding: finding({ rung: 'act', actHandler: 'answer-permission' }), + signal: { tail: 'Do you want to proceed? Read(src/app.js)' }, + rules + }); + expect(result.outcome).toBe('acted'); + expect(sessionManager.writes.map((w) => w.data)).toEqual(['1', '\r']); + }); + + test('a broken notification channel does not stop the action', async () => { + const sessionManager = fakeSessionManager(); + const execute = createExecutor({ + sessionManager, + activityFeed: { track: () => { throw new Error('feed down'); } }, + notificationService: { notify: () => { throw new Error('notify down'); } }, + logger: { warn: () => {}, error: () => {} } + }); + + const result = await execute({ + finding: finding({ rung: 'nudge', nudgeText: 'ping' }), + signal: {}, + rules + }); + expect(result.outcome).toBe('nudged'); + }); + }); +}); diff --git a/tests/unit/supervisorRules.test.js b/tests/unit/supervisorRules.test.js new file mode 100644 index 00000000..c1c4252a --- /dev/null +++ b/tests/unit/supervisorRules.test.js @@ -0,0 +1,113 @@ +const { + loadRules, + normalizeCondition, + matches, + effectiveRung, + evaluate, + DEFAULT_RULES_PATH +} = require('../../server/supervisor/supervisorRules'); + +const signal = (overrides = {}) => ({ + sessionId: 'zoo-game-work1-claude', + type: 'claude', + status: 'idle', + agent: 'claude', + agentPresent: true, + worktreeId: 'work1', + quietSeconds: 0, + tail: '', + lastLine: '', + repeatedLineCount: 1, + git: null, + tier: 3, + ...overrides +}); + +describe('supervisorRules', () => { + test('the shipped rule table loads and grants no autonomy by default', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + expect(rules.autonomy).toBe('observe'); + expect(rules.conditions.length).toBeGreaterThan(0); + expect(rules.conditions.every((c) => ['observe', 'notify', 'nudge', 'act'].includes(c.rung))).toBe(true); + }); + + test('no shipped condition asks to act — defaults never touch a terminal', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + expect(rules.conditions.some((c) => c.rung === 'act')).toBe(false); + }); + + test('a malformed regex in config does not break rule loading', () => { + const condition = normalizeCondition({ id: 'x', when: { tailMatches: ['(unclosed', 'fine'] } }); + expect(condition.when.tailMatches).toHaveLength(1); + }); + + test('quiet-time thresholds gate a condition', () => { + const condition = normalizeCondition({ id: 'stalled', when: { status: ['busy'], minQuietSeconds: 600 } }); + expect(matches(condition, signal({ status: 'busy', quietSeconds: 300 }))).toBe(false); + expect(matches(condition, signal({ status: 'busy', quietSeconds: 900 }))).toBe(true); + }); + + test('tailNotMatches vetoes an otherwise matching condition', () => { + const condition = normalizeCondition({ + id: 'stalled', + when: { status: ['busy'], tailNotMatches: ['limit reached'] } + }); + expect(matches(condition, signal({ status: 'busy', tail: 'working…' }))).toBe(true); + expect(matches(condition, signal({ status: 'busy', tail: '5-hour limit reached ∙ resets 3am' }))).toBe(false); + }); + + test('a git-dependent condition cannot fire without a git read', () => { + const condition = normalizeCondition({ id: 'unpushed', when: { git: { aheadMin: 1 } } }); + expect(matches(condition, signal({ git: null }))).toBe(false); + expect(matches(condition, signal({ git: { ahead: 2, dirty: false } }))).toBe(true); + expect(matches(condition, signal({ git: { ahead: 0, dirty: false } }))).toBe(false); + }); + + test('autonomy caps how far a condition may climb', () => { + expect(effectiveRung('act', 'off')).toBeNull(); + expect(effectiveRung('act', 'observe')).toBe('observe'); + expect(effectiveRung('act', 'assist')).toBe('nudge'); + expect(effectiveRung('act', 'autopilot')).toBe('act'); + expect(effectiveRung('notify', 'autopilot')).toBe('notify'); + }); + + test('autonomy "off" produces no findings at all', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + rules.autonomy = 'off'; + expect(evaluate([signal({ status: 'busy', quietSeconds: 5000 })], rules)).toEqual([]); + }); + + test('only the first matching condition fires per session', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const findings = evaluate([signal({ + status: 'busy', + quietSeconds: 5000, + tail: 'Claude usage limit reached ∙ resets 3am' + })], rules); + + expect(findings).toHaveLength(1); + expect(findings[0].conditionId).toBe('usage-limit-reached'); + expect(findings[0].severity).toBe('critical'); + }); + + test('a stalled session is reported as stalled, not as a usage limit', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const findings = evaluate([signal({ status: 'busy', quietSeconds: 1200, tail: 'still thinking' })], rules); + expect(findings[0].conditionId).toBe('stalled'); + expect(findings[0].requestedRung).toBe('nudge'); + expect(findings[0].rung).toBe('observe'); + expect(findings[0].suppressedByAutonomy).toBe(true); + }); + + test('findings carry a stable id so cooldowns can key on them', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const [finding] = evaluate([signal({ status: 'busy', quietSeconds: 1200 })], rules); + expect(finding.id).toBe('zoo-game-work1-claude:stalled'); + }); + + test('an exited agent is detected from the recovery marker, not the tail', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const findings = evaluate([signal({ status: 'idle', agentPresent: false, quietSeconds: 600 })], rules); + expect(findings[0].conditionId).toBe('agent-exited'); + }); +}); diff --git a/tests/unit/supervisorService.test.js b/tests/unit/supervisorService.test.js new file mode 100644 index 00000000..f73998e8 --- /dev/null +++ b/tests/unit/supervisorService.test.js @@ -0,0 +1,205 @@ +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const SupervisorService = require('../../server/supervisorService'); +const { QuietTracker, maxLineRepeat, stripControlSequences } = require('../../server/supervisor/supervisorSignals'); + +function fakeSession({ id, status = 'idle', buffer = '', type = 'claude' }) { + return { id, type, status, buffer, worktreeId: id.split('-')[0], workspace: 'ws', pty: {} }; +} + +function harness({ sessions = [], autonomy = 'assist' } = {}) { + const writes = []; + const sessionMap = new Map(sessions.map((session) => [session.id, session])); + + const supervisor = new SupervisorService({ logger: { info: () => {}, warn: () => {}, error: () => {} } }); + supervisor.init({ + sessionManager: { + sessions: sessionMap, + writeToSession: (sessionId, data) => { writes.push({ sessionId, data }); return true; }, + getSessionCwd: () => null + }, + sessionRecoveryService: { getSession: () => ({ lastAgent: 'claude', lastAgentActive: true }) }, + taskRecordService: { get: () => null }, + activityFeed: { track: () => {} }, + notificationService: { notify: () => {} } + }); + supervisor.rules.autonomy = autonomy; + return { supervisor, writes, sessionMap }; +} + +describe('supervisor signals', () => { + test('quiet time only accumulates while the buffer is unchanged', () => { + let now = 1_000_000; + const tracker = new QuietTracker({ now: () => now }); + + expect(tracker.observe('a', 100)).toBe(0); + now += 60_000; + expect(tracker.observe('a', 100)).toBe(60); + now += 60_000; + expect(tracker.observe('a', 250)).toBe(0); + now += 30_000; + expect(tracker.observe('a', 250)).toBe(30); + }); + + test('sessions that disappear are pruned from the tracker', () => { + const tracker = new QuietTracker(); + tracker.observe('a', 1); + tracker.observe('b', 1); + tracker.prune(['a']); + expect(tracker.state.has('b')).toBe(false); + }); + + test('repeated-line detection ignores short lines', () => { + const looping = Array(6).fill('Error: cannot find module "widget"').join('\n'); + expect(maxLineRepeat(looping)).toBe(6); + expect(maxLineRepeat(Array(6).fill('ok').join('\n'))).toBe(0); + }); + + test('ANSI escape sequences are stripped before matching', () => { + expect(stripControlSequences('\x1b[31mError\x1b[0m: boom')).toBe('Error: boom'); + }); +}); + +describe('SupervisorService', () => { + let tmpDir; + + beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'supervisor-test-')); + process.env.AGENT_WORKSPACE_DIR = tmpDir; + }); + + afterEach(() => { + delete process.env.AGENT_WORKSPACE_DIR; + fs.rmSync(tmpDir, { recursive: true, force: true }); + }); + + test('observe mode watches without touching anything', async () => { + const { supervisor, writes } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], + autonomy: 'observe' + }); + + supervisor.quietTracker.observe('work1-claude', 8); + supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + + const result = await supervisor.tick(); + expect(result.sessionsWatched).toBe(1); + expect(result.findings[0].conditionId).toBe('stalled'); + expect(result.findings[0].outcome).toBe('observed'); + expect(writes).toEqual([]); + }); + + test('assist mode nudges a stalled session', async () => { + const { supervisor, writes } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], + autonomy: 'assist' + }); + + supervisor.quietTracker.observe('work1-claude', 8); + supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + + const result = await supervisor.tick(); + expect(result.findings[0].outcome).toBe('nudged'); + expect(writes.map((w) => w.data)).toEqual(['status? if you are blocked, say what on and stop.', '\r']); + }); + + test('a finding does not re-fire while it is cooling down', async () => { + const { supervisor, writes } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], + autonomy: 'assist' + }); + + const stall = () => { + supervisor.quietTracker.observe('work1-claude', 8); + supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + }; + + stall(); + await supervisor.tick(); + stall(); + const second = await supervisor.tick(); + + expect(second.findings[0].outcome).toBe('cooling-down'); + expect(writes).toHaveLength(2); + }); + + test('dry run reports what would happen without doing it', async () => { + const { supervisor, writes } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], + autonomy: 'autopilot' + }); + + supervisor.quietTracker.observe('work1-claude', 8); + supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + + const result = await supervisor.tick({ dryRun: true }); + expect(result.findings[0].outcome).toBe('dry-run'); + expect(writes).toEqual([]); + }); + + test('server terminals are not supervised', async () => { + const { supervisor } = harness({ + sessions: [fakeSession({ id: 'work1-server', type: 'server', status: 'busy' })] + }); + const result = await supervisor.tick(); + expect(result.sessionsWatched).toBe(0); + }); + + test('actions are written to the audit log', async () => { + const { supervisor } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], + autonomy: 'assist' + }); + + supervisor.quietTracker.observe('work1-claude', 8); + supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + await supervisor.tick(); + + const audit = fs.readFileSync(supervisor.auditPath(), 'utf8').trim().split('\n').map(JSON.parse); + expect(audit.at(-1)).toMatchObject({ event: 'finding', conditionId: 'stalled', outcome: 'nudged' }); + }); + + test('setAutonomy rejects unknown levels and records real changes', () => { + const { supervisor } = harness(); + expect(() => supervisor.setAutonomy('yolo')).toThrow(/Unknown autonomy level/); + expect(supervisor.setAutonomy('autopilot')).toBe('autopilot'); + + const audit = fs.readFileSync(supervisor.auditPath(), 'utf8').trim().split('\n').map(JSON.parse); + expect(audit.at(-1)).toMatchObject({ event: 'autonomy-changed', to: 'autopilot' }); + }); + + test('start refuses to run when autonomy is off', () => { + const { supervisor } = harness({ autonomy: 'off' }); + expect(supervisor.start()).toMatchObject({ running: false }); + supervisor.stop(); + }); + + test('the briefing reads as a sentence and leads with what is critical', async () => { + const { supervisor } = harness({ + sessions: [ + fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' }), + fakeSession({ id: 'work2-claude', status: 'busy', buffer: 'usage limit reached ∙ resets 3am' }) + ], + autonomy: 'assist' + }); + + for (const id of ['work1-claude', 'work2-claude']) { + supervisor.quietTracker.observe(id, 8); + supervisor.quietTracker.state.get(id).lastGrowthAt = Date.now() - 3_600_000; + } + await supervisor.tick(); + + const briefing = supervisor.getBriefing(); + expect(briefing.counts.critical).toBe(1); + expect(briefing.items[0].severity).toBe('critical'); + expect(briefing.spoken).toMatch(/needs you now/); + }); + + test('an empty fleet briefs as quiet rather than as an error', () => { + const { supervisor } = harness(); + supervisor.running = true; + expect(supervisor.getBriefing().spoken).toMatch(/quiet/); + }); +}); diff --git a/tests/unit/voiceCommandService.test.js b/tests/unit/voiceCommandService.test.js index 8621d2a7..ad8882bf 100644 --- a/tests/unit/voiceCommandService.test.js +++ b/tests/unit/voiceCommandService.test.js @@ -319,3 +319,58 @@ describe('VoiceCommandService parseCommand (rulesOnly)', () => { } }); }); + +describe('VoiceCommandService (free-form routing)', () => { + afterEach(() => { + voiceCommandService.setCommanderForwarder(null); + }); + + test('unmatched speech reaches the Commander instead of dead-ending', async () => { + const forwarded = []; + voiceCommandService.setCommanderForwarder(async (text) => { + forwarded.push(text); + return 'sent'; + }); + + const result = await voiceCommandService.processVoiceCommand( + 'what did work three actually change in the physics code' + ); + + expect(result.success).toBe(true); + expect(result.method).toBe('commander'); + expect(result.forwardedToCommander).toBe(true); + expect(forwarded).toEqual(['what did work three actually change in the physics code']); + }); + + test('a matched command still executes as a command, not as chat', async () => { + const forwarded = []; + voiceCommandService.setCommanderForwarder(async (text) => forwarded.push(text)); + + const result = await voiceCommandService.processVoiceCommand('enter focus mode'); + + expect(result.method).toBe('rules'); + expect(result.command).toBe('set-workflow-mode'); + expect(forwarded).toEqual([]); + }); + + test('with no Commander running the failure is reported, not swallowed', async () => { + const result = await voiceCommandService.processVoiceCommand('ramble ramble unmatched words here'); + expect(result.success).toBe(false); + }); + + test('a throwing forwarder degrades to a normal failure', async () => { + voiceCommandService.setCommanderForwarder(async () => { throw new Error('commander is down'); }); + + const result = await voiceCommandService.processVoiceCommand('another unmatched utterance entirely'); + expect(result.success).toBe(false); + expect(result.forward.reason).toBe('commander is down'); + }); + + test('forwarding can be turned off per call', async () => { + voiceCommandService.setCommanderForwarder(async () => true); + + const result = await voiceCommandService.processVoiceCommand('yet more unmatched words', { forwardUnmatched: false }); + expect(result.success).toBe(false); + expect(result.forwardedToCommander).toBeUndefined(); + }); +}); From 79c7953c74ba11a8667b823aa283805796628a86 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 09:54:55 +1000 Subject: [PATCH 04/69] fix(atlas): CLI crashed with EPIPE when piped to head `atlas list | head` closes stdout early; that is a normal end for a CLI, not an unhandled error event. --- scripts/atlas.js | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/scripts/atlas.js b/scripts/atlas.js index 8b027f8f..d0287c94 100755 --- a/scripts/atlas.js +++ b/scripts/atlas.js @@ -15,6 +15,12 @@ const { formatDecisions } = require('../server/atlas/atlasCompiler'); const atlas = RepoAtlasService.getInstance(); +// `atlas list | head` closes stdout early; that is a normal end, not a crash. +process.stdout.on('error', (error) => { + if (error?.code === 'EPIPE') process.exit(0); + throw error; +}); + function parseArgs(argv) { const positionals = []; const flags = {}; From 841fdc714549ce410448dd86b71109a1b3773199 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 09:59:38 +1000 Subject: [PATCH 05/69] docs: record what shipped and what is left in the design doc MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The plan section read as intent; it is now the record — what landed where, how it was verified, why only two atlas entries were seeded, and the six things still open. --- ...OMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md | 52 ++++++++++++++++--- 1 file changed, 44 insertions(+), 8 deletions(-) diff --git a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md index ebb4ca44..e0cdccc1 100644 --- a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md +++ b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md @@ -195,11 +195,47 @@ Surfaces shipped: standalone CLI (`scripts/atlas.js`, no server required — sym --- -## 5. Order of work - -1. **Repo Atlas** — self-contained, immediately useful, no runtime risk. *(this branch)* -2. **Supervisor** in `observe` mode — accumulate findings, prove the rules are right before granting any autonomy. *(this branch)* -3. **Voice out + free-form routing** — makes 1 and 2 conversational. *(this branch)* -4. Promote supervisor to `assist` after reading a week of findings. *(you, later)* -5. Reach: Discord/mobile push for escalations. *(follow-up)* -6. Atlas write-back: agents propose highlights from work they just did, you approve. *(follow-up — this is what makes it stay current instead of rotting)* +## 5. What shipped (PR #1029) + +All three, on `feature/autopilot-voice-and-repo-atlas`. 709 unit tests green (was 652). + +| Piece | Where | State | +|---|---|---| +| Supervisor loop | `server/supervisorService.js`, `server/supervisor/*` | Running, autonomy `observe` | +| Condition table | `config/supervisor-rules.json` | 8 conditions, none reaching `act` | +| Speech out | `server/speechService.js`, `client/speech-output.js` | Browser backend active | +| Free-form voice | `server/voiceCommandService.js` (`setCommanderForwarder`) | Wired to Commander | +| Repo Atlas | `server/repoAtlasService.js`, `server/atlas/*`, `scripts/atlas.js` | 233 repos mapped, 25 cloned | +| APIs | `server/routes/{supervisor,speech,atlas}Routes.js` | Policy-gated, live-verified | +| Commander docs | `docs/COMMANDER_CLAUDE.md` | All three surfaces documented | +| CLI + skill | `~/.claude/scripts/atlas.sh`, `~/.claude/skills/repo-atlas/` | Installed, on PATH | + +Verified live rather than only in tests: supervisor status/tick/briefing, speech status/say, atlas +status/find/digest/compile, and the autonomy-level guard, all against a running server on a scratch +port. Sharing checked end to end by compiling `core-team` and `contractors` from one registry and +confirming differential redaction, no private entries, and no local paths in the output. + +### Seeded atlas state + +Only highlights with actual evidence behind them were recorded — `box2d-luau` (physics, testing) and +`roblox-mechanics-encyclopedia` (architecture), all sourced from the repos' own descriptions. **No +quality scores were invented for the other 230.** The map is built; the judgement is deliberately +left to you, because a fabricated 4/5 is worse than a blank field — it sends agents somewhere on a +false promise. + +## 6. What is left + +1. **Read a week of findings, then promote the supervisor to `assist`.** This is the whole reason + `observe` is the shipped default. `~/.agent-workspace/logs/supervisor-audit.jsonl`. +2. **Curate the atlas as you go** — `atlas note --topic X --quality N --notes "..."`. The + skill already tells agents to record what they learn, so this should accumulate rather than + needing a curation session. +3. **Tag repos into audiences** before sharing anything: `atlas set --visibility team --groups + core-team`, then `atlas compile core-team --dry-run --explain` and read every line before dropping + the dry-run flag. +4. **Reach** — Discord/mobile push for escalations, borrowing OpenClaw's one genuinely better idea. + `discordIntegrationService` is already half of it. +5. **Atlas write-back** — agents propose highlights from work they just finished, you approve. This + is what keeps the map current instead of letting it rot into another stale doc. +6. **Atlas in the UI** — findings and the digest currently have APIs but no panel. The Commander + status strip proposed in `PLANS/2026-07-15/MULTI_COMMANDER_FEASIBILITY.md` is the natural home. From 47e7ed6d0d7f4295f436bd06cccbbf096f355d06 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:20:19 +1000 Subject: [PATCH 06/69] feat(supervisor): fix it yourself, interrupt last MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Correcting the shipped design. The previous ladder ended in 'tell the human', which is backwards — being narrated at about things the system could have fixed is worse than not having the system. The ordering is now the whole point: fix it myself -> hand it to the Commander -> (only then) interrupt - Every shipped condition declares how to repair itself (asserted in tests). A stall gets nudged, a dead agent gets relaunched, a usage limit parses its own reset time and schedules a 'continue' — that last one used to be a critical alert and is now an info-level non-event, because waiting is not a problem you need to know about. - A finding that has not exhausted its repair attempts is not even eligible to reach a human. Only after N failed self-heals does it become escalatable. - When rules run out of ideas, the problem goes to the Commander with a written brief (label, branch, tier, ticket, output tail) before it goes to you. It is a full agent with the whole API and only costs tokens when something is wrong. - Urgency is weighted by task tier: the same stall scores 96 on T1 focus work and 24 on T4 background work. Background work now structurally cannot pull you out of flow. - An interruption budget (2/hour, 15min apart, optional quiet hours) gates what is left; anything refused goes to a batched digest instead of being dropped. A high enough score overrides quiet hours and the hourly cap, but nothing overrides the per-finding guard — no drumbeat about the same problem. - Problems that heal are removed from the digest and forgotten. Solved problems are never mentioned. Default autonomy is now 'autopilot' (SUPERVISOR_AUTONOMY overrides). The safety invariants are what make that defensible: named handlers only, fail-closed permission approval that refuses credentials/force-push/merge, full audit. Also fixes a Number(null) === 0 bug class in urgency config parsing that made every unset numeric option silently zero. 752 unit tests green. --- config/supervisor-rules.json | 107 +++++--- server/index.js | 28 +- server/routes/supervisorRoutes.js | 15 ++ server/supervisor/supervisorActions.js | 214 ++++++++++----- server/supervisor/supervisorRules.js | 120 ++++++--- server/supervisor/supervisorUrgency.js | 234 +++++++++++++++++ server/supervisorService.js | 347 ++++++++++++++++++------- tests/unit/supervisorActions.test.js | 303 ++++++++++++--------- tests/unit/supervisorRules.test.js | 103 +++++--- tests/unit/supervisorService.test.js | 247 ++++++++++++------ tests/unit/supervisorUrgency.test.js | 206 +++++++++++++++ 11 files changed, 1449 insertions(+), 475 deletions(-) create mode 100644 server/supervisor/supervisorUrgency.js create mode 100644 tests/unit/supervisorUrgency.test.js diff --git a/config/supervisor-rules.json b/config/supervisor-rules.json index 6fda6f86..8f914a1a 100644 --- a/config/supervisor-rules.json +++ b/config/supervisor-rules.json @@ -1,38 +1,58 @@ { - "$comment": "Condition table for the fleet supervisor. Evaluated every tick against zero-token signals (PTY tail, status, quiet time, git state). Override per-machine with ~/.agent-workspace/supervisor-rules.json.", - "schemaVersion": 1, + "$comment": "Condition table for JARVIS, the fleet supervisor. Evaluated every tick against zero-token signals (PTY tail, status, quiet time, git state). Override per-machine with ~/.agent-workspace/supervisor-rules.json.", + "schemaVersion": 2, - "$comment_autonomy": "off = nothing runs. observe = findings recorded, no side effects (start here). assist = may notify and nudge. autopilot = may also run built-in act handlers.", - "autonomy": "observe", + "$comment_autonomy": "off = nothing runs. observe = record only. assist = may resolve things itself. autopilot = may also delegate to the Commander. Interrupting a human is gated separately by interruption policy, NOT by autonomy — the point is to act, not to narrate.", + "autonomy": "autopilot", "tickSeconds": 30, "maxFindingsRetained": 500, + "$comment_interruption": "The human is the most expensive resource in the system and every interruption costs a context switch. A finding only reaches you if it scores above the threshold AND fits the budget; everything else self-heals silently or waits for the digest.", + "interruption": { + "threshold": 60, + "alwaysInterruptAbove": 90, + "maxPerHour": 2, + "minSecondsBetween": 900, + "digestIntervalMinutes": 120, + "quietHours": { "enabled": false, "startHour": 22, "endHour": 7 }, + "$comment_tiers": "Tier weights come from the task record. T1 is what you are actually working on; T3/T4 is background work that should never pull you out of flow — fix it or park it.", + "tierWeights": { "1": 1.5, "2": 1.15, "3": 0.6, "4": 0.35, "none": 0.8 }, + "severityBase": { "info": 10, "warn": 40, "critical": 80 } + }, + "safety": { - "$comment": "Act handlers are named functions in server/supervisor/supervisorActions.js. Rules cannot inject shell commands — they can only select a handler from this list.", - "allowedActHandlers": ["answer-permission", "relaunch-agent", "open-pull-request"], - "$comment_permission": "An auto-answered permission prompt must match one of these AND none of the deny patterns. Anything else escalates to you.", + "$comment": "Resolve/delegate handlers are named functions in server/supervisor/supervisorActions.js. Rules select a handler; they cannot inject shell.", + "allowedHandlers": ["nudge", "answer-permission", "relaunch-agent", "schedule-resume", "commit-and-push", "open-pull-request", "delegate-to-commander"], + "$comment_permission": "An auto-answered permission prompt must match one of these AND none of the deny patterns. Anything else is escalated, never guessed.", "permissionAllowPatterns": [ - "\\b(Read|Glob|Grep|NotebookRead|WebFetch|WebSearch)\\b", - "\\bgit (status|diff|log|show|branch|fetch)\\b", - "\\b(npm|pnpm|yarn) (test|run test|run lint|run typecheck)\\b" + "\\b(Read|Glob|Grep|NotebookRead|WebFetch|WebSearch|TodoWrite)\\b", + "\\b(Edit|Write|Update)\\([^)]*\\)", + "\\bgit (status|diff|log|show|branch|fetch|add|commit)\\b", + "\\b(npm|pnpm|yarn) (test|run test|run lint|run typecheck|run build)\\b", + "\\bjest\\b", "\\bplaywright\\b", "\\bpytest\\b", "\\bcargo (test|check|clippy)\\b" ], "permissionDenyPatterns": [ - "\\brm\\b", "\\bsudo\\b", "\\bgit (push|reset|clean|checkout)\\b", - "\\bgh (pr merge|release|repo delete)\\b", "\\bDELETE\\b", "\\bDROP\\b", - "\\bcurl\\b.*\\|", "\\bchmod\\b", "\\bkill\\b", "> */" + "\\brm\\s+-[rf]", "\\bsudo\\b", "\\bgit (push --force|reset --hard|clean)\\b", + "\\bgh (pr merge|release|repo delete|repo edit)\\b", "\\bDROP\\b", "\\bTRUNCATE\\b", + "\\bDELETE FROM\\b", "\\bcurl\\b[^\\n]*\\|\\s*(sh|bash)", "\\bchmod\\s+777\\b", + "\\bkill(all)?\\b", "\\bnpm publish\\b", "\\bdd\\s+if=", "\\bmkfs\\b", + "~/\\.(ssh|aws|config|codex|claude)\\b", "\\.env\\b" ] }, + "$comment_conditions": "Ordered by urgency — the first match per session wins. `resolve` is what JARVIS does about it without asking. `escalateAfterAttempts` is how many failed self-heals before it is allowed to reach you at all.", "conditions": [ { "id": "awaiting-permission", "label": "Waiting on a permission prompt", "severity": "warn", - "rung": "notify", - "cooldownSeconds": 600, + "cooldownSeconds": 120, + "escalateAfterAttempts": 1, + "urgency": { "blocksWork": true }, + "resolve": { "handler": "answer-permission" }, "when": { "status": ["waiting"], - "minQuietSeconds": 120, + "minQuietSeconds": 45, "tailMatches": [ "Do you want to (proceed|make this edit|create)", "❯\\s*1\\.\\s*Yes", @@ -40,14 +60,16 @@ "\\[y/N\\]" ] }, - "advice": "The agent has been sitting on a permission prompt. Answer it or let autopilot approve read-only prompts." + "advice": "An agent is blocked on a permission prompt that could not be auto-approved safely." }, { "id": "usage-limit-reached", "label": "Usage limit reached", - "severity": "critical", - "rung": "notify", + "severity": "info", "cooldownSeconds": 3600, + "escalateAfterAttempts": 99, + "urgency": { "base": 5 }, + "resolve": { "handler": "schedule-resume" }, "when": { "tailMatches": [ "\\d+-hour limit reached", @@ -56,82 +78,87 @@ "rate.?limit(ed)? .*(retry|reset)" ] }, - "advice": "This session is blocked until the window resets. Nothing to do but wait — resume it after the reset time in the banner." + "advice": "Blocked until the window resets. Resume is scheduled automatically — there is nothing for you to do, so this never interrupts." }, { "id": "error-loop", "label": "Repeating the same error", "severity": "critical", - "rung": "notify", - "cooldownSeconds": 1800, + "cooldownSeconds": 900, + "escalateAfterAttempts": 1, + "urgency": { "blocksWork": true }, + "resolve": { "handler": "delegate-to-commander" }, "when": { "status": ["busy", "idle"], "repeatedTailLine": 4, "tailMatches": ["(?i)\\b(error|failed|exception|traceback|cannot find)\\b"] }, - "advice": "The same error line keeps coming back — the agent is looping. This one needs a human, never automation." + "advice": "The same error keeps recurring — the agent is looping and cannot see it." }, { "id": "stalled", "label": "Busy but silent", "severity": "warn", - "rung": "nudge", - "cooldownSeconds": 900, - "nudgeText": "status? if you are blocked, say what on and stop.", + "cooldownSeconds": 600, + "escalateAfterAttempts": 3, + "resolve": { "handler": "nudge", "text": "status? If you are blocked, say what on, then resolve it yourself if you can." }, "when": { "status": ["busy"], "minQuietSeconds": 900, "tailNotMatches": ["limit reached", "Do you want to (proceed|make this edit)"] }, - "advice": "Marked busy but nothing has come out for 15 minutes." + "advice": "Marked busy with no output for 15 minutes and did not respond to nudges." }, { "id": "agent-exited", "label": "Agent exited, shell left behind", "severity": "warn", - "rung": "notify", - "cooldownSeconds": 900, + "cooldownSeconds": 600, + "escalateAfterAttempts": 2, + "resolve": { "handler": "relaunch-agent" }, "when": { "status": ["idle"], "agentPresent": false, - "minQuietSeconds": 300 + "minQuietSeconds": 180 }, - "advice": "The agent CLI is gone and the terminal dropped back to a shell. Relaunch it or reuse the worktree." + "advice": "The agent CLI is gone and could not be relaunched." }, { "id": "unpushed-work", "label": "Finished with unpushed commits", "severity": "info", - "rung": "nudge", - "cooldownSeconds": 1800, - "nudgeText": "You have local commits that are not pushed. Push the branch and open a PR.", + "cooldownSeconds": 1200, + "escalateAfterAttempts": 3, + "resolve": { "handler": "nudge", "text": "You have local commits that are not pushed. Push the branch and open a PR, then say what you shipped." }, "when": { "status": ["idle"], "minQuietSeconds": 300, "git": { "aheadMin": 1 } }, - "advice": "Work is done on disk but has not left the machine." + "advice": "Finished work has not left the machine and nudges did not shift it." }, { "id": "uncommitted-work", "label": "Idle with uncommitted changes", "severity": "info", - "rung": "nudge", - "cooldownSeconds": 3600, - "nudgeText": "You have uncommitted changes. Commit them with a descriptive message, or say why they should not be committed.", + "cooldownSeconds": 1800, + "escalateAfterAttempts": 3, + "resolve": { "handler": "nudge", "text": "You have uncommitted changes. Commit them with a descriptive message, or say why they should not be committed." }, "when": { "status": ["idle"], "minQuietSeconds": 900, "git": { "dirty": true, "aheadMax": 0 } }, - "advice": "Edits are sitting in the working tree with nothing recorded." + "advice": "Edits sitting in the working tree with nothing recorded." }, { "id": "idle-capacity", "label": "Idle worktree, nothing in flight", "severity": "info", - "rung": "observe", "cooldownSeconds": 3600, + "escalateAfterAttempts": 99, + "urgency": { "base": 5 }, + "resolve": { "handler": "observe" }, "when": { "status": ["idle"], "minQuietSeconds": 1800, diff --git a/server/index.js b/server/index.js index a4ec61ac..cab50594 100644 --- a/server/index.js +++ b/server/index.js @@ -433,6 +433,15 @@ threadService.init({ workspaceManager, sessionManager }); intentHaikuService.setSessionManager(sessionManager); serviceStackRuntimeService.init({ workspaceManager, sessionManager, configPromoterService, io }); auditExportService.init({ activityFeed, schedulerService, userSettingsService }); +// Two writes: agent CLIs treat "text\r" in one chunk as a bracketed paste. +const sendToCommander = async (text) => { + if (!commanderService?.sendInput) return false; + if (commanderService.sendInput(text) === false) return false; + await new Promise((resolve) => setTimeout(resolve, 300)); + commanderService.sendInput('\r'); + return true; +}; + supervisorService.init({ sessionManager, gitHelper, @@ -441,12 +450,13 @@ supervisorService.init({ taskRecordService, activityFeed, notificationService, - speechService + speechService, + // When rules cannot fix something, the Commander gets a written problem brief + // before you do — it is a full agent with the whole API, and it only costs + // tokens when something is actually wrong. + commanderSender: sendToCommander }); -// Shipped default autonomy is `observe`: findings accumulate, nothing is -// touched. That is safe to leave running, and reading a week of it is how you -// decide whether to grant this thing any real autonomy. if (String(process.env.SUPERVISOR_AUTOSTART || 'true').toLowerCase() !== 'false') { const started = supervisorService.start(); logger.info('Supervisor', started); @@ -454,15 +464,7 @@ if (String(process.env.SUPERVISOR_AUTOSTART || 'true').toLowerCase() !== 'false' // Speech that no rule matched is still useful: hand the raw words to the // active Commander so the fallback is an agent, not an error. -voiceCommandService.setCommanderForwarder(async (transcript) => { - if (!commanderService?.sendInput) return false; - // Two writes: agent CLIs treat "text\r" in one chunk as a bracketed paste. - const wrote = commanderService.sendInput(transcript); - if (wrote === false) return false; - await new Promise((resolve) => setTimeout(resolve, 300)); - commanderService.sendInput('\r'); - return true; -}); +voiceCommandService.setCommanderForwarder(sendToCommander); const loadPlugins = async () => { const status = await pluginLoaderService.loadAll({ diff --git a/server/routes/supervisorRoutes.js b/server/routes/supervisorRoutes.js index 918f2386..f894dc94 100644 --- a/server/routes/supervisorRoutes.js +++ b/server/routes/supervisorRoutes.js @@ -58,6 +58,21 @@ function createSupervisorRoutes({ supervisorService, logger = console, requireRe res.json({ ok: true, source: rules.source, conditionCount: rules.conditions.length, autonomy: rules.autonomy }); })); + router.get('/digest', requireRead, handle('digest', (req, res) => { + res.json({ ok: true, pending: supervisorService.digest.pending(), budget: supervisorService.budget.getState() }); + })); + + /** + * Deliver the batch now — "catch me up" — instead of waiting for the timer. + */ + router.post('/digest/deliver', requireWrite, handle('deliver digest', (req, res) => { + res.json({ ok: true, delivered: supervisorService.deliverDigest() }); + })); + + router.post('/interruption-policy', requireWrite, handle('set interruption policy', (req, res) => { + res.json({ ok: true, interruption: supervisorService.setInterruptionPolicy(req.body || {}) }); + })); + return router; } diff --git a/server/supervisor/supervisorActions.js b/server/supervisor/supervisorActions.js index 1fcdc08c..e3855395 100644 --- a/server/supervisor/supervisorActions.js +++ b/server/supervisor/supervisorActions.js @@ -9,9 +9,9 @@ const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms)); */ async function submitText(sessionManager, sessionId, text, { delayMs = SUBMIT_DELAY_MS } = {}) { const wrote = sessionManager?.writeToSession?.(sessionId, text); - if (!wrote) return false; + if (wrote === false) return false; await sleep(delayMs); - return Boolean(sessionManager?.writeToSession?.(sessionId, '\r')); + return sessionManager?.writeToSession?.(sessionId, '\r') !== false; } /** @@ -21,8 +21,7 @@ async function submitText(sessionManager, sessionId, text, { delayMs = SUBMIT_DE * match, or an unreadable prompt all mean "ask the human". */ function classifyPermissionPrompt(tail, safety) { - const text = String(tail || ''); - const window = text.slice(-1200); + const window = String(tail || '').slice(-1200); for (const pattern of safety?.permissionDenyPatterns || []) { if (pattern.test(window)) { @@ -38,10 +37,58 @@ function classifyPermissionPrompt(tail, safety) { return { safe: true, reason: `matched allow pattern ${allowed}` }; } -function buildActHandlers({ sessionManager, gitHelper, agentManager, logger = console }) { +/** + * Usage-limit banners carry their own reset time. Parsing it is what turns + * "blocked for hours" into something that resumes itself. + */ +function parseResetTime(tail, now = new Date()) { + const match = String(tail || '').match(/resets?\s+(?:at\s+)?(\d{1,2})(?::(\d{2}))?\s*(am|pm)?/i); + if (!match) return null; + + let hour = Number(match[1]); + const minute = Number(match[2] || 0); + const meridiem = (match[3] || '').toLowerCase(); + if (meridiem === 'pm' && hour < 12) hour += 12; + if (meridiem === 'am' && hour === 12) hour = 0; + if (!Number.isFinite(hour) || hour > 23) return null; + + const resetAt = new Date(now); + resetAt.setHours(hour, minute, 0, 0); + if (resetAt <= now) resetAt.setDate(resetAt.getDate() + 1); + return resetAt; +} + +function buildProblemBrief(finding, signal) { + const tail = String(signal?.tail || '').split('\n').slice(-25).join('\n'); + return [ + `[JARVIS] ${finding.label} on ${finding.worktreeId || finding.sessionId}.`, + finding.advice, + finding.branch ? `Branch: ${finding.branch}` : '', + finding.ticketTitle ? `Task: ${finding.ticketTitle}` : '', + finding.tier ? `Tier: T${finding.tier}` : '', + `Session: ${finding.sessionId} (${finding.status}, quiet ${finding.quietSeconds}s)`, + '', + 'Last output:', + tail, + '', + 'Diagnose and fix it yourself using the orchestrator API. Do not ask me unless you are genuinely blocked on a decision only I can make.' + ].filter(Boolean).join('\n'); +} + +function buildHandlers({ sessionManager, gitHelper, agentManager, commanderSender, scheduleResume, logger = console }) { return { /** - * Approve a permission prompt only when it is unambiguously read-only. + * Type an instruction into the session and let the agent fix its own problem. + */ + nudge: async ({ finding, plan }) => { + const text = plan.text || finding.advice; + if (!text) return { performed: false, detail: 'no nudge text configured' }; + const submitted = await submitText(sessionManager, finding.sessionId, text); + return { performed: submitted, detail: submitted ? 'nudged the session' : 'write failed' }; + }, + + /** + * Approve a permission prompt only when it is unambiguously safe. */ 'answer-permission': async ({ finding, signal, rules }) => { const verdict = classifyPermissionPrompt(signal?.tail, rules?.safety); @@ -71,8 +118,32 @@ function buildActHandlers({ sessionManager, gitHelper, agentManager, logger = co }, /** - * Outward-facing: creates a real PR. No shipped rule selects this handler — - * it exists for opt-in autopilot configurations. + * A usage limit is not a problem to report — it is a wait to schedule. + */ + 'schedule-resume': async ({ finding, signal }) => { + const resetAt = parseResetTime(signal?.tail); + if (!resetAt) { + return { performed: true, detail: 'usage limit hit; no reset time in the banner, will retry on the next tick' }; + } + try { + scheduleResume?.({ sessionId: finding.sessionId, at: resetAt }); + } catch (error) { + logger.warn?.('Could not schedule a resume', { error: error.message }); + } + return { performed: true, detail: `resume queued for ${resetAt.toISOString()}`, resumeAt: resetAt.toISOString() }; + }, + + 'commit-and-push': async ({ finding }) => { + const submitted = await submitText( + sessionManager, + finding.sessionId, + 'Commit your outstanding work with a descriptive message and push the branch.' + ); + return { performed: submitted, detail: submitted ? 'asked the agent to commit and push' : 'write failed' }; + }, + + /** + * Outward-facing: creates a real PR. Opt-in only. */ 'open-pull-request': async ({ finding, signal }) => { if (!signal?.cwd || !signal?.branch) { @@ -82,101 +153,98 @@ function buildActHandlers({ sessionManager, gitHelper, agentManager, logger = co await gitHelper?.execGh?.(['pr', 'create', '--fill', '--head', signal.branch], { cwd: signal.cwd, timeout: 30_000 }); return { performed: true, detail: `opened a PR for ${signal.branch}` }; } catch (error) { - logger.warn('Supervisor could not open a pull request', { sessionId: finding.sessionId, error: error.message }); return { performed: false, escalate: true, detail: `gh pr create failed: ${error.message}` }; } + }, + + /** + * The tier that exists so you do not have to be the next step. + * + * Rules are cheap and dumb; the Commander is a full agent with the whole + * orchestrator API. When a rule cannot fix something, handing it a written + * problem brief is strictly better than handing it to you — and it only + * spends tokens when something is actually wrong. + */ + 'delegate-to-commander': async ({ finding, signal }) => { + if (!commanderSender) { + return { performed: false, escalate: true, detail: 'no Commander available to delegate to' }; + } + try { + const delivered = await commanderSender(buildProblemBrief(finding, signal)); + return delivered + ? { performed: true, detail: 'handed the problem to the Commander' } + : { performed: false, escalate: true, detail: 'Commander did not accept the brief' }; + } catch (error) { + return { performed: false, escalate: true, detail: `delegation failed: ${error.message}` }; + } } }; } /** - * Executes one finding at its effective rung. + * Carries out the planned intent for one finding. * - * Rungs are cumulative in intent but not in effect: a `nudge` notifies and - * types, an `act` notifies and runs its handler. `observe` deliberately does - * nothing outward, which is what makes it safe to leave running for a week. + * `interrupt` deliberately does not appear here — reaching a human is the + * supervisor's decision to make with the interruption budget, not an action a + * handler performs. */ function createExecutor({ sessionManager, gitHelper, agentManager, + commanderSender, + scheduleResume, activityFeed, - notificationService, - speechService, logger = console } = {}) { - const actHandlers = buildActHandlers({ sessionManager, gitHelper, agentManager, logger }); + const handlers = buildHandlers({ sessionManager, gitHelper, agentManager, commanderSender, scheduleResume, logger }); - const announce = (finding, detail) => { + const record = (finding, plan, result) => { try { - activityFeed?.track?.('supervisor.finding', { + activityFeed?.track?.('supervisor.action', { sessionId: finding.sessionId, conditionId: finding.conditionId, - severity: finding.severity, - label: finding.label, - rung: finding.rung, - detail + intent: plan.intent, + handler: plan.handler || null, + performed: result.performed, + detail: result.detail }); } catch (error) { - logger.warn('Supervisor could not record activity', { error: error.message }); - } - - try { - notificationService?.notify?.( - finding.sessionId, - finding.severity === 'critical' ? 'error' : 'warning', - `${finding.label} — ${finding.worktreeId || finding.sessionId}`, - { conditionId: finding.conditionId, advice: finding.advice } - ); - } catch (error) { - logger.warn('Supervisor could not send a notification', { error: error.message }); - } - - if (finding.severity === 'critical') { - try { - speechService?.speak?.(`${finding.label} on ${finding.worktreeId || finding.sessionId}`, { priority: 'high' }); - } catch (error) { - logger.warn('Supervisor could not speak', { error: error.message }); - } + logger.warn?.('Supervisor could not record activity', { error: error.message }); } }; - return async function execute({ finding, signal, rules }) { - if (!finding.rung || finding.rung === 'observe') { - return { ...finding, performed: false, outcome: 'observed' }; + return async function execute({ finding, plan, signal, rules }) { + if (plan.intent === 'none' || plan.intent === 'observe') { + return { performed: false, outcome: 'observed', detail: plan.reason }; } - announce(finding, finding.advice); - - if (finding.rung === 'notify') { - return { ...finding, performed: true, outcome: 'notified' }; + const handlerId = plan.handler; + if (!handlerId) return { performed: false, outcome: 'skipped', detail: 'no handler configured' }; + if (!(rules?.safety?.allowedHandlers || []).includes(handlerId)) { + return { performed: false, outcome: 'blocked', detail: `handler "${handlerId}" is not in allowedHandlers` }; } + const handler = handlers[handlerId]; + if (!handler) return { performed: false, outcome: 'blocked', detail: `unknown handler "${handlerId}"` }; - if (finding.rung === 'nudge') { - const text = finding.nudgeText; - if (!text) return { ...finding, performed: false, outcome: 'nudge-skipped', detail: 'no nudge text configured' }; - const submitted = await submitText(sessionManager, finding.sessionId, text); - return { ...finding, performed: submitted, outcome: submitted ? 'nudged' : 'nudge-failed' }; - } + const result = await handler({ finding, plan, signal, rules }); + record(finding, plan, result); - const handlerId = finding.actHandler; - if (!handlerId) { - return { ...finding, performed: false, outcome: 'act-skipped', detail: 'no act handler configured' }; - } - if (!(rules?.safety?.allowedActHandlers || []).includes(handlerId)) { - return { ...finding, performed: false, outcome: 'act-blocked', detail: `handler "${handlerId}" is not in allowedActHandlers` }; - } - const handler = actHandlers[handlerId]; - if (!handler) { - return { ...finding, performed: false, outcome: 'act-blocked', detail: `unknown handler "${handlerId}"` }; + if (!result.performed) { + return { + performed: false, + outcome: result.escalate ? 'repair-failed' : 'skipped', + detail: result.detail, + escalate: result.escalate === true + }; } - const result = await handler({ finding, signal, rules }); - if (!result.performed && result.escalate) { - announce({ ...finding, severity: 'critical' }, result.detail); - return { ...finding, performed: false, outcome: 'escalated', detail: result.detail }; - } - return { ...finding, performed: result.performed, outcome: result.performed ? 'acted' : 'act-failed', detail: result.detail }; + return { + performed: true, + outcome: plan.intent === 'delegate' ? 'delegated' : 'resolved', + detail: result.detail, + resumeAt: result.resumeAt + }; }; } @@ -184,6 +252,8 @@ module.exports = { SUBMIT_DELAY_MS, submitText, classifyPermissionPrompt, - buildActHandlers, + parseResetTime, + buildProblemBrief, + buildHandlers, createExecutor }; diff --git a/server/supervisor/supervisorRules.js b/server/supervisor/supervisorRules.js index f88afede..5398bc3c 100644 --- a/server/supervisor/supervisorRules.js +++ b/server/supervisor/supervisorRules.js @@ -2,19 +2,23 @@ const fs = require('fs'); const path = require('path'); const { getAgentWorkspaceDir } = require('../utils/pathUtils'); +const { normalizeInterruptionPolicy } = require('./supervisorUrgency'); -const RUNGS = ['observe', 'notify', 'nudge', 'act']; const SEVERITIES = ['info', 'warn', 'critical']; const AUTONOMY_LEVELS = ['off', 'observe', 'assist', 'autopilot']; -// How far each autonomy level is allowed to climb the ladder. -const AUTONOMY_CEILING = { - off: null, - observe: 'observe', - assist: 'nudge', - autopilot: 'act' +// What each autonomy level is allowed to *do about* a finding. Interrupting a +// human is governed separately by the interruption policy — autonomy is about +// how much the supervisor may fix, not how much it may say. +const AUTONOMY_CAPABILITIES = { + off: { resolve: false, delegate: false }, + observe: { resolve: false, delegate: false }, + assist: { resolve: true, delegate: false }, + autopilot: { resolve: true, delegate: true } }; +const DELEGATE_HANDLERS = new Set(['delegate-to-commander']); + const DEFAULT_RULES_PATH = path.join(__dirname, '..', '..', 'config', 'supervisor-rules.json'); function overrideRulesPath() { @@ -33,22 +37,37 @@ function compilePatterns(patterns) { return out; } +function normalizeResolve(raw) { + const handler = String(raw?.handler || '').trim(); + if (!handler) return null; + return { + handler, + text: String(raw?.text || '').trim(), + delegate: DELEGATE_HANDLERS.has(handler) + }; +} + function normalizeCondition(raw) { const id = String(raw?.id || '').trim(); if (!id) return null; const when = raw?.when || {}; - const rung = RUNGS.includes(raw?.rung) ? raw.rung : 'observe'; + const urgency = raw?.urgency || {}; return { id, label: String(raw?.label || id), severity: SEVERITIES.includes(raw?.severity) ? raw.severity : 'info', - rung, cooldownSeconds: Math.max(0, Number(raw?.cooldownSeconds) || 0), + // How many failed self-heals before this is allowed to reach a human at all. + // A high number means "never bother me about this, just keep handling it". + escalateAfterAttempts: Math.max(0, Number(raw?.escalateAfterAttempts ?? 2)), advice: String(raw?.advice || ''), - nudgeText: String(raw?.nudgeText || '').trim(), - actHandler: String(raw?.actHandler || '').trim(), + resolve: normalizeResolve(raw?.resolve), + urgency: { + base: Number.isFinite(Number(urgency.base)) ? Number(urgency.base) : null, + blocksWork: urgency.blocksWork === true + }, when: { status: (Array.isArray(when.status) ? when.status : []).map((s) => String(s).toLowerCase()), types: (Array.isArray(when.types) ? when.types : []).map((s) => String(s).toLowerCase()), @@ -84,17 +103,24 @@ function readJson(filePath) { */ function loadRules({ rulesPath = null } = {}) { const override = rulesPath || overrideRulesPath(); - const raw = readJson(override) || readJson(DEFAULT_RULES_PATH) || {}; - const source = readJson(override) ? override : DEFAULT_RULES_PATH; + const overrideRaw = readJson(override); + const raw = overrideRaw || readJson(DEFAULT_RULES_PATH) || {}; + const source = overrideRaw ? override : DEFAULT_RULES_PATH; const safety = raw.safety || {}; + const envAutonomy = String(process.env.SUPERVISOR_AUTONOMY || '').trim().toLowerCase(); + const autonomy = AUTONOMY_LEVELS.includes(envAutonomy) + ? envAutonomy + : (AUTONOMY_LEVELS.includes(raw.autonomy) ? raw.autonomy : 'assist'); + return { source, - autonomy: AUTONOMY_LEVELS.includes(raw.autonomy) ? raw.autonomy : 'observe', + autonomy, tickSeconds: Math.max(5, Number(raw.tickSeconds) || 30), maxFindingsRetained: Math.max(20, Number(raw.maxFindingsRetained) || 500), + interruption: normalizeInterruptionPolicy(raw.interruption), safety: { - allowedActHandlers: Array.isArray(safety.allowedActHandlers) ? safety.allowedActHandlers.map(String) : [], + allowedHandlers: Array.isArray(safety.allowedHandlers) ? safety.allowedHandlers.map(String) : [], permissionAllowPatterns: compilePatterns(safety.permissionAllowPatterns), permissionDenyPatterns: compilePatterns(safety.permissionDenyPatterns) }, @@ -132,22 +158,52 @@ function matches(condition, signal) { return true; } -function rungIndex(rung) { - const index = RUNGS.indexOf(rung); - return index === -1 ? 0 : index; +function capabilities(autonomy) { + return AUTONOMY_CAPABILITIES[autonomy] || AUTONOMY_CAPABILITIES.observe; } /** - * The rung a finding may actually reach, given how much autonomy it has been - * granted. Findings above the ceiling are recorded, not acted on. + * What the supervisor intends to do about a finding, before the interruption + * policy gets a say. + * + * The order matters and encodes the whole philosophy: try to fix it, then try + * to have something smarter fix it, and only then consider spending a human's + * attention. A condition that has not yet exhausted its repair attempts is not + * eligible to interrupt at all. */ -function effectiveRung(conditionRung, autonomy) { - const ceiling = AUTONOMY_CEILING[autonomy]; - if (!ceiling) return null; - return rungIndex(conditionRung) <= rungIndex(ceiling) ? conditionRung : ceiling; +function planAction(condition, { autonomy, attempts = 0 }) { + const can = capabilities(autonomy); + const resolve = condition.resolve; + const exhausted = attempts >= condition.escalateAfterAttempts; + + if (autonomy === 'off') return { intent: 'none', reason: 'autonomy off' }; + + if (resolve && resolve.handler === 'observe') { + return { intent: 'observe', reason: 'condition is informational only' }; + } + + if (resolve && !exhausted) { + if (resolve.delegate) { + if (can.delegate) return { intent: 'delegate', handler: resolve.handler, reason: 'handing the problem to the Commander' }; + if (can.resolve) return { intent: 'observe', reason: 'delegation needs autopilot' }; + return { intent: 'observe', reason: `autonomy "${autonomy}" may not act` }; + } + if (can.resolve) { + return { intent: 'resolve', handler: resolve.handler, text: resolve.text, reason: `self-heal attempt ${attempts + 1}` }; + } + return { intent: 'observe', reason: `autonomy "${autonomy}" may not act` }; + } + + if (!resolve) { + return exhausted || condition.escalateAfterAttempts === 0 + ? { intent: 'interrupt', reason: 'nothing can be done automatically' } + : { intent: 'observe', reason: 'no resolve handler configured' }; + } + + return { intent: 'interrupt', reason: `self-heal failed ${attempts} time${attempts === 1 ? '' : 's'}` }; } -function buildFinding(condition, signal, autonomy) { +function buildFinding(condition, signal) { return { id: `${signal.sessionId}:${condition.id}`, conditionId: condition.id, @@ -158,14 +214,10 @@ function buildFinding(condition, signal, autonomy) { repositoryName: signal.repositoryName, branch: signal.branch, tier: signal.tier, + ticketTitle: signal.ticketTitle, status: signal.status, quietSeconds: signal.quietSeconds, advice: condition.advice, - requestedRung: condition.rung, - rung: effectiveRung(condition.rung, autonomy), - suppressedByAutonomy: rungIndex(condition.rung) > rungIndex(AUTONOMY_CEILING[autonomy] || 'observe'), - nudgeText: condition.nudgeText, - actHandler: condition.actHandler, evidence: signal.lastLine, detectedAt: new Date().toISOString() }; @@ -182,7 +234,7 @@ function evaluate(signals, rules) { for (const signal of signals) { for (const condition of rules.conditions) { if (!matches(condition, signal)) continue; - findings.push(buildFinding(condition, signal, rules.autonomy)); + findings.push(buildFinding(condition, signal)); break; } } @@ -190,15 +242,15 @@ function evaluate(signals, rules) { } module.exports = { - RUNGS, SEVERITIES, AUTONOMY_LEVELS, - AUTONOMY_CEILING, + AUTONOMY_CAPABILITIES, DEFAULT_RULES_PATH, overrideRulesPath, loadRules, normalizeCondition, matches, - effectiveRung, + capabilities, + planAction, evaluate }; diff --git a/server/supervisor/supervisorUrgency.js b/server/supervisor/supervisorUrgency.js new file mode 100644 index 00000000..9a249264 --- /dev/null +++ b/server/supervisor/supervisorUrgency.js @@ -0,0 +1,234 @@ +const DEFAULT_INTERRUPTION = { + threshold: 60, + alwaysInterruptAbove: 90, + maxPerHour: 2, + minSecondsBetween: 900, + digestIntervalMinutes: 120, + quietHours: { enabled: false, startHour: 22, endHour: 7 }, + tierWeights: { 1: 1.5, 2: 1.15, 3: 0.6, 4: 0.35, none: 0.8 }, + severityBase: { info: 10, warn: 40, critical: 80 } +}; + +/** + * `Number(null)` and `Number('')` are both 0, which `Number.isFinite` happily + * accepts — so "not configured" silently becomes "zero" unless it is checked + * explicitly. Every numeric option here goes through this. + */ +function numberOrNull(value) { + if (value === null || value === undefined || value === '' || typeof value === 'boolean') return null; + const num = Number(value); + return Number.isFinite(num) ? num : null; +} + +function numberOr(value, fallback) { + const num = numberOrNull(value); + return num === null ? fallback : num; +} + +function normalizeInterruptionPolicy(raw = {}) { + const quiet = raw.quietHours || {}; + return { + threshold: numberOr(raw.threshold, DEFAULT_INTERRUPTION.threshold), + alwaysInterruptAbove: numberOr(raw.alwaysInterruptAbove, DEFAULT_INTERRUPTION.alwaysInterruptAbove), + maxPerHour: Math.max(0, numberOr(raw.maxPerHour, DEFAULT_INTERRUPTION.maxPerHour)), + minSecondsBetween: Math.max(0, numberOr(raw.minSecondsBetween, DEFAULT_INTERRUPTION.minSecondsBetween)), + digestIntervalMinutes: Math.max(1, numberOr(raw.digestIntervalMinutes, DEFAULT_INTERRUPTION.digestIntervalMinutes)), + quietHours: { + enabled: quiet.enabled === true, + startHour: numberOr(quiet.startHour, DEFAULT_INTERRUPTION.quietHours.startHour), + endHour: numberOr(quiet.endHour, DEFAULT_INTERRUPTION.quietHours.endHour) + }, + tierWeights: { ...DEFAULT_INTERRUPTION.tierWeights, ...(raw.tierWeights || {}) }, + severityBase: { ...DEFAULT_INTERRUPTION.severityBase, ...(raw.severityBase || {}) } + }; +} + +/** + * How much this finding deserves to cost you a context switch. + * + * The tier weight is what stops background work from ever pulling you out of + * flow: the same stall on a T1 task you are actively working and a T3 task + * running in the background are not the same event, and treating them the same + * is how notification systems become noise you learn to ignore. + * + * Failed self-heals raise the score, because "I tried three times and it is + * still stuck" is genuinely different information from "this just happened". + */ +function scoreUrgency(finding, { policy, attempts = 0, condition = {} } = {}) { + const config = policy || DEFAULT_INTERRUPTION; + const urgency = condition.urgency || {}; + + const base = numberOr(urgency.base, config.severityBase[finding.severity] ?? 10); + + const tier = numberOrNull(finding.tier); + const tierKey = tier === null ? 'none' : String(Math.round(tier)); + const tierWeight = numberOr(config.tierWeights[tierKey], numberOr(config.tierWeights.none, 1)); + + let score = base * tierWeight; + if (urgency.blocksWork) score += 20; + + // Every failed attempt to fix it adds weight; nothing else in the system + // knows that a problem has resisted repair. + score += Math.min(numberOr(attempts, 0), 5) * 12; + + return Math.max(0, Math.min(100, Math.round(score))); +} + +function isQuietHour(policy, date) { + const quiet = policy?.quietHours; + if (!quiet?.enabled) return false; + const hour = date.getHours(); + const { startHour, endHour } = quiet; + return startHour <= endHour + ? hour >= startHour && hour < endHour + : hour >= startHour || hour < endHour; +} + +/** + * Rate-limits how often the supervisor is allowed to break your concentration. + * + * Anything refused here is not dropped — it goes to the digest, which is the + * whole trade: one batched interruption instead of twelve individual ones. + */ +class InterruptionBudget { + constructor({ policy, now = () => Date.now() } = {}) { + this.policy = normalizeInterruptionPolicy(policy); + this.now = now; + this.recent = []; + } + + setPolicy(policy) { + this.policy = normalizeInterruptionPolicy(policy); + } + + prune() { + const cutoff = this.now() - 3_600_000; + this.recent = this.recent.filter((at) => at > cutoff); + } + + evaluate(score) { + this.prune(); + const { threshold, alwaysInterruptAbove, maxPerHour, minSecondsBetween } = this.policy; + + if (score < threshold) { + return { allow: false, reason: `below interrupt threshold (${score} < ${threshold})` }; + } + + // A genuine emergency outranks quiet hours and the hourly budget. Everything + // else respects them, which is what makes the budget trustworthy. + const overrides = score >= alwaysInterruptAbove; + if (overrides) return { allow: true, reason: `urgency ${score} overrides all limits` }; + + if (isQuietHour(this.policy, new Date(this.now()))) { + return { allow: false, reason: 'quiet hours' }; + } + if (this.recent.length >= maxPerHour) { + return { allow: false, reason: `hourly interruption budget spent (${maxPerHour}/hour)` }; + } + const last = this.recent[this.recent.length - 1]; + if (last && (this.now() - last) / 1000 < minSecondsBetween) { + return { allow: false, reason: `too soon after the last interruption (<${minSecondsBetween}s)` }; + } + + return { allow: true, reason: `urgency ${score} within budget` }; + } + + record() { + this.recent.push(this.now()); + this.prune(); + } + + getState() { + this.prune(); + return { + usedThisHour: this.recent.length, + maxPerHour: this.policy.maxPerHour, + lastInterruptionAt: this.recent.length ? new Date(this.recent[this.recent.length - 1]).toISOString() : null, + quietHoursActive: isQuietHour(this.policy, new Date(this.now())), + policy: this.policy + }; + } +} + +/** + * Everything that needed a human but did not earn an interruption. Delivered on + * a timer or on demand ("what's happening?"), so the cost is one context switch + * at a moment you chose. + */ +class DigestQueue { + constructor({ now = () => Date.now(), maxEntries = 200 } = {}) { + this.now = now; + this.maxEntries = maxEntries; + this.entries = new Map(); + this.lastDeliveredAt = now(); + } + + add(finding, { score, reason }) { + const existing = this.entries.get(finding.id); + if (existing) { + existing.count += 1; + existing.score = Math.max(existing.score, score); + existing.lastSeenAt = new Date(this.now()).toISOString(); + return existing; + } + + const entry = { + id: finding.id, + sessionId: finding.sessionId, + where: finding.worktreeId || finding.sessionId, + label: finding.label, + severity: finding.severity, + advice: finding.advice, + tier: finding.tier, + score, + heldBecause: reason, + count: 1, + firstSeenAt: new Date(this.now()).toISOString(), + lastSeenAt: new Date(this.now()).toISOString() + }; + + this.entries.set(finding.id, entry); + if (this.entries.size > this.maxEntries) { + this.entries.delete(this.entries.keys().next().value); + } + return entry; + } + + /** + * A finding that self-healed after being queued should not still be waiting + * to be mentioned — the whole point is to not report solved problems. + */ + resolve(findingId) { + return this.entries.delete(findingId); + } + + pending() { + return [...this.entries.values()].sort((a, b) => b.score - a.score); + } + + isDue() { + const intervalMs = 60_000 * (this.intervalMinutes || 120); + return this.entries.size > 0 && this.now() - this.lastDeliveredAt >= intervalMs; + } + + setInterval(minutes) { + this.intervalMinutes = Math.max(1, Number(minutes) || 120); + } + + drain() { + const items = this.pending(); + this.entries.clear(); + this.lastDeliveredAt = this.now(); + return items; + } +} + +module.exports = { + DEFAULT_INTERRUPTION, + numberOrNull, + normalizeInterruptionPolicy, + scoreUrgency, + isQuietHour, + InterruptionBudget, + DigestQueue +}; diff --git a/server/supervisorService.js b/server/supervisorService.js index 50958c2c..6bb531a7 100644 --- a/server/supervisorService.js +++ b/server/supervisorService.js @@ -4,20 +4,30 @@ const path = require('path'); const { getAgentWorkspaceDir } = require('./utils/pathUtils'); const { QuietTracker, gatherSignals } = require('./supervisor/supervisorSignals'); const rulesModule = require('./supervisor/supervisorRules'); -const { createExecutor } = require('./supervisor/supervisorActions'); +const { createExecutor, submitText } = require('./supervisor/supervisorActions'); +const { scoreUrgency, InterruptionBudget, DigestQueue } = require('./supervisor/supervisorUrgency'); const AUDIT_FILENAME = 'supervisor-audit.jsonl'; +const MAX_RESUME_DELAY_MS = 12 * 60 * 60 * 1000; /** - * The fleet supervisor. + * JARVIS — the fleet supervisor. * * Watches every agent session on a fixed tick using signals that cost nothing — - * PTY tail, status, how long a buffer has been quiet, git state — matches them - * against a data-driven condition table, and climbs an escalation ladder capped - * by the configured autonomy level. + * PTY tail, status, how long a buffer has been quiet, git state — and then tries + * to make the problem go away. * - * No model is called in the loop. Judgement is only invoked on escalation, which - * is what makes continuous supervision affordable to leave running. + * The ordering is the whole design: + * + * fix it myself → hand it to the Commander → (only then) interrupt a human + * + * A finding that has not exhausted its repair attempts is not even eligible to + * reach you, and one that has must still clear an urgency threshold weighted by + * the task's tier and fit inside an interruption budget. Everything else lands + * in a digest you read when it suits you. Your attention is the scarcest thing + * in the system and the supervisor is built to spend it last. + * + * No model runs in the loop. Judgement is only invoked on delegation. */ class SupervisorService { constructor({ logger = console } = {}) { @@ -30,19 +40,28 @@ class SupervisorService { this.activityFeed = null; this.notificationService = null; this.speechService = null; + this.commanderSender = null; this.rules = rulesModule.loadRules(); this.quietTracker = new QuietTracker(); + this.budget = new InterruptionBudget({ policy: this.rules.interruption }); + this.digest = new DigestQueue(); + this.digest.setInterval(this.rules.interruption.digestIntervalMinutes); + this.executor = null; this.timer = null; this.running = false; this.ticking = false; this.findings = []; + this.attempts = new Map(); + this.cooldowns = new Map(); + this.interruptedAt = new Map(); + this.resumeTimers = new Map(); this.lastTickAt = null; this.lastTickDurationMs = null; this.tickCount = 0; - this.cooldowns = new Map(); + this.stats = { resolved: 0, delegated: 0, interrupted: 0, digested: 0 }; } static getInstance(options = {}) { @@ -54,7 +73,7 @@ class SupervisorService { init({ sessionManager, gitHelper, agentManager, sessionRecoveryService, - taskRecordService, activityFeed, notificationService, speechService + taskRecordService, activityFeed, notificationService, speechService, commanderSender } = {}) { this.sessionManager = sessionManager || this.sessionManager; this.gitHelper = gitHelper || this.gitHelper; @@ -64,14 +83,15 @@ class SupervisorService { this.activityFeed = activityFeed || this.activityFeed; this.notificationService = notificationService || this.notificationService; this.speechService = speechService || this.speechService; + this.commanderSender = commanderSender || this.commanderSender; this.executor = createExecutor({ sessionManager: this.sessionManager, gitHelper: this.gitHelper, agentManager: this.agentManager, + commanderSender: this.commanderSender, + scheduleResume: (options) => this.scheduleResume(options), activityFeed: this.activityFeed, - notificationService: this.notificationService, - speechService: this.speechService, logger: this.logger }); @@ -98,6 +118,8 @@ class SupervisorService { reloadRules({ rulesPath = null } = {}) { this.rules = rulesModule.loadRules({ rulesPath }); + this.budget.setPolicy(this.rules.interruption); + this.digest.setInterval(this.rules.interruption.digestIntervalMinutes); if (this.running) this.restartTimer(); return this.rules; } @@ -112,23 +134,111 @@ class SupervisorService { return level; } - isCoolingDown(finding, condition) { + setInterruptionPolicy(patch = {}) { + this.rules.interruption = { ...this.rules.interruption, ...patch }; + this.budget.setPolicy(this.rules.interruption); + this.digest.setInterval(this.rules.interruption.digestIntervalMinutes); + this.appendAudit({ event: 'interruption-policy-changed', policy: this.rules.interruption }); + return this.rules.interruption; + } + + /** + * A usage limit is a wait, not a problem. Parse the reset time out of the + * banner and type `continue` when it passes. + */ + scheduleResume({ sessionId, at }) { + const delay = Math.min(MAX_RESUME_DELAY_MS, Math.max(0, at.getTime() - Date.now())); + const existing = this.resumeTimers.get(sessionId); + if (existing) clearTimeout(existing.timer); + + const timer = setTimeout(async () => { + this.resumeTimers.delete(sessionId); + await submitText(this.sessionManager, sessionId, 'continue'); + this.appendAudit({ event: 'resumed', sessionId, scheduledFor: at.toISOString() }); + }, delay); + if (typeof timer.unref === 'function') timer.unref(); + + this.resumeTimers.set(sessionId, { timer, at: at.toISOString() }); + return at.toISOString(); + } + + /** + * A high enough urgency score bypasses quiet hours and the hourly budget — + * but nothing bypasses this. Being told twice in five minutes about the same + * problem is the failure mode that teaches people to ignore alerts. + */ + repeatsTooSoon(findingId) { + const last = this.interruptedAt.get(findingId); + if (!last) return false; + return (Date.now() - last) / 1000 < this.rules.interruption.minSecondsBetween; + } + + isCoolingDown(findingId, condition) { const cooldownMs = Math.max(0, Number(condition?.cooldownSeconds || 0)) * 1000; if (!cooldownMs) return false; - const last = this.cooldowns.get(finding.id); + const last = this.cooldowns.get(findingId); return Boolean(last && Date.now() - last < cooldownMs); } - markActed(finding) { - this.cooldowns.set(finding.id, Date.now()); - } - recordFinding(entry) { this.findings.unshift(entry); const cap = this.rules.maxFindingsRetained; if (this.findings.length > cap) this.findings.length = cap; } + /** + * Something the supervisor could not handle, that is urgent enough to be + * worth your attention right now. + */ + interrupt(finding, score) { + try { + this.notificationService?.notify?.( + finding.sessionId, + finding.severity === 'critical' ? 'error' : 'warning', + `${finding.label} — ${finding.worktreeId || finding.sessionId}`, + { conditionId: finding.conditionId, advice: finding.advice, urgency: score } + ); + } catch (error) { + this.logger.warn?.('Supervisor could not send a notification', { error: error.message }); + } + + try { + this.speechService?.speak?.( + `${finding.worktreeId || finding.sessionId}: ${finding.label}. ${finding.advice}`, + { priority: 'high' } + ); + } catch (error) { + this.logger.warn?.('Supervisor could not speak', { error: error.message }); + } + + this.activityFeed?.track?.('supervisor.interrupt', { + sessionId: finding.sessionId, + conditionId: finding.conditionId, + severity: finding.severity, + urgency: score + }); + + this.budget.record(); + this.interruptedAt.set(finding.id, Date.now()); + this.stats.interrupted += 1; + } + + /** + * A problem that stopped matching has gone away — clear its repair counter and + * pull it out of the digest. Reporting solved problems is exactly the noise + * this is built to avoid. + */ + forgetHealed(activeFindingIds) { + for (const id of [...this.attempts.keys()]) { + if (activeFindingIds.has(id)) continue; + this.attempts.delete(id); + this.interruptedAt.delete(id); + if (this.digest.resolve(id)) { + this.appendAudit({ event: 'self-healed', id }); + } + } + } + /** * One pass over the fleet. Safe to call by hand — `POST /api/supervisor/tick` * runs exactly this, which is how you validate rule changes without waiting. @@ -150,49 +260,92 @@ class SupervisorService { const findings = rulesModule.evaluate(signals, this.rules); const signalsById = new Map(signals.map((signal) => [signal.sessionId, signal])); const conditionsById = new Map(this.rules.conditions.map((condition) => [condition.id, condition])); - const results = []; + const activeIds = new Set(findings.map((finding) => finding.id)); + this.forgetHealed(activeIds); + const results = []; for (const finding of findings) { const condition = conditionsById.get(finding.conditionId); - if (this.isCoolingDown(finding, condition)) { - results.push({ ...finding, performed: false, outcome: 'cooling-down' }); + const attempts = this.attempts.get(finding.id) || 0; + const plan = rulesModule.planAction(condition, { autonomy: this.rules.autonomy, attempts }); + const score = scoreUrgency(finding, { policy: this.rules.interruption, attempts, condition }); + const enriched = { ...finding, attempts, urgency: score, intent: plan.intent, planReason: plan.reason }; + + if (this.isCoolingDown(finding.id, condition)) { + results.push({ ...enriched, outcome: 'cooling-down', performed: false }); continue; } - if (dryRun) { - results.push({ ...finding, performed: false, outcome: 'dry-run' }); + results.push({ ...enriched, outcome: 'dry-run', performed: false }); continue; } const executed = await this.executor({ finding, + plan, signal: signalsById.get(finding.sessionId), rules: this.rules }); - if (executed.outcome !== 'observed') { - this.markActed(finding); + let outcome = executed.outcome; + let detail = executed.detail; + + if (plan.intent === 'resolve' || plan.intent === 'delegate') { + this.cooldowns.set(finding.id, Date.now()); + this.attempts.set(finding.id, attempts + 1); + if (executed.performed) { + this.stats[plan.intent === 'delegate' ? 'delegated' : 'resolved'] += 1; + } + } + + // Either the plan was already to interrupt, or the repair itself failed + // in a way that cannot be retried usefully. + const wantsHuman = plan.intent === 'interrupt' || executed.escalate === true; + if (wantsHuman) { + const verdict = this.repeatsTooSoon(finding.id) + ? { allow: false, reason: 'already interrupted about this recently' } + : this.budget.evaluate(score); + + if (verdict.allow) { + this.interrupt(enriched, score); + this.cooldowns.set(finding.id, Date.now()); + outcome = 'interrupted'; + detail = detail || verdict.reason; + } else { + this.digest.add(enriched, { score, reason: verdict.reason }); + this.stats.digested += 1; + outcome = 'digested'; + detail = verdict.reason; + } + } + + const entry = { ...enriched, outcome, detail, performed: executed.performed }; + if (outcome !== 'observed') { this.appendAudit({ event: 'finding', id: finding.id, conditionId: finding.conditionId, sessionId: finding.sessionId, severity: finding.severity, - requestedRung: finding.requestedRung, - rung: finding.rung, - outcome: executed.outcome, - detail: executed.detail || finding.advice + tier: finding.tier, + urgency: score, + attempts, + intent: plan.intent, + outcome, + detail }); } - this.recordFinding(executed); - results.push(executed); + this.recordFinding(entry); + results.push(entry); } this.lastTickAt = new Date().toISOString(); this.lastTickDurationMs = Date.now() - startedAt; this.tickCount += 1; + if (!dryRun && this.digest.isDue()) this.deliverDigest(); + return { at: this.lastTickAt, durationMs: this.lastTickDurationMs, @@ -208,116 +361,126 @@ class SupervisorService { } } - restartTimer() { - if (this.timer) clearInterval(this.timer); - this.timer = setInterval(() => { - this.tick().catch((error) => this.logger.error?.('Supervisor tick threw', { error: error.message })); - }, this.rules.tickSeconds * 1000); - if (typeof this.timer.unref === 'function') this.timer.unref(); - } + /** + * One batched interruption instead of a dozen individual ones. + */ + deliverDigest() { + const items = this.digest.drain(); + if (!items.length) return null; - start() { - if (this.running) return { running: true, alreadyRunning: true }; - if (this.rules.autonomy === 'off') return { running: false, reason: 'autonomy is off' }; - this.running = true; - this.restartTimer(); - this.appendAudit({ event: 'started', autonomy: this.rules.autonomy, tickSeconds: this.rules.tickSeconds }); - return { running: true, tickSeconds: this.rules.tickSeconds, autonomy: this.rules.autonomy }; - } + const summary = `${items.length} thing${items.length === 1 ? '' : 's'} waiting: ${items + .slice(0, 3) + .map((item) => `${item.where} ${item.label.toLowerCase()}`) + .join('; ')}${items.length > 3 ? `; and ${items.length - 3} more` : ''}.`; - stop() { - if (this.timer) clearInterval(this.timer); - this.timer = null; - const wasRunning = this.running; - this.running = false; - if (wasRunning) this.appendAudit({ event: 'stopped' }); - return { running: false, wasRunning }; + this.activityFeed?.track?.('supervisor.digest', { count: items.length, items }); + this.notificationService?.notify?.('supervisor', 'info', summary, { digest: true, count: items.length }); + this.appendAudit({ event: 'digest', count: items.length, items: items.map((item) => item.id) }); + + return { summary, items }; } - getFindings({ limit = 100, severity = '', sessionId = '' } = {}) { + getFindings({ limit = 100, severity = '', sessionId = '', outcome = '' } = {}) { const wantSeverity = String(severity || '').toLowerCase(); const wantSession = String(sessionId || '').trim(); + const wantOutcome = String(outcome || '').trim(); return this.findings .filter((finding) => (!wantSeverity || finding.severity === wantSeverity)) .filter((finding) => (!wantSession || finding.sessionId === wantSession)) + .filter((finding) => (!wantOutcome || finding.outcome === wantOutcome)) .slice(0, Math.max(1, Number(limit) || 100)); } /** - * A one-line-per-item read of what needs a human right now. This is what the - * voice briefing speaks and what the Commander asks for. + * What the supervisor has been doing, and the short list of things it could + * not handle. Deliberately leads with "handled" — if that number is high and + * the waiting list is empty, the system is working. */ getBriefing({ limit = 6 } = {}) { - const bySession = new Map(); - for (const finding of this.findings) { - if (!bySession.has(finding.id)) bySession.set(finding.id, finding); - } - - const rank = { critical: 0, warn: 1, info: 2 }; - const items = [...bySession.values()] - .sort((a, b) => (rank[a.severity] ?? 3) - (rank[b.severity] ?? 3) || String(b.detectedAt).localeCompare(String(a.detectedAt))) - .slice(0, Math.max(1, Number(limit) || 6)) - .map((finding) => ({ - severity: finding.severity, - sessionId: finding.sessionId, - where: finding.worktreeId || finding.sessionId, - label: finding.label, - advice: finding.advice, - outcome: finding.outcome, - detectedAt: finding.detectedAt - })); - - const counts = { critical: 0, warn: 0, info: 0 }; - for (const finding of bySession.values()) counts[finding.severity] = (counts[finding.severity] || 0) + 1; + const waiting = this.digest.pending().slice(0, Math.max(1, Number(limit) || 6)); + const handled = this.findings.filter((f) => f.outcome === 'resolved' || f.outcome === 'delegated').length; return { autonomy: this.rules.autonomy, running: this.running, lastTickAt: this.lastTickAt, - counts, - items, - spoken: this.renderSpokenBriefing(items, counts) + stats: { ...this.stats }, + handledRecently: handled, + waiting, + budget: this.budget.getState(), + spoken: this.renderSpokenBriefing(waiting, handled) }; } - renderSpokenBriefing(items, counts) { - if (!items.length) { - return this.running - ? 'Nothing needs you. The fleet is quiet.' - : 'The supervisor is not running.'; - } + renderSpokenBriefing(waiting, handled) { + if (!this.running) return 'The supervisor is not running.'; + + const handledPart = handled + ? `I handled ${handled} thing${handled === 1 ? '' : 's'} since the last check.` + : 'Nothing has needed handling.'; - const headline = counts.critical - ? `${counts.critical} thing${counts.critical === 1 ? '' : 's'} need${counts.critical === 1 ? 's' : ''} you now.` - : `${items.length} thing${items.length === 1 ? '' : 's'} to look at.`; + if (!waiting.length) return `${handledPart} Nothing is waiting on you.`; - const details = items + const details = waiting .slice(0, 3) .map((item) => `${item.where}: ${item.label.toLowerCase()}`) .join('. '); - return `${headline} ${details}.`; + return `${handledPart} ${waiting.length} thing${waiting.length === 1 ? '' : 's'} still need${waiting.length === 1 ? 's' : ''} you. ${details}.`; + } + + restartTimer() { + if (this.timer) clearInterval(this.timer); + this.timer = setInterval(() => { + this.tick().catch((error) => this.logger.error?.('Supervisor tick threw', { error: error.message })); + }, this.rules.tickSeconds * 1000); + if (typeof this.timer.unref === 'function') this.timer.unref(); + } + + start() { + if (this.running) return { running: true, alreadyRunning: true }; + if (this.rules.autonomy === 'off') return { running: false, reason: 'autonomy is off' }; + this.running = true; + this.restartTimer(); + this.appendAudit({ event: 'started', autonomy: this.rules.autonomy, tickSeconds: this.rules.tickSeconds }); + return { running: true, tickSeconds: this.rules.tickSeconds, autonomy: this.rules.autonomy }; + } + + stop() { + if (this.timer) clearInterval(this.timer); + this.timer = null; + for (const { timer } of this.resumeTimers.values()) clearTimeout(timer); + this.resumeTimers.clear(); + const wasRunning = this.running; + this.running = false; + if (wasRunning) this.appendAudit({ event: 'stopped' }); + return { running: false, wasRunning }; } getStatus() { return { running: this.running, autonomy: this.rules.autonomy, + capabilities: rulesModule.capabilities(this.rules.autonomy), tickSeconds: this.rules.tickSeconds, rulesSource: this.rules.source, conditionCount: this.rules.conditions.length, lastTickAt: this.lastTickAt, lastTickDurationMs: this.lastTickDurationMs, tickCount: this.tickCount, + stats: { ...this.stats }, findingCount: this.findings.length, + digestPending: this.digest.pending().length, + scheduledResumes: [...this.resumeTimers.entries()].map(([sessionId, entry]) => ({ sessionId, at: entry.at })), + budget: this.budget.getState(), auditPath: this.auditPath(), autonomyLevels: rulesModule.AUTONOMY_LEVELS, conditions: this.rules.conditions.map((condition) => ({ id: condition.id, label: condition.label, severity: condition.severity, - rung: condition.rung, - effectiveRung: rulesModule.effectiveRung(condition.rung, this.rules.autonomy), + resolveHandler: condition.resolve?.handler || null, + escalateAfterAttempts: condition.escalateAfterAttempts, cooldownSeconds: condition.cooldownSeconds })) }; diff --git a/tests/unit/supervisorActions.test.js b/tests/unit/supervisorActions.test.js index f9cf0950..0438ff75 100644 --- a/tests/unit/supervisorActions.test.js +++ b/tests/unit/supervisorActions.test.js @@ -1,5 +1,7 @@ const { classifyPermissionPrompt, + parseResetTime, + buildProblemBrief, submitText, createExecutor } = require('../../server/supervisor/supervisorActions'); @@ -14,11 +16,9 @@ const finding = (overrides = {}) => ({ severity: 'warn', sessionId: 'work1-claude', worktreeId: 'work1', - rung: 'notify', - requestedRung: 'notify', advice: 'nothing for 15 minutes', - nudgeText: '', - actHandler: '', + quietSeconds: 900, + status: 'busy', ...overrides }); @@ -33,145 +33,208 @@ function fakeSessionManager() { }; } -describe('supervisorActions', () => { - describe('permission classification', () => { - test('approves an unambiguously read-only prompt', () => { - const verdict = classifyPermissionPrompt('Do you want to proceed? Read(src/index.js)', rules.safety); - expect(verdict.safe).toBe(true); - }); +describe('permission classification', () => { + test('approves an unambiguously safe prompt', () => { + expect(classifyPermissionPrompt('Do you want to proceed? Read(src/index.js)', rules.safety).safe).toBe(true); + expect(classifyPermissionPrompt('Bash(npm run test)', rules.safety).safe).toBe(true); + }); - test('refuses anything matching a deny pattern, even alongside an allow match', () => { - const verdict = classifyPermissionPrompt('Read(x) then Bash(rm -rf build)', rules.safety); - expect(verdict.safe).toBe(false); - expect(verdict.reason).toMatch(/deny pattern/); - }); + test('a deny pattern beats an allow pattern in the same prompt', () => { + const verdict = classifyPermissionPrompt('Read(x) then Bash(rm -rf build)', rules.safety); + expect(verdict.safe).toBe(false); + expect(verdict.reason).toMatch(/deny pattern/); + }); - test('refuses a prompt it does not recognize rather than guessing', () => { - const verdict = classifyPermissionPrompt('Do you want to proceed? SomeNewTool(x)', rules.safety); - expect(verdict.safe).toBe(false); - expect(verdict.reason).toMatch(/no allow pattern/); - }); + test('refuses anything it does not recognize rather than guessing', () => { + const verdict = classifyPermissionPrompt('Do you want to proceed? SomeNewTool(x)', rules.safety); + expect(verdict.safe).toBe(false); + expect(verdict.reason).toMatch(/no allow pattern/); + }); - test('refuses git push even though other git commands are allowed', () => { - expect(classifyPermissionPrompt('Bash(git push origin main)', rules.safety).safe).toBe(false); - expect(classifyPermissionPrompt('Bash(git status)', rules.safety).safe).toBe(true); - }); + test('refuses to touch credentials or force-push', () => { + expect(classifyPermissionPrompt('Read(~/.ssh/id_rsa)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Read(.env)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Bash(git push --force origin main)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Bash(gh pr merge 12)', rules.safety).safe).toBe(false); + }); - test('an empty prompt is not safe', () => { - expect(classifyPermissionPrompt('', rules.safety).safe).toBe(false); - }); + test('ordinary edits and commits are allowed — this has to be usable', () => { + expect(classifyPermissionPrompt('Edit(src/app.js)', rules.safety).safe).toBe(true); + expect(classifyPermissionPrompt('Bash(git commit -m "fix")', rules.safety).safe).toBe(true); }); - describe('submitText', () => { - test('writes the text and the Enter separately', async () => { - const sessionManager = fakeSessionManager(); - await submitText(sessionManager, 'work1-claude', 'status?', { delayMs: 1 }); - expect(sessionManager.writes).toEqual([ - { sessionId: 'work1-claude', data: 'status?' }, - { sessionId: 'work1-claude', data: '\r' } - ]); - }); + test('an empty prompt is not safe', () => { + expect(classifyPermissionPrompt('', rules.safety).safe).toBe(false); + }); +}); + +describe('parseResetTime', () => { + const now = new Date('2026-07-26T14:00:00'); + + test('reads the reset hour out of a limit banner', () => { + expect(parseResetTime('5-hour limit reached ∙ resets 3pm', now).getHours()).toBe(15); + expect(parseResetTime('limit reached ∙ resets 9:30pm', now).getMinutes()).toBe(30); + }); + + test('a reset time already past today means tomorrow', () => { + const reset = parseResetTime('resets 3am', now); + expect(reset.getDate()).toBe(now.getDate() + 1); + }); + + test('returns null when there is no time to parse', () => { + expect(parseResetTime('something else entirely', now)).toBeNull(); + }); +}); - test('does not send Enter when the first write fails', async () => { - const sessionManager = { writeToSession: () => false }; - expect(await submitText(sessionManager, 'x', 'hi', { delayMs: 1 })).toBe(false); +describe('problem brief', () => { + test('gives the Commander enough to act without asking the human', () => { + const brief = buildProblemBrief( + finding({ branch: 'feature/x', tier: 1, ticketTitle: 'Fix physics' }), + { tail: 'line one\nline two' } + ); + expect(brief).toMatch(/Busy but silent on work1/); + expect(brief).toMatch(/Branch: feature\/x/); + expect(brief).toMatch(/Tier: T1/); + expect(brief).toMatch(/line two/); + expect(brief).toMatch(/Diagnose and fix it yourself/); + }); +}); + +describe('submitText', () => { + test('writes the text and the Enter separately', async () => { + const sessionManager = fakeSessionManager(); + await submitText(sessionManager, 'work1-claude', 'status?', { delayMs: 1 }); + expect(sessionManager.writes).toEqual([ + { sessionId: 'work1-claude', data: 'status?' }, + { sessionId: 'work1-claude', data: '\r' } + ]); + }); + + test('does not send Enter when the first write fails', async () => { + expect(await submitText({ writeToSession: () => false }, 'x', 'hi', { delayMs: 1 })).toBe(false); + }); +}); + +describe('executor', () => { + const run = (extra = {}) => createExecutor({ activityFeed: { track: () => {} }, ...extra }); + + test('observe does nothing outward', async () => { + const sessionManager = fakeSessionManager(); + const result = await run({ sessionManager })({ + finding: finding(), + plan: { intent: 'observe', reason: 'informational' }, + signal: {}, + rules }); + expect(result.outcome).toBe('observed'); + expect(sessionManager.writes).toEqual([]); }); - describe('executor', () => { - test('observe does nothing outward', async () => { - const sessionManager = fakeSessionManager(); - const tracked = []; - const execute = createExecutor({ sessionManager, activityFeed: { track: (k, d) => tracked.push([k, d]) } }); + test('resolving a stall types the instruction into the session', async () => { + const sessionManager = fakeSessionManager(); + const result = await run({ sessionManager })({ + finding: finding(), + plan: { intent: 'resolve', handler: 'nudge', text: 'status?' }, + signal: {}, + rules + }); + expect(result.outcome).toBe('resolved'); + expect(sessionManager.writes.map((w) => w.data)).toEqual(['status?', '\r']); + }); - const result = await execute({ finding: finding({ rung: 'observe' }), signal: {}, rules }); - expect(result.outcome).toBe('observed'); - expect(sessionManager.writes).toEqual([]); - expect(tracked).toEqual([]); + test('a handler outside allowedHandlers is blocked', async () => { + const sessionManager = fakeSessionManager(); + const result = await run({ sessionManager })({ + finding: finding(), + plan: { intent: 'resolve', handler: 'delete-everything' }, + signal: {}, + rules }); + expect(result.outcome).toBe('blocked'); + expect(sessionManager.writes).toEqual([]); + }); - test('notify records and alerts but never types', async () => { - const sessionManager = fakeSessionManager(); - const notified = []; - const execute = createExecutor({ - sessionManager, - activityFeed: { track: () => {} }, - notificationService: { notify: (...args) => notified.push(args) } - }); - - const result = await execute({ finding: finding({ rung: 'notify' }), signal: {}, rules }); - expect(result.outcome).toBe('notified'); - expect(sessionManager.writes).toEqual([]); - expect(notified).toHaveLength(1); + test('a risky permission prompt is escalated, never approved', async () => { + const sessionManager = fakeSessionManager(); + const result = await run({ sessionManager })({ + finding: finding(), + plan: { intent: 'resolve', handler: 'answer-permission' }, + signal: { tail: 'Do you want to proceed? Bash(sudo rm -rf /)' }, + rules }); + expect(result.outcome).toBe('repair-failed'); + expect(result.escalate).toBe(true); + expect(sessionManager.writes).toEqual([]); + }); - test('nudge types the configured text into the session', async () => { - const sessionManager = fakeSessionManager(); - const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); - - const result = await execute({ - finding: finding({ rung: 'nudge', nudgeText: 'status?' }), - signal: {}, - rules - }); - expect(result.outcome).toBe('nudged'); - expect(sessionManager.writes.map((w) => w.data)).toEqual(['status?', '\r']); + test('a safe permission prompt is approved without anyone being told', async () => { + const sessionManager = fakeSessionManager(); + const result = await run({ sessionManager })({ + finding: finding(), + plan: { intent: 'resolve', handler: 'answer-permission' }, + signal: { tail: 'Do you want to proceed? Read(src/app.js)' }, + rules }); + expect(result.outcome).toBe('resolved'); + expect(sessionManager.writes.map((w) => w.data)).toEqual(['1', '\r']); + }); - test('a handler outside allowedActHandlers is blocked', async () => { - const sessionManager = fakeSessionManager(); - const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); - - const result = await execute({ - finding: finding({ rung: 'act', actHandler: 'delete-everything' }), - signal: {}, - rules - }); - expect(result.outcome).toBe('act-blocked'); - expect(sessionManager.writes).toEqual([]); + test('a usage limit schedules its own resume instead of reporting', async () => { + const scheduled = []; + const result = await run({ + sessionManager: fakeSessionManager(), + scheduleResume: (options) => scheduled.push(options) + })({ + finding: finding(), + plan: { intent: 'resolve', handler: 'schedule-resume' }, + signal: { tail: '5-hour limit reached ∙ resets 3pm' }, + rules }); - test('auto-answering a risky permission prompt escalates instead', async () => { - const sessionManager = fakeSessionManager(); - const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); - - const result = await execute({ - finding: finding({ rung: 'act', actHandler: 'answer-permission' }), - signal: { tail: 'Do you want to proceed? Bash(sudo rm -rf /)' }, - rules - }); - expect(result.outcome).toBe('escalated'); - expect(sessionManager.writes).toEqual([]); + expect(result.outcome).toBe('resolved'); + expect(scheduled).toHaveLength(1); + expect(scheduled[0].sessionId).toBe('work1-claude'); + }); + + test('delegation hands a written brief to the Commander', async () => { + const briefs = []; + const result = await run({ + sessionManager: fakeSessionManager(), + commanderSender: async (text) => { briefs.push(text); return true; } + })({ + finding: finding({ label: 'Repeating the same error' }), + plan: { intent: 'delegate', handler: 'delegate-to-commander' }, + signal: { tail: 'Error: boom\nError: boom' }, + rules }); - test('auto-answering a read-only permission prompt approves it', async () => { - const sessionManager = fakeSessionManager(); - const execute = createExecutor({ sessionManager, activityFeed: { track: () => {} } }); - - const result = await execute({ - finding: finding({ rung: 'act', actHandler: 'answer-permission' }), - signal: { tail: 'Do you want to proceed? Read(src/app.js)' }, - rules - }); - expect(result.outcome).toBe('acted'); - expect(sessionManager.writes.map((w) => w.data)).toEqual(['1', '\r']); + expect(result.outcome).toBe('delegated'); + expect(briefs[0]).toMatch(/Repeating the same error/); + }); + + test('with no Commander, delegation escalates rather than silently dropping', async () => { + const result = await run({ sessionManager: fakeSessionManager() })({ + finding: finding(), + plan: { intent: 'delegate', handler: 'delegate-to-commander' }, + signal: {}, + rules }); + expect(result.outcome).toBe('repair-failed'); + expect(result.escalate).toBe(true); + }); - test('a broken notification channel does not stop the action', async () => { - const sessionManager = fakeSessionManager(); - const execute = createExecutor({ - sessionManager, - activityFeed: { track: () => { throw new Error('feed down'); } }, - notificationService: { notify: () => { throw new Error('notify down'); } }, - logger: { warn: () => {}, error: () => {} } - }); - - const result = await execute({ - finding: finding({ rung: 'nudge', nudgeText: 'ping' }), - signal: {}, - rules - }); - expect(result.outcome).toBe('nudged'); + test('a broken activity feed does not stop the repair', async () => { + const sessionManager = fakeSessionManager(); + const result = await createExecutor({ + sessionManager, + activityFeed: { track: () => { throw new Error('feed down'); } }, + logger: { warn: () => {}, error: () => {} } + })({ + finding: finding(), + plan: { intent: 'resolve', handler: 'nudge', text: 'ping' }, + signal: {}, + rules }); + expect(result.outcome).toBe('resolved'); }); }); diff --git a/tests/unit/supervisorRules.test.js b/tests/unit/supervisorRules.test.js index c1c4252a..205ee709 100644 --- a/tests/unit/supervisorRules.test.js +++ b/tests/unit/supervisorRules.test.js @@ -2,11 +2,14 @@ const { loadRules, normalizeCondition, matches, - effectiveRung, + planAction, + capabilities, evaluate, DEFAULT_RULES_PATH } = require('../../server/supervisor/supervisorRules'); +const rulesFor = (overrides = {}) => ({ ...loadRules({ rulesPath: DEFAULT_RULES_PATH }), ...overrides }); + const signal = (overrides = {}) => ({ sessionId: 'zoo-game-work1-claude', type: 'claude', @@ -24,16 +27,27 @@ const signal = (overrides = {}) => ({ }); describe('supervisorRules', () => { - test('the shipped rule table loads and grants no autonomy by default', () => { + test('the shipped table defaults to acting, not narrating', () => { const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); - expect(rules.autonomy).toBe('observe'); + expect(rules.autonomy).toBe('autopilot'); expect(rules.conditions.length).toBeGreaterThan(0); - expect(rules.conditions.every((c) => ['observe', 'notify', 'nudge', 'act'].includes(c.rung))).toBe(true); }); - test('no shipped condition asks to act — defaults never touch a terminal', () => { + test('every shipped condition knows how to fix itself', () => { const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); - expect(rules.conditions.some((c) => c.rung === 'act')).toBe(false); + for (const condition of rules.conditions) { + expect(condition.resolve).not.toBeNull(); + expect(rules.safety.allowedHandlers.concat('observe')).toContain(condition.resolve.handler); + } + }); + + test('SUPERVISOR_AUTONOMY overrides the config file', () => { + process.env.SUPERVISOR_AUTONOMY = 'observe'; + try { + expect(loadRules({ rulesPath: DEFAULT_RULES_PATH }).autonomy).toBe('observe'); + } finally { + delete process.env.SUPERVISOR_AUTONOMY; + } }); test('a malformed regex in config does not break rule loading', () => { @@ -63,51 +77,82 @@ describe('supervisorRules', () => { expect(matches(condition, signal({ git: { ahead: 0, dirty: false } }))).toBe(false); }); - test('autonomy caps how far a condition may climb', () => { - expect(effectiveRung('act', 'off')).toBeNull(); - expect(effectiveRung('act', 'observe')).toBe('observe'); - expect(effectiveRung('act', 'assist')).toBe('nudge'); - expect(effectiveRung('act', 'autopilot')).toBe('act'); - expect(effectiveRung('notify', 'autopilot')).toBe('notify'); + describe('planAction — fix first, interrupt last', () => { + const condition = normalizeCondition({ + id: 'stalled', + escalateAfterAttempts: 2, + resolve: { handler: 'nudge', text: 'status?' } + }); + + test('the first response to a problem is to fix it', () => { + const plan = planAction(condition, { autonomy: 'autopilot', attempts: 0 }); + expect(plan.intent).toBe('resolve'); + expect(plan.handler).toBe('nudge'); + }); + + test('a human is only considered once repair attempts are exhausted', () => { + expect(planAction(condition, { autonomy: 'autopilot', attempts: 1 }).intent).toBe('resolve'); + expect(planAction(condition, { autonomy: 'autopilot', attempts: 2 }).intent).toBe('interrupt'); + }); + + test('delegation to the Commander needs autopilot', () => { + const delegating = normalizeCondition({ id: 'loop', resolve: { handler: 'delegate-to-commander' } }); + expect(planAction(delegating, { autonomy: 'autopilot', attempts: 0 }).intent).toBe('delegate'); + expect(planAction(delegating, { autonomy: 'assist', attempts: 0 }).intent).toBe('observe'); + }); + + test('observe watches without acting; off does nothing at all', () => { + expect(planAction(condition, { autonomy: 'observe', attempts: 0 }).intent).toBe('observe'); + expect(planAction(condition, { autonomy: 'off', attempts: 0 }).intent).toBe('none'); + }); + + test('a purely informational condition never escalates', () => { + const informational = normalizeCondition({ id: 'idle', resolve: { handler: 'observe' } }); + expect(planAction(informational, { autonomy: 'autopilot', attempts: 99 }).intent).toBe('observe'); + }); + + test('a condition with no fix available goes straight to a human', () => { + const unfixable = normalizeCondition({ id: 'x', escalateAfterAttempts: 0 }); + expect(planAction(unfixable, { autonomy: 'autopilot', attempts: 0 }).intent).toBe('interrupt'); + }); + }); + + test('autonomy capabilities map to what may be done, not what may be said', () => { + expect(capabilities('observe')).toEqual({ resolve: false, delegate: false }); + expect(capabilities('assist')).toEqual({ resolve: true, delegate: false }); + expect(capabilities('autopilot')).toEqual({ resolve: true, delegate: true }); }); test('autonomy "off" produces no findings at all', () => { - const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); - rules.autonomy = 'off'; - expect(evaluate([signal({ status: 'busy', quietSeconds: 5000 })], rules)).toEqual([]); + expect(evaluate([signal({ status: 'busy', quietSeconds: 5000 })], rulesFor({ autonomy: 'off' }))).toEqual([]); }); test('only the first matching condition fires per session', () => { - const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); const findings = evaluate([signal({ status: 'busy', quietSeconds: 5000, tail: 'Claude usage limit reached ∙ resets 3am' - })], rules); + })], rulesFor()); expect(findings).toHaveLength(1); expect(findings[0].conditionId).toBe('usage-limit-reached'); - expect(findings[0].severity).toBe('critical'); }); - test('a stalled session is reported as stalled, not as a usage limit', () => { + test('a usage limit is treated as a wait, not an emergency', () => { const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); - const findings = evaluate([signal({ status: 'busy', quietSeconds: 1200, tail: 'still thinking' })], rules); - expect(findings[0].conditionId).toBe('stalled'); - expect(findings[0].requestedRung).toBe('nudge'); - expect(findings[0].rung).toBe('observe'); - expect(findings[0].suppressedByAutonomy).toBe(true); + const limit = rules.conditions.find((c) => c.id === 'usage-limit-reached'); + expect(limit.severity).toBe('info'); + expect(limit.resolve.handler).toBe('schedule-resume'); + expect(limit.escalateAfterAttempts).toBeGreaterThan(10); }); - test('findings carry a stable id so cooldowns can key on them', () => { - const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); - const [finding] = evaluate([signal({ status: 'busy', quietSeconds: 1200 })], rules); + test('findings carry a stable id so attempts and cooldowns can key on them', () => { + const [finding] = evaluate([signal({ status: 'busy', quietSeconds: 1200 })], rulesFor()); expect(finding.id).toBe('zoo-game-work1-claude:stalled'); }); test('an exited agent is detected from the recovery marker, not the tail', () => { - const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); - const findings = evaluate([signal({ status: 'idle', agentPresent: false, quietSeconds: 600 })], rules); + const findings = evaluate([signal({ status: 'idle', agentPresent: false, quietSeconds: 600 })], rulesFor()); expect(findings[0].conditionId).toBe('agent-exited'); }); }); diff --git a/tests/unit/supervisorService.test.js b/tests/unit/supervisorService.test.js index f73998e8..8fc63ac6 100644 --- a/tests/unit/supervisorService.test.js +++ b/tests/unit/supervisorService.test.js @@ -9,8 +9,10 @@ function fakeSession({ id, status = 'idle', buffer = '', type = 'claude' }) { return { id, type, status, buffer, worktreeId: id.split('-')[0], workspace: 'ws', pty: {} }; } -function harness({ sessions = [], autonomy = 'assist' } = {}) { +function harness({ sessions = [], autonomy = 'autopilot', tier = null, commanderSender = null } = {}) { const writes = []; + const notifications = []; + const spoken = []; const sessionMap = new Map(sessions.map((session) => [session.id, session])); const supervisor = new SupervisorService({ logger: { info: () => {}, warn: () => {}, error: () => {} } }); @@ -21,12 +23,21 @@ function harness({ sessions = [], autonomy = 'assist' } = {}) { getSessionCwd: () => null }, sessionRecoveryService: { getSession: () => ({ lastAgent: 'claude', lastAgentActive: true }) }, - taskRecordService: { get: () => null }, + taskRecordService: { get: () => (tier ? { tier } : null) }, activityFeed: { track: () => {} }, - notificationService: { notify: () => {} } + notificationService: { notify: (...args) => notifications.push(args) }, + speechService: { speak: (text) => spoken.push(text) }, + commanderSender }); supervisor.rules.autonomy = autonomy; - return { supervisor, writes, sessionMap }; + + // Push a session past every quiet-time threshold without waiting for it. + const goQuiet = (id) => { + supervisor.quietTracker.observe(id, 8); + supervisor.quietTracker.state.get(id).lastGrowthAt = Date.now() - 3_600_000; + }; + + return { supervisor, writes, notifications, spoken, goQuiet }; } describe('supervisor signals', () => { @@ -52,8 +63,7 @@ describe('supervisor signals', () => { }); test('repeated-line detection ignores short lines', () => { - const looping = Array(6).fill('Error: cannot find module "widget"').join('\n'); - expect(maxLineRepeat(looping)).toBe(6); + expect(maxLineRepeat(Array(6).fill('Error: cannot find module "widget"').join('\n'))).toBe(6); expect(maxLineRepeat(Array(6).fill('ok').join('\n'))).toBe(0); }); @@ -75,99 +85,168 @@ describe('SupervisorService', () => { fs.rmSync(tmpDir, { recursive: true, force: true }); }); - test('observe mode watches without touching anything', async () => { - const { supervisor, writes } = harness({ + test('a stall is fixed silently — no notification for something it handled', async () => { + const { supervisor, writes, notifications, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })] + }); + goQuiet('work1-claude'); + + const result = await supervisor.tick(); + expect(result.findings[0].outcome).toBe('resolved'); + expect(writes.map((w) => w.data)).toEqual([expect.stringMatching(/status\?/), '\r']); + expect(notifications).toEqual([]); + }); + + test('a human is only reached after repair attempts are exhausted', async () => { + const { supervisor, notifications, goQuiet } = harness({ sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], - autonomy: 'observe' + tier: 1 }); - supervisor.quietTracker.observe('work1-claude', 8); - supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + const outcomes = []; + for (let i = 0; i < 6 && !outcomes.includes('interrupted'); i += 1) { + goQuiet('work1-claude'); + supervisor.cooldowns.clear(); + outcomes.push((await supervisor.tick()).findings[0].outcome); + } - const result = await supervisor.tick(); - expect(result.sessionsWatched).toBe(1); - expect(result.findings[0].conditionId).toBe('stalled'); - expect(result.findings[0].outcome).toBe('observed'); - expect(writes).toEqual([]); + expect(outcomes.slice(0, 3)).toEqual(['resolved', 'resolved', 'resolved']); + expect(outcomes.at(-1)).toBe('interrupted'); + expect(notifications).toHaveLength(1); }); - test('assist mode nudges a stalled session', async () => { - const { supervisor, writes } = harness({ + test('the same problem never interrupts twice in quick succession', async () => { + const { supervisor, notifications, goQuiet } = harness({ sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], - autonomy: 'assist' + tier: 1 }); - supervisor.quietTracker.observe('work1-claude', 8); - supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + for (let i = 0; i < 8; i += 1) { + goQuiet('work1-claude'); + supervisor.cooldowns.clear(); + await supervisor.tick(); + } - const result = await supervisor.tick(); - expect(result.findings[0].outcome).toBe('nudged'); - expect(writes.map((w) => w.data)).toEqual(['status? if you are blocked, say what on and stop.', '\r']); + // Repeated ticks past the escalation point must not become a drumbeat, even + // at an urgency score that overrides quiet hours and the hourly budget. + expect(notifications).toHaveLength(1); + expect(supervisor.digest.pending()[0].heldBecause).toMatch(/already interrupted/); }); - test('a finding does not re-fire while it is cooling down', async () => { - const { supervisor, writes } = harness({ + test('background work never interrupts — it goes to the digest instead', async () => { + const { supervisor, notifications, goQuiet } = harness({ sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], - autonomy: 'assist' + tier: 4 }); - const stall = () => { - supervisor.quietTracker.observe('work1-claude', 8); - supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; - }; + for (let i = 0; i < 6; i += 1) { + goQuiet('work1-claude'); + supervisor.cooldowns.clear(); + await supervisor.tick(); + } + + expect(notifications).toEqual([]); + expect(supervisor.digest.pending()).toHaveLength(1); + expect(supervisor.digest.pending()[0].heldBecause).toMatch(/below interrupt threshold/); + }); + + test('a problem that heals is dropped from the digest, never mentioned', async () => { + const session = fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' }); + const { supervisor, goQuiet } = harness({ sessions: [session], tier: 4 }); - stall(); + for (let i = 0; i < 5; i += 1) { + goQuiet('work1-claude'); + supervisor.cooldowns.clear(); + await supervisor.tick(); + } + expect(supervisor.digest.pending()).toHaveLength(1); + + session.status = 'idle'; + session.buffer = 'done'; await supervisor.tick(); - stall(); - const second = await supervisor.tick(); - expect(second.findings[0].outcome).toBe('cooling-down'); - expect(writes).toHaveLength(2); + expect(supervisor.digest.pending()).toEqual([]); + expect(supervisor.attempts.has('work1-claude:stalled')).toBe(false); }); - test('dry run reports what would happen without doing it', async () => { - const { supervisor, writes } = harness({ + test('a usage limit resolves itself and schedules a resume', async () => { + const { supervisor, notifications, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: '5-hour limit reached ∙ resets 3am' })] + }); + goQuiet('work1-claude'); + + const result = await supervisor.tick(); + expect(result.findings[0].conditionId).toBe('usage-limit-reached'); + expect(result.findings[0].outcome).toBe('resolved'); + expect(notifications).toEqual([]); + expect(supervisor.getStatus().scheduledResumes).toHaveLength(1); + }); + + test('an error loop is delegated to the Commander, not dumped on the human', async () => { + const briefs = []; + const { supervisor, notifications, goQuiet } = harness({ + sessions: [fakeSession({ + id: 'work1-claude', + status: 'busy', + buffer: Array(6).fill('Error: cannot find module "widget"').join('\n') + })], + commanderSender: async (text) => { briefs.push(text); return true; } + }); + goQuiet('work1-claude'); + + const result = await supervisor.tick(); + expect(result.findings[0].outcome).toBe('delegated'); + expect(briefs).toHaveLength(1); + expect(notifications).toEqual([]); + }); + + test('observe mode watches without touching anything', async () => { + const { supervisor, writes, goQuiet } = harness({ sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], - autonomy: 'autopilot' + autonomy: 'observe' }); + goQuiet('work1-claude'); - supervisor.quietTracker.observe('work1-claude', 8); - supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + const result = await supervisor.tick(); + expect(result.findings[0].outcome).toBe('observed'); + expect(writes).toEqual([]); + }); + + test('dry run reports the plan without carrying it out', async () => { + const { supervisor, writes, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })] + }); + goQuiet('work1-claude'); const result = await supervisor.tick({ dryRun: true }); expect(result.findings[0].outcome).toBe('dry-run'); + expect(result.findings[0].intent).toBe('resolve'); expect(writes).toEqual([]); }); test('server terminals are not supervised', async () => { - const { supervisor } = harness({ - sessions: [fakeSession({ id: 'work1-server', type: 'server', status: 'busy' })] - }); - const result = await supervisor.tick(); - expect(result.sessionsWatched).toBe(0); + const { supervisor } = harness({ sessions: [fakeSession({ id: 'work1-server', type: 'server', status: 'busy' })] }); + expect((await supervisor.tick()).sessionsWatched).toBe(0); }); - test('actions are written to the audit log', async () => { - const { supervisor } = harness({ - sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], - autonomy: 'assist' + test('every action lands in the audit log', async () => { + const { supervisor, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })] }); - - supervisor.quietTracker.observe('work1-claude', 8); - supervisor.quietTracker.state.get('work1-claude').lastGrowthAt = Date.now() - 3_600_000; + goQuiet('work1-claude'); await supervisor.tick(); const audit = fs.readFileSync(supervisor.auditPath(), 'utf8').trim().split('\n').map(JSON.parse); - expect(audit.at(-1)).toMatchObject({ event: 'finding', conditionId: 'stalled', outcome: 'nudged' }); + expect(audit.at(-1)).toMatchObject({ event: 'finding', conditionId: 'stalled', outcome: 'resolved', intent: 'resolve' }); }); test('setAutonomy rejects unknown levels and records real changes', () => { const { supervisor } = harness(); expect(() => supervisor.setAutonomy('yolo')).toThrow(/Unknown autonomy level/); - expect(supervisor.setAutonomy('autopilot')).toBe('autopilot'); + expect(supervisor.setAutonomy('assist')).toBe('assist'); const audit = fs.readFileSync(supervisor.auditPath(), 'utf8').trim().split('\n').map(JSON.parse); - expect(audit.at(-1)).toMatchObject({ event: 'autonomy-changed', to: 'autopilot' }); + expect(audit.at(-1)).toMatchObject({ event: 'autonomy-changed', to: 'assist' }); }); test('start refuses to run when autonomy is off', () => { @@ -176,30 +255,48 @@ describe('SupervisorService', () => { supervisor.stop(); }); - test('the briefing reads as a sentence and leads with what is critical', async () => { - const { supervisor } = harness({ - sessions: [ - fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' }), - fakeSession({ id: 'work2-claude', status: 'busy', buffer: 'usage limit reached ∙ resets 3am' }) - ], - autonomy: 'assist' + test('the briefing leads with what was handled, not with problems', async () => { + const { supervisor, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })] }); - - for (const id of ['work1-claude', 'work2-claude']) { - supervisor.quietTracker.observe(id, 8); - supervisor.quietTracker.state.get(id).lastGrowthAt = Date.now() - 3_600_000; - } + supervisor.running = true; + goQuiet('work1-claude'); await supervisor.tick(); const briefing = supervisor.getBriefing(); - expect(briefing.counts.critical).toBe(1); - expect(briefing.items[0].severity).toBe('critical'); - expect(briefing.spoken).toMatch(/needs you now/); + expect(briefing.handledRecently).toBe(1); + expect(briefing.waiting).toEqual([]); + expect(briefing.spoken).toMatch(/I handled 1 thing/); + expect(briefing.spoken).toMatch(/Nothing is waiting on you/); }); - test('an empty fleet briefs as quiet rather than as an error', () => { - const { supervisor } = harness(); - supervisor.running = true; - expect(supervisor.getBriefing().spoken).toMatch(/quiet/); + test('delivering the digest batches everything into one message', async () => { + const { supervisor, notifications, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'thinking' })], + tier: 4 + }); + + for (let i = 0; i < 5; i += 1) { + goQuiet('work1-claude'); + supervisor.cooldowns.clear(); + await supervisor.tick(); + } + + const delivered = supervisor.deliverDigest(); + expect(delivered.items).toHaveLength(1); + expect(notifications).toHaveLength(1); + expect(supervisor.digest.pending()).toEqual([]); + }); + + test('stopping clears scheduled resumes so nothing fires after shutdown', async () => { + const { supervisor, goQuiet } = harness({ + sessions: [fakeSession({ id: 'work1-claude', status: 'busy', buffer: 'limit reached ∙ resets 3am' })] + }); + goQuiet('work1-claude'); + await supervisor.tick(); + expect(supervisor.resumeTimers.size).toBe(1); + + supervisor.stop(); + expect(supervisor.resumeTimers.size).toBe(0); }); }); diff --git a/tests/unit/supervisorUrgency.test.js b/tests/unit/supervisorUrgency.test.js new file mode 100644 index 00000000..1b75cccd --- /dev/null +++ b/tests/unit/supervisorUrgency.test.js @@ -0,0 +1,206 @@ +const { + scoreUrgency, + isQuietHour, + normalizeInterruptionPolicy, + InterruptionBudget, + DigestQueue +} = require('../../server/supervisor/supervisorUrgency'); + +const policy = normalizeInterruptionPolicy({}); + +const finding = (overrides = {}) => ({ + id: 'work1-claude:stalled', + sessionId: 'work1-claude', + worktreeId: 'work1', + label: 'Busy but silent', + severity: 'warn', + advice: 'nothing for 15 minutes', + tier: 3, + ...overrides +}); + +describe('urgency scoring', () => { + test('the same problem on background work scores far below focus work', () => { + const background = scoreUrgency(finding({ tier: 3 }), { policy }); + const focus = scoreUrgency(finding({ tier: 1 }), { policy }); + + expect(focus).toBeGreaterThan(background); + expect(background).toBeLessThan(policy.threshold); + expect(focus).toBeGreaterThanOrEqual(policy.threshold); + }); + + test('tier 4 is effectively never worth an interruption', () => { + expect(scoreUrgency(finding({ tier: 4, severity: 'warn' }), { policy })).toBeLessThan(policy.threshold); + }); + + test('a critical problem on focus work outranks everything', () => { + const score = scoreUrgency(finding({ tier: 1, severity: 'critical' }), { policy }); + expect(score).toBeGreaterThanOrEqual(policy.alwaysInterruptAbove); + }); + + test('repeated failed repairs raise urgency — resisting a fix is information', () => { + const first = scoreUrgency(finding({ tier: 2 }), { policy, attempts: 0 }); + const fourth = scoreUrgency(finding({ tier: 2 }), { policy, attempts: 3 }); + expect(fourth).toBeGreaterThan(first); + }); + + test('a condition can pin its own base score regardless of severity', () => { + const score = scoreUrgency(finding({ tier: 1, severity: 'critical' }), { + policy, + condition: { urgency: { base: 5 } } + }); + expect(score).toBeLessThan(policy.threshold); + }); + + test('blocking work adds weight', () => { + const plain = scoreUrgency(finding({ tier: 2 }), { policy }); + const blocking = scoreUrgency(finding({ tier: 2 }), { policy, condition: { urgency: { blocksWork: true } } }); + expect(blocking).toBeGreaterThan(plain); + }); + + test('an untiered session sits between focus and background', () => { + const untiered = scoreUrgency(finding({ tier: null }), { policy }); + expect(untiered).toBeGreaterThan(scoreUrgency(finding({ tier: 3 }), { policy })); + expect(untiered).toBeLessThan(scoreUrgency(finding({ tier: 1 }), { policy })); + }); +}); + +describe('quiet hours', () => { + const quiet = normalizeInterruptionPolicy({ quietHours: { enabled: true, startHour: 22, endHour: 7 } }); + + test('an overnight window wraps midnight correctly', () => { + expect(isQuietHour(quiet, new Date('2026-07-26T23:30:00'))).toBe(true); + expect(isQuietHour(quiet, new Date('2026-07-26T03:00:00'))).toBe(true); + expect(isQuietHour(quiet, new Date('2026-07-26T12:00:00'))).toBe(false); + }); + + test('disabled quiet hours never apply', () => { + expect(isQuietHour(policy, new Date('2026-07-26T03:00:00'))).toBe(false); + }); +}); + +describe('InterruptionBudget', () => { + let now; + const budget = () => new InterruptionBudget({ policy: { maxPerHour: 2, minSecondsBetween: 600 }, now: () => now }); + + beforeEach(() => { now = Date.parse('2026-07-26T12:00:00Z'); }); + + test('below-threshold findings never interrupt', () => { + expect(budget().evaluate(30).allow).toBe(false); + }); + + test('the hourly budget stops a third interruption', () => { + const b = budget(); + expect(b.evaluate(70).allow).toBe(true); + b.record(); + now += 700_000; + expect(b.evaluate(70).allow).toBe(true); + b.record(); + now += 700_000; + + const third = b.evaluate(70); + expect(third.allow).toBe(false); + expect(third.reason).toMatch(/budget spent/); + }); + + test('two interruptions in quick succession are throttled', () => { + const b = budget(); + b.record(); + now += 60_000; + expect(b.evaluate(70).reason).toMatch(/too soon/); + }); + + test('a genuine emergency overrides both the budget and quiet hours', () => { + const b = new InterruptionBudget({ + policy: { maxPerHour: 0, quietHours: { enabled: true, startHour: 0, endHour: 23 } }, + now: () => now + }); + expect(b.evaluate(95).allow).toBe(true); + expect(b.evaluate(70).allow).toBe(false); + }); + + test('the budget forgets interruptions older than an hour', () => { + const b = budget(); + b.record(); + b.record(); + now += 3_700_000; + expect(b.evaluate(70).allow).toBe(true); + }); +}); + +describe('DigestQueue', () => { + let now; + const queue = () => new DigestQueue({ now: () => now }); + + beforeEach(() => { now = Date.parse('2026-07-26T12:00:00Z'); }); + + test('repeats of the same finding collapse into one entry with a count', () => { + const q = queue(); + q.add(finding(), { score: 40, reason: 'below threshold' }); + q.add(finding(), { score: 55, reason: 'below threshold' }); + + const pending = q.pending(); + expect(pending).toHaveLength(1); + expect(pending[0].count).toBe(2); + expect(pending[0].score).toBe(55); + }); + + test('a finding that heals is pulled from the digest and never mentioned', () => { + const q = queue(); + q.add(finding(), { score: 40, reason: 'below threshold' }); + expect(q.resolve('work1-claude:stalled')).toBe(true); + expect(q.pending()).toEqual([]); + }); + + test('pending items are ordered by urgency', () => { + const q = queue(); + q.add(finding({ id: 'a' }), { score: 20, reason: 'x' }); + q.add(finding({ id: 'b' }), { score: 55, reason: 'x' }); + expect(q.pending().map((item) => item.id)).toEqual(['b', 'a']); + }); + + test('an empty digest is never due', () => { + const q = queue(); + q.setInterval(1); + now += 3_600_000; + expect(q.isDue()).toBe(false); + }); + + test('the digest becomes due once the interval passes with something waiting', () => { + const q = queue(); + q.setInterval(60); + q.add(finding(), { score: 20, reason: 'x' }); + expect(q.isDue()).toBe(false); + now += 3_700_000; + expect(q.isDue()).toBe(true); + }); + + test('draining empties the queue and resets the clock', () => { + const q = queue(); + q.setInterval(60); + q.add(finding(), { score: 20, reason: 'x' }); + now += 3_700_000; + + expect(q.drain()).toHaveLength(1); + expect(q.pending()).toEqual([]); + expect(q.isDue()).toBe(false); + }); +}); + +describe('null-vs-zero handling', () => { + test('an unset urgency base falls back to severity, it does not become zero', () => { + const explicit = scoreUrgency(finding({ tier: 1 }), { policy, condition: { urgency: { base: null } } }); + const absent = scoreUrgency(finding({ tier: 1 }), { policy, condition: {} }); + expect(explicit).toBe(absent); + expect(explicit).toBeGreaterThan(0); + }); + + test('an explicit zero base really is zero', () => { + expect(scoreUrgency(finding({ tier: 1 }), { policy, condition: { urgency: { base: 0 } } })).toBe(0); + }); + + test('an unset maxPerHour keeps the default rather than becoming NaN', () => { + expect(normalizeInterruptionPolicy({}).maxPerHour).toBe(2); + expect(normalizeInterruptionPolicy({ maxPerHour: 0 }).maxPerHour).toBe(0); + }); +}); From 7707ac28e5edf5d3567c990f66c0ac884c70e7b5 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:25:35 +1000 Subject: [PATCH 07/69] feat(atlas): git-backed registry so the map follows you between machines MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A map that only exists on one laptop is not a map you can rely on. The blocker was that syncing it collides with the sharing model — the master registry describes private repos, so it cannot just be pushed somewhere shared. The split that resolves it: - REGISTRY (portable judgement: what a repo is worth reading for, and who may see it) lives in a PRIVATE git repo you control. `atlas remote set` + `atlas sync` pulls, merges and pushes it. Multi-machine and backed up. - DISCOVERY (what this particular computer has cloned) stays local and is never synced — it would be wrong on every other machine. - BUNDLES (the subsets you share) are published into whichever repo that audience already has access to. GitHub permissions stay the enforcement. One file per repo under entries/. That detail is what makes it work: two machines curating different repos touch different files, so git merges them with no conflict at all — verified with a real remote and interleaved edits. Subscriptions close the loop the other way: `atlas subscribe` reads a bundle someone published, so their map shows up in your searches, attributed to them, at the lowest precedence — your own notes always win. And what was shared with you is never re-shared by you: foreign entries are excluded from compilation. Legacy single-file registries migrate automatically on first read. 762 unit tests green. --- scripts/atlas.js | 100 ++++++++++- server/atlas/atlasStore.js | 223 +++++++++++++++++++----- server/atlas/atlasSync.js | 256 ++++++++++++++++++++++++++++ server/repoAtlasService.js | 113 +++++++++++- server/routes/atlasRoutes.js | 33 ++++ tests/unit/repoAtlasService.test.js | 3 +- tests/unit/repoAtlasSync.test.js | 179 +++++++++++++++++++ 7 files changed, 853 insertions(+), 54 deletions(-) create mode 100644 server/atlas/atlasSync.js create mode 100644 tests/unit/repoAtlasSync.test.js diff --git a/scripts/atlas.js b/scripts/atlas.js index d0287c94..f010c99a 100755 --- a/scripts/atlas.js +++ b/scripts/atlas.js @@ -80,14 +80,27 @@ const commands = { out('Next: `atlas doctor` to see what needs curating, `atlas note --topic X --quality N` to record what a repo is good at.'); }, - status() { + async status() { const status = atlas.getStatus(); out(`atlas dir ${status.atlasDir}`); - out(`registry ${status.registryPath}`); + out(`registry ${status.registryDir}`); out(`scan roots ${status.scanRoots.join(', ')}`); - out(`repos ${status.entryCount} (${status.clonedCount} cloned locally)`); + out(`repos ${status.entryCount} (${status.clonedCount} cloned locally, ${status.curatedCount} curated)`); out(`highlights ${status.highlightCount}`); out(`audiences ${status.audiences.join(', ') || 'none configured'}`); + out(`remote ${status.remote || 'not configured — run `atlas remote set `'}`); + if (status.subscriptions.length) { + out(`subscribed ${status.subscriptions.map((s) => `${s.name} (${s.entryCount})`).join(', ')}`); + } + + const git = await atlas.getSyncStatus(); + if (git.tracked) { + const pending = [git.dirty ? 'uncommitted changes' : '', git.unpushed ? `${git.unpushed} unpushed` : ''] + .filter(Boolean).join(', '); + out(`sync ${git.branch}${pending ? ` — ${pending}` : ' — up to date'}`); + } else { + out('sync not tracked in git yet'); + } if (!status.discovery) { out('discovery never run — start with `atlas scan`'); } else { @@ -240,13 +253,76 @@ const commands = { id, label: flags.label === true ? '' : String(flags.label || ''), description: flags.description === true ? '' : String(flags.description || ''), - outputPath: flags.out === true ? '' : String(flags.out || '') + outputPath: flags.out === true ? '' : String(flags.out || ''), + outputRemote: flags.outRemote === true ? '' : String(flags.outRemote || '') }); return out(`Audience "${id}" saved. Tag repos into it with \`atlas set --visibility team --groups ${id}\`.`); } return fail(`unknown audience action "${action}"`); }, + async remote(positionals, flags) { + const action = positionals[0]; + if (action === 'set') { + const url = positionals[1]; + if (!url) return fail('usage: atlas remote set '); + await atlas.setRemote(url); + out(`Registry remote set to ${url}.`); + out('Run `atlas sync` to push your curated entries and pull anything from your other machines.'); + return undefined; + } + const status = await atlas.getSyncStatus(); + out(`remote ${status.remote || 'not configured'}`); + out(`dir ${status.dir}`); + if (status.tracked) out(`branch ${status.branch}${status.dirty ? ' (dirty)' : ''}`); + return undefined; + }, + + async sync(_positionals, flags) { + out('Syncing registry…'); + const result = await atlas.sync({ message: flags.message === true ? '' : String(flags.message || '') }); + for (const step of result.steps || []) { + out(` ${step.step.padEnd(14)} ${step.ok === false ? 'failed' : 'ok'}${step.detail ? ` — ${String(step.detail).split('\n')[0]}` : ''}`); + } + if (!result.ok) return fail(result.error); + out(`Synced ${result.entryCount} curated entries with ${result.remote} (${result.branch}).`); + return undefined; + }, + + async publish(positionals, flags) { + const audience = positionals[0]; + if (!audience) return fail('usage: atlas publish [--no-push]'); + + const result = await atlas.publish(audience, { push: flags.noPush !== true }); + out(`${audience}: ${result.counts.included} shared / ${result.counts.excluded} withheld / ${result.counts.redacted} partly redacted`); + if (result.published?.error) return fail(result.published.error); + out(`wrote ${result.published.path}`); + if (result.published.committed) out(result.published.pushed ? 'committed and pushed' : 'committed (push failed)'); + return undefined; + }, + + async subscribe(positionals, flags) { + const action = positionals[0]; + if (action === 'list' || !action) { + const subs = atlas.listSubscriptions(); + if (!subs.length) return out('No subscriptions. Add one: `atlas subscribe add `'); + for (const sub of subs) out(`${sub.name.padEnd(20)} ${sub.entryCount} repos ${sub.generatedAt || ''}`); + return undefined; + } + if (action === 'add') { + const [, name, source] = positionals; + if (!name || !source) return fail('usage: atlas subscribe add '); + const result = await atlas.subscribe({ name, source }); + return out(`Subscribed to ${result.name} — ${result.entryCount} repos now searchable (marked as theirs).`); + } + if (action === 'remove') { + const name = positionals[1]; + if (!name) return fail('usage: atlas subscribe remove '); + return out(atlas.unsubscribe(name) ? `Removed ${name}.` : `No subscription "${name}".`); + } + return fail(`unknown subscribe action "${action}"`); + }, + compile(positionals, flags) { const audience = positionals[0]; if (!audience) return fail('usage: atlas compile [--dry-run] [--explain]'); @@ -310,8 +386,15 @@ const commands = { atlas avoid --topic X --reason "..." atlas set [--visibility ...] [--groups a,b] [--kind ...] [--summary "..."] - atlas audience list | add [--label "..."] [--out ] + atlas audience list | add [--label "..."] [--out ] [--out-remote ] atlas compile [--dry-run] [--explain] + +multi-machine: + atlas remote set track the registry in a PRIVATE git repo + atlas sync [--message "..."] pull + merge + push your curated entries + atlas publish [--no-push] compile a bundle and commit it where that audience can read it + atlas subscribe add read someone else's published bundle + atlas subscribe list | remove atlas doctor atlas init [path] [--visibility ...] [--groups a,b] @@ -324,7 +407,12 @@ values: maturity ${MATURITIES.join('|')} visibility ${VISIBILITIES.join('| Sharing model: entries are private by default. \`visibility: public\` goes in every bundle, \`team\` goes only to audiences listed in its groups, \`private\` never leaves this machine. Compiled bundles are metadata distribution — GitHub permissions are -still the real access control.`); +still the real access control. + +Multi-machine: your registry (judgement, portable) lives in a PRIVATE git repo, +one file per repo so machines never conflict. Discovery (what this computer has +cloned) stays local and is never synced. Audience bundles are published into +whichever shared repo that audience already has access to.`); } }; diff --git a/server/atlas/atlasStore.js b/server/atlas/atlasStore.js index cf271c88..e4fd74ac 100644 --- a/server/atlas/atlasStore.js +++ b/server/atlas/atlasStore.js @@ -7,16 +7,42 @@ const { SCHEMA_VERSION, kebab, normalizeEntry } = require('./atlasSchema'); const MANIFEST_FILENAME = '.repo-atlas.json'; const MANIFEST_SEARCH_SUBDIRS = ['', 'master', 'main']; const DISCOVERY_CACHE_TTL_MS = 12 * 60 * 60 * 1000; +const CONFIG_FILENAME = 'atlas.config.json'; +const LEGACY_REGISTRY_FILENAME = 'registry.json'; function atlasDir() { const override = String(process.env.AGENT_WORKSPACE_ATLAS_DIR || '').trim(); return override ? path.resolve(override) : path.join(getAgentWorkspaceDir(), 'atlas'); } -function registryPath() { - return path.join(atlasDir(), 'registry.json'); +/** + * The portable half of the atlas — your judgement about repos, which is the + * same on every machine you work from and is what belongs in git. + */ +function registryDir() { + return path.join(atlasDir(), 'registry'); +} + +/** + * One file per curated repo. This is the detail that makes multi-machine sync + * work: two machines curating different repos touch different files, so git + * merges them without a conflict. A single registry.json would collide on + * every concurrent edit. + */ +function entriesDir() { + return path.join(registryDir(), 'entries'); +} + +function configPath() { + return path.join(registryDir(), CONFIG_FILENAME); } +function legacyRegistryPath() { + return path.join(atlasDir(), LEGACY_REGISTRY_FILENAME); +} + +// Machine-local: what this particular computer happens to have cloned. Never +// synced — it would be wrong on every other machine. function discoveryCachePath() { return path.join(atlasDir(), 'discovery.json'); } @@ -25,6 +51,10 @@ function bundlesDir() { return path.join(atlasDir(), 'bundles'); } +function subscriptionsDir() { + return path.join(atlasDir(), 'subscriptions'); +} + function ensureDir(dirPath) { fs.mkdirSync(dirPath, { recursive: true }); return dirPath; @@ -49,6 +79,7 @@ function emptyRegistry() { schemaVersion: SCHEMA_VERSION, scanRoots: [], audiences: [], + remote: '', defaults: { visibility: 'private', groups: [] }, entries: {} }; @@ -61,61 +92,126 @@ function normalizeAudience(raw) { id, label: String(raw?.label || id).trim(), description: String(raw?.description || '').trim(), - // Where compiled bundles for this audience should be copied, if anywhere. - outputPath: String(raw?.outputPath || '').trim() + // Where compiled bundles for this audience get published — a path inside a + // repo that audience already has access to. + outputPath: String(raw?.outputPath || '').trim(), + // Optional git repo to commit that bundle into. + outputRemote: String(raw?.outputRemote || '').trim() }; } -function loadRegistry() { - const raw = readJson(registryPath(), null); - if (!raw) return emptyRegistry(); - - const registry = emptyRegistry(); - registry.scanRoots = Array.isArray(raw.scanRoots) ? raw.scanRoots.map(String).filter(Boolean) : []; - registry.audiences = (Array.isArray(raw.audiences) ? raw.audiences : []).map(normalizeAudience).filter(Boolean); - registry.defaults = { - visibility: String(raw?.defaults?.visibility || 'private'), - groups: Array.isArray(raw?.defaults?.groups) ? raw.defaults.groups.map(kebab).filter(Boolean) : [] +function loadConfig() { + const raw = readJson(configPath(), null) || readJson(legacyRegistryPath(), null) || {}; + return { + schemaVersion: SCHEMA_VERSION, + scanRoots: Array.isArray(raw.scanRoots) ? raw.scanRoots.map(String).filter(Boolean) : [], + audiences: (Array.isArray(raw.audiences) ? raw.audiences : []).map(normalizeAudience).filter(Boolean), + remote: String(raw.remote || '').trim(), + defaults: { + visibility: String(raw?.defaults?.visibility || 'private'), + groups: Array.isArray(raw?.defaults?.groups) ? raw.defaults.groups.map(kebab).filter(Boolean) : [] + } }; +} + +function saveConfig(config) { + return writeJson(configPath(), { + schemaVersion: SCHEMA_VERSION, + scanRoots: config?.scanRoots || [], + audiences: (config?.audiences || []).map(normalizeAudience).filter(Boolean), + remote: String(config?.remote || '').trim(), + defaults: config?.defaults || { visibility: 'private', groups: [] } + }); +} + +function entryPath(id) { + return path.join(entriesDir(), `${kebab(id)}.json`); +} + +function loadEntries() { + const entries = {}; + + let files = []; + try { + files = fs.readdirSync(entriesDir()).filter((name) => name.endsWith('.json')); + } catch { + files = []; + } + + for (const file of files) { + const raw = readJson(path.join(entriesDir(), file), null); + const id = kebab(raw?.id || path.basename(file, '.json')); + if (!id) continue; + entries[id] = { ...normalizeEntry({ ...raw, id }), id }; + } + + return entries; +} + +/** + * Fold a pre-split `registry.json` into per-entry files. Runs once, keeps the + * old file as a backup, and is a no-op afterwards. + */ +function migrateLegacyRegistry() { + const legacy = readJson(legacyRegistryPath(), null); + if (!legacy) return { migrated: 0 }; + if (fs.existsSync(configPath())) return { migrated: 0, reason: 'already migrated' }; - const entries = raw.entries && typeof raw.entries === 'object' ? raw.entries : {}; - for (const [key, value] of Object.entries(entries)) { + ensureDir(entriesDir()); + let migrated = 0; + for (const [key, value] of Object.entries(legacy.entries || {})) { const id = kebab(value?.id || key); if (!id) continue; - registry.entries[id] = { ...normalizeEntry({ ...value, id }), id }; + writeJson(entryPath(id), { ...normalizeEntry({ ...value, id }), id }); + migrated += 1; + } + + saveConfig({ + scanRoots: legacy.scanRoots || [], + audiences: legacy.audiences || [], + remote: legacy.remote || '', + defaults: legacy.defaults + }); + + try { + fs.renameSync(legacyRegistryPath(), `${legacyRegistryPath()}.migrated`); + } catch { + // Keeping the original in place is harmless — config presence gates re-runs. } - return registry; + return { migrated }; +} + +function loadRegistry() { + migrateLegacyRegistry(); + const config = loadConfig(); + return { ...emptyRegistry(), ...config, entries: loadEntries() }; } function saveRegistry(registry) { - const next = { - schemaVersion: SCHEMA_VERSION, - scanRoots: registry?.scanRoots || [], - audiences: (registry?.audiences || []).map(normalizeAudience).filter(Boolean), - defaults: registry?.defaults || { visibility: 'private', groups: [] }, - entries: registry?.entries || {} - }; - return writeJson(registryPath(), next); + saveConfig(registry); + for (const [id, entry] of Object.entries(registry?.entries || {})) { + writeJson(entryPath(id), { ...entry, id }); + } + return registryDir(); } function upsertRegistryEntry(id, patch) { - const registry = loadRegistry(); const key = kebab(id); if (!key) throw new Error('An atlas entry needs an id'); - const existing = registry.entries[key] || { id: key }; - registry.entries[key] = { ...existing, ...normalizeEntry({ ...patch, id: key }), id: key }; - saveRegistry(registry); - return registry.entries[key]; + + migrateLegacyRegistry(); + const existing = readJson(entryPath(key), null) || { id: key }; + const next = { ...existing, ...normalizeEntry({ ...patch, id: key }), id: key }; + writeJson(entryPath(key), next); + return next; } function removeRegistryEntry(id) { - const registry = loadRegistry(); - const key = kebab(id); - const existed = Boolean(registry.entries[key]); - delete registry.entries[key]; - saveRegistry(registry); - return existed; + const target = entryPath(id); + if (!fs.existsSync(target)) return false; + fs.unlinkSync(target); + return true; } function loadDiscoveryCache({ maxAgeMs = DISCOVERY_CACHE_TTL_MS } = {}) { @@ -170,14 +266,62 @@ function saveBundle(audienceId, bundle, outputPath = '') { return written; } +/** + * Bundles published by other people. Read-only, lowest precedence, and always + * attributed — a teammate's map should never silently overwrite your own notes. + */ +function loadSubscriptions() { + let files = []; + try { + files = fs.readdirSync(subscriptionsDir()).filter((name) => name.endsWith('.json')); + } catch { + return []; + } + + const bundles = []; + for (const file of files) { + const raw = readJson(path.join(subscriptionsDir(), file), null); + if (!raw || !Array.isArray(raw.entries)) continue; + bundles.push({ + name: path.basename(file, '.json'), + audience: raw.audience || '', + generatedAt: raw.generatedAt || null, + entries: raw.entries + }); + } + return bundles; +} + +function saveSubscription(name, bundle) { + return writeJson(path.join(subscriptionsDir(), `${kebab(name)}.json`), bundle); +} + +function removeSubscription(name) { + const target = path.join(subscriptionsDir(), `${kebab(name)}.json`); + if (!fs.existsSync(target)) return false; + fs.unlinkSync(target); + return true; +} + module.exports = { MANIFEST_FILENAME, + CONFIG_FILENAME, DISCOVERY_CACHE_TTL_MS, atlasDir, - registryPath, + registryDir, + entriesDir, + configPath, + legacyRegistryPath, + registryPath: configPath, discoveryCachePath, bundlesDir, + subscriptionsDir, emptyRegistry, + loadConfig, + saveConfig, + loadEntries, + entryPath, + migrateLegacyRegistry, loadRegistry, saveRegistry, upsertRegistryEntry, @@ -188,6 +332,9 @@ module.exports = { loadManifest, writeManifest, saveBundle, + loadSubscriptions, + saveSubscription, + removeSubscription, readJson, writeJson }; diff --git a/server/atlas/atlasSync.js b/server/atlas/atlasSync.js new file mode 100644 index 00000000..12319480 --- /dev/null +++ b/server/atlas/atlasSync.js @@ -0,0 +1,256 @@ +const fs = require('fs'); +const path = require('path'); +const { execFile } = require('child_process'); + +const store = require('./atlasStore'); +const { kebab } = require('./atlasSchema'); + +const GIT_TIMEOUT_MS = 60_000; + +const REGISTRY_README = `# Repo Atlas — private registry + +Your judgement about your repos: what each one is worth reading for, scored per +topic, plus who each entry may be shared with. + +One file per repo under \`entries/\` so two machines curating different repos +never produce a merge conflict. + +**Keep this repository private.** It describes repos your collaborators may not +have access to. Compiled bundles — the subsets you actually share — are produced +with \`atlas publish \` and go somewhere else entirely. +`; + +const REGISTRY_GITIGNORE = `# Machine-local: what this particular computer happens to have cloned. +discovery.json +bundles/ +subscriptions/ +*.migrated +`; + +function git(args, { cwd, timeout = GIT_TIMEOUT_MS } = {}) { + return new Promise((resolve) => { + execFile('git', args, { cwd, timeout, windowsHide: true, maxBuffer: 4 * 1024 * 1024 }, (error, stdout, stderr) => { + resolve({ + ok: !error, + stdout: String(stdout || '').trim(), + stderr: String(stderr || '').trim(), + error: error ? error.message : null + }); + }); + }); +} + +function isGitRepo(dir) { + return fs.existsSync(path.join(dir, '.git')); +} + +async function ensureRepo(dir, remote) { + fs.mkdirSync(dir, { recursive: true }); + + if (!isGitRepo(dir)) { + const init = await git(['init', '-b', 'main'], { cwd: dir }); + if (!init.ok) return { ok: false, error: `git init failed: ${init.stderr || init.error}` }; + } + + if (remote) { + const current = await git(['remote', 'get-url', 'origin'], { cwd: dir }); + if (!current.ok) { + const added = await git(['remote', 'add', 'origin', remote], { cwd: dir }); + if (!added.ok) return { ok: false, error: `could not add remote: ${added.stderr || added.error}` }; + } else if (current.stdout !== remote) { + await git(['remote', 'set-url', 'origin', remote], { cwd: dir }); + } + } + + return { ok: true }; +} + +function seedRepoFiles(dir) { + const readme = path.join(dir, 'README.md'); + if (!fs.existsSync(readme)) fs.writeFileSync(readme, REGISTRY_README, 'utf8'); + const ignore = path.join(dir, '.gitignore'); + if (!fs.existsSync(ignore)) fs.writeFileSync(ignore, REGISTRY_GITIGNORE, 'utf8'); + fs.mkdirSync(path.join(dir, 'entries'), { recursive: true }); +} + +async function hasChanges(dir) { + const status = await git(['status', '--porcelain'], { cwd: dir }); + return status.ok && status.stdout.length > 0; +} + +async function commitAll(dir, message) { + await git(['add', '-A'], { cwd: dir }); + if (!(await hasChanges(dir))) return { committed: false }; + const commit = await git(['commit', '-m', message], { cwd: dir }); + return { committed: commit.ok, error: commit.ok ? null : (commit.stderr || commit.error) }; +} + +/** + * Pull, merge, push — the whole point of which is that the atlas survives you + * moving between machines, and survives the machine. + * + * Rebase is deliberate: per-entry files make conflicts rare, and when one does + * happen it is a single small JSON file you can read, not a merge commit in the + * middle of a registry blob. + */ +async function syncRegistry({ remote = '', message = '' } = {}) { + const dir = store.registryDir(); + const config = store.loadConfig(); + const target = remote || config.remote; + + if (!target) { + return { ok: false, error: 'No registry remote configured — run `atlas remote set ` first' }; + } + + const prepared = await ensureRepo(dir, target); + if (!prepared.ok) return { ok: false, error: prepared.error }; + seedRepoFiles(dir); + + const steps = []; + + const localCommit = await commitAll(dir, message || `atlas: sync from ${require('os').hostname()}`); + steps.push({ step: 'commit-local', ...localCommit }); + + const fetched = await git(['fetch', 'origin'], { cwd: dir }); + steps.push({ step: 'fetch', ok: fetched.ok, detail: fetched.stderr || null }); + + const branch = (await git(['rev-parse', '--abbrev-ref', 'HEAD'], { cwd: dir })).stdout || 'main'; + const remoteExists = (await git(['rev-parse', '--verify', `origin/${branch}`], { cwd: dir })).ok; + + if (remoteExists) { + const pulled = await git(['pull', '--rebase', 'origin', branch], { cwd: dir }); + steps.push({ step: 'pull', ok: pulled.ok, detail: pulled.stderr || pulled.stdout || null }); + if (!pulled.ok) { + await git(['rebase', '--abort'], { cwd: dir }); + return { + ok: false, + error: 'Registry pull conflicted. Resolve it by hand in the registry directory, then sync again.', + dir, + steps + }; + } + } + + const pushed = await git(['push', '-u', 'origin', branch], { cwd: dir }); + steps.push({ step: 'push', ok: pushed.ok, detail: pushed.stderr || null }); + + return { + ok: pushed.ok, + dir, + remote: target, + branch, + entryCount: Object.keys(store.loadEntries()).length, + steps, + error: pushed.ok ? null : `push failed: ${pushed.stderr || pushed.error}` + }; +} + +async function setRemote(remote) { + const config = store.loadConfig(); + config.remote = String(remote || '').trim(); + store.saveConfig(config); + if (config.remote) await ensureRepo(store.registryDir(), config.remote); + return config.remote; +} + +/** + * Clone someone else's published bundle so their map shows up in your searches, + * attributed to them and never overwriting your own notes. + */ +async function subscribe({ name, source }) { + const key = kebab(name); + if (!key) throw new Error('A subscription needs a name'); + const from = String(source || '').trim(); + if (!from) throw new Error('A subscription needs a path or git URL'); + + if (!/^(https?:|git@|ssh:)/.test(from)) { + const bundle = store.readJson(path.resolve(from), null); + if (!bundle || !Array.isArray(bundle.entries)) { + throw new Error(`${from} is not an atlas bundle`); + } + return { name: key, entryCount: bundle.entries.length, path: store.saveSubscription(key, bundle) }; + } + + const checkoutDir = path.join(store.subscriptionsDir(), '.repos', key); + const prepared = await ensureRepo(checkoutDir, from); + if (!prepared.ok) throw new Error(prepared.error); + + const fetched = await git(['fetch', 'origin'], { cwd: checkoutDir }); + if (!fetched.ok) throw new Error(`could not fetch ${from}: ${fetched.stderr}`); + const branch = (await git(['rev-parse', '--abbrev-ref', 'origin/HEAD'], { cwd: checkoutDir })).stdout.replace('origin/', '') || 'main'; + await git(['checkout', '-B', branch, `origin/${branch}`], { cwd: checkoutDir }); + + const candidates = fs.readdirSync(checkoutDir).filter((file) => /^atlas\..*\.json$/.test(file)); + if (!candidates.length) throw new Error(`no atlas bundle found in ${from}`); + + const bundle = store.readJson(path.join(checkoutDir, candidates[0]), null); + if (!bundle || !Array.isArray(bundle.entries)) throw new Error(`${candidates[0]} is not an atlas bundle`); + + return { name: key, entryCount: bundle.entries.length, path: store.saveSubscription(key, bundle) }; +} + +/** + * Write a compiled bundle into a repo the audience already has access to, and + * commit it. GitHub permissions on that repo are the actual access control — + * this just puts the file where they can see it. + */ +async function publishBundle({ audience, bundle, outputPath, outputRemote }) { + if (!outputPath) { + return { ok: false, error: `Audience "${audience}" has no outputPath — set one with \`atlas audience add ${audience} --out \`` }; + } + + const target = path.resolve(outputPath); + store.writeJson(target, bundle); + + const repoDir = outputRemote ? path.dirname(target) : null; + if (!repoDir) return { ok: true, path: target, committed: false }; + + const prepared = await ensureRepo(repoDir, outputRemote); + if (!prepared.ok) return { ok: true, path: target, committed: false, warning: prepared.error }; + + const committed = await commitAll(repoDir, `atlas: publish ${audience} bundle (${bundle.entryCount} repos)`); + if (!committed.committed) return { ok: true, path: target, committed: false, detail: 'nothing changed' }; + + const branch = (await git(['rev-parse', '--abbrev-ref', 'HEAD'], { cwd: repoDir })).stdout || 'main'; + const pushed = await git(['push', 'origin', branch], { cwd: repoDir }); + + return { ok: true, path: target, committed: true, pushed: pushed.ok, detail: pushed.ok ? null : pushed.stderr }; +} + +async function getSyncStatus() { + const dir = store.registryDir(); + const config = store.loadConfig(); + + if (!isGitRepo(dir)) { + return { tracked: false, remote: config.remote || null, dir, hint: 'run `atlas sync` to start tracking the registry in git' }; + } + + const [branch, status, ahead] = await Promise.all([ + git(['rev-parse', '--abbrev-ref', 'HEAD'], { cwd: dir }), + git(['status', '--porcelain'], { cwd: dir }), + git(['rev-list', '--count', '@{upstream}..HEAD'], { cwd: dir }) + ]); + + return { + tracked: true, + dir, + remote: config.remote || null, + branch: branch.stdout || null, + dirty: status.stdout.length > 0, + unpushed: ahead.ok ? Number(ahead.stdout) || 0 : null, + subscriptions: store.loadSubscriptions().map((sub) => ({ name: sub.name, entries: sub.entries.length, generatedAt: sub.generatedAt })) + }; +} + +module.exports = { + git, + isGitRepo, + ensureRepo, + seedRepoFiles, + commitAll, + syncRegistry, + setRemote, + subscribe, + publishBundle, + getSyncStatus +}; diff --git a/server/repoAtlasService.js b/server/repoAtlasService.js index 1ed0675e..2df62a1e 100644 --- a/server/repoAtlasService.js +++ b/server/repoAtlasService.js @@ -6,6 +6,7 @@ const store = require('./atlas/atlasStore'); const discovery = require('./atlas/atlasDiscovery'); const query = require('./atlas/atlasQuery'); const compiler = require('./atlas/atlasCompiler'); +const sync = require('./atlas/atlasSync'); const { getProjectsRoot, getLegacyProjectsRoot } = require('./utils/pathUtils'); const ATLAS_CACHE_MS = 60_000; @@ -81,9 +82,30 @@ class RepoAtlasService { const registry = store.loadRegistry(); const byId = new Map(); + + // Bundles other people published sit underneath everything: their map is + // useful, but your own discovery and judgement always win over it. + for (const bundle of store.loadSubscriptions()) { + for (const entry of bundle.entries) { + if (!entry?.id) continue; + const slot = byId.get(entry.id) || {}; + slot.subscription = { + ...entry, + __source: 'subscription', + foreign: true, + sharedBy: bundle.name, + cloned: false, + localPath: null + }; + byId.set(entry.id, slot); + } + } + for (const entry of discovered) { if (!entry?.id) continue; - byId.set(entry.id, { discovery: { ...entry, __source: 'discovery' } }); + const slot = byId.get(entry.id) || {}; + slot.discovery = { ...entry, __source: 'discovery' }; + byId.set(entry.id, slot); } for (const entry of discovered) { @@ -110,10 +132,15 @@ class RepoAtlasService { for (const [id, layers] of byId.entries()) { const merged = schema.mergeEntries( { id, visibility: registry.defaults?.visibility, groups: registry.defaults?.groups, __source: 'defaults' }, + layers.subscription, layers.discovery, layers.manifest, layers.registry ); + if (layers.subscription && !layers.discovery && !layers.registry) { + merged.foreign = true; + merged.sharedBy = layers.subscription.sharedBy; + } merged.id = id; merged.sources = (merged.sources || []).filter((s) => s !== 'defaults'); entries.push(merged); @@ -206,21 +233,24 @@ class RepoAtlasService { return store.loadRegistry().audiences || []; } - setAudience({ id, label = '', description = '', outputPath = '' } = {}) { - const registry = store.loadRegistry(); + setAudience({ id, label = '', description = '', outputPath = '', outputRemote = '' } = {}) { + const config = store.loadConfig(); const key = schema.kebab(id); if (!key) throw new Error('An audience needs an id'); - const audiences = (registry.audiences || []).filter((a) => a.id !== key); - audiences.push({ id: key, label: label || key, description, outputPath }); - registry.audiences = audiences; - store.saveRegistry(registry); + const audiences = (config.audiences || []).filter((a) => a.id !== key); + audiences.push({ id: key, label: label || key, description, outputPath, outputRemote }); + config.audiences = audiences; + store.saveConfig(config); this.invalidate(); return audiences; } compile(audience, { write = true } = {}) { const meta = this.listAudiences().find((a) => a.id === schema.kebab(audience)) || {}; - const result = compiler.compileBundle(this.getEntries(), { + // Never re-share what someone else shared with you — attribution and + // permission both belong to whoever published it. + const own = this.getEntries().filter((entry) => entry.foreign !== true); + const result = compiler.compileBundle(own, { audience, label: meta.label, description: meta.description @@ -229,6 +259,67 @@ class RepoAtlasService { return result; } + /** + * Pull, merge and push the registry. This is what makes the atlas survive + * working across machines — and survive the machine. + */ + async sync(options = {}) { + const result = await sync.syncRegistry(options); + this.invalidate(); + return result; + } + + async setRemote(remote) { + const value = await sync.setRemote(remote); + this.invalidate(); + return value; + } + + async getSyncStatus() { + return sync.getSyncStatus(); + } + + /** + * Publish an audience bundle into a repo that audience already has access to. + */ + async publish(audience, { push = true } = {}) { + const meta = this.listAudiences().find((a) => a.id === schema.kebab(audience)); + if (!meta) throw new Error(`Unknown audience "${audience}" — add it with \`atlas audience add ${audience}\``); + + const compiled = this.compile(audience, { write: true }); + const published = push + ? await sync.publishBundle({ + audience: meta.id, + bundle: compiled.bundle, + outputPath: meta.outputPath, + outputRemote: meta.outputRemote + }) + : { ok: true, committed: false, path: meta.outputPath || compiled.written[0], detail: 'push disabled' }; + + return { ...compiled, published }; + } + + async subscribe({ name, source }) { + const result = await sync.subscribe({ name, source }); + this.invalidate(); + return result; + } + + listSubscriptions() { + return store.loadSubscriptions().map((bundle) => ({ + name: bundle.name, + audience: bundle.audience, + generatedAt: bundle.generatedAt, + entryCount: bundle.entries.length + })); + } + + unsubscribe(name) { + const removed = store.removeSubscription(name); + this.invalidate(); + return removed; + } + validate() { const entries = this.getEntries(); const reports = entries.map((entry) => schema.validateEntry(entry)); @@ -268,12 +359,16 @@ class RepoAtlasService { const entries = this.getEntries(); return { atlasDir: store.atlasDir(), - registryPath: store.registryPath(), + registryDir: store.registryDir(), scanRoots: this.getScanRoots(), entryCount: entries.length, clonedCount: entries.filter((e) => e.cloned).length, + foreignCount: entries.filter((e) => e.foreign).length, + curatedCount: Object.keys(store.loadEntries()).length, highlightCount: entries.reduce((sum, e) => sum + (e.highlights || []).length, 0), audiences: this.listAudiences().map((a) => a.id), + subscriptions: this.listSubscriptions(), + remote: store.loadConfig().remote || null, discovery: meta ? { generatedAt: meta.generatedAt, stale: meta.stale, githubAvailable: meta.githubAvailable !== false } : null diff --git a/server/routes/atlasRoutes.js b/server/routes/atlasRoutes.js index 321f6e7b..53586fec 100644 --- a/server/routes/atlasRoutes.js +++ b/server/routes/atlasRoutes.js @@ -127,6 +127,39 @@ function createAtlasRoutes({ repoAtlasService, logger = console, requireRead = p res.json({ ok: true, audiences }); })); + router.get('/sync', requireRead, handle('sync status', async (req, res) => { + res.json({ ok: true, sync: await repoAtlasService.getSyncStatus() }); + })); + + router.post('/sync', requireWrite, handle('sync', async (req, res) => { + const result = await repoAtlasService.sync({ message: req.body?.message || '' }); + res.json({ ok: result.ok, ...result }); + })); + + router.post('/remote', requireWrite, handle('set remote', async (req, res) => { + res.json({ ok: true, remote: await repoAtlasService.setRemote(req.body?.remote || '') }); + })); + + router.post('/publish', requireWrite, handle('publish', async (req, res) => { + const audience = req.body?.audience; + if (!audience) return res.status(400).json({ ok: false, error: 'audience is required' }); + const result = await repoAtlasService.publish(audience, { push: req.body?.push !== false }); + return res.json({ ok: true, audience, counts: result.counts, published: result.published }); + })); + + router.get('/subscriptions', requireRead, handle('list subscriptions', (req, res) => { + res.json({ ok: true, subscriptions: repoAtlasService.listSubscriptions() }); + })); + + router.post('/subscriptions', requireWrite, handle('subscribe', async (req, res) => { + const result = await repoAtlasService.subscribe({ name: req.body?.name, source: req.body?.source }); + res.json({ ok: true, ...result }); + })); + + router.delete('/subscriptions/:name', requireWrite, handle('unsubscribe', (req, res) => { + res.json({ ok: true, removed: repoAtlasService.unsubscribe(req.params.name) }); + })); + router.post('/compile', requireWrite, handle('compile bundle', (req, res) => { const audience = req.body?.audience; if (!audience) return res.status(400).json({ ok: false, error: 'audience is required' }); diff --git a/tests/unit/repoAtlasService.test.js b/tests/unit/repoAtlasService.test.js index 8eef96d1..70cc5c00 100644 --- a/tests/unit/repoAtlasService.test.js +++ b/tests/unit/repoAtlasService.test.js @@ -143,6 +143,7 @@ describe('RepoAtlasService', () => { expect(status.entryCount).toBe(1); expect(status.clonedCount).toBe(1); expect(status.highlightCount).toBe(1); - expect(status.registryPath).toContain('registry.json'); + expect(status.registryDir).toContain('registry'); + expect(status.curatedCount).toBe(1); }); }); diff --git a/tests/unit/repoAtlasSync.test.js b/tests/unit/repoAtlasSync.test.js new file mode 100644 index 00000000..bd0e7d93 --- /dev/null +++ b/tests/unit/repoAtlasSync.test.js @@ -0,0 +1,179 @@ +const fs = require('fs'); +const os = require('os'); +const path = require('path'); +const { execFileSync } = require('child_process'); + +const RepoAtlasService = require('../../server/repoAtlasService'); +const store = require('../../server/atlas/atlasStore'); + +const git = (args, cwd) => execFileSync('git', args, { cwd, stdio: 'pipe' }); + +describe('Repo Atlas multi-machine sync', () => { + let root; + let remote; + + const machine = (name) => { + process.env.AGENT_WORKSPACE_ATLAS_DIR = path.join(root, name); + const atlas = new RepoAtlasService(); + atlas.invalidate(); + return atlas; + }; + + beforeEach(() => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-sync-')); + remote = path.join(root, 'remote.git'); + git(['init', '-q', '--bare', remote], root); + }); + + afterEach(() => { + delete process.env.AGENT_WORKSPACE_ATLAS_DIR; + fs.rmSync(root, { recursive: true, force: true }); + }); + + test('curated entries are one file per repo so machines cannot conflict', () => { + const atlas = machine('a'); + atlas.addHighlight('box2d-luau', { topic: 'testing', quality: 5 }); + atlas.addHighlight('zoo-game', { topic: 'networking', quality: 3 }); + + expect(fs.readdirSync(store.entriesDir()).sort()).toEqual(['box2d-luau.json', 'zoo-game.json']); + }); + + test('judgement travels between machines; local discovery does not', async () => { + const a = machine('a'); + await a.setRemote(remote); + a.addHighlight('box2d-luau', { topic: 'testing', quality: 5, notes: 'best harness we have' }); + store.saveDiscoveryCache([{ id: 'only-on-a', name: 'only-on-a', localPath: '/machine/a/only-on-a', cloned: true }]); + expect((await a.sync()).ok).toBe(true); + + const b = machine('b'); + await b.setRemote(remote); + expect((await b.sync()).ok).toBe(true); + + // B inherited the judgement… + const hits = b.find('testing'); + expect(hits).toHaveLength(1); + expect(hits[0].notes).toBe('best harness we have'); + + // …but not A's idea of what is on disk. + expect(b.getEntry('only-on-a')).toBeNull(); + }); + + test('two machines editing different repos merge without conflict', async () => { + const a = machine('a'); + await a.setRemote(remote); + a.addHighlight('repo-one', { topic: 'testing', quality: 5 }); + await a.sync(); + + const b = machine('b'); + await b.setRemote(remote); + await b.sync(); + b.addHighlight('repo-two', { topic: 'physics', quality: 4 }); + expect((await b.sync()).ok).toBe(true); + + const backOnA = machine('a'); + expect((await backOnA.sync()).ok).toBe(true); + expect(backOnA.topics().map((t) => t.topic).sort()).toEqual(['physics', 'testing']); + }); + + test('sync refuses to run without a remote rather than failing silently', async () => { + const result = await machine('a').sync(); + expect(result.ok).toBe(false); + expect(result.error).toMatch(/No registry remote configured/); + }); + + test('a legacy single-file registry migrates into per-entry files', () => { + process.env.AGENT_WORKSPACE_ATLAS_DIR = path.join(root, 'legacy'); + store.writeJson(store.legacyRegistryPath(), { + schemaVersion: 1, + audiences: [{ id: 'core-team', label: 'Core team' }], + entries: { + 'old-repo': { id: 'old-repo', summary: 'from before the split', highlights: [{ topic: 'testing', quality: 4 }] } + } + }); + + const atlas = new RepoAtlasService(); + const entry = atlas.getEntry('old-repo'); + + expect(entry.summary).toBe('from before the split'); + expect(fs.existsSync(store.entryPath('old-repo'))).toBe(true); + expect(atlas.listAudiences().map((a) => a.id)).toEqual(['core-team']); + }); + + describe('subscriptions', () => { + const publishBundle = (dir, entries) => { + const file = path.join(dir, 'atlas.core-team.json'); + store.writeJson(file, { schemaVersion: 1, audience: 'core-team', entryCount: entries.length, entries }); + return file; + }; + + test('a teammate can search what was shared with them', async () => { + const shared = publishBundle(root, [{ + id: 'their-repo', + name: 'their-repo', + summary: 'shared with me', + visibility: 'team', + highlights: [{ topic: 'auth', quality: 4, notes: 'clean oauth flow' }] + }]); + + const me = machine('me'); + await me.subscribe({ name: 'them', source: shared }); + + const hits = me.find('auth'); + expect(hits).toHaveLength(1); + expect(hits[0].cloned).toBe(false); + expect(me.getEntry('their-repo').sharedBy).toBe('them'); + }); + + test('what someone shared with you is never re-shared by you', async () => { + const shared = publishBundle(root, [{ + id: 'their-repo', + name: 'their-repo', + visibility: 'public', + highlights: [{ topic: 'auth', quality: 4 }] + }]); + + const me = machine('me'); + await me.subscribe({ name: 'them', source: shared }); + me.addHighlight('my-repo', { topic: 'testing', quality: 5 }); + me.setEntry('my-repo', { visibility: 'public' }); + + const compiled = me.compile('anyone', { write: false }); + expect(compiled.bundle.entries.map((e) => e.id)).toEqual(['my-repo']); + }); + + test('your own notes outrank anything a subscription says about the same repo', async () => { + const shared = publishBundle(root, [{ + id: 'shared-repo', + name: 'shared-repo', + summary: 'their description', + visibility: 'public', + highlights: [{ topic: 'testing', quality: 2 }] + }]); + + const me = machine('me'); + await me.subscribe({ name: 'them', source: shared }); + me.addHighlight('shared-repo', { topic: 'testing', quality: 5, notes: 'actually excellent' }); + + const entry = me.getEntry('shared-repo'); + expect(entry.highlights[0].quality).toBe(5); + expect(entry.summary).toBe('their description'); + expect(entry.foreign).toBeUndefined(); + }); + + test('subscribing to something that is not a bundle fails loudly', async () => { + const notABundle = path.join(root, 'nope.json'); + store.writeJson(notABundle, { hello: 'world' }); + await expect(machine('me').subscribe({ name: 'x', source: notABundle })).rejects.toThrow(/not an atlas bundle/); + }); + + test('unsubscribing removes those repos from search', async () => { + const shared = publishBundle(root, [{ id: 'their-repo', name: 'their-repo', highlights: [{ topic: 'auth', quality: 4 }] }]); + const me = machine('me'); + await me.subscribe({ name: 'them', source: shared }); + expect(me.find('auth')).toHaveLength(1); + + expect(me.unsubscribe('them')).toBe(true); + expect(me.find('auth')).toEqual([]); + }); + }); +}); From de95f0b59f15167f01156627f027b27d7a324487 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:27:31 +1000 Subject: [PATCH 08/69] =?UTF-8?q?docs:=20research=20=E2=80=94=20Codex=20ap?= =?UTF-8?q?p-server,=20Hermes=20Agent,=20and=20the=20Discord=20rebuild?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Codex finding is the significant one. openai/codex is Apache 2.0 and the CLI already installed here ships `codex app-server`: JSON-RPC 2.0 over stdio/websocket/unix socket, the same interface that powers the VS Code extension and the Codex app. It emits, as structured events, everything the supervisor currently reconstructs by regex-scraping terminal output — requestApproval instead of matching 'Do you want to proceed', turn/completed instead of spotting a cost line, thread/status/changed instead of buffer-growth heuristics — plus token usage and rate limits we cannot see at all today. And thread/realtime/* is the full-duplex voice pipeline OpenAI shipped on 2026-07-23, WebRTC SDP included. Answers: yes there is an API (SDK + app-server); no reverse engineering needed; and the broader play is real — the Codex app is Codex-only and macOS-only, so speaking app-server for Codex while keeping PTY scraping as the universal fallback beats both products. Hermes Agent: model-agnostic and genuinely good, but it cannot run on the Codex subscription (it wants an OpenAI-compatible endpoint, which the CLI is not), and its runtime duplicates ours without knowing worktrees, tiers or queue state. Its messaging gateway and self-improving skill loop are worth stealing as patterns. Verdict: no. Discord: the current file-drop queue only sees what was explicitly queued and loses anything that arrived while it was down. Replacement is cursor-based polling (a missed message stops being possible rather than being retried), ambient extraction of work items, and status publishing so 'is their agent working on it' stops being a question anyone has to ask. --- .../RESEARCH_HERMES_CODEX_AND_DISCORD.md | 193 ++++++++++++++++++ 1 file changed, 193 insertions(+) create mode 100644 PLANS/2026-07-26/RESEARCH_HERMES_CODEX_AND_DISCORD.md diff --git a/PLANS/2026-07-26/RESEARCH_HERMES_CODEX_AND_DISCORD.md b/PLANS/2026-07-26/RESEARCH_HERMES_CODEX_AND_DISCORD.md new file mode 100644 index 00000000..62e2cb2c --- /dev/null +++ b/PLANS/2026-07-26/RESEARCH_HERMES_CODEX_AND_DISCORD.md @@ -0,0 +1,193 @@ +# Research: Hermes Agent, the Codex app-server, and the Discord rebuild (2026-07-26) + +Three questions, one conclusion: the most valuable thing found here is not a product to adopt, +it is a **protocol we can already speak**. + +--- + +## 1. The Codex finding (this is the important one) + +`openai/codex` is **Apache 2.0 and open source**, and the CLI already installed on this machine +ships a component called `codex app-server`. From its own README: + +> Similar to MCP, `codex app-server` supports bidirectional communication using JSON-RPC 2.0 +> messages. Supported transports: stdio, websocket, unix socket. + +This is the interface that powers the Codex VS Code extension and the Codex app. It is documented, +versioned (`v1`/`v2` protocol modules), and speakable by anything that can write JSON lines. + +### What it exposes that we currently guess at + +The supervisor today infers agent state by regex-scraping terminal output — "Do you want to +proceed" means a permission prompt, a cost line means a turn ended. The app-server emits these as +**structured events**: + +| We currently scrape | app-server emits | +|---|---| +| "Do you want to proceed?" | `item/commandExecution/requestApproval` | +| Cost/summary line = done | `turn/completed`, `item/completed` | +| Busy/idle heuristics on buffer growth | `thread/status/changed` | +| Nothing — invisible to us | `thread/tokenUsage/updated` | +| Nothing — invisible to us | `turn/plan/updated`, `turn/diff/updated` | +| Guessed from banner text | `account/rateLimits/updated` | + +Every one of those is a supervisor signal we are currently reconstructing unreliably from a byte +stream. Approval prompts in particular: instead of pattern-matching prose that changes between +releases, we would receive the actual command and answer it over the wire. + +### And the voice pipeline is right there + +``` +thread/realtime/start thread/realtime/appendAudio thread/realtime/outputAudio/delta +thread/realtime/appendText thread/realtime/transcript/delta thread/realtime/transcript/done +thread/realtime/listVoices thread/realtime/sdp thread/realtime/stop +``` + +`sdp` means WebRTC. This is the full-duplex voice OpenAI shipped to the Codex desktop app on +2026-07-23 — the same thing described as "orchestrate multi-threaded coding jobs by voice" — and it +is addressable locally. + +### Answering the three questions directly + +1. **Is there an API?** Two. The **Codex SDK** (TypeScript and Python) embeds the agent in your own + app. The **app-server protocol** drives a local Codex the way the official app does. Codex CLI + can also run as an MCP server. +2. **Can we reverse-engineer it?** No need. It is Apache 2.0 with the protocol documented in-repo. +3. **Can we do broader/better?** Yes, and this is the actual opportunity. **The Codex app is + Codex-only, and macOS-only.** Ours is agent-agnostic and cross-platform. So: + +> **Speak app-server for Codex sessions to get structured signals; keep PTY scraping as the +> universal fallback for Claude, Gemini, aider and anything else. One supervisor, best-available +> signal per agent.** + +That is a strictly better position than either product: OpenAI cannot generalise to Claude, and we +would not be throwing away the agent-agnostic layer to get the fidelity. + +### Cost note + +The app-server drives the **local Codex CLI**, which bills the Codex subscription — the same +"drive the CLI, don't call the API" property the rest of this system relies on. + +### Recommended next step + +An adapter seam in the supervisor: `signalSource: 'app-server' | 'pty'` per session, resolved from +the agent registry. PTY stays the default and the fallback; nothing regresses if the app-server is +unavailable. Sized at roughly a day, and it upgrades every condition in the rule table at once. + +--- + +## 2. Hermes Agent — worth knowing, not worth adopting + +[Hermes Agent](https://hermes-agent.nousresearch.com/docs/) (Nous Research, MIT, launched 2026-02-25) +is model-agnostic and self-hostable on Linux, macOS, WSL2, Windows and Android/Termux. It runs as a +CLI, a desktop app, an **OpenAI-compatible API server**, and a **gateway across 20+ messaging +platforms** — Telegram, Discord, Slack, WhatsApp, Teams. It works with Nous Portal, OpenRouter, +OpenAI, Anthropic, Gemini, DeepSeek, Qwen, or any OpenAI-compatible endpoint including Ollama. + +**Can Codex run it?** Not in the sense of "Hermes powered by your Codex subscription". Hermes wants +an OpenAI-compatible `/v1/chat/completions` endpoint; the Codex CLI is not one. You could put a +proxy in between, but then you are paying per token through whatever the proxy talks to, which +throws away the subscription-billing advantage that makes continuous autonomy affordable here. + +**Is it still beneficial?** Two things about it are genuinely interesting, and neither requires +adopting it: + +1. **The messaging gateway.** 20+ platforms with one integration is real engineering we would not + want to redo. If the Discord bridge ever needs to become a Slack/Telegram/WhatsApp bridge, look + here first — as a component, behind our own work model. +2. **The self-improving skill loop** (agent-curated `MEMORY.md`, skills it writes and then refines + during use). That is the same instinct as Atlas write-back: the system recording what it learned + so the next run starts smarter. Worth stealing as a pattern. + +**What is not useful:** its agent runtime. We already have one, and ours knows what a worktree is, +what tier a task is, and which PR is waiting on evidence. Running a second runtime that knows none +of that adds a process without adding capability. + +**Verdict: no.** Revisit only if multi-platform messaging becomes the requirement. + +--- + +## 3. Discord: what is wrong and what replaces it + +### What exists today + +`server/discordIntegrationService.js` is a **file-drop queue**. An external bot repo writes +`~/.claude/discord-queue/pending-tasks.json`; the orchestrator ensures a Services workspace and +sends a processing prompt to a Claude terminal. It has real hardening (signed queue verification, +idempotency keys, an audit log) but the *shape* is wrong: + +- **It only sees what was explicitly queued.** Someone has to address the bot. Ordinary conversation + — which is where the actual assignments happen — is invisible. +- **A restart loses whatever arrived while it was down.** There is no cursor and no backfill. +- **Hardcoded paths** to another repo's queue directory. +- **It runs on one laptop**, so "is the bot up?" is a question with a real answer. + +### The team-coordination gap underneath it + +The tooling is Discord + Trello + GitHub, and between them nothing answers: + +- What is the **priority** of what I just asked someone to do? +- Is anyone **working on it**, right now? +- Is their **agent** running, or did they forget to prompt it? +- Did a ticket ever get **created**? +- Did the work **land**? + +Every one of those is knowable — the orchestrator already knows session status, tier, branch, and PR +state — it is just never published anywhere the team can see. + +### The replacement, in three parts + +**a) Durable ingest, cursor-based.** Poll `GET /channels/{id}/messages?after={lastSeenId}` instead +of holding a gateway socket. This is the fix for "it couldn't pick up what it missed", and it is a +fix by *construction* rather than by retry logic: there is no such thing as a missed message when +your read position is persisted. A restart after three days is just a longer page-through. Polling a +team chat every 10 seconds is entirely adequate and removes a whole class of connection-state bugs. + +**b) Ambient extraction.** Read every message, not just mentions. A cheap rule pass finds the +obvious cases (a mention plus an imperative, a question directed at someone, a link to a PR or +card). Only the ambiguous middle goes to a model, batched — same rules-first/LLM-on-event economics +as the supervisor. Output is a **work item**: who, what, priority, source message, permalink. + +**c) Status publishing — the part that closes the loop.** When a session picks up a work item, the +orchestrator posts back to the thread. When the branch pushes, when the PR opens, when it merges. +Nobody types a status update; the status *is* the system's own knowledge, published. "Is their agent +working on it" stops being a question you have to ask. + +### Where does it run, and on whose computer + +Honest answer: **it does not need a VPS to start, and it should not start with one.** + +- The **ingest is stateless given its cursor.** Whoever's orchestrator is designated the hub polls + and publishes. If that machine sleeps, nothing is lost — it catches up on wake. That is a very + different failure mode from a dropped gateway connection. +- **Everyone else's orchestrator only publishes its own status**, which needs no inbound + connectivity at all. +- The upgrade path, if and when the hub machine being asleep becomes annoying: move *only* the + ingest and cursor to a cheap always-on box. It is a poller with a JSON file. The agents stay on + the machines that have the code and the credentials — those never move, because that is where the + work is. + +Deliberately **not** proposed: a shared server that runs agents. Auth doesn't travel, worktrees +don't travel, and the moment the brain is remote you are proxying every signal it needs back to it. + +--- + +## 4. What this changes about the roadmap + +1. **Discord ambient ingest + work items + status publishing** — replaces the file-drop queue. +2. **App-server adapter for Codex sessions** — upgrades every supervisor condition at once by + replacing scraped signals with structured ones. Highest ratio of capability to effort on this + list. +3. **Realtime voice via `thread/realtime/*`** — full-duplex voice for Codex threads, using the same + pipeline OpenAI shipped, without being locked to their app or to macOS. +4. **Atlas write-back** (agents propose highlights from work they just did) — the Hermes + self-improving-skills idea, applied to the map. + +## Sources + +- [openai/codex](https://github.com/openai/codex) — Apache 2.0; `codex-rs/app-server/README.md` documents the protocol +- [Codex SDK](https://developers.openai.com/codex/sdk) · [Codex with the Agents SDK](https://developers.openai.com/codex/guides/agents-sdk) +- [Introducing the Codex app](https://openai.com/index/introducing-the-codex-app/) +- [VentureBeat — GPT-Live full-duplex voice control comes to Codex](https://venturebeat.com/orchestration/agentic-coding-goes-hands-free-as-openai-brings-gpt-lives-full-duplex-voice-control-to-codex-and-chatgpt-on-the-desktop) +- [Hermes Agent docs](https://hermes-agent.nousresearch.com/docs/) · [AI providers](https://hermes-agent.nousresearch.com/docs/integrations/providers) +- [Hermes Agent vs OpenClaw comparison](https://contabo.com/blog/hermes-agent-vs-openclaw-paperclip-and-the-best-open-source-ai-agents-in-2026/) From fae416612ae7ae44a1198f67d2be31519b6223f9 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:33:16 +1000 Subject: [PATCH 09/69] feat(discord): ambient watcher that tracks team work and publishes status MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Replaces the file-drop queue. That design only saw what was explicitly queued, lost anything that arrived while it was down, and hardcoded paths into another repo — so the assignments that actually happen in conversation were invisible. INGEST — cursor-based, not gateway-based Polls `GET /channels/{id}/messages?after={lastSeenId}` with the position persisted. A message missed while the process was down stops being a thing that can happen, rather than a thing retry logic has to handle: a restart after three days is just a longer page-through. Verified by killing and rebuilding the service mid-test and confirming it resumes from the cursor without replaying. EXTRACTION — read everything, track what matters Rules over every message, LLM-free: a mention plus a request is an assignment even though nobody filed a ticket; 'urgent'/'prod is down' is tier 1 and 'when you get a chance' is tier 4; 'on it' claims the open item and 'done' closes it rather than each creating more noise. Questions, bots and one-word replies are ignored on purpose. STATUS — the gap underneath the whole ask The orchestrator already knows session status, tier, branch and PR state; it was just never published anywhere the team could see. `linkSession()` binds a work item to the session doing it, writes a task record with the right tier, and announces it in-thread. 'Is their agent working on it' stops being a question anyone has to ask, and /untracked is the list of what was asked for that nobody has started — which until now existed only in scrollback. Off by default; needs DISCORD_BOT_TOKEN and channel ids. 783 unit tests green. --- CODEBASE_DOCUMENTATION.md | 43 ++- config/discord-watch.json | 40 +++ docs/COMMANDER_CLAUDE.md | 38 ++- server/discord/discordClient.js | 120 ++++++++ server/discord/workExtractor.js | 189 +++++++++++++ server/discordWatchService.js | 376 +++++++++++++++++++++++++ server/index.js | 15 + server/routes/discordWatchRoutes.js | 88 ++++++ tests/unit/discordWatchService.test.js | 278 ++++++++++++++++++ 9 files changed, 1182 insertions(+), 5 deletions(-) create mode 100644 config/discord-watch.json create mode 100644 server/discord/discordClient.js create mode 100644 server/discord/workExtractor.js create mode 100644 server/discordWatchService.js create mode 100644 server/routes/discordWatchRoutes.js create mode 100644 tests/unit/discordWatchService.test.js diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index ec9af242..a79990ac 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -126,14 +126,17 @@ server/commanderService.js - Top-level Commander PTY (Claude/Codex) + la ├─ Packaged CWD: uses `ORCHESTRATOR_DATA_DIR/commander` so desktop users can edit `CLAUDE.md` / `AGENTS.md` safely └─ First-run seed: copies the packaged `docs/COMMANDER_CLAUDE.md` into the Commander data directory when missing server/repoAtlasService.js - Repo Atlas facade: one queryable map of every repo you own, cloned or not -├─ Layers (later wins): discovery (disk + `gh repo list`) < in-repo `.repo-atlas.json` manifest < `~/.agent-workspace/atlas/registry.json` (your override) +├─ Layers (later wins): subscriptions (bundles others shared) < discovery (disk + `gh repo list`) < in-repo `.repo-atlas.json` < your registry override ├─ Query: `find(topic)` ranked by per-topic quality 1-5, `digest()` compact paste-into-a-prompt map, `search()`, `topics()` ├─ Curation: `addHighlight()` / `addAvoid()` persist into the registry — quality is scored per topic, so a rough repo can still be the best example of one thing └─ Sharing: `compile(audience)` emits audience-scoped bundles — `private` never leaves the machine, `team` needs a group match, `public` goes everywhere server/supervisorService.js - Fleet supervisor: rule-driven watchdog over every agent session ├─ Loop: rules run on a tick (default 30s) from zero-token signals; no model is called in the loop, only on escalation -├─ Ladder: observe → notify → nudge → act, capped by autonomy (`off` | `observe` (default) | `assist` | `autopilot`) -├─ Safety: shipped conditions never reach `act`; act handlers are named functions, so rules cannot inject shell +├─ Ordering: fix it itself → hand it to the Commander → only then interrupt a human +├─ Autonomy (`off` | `observe` | `assist` | `autopilot`, default autopilot; `SUPERVISOR_AUTONOMY` overrides) governs what it may FIX, not what it may say +├─ Escalation: a finding cannot reach a human until `escalateAfterAttempts` self-heals have failed +├─ Interruption: urgency = severity x task tier + failed repairs; gated by a budget (2/hour, 15min apart, optional quiet hours); anything refused batches into a digest +├─ Safety: named handlers only, so rules cannot inject shell; permission auto-approval fails closed on credentials/force-push/merge ├─ Audit: every action appended to `~/.agent-workspace/logs/supervisor-audit.jsonl` with the finding that caused it └─ `getBriefing()` renders the spoken/at-a-glance "what needs you now" summary server/supervisor/supervisorSignals.js - Per-session signal collection (PTY tail, quiet-time tracker, repeated-line detection, git ahead/dirty for quiet sessions only) @@ -152,6 +155,22 @@ client/speech-output.js - Web Speech API listener for the browser bac server/voiceCommandService.js - (existing) rule/LLM voice parsing, now with `setCommanderForwarder()`: unmatched speech is handed to the Commander agent instead of dead-ending tests/unit/speechService.test.js - Sanitization, repeat suppression, backend resolution +server/supervisor/supervisorUrgency.js - Urgency scoring (severity x task tier + failed-repair weight), interruption budget, digest queue +server/discordWatchService.js - Ambient Discord watching: read the conversation, track the work, publish status back +├─ Ingest: cursor-based polling (`?after=`), so a message missed while the process was down is not possible rather than retried +├─ Extraction: rules over every message -> work items with assignee, priority and tier; claim/done/drop update existing items +├─ Publishing: `linkSession()` binds an item to the session doing it, writes a task record, and announces it in-channel +└─ State: `~/.agent-workspace/discord/discord-watch.json` (cursors + items + member names) +server/discord/discordClient.js - Minimal Discord REST client (paged `after` reads, rate-limit backoff, reply-threaded posts) +server/discord/workExtractor.js - Message -> work item classification (`config/discord-watch.json`, override `~/.agent-workspace/discord-watch.json`) +server/routes/discordWatchRoutes.js - Express router for `/api/discord-watch/*` +config/discord-watch.json - Priority/kind/claim/done patterns, watched channels, poll cadence +tests/unit/discordWatchService.test.js - Extraction, cursor durability, restart resume, status publishing + +server/atlas/atlasSync.js - Git operations for the atlas: registry sync, audience publishing, subscriptions +├─ Registry lives in a PRIVATE git repo (`atlas remote set` + `atlas sync`) so judgement follows you between machines +├─ One file per repo under `entries/` — two machines curating different repos never conflict +└─ `subscribe` reads a bundle someone else published; foreign entries are lowest precedence and are never re-shared server/atlas/atlasSchema.js - Entry normalization, layered merge, topic-alias folding (`config/repo-atlas-topics.json`), validation server/atlas/atlasDiscovery.js - Local git scan (worktree siblings collapse into one project entry) + `gh repo list` + source merge server/atlas/atlasStore.js - Persistence under `~/.agent-workspace/atlas/` (registry, discovery cache, compiled bundles) + in-repo manifest read/write @@ -712,6 +731,24 @@ POST /api/supervisor/start | /stop - Control the loop POST /api/supervisor/autonomy - Set autonomy (`off` | `observe` | `assist` | `autopilot`) POST /api/supervisor/reload-rules - Re-read the condition table from disk +GET /api/supervisor/digest - What is waiting but did not earn an interruption +POST /api/supervisor/digest/deliver - "Catch me up" — deliver the batch now +POST /api/supervisor/interruption-policy - Tune threshold/budget/quiet hours + +GET /api/atlas/sync | POST /api/atlas/sync - Registry git status / pull+merge+push +POST /api/atlas/remote - Point the registry at a private git repo +POST /api/atlas/publish - Compile + commit an audience bundle where they can read it +GET|POST /api/atlas/subscriptions - Read bundles other people published +DELETE /api/atlas/subscriptions/:name - Stop reading one + +GET /api/discord-watch/status - Watcher state, cursors, item counts +GET /api/discord-watch/items?status=&assignee=&channelId= - Tracked work extracted from chat +GET /api/discord-watch/untracked - Asked for, nobody started — ordered by priority +POST /api/discord-watch/poll | /start | /stop - Control the watcher +GET|POST /api/discord-watch/channels - Which channels are watched +POST /api/discord-watch/items/:id/link - Bind an item to the session doing it (and announce it) +POST /api/discord-watch/items/:id/status - Post a status update back into the channel + GET /api/speech/status - Enabled state, resolved backend, available backends, listeners POST /api/speech/say - Speak text (`priority: high` interrupts, `force` skips repeat suppression) POST /api/speech/backend - Choose a backend diff --git a/config/discord-watch.json b/config/discord-watch.json new file mode 100644 index 00000000..72b30cb6 --- /dev/null +++ b/config/discord-watch.json @@ -0,0 +1,40 @@ +{ + "$comment": "Ambient Discord watching: what counts as work, and how urgent. Override per-machine with ~/.agent-workspace/discord-watch.json.", + "schemaVersion": 1, + + "enabled": false, + "pollSeconds": 15, + "$comment_channels": "Channel ids to watch. Add via `POST /api/discord-watch/channels` or here.", + "channels": [], + "$comment_backfill": "On first sight of a channel, how far back to read. 0 = start from now and never look back.", + "backfillMessages": 50, + + "$comment_publish": "Post status updates back into the channel so nobody has to ask whether an agent picked something up.", + "publishStatus": true, + + "$comment_priority": "First match wins, most urgent first. Matched against the message text, case-insensitively.", + "priority": [ + { "level": "urgent", "tier": 1, "patterns": ["\\burgent\\b", "\\basap\\b", "\\bcritical\\b", "\\bblocker\\b", "\\bprod(uction)? (is )?(down|broken)\\b", "\\bp0\\b", "🚨"] }, + { "level": "high", "tier": 2, "patterns": ["\\bimportant\\b", "\\bhigh priority\\b", "\\btoday\\b", "\\bbefore (the )?(demo|release|launch)\\b", "\\bp1\\b"] }, + { "level": "low", "tier": 4, "patterns": ["\\bwhen you (get a chance|have time)\\b", "\\bno rush\\b", "\\blow priority\\b", "\\bsomeday\\b", "\\bnice to have\\b", "\\bbacklog\\b"] } + ], + "defaultPriority": { "level": "normal", "tier": 3 }, + + "$comment_kind": "What sort of message this is. Drives whether it becomes trackable work at all.", + "kinds": [ + { "kind": "bug", "trackable": true, "patterns": ["\\bbug\\b", "\\bbroken\\b", "\\bcrash(ing|ed)?\\b", "\\berror\\b", "\\bfail(ing|ed|s)\\b", "\\bregression\\b", "doesn'?t work"] }, + { "kind": "assignment", "trackable": true, "patterns": ["\\b(can|could|would) you\\b", "\\bplease\\b", "\\bneed(s)? (you )?to\\b", "\\blet'?s\\b", "\\bmake sure\\b", "\\badd\\b", "\\bfix\\b", "\\bbuild\\b", "\\bimplement\\b", "\\bupdate\\b", "\\bship\\b", "\\bcreate\\b", "\\bwrite\\b"] }, + { "kind": "question", "trackable": false, "patterns": ["\\?\\s*$", "^(what|why|how|when|where|who|is|are|do|does|did|can|should)\\b"] }, + { "kind": "idea", "trackable": false, "patterns": ["\\bidea\\b", "\\bwhat if\\b", "\\bmaybe we (should|could)\\b", "\\bthinking about\\b"] } + ], + + "$comment_signals": "Phrases that update an existing item rather than creating a new one.", + "claimPatterns": ["\\bon it\\b", "\\bi'?ll (do|take|handle|look at)\\b", "\\btaking (this|that|it)\\b", "\\bstarting (this|that|it|on)\\b", "\\bpicking (this|that|it) up\\b"], + "donePatterns": ["\\bdone\\b", "\\bshipped\\b", "\\bmerged\\b", "\\bfixed\\b", "\\bcompleted?\\b", "\\bthat'?s live\\b", "✅"], + "dropPatterns": ["\\bnevermind\\b", "\\bnever mind\\b", "\\bignore (that|this)\\b", "\\bcancel(led)? (that|this)\\b", "\\bnot needed\\b", "\\bwon'?t do\\b"], + + "$comment_ignore": "Never create work from these — bots talking to bots is how a queue fills with noise.", + "ignoreBots": true, + "ignorePrefixes": ["!", "/", "```"], + "minLength": 12 +} diff --git a/docs/COMMANDER_CLAUDE.md b/docs/COMMANDER_CLAUDE.md index 2fe045b3..5115caa6 100644 --- a/docs/COMMANDER_CLAUDE.md +++ b/docs/COMMANDER_CLAUDE.md @@ -138,7 +138,20 @@ curl -sS -X POST "$BASE_URL/api/supervisor/tick" \ -d '{"dryRun": true}' | jq ``` -**Autonomy levels** — `off` (nothing runs) | `observe` (default: record only, zero side effects) | `assist` (may notify and type nudges into sessions) | `autopilot` (may also run allowlisted act handlers). +**Autonomy levels** — `off` (nothing runs) | `observe` (record only) | `assist` (may repair things itself) | `autopilot` (default: may also delegate to a Commander). + +Autonomy governs what JARVIS may **fix**, not what it may say. Reaching a human is gated separately: a finding must exhaust its repair attempts, then clear an urgency threshold weighted by the task's tier, then fit inside an interruption budget. Everything else batches into a digest. + +**You may be on the receiving end of this.** When rules cannot fix something, the problem is delegated to a Commander as a `[JARVIS]` problem brief with the session, branch, tier and output tail. That is a request to diagnose and fix it — not to relay it to the user. Escalate to a human only if you are genuinely blocked on a decision only they can make. + +```bash +# What is waiting but did not earn an interruption +curl -sS "$BASE_URL/api/supervisor/digest" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# "Catch me up" — deliver the batch now +curl -sS -X POST "$BASE_URL/api/supervisor/digest/deliver" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" -d '{}' | jq +``` ```bash curl -sS -X POST "$BASE_URL/api/supervisor/autonomy" \ @@ -187,7 +200,28 @@ curl -sS -X POST "$BASE_URL/api/atlas/entries/zoo-game/highlights" \ Also available as a CLI anywhere: `node scripts/atlas.js find `. -**Do not change a repo's `visibility` or `groups`, and do not compile sharing bundles, without being asked.** Those decide what leaves the machine. +**Do not change a repo's `visibility` or `groups`, and do not compile or publish sharing bundles, without being asked.** Those decide what leaves the machine. + +The registry syncs between machines via a private git repo (`GET/POST /api/atlas/sync`). Entries marked `foreign: true` were shared with you by someone else — read them, never re-share them. + +## Discord (ambient team work) + +The watcher reads whole channels rather than waiting to be addressed, turns assignments into tracked work with a priority, and publishes status back so nobody has to ask whether an agent picked something up. + +```bash +# Asked for, nobody started — ordered by priority +curl -sS "$BASE_URL/api/discord-watch/untracked" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +curl -sS "$BASE_URL/api/discord-watch/items?status=in-progress" -H "X-Auth-Token: $AUTH_TOKEN" | jq +curl -sS "$BASE_URL/api/discord-watch/status" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Bind a work item to the session doing it — this is what makes agent status visible to the team +curl -sS -X POST "$BASE_URL/api/discord-watch/items/discord:123/link" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"sessionId": "zoo-game-work1-claude"}' +``` + +Link a work item whenever you start a session for one — an unlinked item looks untouched to everyone else. Work item tiers come from how urgently the message was phrased, and they flow into the task record, so linking also sets the session's tier correctly. ## Session Control diff --git a/server/discord/discordClient.js b/server/discord/discordClient.js new file mode 100644 index 00000000..62388729 --- /dev/null +++ b/server/discord/discordClient.js @@ -0,0 +1,120 @@ +const API_BASE = 'https://discord.com/api/v10'; +const DEFAULT_TIMEOUT_MS = 15_000; +const MAX_PAGE = 100; + +/** + * Minimal Discord REST client. + * + * Deliberately not a gateway client. Reading with `?after=` means a + * missed message is not a thing that can happen — there is no connection state + * to lose, and a restart after three days is just a longer page-through. That + * trades a few seconds of latency for removing an entire class of bug. + */ +class DiscordClient { + constructor({ token = '', fetchImpl = null, logger = console } = {}) { + this.token = String(token || process.env.DISCORD_BOT_TOKEN || '').trim(); + this.fetch = fetchImpl || globalThis.fetch; + this.logger = logger; + this.rateLimitedUntil = 0; + } + + isConfigured() { + return Boolean(this.token) && typeof this.fetch === 'function'; + } + + async request(pathname, { method = 'GET', body = null, timeoutMs = DEFAULT_TIMEOUT_MS } = {}) { + if (!this.isConfigured()) { + return { ok: false, status: 0, error: 'Discord bot token is not configured (DISCORD_BOT_TOKEN)' }; + } + if (Date.now() < this.rateLimitedUntil) { + return { ok: false, status: 429, error: 'rate limited', retryAfterMs: this.rateLimitedUntil - Date.now() }; + } + + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), timeoutMs); + + try { + const response = await this.fetch(`${API_BASE}${pathname}`, { + method, + headers: { + Authorization: `Bot ${this.token}`, + 'Content-Type': 'application/json', + 'User-Agent': 'AgentWorkspace (https://github.com/web3dev1337/agent-workspace, 1.0)' + }, + body: body ? JSON.stringify(body) : undefined, + signal: controller.signal + }); + + if (response.status === 429) { + const retryAfter = Number(response.headers?.get?.('retry-after') || 5); + this.rateLimitedUntil = Date.now() + retryAfter * 1000; + return { ok: false, status: 429, error: 'rate limited', retryAfterMs: retryAfter * 1000 }; + } + + const text = await response.text(); + const data = text ? JSON.parse(text) : null; + if (!response.ok) { + return { ok: false, status: response.status, error: data?.message || `HTTP ${response.status}` }; + } + return { ok: true, status: response.status, data }; + } catch (error) { + return { ok: false, status: 0, error: error.name === 'AbortError' ? 'timed out' : error.message }; + } finally { + clearTimeout(timer); + } + } + + /** + * Page forward from a cursor. Discord returns newest-first even with `after`, + * so pages are reversed into chronological order — work items should be + * created in the order the conversation actually happened. + */ + async fetchMessagesAfter(channelId, afterId, { maxMessages = 400 } = {}) { + const collected = []; + let cursor = afterId; + + while (collected.length < maxMessages) { + const query = new URLSearchParams({ limit: String(MAX_PAGE) }); + if (cursor) query.set('after', cursor); + + const result = await this.request(`/channels/${channelId}/messages?${query}`); + if (!result.ok) return { ok: false, error: result.error, status: result.status, messages: collected }; + + const page = Array.isArray(result.data) ? result.data.slice().reverse() : []; + if (!page.length) break; + + collected.push(...page); + cursor = page[page.length - 1].id; + if (page.length < MAX_PAGE) break; + } + + return { ok: true, messages: collected.slice(0, maxMessages), cursor: cursor || afterId }; + } + + async getLatestMessageId(channelId) { + const result = await this.request(`/channels/${channelId}/messages?limit=1`); + if (!result.ok) return { ok: false, error: result.error }; + const latest = Array.isArray(result.data) ? result.data[0] : null; + return { ok: true, id: latest?.id || null }; + } + + async getChannel(channelId) { + const result = await this.request(`/channels/${channelId}`); + return result.ok ? { ok: true, channel: result.data } : { ok: false, error: result.error }; + } + + async postMessage(channelId, content, { replyToMessageId = null } = {}) { + const body = { content: String(content || '').slice(0, 1900) }; + if (replyToMessageId) { + body.message_reference = { message_id: replyToMessageId, fail_if_not_exists: false }; + } + const result = await this.request(`/channels/${channelId}/messages`, { method: 'POST', body }); + return result.ok ? { ok: true, message: result.data } : { ok: false, error: result.error }; + } +} + +function messagePermalink({ guildId, channelId, messageId }) { + return `https://discord.com/channels/${guildId || '@me'}/${channelId}/${messageId}`; +} + +module.exports = { DiscordClient, messagePermalink, API_BASE }; diff --git a/server/discord/workExtractor.js b/server/discord/workExtractor.js new file mode 100644 index 00000000..59640597 --- /dev/null +++ b/server/discord/workExtractor.js @@ -0,0 +1,189 @@ +const fs = require('fs'); +const path = require('path'); + +const { getAgentWorkspaceDir } = require('../utils/pathUtils'); +const { messagePermalink } = require('./discordClient'); + +const DEFAULT_CONFIG_PATH = path.join(__dirname, '..', '..', 'config', 'discord-watch.json'); + +function overrideConfigPath() { + return path.join(getAgentWorkspaceDir(), 'discord-watch.json'); +} + +function compile(patterns) { + const out = []; + for (const pattern of Array.isArray(patterns) ? patterns : []) { + try { + out.push(new RegExp(String(pattern), 'i')); + } catch { + // One bad pattern must not disable extraction entirely. + } + } + return out; +} + +function readJson(filePath) { + try { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); + } catch { + return null; + } +} + +function loadConfig({ configPath = null } = {}) { + const override = configPath || overrideConfigPath(); + const raw = readJson(override) || readJson(DEFAULT_CONFIG_PATH) || {}; + const source = readJson(override) ? override : DEFAULT_CONFIG_PATH; + + return { + source, + enabled: raw.enabled === true, + pollSeconds: Math.max(5, Number(raw.pollSeconds) || 15), + channels: Array.isArray(raw.channels) ? raw.channels.map(String).filter(Boolean) : [], + backfillMessages: Math.max(0, Number(raw.backfillMessages) ?? 50), + publishStatus: raw.publishStatus !== false, + priority: (Array.isArray(raw.priority) ? raw.priority : []).map((row) => ({ + level: String(row.level || 'normal'), + tier: Number(row.tier) || 3, + patterns: compile(row.patterns) + })), + defaultPriority: { + level: String(raw.defaultPriority?.level || 'normal'), + tier: Number(raw.defaultPriority?.tier) || 3 + }, + kinds: (Array.isArray(raw.kinds) ? raw.kinds : []).map((row) => ({ + kind: String(row.kind || 'fyi'), + trackable: row.trackable === true, + patterns: compile(row.patterns) + })), + claimPatterns: compile(raw.claimPatterns), + donePatterns: compile(raw.donePatterns), + dropPatterns: compile(raw.dropPatterns), + ignoreBots: raw.ignoreBots !== false, + ignorePrefixes: Array.isArray(raw.ignorePrefixes) ? raw.ignorePrefixes.map(String) : [], + minLength: Math.max(0, Number(raw.minLength) ?? 12) + }; +} + +const MENTION_RE = /<@!?(\d+)>/g; + +function extractMentions(message) { + const ids = new Set(); + for (const match of String(message?.content || '').matchAll(MENTION_RE)) ids.add(match[1]); + for (const mention of Array.isArray(message?.mentions) ? message.mentions : []) { + if (mention?.id) ids.add(String(mention.id)); + } + return [...ids]; +} + +function cleanContent(message, mentionNames = {}) { + return String(message?.content || '') + .replace(MENTION_RE, (_, id) => `@${mentionNames[id] || 'someone'}`) + .replace(/\s+/g, ' ') + .trim(); +} + +function firstMatch(rows, text) { + for (const row of rows) { + if (row.patterns.some((re) => re.test(text))) return row; + } + return null; +} + +function classifyPriority(text, config) { + const hit = firstMatch(config.priority, text); + return hit ? { level: hit.level, tier: hit.tier } : { ...config.defaultPriority }; +} + +function classifyKind(text, config) { + const hit = firstMatch(config.kinds, text); + return hit ? { kind: hit.kind, trackable: hit.trackable } : { kind: 'fyi', trackable: false }; +} + +function shouldIgnore(message, config) { + if (config.ignoreBots && message?.author?.bot) return 'bot'; + const content = String(message?.content || '').trim(); + if (!content) return 'empty'; + if (content.length < config.minLength) return 'too short'; + if (config.ignorePrefixes.some((prefix) => content.startsWith(prefix))) return 'ignored prefix'; + return null; +} + +/** + * Decide what a single message means. + * + * Rules first, deliberately: most team chat is unambiguous, and a rule pass over + * every message costs nothing. `classifier` is the seam for handing the genuinely + * ambiguous middle to a model — batched, and only for messages that look like + * work but did not match cleanly. + */ +function extractFromMessage(message, { config, guildId = '', memberNames = {} } = {}) { + const ignored = shouldIgnore(message, config); + if (ignored) return { action: 'ignore', reason: ignored, messageId: message?.id }; + + const text = cleanContent(message, memberNames); + const mentions = extractMentions(message); + + // A signal about existing work beats creating new work — "done" following an + // assignment is a status change, not a new task. + if (config.donePatterns.some((re) => re.test(text))) { + return { action: 'complete', messageId: message.id, authorId: message.author?.id, text, mentions }; + } + if (config.dropPatterns.some((re) => re.test(text))) { + return { action: 'drop', messageId: message.id, authorId: message.author?.id, text, mentions }; + } + if (config.claimPatterns.some((re) => re.test(text))) { + return { action: 'claim', messageId: message.id, authorId: message.author?.id, text, mentions }; + } + + const kind = classifyKind(text, config); + const priority = classifyPriority(text, config); + + // Directed at a person and phrased as a request: that is an assignment even + // if nobody creates a ticket, and it is exactly the thing that gets lost today. + const directed = mentions.length > 0; + const trackable = kind.trackable || (directed && kind.kind !== 'question'); + + if (!trackable) { + return { action: 'ignore', reason: `not trackable (${kind.kind})`, messageId: message.id, kind: kind.kind }; + } + + return { + action: 'create', + messageId: message.id, + channelId: message.channel_id, + permalink: messagePermalink({ guildId, channelId: message.channel_id, messageId: message.id }), + authorId: message.author?.id || null, + authorName: message.author?.username || memberNames[message.author?.id] || 'someone', + assigneeIds: mentions, + assigneeNames: mentions.map((id) => memberNames[id] || id), + text, + summary: text.length > 140 ? `${text.slice(0, 137)}…` : text, + kind: kind.kind, + priority: priority.level, + tier: priority.tier, + createdAt: message.timestamp || new Date().toISOString(), + confidence: directed && kind.trackable ? 'high' : 'medium' + }; +} + +function extractBatch(messages, options) { + const results = []; + for (const message of Array.isArray(messages) ? messages : []) { + results.push(extractFromMessage(message, options)); + } + return results; +} + +module.exports = { + DEFAULT_CONFIG_PATH, + overrideConfigPath, + loadConfig, + extractMentions, + cleanContent, + classifyPriority, + classifyKind, + shouldIgnore, + extractFromMessage, + extractBatch +}; diff --git a/server/discordWatchService.js b/server/discordWatchService.js new file mode 100644 index 00000000..42f3b51e --- /dev/null +++ b/server/discordWatchService.js @@ -0,0 +1,376 @@ +const fs = require('fs'); +const path = require('path'); + +const { getAgentWorkspaceDir } = require('./utils/pathUtils'); +const { DiscordClient } = require('./discord/discordClient'); +const extractor = require('./discord/workExtractor'); + +const STATE_FILENAME = 'discord-watch.json'; +const MAX_ITEMS = 500; + +/** + * Ambient Discord watching. + * + * Reads the whole conversation rather than waiting to be addressed, turns + * assignments into tracked work with a priority, and publishes back what the + * orchestrator already knows — so "did that get picked up?" and "is anyone's + * agent actually on it?" stop being questions someone has to ask. + * + * Reading is cursor-based rather than gateway-based on purpose: with a persisted + * `after` position there is no such thing as a message missed while the process + * was down, which was the previous integration's worst failure mode. + */ +class DiscordWatchService { + constructor({ logger = console, client = null } = {}) { + this.logger = logger; + this.config = extractor.loadConfig(); + this.client = client || new DiscordClient({ logger }); + this.taskRecordService = null; + this.activityFeed = null; + + this.timer = null; + this.running = false; + this.polling = false; + this.lastPollAt = null; + this.lastError = null; + this.stats = { messagesRead: 0, itemsCreated: 0, itemsUpdated: 0, statusesPublished: 0 }; + + this.state = this.loadState(); + } + + static getInstance(options = {}) { + if (!DiscordWatchService.instance) { + DiscordWatchService.instance = new DiscordWatchService(options); + } + return DiscordWatchService.instance; + } + + init({ taskRecordService, activityFeed } = {}) { + this.taskRecordService = taskRecordService || this.taskRecordService; + this.activityFeed = activityFeed || this.activityFeed; + return this; + } + + statePath() { + const dir = path.join(getAgentWorkspaceDir(), 'discord'); + try { + fs.mkdirSync(dir, { recursive: true }); + } catch { + // A missing state dir is recreated on the next write attempt. + } + return path.join(dir, STATE_FILENAME); + } + + loadState() { + try { + const raw = JSON.parse(fs.readFileSync(this.statePath(), 'utf8')); + return { + cursors: raw.cursors && typeof raw.cursors === 'object' ? raw.cursors : {}, + items: Array.isArray(raw.items) ? raw.items : [], + memberNames: raw.memberNames && typeof raw.memberNames === 'object' ? raw.memberNames : {} + }; + } catch { + return { cursors: {}, items: [], memberNames: {} }; + } + } + + saveState() { + try { + fs.writeFileSync(this.statePath(), `${JSON.stringify(this.state, null, 2)}\n`, 'utf8'); + } catch (error) { + this.logger.warn?.('Discord watch could not persist state', { error: error.message }); + } + } + + reloadConfig() { + this.config = extractor.loadConfig(); + if (this.running) this.restartTimer(); + return this.config; + } + + getChannels() { + return this.config.channels; + } + + addChannel(channelId) { + const id = String(channelId || '').trim(); + if (!/^\d+$/.test(id)) throw new Error('A Discord channel id is a numeric snowflake'); + if (!this.config.channels.includes(id)) this.config.channels.push(id); + return this.config.channels; + } + + removeChannel(channelId) { + const id = String(channelId || '').trim(); + this.config.channels = this.config.channels.filter((existing) => existing !== id); + delete this.state.cursors[id]; + this.saveState(); + return this.config.channels; + } + + findItemByMessage(messageId) { + return this.state.items.find((item) => item.messageId === messageId) || null; + } + + /** + * The most recent open item in a channel — what "on it" or "done" refers to + * when nobody quotes the original message, which is almost always. + */ + findOpenItem(channelId, { authorId = null } = {}) { + return this.state.items.find((item) => ( + item.channelId === channelId + && item.status !== 'done' + && item.status !== 'dropped' + && (!authorId || !item.assigneeIds?.length || item.assigneeIds.includes(authorId)) + )) || null; + } + + recordItem(item) { + this.state.items.unshift(item); + if (this.state.items.length > MAX_ITEMS) this.state.items.length = MAX_ITEMS; + this.stats.itemsCreated += 1; + this.activityFeed?.track?.('discord.work-item', { + id: item.id, + kind: item.kind, + priority: item.priority, + assignees: item.assigneeNames + }); + return item; + } + + updateItem(item, patch) { + Object.assign(item, patch, { updatedAt: new Date().toISOString() }); + this.stats.itemsUpdated += 1; + return item; + } + + applyExtraction(extraction, { channelId }) { + if (extraction.action === 'ignore') return null; + + if (extraction.action === 'create') { + if (this.findItemByMessage(extraction.messageId)) return null; + return this.recordItem({ + id: `discord:${extraction.messageId}`, + source: 'discord', + channelId: extraction.channelId || channelId, + messageId: extraction.messageId, + permalink: extraction.permalink, + authorId: extraction.authorId, + authorName: extraction.authorName, + assigneeIds: extraction.assigneeIds, + assigneeNames: extraction.assigneeNames, + text: extraction.text, + summary: extraction.summary, + kind: extraction.kind, + priority: extraction.priority, + tier: extraction.tier, + confidence: extraction.confidence, + status: 'new', + claimedBy: null, + sessionId: null, + ticketUrl: null, + prUrl: null, + createdAt: extraction.createdAt, + updatedAt: new Date().toISOString() + }); + } + + const target = this.findOpenItem(channelId, { authorId: extraction.authorId }); + if (!target) return null; + + if (extraction.action === 'claim') { + return this.updateItem(target, { status: 'claimed', claimedBy: extraction.authorId }); + } + if (extraction.action === 'complete') { + return this.updateItem(target, { status: 'done', completedBy: extraction.authorId }); + } + if (extraction.action === 'drop') { + return this.updateItem(target, { status: 'dropped' }); + } + return null; + } + + /** + * Read one channel forward from wherever we left off. + */ + async pollChannel(channelId) { + const cursor = this.state.cursors[channelId] || null; + + // First sight of a channel: start from now (or a short backfill) rather than + // ingesting years of history and inventing a hundred stale tasks. + if (!cursor) { + const latest = await this.client.getLatestMessageId(channelId); + if (!latest.ok) return { channelId, ok: false, error: latest.error }; + if (!this.config.backfillMessages) { + this.state.cursors[channelId] = latest.id; + this.saveState(); + return { channelId, ok: true, messages: 0, seeded: true }; + } + } + + const fetched = await this.client.fetchMessagesAfter(channelId, cursor, { + maxMessages: cursor ? 400 : this.config.backfillMessages + }); + if (!fetched.ok) return { channelId, ok: false, error: fetched.error }; + + const created = []; + const updated = []; + for (const message of fetched.messages) { + if (message.author?.username && message.author?.id) { + this.state.memberNames[message.author.id] = message.author.username; + } + const extraction = extractor.extractFromMessage(message, { + config: this.config, + memberNames: this.state.memberNames + }); + const result = this.applyExtraction(extraction, { channelId }); + if (!result) continue; + (extraction.action === 'create' ? created : updated).push(result); + } + + this.stats.messagesRead += fetched.messages.length; + if (fetched.messages.length) { + this.state.cursors[channelId] = fetched.messages[fetched.messages.length - 1].id; + } + this.saveState(); + + return { channelId, ok: true, messages: fetched.messages.length, created: created.length, updated: updated.length, items: created }; + } + + async poll() { + if (this.polling) return { skipped: 'already polling' }; + this.polling = true; + + try { + const results = []; + for (const channelId of this.config.channels) { + results.push(await this.pollChannel(channelId)); + } + this.lastPollAt = new Date().toISOString(); + this.lastError = results.find((r) => !r.ok)?.error || null; + return { at: this.lastPollAt, channels: results }; + } catch (error) { + this.lastError = error.message; + this.logger.error?.('Discord watch poll failed', { error: error.message }); + return { error: error.message }; + } finally { + this.polling = false; + } + } + + /** + * Say back into the channel what the orchestrator already knows. Nobody types + * a status update; the status is published from the system's own state. + */ + async publishStatus(itemId, text) { + const item = this.state.items.find((row) => row.id === itemId); + if (!item) return { ok: false, error: `no work item "${itemId}"` }; + if (!this.config.publishStatus) return { ok: false, error: 'status publishing is disabled' }; + + const posted = await this.client.postMessage(item.channelId, text, { replyToMessageId: item.messageId }); + if (posted.ok) this.stats.statusesPublished += 1; + return posted; + } + + /** + * Bind a work item to the session that is actually doing it — this is what + * makes "is their agent working on it" answerable. + */ + async linkSession(itemId, sessionId, { announce = true } = {}) { + const item = this.state.items.find((row) => row.id === itemId); + if (!item) throw new Error(`No work item "${itemId}"`); + + this.updateItem(item, { status: 'in-progress', sessionId }); + this.saveState(); + + try { + this.taskRecordService?.upsert?.(`session:${sessionId}`, { + tier: item.tier, + ticketProvider: 'discord', + ticketCardId: item.messageId, + ticketCardUrl: item.permalink, + ticketTitle: item.summary + }); + } catch (error) { + this.logger.warn?.('Could not write a task record for a Discord work item', { error: error.message }); + } + + if (announce) { + await this.publishStatus(itemId, `🤖 picked up by \`${sessionId}\` — ${item.summary}`); + } + return item; + } + + getItems({ status = '', assignee = '', channelId = '', limit = 50 } = {}) { + const wantStatus = String(status || '').trim(); + const wantAssignee = String(assignee || '').trim(); + const wantChannel = String(channelId || '').trim(); + + return this.state.items + .filter((item) => (!wantStatus || item.status === wantStatus)) + .filter((item) => (!wantChannel || item.channelId === wantChannel)) + .filter((item) => (!wantAssignee + || item.assigneeIds?.includes(wantAssignee) + || item.assigneeNames?.includes(wantAssignee))) + .slice(0, Math.max(1, Number(limit) || 50)); + } + + /** + * What the team asked for that nobody has started — the list that currently + * exists only in people's heads and scrollback. + */ + getUntracked() { + return this.state.items + .filter((item) => item.status === 'new') + .sort((a, b) => a.tier - b.tier || String(a.createdAt).localeCompare(String(b.createdAt))); + } + + restartTimer() { + if (this.timer) clearInterval(this.timer); + this.timer = setInterval(() => { + this.poll().catch((error) => this.logger.error?.('Discord poll threw', { error: error.message })); + }, this.config.pollSeconds * 1000); + if (typeof this.timer.unref === 'function') this.timer.unref(); + } + + start() { + if (this.running) return { running: true, alreadyRunning: true }; + if (!this.config.enabled) return { running: false, reason: 'discord watch is disabled in config' }; + if (!this.client.isConfigured()) return { running: false, reason: 'DISCORD_BOT_TOKEN is not set' }; + if (!this.config.channels.length) return { running: false, reason: 'no channels configured' }; + + this.running = true; + this.restartTimer(); + return { running: true, pollSeconds: this.config.pollSeconds, channels: this.config.channels.length }; + } + + stop() { + if (this.timer) clearInterval(this.timer); + this.timer = null; + const wasRunning = this.running; + this.running = false; + return { running: false, wasRunning }; + } + + getStatus() { + return { + running: this.running, + enabled: this.config.enabled, + configured: this.client.isConfigured(), + configSource: this.config.source, + pollSeconds: this.config.pollSeconds, + publishStatus: this.config.publishStatus, + channels: this.config.channels.map((id) => ({ id, cursor: this.state.cursors[id] || null })), + stats: { ...this.stats }, + lastPollAt: this.lastPollAt, + lastError: this.lastError, + itemCounts: this.state.items.reduce((counts, item) => { + counts[item.status] = (counts[item.status] || 0) + 1; + return counts; + }, {}), + statePath: this.statePath() + }; + } +} + +module.exports = DiscordWatchService; +module.exports.DiscordWatchService = DiscordWatchService; diff --git a/server/index.js b/server/index.js index cab50594..9bb970e2 100644 --- a/server/index.js +++ b/server/index.js @@ -116,6 +116,8 @@ const { SupervisorService } = require('./supervisorService'); const { createSupervisorRoutes } = require('./routes/supervisorRoutes'); const { SpeechService } = require('./speechService'); const { createSpeechRoutes } = require('./routes/speechRoutes'); +const { DiscordWatchService } = require('./discordWatchService'); +const { createDiscordWatchRoutes } = require('./routes/discordWatchRoutes'); const { ProductLauncherService } = require('./productLauncherService'); const { CommanderService } = require('./commanderService'); const { ConversationService } = require('./conversationService'); @@ -359,6 +361,7 @@ const repoAtlasService = RepoAtlasService.getInstance({ logger }); const speechService = SpeechService.getInstance({ logger }); speechService.setIO(io); const supervisorService = SupervisorService.getInstance({ logger }); +const discordWatchService = DiscordWatchService.getInstance({ logger }); const activityFeed = ActivityFeedService.getInstance(); activityFeed.setIO(io); activityFeed.track('server.started', { port: Number(process.env.ORCHESTRATOR_PORT || 9460) }); @@ -462,6 +465,11 @@ if (String(process.env.SUPERVISOR_AUTOSTART || 'true').toLowerCase() !== 'false' logger.info('Supervisor', started); } +// Off unless explicitly configured — it needs a bot token and channel ids. +discordWatchService.init({ taskRecordService, activityFeed }); +const discordWatchStarted = discordWatchService.start(); +if (discordWatchStarted.running) logger.info('Discord watch', discordWatchStarted); + // Speech that no rule matched is still useful: hand the raw words to the // active Commander so the fallback is an agent, not an error. voiceCommandService.setCommanderForwarder(sendToCommander); @@ -1392,6 +1400,13 @@ app.use('/api/supervisor', createSupervisorRoutes({ requireWrite: requirePolicyAction('write') })); +app.use('/api/discord-watch', createDiscordWatchRoutes({ + discordWatchService, + logger, + requireRead: requirePolicyAction('read'), + requireWrite: requirePolicyAction('write') +})); + app.use('/api/speech', createSpeechRoutes({ speechService, supervisorService, diff --git a/server/routes/discordWatchRoutes.js b/server/routes/discordWatchRoutes.js new file mode 100644 index 00000000..ced3e621 --- /dev/null +++ b/server/routes/discordWatchRoutes.js @@ -0,0 +1,88 @@ +const express = require('express'); + +const passthrough = (req, res, next) => next(); + +/** + * Ambient Discord watching: read the conversation, track the work, publish + * status back. Anything that writes into a channel is policy-`write`. + */ +function createDiscordWatchRoutes({ discordWatchService, logger = console, requireRead = passthrough, requireWrite = passthrough } = {}) { + const router = express.Router(); + + const handle = (label, handler) => async (req, res) => { + try { + await handler(req, res); + } catch (error) { + logger.error(`Discord watch: ${label} failed`, { error: error.message, stack: error.stack }); + res.status(400).json({ ok: false, error: error.message }); + } + }; + + router.get('/status', requireRead, handle('status', (req, res) => { + res.json({ ok: true, status: discordWatchService.getStatus() }); + })); + + router.get('/items', requireRead, handle('list items', (req, res) => { + const items = discordWatchService.getItems({ + status: req.query.status, + assignee: req.query.assignee, + channelId: req.query.channelId, + limit: req.query.limit + }); + res.json({ ok: true, count: items.length, items }); + })); + + /** + * Asked for, nobody started. The list that otherwise only exists in scrollback. + */ + router.get('/untracked', requireRead, handle('untracked', (req, res) => { + const items = discordWatchService.getUntracked(); + res.json({ ok: true, count: items.length, items }); + })); + + router.post('/poll', requireWrite, handle('poll', async (req, res) => { + res.json({ ok: true, ...(await discordWatchService.poll()) }); + })); + + router.post('/start', requireWrite, handle('start', (req, res) => { + res.json({ ok: true, ...discordWatchService.start() }); + })); + + router.post('/stop', requireWrite, handle('stop', (req, res) => { + res.json({ ok: true, ...discordWatchService.stop() }); + })); + + router.post('/reload-config', requireWrite, handle('reload config', (req, res) => { + const config = discordWatchService.reloadConfig(); + res.json({ ok: true, source: config.source, enabled: config.enabled, channels: config.channels }); + })); + + router.get('/channels', requireRead, handle('list channels', (req, res) => { + res.json({ ok: true, channels: discordWatchService.getChannels() }); + })); + + router.post('/channels', requireWrite, handle('add channel', (req, res) => { + res.json({ ok: true, channels: discordWatchService.addChannel(req.body?.channelId) }); + })); + + router.delete('/channels/:channelId', requireWrite, handle('remove channel', (req, res) => { + res.json({ ok: true, channels: discordWatchService.removeChannel(req.params.channelId) }); + })); + + router.post('/items/:id/link', requireWrite, handle('link session', async (req, res) => { + const item = await discordWatchService.linkSession(req.params.id, req.body?.sessionId, { + announce: req.body?.announce !== false + }); + res.json({ ok: true, item }); + })); + + router.post('/items/:id/status', requireWrite, handle('publish status', async (req, res) => { + const text = String(req.body?.text || '').trim(); + if (!text) return res.status(400).json({ ok: false, error: 'text is required' }); + return res.json({ ok: true, ...(await discordWatchService.publishStatus(req.params.id, text)) }); + })); + + return router; +} + +module.exports = { createDiscordWatchRoutes }; diff --git a/tests/unit/discordWatchService.test.js b/tests/unit/discordWatchService.test.js new file mode 100644 index 00000000..040f0b22 --- /dev/null +++ b/tests/unit/discordWatchService.test.js @@ -0,0 +1,278 @@ +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const DiscordWatchService = require('../../server/discordWatchService'); +const extractor = require('../../server/discord/workExtractor'); +const { DiscordClient } = require('../../server/discord/discordClient'); + +const config = extractor.loadConfig({ configPath: extractor.DEFAULT_CONFIG_PATH }); + +const message = (overrides = {}) => ({ + id: '100', + channel_id: 'chan1', + content: 'hello world this is long enough', + author: { id: '1001', username: 'ab', bot: false }, + timestamp: '2026-07-26T12:00:00Z', + ...overrides +}); + +describe('workExtractor', () => { + test('a mention plus a request is an assignment, even with no ticket', () => { + const result = extractor.extractFromMessage( + message({ content: '<@2002> can you fix the physics stutter on the boss fight' }), + { config, memberNames: { 2002: 'sam' } } + ); + + expect(result.action).toBe('create'); + expect(result.kind).toBe('assignment'); + expect(result.assigneeIds).toEqual(['2002']); + expect(result.assigneeNames).toEqual(['sam']); + expect(result.confidence).toBe('high'); + }); + + test('priority comes out of how people actually talk', () => { + const urgent = extractor.extractFromMessage(message({ content: '<@2002> URGENT the build is broken on main' }), { config }); + const later = extractor.extractFromMessage(message({ content: '<@2002> when you get a chance please update the readme' }), { config }); + + expect(urgent.priority).toBe('urgent'); + expect(urgent.tier).toBe(1); + expect(later.priority).toBe('low'); + expect(later.tier).toBe(4); + }); + + test('a bug report is trackable even with nobody mentioned', () => { + const result = extractor.extractFromMessage(message({ content: 'the save system is broken after the last merge' }), { config }); + expect(result.action).toBe('create'); + expect(result.kind).toBe('bug'); + }); + + test('a plain question is not turned into a task', () => { + const result = extractor.extractFromMessage(message({ content: 'what did we decide about the camera angle?' }), { config }); + expect(result.action).toBe('ignore'); + }); + + test('claim, done and drop update existing work instead of creating more', () => { + expect(extractor.extractFromMessage(message({ content: "on it, starting now" }), { config }).action).toBe('claim'); + expect(extractor.extractFromMessage(message({ content: 'done, merged to main' }), { config }).action).toBe('complete'); + expect(extractor.extractFromMessage(message({ content: 'nevermind, not needed anymore' }), { config }).action).toBe('drop'); + }); + + test('bots, commands and one-word replies are ignored', () => { + expect(extractor.extractFromMessage(message({ author: { id: 'b', bot: true }, content: 'please fix the thing' }), { config }).reason).toBe('bot'); + expect(extractor.extractFromMessage(message({ content: '!deploy production now' }), { config }).reason).toBe('ignored prefix'); + expect(extractor.extractFromMessage(message({ content: 'ok' }), { config }).reason).toBe('too short'); + }); + + test('mentions are rendered as names so the text reads back sensibly', () => { + const result = extractor.extractFromMessage( + message({ content: '<@2002> please add the leaderboard' }), + { config, memberNames: { 2002: 'sam' } } + ); + expect(result.text).toBe('@sam please add the leaderboard'); + }); +}); + +describe('DiscordClient', () => { + test('reports missing configuration rather than throwing', async () => { + const client = new DiscordClient({ token: '' }); + expect(client.isConfigured()).toBe(false); + expect((await client.request('/anything')).error).toMatch(/not configured/); + }); + + test('pages forward from the cursor and returns messages oldest-first', async () => { + const pages = [ + [{ id: '3' }, { id: '2' }], + [] + ]; + const seen = []; + const client = new DiscordClient({ + token: 't', + fetchImpl: async (url) => { + seen.push(url); + return { ok: true, status: 200, headers: { get: () => null }, text: async () => JSON.stringify(pages.shift() || []) }; + } + }); + + const result = await client.fetchMessagesAfter('chan1', '1'); + expect(result.ok).toBe(true); + expect(result.messages.map((m) => m.id)).toEqual(['2', '3']); + expect(seen[0]).toContain('after=1'); + }); + + test('a 429 backs off instead of hammering', async () => { + const client = new DiscordClient({ + token: 't', + fetchImpl: async () => ({ ok: false, status: 429, headers: { get: () => '3' }, text: async () => '{}' }) + }); + + expect((await client.request('/x')).status).toBe(429); + const second = await client.request('/x'); + expect(second.error).toBe('rate limited'); + expect(second.retryAfterMs).toBeGreaterThan(0); + }); +}); + +describe('DiscordWatchService', () => { + let tmpDir; + + const harness = ({ messages = [], latestId = '9' } = {}) => { + const posted = []; + const client = { + isConfigured: () => true, + getLatestMessageId: async () => ({ ok: true, id: latestId }), + fetchMessagesAfter: async () => ({ ok: true, messages }), + postMessage: async (channelId, content, options) => { + posted.push({ channelId, content, options }); + return { ok: true, message: { id: 'posted' } }; + } + }; + + const service = new DiscordWatchService({ logger: { warn: () => {}, error: () => {}, info: () => {} }, client }); + service.config = { ...extractor.loadConfig({ configPath: extractor.DEFAULT_CONFIG_PATH }), enabled: true, channels: ['chan1'] }; + service.init({ taskRecordService: { upsert: () => {} }, activityFeed: { track: () => {} } }); + return { service, posted }; + }; + + beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'discord-watch-')); + process.env.AGENT_WORKSPACE_DIR = tmpDir; + }); + + afterEach(() => { + delete process.env.AGENT_WORKSPACE_DIR; + fs.rmSync(tmpDir, { recursive: true, force: true }); + }); + + test('a conversation becomes tracked work without anyone filing a ticket', async () => { + const { service } = harness({ + messages: [ + message({ id: '10', content: '<@2002> please fix the crash on level three, its urgent' }), + message({ id: '11', content: 'sounds good' }) + ] + }); + + const result = await service.poll(); + expect(result.channels[0].created).toBe(1); + + const [item] = service.getItems(); + expect(item.priority).toBe('urgent'); + expect(item.tier).toBe(1); + expect(item.status).toBe('new'); + expect(item.permalink).toContain('/chan1/10'); + }); + + test('the cursor advances so nothing is read twice', async () => { + const { service } = harness({ messages: [message({ id: '10', content: '<@2002> please fix the crash on level three' })] }); + await service.poll(); + expect(service.state.cursors.chan1).toBe('10'); + + service.client.fetchMessagesAfter = async () => ({ ok: true, messages: [] }); + await service.poll(); + expect(service.getItems()).toHaveLength(1); + }); + + test('a restart resumes from the stored cursor rather than replaying history', async () => { + const { service } = harness({ messages: [message({ id: '10', content: '<@2002> please fix the crash on level three' })] }); + await service.poll(); + + const revived = new DiscordWatchService({ logger: { warn: () => {} }, client: service.client }); + expect(revived.state.cursors.chan1).toBe('10'); + expect(revived.getItems()).toHaveLength(1); + }); + + test('a first-seen channel seeds from now instead of ingesting all history', async () => { + const { service } = harness({ latestId: '500' }); + service.config.backfillMessages = 0; + + const result = await service.poll(); + expect(result.channels[0].seeded).toBe(true); + expect(service.state.cursors.chan1).toBe('500'); + expect(service.getItems()).toEqual([]); + }); + + test('"on it" claims the open item rather than creating another', async () => { + const { service } = harness({ + messages: [ + message({ id: '10', content: '<@2002> please fix the crash on level three' }), + message({ id: '11', author: { id: '2002', username: 'sam' }, content: 'on it, taking this now' }) + ] + }); + + await service.poll(); + const items = service.getItems(); + expect(items).toHaveLength(1); + expect(items[0].status).toBe('claimed'); + expect(items[0].claimedBy).toBe('2002'); + }); + + test('"done" closes the item so it stops showing as outstanding', async () => { + const { service } = harness({ + messages: [ + message({ id: '10', content: '<@2002> please fix the crash on level three' }), + message({ id: '11', author: { id: '2002', username: 'sam' }, content: 'done, that is shipped and merged' }) + ] + }); + + await service.poll(); + expect(service.getItems()[0].status).toBe('done'); + expect(service.getUntracked()).toEqual([]); + }); + + test('untracked work is ordered by priority — the list nobody currently has', async () => { + const { service } = harness({ + messages: [ + message({ id: '10', content: '<@2002> when you get a chance please update the readme' }), + message({ id: '11', content: '<@2002> URGENT production is down, please look now' }) + ] + }); + + await service.poll(); + expect(service.getUntracked().map((item) => item.priority)).toEqual(['urgent', 'low']); + }); + + test('linking a session announces it, which is what makes agent status visible', async () => { + const { service, posted } = harness({ + messages: [message({ id: '10', content: '<@2002> please fix the crash on level three' })] + }); + await service.poll(); + + const item = await service.linkSession('discord:10', 'zoo-game-work1-claude'); + expect(item.status).toBe('in-progress'); + expect(item.sessionId).toBe('zoo-game-work1-claude'); + expect(posted[0].content).toContain('zoo-game-work1-claude'); + expect(posted[0].options.replyToMessageId).toBe('10'); + }); + + test('linking writes a task record so the item gets the right tier', async () => { + const records = []; + const { service } = harness({ messages: [message({ id: '10', content: '<@2002> URGENT please fix the crash now' })] }); + service.init({ taskRecordService: { upsert: (id, patch) => records.push({ id, patch }) } }); + await service.poll(); + + await service.linkSession('discord:10', 'work1-claude', { announce: false }); + expect(records[0].id).toBe('session:work1-claude'); + expect(records[0].patch.tier).toBe(1); + expect(records[0].patch.ticketProvider).toBe('discord'); + }); + + test('start refuses without a token, channels, or being enabled', () => { + const { service } = harness(); + service.config.enabled = false; + expect(service.start().reason).toMatch(/disabled/); + + service.config.enabled = true; + service.config.channels = []; + expect(service.start().reason).toMatch(/no channels/); + }); + + test('a Discord outage is reported, not thrown', async () => { + const { service } = harness(); + service.client.fetchMessagesAfter = async () => ({ ok: false, error: 'HTTP 503' }); + service.state.cursors.chan1 = '1'; + + const result = await service.poll(); + expect(result.channels[0].ok).toBe(false); + expect(service.getStatus().lastError).toBe('HTTP 503'); + }); +}); From fe9221d0ebf935b8e62c344cb3209cc4d524a171 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:42:55 +1000 Subject: [PATCH 10/69] feat(codex): speak the app-server protocol instead of scraping terminals MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The research said this was the highest-value next step, so here it is. `codex app-server` is Apache 2.0, ships in the CLI already installed, and is the JSON-RPC interface behind the Codex app and VS Code extension. Built and verified against the real thing, not against the docs: `initialize` handshake, `thread/list` (which returns `data`, not `threads` — the schema names imply otherwise), and live notification flow. What this replaces: scraped guess -> reported fact 'Do you want to proceed' regex -> ThreadActiveFlag waitingOnApproval cost line means the turn ended -> turn/completed with status + duration buffer-growth busy/idle heuristics -> thread/status/changed (invisible) -> systemError thread state (invisible) -> token usage, rate limits Two new conditions use the structured facts: `structured-approval` fires in seconds because a thread reporting waitingOnApproval is not something you need to wait out a quiet-time threshold to be confident about, and `thread-system-error` delegates a runtime failure rather than nudging it. Approvals can now be granted over the wire with the actual command in hand, instead of typing a keystroke at whatever prompt happens to be on screen. Degradation is the design: `getSignalForSession` returning null means 'no better information than the PTY', so Claude, Gemini and aider are unaffected and everything still works with the app-server off. Opt in with CODEX_APP_SERVER=true. Also lays the realtime groundwork — thread/realtime/* wired for full-duplex voice, transports websocket and webrtc. 803 unit tests green. --- config/supervisor-rules.json | 26 +++ server/agents/appServerClient.js | 237 +++++++++++++++++++++++ server/agents/appServerSignals.js | 234 +++++++++++++++++++++++ server/appServerService.js | 251 +++++++++++++++++++++++++ server/index.js | 20 ++ server/routes/appServerRoutes.js | 109 +++++++++++ server/supervisor/supervisorRules.js | 7 + server/supervisor/supervisorSignals.js | 38 +++- server/supervisorService.js | 7 +- tests/unit/appServerService.test.js | 231 +++++++++++++++++++++++ 10 files changed, 1156 insertions(+), 4 deletions(-) create mode 100644 server/agents/appServerClient.js create mode 100644 server/agents/appServerSignals.js create mode 100644 server/appServerService.js create mode 100644 server/routes/appServerRoutes.js create mode 100644 tests/unit/appServerService.test.js diff --git a/config/supervisor-rules.json b/config/supervisor-rules.json index 8f914a1a..594b311f 100644 --- a/config/supervisor-rules.json +++ b/config/supervisor-rules.json @@ -42,6 +42,32 @@ "$comment_conditions": "Ordered by urgency — the first match per session wins. `resolve` is what JARVIS does about it without asking. `escalateAfterAttempts` is how many failed self-heals before it is allowed to reach you at all.", "conditions": [ + { + "id": "structured-approval", + "label": "Waiting on an approval (reported, not guessed)", + "severity": "warn", + "cooldownSeconds": 60, + "escalateAfterAttempts": 1, + "urgency": { "blocksWork": true }, + "resolve": { "handler": "answer-permission" }, + "$comment": "The Codex app-server reports waitingOnApproval as a fact, so this fires in seconds rather than waiting out a quiet-time threshold to be confident a prompt is really stuck.", + "when": { + "awaitingApproval": true, + "minQuietSeconds": 10 + }, + "advice": "An agent reported it is blocked on an approval that could not be auto-granted safely." + }, + { + "id": "thread-system-error", + "label": "Thread reported a system error", + "severity": "critical", + "cooldownSeconds": 600, + "escalateAfterAttempts": 1, + "urgency": { "blocksWork": true }, + "resolve": { "handler": "delegate-to-commander" }, + "when": { "status": ["error"] }, + "advice": "The agent runtime itself reported a system error — this is not something a nudge fixes." + }, { "id": "awaiting-permission", "label": "Waiting on a permission prompt", diff --git a/server/agents/appServerClient.js b/server/agents/appServerClient.js new file mode 100644 index 00000000..7864a7d7 --- /dev/null +++ b/server/agents/appServerClient.js @@ -0,0 +1,237 @@ +const { EventEmitter } = require('events'); +const { spawn } = require('child_process'); + +const { augmentProcessEnv, getHiddenProcessOptions } = require('../utils/processUtils'); + +const DEFAULT_REQUEST_TIMEOUT_MS = 30_000; +const RESTART_BACKOFF_MS = [1_000, 2_000, 5_000, 15_000, 30_000]; +const MAX_LINE_BYTES = 8 * 1024 * 1024; + +/** + * JSON-RPC client for `codex app-server`. + * + * This is the interface the Codex app and VS Code extension speak. It matters + * because it replaces guesswork with facts: instead of matching "Do you want to + * proceed" in a byte stream, a thread reports `active` with an explicit + * `waitingOnApproval` flag; instead of spotting a cost line, a turn completes. + * + * Per the protocol README the `"jsonrpc":"2.0"` header is omitted on the wire; + * framing is newline-delimited JSON over stdio. + */ +class AppServerClient extends EventEmitter { + constructor({ command = 'codex', args = ['app-server'], cwd = null, logger = console, autoRestart = true } = {}) { + super(); + this.command = command; + this.args = args; + this.cwd = cwd; + this.logger = logger; + this.autoRestart = autoRestart; + + this.child = null; + this.buffer = ''; + this.nextId = 1; + this.pending = new Map(); + this.starting = null; + this.stopped = false; + this.restartAttempts = 0; + this.lastError = null; + this.startedAt = null; + } + + isRunning() { + return Boolean(this.child && !this.child.killed && this.child.exitCode === null); + } + + async start() { + if (this.isRunning()) return { running: true, alreadyRunning: true }; + if (this.starting) return this.starting; + + this.stopped = false; + this.starting = new Promise((resolve) => { + try { + this.child = spawn(this.command, this.args, { + ...getHiddenProcessOptions({ stdio: ['pipe', 'pipe', 'pipe'] }), + cwd: this.cwd || undefined, + env: augmentProcessEnv(process.env) + }); + } catch (error) { + this.lastError = error.message; + this.starting = null; + resolve({ running: false, error: error.message }); + return; + } + + this.child.stdout.setEncoding('utf8'); + this.child.stdout.on('data', (chunk) => this.consume(chunk)); + this.child.stderr.setEncoding('utf8'); + this.child.stderr.on('data', (chunk) => { + const text = String(chunk).trim(); + if (text) this.logger.debug?.('[app-server]', text); + }); + + this.child.on('error', (error) => { + this.lastError = error.message; + this.emit('error', error); + }); + + this.child.on('exit', (code, signal) => { + this.rejectAllPending(new Error(`app-server exited (code ${code}, signal ${signal})`)); + this.child = null; + this.emit('exit', { code, signal }); + if (!this.stopped && this.autoRestart) this.scheduleRestart(); + }); + + this.startedAt = new Date().toISOString(); + this.restartAttempts = 0; + this.starting = null; + resolve({ running: true, pid: this.child.pid }); + }); + + return this.starting; + } + + scheduleRestart() { + const delay = RESTART_BACKOFF_MS[Math.min(this.restartAttempts, RESTART_BACKOFF_MS.length - 1)]; + this.restartAttempts += 1; + const timer = setTimeout(() => { + if (!this.stopped) this.start().catch(() => {}); + }, delay); + if (typeof timer.unref === 'function') timer.unref(); + } + + stop() { + this.stopped = true; + this.rejectAllPending(new Error('app-server client stopped')); + if (this.child) { + try { + this.child.kill('SIGTERM'); + } catch { + // Already gone. + } + this.child = null; + } + return { running: false }; + } + + rejectAllPending(error) { + for (const [, entry] of this.pending) { + clearTimeout(entry.timer); + entry.reject(error); + } + this.pending.clear(); + } + + consume(chunk) { + this.buffer += chunk; + if (this.buffer.length > MAX_LINE_BYTES) { + this.logger.warn?.('app-server output exceeded the line buffer; dropping it'); + this.buffer = ''; + return; + } + + let index = this.buffer.indexOf('\n'); + while (index !== -1) { + const line = this.buffer.slice(0, index).trim(); + this.buffer = this.buffer.slice(index + 1); + if (line) this.handleLine(line); + index = this.buffer.indexOf('\n'); + } + } + + handleLine(line) { + let message; + try { + message = JSON.parse(line); + } catch { + // The app-server occasionally logs non-JSON on stdout during startup. + return; + } + + if (message.id !== undefined && (message.result !== undefined || message.error !== undefined)) { + const entry = this.pending.get(message.id); + if (!entry) return; + clearTimeout(entry.timer); + this.pending.delete(message.id); + if (message.error) entry.reject(Object.assign(new Error(message.error.message || 'app-server error'), { data: message.error })); + else entry.resolve(message.result); + return; + } + + if (message.method && message.id !== undefined) { + // A server->client request (approvals, elicitation). Emit it so a policy + // layer can answer; unanswered requests are the caller's problem, not ours. + this.emit('request', { id: message.id, method: message.method, params: message.params || {} }); + return; + } + + if (message.method) { + this.emit('notification', { method: message.method, params: message.params || {} }); + this.emit(message.method, message.params || {}); + } + } + + send(payload) { + if (!this.isRunning()) return false; + try { + this.child.stdin.write(`${JSON.stringify(payload)}\n`); + return true; + } catch (error) { + this.lastError = error.message; + return false; + } + } + + request(method, params = {}, { timeoutMs = DEFAULT_REQUEST_TIMEOUT_MS } = {}) { + return new Promise((resolve, reject) => { + if (!this.isRunning()) { + reject(new Error('app-server is not running')); + return; + } + + const id = this.nextId; + this.nextId += 1; + + const timer = setTimeout(() => { + this.pending.delete(id); + reject(new Error(`app-server request "${method}" timed out`)); + }, timeoutMs); + if (typeof timer.unref === 'function') timer.unref(); + + this.pending.set(id, { resolve, reject, timer, method }); + if (!this.send({ id, method, params })) { + clearTimeout(timer); + this.pending.delete(id); + reject(new Error(`could not write "${method}" to app-server`)); + } + }); + } + + notify(method, params = {}) { + return this.send({ method, params }); + } + + /** + * Answer a server->client request, which is how approvals are granted over the + * protocol instead of by typing "1" into a terminal and hoping. + */ + respond(id, result) { + return this.send({ id, result }); + } + + respondError(id, message, code = -32000) { + return this.send({ id, error: { code, message } }); + } + + getStatus() { + return { + running: this.isRunning(), + pid: this.child?.pid || null, + startedAt: this.startedAt, + pendingRequests: this.pending.size, + restartAttempts: this.restartAttempts, + lastError: this.lastError + }; + } +} + +module.exports = { AppServerClient, DEFAULT_REQUEST_TIMEOUT_MS }; diff --git a/server/agents/appServerSignals.js b/server/agents/appServerSignals.js new file mode 100644 index 00000000..0fef08bd --- /dev/null +++ b/server/agents/appServerSignals.js @@ -0,0 +1,234 @@ +const { EventEmitter } = require('events'); + +// From the protocol: ThreadStatus is a tagged union, and an active thread +// carries flags saying *why* it is active. `waitingOnApproval` is the fact we +// currently reconstruct by pattern-matching prose that changes between releases. +const ACTIVE_FLAGS = { WAITING_ON_APPROVAL: 'waitingOnApproval', WAITING_ON_USER_INPUT: 'waitingOnUserInput' }; + +const NOTIFICATIONS = { + THREAD_STATUS_CHANGED: 'thread/status/changed', + THREAD_STARTED: 'thread/started', + THREAD_CLOSED: 'thread/closed', + TURN_STARTED: 'turn/started', + TURN_COMPLETED: 'turn/completed', + TOKEN_USAGE: 'thread/tokenUsage/updated', + RATE_LIMITS: 'account/rateLimits/updated', + ERROR: 'error', + ITEM_COMPLETED: 'item/completed', + COMMAND_APPROVAL: 'item/commandExecution/requestApproval', + FILE_APPROVAL: 'item/fileChange/requestApproval' +}; + +/** + * Translates app-server notifications into the shape the supervisor already + * understands, so structured signals slot in beside scraped ones rather than + * requiring a second rule engine. + * + * Where a thread is tracked here, its state is *known* rather than inferred: + * `waiting` means the thread told us it is waiting, and for what. + */ +class AppServerSignalSource extends EventEmitter { + constructor({ client, logger = console } = {}) { + super(); + this.client = client; + this.logger = logger; + this.threads = new Map(); + this.pendingApprovals = new Map(); + this.bound = false; + } + + bind() { + if (this.bound || !this.client) return this; + this.bound = true; + + this.client.on('notification', ({ method, params }) => { + try { + this.handleNotification(method, params); + } catch (error) { + this.logger.warn?.('app-server signal mapping failed', { method, error: error.message }); + } + }); + + this.client.on('request', ({ id, method, params }) => { + if (method === NOTIFICATIONS.COMMAND_APPROVAL || method === NOTIFICATIONS.FILE_APPROVAL) { + this.recordApprovalRequest(id, method, params); + } + }); + + this.client.on('exit', () => { + for (const state of this.threads.values()) state.stale = true; + }); + + return this; + } + + ensureThread(threadId) { + if (!threadId) return null; + if (!this.threads.has(threadId)) { + this.threads.set(threadId, { + threadId, + status: 'unknown', + activeFlags: [], + lastEventAt: Date.now(), + turnId: null, + turnStartedAt: null, + lastTurnStatus: null, + lastTurnError: null, + tokenUsage: null, + rateLimits: null, + lastError: null, + stale: false + }); + } + return this.threads.get(threadId); + } + + /** + * The union arrives as `{ type, activeFlags? }`; flatten it to the vocabulary + * the supervisor rule table already uses (busy / waiting / idle / error). + */ + mapStatus(raw) { + const type = String(raw?.type || '').trim(); + const flags = Array.isArray(raw?.activeFlags) ? raw.activeFlags : []; + + if (type === 'active') { + const waiting = flags.includes(ACTIVE_FLAGS.WAITING_ON_APPROVAL) || flags.includes(ACTIVE_FLAGS.WAITING_ON_USER_INPUT); + return { status: waiting ? 'waiting' : 'busy', activeFlags: flags }; + } + if (type === 'idle') return { status: 'idle', activeFlags: [] }; + if (type === 'systemError') return { status: 'error', activeFlags: [] }; + return { status: 'unknown', activeFlags: [] }; + } + + handleNotification(method, params) { + const threadId = params?.threadId || params?.thread_id || null; + const state = this.ensureThread(threadId); + if (!state) return; + + state.lastEventAt = Date.now(); + state.stale = false; + + switch (method) { + case NOTIFICATIONS.THREAD_STATUS_CHANGED: { + const mapped = this.mapStatus(params?.status); + state.status = mapped.status; + state.activeFlags = mapped.activeFlags; + this.emit('status', { threadId, ...mapped }); + break; + } + case NOTIFICATIONS.TURN_STARTED: + state.turnId = params?.turn?.id || null; + state.turnStartedAt = Date.now(); + state.status = state.status === 'waiting' ? 'waiting' : 'busy'; + break; + case NOTIFICATIONS.TURN_COMPLETED: + state.turnId = null; + state.lastTurnStatus = params?.turn?.status || null; + state.lastTurnError = params?.turn?.error || null; + state.lastTurnDurationMs = params?.turn?.durationMs ?? null; + state.status = 'idle'; + this.emit('turn-completed', { threadId, turn: params?.turn || null }); + break; + case NOTIFICATIONS.TOKEN_USAGE: + state.tokenUsage = params?.tokenUsage || null; + break; + case NOTIFICATIONS.RATE_LIMITS: + state.rateLimits = params || null; + this.emit('rate-limits', params || {}); + break; + case NOTIFICATIONS.THREAD_CLOSED: + this.threads.delete(threadId); + break; + case NOTIFICATIONS.ERROR: + state.lastError = params?.message || params?.error || 'unknown error'; + break; + default: + break; + } + } + + recordApprovalRequest(id, method, params) { + const threadId = params?.threadId || null; + const state = this.ensureThread(threadId); + const entry = { + requestId: id, + method, + threadId, + command: params?.command || params?.changes || null, + reason: params?.reason || '', + requestedAt: Date.now() + }; + + this.pendingApprovals.set(id, entry); + if (state) { + state.status = 'waiting'; + state.activeFlags = [ACTIVE_FLAGS.WAITING_ON_APPROVAL]; + state.lastEventAt = Date.now(); + } + this.emit('approval-request', entry); + } + + /** + * Grant or refuse an approval over the wire. Compare with the PTY path, which + * can only type a keystroke at whatever prompt happens to be showing. + */ + answerApproval(requestId, approved, { note = '' } = {}) { + const entry = this.pendingApprovals.get(requestId); + if (!entry) return { ok: false, error: `no pending approval "${requestId}"` }; + + const sent = this.client?.respond(requestId, { decision: approved ? 'approved' : 'denied', note }); + this.pendingApprovals.delete(requestId); + + const state = this.threads.get(entry.threadId); + if (state && approved) { + state.status = 'busy'; + state.activeFlags = []; + } + return { ok: Boolean(sent), approved, threadId: entry.threadId }; + } + + listPendingApprovals() { + return [...this.pendingApprovals.values()]; + } + + /** + * A supervisor-shaped signal for one thread, or null when this source knows + * nothing about it — in which case the PTY scraper remains authoritative. + */ + getSignal(threadId) { + const state = this.threads.get(threadId); + if (!state || state.stale) return null; + + const quietSeconds = Math.max(0, Math.round((Date.now() - state.lastEventAt) / 1000)); + return { + source: 'app-server', + threadId, + status: state.status, + activeFlags: state.activeFlags, + awaitingApproval: state.activeFlags.includes(ACTIVE_FLAGS.WAITING_ON_APPROVAL), + awaitingUserInput: state.activeFlags.includes(ACTIVE_FLAGS.WAITING_ON_USER_INPUT), + quietSeconds, + turnId: state.turnId, + lastTurnStatus: state.lastTurnStatus, + lastTurnError: state.lastTurnError, + tokenUsage: state.tokenUsage, + rateLimits: state.rateLimits, + lastError: state.lastError + }; + } + + listThreads() { + return [...this.threads.keys()].map((threadId) => this.getSignal(threadId)).filter(Boolean); + } + + getStatus() { + return { + bound: this.bound, + threadCount: this.threads.size, + pendingApprovals: this.pendingApprovals.size, + threads: this.listThreads() + }; + } +} + +module.exports = { AppServerSignalSource, ACTIVE_FLAGS, NOTIFICATIONS }; diff --git a/server/appServerService.js b/server/appServerService.js new file mode 100644 index 00000000..eaff3bc4 --- /dev/null +++ b/server/appServerService.js @@ -0,0 +1,251 @@ +const { AppServerClient } = require('./agents/appServerClient'); +const { AppServerSignalSource } = require('./agents/appServerSignals'); + +const REALTIME = { + START: 'thread/realtime/start', + STOP: 'thread/realtime/stop', + APPEND_TEXT: 'thread/realtime/appendText', + APPEND_AUDIO: 'thread/realtime/appendAudio', + APPEND_SPEECH: 'thread/realtime/appendSpeech', + LIST_VOICES: 'thread/realtime/listVoices' +}; + +const REALTIME_EVENTS = [ + 'thread/realtime/started', + 'thread/realtime/closed', + 'thread/realtime/error', + 'thread/realtime/itemAdded', + 'thread/realtime/transcript/delta', + 'thread/realtime/transcript/done', + 'thread/realtime/outputAudio/delta', + 'thread/realtime/sdp' +]; + +/** + * Talks to `codex app-server` so Codex sessions can report facts instead of + * being guessed at, and so the full-duplex realtime voice pipeline is usable + * from here rather than only from OpenAI's own (Codex-only, macOS-only) app. + * + * Opt-in. Nothing regresses when it is off: the PTY scraper stays the universal + * signal source for Claude, Gemini, aider and anything else. + */ +class AppServerService { + constructor({ logger = console } = {}) { + this.logger = logger; + this.enabled = String(process.env.CODEX_APP_SERVER || '').toLowerCase() === 'true'; + this.client = new AppServerClient({ logger }); + this.signals = new AppServerSignalSource({ client: this.client, logger }); + this.io = null; + this.speechService = null; + this.realtimeThreads = new Map(); + this.transcripts = []; + this.maxTranscripts = 200; + this.serverInfo = null; + } + + static getInstance(options = {}) { + if (!AppServerService.instance) { + AppServerService.instance = new AppServerService(options); + } + return AppServerService.instance; + } + + init({ io, speechService } = {}) { + this.io = io || this.io; + this.speechService = speechService || this.speechService; + this.signals.bind(); + this.bindRealtime(); + return this; + } + + bindRealtime() { + if (this.realtimeBound || !this.client) return; + this.realtimeBound = true; + + for (const event of REALTIME_EVENTS) { + this.client.on(event, (params) => this.handleRealtimeEvent(event, params)); + } + + // An approval arriving over the wire is worth surfacing immediately — it is + // the thing most likely to be silently blocking a session. + this.signals.on('approval-request', (entry) => { + this.io?.emit('app-server-approval', entry); + }); + } + + handleRealtimeEvent(event, params = {}) { + const threadId = params?.threadId || null; + + if (event === 'thread/realtime/started') { + this.realtimeThreads.set(threadId, { threadId, startedAt: new Date().toISOString() }); + } + if (event === 'thread/realtime/closed' || event === 'thread/realtime/error') { + this.realtimeThreads.delete(threadId); + } + + if (event === 'thread/realtime/transcript/delta' || event === 'thread/realtime/transcript/done') { + this.recordTranscript({ + threadId, + role: params?.role || 'assistant', + text: params?.delta || params?.text || '', + done: event.endsWith('done'), + at: new Date().toISOString() + }); + } + + // Audio deltas are large and high-frequency; forward the fact, not the bytes. + const payload = event === 'thread/realtime/outputAudio/delta' + ? { threadId, bytes: (params?.delta || '').length } + : params; + + this.io?.emit('app-server-realtime', { event, payload }); + } + + recordTranscript(entry) { + this.transcripts.unshift(entry); + if (this.transcripts.length > this.maxTranscripts) this.transcripts.length = this.maxTranscripts; + this.io?.emit('app-server-transcript', entry); + } + + async start() { + if (!this.enabled) return { running: false, reason: 'CODEX_APP_SERVER is not enabled' }; + + const started = await this.client.start(); + if (!started.running) return started; + + this.signals.bind(); + this.bindRealtime(); + + // The protocol requires an initialize handshake before anything else; the + // server answers with codexHome and platform details. + try { + this.serverInfo = await this.client.request('initialize', { + clientInfo: { name: 'agent-workspace', version: require('../package.json').version || '1.0.0' } + }, { timeoutMs: 15_000 }); + } catch (error) { + this.logger.warn?.('app-server initialize failed', { error: error.message }); + this.serverInfo = null; + } + + return { ...started, initialized: Boolean(this.serverInfo) }; + } + + stop() { + this.realtimeThreads.clear(); + return this.client.stop(); + } + + setEnabled(enabled) { + this.enabled = enabled === true; + if (!this.enabled) this.stop(); + return this.enabled; + } + + /** + * The protocol returns paged threads under `data`, not `threads` — verified + * against a live app-server rather than assumed from the schema names. + */ + async listThreads({ limit = 50 } = {}) { + const result = await this.client.request('thread/list', { limit }); + if (Array.isArray(result?.data)) return result.data; + if (Array.isArray(result?.threads)) return result.threads; + return Array.isArray(result) ? result : []; + } + + async startThread(params = {}) { + return this.client.request('thread/start', params); + } + + async resumeThread(threadId, params = {}) { + return this.client.request('thread/resume', { threadId, ...params }); + } + + async startTurn(threadId, input) { + return this.client.request('turn/start', { threadId, input }); + } + + async interruptTurn(threadId) { + return this.client.request('turn/interrupt', { threadId }); + } + + // ---- Realtime voice ----------------------------------------------------- + + /** + * Open a full-duplex realtime session on a thread. `websocket` is the default + * transport because it needs no peer-connection setup; `webrtc` is available + * for a browser that wants to negotiate directly. + */ + async startRealtime(threadId, { transport = 'websocket', sdp = '', voice = '' } = {}) { + const params = { + threadId, + transport: transport === 'webrtc' ? { type: 'webrtc', sdp } : { type: 'websocket' } + }; + if (voice) params.voice = voice; + const result = await this.client.request(REALTIME.START, params); + this.realtimeThreads.set(threadId, { threadId, transport, startedAt: new Date().toISOString() }); + return result; + } + + async stopRealtime(threadId) { + const result = await this.client.request(REALTIME.STOP, { threadId }); + this.realtimeThreads.delete(threadId); + return result; + } + + async sendRealtimeText(threadId, text) { + return this.client.request(REALTIME.APPEND_TEXT, { threadId, text: String(text || '') }); + } + + async sendRealtimeAudio(threadId, audioBase64) { + // Audio is streamed, so this is a notification rather than a round trip. + return this.client.notify(REALTIME.APPEND_AUDIO, { threadId, audio: audioBase64 }); + } + + async listVoices() { + return this.client.request(REALTIME.LIST_VOICES, {}); + } + + getTranscripts({ threadId = '', limit = 50 } = {}) { + const wanted = String(threadId || '').trim(); + return this.transcripts + .filter((entry) => (!wanted || entry.threadId === wanted)) + .slice(0, Math.max(1, Number(limit) || 50)); + } + + // ---- Supervisor integration -------------------------------------------- + + /** + * Structured signal for a session, if this source knows about it. Returning + * null is meaningful: it means "no better information than the PTY", and the + * supervisor keeps scraping. + */ + getSignalForSession(session) { + const threadId = session?.appServerThreadId || session?.threadId || null; + if (!threadId) return null; + return this.signals.getSignal(threadId); + } + + listPendingApprovals() { + return this.signals.listPendingApprovals(); + } + + answerApproval(requestId, approved, options = {}) { + return this.signals.answerApproval(requestId, approved, options); + } + + getStatus() { + return { + enabled: this.enabled, + serverInfo: this.serverInfo, + client: this.client.getStatus(), + signals: this.signals.getStatus(), + realtimeThreads: [...this.realtimeThreads.values()], + transcriptCount: this.transcripts.length + }; + } +} + +module.exports = AppServerService; +module.exports.AppServerService = AppServerService; +module.exports.REALTIME = REALTIME; +module.exports.REALTIME_EVENTS = REALTIME_EVENTS; diff --git a/server/index.js b/server/index.js index 9bb970e2..6280336f 100644 --- a/server/index.js +++ b/server/index.js @@ -118,6 +118,8 @@ const { SpeechService } = require('./speechService'); const { createSpeechRoutes } = require('./routes/speechRoutes'); const { DiscordWatchService } = require('./discordWatchService'); const { createDiscordWatchRoutes } = require('./routes/discordWatchRoutes'); +const { AppServerService } = require('./appServerService'); +const { createAppServerRoutes } = require('./routes/appServerRoutes'); const { ProductLauncherService } = require('./productLauncherService'); const { CommanderService } = require('./commanderService'); const { ConversationService } = require('./conversationService'); @@ -362,6 +364,7 @@ const speechService = SpeechService.getInstance({ logger }); speechService.setIO(io); const supervisorService = SupervisorService.getInstance({ logger }); const discordWatchService = DiscordWatchService.getInstance({ logger }); +const appServerService = AppServerService.getInstance({ logger }); const activityFeed = ActivityFeedService.getInstance(); activityFeed.setIO(io); activityFeed.track('server.started', { port: Number(process.env.ORCHESTRATOR_PORT || 9460) }); @@ -454,6 +457,10 @@ supervisorService.init({ activityFeed, notificationService, speechService, + // Facts beat inference: where the Codex app-server knows a thread's state, + // it replaces the scraped guess. Null means "no better information", and the + // PTY scraper stays authoritative. + structuredSource: appServerService, // When rules cannot fix something, the Commander gets a written problem brief // before you do — it is a full agent with the whole API, and it only costs // tokens when something is actually wrong. @@ -465,6 +472,12 @@ if (String(process.env.SUPERVISOR_AUTOSTART || 'true').toLowerCase() !== 'false' logger.info('Supervisor', started); } +// Opt-in via CODEX_APP_SERVER=true; everything degrades to PTY scraping without it. +appServerService.init({ io, speechService }); +appServerService.start().then((started) => { + if (started.running) logger.info('Codex app-server', started); +}).catch((error) => logger.warn('Codex app-server did not start', { error: error.message })); + // Off unless explicitly configured — it needs a bot token and channel ids. discordWatchService.init({ taskRecordService, activityFeed }); const discordWatchStarted = discordWatchService.start(); @@ -1400,6 +1413,13 @@ app.use('/api/supervisor', createSupervisorRoutes({ requireWrite: requirePolicyAction('write') })); +app.use('/api/app-server', createAppServerRoutes({ + appServerService, + logger, + requireRead: requirePolicyAction('read'), + requireWrite: requirePolicyAction('write') +})); + app.use('/api/discord-watch', createDiscordWatchRoutes({ discordWatchService, logger, diff --git a/server/routes/appServerRoutes.js b/server/routes/appServerRoutes.js new file mode 100644 index 00000000..ddc490ad --- /dev/null +++ b/server/routes/appServerRoutes.js @@ -0,0 +1,109 @@ +const express = require('express'); + +const passthrough = (req, res, next) => next(); + +/** + * `/api/app-server/*` — structured Codex control and the realtime voice bridge. + * + * Answering an approval is a `write`: it authorizes an agent to do something. + */ +function createAppServerRoutes({ appServerService, logger = console, requireRead = passthrough, requireWrite = passthrough } = {}) { + const router = express.Router(); + + const handle = (label, handler) => async (req, res) => { + try { + await handler(req, res); + } catch (error) { + logger.error(`App-server: ${label} failed`, { error: error.message }); + res.status(400).json({ ok: false, error: error.message }); + } + }; + + router.get('/status', requireRead, handle('status', (req, res) => { + res.json({ ok: true, status: appServerService.getStatus() }); + })); + + router.post('/start', requireWrite, handle('start', async (req, res) => { + if (req.body?.enable === true) appServerService.setEnabled(true); + res.json({ ok: true, ...(await appServerService.start()) }); + })); + + router.post('/stop', requireWrite, handle('stop', (req, res) => { + res.json({ ok: true, ...appServerService.stop() }); + })); + + router.get('/threads', requireRead, handle('list threads', async (req, res) => { + res.json({ ok: true, threads: await appServerService.listThreads() }); + })); + + router.get('/signals', requireRead, handle('signals', (req, res) => { + res.json({ ok: true, signals: appServerService.signals.listThreads() }); + })); + + router.post('/threads', requireWrite, handle('start thread', async (req, res) => { + res.json({ ok: true, thread: await appServerService.startThread(req.body || {}) }); + })); + + router.post('/threads/:threadId/turn', requireWrite, handle('start turn', async (req, res) => { + res.json({ ok: true, turn: await appServerService.startTurn(req.params.threadId, req.body?.input) }); + })); + + router.post('/threads/:threadId/interrupt', requireWrite, handle('interrupt turn', async (req, res) => { + res.json({ ok: true, result: await appServerService.interruptTurn(req.params.threadId) }); + })); + + /** + * Approvals answered over the wire, with the actual command in hand — rather + * than typing a keystroke at whatever prompt happens to be on screen. + */ + router.get('/approvals', requireRead, handle('list approvals', (req, res) => { + res.json({ ok: true, approvals: appServerService.listPendingApprovals() }); + })); + + router.post('/approvals/:requestId', requireWrite, handle('answer approval', (req, res) => { + const approved = req.body?.approved === true; + res.json({ ok: true, ...appServerService.answerApproval(req.params.requestId, approved, { note: req.body?.note || '' }) }); + })); + + // ---- Realtime voice ---------------------------------------------------- + + router.get('/realtime/voices', requireRead, handle('list voices', async (req, res) => { + res.json({ ok: true, voices: await appServerService.listVoices() }); + })); + + router.post('/realtime/:threadId/start', requireWrite, handle('start realtime', async (req, res) => { + const result = await appServerService.startRealtime(req.params.threadId, { + transport: req.body?.transport || 'websocket', + sdp: req.body?.sdp || '', + voice: req.body?.voice || '' + }); + res.json({ ok: true, realtime: result }); + })); + + router.post('/realtime/:threadId/stop', requireWrite, handle('stop realtime', async (req, res) => { + res.json({ ok: true, result: await appServerService.stopRealtime(req.params.threadId) }); + })); + + router.post('/realtime/:threadId/text', requireWrite, handle('send realtime text', async (req, res) => { + const text = String(req.body?.text || '').trim(); + if (!text) return res.status(400).json({ ok: false, error: 'text is required' }); + return res.json({ ok: true, result: await appServerService.sendRealtimeText(req.params.threadId, text) }); + })); + + router.post('/realtime/:threadId/audio', requireWrite, handle('send realtime audio', async (req, res) => { + const audio = String(req.body?.audio || ''); + if (!audio) return res.status(400).json({ ok: false, error: 'audio (base64) is required' }); + return res.json({ ok: true, sent: await appServerService.sendRealtimeAudio(req.params.threadId, audio) }); + })); + + router.get('/realtime/transcripts', requireRead, handle('transcripts', (req, res) => { + res.json({ + ok: true, + transcripts: appServerService.getTranscripts({ threadId: req.query.threadId, limit: req.query.limit }) + }); + })); + + return router; +} + +module.exports = { createAppServerRoutes }; diff --git a/server/supervisor/supervisorRules.js b/server/supervisor/supervisorRules.js index 5398bc3c..a641f00b 100644 --- a/server/supervisor/supervisorRules.js +++ b/server/supervisor/supervisorRules.js @@ -77,6 +77,9 @@ function normalizeCondition(raw) { maxQuietSeconds: Number.isFinite(Number(when.maxQuietSeconds)) ? Number(when.maxQuietSeconds) : null, repeatedTailLine: Number.isFinite(Number(when.repeatedTailLine)) ? Number(when.repeatedTailLine) : null, agentPresent: typeof when.agentPresent === 'boolean' ? when.agentPresent : null, + awaitingApproval: typeof when.awaitingApproval === 'boolean' ? when.awaitingApproval : null, + awaitingUserInput: typeof when.awaitingUserInput === 'boolean' ? when.awaitingUserInput : null, + signalSource: String(when.signalSource || '').trim(), tiers: (Array.isArray(when.tiers) ? when.tiers : []).map(Number).filter(Number.isFinite), git: when.git && typeof when.git === 'object' ? { dirty: typeof when.git.dirty === 'boolean' ? when.git.dirty : null, @@ -145,6 +148,9 @@ function matches(condition, signal) { if (when.status.length && !when.status.includes(signal.status)) return false; if (when.types.length && !when.types.includes(signal.type)) return false; if (when.agentPresent !== null && Boolean(signal.agentPresent) !== when.agentPresent) return false; + if (when.awaitingApproval !== null && Boolean(signal.awaitingApproval) !== when.awaitingApproval) return false; + if (when.awaitingUserInput !== null && Boolean(signal.awaitingUserInput) !== when.awaitingUserInput) return false; + if (when.signalSource && signal.signalSource !== when.signalSource) return false; if (when.tiers.length && !when.tiers.includes(Number(signal.tier))) return false; if (when.minQuietSeconds !== null && signal.quietSeconds < when.minQuietSeconds) return false; if (when.maxQuietSeconds !== null && signal.quietSeconds > when.maxQuietSeconds) return false; @@ -216,6 +222,7 @@ function buildFinding(condition, signal) { tier: signal.tier, ticketTitle: signal.ticketTitle, status: signal.status, + signalSource: signal.signalSource || 'pty', quietSeconds: signal.quietSeconds, advice: condition.advice, evidence: signal.lastLine, diff --git a/server/supervisor/supervisorSignals.js b/server/supervisor/supervisorSignals.js index a223c5c3..64fd6b19 100644 --- a/server/supervisor/supervisorSignals.js +++ b/server/supervisor/supervisorSignals.js @@ -142,12 +142,42 @@ async function collectGitState(gitHelper, cwd) { } } +/** + * Prefer facts over inference. + * + * When a structured source (the Codex app-server) knows a thread's state, it + * replaces the scraped guess: `waiting` because the thread said it is waiting on + * an approval beats `waiting` because a regex matched some prose. Quiet time + * stays PTY-derived where the PTY is the thing actually being watched, and the + * scraped tail is kept either way so tail-matching rules still work. + */ +function applyStructuredSignal(signal, structured) { + if (!structured) return signal; + + return { + ...signal, + signalSource: structured.source || 'app-server', + status: structured.status && structured.status !== 'unknown' ? structured.status : signal.status, + quietSeconds: Number.isFinite(structured.quietSeconds) ? structured.quietSeconds : signal.quietSeconds, + awaitingApproval: structured.awaitingApproval === true, + awaitingUserInput: structured.awaitingUserInput === true, + activeFlags: structured.activeFlags || [], + tokenUsage: structured.tokenUsage || null, + rateLimits: structured.rateLimits || null, + lastTurnStatus: structured.lastTurnStatus || null, + lastTurnError: structured.lastTurnError || null, + structuredError: structured.lastError || null, + threadId: structured.threadId || null + }; +} + async function gatherSignals({ sessionManager, gitHelper, sessionRecoveryService, taskRecordService, quietTracker, + structuredSource = null, gitQuietThresholdSeconds = 120 } = {}) { const supervised = listSupervisedSessions(sessionManager); @@ -170,8 +200,9 @@ async function gatherSignals({ const record = taskRecordService?.get?.(`session:${id}`) || null; - signals.push({ + const base = { sessionId: id, + signalSource: 'pty', type: String(session.type || '').toLowerCase(), status: String(session.status || 'idle').toLowerCase(), agent: recovery?.lastAgent || (session.type === 'codex' ? 'codex' : null), @@ -188,7 +219,9 @@ async function gatherSignals({ git, tier: Number(record?.tier) || null, ticketTitle: record?.ticketTitle || null - }); + }; + + signals.push(applyStructuredSignal(base, structuredSource?.getSignalForSession?.(session) || null)); } return signals; @@ -202,6 +235,7 @@ module.exports = { lastNonEmptyLines, maxLineRepeat, listSupervisedSessions, + applyStructuredSignal, countUnpushedCommits, collectGitState, gatherSignals diff --git a/server/supervisorService.js b/server/supervisorService.js index 6bb531a7..1c3b748d 100644 --- a/server/supervisorService.js +++ b/server/supervisorService.js @@ -41,6 +41,7 @@ class SupervisorService { this.notificationService = null; this.speechService = null; this.commanderSender = null; + this.structuredSource = null; this.rules = rulesModule.loadRules(); this.quietTracker = new QuietTracker(); @@ -73,7 +74,7 @@ class SupervisorService { init({ sessionManager, gitHelper, agentManager, sessionRecoveryService, - taskRecordService, activityFeed, notificationService, speechService, commanderSender + taskRecordService, activityFeed, notificationService, speechService, commanderSender, structuredSource } = {}) { this.sessionManager = sessionManager || this.sessionManager; this.gitHelper = gitHelper || this.gitHelper; @@ -84,6 +85,7 @@ class SupervisorService { this.notificationService = notificationService || this.notificationService; this.speechService = speechService || this.speechService; this.commanderSender = commanderSender || this.commanderSender; + this.structuredSource = structuredSource || this.structuredSource; this.executor = createExecutor({ sessionManager: this.sessionManager, @@ -254,7 +256,8 @@ class SupervisorService { gitHelper: this.gitHelper, sessionRecoveryService: this.sessionRecoveryService, taskRecordService: this.taskRecordService, - quietTracker: this.quietTracker + quietTracker: this.quietTracker, + structuredSource: this.structuredSource }); const findings = rulesModule.evaluate(signals, this.rules); diff --git a/tests/unit/appServerService.test.js b/tests/unit/appServerService.test.js new file mode 100644 index 00000000..367102e5 --- /dev/null +++ b/tests/unit/appServerService.test.js @@ -0,0 +1,231 @@ +const { EventEmitter } = require('events'); + +const { AppServerClient } = require('../../server/agents/appServerClient'); +const { AppServerSignalSource, ACTIVE_FLAGS } = require('../../server/agents/appServerSignals'); +const { applyStructuredSignal } = require('../../server/supervisor/supervisorSignals'); +const { loadRules, DEFAULT_RULES_PATH, evaluate } = require('../../server/supervisor/supervisorRules'); + +/** A client stand-in that lets tests push protocol frames without spawning codex. */ +class FakeClient extends EventEmitter { + constructor() { + super(); + this.responses = []; + } + + respond(id, result) { + this.responses.push({ id, result }); + return true; + } + + emitNotification(method, params) { + this.emit('notification', { method, params }); + this.emit(method, params); + } + + emitRequest(id, method, params) { + this.emit('request', { id, method, params }); + } +} + +describe('AppServerClient framing', () => { + test('parses newline-delimited JSON and resolves the matching request', async () => { + const client = new AppServerClient({ autoRestart: false }); + const resolved = []; + client.pending.set(7, { resolve: (v) => resolved.push(v), reject: () => {}, timer: setTimeout(() => {}, 0), method: 'x' }); + + client.consume('{"id":7,"result":{"threadId":"t1"}}\n'); + expect(resolved).toEqual([{ threadId: 't1' }]); + }); + + test('a frame split across chunks is reassembled', () => { + const client = new AppServerClient({ autoRestart: false }); + const seen = []; + client.on('notification', (n) => seen.push(n.method)); + + client.consume('{"method":"turn/'); + client.consume('started","params":{"threadId":"t1"}}\n'); + expect(seen).toEqual(['turn/started']); + }); + + test('non-JSON stdout noise is skipped rather than throwing', () => { + const client = new AppServerClient({ autoRestart: false }); + expect(() => client.consume('starting up...\n{"method":"x"}\n')).not.toThrow(); + }); + + test('a server error response rejects with the protocol message', async () => { + const client = new AppServerClient({ autoRestart: false }); + const rejected = []; + client.pending.set(1, { resolve: () => {}, reject: (e) => rejected.push(e.message), timer: setTimeout(() => {}, 0), method: 'x' }); + + client.consume('{"id":1,"error":{"code":-32000,"message":"nope"}}\n'); + expect(rejected).toEqual(['nope']); + }); + + test('requests fail fast when the server is not running', async () => { + await expect(new AppServerClient({ autoRestart: false }).request('thread/list')).rejects.toThrow(/not running/); + }); +}); + +describe('AppServerSignalSource', () => { + const source = () => { + const client = new FakeClient(); + return { client, signals: new AppServerSignalSource({ client, logger: { warn: () => {} } }).bind() }; + }; + + test('an active thread waiting on approval is reported as waiting, not guessed', () => { + const { client, signals } = source(); + client.emitNotification('thread/status/changed', { + threadId: 't1', + status: { type: 'active', activeFlags: [ACTIVE_FLAGS.WAITING_ON_APPROVAL] } + }); + + const signal = signals.getSignal('t1'); + expect(signal.status).toBe('waiting'); + expect(signal.awaitingApproval).toBe(true); + expect(signal.source).toBe('app-server'); + }); + + test('an active thread with no flags is simply busy', () => { + const { client, signals } = source(); + client.emitNotification('thread/status/changed', { threadId: 't1', status: { type: 'active', activeFlags: [] } }); + expect(signals.getSignal('t1').status).toBe('busy'); + expect(signals.getSignal('t1').awaitingApproval).toBe(false); + }); + + test('systemError surfaces as an error state we cannot see from a PTY at all', () => { + const { client, signals } = source(); + client.emitNotification('thread/status/changed', { threadId: 't1', status: { type: 'systemError' } }); + expect(signals.getSignal('t1').status).toBe('error'); + }); + + test('turn completion is an event, not a cost line to be spotted', () => { + const { client, signals } = source(); + client.emitNotification('turn/started', { threadId: 't1', turn: { id: 'turn1' } }); + expect(signals.getSignal('t1').status).toBe('busy'); + + client.emitNotification('turn/completed', { threadId: 't1', turn: { id: 'turn1', status: 'completed', durationMs: 4200 } }); + const signal = signals.getSignal('t1'); + expect(signal.status).toBe('idle'); + expect(signal.lastTurnStatus).toBe('completed'); + expect(signal.turnId).toBeNull(); + }); + + test('token usage and rate limits are captured — invisible to PTY scraping', () => { + const { client, signals } = source(); + client.emitNotification('thread/tokenUsage/updated', { threadId: 't1', turnId: 'x', tokenUsage: { input: 100, output: 20 } }); + client.emitNotification('account/rateLimits/updated', { threadId: 't1', primary: { usedPercent: 82 } }); + + const signal = signals.getSignal('t1'); + expect(signal.tokenUsage).toEqual({ input: 100, output: 20 }); + expect(signal.rateLimits.primary.usedPercent).toBe(82); + }); + + test('an approval request marks the thread waiting and is answerable over the wire', () => { + const { client, signals } = source(); + client.emitRequest(42, 'item/commandExecution/requestApproval', { threadId: 't1', command: 'npm test' }); + + expect(signals.getSignal('t1').awaitingApproval).toBe(true); + expect(signals.listPendingApprovals()[0].command).toBe('npm test'); + + const result = signals.answerApproval(42, true); + expect(result.ok).toBe(true); + expect(client.responses[0].result.decision).toBe('approved'); + expect(signals.listPendingApprovals()).toEqual([]); + }); + + test('answering an unknown approval is refused rather than silently dropped', () => { + const { signals } = source(); + expect(signals.answerApproval('nope', true).ok).toBe(false); + }); + + test('a closed thread stops producing signals', () => { + const { client, signals } = source(); + client.emitNotification('thread/status/changed', { threadId: 't1', status: { type: 'idle' } }); + client.emitNotification('thread/closed', { threadId: 't1' }); + expect(signals.getSignal('t1')).toBeNull(); + }); + + test('signals go stale when the app-server dies, so the PTY takes over again', () => { + const { client, signals } = source(); + client.emitNotification('thread/status/changed', { threadId: 't1', status: { type: 'idle' } }); + client.emit('exit', { code: 1 }); + expect(signals.getSignal('t1')).toBeNull(); + }); +}); + +describe('supervisor integration', () => { + test('a structured signal overrides the scraped guess but keeps the tail', () => { + const scraped = { sessionId: 's1', signalSource: 'pty', status: 'busy', quietSeconds: 400, tail: 'some output' }; + const merged = applyStructuredSignal(scraped, { + source: 'app-server', + threadId: 't1', + status: 'waiting', + activeFlags: ['waitingOnApproval'], + awaitingApproval: true, + quietSeconds: 12 + }); + + expect(merged.status).toBe('waiting'); + expect(merged.signalSource).toBe('app-server'); + expect(merged.quietSeconds).toBe(12); + expect(merged.tail).toBe('some output'); + }); + + test('no structured signal leaves the scraped one untouched', () => { + const scraped = { sessionId: 's1', signalSource: 'pty', status: 'busy', quietSeconds: 400 }; + expect(applyStructuredSignal(scraped, null)).toBe(scraped); + }); + + test('an unknown structured status does not clobber a known scraped one', () => { + const merged = applyStructuredSignal( + { status: 'busy', quietSeconds: 30 }, + { source: 'app-server', status: 'unknown', quietSeconds: 5 } + ); + expect(merged.status).toBe('busy'); + }); + + test('a reported approval fires the structured rule in seconds, not minutes', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const findings = evaluate([{ + sessionId: 'work1-codex', + type: 'codex', + status: 'waiting', + signalSource: 'app-server', + awaitingApproval: true, + agentPresent: true, + quietSeconds: 15, + tail: '', + repeatedLineCount: 0, + git: null, + tier: 2 + }], rules); + + expect(findings[0].conditionId).toBe('structured-approval'); + expect(findings[0].signalSource).toBe('app-server'); + }); + + test('a reported system error is delegated, not nudged', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const condition = rules.conditions.find((c) => c.id === 'thread-system-error'); + expect(condition.resolve.handler).toBe('delegate-to-commander'); + expect(condition.severity).toBe('critical'); + }); + + test('PTY sessions never match the structured-only rules', () => { + const rules = loadRules({ rulesPath: DEFAULT_RULES_PATH }); + const findings = evaluate([{ + sessionId: 'work1-claude', + type: 'claude', + status: 'busy', + signalSource: 'pty', + agentPresent: true, + quietSeconds: 15, + tail: 'working', + repeatedLineCount: 0, + git: null, + tier: 2 + }], rules); + + expect(findings.map((f) => f.conditionId)).not.toContain('structured-approval'); + }); +}); From e394b71aac2fb24fe2275440a9a33ce16ea13a5b Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:45:01 +1000 Subject: [PATCH 11/69] feat(atlas): write-back so the map improves instead of rotting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An atlas nobody updates becomes another stale doc. But agents cannot be given write access either — left to write freely, every repo an agent touched would end up rated 5/5 and the quality scores would stop meaning anything, which is the entire value. So: agents propose, you decide. A proposal carries the evidence for its claim ('40 tests added in tests/unit, all green'), which is what makes reviewing one take two seconds instead of requiring you to go and look. Approving writes through the exact same path manual curation uses, so an approved proposal is indistinguishable from a note you wrote yourself and syncs identically. Guards that matter: quality clamps to 1-5 so an over-eager agent cannot invent a 9; topic aliases normalize so proposals do not fragment the vocabulary; a second proposal for the same topic supersedes the first rather than stacking. The skill now tells agents to propose at the end of substantial work, and tells them plainly that they cannot write directly and should not try. 815 unit tests green. --- scripts/atlas.js | 57 ++++++++++ server/atlas/atlasProposals.js | 149 ++++++++++++++++++++++++++ server/repoAtlasService.js | 34 ++++++ server/routes/atlasRoutes.js | 29 +++++ skills/public/repo-atlas/SKILL.md | 17 ++- tests/unit/repoAtlasProposals.test.js | 113 +++++++++++++++++++ 6 files changed, 396 insertions(+), 3 deletions(-) create mode 100644 server/atlas/atlasProposals.js create mode 100644 tests/unit/repoAtlasProposals.test.js diff --git a/scripts/atlas.js b/scripts/atlas.js index f010c99a..463f6616 100755 --- a/scripts/atlas.js +++ b/scripts/atlas.js @@ -261,6 +261,59 @@ const commands = { return fail(`unknown audience action "${action}"`); }, + propose(positionals, flags) { + const id = positionals[0]; + if (!id || !flags.topic) { + return fail('usage: atlas propose --topic [--quality 1-5] [--paths a,b] [--notes "..."] [--evidence "why"] [--avoid]'); + } + const proposal = atlas.proposeHighlight({ + repoId: id, + topic: flags.topic, + kind: flags.avoid === true ? 'avoid' : 'highlight', + quality: flags.quality === undefined ? null : Number(flags.quality), + paths: listFlag(flags.paths), + notes: flags.notes === true ? '' : String(flags.notes || ''), + evidence: flags.evidence === true ? '' : String(flags.evidence || ''), + proposedBy: flags.by === true ? 'agent' : String(flags.by || 'agent') + }); + out(`Proposed ${proposal.kind} ${proposal.repoId} ${proposal.topic}${proposal.quality ? `:${proposal.quality}` : ''} — waiting for review.`); + return undefined; + }, + + proposals(positionals, flags) { + const action = positionals[0] || 'list'; + + if (action === 'list') { + const list = atlas.listProposals({ status: flags.status === true ? 'pending' : (flags.status || 'pending') }); + if (flags.json) return printJson(list); + if (!list.length) return out('No proposals waiting.'); + for (const p of list) { + out(`${p.id}`); + out(` ${p.kind} ${p.quality ? `${p.quality}/5` : ''} by ${p.proposedBy} ${p.proposedAt.slice(0, 16).replace('T', ' ')}`); + if (p.notes) out(` ${p.notes}`); + if (p.evidence) out(` evidence: ${p.evidence}`); + if (p.paths?.length) out(` paths: ${p.paths.join(', ')}`); + } + out(''); + out(`${list.length} waiting — \`atlas proposals approve \` or \`reject \``); + return undefined; + } + + if (action === 'approve' || action === 'reject') { + const id = positionals[1]; + if (!id) return fail(`usage: atlas proposals ${action} [--note "..."]`); + const note = flags.note === true ? '' : String(flags.note || ''); + const result = action === 'approve' ? atlas.approveProposal(id, { note }) : atlas.rejectProposal(id, { note }); + if (!result.ok) return fail(result.error); + return out(action === 'approve' ? `Approved and written to the registry: ${id}` : `Rejected: ${id}`); + } + + if (action === 'clear') { + return out(`Cleared decided proposals; ${atlas.clearDecidedProposals()} still waiting.`); + } + return fail(`unknown proposals action "${action}"`); + }, + async remote(positionals, flags) { const action = positionals[0]; if (action === 'set') { @@ -389,6 +442,10 @@ const commands = { atlas audience list | add [--label "..."] [--out ] [--out-remote ] atlas compile [--dry-run] [--explain] +write-back (agents propose, you decide): + atlas propose --topic X [--quality N] [--notes "..."] [--evidence "why"] [--avoid] + atlas proposals [list|approve |reject |clear] [--status all] + multi-machine: atlas remote set track the registry in a PRIVATE git repo atlas sync [--message "..."] pull + merge + push your curated entries diff --git a/server/atlas/atlasProposals.js b/server/atlas/atlasProposals.js new file mode 100644 index 00000000..e89ce3af --- /dev/null +++ b/server/atlas/atlasProposals.js @@ -0,0 +1,149 @@ +const fs = require('fs'); +const path = require('path'); + +const store = require('./atlasStore'); +const { kebab, normalizeTopic } = require('./atlasSchema'); + +const PROPOSALS_FILENAME = 'proposals.json'; +const MAX_PROPOSALS = 300; + +function proposalsPath() { + return path.join(store.atlasDir(), PROPOSALS_FILENAME); +} + +function load() { + try { + const raw = JSON.parse(fs.readFileSync(proposalsPath(), 'utf8')); + return Array.isArray(raw?.proposals) ? raw.proposals : []; + } catch { + return []; + } +} + +function save(proposals) { + fs.mkdirSync(path.dirname(proposalsPath()), { recursive: true }); + fs.writeFileSync(proposalsPath(), `${JSON.stringify({ proposals }, null, 2)}\n`, 'utf8'); + return proposals; +} + +function proposalId(repoId, topic) { + return `${kebab(repoId)}:${normalizeTopic(topic)}`; +} + +/** + * Agents propose; you decide. + * + * The map is only worth trusting if its quality scores mean something, so an + * agent that just finished work cannot write to it directly — it queues a + * proposal with the evidence for the claim. Left to write freely, every repo + * an agent touched would end up rated 5/5 and the whole thing would be noise. + */ +function propose({ repoId, topic, quality = null, paths = [], notes = '', evidence = '', proposedBy = 'agent', kind = 'highlight' } = {}) { + const id = kebab(repoId); + const normalizedTopic = normalizeTopic(topic); + if (!id) throw new Error('A proposal needs a repo id'); + if (!normalizedTopic) throw new Error('A proposal needs a topic'); + if (kind !== 'highlight' && kind !== 'avoid') throw new Error(`Unknown proposal kind "${kind}"`); + + const proposals = load(); + const key = proposalId(id, normalizedTopic); + const existing = proposals.find((p) => p.id === key && p.status === 'pending'); + + const entry = { + id: key, + repoId: id, + topic: normalizedTopic, + kind, + quality: quality === null || quality === undefined ? null : Math.min(5, Math.max(1, Math.round(Number(quality)))), + paths: Array.isArray(paths) ? paths.filter(Boolean) : String(paths || '').split(',').map((p) => p.trim()).filter(Boolean), + notes: String(notes || '').trim(), + // Why the agent believes this — the thing that makes a proposal reviewable + // in a couple of seconds instead of requiring you to go and look. + evidence: String(evidence || '').trim(), + proposedBy: String(proposedBy || 'agent'), + status: 'pending', + proposedAt: new Date().toISOString(), + supersedes: existing ? existing.proposedAt : null + }; + + const remaining = proposals.filter((p) => !(p.id === key && p.status === 'pending')); + remaining.unshift(entry); + if (remaining.length > MAX_PROPOSALS) remaining.length = MAX_PROPOSALS; + save(remaining); + + return entry; +} + +function list({ status = 'pending', repoId = '' } = {}) { + const wantStatus = String(status || '').trim(); + const wantRepo = kebab(repoId); + return load() + .filter((p) => (!wantStatus || wantStatus === 'all' || p.status === wantStatus)) + .filter((p) => (!wantRepo || p.repoId === wantRepo)); +} + +function decide(id, decision, { note = '' } = {}) { + const proposals = load(); + const entry = proposals.find((p) => p.id === id && p.status === 'pending'); + if (!entry) return { ok: false, error: `no pending proposal "${id}"` }; + + entry.status = decision; + entry.decidedAt = new Date().toISOString(); + if (note) entry.decisionNote = note; + save(proposals); + + return { ok: true, proposal: entry }; +} + +/** + * Approving is what actually writes to the registry, via the same code path + * manual curation uses — so an approved proposal is indistinguishable from a + * note you wrote yourself, and syncs the same way. + */ +function approve(id, atlas, { note = '' } = {}) { + const result = decide(id, 'approved', { note }); + if (!result.ok) return result; + + const { proposal } = result; + const applied = proposal.kind === 'avoid' + ? atlas.addAvoid(proposal.repoId, { topic: proposal.topic, reason: proposal.notes }) + : atlas.addHighlight(proposal.repoId, { + topic: proposal.topic, + quality: proposal.quality, + paths: proposal.paths, + notes: proposal.notes + }); + + return { ok: true, proposal, entry: applied }; +} + +function reject(id, { note = '' } = {}) { + return decide(id, 'rejected', { note }); +} + +function clearDecided() { + const remaining = load().filter((p) => p.status === 'pending'); + save(remaining); + return remaining.length; +} + +function getStats() { + const all = load(); + return { + pending: all.filter((p) => p.status === 'pending').length, + approved: all.filter((p) => p.status === 'approved').length, + rejected: all.filter((p) => p.status === 'rejected').length, + path: proposalsPath() + }; +} + +module.exports = { + PROPOSALS_FILENAME, + proposalsPath, + propose, + list, + approve, + reject, + clearDecided, + getStats +}; diff --git a/server/repoAtlasService.js b/server/repoAtlasService.js index 2df62a1e..843863e4 100644 --- a/server/repoAtlasService.js +++ b/server/repoAtlasService.js @@ -7,6 +7,7 @@ const discovery = require('./atlas/atlasDiscovery'); const query = require('./atlas/atlasQuery'); const compiler = require('./atlas/atlasCompiler'); const sync = require('./atlas/atlasSync'); +const proposals = require('./atlas/atlasProposals'); const { getProjectsRoot, getLegacyProjectsRoot } = require('./utils/pathUtils'); const ATLAS_CACHE_MS = 60_000; @@ -305,6 +306,37 @@ class RepoAtlasService { return result; } + /** + * Write-back: an agent that just finished work proposes what it learned, and + * the proposal waits for you. This is what keeps the map current instead of + * letting it rot into another stale doc. + */ + proposeHighlight(input) { + return proposals.propose(input); + } + + listProposals(filters = {}) { + return proposals.list(filters); + } + + approveProposal(id, options = {}) { + const result = proposals.approve(id, this, options); + this.invalidate(); + return result; + } + + rejectProposal(id, options = {}) { + return proposals.reject(id, options); + } + + clearDecidedProposals() { + return proposals.clearDecided(); + } + + getProposalStats() { + return proposals.getStats(); + } + listSubscriptions() { return store.loadSubscriptions().map((bundle) => ({ name: bundle.name, @@ -368,6 +400,7 @@ class RepoAtlasService { highlightCount: entries.reduce((sum, e) => sum + (e.highlights || []).length, 0), audiences: this.listAudiences().map((a) => a.id), subscriptions: this.listSubscriptions(), + proposals: proposals.getStats(), remote: store.loadConfig().remote || null, discovery: meta ? { generatedAt: meta.generatedAt, stale: meta.stale, githubAvailable: meta.githubAvailable !== false } @@ -383,4 +416,5 @@ module.exports.store = store; module.exports.discovery = discovery; module.exports.query = query; module.exports.compiler = compiler; +module.exports.proposals = proposals; module.exports.defaultScanRoots = defaultScanRoots; diff --git a/server/routes/atlasRoutes.js b/server/routes/atlasRoutes.js index 53586fec..cbbbf520 100644 --- a/server/routes/atlasRoutes.js +++ b/server/routes/atlasRoutes.js @@ -127,6 +127,35 @@ function createAtlasRoutes({ repoAtlasService, logger = console, requireRead = p res.json({ ok: true, audiences }); })); + /** + * Write-back. Agents propose; the human decides. Proposing is a `read`-level + * action because it changes nothing — approving is what writes. + */ + router.get('/proposals', requireRead, handle('list proposals', (req, res) => { + const list = repoAtlasService.listProposals({ status: req.query.status || 'pending', repoId: req.query.repoId }); + res.json({ ok: true, count: list.length, proposals: list, stats: repoAtlasService.getProposalStats() }); + })); + + router.post('/proposals', requireRead, handle('propose', (req, res) => { + res.json({ ok: true, proposal: repoAtlasService.proposeHighlight(req.body || {}) }); + })); + + router.post('/proposals/:id/approve', requireWrite, handle('approve proposal', (req, res) => { + const result = repoAtlasService.approveProposal(req.params.id, { note: req.body?.note || '' }); + if (!result.ok) return res.status(404).json(result); + return res.json({ ok: true, ...result }); + })); + + router.post('/proposals/:id/reject', requireWrite, handle('reject proposal', (req, res) => { + const result = repoAtlasService.rejectProposal(req.params.id, { note: req.body?.note || '' }); + if (!result.ok) return res.status(404).json(result); + return res.json({ ok: true, ...result }); + })); + + router.delete('/proposals', requireWrite, handle('clear decided proposals', (req, res) => { + res.json({ ok: true, remaining: repoAtlasService.clearDecidedProposals() }); + })); + router.get('/sync', requireRead, handle('sync status', async (req, res) => { res.json({ ok: true, sync: await repoAtlasService.getSyncStatus() }); })); diff --git a/skills/public/repo-atlas/SKILL.md b/skills/public/repo-atlas/SKILL.md index 5def9db9..c1f1fda8 100644 --- a/skills/public/repo-atlas/SKILL.md +++ b/skills/public/repo-atlas/SKILL.md @@ -59,13 +59,24 @@ Never clone into the user's `~/GitHub` tree to "just take a look" — use `/tmp` ## Recording what you learn -When you finish work that produced something genuinely reusable — or discover that a repo's approach to something is excellent or awful — write it down. This is what keeps the map alive. +**Do this at the end of any substantial piece of work.** If you built something genuinely reusable, or discovered that a repo's approach to something is excellent or awful, propose it. This is the single thing that keeps the map alive instead of letting it rot. ```bash -atlas note --topic --quality 1-5 --paths a/b.ts,c/ --notes "why it is worth copying" -atlas avoid --topic --reason "why nobody should copy this" +atlas propose --topic --quality 1-5 \ + --paths src/a.ts,tests/ \ + --notes "why it is worth copying" \ + --evidence "what you actually saw that supports this" \ + --by "" + +atlas propose --topic --avoid --notes "why nobody should copy this" ``` +Proposals wait for the user to approve — **you cannot write to the map directly, and should not try.** That is deliberate: if agents wrote freely, every repo anyone touched would end up rated 5/5 and the quality scores would stop meaning anything. + +Always fill in `--evidence`. "40 tests added in tests/unit, all green" is reviewable in two seconds; "it's good" is not, and will be rejected. + +If the user is curating directly, `atlas note` / `atlas avoid` write immediately — those are for them, not for you. + Guidance on scores: **5** = copy this exactly; **4** = solid, adapt it; **3** = works, read for ideas; **2** = only if nothing better; **1** = cautionary example. Score the *topic*, not the repo — a prototype can be a 5 at one thing and a 2 at everything else. Use `atlas topics --vocabulary` for canonical topic names. Aliases fold automatically (`multiplayer` → `networking`, `tests` → `testing`), and unrecognized topics are kept rather than dropped. diff --git a/tests/unit/repoAtlasProposals.test.js b/tests/unit/repoAtlasProposals.test.js new file mode 100644 index 00000000..b6fd0e85 --- /dev/null +++ b/tests/unit/repoAtlasProposals.test.js @@ -0,0 +1,113 @@ +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const RepoAtlasService = require('../../server/repoAtlasService'); + +describe('Atlas write-back', () => { + let tmpDir; + let atlas; + + beforeEach(() => { + tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-proposals-')); + process.env.AGENT_WORKSPACE_ATLAS_DIR = tmpDir; + atlas = new RepoAtlasService(); + }); + + afterEach(() => { + delete process.env.AGENT_WORKSPACE_ATLAS_DIR; + fs.rmSync(tmpDir, { recursive: true, force: true }); + }); + + test('a proposal does not touch the map until it is approved', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5, proposedBy: 'work1-claude' }); + + expect(atlas.find('testing')).toEqual([]); + expect(atlas.listProposals()).toHaveLength(1); + }); + + test('approving writes through the same path manual curation uses', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5, notes: 'good harness' }); + const result = atlas.approveProposal('zoo-game:testing'); + + expect(result.ok).toBe(true); + const hits = atlas.find('testing'); + expect(hits).toHaveLength(1); + expect(hits[0].quality).toBe(5); + expect(hits[0].notes).toBe('good harness'); + }); + + test('rejecting leaves the map untouched and clears the queue', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5 }); + expect(atlas.rejectProposal('zoo-game:testing').ok).toBe(true); + + expect(atlas.find('testing')).toEqual([]); + expect(atlas.listProposals()).toEqual([]); + expect(atlas.listProposals({ status: 'rejected' })).toHaveLength(1); + }); + + test('a second proposal for the same topic supersedes the first rather than stacking', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 3 }); + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5 }); + + const pending = atlas.listProposals(); + expect(pending).toHaveLength(1); + expect(pending[0].quality).toBe(5); + expect(pending[0].supersedes).toBeTruthy(); + }); + + test('topic aliases are normalized so proposals do not fragment the vocabulary', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'unit-tests', quality: 4 }); + expect(atlas.listProposals()[0].topic).toBe('testing'); + }); + + test('an avoid proposal records a do-not-copy note when approved', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'ui', kind: 'avoid', notes: 'hand-rolled, superseded' }); + atlas.approveProposal('zoo-game:ui'); + + expect(atlas.getEntry('zoo-game').avoid).toEqual([{ topic: 'ui', reason: 'hand-rolled, superseded' }]); + }); + + test('quality is clamped, so an over-eager agent cannot invent a 9/5', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 9 }); + expect(atlas.listProposals()[0].quality).toBe(5); + }); + + test('proposals without a repo or topic are refused', () => { + expect(() => atlas.proposeHighlight({ topic: 'testing' })).toThrow(/repo id/); + expect(() => atlas.proposeHighlight({ repoId: 'zoo-game' })).toThrow(/topic/); + }); + + test('deciding an unknown proposal reports it instead of failing silently', () => { + expect(atlas.approveProposal('nope:testing').ok).toBe(false); + expect(atlas.rejectProposal('nope:testing').ok).toBe(false); + }); + + test('evidence travels with the proposal so review takes seconds', () => { + atlas.proposeHighlight({ + repoId: 'zoo-game', + topic: 'testing', + quality: 4, + evidence: 'added 40 tests in tests/unit, all green', + proposedBy: 'work1-claude' + }); + + const [proposal] = atlas.listProposals(); + expect(proposal.evidence).toBe('added 40 tests in tests/unit, all green'); + expect(proposal.proposedBy).toBe('work1-claude'); + }); + + test('clearing decided proposals keeps the pending ones', () => { + atlas.proposeHighlight({ repoId: 'a', topic: 'testing', quality: 4 }); + atlas.proposeHighlight({ repoId: 'b', topic: 'physics', quality: 4 }); + atlas.rejectProposal('a:testing'); + + expect(atlas.clearDecidedProposals()).toBe(1); + expect(atlas.listProposals({ status: 'all' })).toHaveLength(1); + }); + + test('proposal stats surface in atlas status', () => { + atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 4 }); + expect(atlas.getStatus().proposals.pending).toBe(1); + }); +}); From 0cd18a9477dc5eb49afb3fc28019ee93421749d9 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:48:16 +1000 Subject: [PATCH 12/69] =?UTF-8?q?feat(ui):=20JARVIS=20panel=20=E2=80=94=20?= =?UTF-8?q?Alt+J=20for=20the=20whole=20picture?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The APIs existed but nothing rendered them, which meant the system could only be read by curling it. Leads with 'handled' on purpose: that number rising while the waiting list stays empty is what the system working looks like. A dashboard that only ever shows problems trains you to read it as a problem list, which is the habit this whole design is trying to break. Four sections: what still needs you (the digest, with urgency and how many times it recurred), what the team asked for that nobody started, Atlas proposals with one-click approve/reject, and Atlas topic search. Browser-verified rather than eyeballed: headless Chrome under Xvfb confirms the assets load, the CSS applies, all four sections render, and clicking Approve writes through the API into the registry — 1 proposal in, 0 remaining after the click, entry present in the map. Zero console errors, zero failed requests. Styling follows the house rules: rem throughout, 100dvh, safe-area inset, 44px touch targets, one mobile breakpoint, and white/bright-accent text on dark with no grey-on-grey anywhere. 815 unit tests green. --- client/index.html | 2 + client/jarvis-panel.js | 303 +++++++++++++++++++++++++++++++++++++++ client/styles/jarvis.css | 218 ++++++++++++++++++++++++++++ 3 files changed, 523 insertions(+) create mode 100644 client/jarvis-panel.js create mode 100644 client/styles/jarvis.css diff --git a/client/index.html b/client/index.html index 572417ae..a50ef311 100644 --- a/client/index.html +++ b/client/index.html @@ -13,6 +13,7 @@ + @@ -549,6 +550,7 @@

Notifications

+ diff --git a/client/jarvis-panel.js b/client/jarvis-panel.js new file mode 100644 index 00000000..a8cbe96b --- /dev/null +++ b/client/jarvis-panel.js @@ -0,0 +1,303 @@ +/** + * JARVIS panel — what the supervisor handled, what is still waiting, the work + * the team asked for, and the map's pending proposals. + * + * Deliberately leads with "handled", because that number going up while the + * waiting list stays empty is the system working. A dashboard that only shows + * problems trains you to read it as a problem list. + */ +(function initJarvisPanel() { + const REFRESH_MS = 20_000; + + const api = async (path, options = {}) => { + const token = window.AUTH_TOKEN || localStorage.getItem('authToken') || ''; + const response = await fetch(path, { + ...options, + headers: { + 'Content-Type': 'application/json', + ...(token ? { 'X-Auth-Token': token } : {}), + ...(options.headers || {}) + } + }); + if (!response.ok) throw new Error(`${options.method || 'GET'} ${path} -> ${response.status}`); + return response.json(); + }; + + const el = (tag, className, text) => { + const node = document.createElement(tag); + if (className) node.className = className; + if (text !== undefined) node.textContent = text; + return node; + }; + + const relativeTime = (iso) => { + const ms = Date.parse(String(iso || '')); + if (!Number.isFinite(ms)) return ''; + const seconds = Math.round((Date.now() - ms) / 1000); + if (seconds < 60) return `${seconds}s ago`; + if (seconds < 3600) return `${Math.round(seconds / 60)}m ago`; + if (seconds < 86400) return `${Math.round(seconds / 3600)}h ago`; + return `${Math.round(seconds / 86400)}d ago`; + }; + + class JarvisPanel { + constructor() { + this.root = null; + this.timer = null; + this.open = false; + this.state = { supervisor: null, digest: [], discord: [], proposals: [], atlasHits: null }; + } + + mount() { + if (this.root) return this.root; + + this.root = el('div', 'jarvis-panel'); + this.root.innerHTML = ` +
+ JARVIS + +
+ + + +
+
+
+
+
+
+
+
+

Atlas

+
+ + +
+
+
+
+ `; + + this.root.addEventListener('click', (event) => this.onClick(event)); + this.root.querySelector('[data-role="atlas-input"]').addEventListener('keydown', (event) => { + if (event.key === 'Enter') this.findInAtlas(); + }); + + document.body.appendChild(this.root); + return this.root; + } + + async onClick(event) { + const action = event.target?.dataset?.action; + if (!action) return; + + if (action === 'close') return this.hide(); + if (action === 'refresh') return this.refresh(); + if (action === 'atlas-find') return this.findInAtlas(); + + if (action === 'catch-up') { + await api('/api/supervisor/digest/deliver', { method: 'POST', body: '{}' }); + return this.refresh(); + } + if (action === 'approve-proposal' || action === 'reject-proposal') { + const id = event.target.dataset.id; + const verb = action === 'approve-proposal' ? 'approve' : 'reject'; + await api(`/api/atlas/proposals/${encodeURIComponent(id)}/${verb}`, { method: 'POST', body: '{}' }); + return this.refresh(); + } + return undefined; + } + + async refresh() { + const settled = await Promise.allSettled([ + api('/api/supervisor/briefing'), + api('/api/discord-watch/untracked'), + api('/api/atlas/proposals') + ]); + + const [briefing, discord, proposals] = settled.map((r) => (r.status === 'fulfilled' ? r.value : null)); + this.state.supervisor = briefing?.briefing || null; + this.state.digest = briefing?.briefing?.waiting || []; + this.state.discord = discord?.items || []; + this.state.proposals = proposals?.proposals || []; + this.render(); + } + + async findInAtlas() { + const input = this.root.querySelector('[data-role="atlas-input"]'); + const topic = String(input.value || '').trim(); + if (!topic) return; + + try { + const result = await api(`/api/atlas/find?topic=${encodeURIComponent(topic)}`); + this.state.atlasHits = { topic, hits: result.hits || [] }; + } catch { + this.state.atlasHits = { topic, hits: [], error: true }; + } + this.renderAtlas(); + } + + renderStats() { + const target = this.root.querySelector('[data-role="stats"]'); + const supervisor = this.state.supervisor; + target.innerHTML = ''; + + if (!supervisor) { + target.appendChild(el('div', 'jarvis-stat-empty', 'Supervisor unreachable')); + return; + } + + const stats = supervisor.stats || {}; + const cells = [ + ['handled', (stats.resolved || 0) + (stats.delegated || 0)], + ['delegated', stats.delegated || 0], + ['waiting', this.state.digest.length], + ['interrupts', `${supervisor.budget?.usedThisHour ?? 0}/${supervisor.budget?.maxPerHour ?? 0}`] + ]; + + for (const [label, value] of cells) { + const cell = el('div', 'jarvis-stat'); + cell.appendChild(el('span', 'jarvis-stat-value', String(value))); + cell.appendChild(el('span', 'jarvis-stat-label', label)); + target.appendChild(cell); + } + + this.root.querySelector('[data-role="autonomy"]').textContent = supervisor.autonomy || ''; + } + + renderWaiting() { + const target = this.root.querySelector('[data-role="waiting"]'); + target.innerHTML = '

Needs you

'; + + if (!this.state.digest.length) { + target.appendChild(el('p', 'jarvis-empty', 'Nothing. Everything else was handled.')); + return; + } + + for (const item of this.state.digest) { + const row = el('div', `jarvis-item jarvis-sev-${item.severity}`); + row.appendChild(el('span', 'jarvis-item-where', item.where)); + row.appendChild(el('span', 'jarvis-item-label', item.label)); + if (item.advice) row.appendChild(el('span', 'jarvis-item-advice', item.advice)); + row.appendChild(el('span', 'jarvis-item-meta', `urgency ${item.score} · seen ${item.count}× · ${relativeTime(item.lastSeenAt)}`)); + target.appendChild(row); + } + } + + renderDiscord() { + const target = this.root.querySelector('[data-role="discord"]'); + target.innerHTML = '

Asked for, not started

'; + + if (!this.state.discord.length) { + target.appendChild(el('p', 'jarvis-empty', 'Nothing outstanding from chat.')); + return; + } + + for (const item of this.state.discord) { + const row = el('div', `jarvis-item jarvis-tier-${item.tier}`); + row.appendChild(el('span', 'jarvis-item-where', `T${item.tier} ${item.priority}`)); + row.appendChild(el('span', 'jarvis-item-label', item.summary)); + row.appendChild(el('span', 'jarvis-item-meta', `${item.assigneeNames?.join(', ') || 'unassigned'} · ${relativeTime(item.createdAt)}`)); + if (item.permalink) { + const link = el('a', 'jarvis-item-link', 'open in Discord'); + link.href = item.permalink; + link.target = '_blank'; + link.rel = 'noopener noreferrer'; + row.appendChild(link); + } + target.appendChild(row); + } + } + + renderProposals() { + const target = this.root.querySelector('[data-role="proposals"]'); + target.innerHTML = '

Atlas proposals

'; + + if (!this.state.proposals.length) { + target.appendChild(el('p', 'jarvis-empty', 'No proposals waiting.')); + return; + } + + for (const proposal of this.state.proposals) { + const row = el('div', 'jarvis-item'); + row.appendChild(el('span', 'jarvis-item-where', `${proposal.repoId} · ${proposal.topic}${proposal.quality ? ` ${proposal.quality}/5` : ''}`)); + if (proposal.notes) row.appendChild(el('span', 'jarvis-item-label', proposal.notes)); + if (proposal.evidence) row.appendChild(el('span', 'jarvis-item-advice', `evidence: ${proposal.evidence}`)); + row.appendChild(el('span', 'jarvis-item-meta', `by ${proposal.proposedBy} · ${relativeTime(proposal.proposedAt)}`)); + + const actions = el('div', 'jarvis-item-actions'); + const approve = el('button', 'jarvis-approve', 'Approve'); + approve.dataset.action = 'approve-proposal'; + approve.dataset.id = proposal.id; + const reject = el('button', 'jarvis-reject', 'Reject'); + reject.dataset.action = 'reject-proposal'; + reject.dataset.id = proposal.id; + actions.append(approve, reject); + row.appendChild(actions); + + target.appendChild(row); + } + } + + renderAtlas() { + const target = this.root.querySelector('[data-role="atlas-results"]'); + target.innerHTML = ''; + const found = this.state.atlasHits; + if (!found) return; + + if (!found.hits.length) { + target.appendChild(el('p', 'jarvis-empty', `Nothing recorded for "${found.topic}".`)); + return; + } + + for (const hit of found.hits) { + const row = el('div', 'jarvis-item'); + row.appendChild(el('span', 'jarvis-item-where', `${hit.quality ?? '?'}/5 ${hit.id}${hit.stale ? ' ⚠old' : ''}`)); + if (hit.notes) row.appendChild(el('span', 'jarvis-item-label', hit.notes)); + row.appendChild(el('span', 'jarvis-item-meta', hit.cloned ? hit.localPath : (hit.remoteUrl || 'not cloned'))); + target.appendChild(row); + } + } + + render() { + if (!this.root) return; + this.renderStats(); + this.renderWaiting(); + this.renderDiscord(); + this.renderProposals(); + this.renderAtlas(); + } + + show() { + this.mount(); + this.root.classList.add('jarvis-open'); + this.open = true; + this.refresh(); + if (!this.timer) this.timer = setInterval(() => this.open && this.refresh(), REFRESH_MS); + } + + hide() { + this.open = false; + this.root?.classList.remove('jarvis-open'); + if (this.timer) { + clearInterval(this.timer); + this.timer = null; + } + } + + toggle() { + return this.open ? this.hide() : this.show(); + } + } + + const panel = new JarvisPanel(); + window.JarvisPanel = panel; + + // Alt+J — the fleet summary should be one keystroke away, not buried. + document.addEventListener('keydown', (event) => { + if (event.altKey && (event.key === 'j' || event.key === 'J')) { + event.preventDefault(); + panel.toggle(); + } + }); +})(); diff --git a/client/styles/jarvis.css b/client/styles/jarvis.css new file mode 100644 index 00000000..32e6239e --- /dev/null +++ b/client/styles/jarvis.css @@ -0,0 +1,218 @@ +/* + * JARVIS panel. + * + * Dark surface, white and bright accent text only — no grey-on-grey. Sizes are + * in rem so the panel scales with the user's root font size instead of pinning + * itself to one display. + */ + +.jarvis-panel { + position: fixed; + top: 0; + right: 0; + width: min(100%, 30rem); + height: 100dvh; + padding-right: env(safe-area-inset-right, 0); + background: #12161d; + border-left: 2px solid #2a3340; + box-shadow: -0.5rem 0 2rem rgba(0, 0, 0, 0.45); + transform: translateX(100%); + transition: transform 0.18s ease-out; + display: flex; + flex-direction: column; + z-index: 9000; + font-size: 0.875rem; + color: #ffffff; +} + +.jarvis-panel.jarvis-open { + transform: translateX(0); +} + +.jarvis-header { + display: flex; + align-items: center; + gap: 0.75rem; + padding: 1rem 1.25rem; + border-bottom: 1px solid #2a3340; + flex-shrink: 0; +} + +.jarvis-title { + font-weight: 700; + letter-spacing: 0.08em; + color: #a8d8f0; +} + +.jarvis-autonomy { + font-size: 0.75rem; + text-transform: uppercase; + letter-spacing: 0.06em; + color: #ffd685; + border: 1px solid #4a3f22; + border-radius: 0.25rem; + padding: 0.125rem 0.5rem; +} + +.jarvis-header-actions { + margin-left: auto; + display: flex; + gap: 0.375rem; +} + +.jarvis-panel button { + background: #22303f; + color: #ffffff; + border: 1px solid #35485c; + border-radius: 0.25rem; + padding: 0.375rem 0.625rem; + font-size: 0.8125rem; + font-weight: 600; + cursor: pointer; + min-height: 2.75rem; + min-width: 2.75rem; +} + +.jarvis-panel button:hover { + background: #2d3f52; +} + +.jarvis-stats { + display: grid; + grid-template-columns: repeat(4, 1fr); + gap: 0.5rem; + padding: 1rem 1.25rem; + border-bottom: 1px solid #2a3340; + flex-shrink: 0; +} + +.jarvis-stat { + display: flex; + flex-direction: column; + align-items: center; +} + +.jarvis-stat-value { + font-size: 1.5rem; + font-weight: 700; + color: #7ee787; +} + +.jarvis-stat-label { + font-size: 0.6875rem; + text-transform: uppercase; + letter-spacing: 0.05em; + color: #a8d8f0; +} + +.jarvis-stat-empty { + grid-column: 1 / -1; + color: #ff9a9a; + text-align: center; +} + +.jarvis-sections { + flex: 1; + overflow-y: auto; + padding: 0 1.25rem 2rem; +} + +.jarvis-sections h4 { + margin: 1.25rem 0 0.5rem; + font-size: 0.75rem; + text-transform: uppercase; + letter-spacing: 0.08em; + color: #a8d8f0; +} + +.jarvis-empty { + margin: 0; + color: #7ee787; + font-size: 0.8125rem; +} + +.jarvis-item { + display: flex; + flex-direction: column; + gap: 0.1875rem; + padding: 0.625rem 0.75rem; + margin-bottom: 0.5rem; + background: #1a212b; + border-left: 3px solid #35485c; + border-radius: 0.25rem; +} + +.jarvis-sev-critical { border-left-color: #ff6b6b; } +.jarvis-sev-warn { border-left-color: #ffd685; } +.jarvis-sev-info { border-left-color: #a8d8f0; } +.jarvis-tier-1 { border-left-color: #ff6b6b; } +.jarvis-tier-2 { border-left-color: #ffd685; } + +.jarvis-item-where { + font-weight: 700; + color: #ffd685; + font-size: 0.8125rem; +} + +.jarvis-item-label { + color: #ffffff; +} + +.jarvis-item-advice { + color: #a8d8f0; + font-size: 0.8125rem; +} + +.jarvis-item-meta { + font-size: 0.75rem; + color: #8fb8d0; +} + +.jarvis-item-link { + font-size: 0.75rem; + color: #a8d8f0; + text-decoration: underline; +} + +.jarvis-item-actions { + display: flex; + gap: 0.375rem; + margin-top: 0.375rem; +} + +.jarvis-approve { background: #1f5132; border-color: #2f7a4c; } +.jarvis-approve:hover { background: #2a6b43; } +.jarvis-reject { background: #5a2020; border-color: #8a3030; } +.jarvis-reject:hover { background: #742a2a; } + +.jarvis-atlas-search { + display: flex; + gap: 0.375rem; +} + +.jarvis-atlas-search input { + flex: 1; + min-width: 0; + background: #0d1117; + color: #ffffff; + border: 1px solid #35485c; + border-radius: 0.25rem; + padding: 0.5rem 0.625rem; + font-size: 0.8125rem; + min-height: 2.75rem; +} + +.jarvis-atlas-search input::placeholder { + color: #7fa3bd; +} + +@media (max-width: 640px) { + .jarvis-panel { + width: 100%; + border-left: none; + } + + .jarvis-stats { + grid-template-columns: repeat(2, 1fr); + } +} From 61fa05eac29f00c3948730b1d095b7593e15e98f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:50:04 +1000 Subject: [PATCH 13/69] feat(voice): full-duplex realtime loop against Codex threads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the loop OpenAI shipped to their desktop app on 2026-07-23, without being locked to their app or to macOS. Browser speech recognition transcribes you, the text goes to thread/realtime/appendText, and the assistant's transcript deltas arrive over the socket and are spoken by SpeechOutput. Hands-free, nothing to install. Two details that matter: only *completed* assistant turns are spoken, because speaking every delta stutters and speaking your own words back is absurd (browser-verified — 'Build is green.' spoken, the partial delta ignored); and continuous recognition restarts itself, because browsers end the stream periodically whether or not you are done talking. Honest about maturity: the raw-audio path to appendAudio is wired but the exact PCM framing has not been confirmed against a live authenticated realtime session, so it is opt-in and the text path is the default rather than pretending otherwise. The text path is a complete working voice loop today. 815 unit tests green; browser-verified under Xvfb with zero console errors. --- client/index.html | 1 + client/realtime-voice.js | 210 +++++++++++++++++++++++++++++++++++++++ 2 files changed, 211 insertions(+) create mode 100644 client/realtime-voice.js diff --git a/client/index.html b/client/index.html index a50ef311..3b776ab9 100644 --- a/client/index.html +++ b/client/index.html @@ -550,6 +550,7 @@

Notifications

+ diff --git a/client/realtime-voice.js b/client/realtime-voice.js new file mode 100644 index 00000000..040f8bae --- /dev/null +++ b/client/realtime-voice.js @@ -0,0 +1,210 @@ +/** + * Realtime voice loop against a Codex thread. + * + * Two paths exist, and they have different maturity: + * + * TEXT (default, works today) — browser speech recognition transcribes you, + * the text goes to `thread/realtime/appendText`, and the assistant's + * transcript deltas come back over the socket and are spoken by SpeechOutput. + * A complete hands-free loop with nothing to install. + * + * AUDIO (wired, unverified) — raw capture straight to `appendAudio`. The + * protocol accepts it, but the exact PCM framing has not been confirmed + * against a live authenticated realtime session, so it is opt-in and the text + * path stays the default rather than pretending otherwise. + */ +(function initRealtimeVoice() { + const api = async (path, options = {}) => { + const token = window.AUTH_TOKEN || localStorage.getItem('authToken') || ''; + const response = await fetch(path, { + ...options, + headers: { + 'Content-Type': 'application/json', + ...(token ? { 'X-Auth-Token': token } : {}), + ...(options.headers || {}) + } + }); + const data = await response.json().catch(() => ({})); + if (!response.ok || data.ok === false) throw new Error(data.error || `${path} -> ${response.status}`); + return data; + }; + + class RealtimeVoice { + constructor() { + this.threadId = null; + this.active = false; + this.recognition = null; + this.listening = false; + this.transcript = []; + this.listeners = new Set(); + this.speakReplies = true; + } + + onUpdate(listener) { + this.listeners.add(listener); + return () => this.listeners.delete(listener); + } + + emit() { + for (const listener of this.listeners) { + try { + listener(this.getState()); + } catch { + // A broken listener must not stop the conversation. + } + } + } + + attach(socket) { + if (!socket || socket.__realtimeVoiceAttached) return; + socket.__realtimeVoiceAttached = true; + + socket.on('app-server-transcript', (entry) => { + this.transcript.unshift(entry); + if (this.transcript.length > 100) this.transcript.length = 100; + + // Only speak completed assistant turns — speaking every delta would + // stutter, and speaking your own words back is absurd. + if (this.speakReplies && entry.done && entry.role !== 'user' && entry.text) { + window.SpeechOutput?.speak(entry.text, { priority: 'normal' }); + } + this.emit(); + }); + + socket.on('app-server-realtime', ({ event }) => { + if (event === 'thread/realtime/started') this.active = true; + if (event === 'thread/realtime/closed' || event === 'thread/realtime/error') { + this.active = false; + this.stopListening(); + } + this.emit(); + }); + } + + async start(threadId, { voice = '', transport = 'websocket' } = {}) { + if (!threadId) throw new Error('A realtime session needs a thread id'); + await api(`/api/app-server/realtime/${encodeURIComponent(threadId)}/start`, { + method: 'POST', + body: JSON.stringify({ voice, transport }) + }); + this.threadId = threadId; + this.active = true; + this.emit(); + return { threadId, active: true }; + } + + async stop() { + this.stopListening(); + if (this.threadId) { + await api(`/api/app-server/realtime/${encodeURIComponent(this.threadId)}/stop`, { method: 'POST', body: '{}' }) + .catch(() => {}); + } + this.active = false; + this.emit(); + return { active: false }; + } + + async say(text) { + const clean = String(text || '').trim(); + if (!clean) return null; + if (!this.threadId) throw new Error('No realtime session is open'); + + this.transcript.unshift({ role: 'user', text: clean, done: true, at: new Date().toISOString(), local: true }); + this.emit(); + + return api(`/api/app-server/realtime/${encodeURIComponent(this.threadId)}/text`, { + method: 'POST', + body: JSON.stringify({ text: clean }) + }); + } + + /** + * Continuous listening. Interim results are ignored — only a finalized + * phrase is worth sending, otherwise the agent receives half-sentences. + */ + startListening() { + const SpeechRecognition = window.SpeechRecognition || window.webkitSpeechRecognition; + if (!SpeechRecognition) throw new Error('This browser has no speech recognition'); + if (this.listening) return true; + + this.recognition = new SpeechRecognition(); + this.recognition.continuous = true; + this.recognition.interimResults = false; + this.recognition.lang = 'en-US'; + + this.recognition.onresult = (event) => { + for (let i = event.resultIndex; i < event.results.length; i += 1) { + const result = event.results[i]; + if (!result.isFinal) continue; + const text = String(result[0]?.transcript || '').trim(); + if (text) this.say(text).catch(() => {}); + } + }; + + // Continuous recognition stops itself periodically; restart while wanted. + this.recognition.onend = () => { + if (this.listening) { + try { + this.recognition.start(); + } catch { + this.listening = false; + this.emit(); + } + } + }; + + this.recognition.onerror = () => {}; + + this.recognition.start(); + this.listening = true; + this.emit(); + return true; + } + + stopListening() { + this.listening = false; + try { + this.recognition?.stop(); + } catch { + // Already stopped. + } + this.recognition = null; + this.emit(); + } + + toggleListening() { + return this.listening ? this.stopListening() : this.startListening(); + } + + setSpeakReplies(enabled) { + this.speakReplies = enabled !== false; + return this.speakReplies; + } + + async listVoices() { + const result = await api('/api/app-server/realtime/voices'); + return result.voices || []; + } + + async listThreads() { + const result = await api('/api/app-server/threads'); + return result.threads || []; + } + + getState() { + return { + threadId: this.threadId, + active: this.active, + listening: this.listening, + speakReplies: this.speakReplies, + transcript: this.transcript.slice(0, 30) + }; + } + } + + const realtime = new RealtimeVoice(); + window.RealtimeVoice = realtime; + + if (window.socket) realtime.attach(window.socket); + document.addEventListener('orchestrator-socket-ready', (event) => realtime.attach(event.detail?.socket || window.socket)); +})(); From 6847a0648a84e1a5828684149d0e2b69204b5f73 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:50:35 +1000 Subject: [PATCH 14/69] docs: app-server bridge, realtime voice, atlas write-back, JARVIS panel --- CODEBASE_DOCUMENTATION.md | 30 ++++++++++++++++++++++++++++++ docs/COMMANDER_CLAUDE.md | 35 +++++++++++++++++++++++++++++++++++ 2 files changed, 65 insertions(+) diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index a79990ac..4e21d810 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -155,6 +155,20 @@ client/speech-output.js - Web Speech API listener for the browser bac server/voiceCommandService.js - (existing) rule/LLM voice parsing, now with `setCommanderForwarder()`: unmatched speech is handed to the Commander agent instead of dead-ending tests/unit/speechService.test.js - Sanitization, repeat suppression, backend resolution +server/appServerService.js - Codex app-server bridge: structured signals + realtime voice +├─ Protocol: JSON-RPC 2.0 over stdio to `codex app-server` (Apache 2.0, ships in the CLI); `initialize` handshake required first +├─ Signals: thread/status/changed, turn/started|completed, tokenUsage, rateLimits, and approval requests replace regex-scraped PTY output +├─ Realtime: thread/realtime/* full-duplex voice (websocket + webrtc transports), transcripts relayed over Socket.IO +└─ Degrades: `getSignalForSession` returning null means the PTY scraper stays authoritative; opt in with CODEX_APP_SERVER=true +server/agents/appServerClient.js - JSON-RPC client (newline-delimited framing, request/response correlation, restart backoff) +server/agents/appServerSignals.js - Notification -> supervisor-signal mapping; ThreadActiveFlag waitingOnApproval, over-the-wire approvals +server/routes/appServerRoutes.js - `/api/app-server/*` (threads, turns, approvals, realtime voice, transcripts) +server/atlas/atlasProposals.js - Write-back queue: agents propose highlights with evidence, the human approves +client/jarvis-panel.js - Alt+J panel: what was handled, what needs you, untracked chat work, atlas proposals + search +client/realtime-voice.js - Browser side of the realtime loop (`window.RealtimeVoice`) +client/styles/jarvis.css - JARVIS panel styling +tests/unit/appServerService.test.js, repoAtlasProposals.test.js - App-server framing/signal mapping, write-back approval flow + server/supervisor/supervisorUrgency.js - Urgency scoring (severity x task tier + failed-repair weight), interruption budget, digest queue server/discordWatchService.js - Ambient Discord watching: read the conversation, track the work, publish status back ├─ Ingest: cursor-based polling (`?after=`), so a message missed while the process was down is not possible rather than retried @@ -525,6 +539,9 @@ notification: {type, message, level} - System notifications workspace-changed: {workspaceId, sessions} - Workspace switch completed workspace-list: {workspaces} - Available workspaces update speech-speak: {text, priority, at} - Say this out loud (browser speech backend) +app-server-approval: {requestId, threadId, command} - A Codex thread is blocked on an approval +app-server-realtime: {event, payload} - Realtime voice lifecycle +app-server-transcript: {threadId, role, text, done} - Realtime transcript delta ``` ### Client → Server Events @@ -749,6 +766,19 @@ GET|POST /api/discord-watch/channels - Which channels a POST /api/discord-watch/items/:id/link - Bind an item to the session doing it (and announce it) POST /api/discord-watch/items/:id/status - Post a status update back into the channel +GET /api/atlas/proposals?status=&repoId= - Agent-proposed highlights waiting for review +POST /api/atlas/proposals - Propose a highlight (agents; changes nothing until approved) +POST /api/atlas/proposals/:id/approve | /reject - Decide a proposal (approve writes it into the registry) +DELETE /api/atlas/proposals - Clear decided proposals + +GET /api/app-server/status | /threads | /signals - Bridge state, Codex threads, structured signals +POST /api/app-server/start | /stop - Control the app-server process +GET /api/app-server/approvals - Approvals reported over the wire, with the command in hand +POST /api/app-server/approvals/:requestId - Grant or refuse one +POST /api/app-server/threads | /threads/:id/turn | /interrupt - Start a thread, run a turn, interrupt it +GET /api/app-server/realtime/voices | /realtime/transcripts - Available voices, transcript history +POST /api/app-server/realtime/:threadId/start|stop|text|audio - Full-duplex realtime voice + GET /api/speech/status - Enabled state, resolved backend, available backends, listeners POST /api/speech/say - Speak text (`priority: high` interrupts, `force` skips repeat suppression) POST /api/speech/backend - Choose a backend diff --git a/docs/COMMANDER_CLAUDE.md b/docs/COMMANDER_CLAUDE.md index 5115caa6..1d24e6c2 100644 --- a/docs/COMMANDER_CLAUDE.md +++ b/docs/COMMANDER_CLAUDE.md @@ -204,6 +204,41 @@ Also available as a CLI anywhere: `node scripts/atlas.js find `. The registry syncs between machines via a private git repo (`GET/POST /api/atlas/sync`). Entries marked `foreign: true` were shared with you by someone else — read them, never re-share them. +## Codex app-server (structured signals + realtime voice) + +Opt-in via `CODEX_APP_SERVER=true`. When on, Codex threads report state as facts instead of being scraped, and approvals arrive with the command attached. + +```bash +curl -sS "$BASE_URL/api/app-server/status" -H "X-Auth-Token: $AUTH_TOKEN" | jq +curl -sS "$BASE_URL/api/app-server/approvals" -H "X-Auth-Token: $AUTH_TOKEN" | jq + +# Grant or refuse an approval over the wire +curl -sS -X POST "$BASE_URL/api/app-server/approvals/" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"approved": true}' + +# Full-duplex voice on a thread +curl -sS -X POST "$BASE_URL/api/app-server/realtime//start" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" -d '{}' +``` + +Only approve something you would approve yourself — the same fail-closed rules apply, and the command is right there in the request. + +## Atlas write-back + +After substantial work, propose what you learned. You cannot write to the map directly. + +```bash +curl -sS -X POST "$BASE_URL/api/atlas/proposals" \ + -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ + -d '{"repoId":"zoo-game","topic":"data-compression","quality":5, + "paths":["src/data/"],"notes":"bitpacked saves", + "evidence":"12x smaller than the JSON it replaced, benchmarked", + "proposedBy":"zoo-game-work1-claude"}' +``` + +Always include `evidence`. Proposals without it get rejected, and rightly so. + ## Discord (ambient team work) The watcher reads whole channels rather than waiting to be addressed, turns assignments into tracked work with a priority, and publishes status back so nobody has to ask whether an agent picked something up. From a339779e0082015d09954e562e0ed631dac4aaab Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 10:50:59 +1000 Subject: [PATCH 15/69] docs: correct stale autonomy default, record what remains The Commander reference still described 'observe' as the shipped default after that changed to autopilot, and the design doc's open-items list was written before most of it shipped. --- ...OMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md | 37 ++++++++++++------- docs/COMMANDER_CLAUDE.md | 4 +- 2 files changed, 25 insertions(+), 16 deletions(-) diff --git a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md index e0cdccc1..78ca8123 100644 --- a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md +++ b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md @@ -225,17 +225,26 @@ false promise. ## 6. What is left -1. **Read a week of findings, then promote the supervisor to `assist`.** This is the whole reason - `observe` is the shipped default. `~/.agent-workspace/logs/supervisor-audit.jsonl`. -2. **Curate the atlas as you go** — `atlas note --topic X --quality N --notes "..."`. The - skill already tells agents to record what they learn, so this should accumulate rather than - needing a curation session. -3. **Tag repos into audiences** before sharing anything: `atlas set --visibility team --groups - core-team`, then `atlas compile core-team --dry-run --explain` and read every line before dropping - the dry-run flag. -4. **Reach** — Discord/mobile push for escalations, borrowing OpenClaw's one genuinely better idea. - `discordIntegrationService` is already half of it. -5. **Atlas write-back** — agents propose highlights from work they just finished, you approve. This - is what keeps the map current instead of letting it rot into another stale doc. -6. **Atlas in the UI** — findings and the digest currently have APIs but no panel. The Commander - status strip proposed in `PLANS/2026-07-15/MULTI_COMMANDER_FEASIBILITY.md` is the natural home. +Everything scoped in section 5 and in `RESEARCH_HERMES_CODEX_AND_DISCORD.md` has shipped: +the fix-first supervisor, git-backed Atlas sync, ambient Discord tracking, the Codex +app-server adapter, realtime voice, Atlas write-back, and the JARVIS panel. + +What genuinely remains is operational rather than unbuilt: + +1. **Point the Atlas registry at a private repo** — `atlas remote set ` then + `atlas sync`. Until that runs, the map is still single-machine. +2. **Curate.** 233 repos are mapped; only a handful are scored. `atlas note` when you know + something, and approve the proposals agents file as they work. +3. **Set `DISCORD_BOT_TOKEN` and add channels** to turn the ambient watcher on. +4. **Set `CODEX_APP_SERVER=true`** to get structured Codex signals in place of scraping. +5. **Read a week of `supervisor-audit.jsonl`** and tune. The defaults are opinions — + interruption threshold, tier weights and `escalateAfterAttempts` are the dials. + +Known limits, stated plainly: + +- The raw-audio realtime path (`appendAudio`) is wired but its PCM framing is unverified + against a live authenticated session. The text path is the default and works. +- Structured signals only cover Codex. Claude, Gemini and aider stay on PTY scraping — + which is why the scraper remains the universal fallback rather than being removed. +- Discord extraction is rules-only. The `classifier` seam for handing ambiguous messages + to a cheap model exists but is unused. diff --git a/docs/COMMANDER_CLAUDE.md b/docs/COMMANDER_CLAUDE.md index 1d24e6c2..926d0a87 100644 --- a/docs/COMMANDER_CLAUDE.md +++ b/docs/COMMANDER_CLAUDE.md @@ -120,7 +120,7 @@ curl -sS "$BASE_URL/api/commander/execute" \ ## Supervisor (the fleet watchdog) -A rule-driven loop classifies every agent session every 30s from zero-token signals (PTY tail, status, quiet time, git state) and climbs an escalation ladder capped by an autonomy level. Ask it what needs attention instead of reading 16 terminals yourself. +A rule-driven loop classifies every agent session every 30s from zero-token signals (PTY tail, status, quiet time, git state — plus structured app-server events for Codex threads) and tries to fix what it finds. Ask it what needs attention instead of reading 16 terminals yourself. ```bash # What needs a human right now — start here @@ -159,7 +159,7 @@ curl -sS -X POST "$BASE_URL/api/supervisor/autonomy" \ -d '{"level": "assist"}' ``` -**Never raise the autonomy level on your own.** That is the user's decision, and `observe` is deliberately the shipped default so the rules can be judged before they are trusted. Rules live in `config/supervisor-rules.json`, overridable at `~/.agent-workspace/supervisor-rules.json`; every action is appended to `~/.agent-workspace/logs/supervisor-audit.jsonl`. +**Never change the autonomy level on your own** — raising or lowering it is the user's decision. Rules live in `config/supervisor-rules.json`, overridable at `~/.agent-workspace/supervisor-rules.json` and by `SUPERVISOR_AUTONOMY`; every action is appended to `~/.agent-workspace/logs/supervisor-audit.jsonl`. ## Speech From 6555d9e4768d3cc43abf257989b01d3531d3612d Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:31:51 +1000 Subject: [PATCH 16/69] fix(codex): app-server client crash on error notif + broken respawn MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two bugs in the app-server bridge, both fatal when CODEX_APP_SERVER=true: 1. A protocol notification with method 'error' (a real, expected type — systemError threads) and any child-process error event were re-emitted as the EventEmitter-reserved 'error' event with no listener, which Node throws for. That surfaced as an uncaughtException and shut the whole orchestrator down, killing every session in every workspace over one Codex thread's problem. Add a default 'error' listener and stop re-emitting the reserved name (the 'error' notification is still delivered via 'notification'). 2. start() nulled this.starting inside the Promise executor, which runs synchronously and is immediately clobbered by the outer assignment — so this.starting held a stale resolved promise forever. Every later start() short-circuited on it, meaning auto-restart after a crash (and manual restart) never spawned again. Clear the marker after the promise settles. Co-Authored-By: Claude Opus 4.8 --- server/agents/appServerClient.js | 29 +++++-- tests/unit/appServerClientLifecycle.test.js | 85 +++++++++++++++++++++ 2 files changed, 109 insertions(+), 5 deletions(-) create mode 100644 tests/unit/appServerClientLifecycle.test.js diff --git a/server/agents/appServerClient.js b/server/agents/appServerClient.js index 7864a7d7..d61ca15e 100644 --- a/server/agents/appServerClient.js +++ b/server/agents/appServerClient.js @@ -36,6 +36,14 @@ class AppServerClient extends EventEmitter { this.restartAttempts = 0; this.lastError = null; this.startedAt = null; + + // EventEmitter throws on an 'error' emit with no listener, which would take + // the whole orchestrator down via uncaughtException. A default listener + // turns a client-level error into a recorded fact instead of a crash. + this.on('error', (error) => { + this.lastError = error?.message || String(error); + this.logger.warn?.('[app-server] client error', { error: this.lastError }); + }); } isRunning() { @@ -47,7 +55,7 @@ class AppServerClient extends EventEmitter { if (this.starting) return this.starting; this.stopped = false; - this.starting = new Promise((resolve) => { + const startPromise = new Promise((resolve) => { try { this.child = spawn(this.command, this.args, { ...getHiddenProcessOptions({ stdio: ['pipe', 'pipe', 'pipe'] }), @@ -56,7 +64,6 @@ class AppServerClient extends EventEmitter { }); } catch (error) { this.lastError = error.message; - this.starting = null; resolve({ running: false, error: error.message }); return; } @@ -83,11 +90,19 @@ class AppServerClient extends EventEmitter { this.startedAt = new Date().toISOString(); this.restartAttempts = 0; - this.starting = null; resolve({ running: true, pid: this.child.pid }); }); - return this.starting; + // Clear the in-flight marker once it settles. The Promise executor runs + // synchronously, so nulling `this.starting` from inside it would be + // clobbered by the assignment below — leaving a stale resolved promise that + // makes every later start() (auto-restart included) a permanent no-op. + this.starting = startPromise; + startPromise.finally(() => { + if (this.starting === startPromise) this.starting = null; + }); + + return startPromise; } scheduleRestart() { @@ -166,7 +181,11 @@ class AppServerClient extends EventEmitter { if (message.method) { this.emit('notification', { method: message.method, params: message.params || {} }); - this.emit(message.method, message.params || {}); + // Re-emit under the method name so listeners can subscribe to a specific + // notification (realtime events do this). Never for 'error': that is a + // reserved EventEmitter event that throws without a listener, and the + // 'error' *notification* is already delivered via 'notification' above. + if (message.method !== 'error') this.emit(message.method, message.params || {}); } } diff --git a/tests/unit/appServerClientLifecycle.test.js b/tests/unit/appServerClientLifecycle.test.js new file mode 100644 index 00000000..277399d6 --- /dev/null +++ b/tests/unit/appServerClientLifecycle.test.js @@ -0,0 +1,85 @@ +const { EventEmitter } = require('events'); + +const mockSpawn = jest.fn(); +jest.mock('child_process', () => ({ spawn: (...args) => mockSpawn(...args) })); + +const { AppServerClient } = require('../../server/agents/appServerClient'); + +const quietLogger = { warn: () => {}, debug: () => {}, info: () => {} }; + +function makeChild(pid = 4242) { + const child = new EventEmitter(); + child.pid = pid; + child.killed = false; + child.exitCode = null; + child.stdout = new EventEmitter(); + child.stdout.setEncoding = () => {}; + child.stderr = new EventEmitter(); + child.stderr.setEncoding = () => {}; + child.stdin = { write: jest.fn() }; + child.kill = jest.fn(() => { child.killed = true; }); + return child; +} + +beforeEach(() => mockSpawn.mockReset()); + +describe('AppServerClient lifecycle', () => { + test('an "error" notification does not crash the process (reserved event guard)', () => { + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + // Before the fix this emitted a listener-less 'error' event, which Node + // throws for — taking the whole orchestrator down via uncaughtException. + expect(() => client.consume('{"method":"error","params":{"message":"boom"}}\n')).not.toThrow(); + }); + + test('the "error" notification is still delivered via the generic notification event', () => { + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + const seen = []; + client.on('notification', (n) => seen.push(n.method)); + client.consume('{"method":"error","params":{"message":"boom"}}\n'); + expect(seen).toEqual(['error']); + }); + + test('a child-process error event is recorded instead of crashing', async () => { + const child = makeChild(); + mockSpawn.mockReturnValue(child); + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + await client.start(); + + expect(() => child.emit('error', new Error('spawn EACCES'))).not.toThrow(); + expect(client.lastError).toMatch(/EACCES/); + }); + + test('start() spawns again after the child exits (no stale in-flight promise)', async () => { + mockSpawn.mockImplementation(() => makeChild()); + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + + const first = await client.start(); + expect(first.running).toBe(true); + expect(mockSpawn).toHaveBeenCalledTimes(1); + + // The child dies; its exit handler nulls this.child. + client.child.emit('exit', 0, null); + expect(client.isRunning()).toBe(false); + + // Before the fix, `this.starting` held a stale resolved promise, so this + // returned the old result and never respawned. + const second = await client.start(); + expect(second.running).toBe(true); + expect(mockSpawn).toHaveBeenCalledTimes(2); + }); + + test('a spawn failure resolves cleanly and leaves start() retryable', async () => { + mockSpawn.mockImplementationOnce(() => { throw new Error('ENOENT'); }); + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + + const failed = await client.start(); + expect(failed.running).toBe(false); + expect(failed.error).toMatch(/ENOENT/); + + // The in-flight marker must be cleared so a later attempt actually retries. + mockSpawn.mockImplementationOnce(() => makeChild()); + const recovered = await client.start(); + expect(recovered.running).toBe(true); + expect(mockSpawn).toHaveBeenCalledTimes(2); + }); +}); From ee251f3ca40e0c43a0e687cbbc8120ead80ca10a Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:32:14 +1000 Subject: [PATCH 17/69] fix(shutdown): stop supervisor, discord-watch and app-server on exit shutdown() only cleaned up sessions/io/http. The app-server child process (codex app-server) was never sent SIGTERM, so every nodemon reload or restart with CODEX_APP_SERVER=true leaked one orphaned process, invisible to a 'ps aux | grep node'. Stop all three new services first. Co-Authored-By: Claude Opus 4.8 --- server/index.js | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/server/index.js b/server/index.js index 6280336f..bb48dff4 100644 --- a/server/index.js +++ b/server/index.js @@ -8727,7 +8727,23 @@ function shutdown(signal = 'unknown') { isShuttingDown = true; logger.info('Shutting down server...', { signal }); - + + // Stop the services this branch added. The app-server one owns a real child + // process (`codex app-server`) that survives a parent restart otherwise — + // nodemon reloads would then leak one orphaned process each. The others only + // clear unref'd JS intervals, stopped here for symmetry. + for (const [name, service] of [ + ['appServerService', appServerService], + ['discordWatchService', discordWatchService], + ['supervisorService', supervisorService] + ]) { + try { + service?.stop?.(); + } catch (error) { + logger.warn(`Failed to stop ${name} during shutdown`, { error: error.message }); + } + } + // Clean up sessions first sessionManager.cleanup(); From 9c2244a60e018ad5efa79fa72ace092ee3a48224 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:33:21 +1000 Subject: [PATCH 18/69] fix(atlas): scrub real private repo names from public template + skill config/repo-atlas.example.json and skills/public/repo-atlas/SKILL.md shipped with real PRIVATE repo names (zoo-game, box2d-luau, drain-the-lake, hyfire2, hytopia-client-tracker) plus a real local path exposing the OS username, all committed to this PUBLIC repo. The design doc's own goal for these artifacts is 'ships with zero personal data; an example manifest only'. Replace with clearly fictional placeholders (acme-tycoon, physics-kit, puzzle-proto, ~/GitHub/...). Co-Authored-By: Claude Opus 4.8 --- config/repo-atlas.example.json | 10 +++++----- skills/public/repo-atlas/SKILL.md | 8 ++++---- 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/config/repo-atlas.example.json b/config/repo-atlas.example.json index 78a8f381..6840aa47 100644 --- a/config/repo-atlas.example.json +++ b/config/repo-atlas.example.json @@ -1,13 +1,13 @@ { "$comment": "Annotated example of a .repo-atlas.json manifest. Copy this into a repo root (or generate one with `node scripts/atlas.js init`), delete the $comment keys, and commit it. Every field is optional — discovery fills in what you leave out.", - "id": "zoo-game", - "name": "Zoo Game", - "summary": "Multiplayer zoo tycoon built on Hytopia. Real players, real economy.", + "id": "acme-tycoon", + "name": "Acme Tycoon", + "summary": "Example multiplayer tycoon game. Replace this whole file with your own repo's details.", "$comment_classification": "kind: game|library|tool|website|service|reference|writing|experiment|infra|other", "kind": "game", - "platforms": ["hytopia"], + "platforms": ["example-engine"], "languages": ["TypeScript"], "dimension": "3d", "tags": ["multiplayer", "tycoon"], @@ -41,7 +41,7 @@ { "topic": "ui", "reason": "hand-rolled HUD, superseded by the shared component kit" } ], - "seeAlso": ["hyfire2", "hytopia-client-tracker"], + "seeAlso": ["acme-shooter", "acme-client-tools"], "$comment_redaction": "Fields stripped from every shared bundle. Redactable: summary, notes, paths, highlights, avoid, seeAlso, tags. The repo still appears in the bundle — it is just opaque.", "redact": [], diff --git a/skills/public/repo-atlas/SKILL.md b/skills/public/repo-atlas/SKILL.md index c1f1fda8..0de74d12 100644 --- a/skills/public/repo-atlas/SKILL.md +++ b/skills/public/repo-atlas/SKILL.md @@ -21,10 +21,10 @@ atlas find # who did this well, and where in the repo Example: ```bash $ atlas find data-compression -5/5 zoo-game +5/5 acme-tycoon bitpacked player save — 12x smaller than the JSON we started with paths: src/data/packSave.ts - /home/ab/GitHub/games/hytopia/zoo-game + ~/GitHub/acme-tycoon ``` Results are ranked by quality (1–5, recorded **per topic**), and repos the user has explicitly marked do-not-copy for that topic are excluded. `⚠old` means untouched for over a year — still readable, just check it against current conventions. @@ -42,8 +42,8 @@ atlas find testing --min-quality 4 `atlas digest` is the right first call when you want orientation rather than an answer. It is deliberately terse: ``` -roblox box2d-luau(physics:5, testing:5) drain-the-lake(testing:4 ⚠old) -hytopia zoo-game(data-compression:5, worldgen:4) +roblox physics-kit(physics:5, testing:5) puzzle-proto(testing:4 ⚠old) +example acme-tycoon(data-compression:5, worldgen:4) ``` ## Not cloned? Still useful From 2460cfb04656e0d16dcb8c7adf18e71c3af27b7b Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:34:55 +1000 Subject: [PATCH 19/69] fix(supervisor): stop auto-approving writes to exec-on-next-op targets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Edit/Write/Update allow pattern had no path restriction, so with the shipped default autonomy 'autopilot' the supervisor would auto-approve a write to .git/hooks, package.json (scripts), .github/workflows, a shell rc, Makefile, ~/.npmrc or /etc — each of which executes on the next already-allowlisted 'npm run build'/'git commit', i.e. unattended RCE. Add path-based deny patterns so those fail closed to a human while ordinary source edits stay auto-approved. Co-Authored-By: Claude Opus 4.8 --- config/supervisor-rules.json | 8 +++++++- tests/unit/supervisorActions.test.js | 22 ++++++++++++++++++++++ 2 files changed, 29 insertions(+), 1 deletion(-) diff --git a/config/supervisor-rules.json b/config/supervisor-rules.json index 594b311f..e6cec253 100644 --- a/config/supervisor-rules.json +++ b/config/supervisor-rules.json @@ -31,12 +31,18 @@ "\\b(npm|pnpm|yarn) (test|run test|run lint|run typecheck|run build)\\b", "\\bjest\\b", "\\bplaywright\\b", "\\bpytest\\b", "\\bcargo (test|check|clippy)\\b" ], + "$comment_deny_paths": "A permission prompt to Edit/Write is auto-approvable in general, but NOT for targets that execute on the next ordinary operation — a poisoned build script, git hook, CI workflow or shell rc turns an approved edit plus an already-allowlisted `npm run build`/`git commit` into arbitrary code execution. Any write to these fails closed to a human.", "permissionDenyPatterns": [ "\\brm\\s+-[rf]", "\\bsudo\\b", "\\bgit (push --force|reset --hard|clean)\\b", "\\bgh (pr merge|release|repo delete|repo edit)\\b", "\\bDROP\\b", "\\bTRUNCATE\\b", "\\bDELETE FROM\\b", "\\bcurl\\b[^\\n]*\\|\\s*(sh|bash)", "\\bchmod\\s+777\\b", "\\bkill(all)?\\b", "\\bnpm publish\\b", "\\bdd\\s+if=", "\\bmkfs\\b", - "~/\\.(ssh|aws|config|codex|claude)\\b", "\\.env\\b" + "~/\\.(ssh|aws|config|codex|claude)\\b", "\\.env\\b", + "\\.git/(hooks|config)", "\\.github/(workflows|actions)\\b", + "(^|[/(])(package(-lock)?\\.json|pnpm-lock\\.yaml|yarn\\.lock)\\b", + "(^|[/(])(Makefile|Rakefile|Gemfile|Cargo\\.toml|pyproject\\.toml|setup\\.py|build\\.gradle|pom\\.xml)\\b", + "(^|[/(.])(bashrc|zshrc|bash_profile|zshenv|profile|gitconfig|npmrc|pre-commit-config\\.yaml)\\b", + "(^|[\\s(])/etc/" ] }, diff --git a/tests/unit/supervisorActions.test.js b/tests/unit/supervisorActions.test.js index 0438ff75..55bf922a 100644 --- a/tests/unit/supervisorActions.test.js +++ b/tests/unit/supervisorActions.test.js @@ -60,9 +60,31 @@ describe('permission classification', () => { test('ordinary edits and commits are allowed — this has to be usable', () => { expect(classifyPermissionPrompt('Edit(src/app.js)', rules.safety).safe).toBe(true); + expect(classifyPermissionPrompt('Write(src/components/Button.tsx)', rules.safety).safe).toBe(true); + expect(classifyPermissionPrompt('Edit(docs/README.md)', rules.safety).safe).toBe(true); expect(classifyPermissionPrompt('Bash(git commit -m "fix")', rules.safety).safe).toBe(true); }); + test('refuses to auto-approve writes that execute on the next ordinary operation', () => { + // Each of these, once auto-approved, runs code via an already-allowlisted + // `npm run build` / `git commit` — so they must fail closed to a human. + for (const prompt of [ + 'Edit(.git/hooks/post-commit)', + 'Write(package.json)', + 'Edit(package-lock.json)', + 'Write(.github/workflows/ci.yml)', + 'Edit(Makefile)', + 'Write(~/.bashrc)', + 'Edit(~/.npmrc)', + 'Write(/etc/passwd)', + 'Edit(pyproject.toml)' + ]) { + const verdict = classifyPermissionPrompt(prompt, rules.safety); + expect(verdict.safe).toBe(false); + expect(verdict.reason).toMatch(/deny pattern/); + } + }); + test('an empty prompt is not safe', () => { expect(classifyPermissionPrompt('', rules.safety).safe).toBe(false); }); From 75cff6ef8c236d0d8abe633d5b1c5238a18f5639 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:36:03 +1000 Subject: [PATCH 20/69] fix(atlas): keep unscored highlights null instead of forcing quality 1 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit qualityScore() ran Number(value) before the finite check, and Number(null) is 0, which clamped to 1. An intentionally-unscored highlight (quality: null, which atlasProposals explicitly supports for 'no opinion yet') therefore became the worst score once approved — inventing a cautionary rating the design doc's own 'a fabricated score is worse than a blank field' principle forbids. Guard the null-ish cases; a real out-of-range 0 still floors to 1 as before. Co-Authored-By: Claude Opus 4.8 --- server/atlas/atlasSchema.js | 5 +++++ tests/unit/repoAtlasSchema.test.js | 11 +++++++++++ 2 files changed, 16 insertions(+) diff --git a/server/atlas/atlasSchema.js b/server/atlas/atlasSchema.js index 08d63c5f..98620ce9 100644 --- a/server/atlas/atlasSchema.js +++ b/server/atlas/atlasSchema.js @@ -96,6 +96,11 @@ function slugList(value) { } function qualityScore(value) { + // `Number(null)` and `Number('')` are 0, which would clamp to 1 — silently + // turning an intentionally-unscored highlight ("no opinion yet") into the + // worst possible score. Unscored must stay null; only real numbers clamp + // (an explicit out-of-range 0 still floors to 1). + if (value === null || value === undefined || value === '' || typeof value === 'boolean') return null; const num = Number(value); if (!Number.isFinite(num)) return null; return Math.min(5, Math.max(1, Math.round(num))); diff --git a/tests/unit/repoAtlasSchema.test.js b/tests/unit/repoAtlasSchema.test.js index 443678f1..57d48b0c 100644 --- a/tests/unit/repoAtlasSchema.test.js +++ b/tests/unit/repoAtlasSchema.test.js @@ -41,6 +41,17 @@ describe('atlasSchema', () => { ]); }); + test('an unscored highlight stays unscored rather than becoming quality 1', () => { + const entry = normalizeEntry({ + id: 'x', + highlights: [ + { topic: 'testing', quality: null, notes: 'no opinion yet' }, + { topic: 'physics' } + ] + }); + expect(entry.highlights.map((h) => h.quality)).toEqual([null, null]); + }); + test('normalizeEntry only accepts redactions for known fields', () => { const entry = normalizeEntry({ id: 'x', redact: ['notes', 'secrets', 'paths'] }); expect(entry.redact).toEqual(['notes', 'paths']); From b2f1762e063d4ea242b155db38b270690d5ff846 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:37:35 +1000 Subject: [PATCH 21/69] fix(voice): filter realtime socket events by thread id app-server-transcript and app-server-realtime are broadcast to every client (io.emit), but both handlers acted on all of them. Two tabs on different Codex threads would hear each other's assistant replies spoken aloud, and one thread closing would stop another tab's listening loop. Ignore events whose threadId isn't this client's, matching the server-side getTranscripts() filter. Co-Authored-By: Claude Opus 4.8 --- client/realtime-voice.js | 12 +++++++++++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/client/realtime-voice.js b/client/realtime-voice.js index 040f8bae..af7e3df6 100644 --- a/client/realtime-voice.js +++ b/client/realtime-voice.js @@ -60,6 +60,11 @@ socket.__realtimeVoiceAttached = true; socket.on('app-server-transcript', (entry) => { + // These events are broadcast to every client. Only react to our own + // thread's — otherwise a second tab on a different thread would hear + // this one's replies spoken aloud. + if (!this.threadId || entry?.threadId !== this.threadId) return; + this.transcript.unshift(entry); if (this.transcript.length > 100) this.transcript.length = 100; @@ -71,7 +76,12 @@ this.emit(); }); - socket.on('app-server-realtime', ({ event }) => { + socket.on('app-server-realtime', ({ event, payload }) => { + // Same fleet-wide broadcast: ignore other threads' lifecycle events so + // one thread closing can't stop this tab from listening. + const threadId = payload?.threadId || null; + if (this.threadId && threadId && threadId !== this.threadId) return; + if (event === 'thread/realtime/started') this.active = true; if (event === 'thread/realtime/closed' || event === 'thread/realtime/error') { this.active = false; From 2527df6b561c29064857b2917001fd8b2d54f32f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:40:34 +1000 Subject: [PATCH 22/69] fix(discord): merge override config, fix NaN defaults, guard negated signals Three bugs in the ambient watcher's config + extraction: - loadConfig picked the override file wholesale, so the documented minimal override ({enabled, channels}) silently emptied every priority/kind/claim/ done/drop pattern table. Layer it on top of the defaults instead. - Number(x) ?? default leaves NaN when the field is absent (?? only catches null/undefined), so a missing backfillMessages/minLength became NaN. Use a finite-check helper; an explicit backfillMessages:0 is still preserved. - 'not done yet' / 'isn't fixed' matched the done patterns and closed an open item. Guard complete/claim with a negation check (drop keeps its intentional negatives like 'not needed'). Co-Authored-By: Claude Opus 4.8 --- server/discord/workExtractor.js | 43 ++++++++++++++++++++------ tests/unit/discordWatchService.test.js | 41 ++++++++++++++++++++++++ 2 files changed, 75 insertions(+), 9 deletions(-) diff --git a/server/discord/workExtractor.js b/server/discord/workExtractor.js index 59640597..2c400d06 100644 --- a/server/discord/workExtractor.js +++ b/server/discord/workExtractor.js @@ -30,17 +30,30 @@ function readJson(filePath) { } } +// `Number(undefined)` is NaN and `NaN ?? fallback` is still NaN (?? only catches +// null/undefined), so a missing numeric field must be caught explicitly. An +// explicit 0 (e.g. backfillMessages: 0 = "never look back") is preserved. +function numberOr(value, fallback) { + const num = Number(value); + return Number.isFinite(num) ? num : fallback; +} + function loadConfig({ configPath = null } = {}) { - const override = configPath || overrideConfigPath(); - const raw = readJson(override) || readJson(DEFAULT_CONFIG_PATH) || {}; - const source = readJson(override) ? override : DEFAULT_CONFIG_PATH; + const overridePath = configPath || overrideConfigPath(); + const override = readJson(overridePath); + const defaults = readJson(DEFAULT_CONFIG_PATH) || {}; + // Layer the per-machine override on top of the shipped defaults. Picking one + // file wholesale meant the documented minimal override ({enabled, channels}) + // silently wiped every priority/kind/claim/done/drop pattern table. + const raw = override ? { ...defaults, ...override } : defaults; + const source = override ? overridePath : DEFAULT_CONFIG_PATH; return { source, enabled: raw.enabled === true, - pollSeconds: Math.max(5, Number(raw.pollSeconds) || 15), + pollSeconds: Math.max(5, numberOr(raw.pollSeconds, 15)), channels: Array.isArray(raw.channels) ? raw.channels.map(String).filter(Boolean) : [], - backfillMessages: Math.max(0, Number(raw.backfillMessages) ?? 50), + backfillMessages: Math.max(0, numberOr(raw.backfillMessages, 50)), publishStatus: raw.publishStatus !== false, priority: (Array.isArray(raw.priority) ? raw.priority : []).map((row) => ({ level: String(row.level || 'normal'), @@ -61,10 +74,20 @@ function loadConfig({ configPath = null } = {}) { dropPatterns: compile(raw.dropPatterns), ignoreBots: raw.ignoreBots !== false, ignorePrefixes: Array.isArray(raw.ignorePrefixes) ? raw.ignorePrefixes.map(String) : [], - minLength: Math.max(0, Number(raw.minLength) ?? 12) + minLength: Math.max(0, numberOr(raw.minLength, 12)) }; } +// A completion/claim keyword directly preceded by a negator means the opposite +// ("not done yet", "isn't fixed"). A punctuation break resets it, so "not a +// problem, it's done" is still a completion. Drop patterns are left alone — +// their negatives ("not needed", "won't do") are intentional. +const NEGATED_SIGNAL_RE = /\b(not|isn'?t|aren'?t|wasn'?t|weren'?t|haven'?t|hasn'?t|hadn'?t|didn'?t|don'?t|doesn'?t|won'?t|can'?t|never|no longer)\b[\s\w']{0,20}?\b(done|shipped?|merged?|fixed?|completed?|ready|live|handled|taking|starting|on it)\b/i; + +function isNegatedSignal(text) { + return NEGATED_SIGNAL_RE.test(String(text || '')); +} + const MENTION_RE = /<@!?(\d+)>/g; function extractMentions(message) { @@ -125,14 +148,15 @@ function extractFromMessage(message, { config, guildId = '', memberNames = {} } const mentions = extractMentions(message); // A signal about existing work beats creating new work — "done" following an - // assignment is a status change, not a new task. - if (config.donePatterns.some((re) => re.test(text))) { + // assignment is a status change, not a new task. A negated signal ("not done + // yet") is neither, so it falls through to normal classification. + if (config.donePatterns.some((re) => re.test(text)) && !isNegatedSignal(text)) { return { action: 'complete', messageId: message.id, authorId: message.author?.id, text, mentions }; } if (config.dropPatterns.some((re) => re.test(text))) { return { action: 'drop', messageId: message.id, authorId: message.author?.id, text, mentions }; } - if (config.claimPatterns.some((re) => re.test(text))) { + if (config.claimPatterns.some((re) => re.test(text)) && !isNegatedSignal(text)) { return { action: 'claim', messageId: message.id, authorId: message.author?.id, text, mentions }; } @@ -184,6 +208,7 @@ module.exports = { classifyPriority, classifyKind, shouldIgnore, + isNegatedSignal, extractFromMessage, extractBatch }; diff --git a/tests/unit/discordWatchService.test.js b/tests/unit/discordWatchService.test.js index 040f0b22..48280c1c 100644 --- a/tests/unit/discordWatchService.test.js +++ b/tests/unit/discordWatchService.test.js @@ -71,6 +71,47 @@ describe('workExtractor', () => { ); expect(result.text).toBe('@sam please add the leaderboard'); }); + + test('a negated completion is not treated as done', () => { + expect(extractor.extractFromMessage(message({ content: 'that bug is not done yet, still crashing' }), { config }).action).not.toBe('complete'); + expect(extractor.extractFromMessage(message({ content: "the payment flow isn't fixed, more work needed" }), { config }).action).not.toBe('complete'); + // A punctuation break resets the negation, so this is still a completion. + expect(extractor.extractFromMessage(message({ content: 'not a problem, that one is done and merged' }), { config }).action).toBe('complete'); + }); +}); + +describe('workExtractor loadConfig', () => { + const withConfig = (obj, fn) => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'disc-cfg-')); + const file = path.join(dir, 'discord-watch.json'); + fs.writeFileSync(file, JSON.stringify(obj)); + try { return fn(extractor.loadConfig({ configPath: file })); } + finally { fs.rmSync(dir, { recursive: true, force: true }); } + }; + + test('a minimal override keeps the shipped pattern tables instead of wiping them', () => { + withConfig({ enabled: true, channels: ['123'] }, (cfg) => { + expect(cfg.enabled).toBe(true); + expect(cfg.channels).toEqual(['123']); + // These come from the defaults — a wholesale replace would have emptied them. + expect(cfg.priority.length).toBeGreaterThan(0); + expect(cfg.kinds.length).toBeGreaterThan(0); + expect(cfg.donePatterns.length).toBeGreaterThan(0); + }); + }); + + test('a missing numeric field falls back to its default rather than NaN', () => { + withConfig({ enabled: true, channels: ['123'] }, (cfg) => { + expect(cfg.backfillMessages).toBe(50); + expect(cfg.minLength).toBe(12); + }); + }); + + test('an explicit backfill of 0 (never look back) is preserved', () => { + withConfig({ enabled: true, channels: ['123'], backfillMessages: 0 }, (cfg) => { + expect(cfg.backfillMessages).toBe(0); + }); + }); }); describe('DiscordClient', () => { From 3174e593e53feed079f8f921cdb11ac14f16b49f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:42:55 +1000 Subject: [PATCH 23/69] fix(discord,atlas): correct backfill slice + atomic state/registry writes - fetchMessagesAfter kept the OLDEST N of a first-sight backfill (slice(0,N)) after fetching newest-first, so it skipped exactly the most recent messages a backfill exists to catch. Keep the tail (most recent N) when there's no cursor; forward paging still keeps the oldest N after the cursor. Also point the returned cursor at the last kept message so a >cap page can't skip the overflow. - atlasStore.writeJson and DiscordWatchService.saveState wrote straight to the final path; a crash mid-write corrupts the file. The atlas registry is git-synced, so a half-write would propagate everywhere. Write-then-rename. Co-Authored-By: Claude Opus 4.8 --- server/atlas/atlasStore.js | 8 +++++++- server/discord/discordClient.js | 10 +++++++++- server/discordWatchService.js | 7 ++++++- tests/unit/discordWatchService.test.js | 16 ++++++++++++++++ 4 files changed, 38 insertions(+), 3 deletions(-) diff --git a/server/atlas/atlasStore.js b/server/atlas/atlasStore.js index e4fd74ac..81112033 100644 --- a/server/atlas/atlasStore.js +++ b/server/atlas/atlasStore.js @@ -70,7 +70,13 @@ function readJson(filePath, fallback = null) { function writeJson(filePath, value) { ensureDir(path.dirname(filePath)); - fs.writeFileSync(filePath, `${JSON.stringify(value, null, 2)}\n`, 'utf8'); + // Write-then-rename so a crash mid-write can't leave a truncated file. The + // registry is git-synced, so a corrupted half-write would otherwise be + // committed and propagated to every other machine. rename is atomic on the + // same filesystem; the pid keeps concurrent writers from sharing a temp path. + const tmp = `${filePath}.${process.pid}.tmp`; + fs.writeFileSync(tmp, `${JSON.stringify(value, null, 2)}\n`, 'utf8'); + fs.renameSync(tmp, filePath); return filePath; } diff --git a/server/discord/discordClient.js b/server/discord/discordClient.js index 62388729..ccb07063 100644 --- a/server/discord/discordClient.js +++ b/server/discord/discordClient.js @@ -68,6 +68,11 @@ class DiscordClient { * Page forward from a cursor. Discord returns newest-first even with `after`, * so pages are reversed into chronological order — work items should be * created in the order the conversation actually happened. + * + * With no cursor (first-sight backfill) there is nothing to page forward from, + * so a single newest-first page is taken and trimmed to the most recent N. + * Trimming the oldest N instead would silently skip the newest messages — + * exactly the ones a backfill is meant to catch. */ async fetchMessagesAfter(channelId, afterId, { maxMessages = 400 } = {}) { const collected = []; @@ -88,7 +93,10 @@ class DiscordClient { if (page.length < MAX_PAGE) break; } - return { ok: true, messages: collected.slice(0, maxMessages), cursor: cursor || afterId }; + // Forward paging keeps the oldest N after the cursor; a backfill keeps the + // most recent N (the tail of the chronological list). + const messages = afterId ? collected.slice(0, maxMessages) : collected.slice(-maxMessages); + return { ok: true, messages, cursor: messages.length ? messages[messages.length - 1].id : (cursor || afterId) }; } async getLatestMessageId(channelId) { diff --git a/server/discordWatchService.js b/server/discordWatchService.js index 42f3b51e..353add20 100644 --- a/server/discordWatchService.js +++ b/server/discordWatchService.js @@ -76,7 +76,12 @@ class DiscordWatchService { saveState() { try { - fs.writeFileSync(this.statePath(), `${JSON.stringify(this.state, null, 2)}\n`, 'utf8'); + // Write-then-rename: a crash mid-write must not corrupt the cursor state, + // or the "no missed messages" guarantee turns into duplicates or losses. + const target = this.statePath(); + const tmp = `${target}.${process.pid}.tmp`; + fs.writeFileSync(tmp, `${JSON.stringify(this.state, null, 2)}\n`, 'utf8'); + fs.renameSync(tmp, target); } catch (error) { this.logger.warn?.('Discord watch could not persist state', { error: error.message }); } diff --git a/tests/unit/discordWatchService.test.js b/tests/unit/discordWatchService.test.js index 48280c1c..84017481 100644 --- a/tests/unit/discordWatchService.test.js +++ b/tests/unit/discordWatchService.test.js @@ -141,6 +141,22 @@ describe('DiscordClient', () => { expect(seen[0]).toContain('after=1'); }); + test('a first-sight backfill keeps the most recent N, not the oldest N', async () => { + // 150 messages, newest-first per page (ids 150..51, then 50..1). + const all = Array.from({ length: 150 }, (_, i) => ({ id: String(150 - i) })); + const pages = [all.slice(0, 100), all.slice(100), []]; + const client = new DiscordClient({ + token: 't', + fetchImpl: async () => ({ ok: true, status: 200, headers: { get: () => null }, text: async () => JSON.stringify(pages.shift() || []) }) + }); + + // No cursor => backfill. Want the 50 most recent (ids 101..150), chronological. + const result = await client.fetchMessagesAfter('chan1', null, { maxMessages: 50 }); + expect(result.messages).toHaveLength(50); + expect(result.messages[result.messages.length - 1].id).toBe('150'); + expect(result.messages[0].id).toBe('101'); + }); + test('a 429 backs off instead of hammering', async () => { const client = new DiscordClient({ token: 't', From 2e986510de7e5d4a02597210e41eeecf0e228acb Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:44:10 +1000 Subject: [PATCH 24/69] fix(ui): guard Alt+J while typing + surface panel action failures - Alt+J fired even inside inputs/textareas (including the panel's own atlas search box) and on Ctrl+Alt/AltGr layouts, swallowing the keystroke. Add the same typing/modifier guard every other Alt-shortcut in the app uses. - The catch-up / approve / reject buttons awaited network calls with no try/catch, so a failed request became an unhandled rejection and the button looked dead. Catch and show a dismissable error banner (white on dark red). Co-Authored-By: Claude Opus 4.8 --- client/jarvis-panel.js | 57 +++++++++++++++++++++++++++++----------- client/styles/jarvis.css | 16 +++++++++++ 2 files changed, 58 insertions(+), 15 deletions(-) diff --git a/client/jarvis-panel.js b/client/jarvis-panel.js index a8cbe96b..51b3895e 100644 --- a/client/jarvis-panel.js +++ b/client/jarvis-panel.js @@ -63,6 +63,7 @@
+
@@ -92,22 +93,41 @@ if (!action) return; if (action === 'close') return this.hide(); - if (action === 'refresh') return this.refresh(); - if (action === 'atlas-find') return this.findInAtlas(); - if (action === 'catch-up') { - await api('/api/supervisor/digest/deliver', { method: 'POST', body: '{}' }); - return this.refresh(); - } - if (action === 'approve-proposal' || action === 'reject-proposal') { - const id = event.target.dataset.id; - const verb = action === 'approve-proposal' ? 'approve' : 'reject'; - await api(`/api/atlas/proposals/${encodeURIComponent(id)}/${verb}`, { method: 'POST', body: '{}' }); - return this.refresh(); + // Every branch below hits the network; a failure must surface, not vanish + // into an unhandled rejection that leaves the button looking dead. + try { + if (action === 'refresh') return await this.refresh(); + if (action === 'atlas-find') return await this.findInAtlas(); + + if (action === 'catch-up') { + await api('/api/supervisor/digest/deliver', { method: 'POST', body: '{}' }); + return await this.refresh(); + } + if (action === 'approve-proposal' || action === 'reject-proposal') { + const id = event.target.dataset.id; + const verb = action === 'approve-proposal' ? 'approve' : 'reject'; + await api(`/api/atlas/proposals/${encodeURIComponent(id)}/${verb}`, { method: 'POST', body: '{}' }); + return await this.refresh(); + } + } catch (error) { + this.showActionError(action, error); } return undefined; } + showActionError(action, error) { + const banner = this.root?.querySelector('[data-role="action-error"]'); + const message = `${action} failed: ${error?.message || error}`; + if (banner) { + banner.textContent = message; + banner.classList.add('jarvis-error-visible'); + clearTimeout(this._errorTimer); + this._errorTimer = setTimeout(() => banner.classList.remove('jarvis-error-visible'), 5000); + } + console.error('[JARVIS]', message); + } + async refresh() { const settled = await Promise.allSettled([ api('/api/supervisor/briefing'), @@ -295,9 +315,16 @@ // Alt+J — the fleet summary should be one keystroke away, not buried. document.addEventListener('keydown', (event) => { - if (event.altKey && (event.key === 'j' || event.key === 'J')) { - event.preventDefault(); - panel.toggle(); - } + if (!event.altKey || event.ctrlKey || event.metaKey) return; + if (event.key !== 'j' && event.key !== 'J') return; + + // Don't hijack the key while the user is typing (the same guard every other + // Alt-shortcut in the app uses) — including the panel's own atlas search box. + const target = event.target; + const tag = String(target?.tagName || '').toLowerCase(); + if (tag === 'input' || tag === 'textarea' || tag === 'select' || target?.isContentEditable) return; + + event.preventDefault(); + panel.toggle(); }); })(); diff --git a/client/styles/jarvis.css b/client/styles/jarvis.css index 32e6239e..4d5aa965 100644 --- a/client/styles/jarvis.css +++ b/client/styles/jarvis.css @@ -86,6 +86,22 @@ flex-shrink: 0; } +.jarvis-error { + display: none; + margin: 0 1.25rem 0.75rem; + padding: 0.5rem 0.75rem; + border-radius: 0.375rem; + background: #5a2020; + border: 1px solid #8a3030; + color: #ffffff; + font-size: 0.8125rem; + font-weight: 600; +} + +.jarvis-error.jarvis-error-visible { + display: block; +} + .jarvis-stat { display: flex; flex-direction: column; From 4cc3f653b82c91fe5d13e0ae56f82d0149cf7885 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:45:48 +1000 Subject: [PATCH 25/69] fix(atlas cli): preserve '=' in flag values, reject value-less --quality - parseArgs split flag tokens on every '=', truncating any value containing one (--notes="a = b" became "a "). Split on the first '=' only. - A forgotten --quality value parsed as boolean true and Number(true) silently became quality 1 (the worst score). Validate: a value-less or non-numeric --quality now errors instead of inventing a rating. Co-Authored-By: Claude Opus 4.8 --- scripts/atlas.js | 32 ++++++++++++++++++++++++++++---- 1 file changed, 28 insertions(+), 4 deletions(-) diff --git a/scripts/atlas.js b/scripts/atlas.js index 463f6616..b9788d85 100755 --- a/scripts/atlas.js +++ b/scripts/atlas.js @@ -31,7 +31,12 @@ function parseArgs(argv) { positionals.push(token); continue; } - const [rawKey, inlineValue] = token.slice(2).split('='); + // Split on the FIRST '=' only, so a value containing '=' survives + // (e.g. --notes="a = b" or --evidence="x == y"). + const body = token.slice(2); + const eq = body.indexOf('='); + const rawKey = eq === -1 ? body : body.slice(0, eq); + const inlineValue = eq === -1 ? undefined : body.slice(eq + 1); const key = rawKey.replace(/-([a-z])/g, (_, c) => c.toUpperCase()); if (inlineValue !== undefined) { flags[key] = inlineValue; @@ -54,6 +59,17 @@ const listFlag = (value) => String(value === true ? '' : value || '') .map((v) => v.trim()) .filter(Boolean); +// A value-less `--quality` (parsed as `true`) or a non-number must not be +// silently coerced to a score — Number(true) is 1, the worst rating. +// Returns null for "not given", a number, or a QUALITY_INVALID sentinel. +const QUALITY_INVALID = Symbol('quality-invalid'); +const qualityFlag = (value) => { + if (value === undefined) return null; + if (value === true) return QUALITY_INVALID; + const num = Number(value); + return Number.isFinite(num) ? num : QUALITY_INVALID; +}; + const out = (text) => process.stdout.write(`${text}\n`); const fail = (message) => { process.stderr.write(`atlas: ${message}\n`); @@ -198,9 +214,11 @@ const commands = { note(positionals, flags) { const id = positionals[0]; if (!id || !flags.topic) return fail('usage: atlas note --topic [--quality 1-5] [--paths a,b] [--notes "..."]'); + const quality = qualityFlag(flags.quality); + if (quality === QUALITY_INVALID) return fail('--quality needs a number 1-5'); const saved = atlas.addHighlight(id, { topic: flags.topic, - quality: flags.quality === undefined ? null : Number(flags.quality), + quality, paths: listFlag(flags.paths), notes: flags.notes === true ? '' : String(flags.notes || '') }); @@ -229,7 +247,11 @@ const commands = { if (flags.platforms !== undefined) patch.platforms = listFlag(flags.platforms); if (flags.tags !== undefined) patch.tags = listFlag(flags.tags); if (flags.redact !== undefined) patch.redact = listFlag(flags.redact); - if (flags.quality !== undefined) patch.quality = Number(flags.quality); + if (flags.quality !== undefined) { + const quality = qualityFlag(flags.quality); + if (quality === QUALITY_INVALID) return fail('--quality needs a number 1-5'); + patch.quality = quality; + } if (!Object.keys(patch).length) return fail('nothing to set'); const saved = atlas.setEntry(id, patch); @@ -266,11 +288,13 @@ const commands = { if (!id || !flags.topic) { return fail('usage: atlas propose --topic [--quality 1-5] [--paths a,b] [--notes "..."] [--evidence "why"] [--avoid]'); } + const quality = qualityFlag(flags.quality); + if (quality === QUALITY_INVALID) return fail('--quality needs a number 1-5'); const proposal = atlas.proposeHighlight({ repoId: id, topic: flags.topic, kind: flags.avoid === true ? 'avoid' : 'highlight', - quality: flags.quality === undefined ? null : Number(flags.quality), + quality, paths: listFlag(flags.paths), notes: flags.notes === true ? '' : String(flags.notes || ''), evidence: flags.evidence === true ? '' : String(flags.evidence || ''), From 3d7190f126325915682cec0ae8a8c4bf9b6a758a Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:48:03 +1000 Subject: [PATCH 26/69] fix(atlas): a local note must not make a subscribed repo re-shareable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit An entry was marked foreign (excluded from compiled bundles) only when it was subscription-ONLY. Adding any local registry note to a repo a teammate shared added a registry layer, cleared the foreign flag, and compile() then re-published the whole entry — including summary/highlights inherited from the teammate's bundle — defeating the 'subscriber provably unable to re-share' guarantee. Foreignness now depends on whether you actually have the repo locally (discovery), not on whether you annotated it. Co-Authored-By: Claude Opus 4.8 --- server/repoAtlasService.js | 7 ++++++- tests/unit/repoAtlasSync.test.js | 22 +++++++++++++++++++++- 2 files changed, 27 insertions(+), 2 deletions(-) diff --git a/server/repoAtlasService.js b/server/repoAtlasService.js index 843863e4..dc0550a7 100644 --- a/server/repoAtlasService.js +++ b/server/repoAtlasService.js @@ -138,7 +138,12 @@ class RepoAtlasService { layers.manifest, layers.registry ); - if (layers.subscription && !layers.discovery && !layers.registry) { + // Foreign = it only exists here because a teammate shared it. A local + // registry note (annotating their entry for your own searches) must NOT + // declassify it — otherwise compile() would re-publish their inherited + // fields under your bundle, breaking the "never re-share" guarantee. Only + // actually having the repo locally (discovery) makes it yours to share. + if (layers.subscription && !layers.discovery) { merged.foreign = true; merged.sharedBy = layers.subscription.sharedBy; } diff --git a/tests/unit/repoAtlasSync.test.js b/tests/unit/repoAtlasSync.test.js index bd0e7d93..8d7b4d13 100644 --- a/tests/unit/repoAtlasSync.test.js +++ b/tests/unit/repoAtlasSync.test.js @@ -157,7 +157,27 @@ describe('Repo Atlas multi-machine sync', () => { const entry = me.getEntry('shared-repo'); expect(entry.highlights[0].quality).toBe(5); expect(entry.summary).toBe('their description'); - expect(entry.foreign).toBeUndefined(); + }); + + test('annotating a shared repo does not make it re-shareable', async () => { + const shared = publishBundle(root, [{ + id: 'shared-repo', + name: 'shared-repo', + summary: 'their description', + visibility: 'public', + highlights: [{ topic: 'testing', quality: 2 }] + }]); + + const me = machine('me'); + await me.subscribe({ name: 'them', source: shared }); + // A local note must not declassify a repo you only know about because a + // teammate shared it — re-publishing it would leak their inherited fields. + me.addHighlight('shared-repo', { topic: 'testing', quality: 5, notes: 'actually excellent' }); + me.setEntry('shared-repo', { visibility: 'public' }); + + expect(me.getEntry('shared-repo').foreign).toBe(true); + const compiled = me.compile('anyone', { write: false }); + expect(compiled.bundle.entries.map((e) => e.id)).not.toContain('shared-repo'); }); test('subscribing to something that is not a bundle fails loudly', async () => { From 34f2882ba36ad3a01e79207049e5e75d2e54c7c0 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:49:46 +1000 Subject: [PATCH 27/69] docs: note the app-server thread linkage is still dormant The bridge works against a real codex app-server, but no session-to-thread-id link exists yet, so the supervisor runs on PTY signals in practice. State that plainly rather than implying structured signals are live. --- .../AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md index 78ca8123..e62eb289 100644 --- a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md +++ b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md @@ -246,5 +246,10 @@ Known limits, stated plainly: against a live authenticated session. The text path is the default and works. - Structured signals only cover Codex. Claude, Gemini and aider stay on PTY scraping — which is why the scraper remains the universal fallback rather than being removed. +- The app-server bridge is verified working in isolation (initialize handshake, thread + list, live notifications), but nothing yet links a running PTY session to its + app-server thread id, so `getSignalForSession` returns null for every session today and + the supervisor runs entirely on PTY signals. Wiring that correlation is what turns the + structured path on; until then it is dormant, not active. - Discord extraction is rules-only. The `classifier` seam for handing ambiguous messages to a cheap model exists but is unused. From f1449c4cb5efcc7708fae3cffb7a65425e7a2c39 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:50:52 +1000 Subject: [PATCH 28/69] fix(speech): prune the repeat-dedup map so it can't grow unbounded lastSpokenAt kept one entry per distinct spoken string forever. Over a long-lived server that is a slow leak. Drop entries past the repeat window (they can never match isRepeat again) each time a new one is recorded. Co-Authored-By: Claude Opus 4.8 --- server/speechService.js | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/server/speechService.js b/server/speechService.js index e800f1de..17b45b9c 100644 --- a/server/speechService.js +++ b/server/speechService.js @@ -204,7 +204,15 @@ class SpeechService { result = { spoken: false, reason: error.message }; } - if (result.spoken) this.lastSpokenAt.set(text, Date.now()); + if (result.spoken) { + const now = Date.now(); + // Entries older than the repeat window are dead weight — prune them so + // this dedup map can't grow unbounded over a long-lived server. + for (const [key, at] of this.lastSpokenAt) { + if (now - at >= REPEAT_WINDOW_MS) this.lastSpokenAt.delete(key); + } + this.lastSpokenAt.set(text, now); + } return this.record({ text, backend, priority, at: new Date().toISOString(), ...result }); } From bab9037ba64246ffe47786b762d57abe3411091f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 14:56:04 +1000 Subject: [PATCH 29/69] fix: proposals atomic write, app-server buffer overflow, supervisor robustness Second review pass, lower-severity items confirmed by the scout reports: - atlasProposals.save wrote the whole proposal queue non-atomically (same class as the registry fix, bigger blast radius). Route through store.writeJson. - app-server consume() dropped the ENTIRE buffer on overflow, discarding any complete frames queued ahead of the oversized line. Process complete lines first, then drop only the oversized incomplete tail. - supervisor nudge conditions (stalled, unpushed-work, uncommitted-work) relied on array ordering to avoid typing into an exited agent's bare shell. Require agentPresent:true so the invariant holds regardless of operator reordering. - supervisorSignals tier used Number(x)||null (the 0-becomes-null gotcha this repo's CLAUDE.md calls out). Use a finite check. - voice-control showed a literal 'null' status when speech was forwarded to the Commander (no command name). Show 'Sent to Commander' instead. Co-Authored-By: Claude Opus 4.8 --- client/voice-control.js | 5 ++++- config/supervisor-rules.json | 3 +++ server/agents/appServerClient.js | 13 ++++++++----- server/atlas/atlasProposals.js | 6 ++++-- server/supervisor/supervisorSignals.js | 2 +- tests/unit/appServerClientLifecycle.test.js | 14 ++++++++++++++ 6 files changed, 34 insertions(+), 9 deletions(-) diff --git a/client/voice-control.js b/client/voice-control.js index 222c8f27..4e0d78cb 100644 --- a/client/voice-control.js +++ b/client/voice-control.js @@ -412,7 +412,10 @@ class VoiceControl { if (result.success) { this.transcriptEl.textContent = result.transcript; - this.setStatus(`${result.command} (${result.transcriptionTime}ms)`, 'success'); + // Unmatched speech is forwarded to the Commander and has no command name; + // showing the literal "null" reads as a bug. + const label = result.forwardedToCommander ? 'Sent to Commander' : (result.command || 'done'); + this.setStatus(`${label} (${result.transcriptionTime}ms)`, 'success'); this.showFeedback(result); } else { this.transcriptEl.textContent = result.transcript || ''; diff --git a/config/supervisor-rules.json b/config/supervisor-rules.json index e6cec253..dad5bc09 100644 --- a/config/supervisor-rules.json +++ b/config/supervisor-rules.json @@ -136,6 +136,7 @@ "resolve": { "handler": "nudge", "text": "status? If you are blocked, say what on, then resolve it yourself if you can." }, "when": { "status": ["busy"], + "agentPresent": true, "minQuietSeconds": 900, "tailNotMatches": ["limit reached", "Do you want to (proceed|make this edit)"] }, @@ -164,6 +165,7 @@ "resolve": { "handler": "nudge", "text": "You have local commits that are not pushed. Push the branch and open a PR, then say what you shipped." }, "when": { "status": ["idle"], + "agentPresent": true, "minQuietSeconds": 300, "git": { "aheadMin": 1 } }, @@ -178,6 +180,7 @@ "resolve": { "handler": "nudge", "text": "You have uncommitted changes. Commit them with a descriptive message, or say why they should not be committed." }, "when": { "status": ["idle"], + "agentPresent": true, "minQuietSeconds": 900, "git": { "dirty": true, "aheadMax": 0 } }, diff --git a/server/agents/appServerClient.js b/server/agents/appServerClient.js index d61ca15e..4aabc082 100644 --- a/server/agents/appServerClient.js +++ b/server/agents/appServerClient.js @@ -138,11 +138,6 @@ class AppServerClient extends EventEmitter { consume(chunk) { this.buffer += chunk; - if (this.buffer.length > MAX_LINE_BYTES) { - this.logger.warn?.('app-server output exceeded the line buffer; dropping it'); - this.buffer = ''; - return; - } let index = this.buffer.indexOf('\n'); while (index !== -1) { @@ -151,6 +146,14 @@ class AppServerClient extends EventEmitter { if (line) this.handleLine(line); index = this.buffer.indexOf('\n'); } + + // Only an unterminated trailing line can remain. Drop it if it alone blows + // the cap — the complete messages ahead of it were already handled, so this + // no longer discards queued-but-parseable frames along with the oversized one. + if (this.buffer.length > MAX_LINE_BYTES) { + this.logger.warn?.('app-server line exceeded the buffer; dropping the incomplete line'); + this.buffer = ''; + } } handleLine(line) { diff --git a/server/atlas/atlasProposals.js b/server/atlas/atlasProposals.js index e89ce3af..bd89dad5 100644 --- a/server/atlas/atlasProposals.js +++ b/server/atlas/atlasProposals.js @@ -21,8 +21,10 @@ function load() { } function save(proposals) { - fs.mkdirSync(path.dirname(proposalsPath()), { recursive: true }); - fs.writeFileSync(proposalsPath(), `${JSON.stringify({ proposals }, null, 2)}\n`, 'utf8'); + // Route through store.writeJson for the same atomic write-then-rename the + // registry uses — this is one file for every pending proposal, so a truncating + // half-write here loses the whole review queue. + store.writeJson(proposalsPath(), { proposals }); return proposals; } diff --git a/server/supervisor/supervisorSignals.js b/server/supervisor/supervisorSignals.js index 64fd6b19..ab406e62 100644 --- a/server/supervisor/supervisorSignals.js +++ b/server/supervisor/supervisorSignals.js @@ -217,7 +217,7 @@ async function gatherSignals({ lastLine: lastNonEmptyLines(tail, 1)[0] || '', repeatedLineCount: maxLineRepeat(tail), git, - tier: Number(record?.tier) || null, + tier: Number.isFinite(Number(record?.tier)) ? Number(record.tier) : null, ticketTitle: record?.ticketTitle || null }; diff --git a/tests/unit/appServerClientLifecycle.test.js b/tests/unit/appServerClientLifecycle.test.js index 277399d6..de40c9a3 100644 --- a/tests/unit/appServerClientLifecycle.test.js +++ b/tests/unit/appServerClientLifecycle.test.js @@ -68,6 +68,20 @@ describe('AppServerClient lifecycle', () => { expect(mockSpawn).toHaveBeenCalledTimes(2); }); + test('an oversized trailing line is dropped without discarding complete frames ahead of it', () => { + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + const seen = []; + client.on('notification', (n) => seen.push(n.method)); + + // A complete frame, then an unterminated 9MB line (no newline). + client.consume('{"method":"turn/completed","params":{}}\n'); + client.consume(`{"method":"x","params":{"blob":"${'a'.repeat(9 * 1024 * 1024)}`); + + // The complete frame was handled; the oversized incomplete line was dropped. + expect(seen).toEqual(['turn/completed']); + expect(client.buffer).toBe(''); + }); + test('a spawn failure resolves cleanly and leaves start() retryable', async () => { mockSpawn.mockImplementationOnce(() => { throw new Error('ENOENT'); }); const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); From e6580328d2b0721bf536de8ed8f2d08d97d3f55f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 21:25:00 +1000 Subject: [PATCH 30/69] docs: design doc ladder matches the code, not an earlier draft Section 2 still said observe was the default and that assist could not run commands; section 5's table said autonomy observe / 8 conditions / 709 tests. The code ships autopilot, assist runs the named resolve handlers, the table has 10 conditions, and the suite is at 829 after the review pass. Say what is true. --- ...OMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md | 24 ++++++++++++------- 1 file changed, 15 insertions(+), 9 deletions(-) diff --git a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md index e62eb289..3606f061 100644 --- a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md +++ b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md @@ -87,15 +87,20 @@ The missing piece is a **push-based supervisor**: something that continuously cl Every condition resolves to one rung, and the rung is capped by a global autonomy level: ``` -observe → notify → nudge → act → escalate(human) - ↑ ↑ - autonomy: assist always available +observe → resolve (self-heal) → delegate (Commander) → interrupt(human) + ↑ assist ↑ autopilot ↑ gated by the + interruption budget, + not by autonomy ``` - `off` — nothing runs. -- `observe` — findings recorded and visible; zero side effects. **Default.** Run it for a week and read the log before you let it touch anything. -- `assist` — may notify, speak, and nudge (text into a session). Cannot run commands. -- `autopilot` — may also take listed `act` steps. +- `observe` — findings recorded and visible; zero side effects. +- `assist` — may run the named resolve handlers itself (nudge text, answer a safe + permission prompt, relaunch an agent, schedule a resume, `gh pr create`). +- `autopilot` — may also hand a written problem brief to the Commander when a rule + can't fix something. **Shipped default** (originally `observe`; promoted after the + fix-first redesign — reaching a human is rate-limited separately by the + interruption budget, so acting is not the same as interrupting). Hard invariants regardless of level: - **Never** auto-act on anything matching the scheduler's blocked-command patterns (merge, approve, stop-session, remove-worktree, destroy). @@ -197,12 +202,13 @@ Surfaces shipped: standalone CLI (`scripts/atlas.js`, no server required — sym ## 5. What shipped (PR #1029) -All three, on `feature/autopilot-voice-and-repo-atlas`. 709 unit tests green (was 652). +All three, on `feature/autopilot-voice-and-repo-atlas`. 829 unit tests green (652 on main; +815 at first ship, +14 from the review pass). | Piece | Where | State | |---|---|---| -| Supervisor loop | `server/supervisorService.js`, `server/supervisor/*` | Running, autonomy `observe` | -| Condition table | `config/supervisor-rules.json` | 8 conditions, none reaching `act` | +| Supervisor loop | `server/supervisorService.js`, `server/supervisor/*` | Running, autonomy `autopilot` | +| Condition table | `config/supervisor-rules.json` | 10 conditions, all with a self-heal or observe path | | Speech out | `server/speechService.js`, `client/speech-output.js` | Browser backend active | | Free-form voice | `server/voiceCommandService.js` (`setCommanderForwarder`) | Wired to Commander | | Repo Atlas | `server/repoAtlasService.js`, `server/atlas/*`, `scripts/atlas.js` | 233 repos mapped, 25 cloned | From 6d657bd46e4f0df328eeefc1292354087508a603 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 21:25:44 +1000 Subject: [PATCH 31/69] =?UTF-8?q?docs:=20codebase=20map=20=E2=80=94=20corr?= =?UTF-8?q?ect=20autonomy=20default,=20complete=20test-file=20listing?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- CODEBASE_DOCUMENTATION.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index 4e21d810..b281f4e0 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -143,8 +143,8 @@ server/supervisor/supervisorSignals.js - Per-session signal collection (PTY tail server/supervisor/supervisorRules.js - Condition table loader/matcher (`config/supervisor-rules.json`, override `~/.agent-workspace/supervisor-rules.json`) + autonomy ceilings server/supervisor/supervisorActions.js - Ladder executor: notify/nudge/act, two-write submit, fail-closed permission-prompt classification server/routes/supervisorRoutes.js - Express router for `/api/supervisor/*` -config/supervisor-rules.json - Shipped condition table (autonomy `observe`, permission allow/deny patterns, act-handler allowlist) -tests/unit/supervisorRules.test.js, supervisorActions.test.js, supervisorService.test.js - Supervisor coverage (matching, autonomy ceilings, cooldowns, fail-closed approvals, audit) +config/supervisor-rules.json - Shipped condition table (autonomy `autopilot`, permission allow/deny patterns incl. exec-on-next-op path denials, act-handler allowlist) +tests/unit/supervisorRules.test.js, supervisorActions.test.js, supervisorService.test.js, supervisorUrgency.test.js - Supervisor coverage (matching, autonomy ceilings, cooldowns, fail-closed approvals, interruption budget, audit) server/speechService.js - Speech output with degrading backends ├─ Default `browser` backend emits a `speech-speak` socket event — works on a fresh clone with nothing installed @@ -167,7 +167,7 @@ server/atlas/atlasProposals.js - Write-back queue: agents propose highlights client/jarvis-panel.js - Alt+J panel: what was handled, what needs you, untracked chat work, atlas proposals + search client/realtime-voice.js - Browser side of the realtime loop (`window.RealtimeVoice`) client/styles/jarvis.css - JARVIS panel styling -tests/unit/appServerService.test.js, repoAtlasProposals.test.js - App-server framing/signal mapping, write-back approval flow +tests/unit/appServerService.test.js, appServerClientLifecycle.test.js, repoAtlasProposals.test.js - App-server framing/signal mapping, child-process lifecycle (crash/restart/overflow), write-back approval flow server/supervisor/supervisorUrgency.js - Urgency scoring (severity x task tier + failed-repair weight), interruption budget, digest queue server/discordWatchService.js - Ambient Discord watching: read the conversation, track the work, publish status back @@ -196,7 +196,7 @@ config/repo-atlas-topics.json - Canonical topic vocabulary + aliases config/repo-atlas.example.json - Annotated manifest example .repo-atlas.json - This repo's own manifest skills/public/repo-atlas/SKILL.md - Agent skill: query prior art instead of grepping the filesystem -tests/unit/repoAtlasSchema.test.js, repoAtlasQuery.test.js, repoAtlasCompiler.test.js, repoAtlasService.test.js - Atlas coverage (merge precedence, quality floors, sharing decisions, redaction) +tests/unit/repoAtlasSchema.test.js, repoAtlasQuery.test.js, repoAtlasCompiler.test.js, repoAtlasService.test.js, repoAtlasSync.test.js - Atlas coverage (merge precedence, quality floors, sharing decisions, redaction, real-git multi-machine sync) scripts/tauri/prepare-backend-resources.js - Tauri backend packager ├─ Bundles: server/client/config/templates/scripts + optional Node runtime into `src-tauri/resources/backend` ├─ Commander instructions: copies `docs/COMMANDER_CLAUDE.md` into `resources/backend/{COMMANDER_CLAUDE.md,CLAUDE.md,AGENTS.md}` for desktop builds From fefe095f55317a205bb52a43bac1e3f2db508f35 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 22:53:33 +1000 Subject: [PATCH 32/69] fix(atlas): sync reports the real commit/fetch failure, not a downstream one MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by the live end-to-end sweep: on a machine where the local commit fails (no git identity in this repro; a rejecting hook or full disk equally), sync carried on and reported 'push failed: src refspec HEAD does not match any' — pointing at entirely the wrong cause. Stop at the failed step and say what actually happened; a failed fetch likewise reports 'could not reach the registry remote' instead of decaying into a bogus pull/push error. Co-Authored-By: Claude Fable 5 --- server/atlas/atlasSync.js | 9 +++++++++ tests/unit/repoAtlasSync.test.js | 21 +++++++++++++++++++++ 2 files changed, 30 insertions(+) diff --git a/server/atlas/atlasSync.js b/server/atlas/atlasSync.js index 12319480..7b0cb15b 100644 --- a/server/atlas/atlasSync.js +++ b/server/atlas/atlasSync.js @@ -110,9 +110,18 @@ async function syncRegistry({ remote = '', message = '' } = {}) { const localCommit = await commitAll(dir, message || `atlas: sync from ${require('os').hostname()}`); steps.push({ step: 'commit-local', ...localCommit }); + // A failed commit (identity unset, rejecting hook, disk full) must stop the + // sync here — carrying on used to surface as a baffling "push failed: src + // refspec HEAD does not match any" that pointed at entirely the wrong thing. + if (localCommit.error) { + return { ok: false, dir, steps, error: `local commit failed: ${localCommit.error}` }; + } const fetched = await git(['fetch', 'origin'], { cwd: dir }); steps.push({ step: 'fetch', ok: fetched.ok, detail: fetched.stderr || null }); + if (!fetched.ok) { + return { ok: false, dir, steps, error: `could not reach the registry remote: ${fetched.stderr || fetched.error}` }; + } const branch = (await git(['rev-parse', '--abbrev-ref', 'HEAD'], { cwd: dir })).stdout || 'main'; const remoteExists = (await git(['rev-parse', '--verify', `origin/${branch}`], { cwd: dir })).ok; diff --git a/tests/unit/repoAtlasSync.test.js b/tests/unit/repoAtlasSync.test.js index 8d7b4d13..5d156aa7 100644 --- a/tests/unit/repoAtlasSync.test.js +++ b/tests/unit/repoAtlasSync.test.js @@ -58,6 +58,27 @@ describe('Repo Atlas multi-machine sync', () => { expect(b.getEntry('only-on-a')).toBeNull(); }); + test('a failing local commit reports itself, not a misleading push error', async () => { + const a = machine('a'); + await a.setRemote(remote); + a.addHighlight('repo-one', { topic: 'testing', quality: 5 }); + expect((await a.sync()).ok).toBe(true); + + // A rejecting pre-commit hook stands in for any real commit failure + // (identity unset, disk full). The sync must stop there and say so. + const hookDir = path.join(store.registryDir(), '.git', 'hooks'); + fs.mkdirSync(hookDir, { recursive: true }); + const hook = path.join(hookDir, 'pre-commit'); + fs.writeFileSync(hook, '#!/bin/sh\necho "hook says no" >&2\nexit 1\n'); + fs.chmodSync(hook, 0o755); + + a.addHighlight('repo-one', { topic: 'testing', quality: 4, notes: 'changed' }); + const result = await a.sync(); + expect(result.ok).toBe(false); + expect(result.error).toMatch(/local commit failed/); + expect(result.error).not.toMatch(/push failed/); + }); + test('two machines editing different repos merge without conflict', async () => { const a = machine('a'); await a.setRemote(remote); From 42b453ce7d7ae22388729cc23651a16a63553fb3 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 22:57:21 +1000 Subject: [PATCH 33/69] fix(codex): stale-child race on quick restart + re-handshake on every spawn Found by driving the real API end-to-end: POST /api/app-server/stop followed by /start left the bridge answering 'Not initialized' forever. Two causes: 1. A SIGTERM'd child's exit event lands on a later tick. Its handler ran against the CURRENT state, so it rejected the NEW child's pending initialize, nulled the new child reference, and could schedule a spurious extra respawn. Handlers are now bound to their own child and bail once replaced; the stream buffer resets per spawn so a stale partial line can't corrupt the new frame. 2. The initialize handshake only ran inside service.start(), so any client auto-restart came back running but never initialized. The client now emits 'started' on every spawn and the service re-handshakes on it (single-flight, serverInfo nulled on exit/stop since it belongs to the connection). Co-Authored-By: Claude Fable 5 --- server/agents/appServerClient.js | 33 +++++++++++---- server/appServerService.js | 45 ++++++++++++++++----- tests/unit/appServerClientLifecycle.test.js | 36 +++++++++++++++++ 3 files changed, 97 insertions(+), 17 deletions(-) diff --git a/server/agents/appServerClient.js b/server/agents/appServerClient.js index 4aabc082..3ca20df8 100644 --- a/server/agents/appServerClient.js +++ b/server/agents/appServerClient.js @@ -56,8 +56,9 @@ class AppServerClient extends EventEmitter { this.stopped = false; const startPromise = new Promise((resolve) => { + let child; try { - this.child = spawn(this.command, this.args, { + child = spawn(this.command, this.args, { ...getHiddenProcessOptions({ stdio: ['pipe', 'pipe', 'pipe'] }), cwd: this.cwd || undefined, env: augmentProcessEnv(process.env) @@ -68,20 +69,34 @@ class AppServerClient extends EventEmitter { return; } - this.child.stdout.setEncoding('utf8'); - this.child.stdout.on('data', (chunk) => this.consume(chunk)); - this.child.stderr.setEncoding('utf8'); - this.child.stderr.on('data', (chunk) => { + this.child = child; + // A partial line left over from a previous process must not prefix the + // new stream — it would corrupt the first frame the new child sends. + this.buffer = ''; + + // Every handler below is bound to THIS child and bails if a newer one + // has replaced it. A SIGTERM'd child's 'exit' event arrives on a later + // tick — without the guard, a quick stop()+start() let the OLD child's + // exit reject the NEW child's pending requests (killing the initialize + // handshake), null the new child, and schedule a spurious extra respawn. + child.stdout.setEncoding('utf8'); + child.stdout.on('data', (chunk) => { + if (this.child === child) this.consume(chunk); + }); + child.stderr.setEncoding('utf8'); + child.stderr.on('data', (chunk) => { const text = String(chunk).trim(); if (text) this.logger.debug?.('[app-server]', text); }); - this.child.on('error', (error) => { + child.on('error', (error) => { + if (this.child !== child) return; this.lastError = error.message; this.emit('error', error); }); - this.child.on('exit', (code, signal) => { + child.on('exit', (code, signal) => { + if (this.child !== child) return; this.rejectAllPending(new Error(`app-server exited (code ${code}, signal ${signal})`)); this.child = null; this.emit('exit', { code, signal }); @@ -90,7 +105,8 @@ class AppServerClient extends EventEmitter { this.startedAt = new Date().toISOString(); this.restartAttempts = 0; - resolve({ running: true, pid: this.child.pid }); + this.emit('started', { pid: child.pid }); + resolve({ running: true, pid: child.pid }); }); // Clear the in-flight marker once it settles. The Promise executor runs @@ -116,6 +132,7 @@ class AppServerClient extends EventEmitter { stop() { this.stopped = true; + this.buffer = ''; this.rejectAllPending(new Error('app-server client stopped')); if (this.child) { try { diff --git a/server/appServerService.js b/server/appServerService.js index eaff3bc4..97aa0dcb 100644 --- a/server/appServerService.js +++ b/server/appServerService.js @@ -55,9 +55,41 @@ class AppServerService { this.speechService = speechService || this.speechService; this.signals.bind(); this.bindRealtime(); + if (!this.handshakeBound) { + this.handshakeBound = true; + // The protocol handshake belongs to a CONNECTION, not to the service: + // every spawned process needs its own initialize, including the ones the + // client's auto-restart brings up — otherwise the bridge comes back + // running but permanently answers "Not initialized". + this.client.on('started', () => { + this.serverInfo = null; + this.handshake().catch(() => {}); + }); + this.client.on('exit', () => { + this.serverInfo = null; + }); + } return this; } + async handshake() { + if (this.handshaking) return this.handshaking; + this.handshaking = (async () => { + try { + this.serverInfo = await this.client.request('initialize', { + clientInfo: { name: 'agent-workspace', version: require('../package.json').version || '1.0.0' } + }, { timeoutMs: 15_000 }); + } catch (error) { + this.logger.warn?.('app-server initialize failed', { error: error.message }); + this.serverInfo = null; + } finally { + this.handshaking = null; + } + return this.serverInfo; + })(); + return this.handshaking; + } + bindRealtime() { if (this.realtimeBound || !this.client) return; this.realtimeBound = true; @@ -117,21 +149,16 @@ class AppServerService { this.bindRealtime(); // The protocol requires an initialize handshake before anything else; the - // server answers with codexHome and platform details. - try { - this.serverInfo = await this.client.request('initialize', { - clientInfo: { name: 'agent-workspace', version: require('../package.json').version || '1.0.0' } - }, { timeoutMs: 15_000 }); - } catch (error) { - this.logger.warn?.('app-server initialize failed', { error: error.message }); - this.serverInfo = null; - } + // 'started' listener bound in init() also fired it, so this await mostly + // just surfaces the result — awaiting twice is harmless (idempotent call). + if (!this.serverInfo) await this.handshake(); return { ...started, initialized: Boolean(this.serverInfo) }; } stop() { this.realtimeThreads.clear(); + this.serverInfo = null; return this.client.stop(); } diff --git a/tests/unit/appServerClientLifecycle.test.js b/tests/unit/appServerClientLifecycle.test.js index de40c9a3..c0510a6d 100644 --- a/tests/unit/appServerClientLifecycle.test.js +++ b/tests/unit/appServerClientLifecycle.test.js @@ -68,6 +68,42 @@ describe('AppServerClient lifecycle', () => { expect(mockSpawn).toHaveBeenCalledTimes(2); }); + test('a stopped child exiting late cannot clobber its replacement', async () => { + const first = makeChild(1001); + const second = makeChild(1002); + mockSpawn.mockReturnValueOnce(first).mockReturnValueOnce(second); + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + + await client.start(); + client.stop(); + await client.start(); + expect(client.child).toBe(second); + + // The new child has a request in flight when the OLD child's SIGTERM'd + // exit event finally lands (it always arrives on a later tick). + const pending = client.request('initialize', {}); + first.emit('exit', 0, null); + + // The new child and its pending request must be untouched. + expect(client.child).toBe(second); + expect(client.pending.size).toBe(1); + second.emit('data-noop'); + client.consume('{"id":1,"result":{"ok":true}}\n'); + await expect(pending).resolves.toEqual({ ok: true }); + }); + + test('every successful spawn emits "started" so the handshake can re-run', async () => { + mockSpawn.mockImplementation(() => makeChild()); + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + const started = []; + client.on('started', (info) => started.push(info.pid)); + + await client.start(); + client.child.emit('exit', 1, null); + await client.start(); + expect(started).toHaveLength(2); + }); + test('an oversized trailing line is dropped without discarding complete frames ahead of it', () => { const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); const seen = []; From 390693548f2aa0617bbbd7d86b944579c74664a7 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 23:03:08 +1000 Subject: [PATCH 34/69] fix(codex): approvals with numeric ids are answerable over the API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by driving a real approval loop live: the first JSON-RPC request id a real app-server sends is the NUMBER 0, but an Express path param is the string '0' — the Map lookup missed and every wire answer failed 'no pending approval'. Key the pending map by String(id); the wire response still carries the original id with its exact type, since JSON-RPC ids must match exactly. Co-Authored-By: Claude Fable 5 --- server/agents/appServerSignals.js | 13 +++++++++---- tests/unit/appServerService.test.js | 14 ++++++++++++++ 2 files changed, 23 insertions(+), 4 deletions(-) diff --git a/server/agents/appServerSignals.js b/server/agents/appServerSignals.js index 0fef08bd..04560a37 100644 --- a/server/agents/appServerSignals.js +++ b/server/agents/appServerSignals.js @@ -159,7 +159,11 @@ class AppServerSignalSource extends EventEmitter { requestedAt: Date.now() }; - this.pendingApprovals.set(id, entry); + // Keyed by String(id): JSON-RPC ids are numbers (the first is literally 0) + // but an HTTP route param arrives as a string, and a Map lookup with the + // wrong type silently misses. The entry keeps the ORIGINAL id because the + // response on the wire must carry it back with its exact type. + this.pendingApprovals.set(String(id), entry); if (state) { state.status = 'waiting'; state.activeFlags = [ACTIVE_FLAGS.WAITING_ON_APPROVAL]; @@ -173,11 +177,12 @@ class AppServerSignalSource extends EventEmitter { * can only type a keystroke at whatever prompt happens to be showing. */ answerApproval(requestId, approved, { note = '' } = {}) { - const entry = this.pendingApprovals.get(requestId); + const key = String(requestId); + const entry = this.pendingApprovals.get(key); if (!entry) return { ok: false, error: `no pending approval "${requestId}"` }; - const sent = this.client?.respond(requestId, { decision: approved ? 'approved' : 'denied', note }); - this.pendingApprovals.delete(requestId); + const sent = this.client?.respond(entry.requestId, { decision: approved ? 'approved' : 'denied', note }); + this.pendingApprovals.delete(key); const state = this.threads.get(entry.threadId); if (state && approved) { diff --git a/tests/unit/appServerService.test.js b/tests/unit/appServerService.test.js index 367102e5..a205afdf 100644 --- a/tests/unit/appServerService.test.js +++ b/tests/unit/appServerService.test.js @@ -138,6 +138,20 @@ describe('AppServerSignalSource', () => { expect(signals.answerApproval('nope', true).ok).toBe(false); }); + test('a numeric request id 0 is answerable via the string an HTTP route delivers', () => { + const { client, signals } = source(); + // The first JSON-RPC request a real app-server sends has id 0 (a number); + // an Express path param arrives as the string "0". + client.emitRequest(0, 'item/commandExecution/requestApproval', { threadId: 't1', command: 'printf ok' }); + expect(signals.listPendingApprovals()).toHaveLength(1); + + const result = signals.answerApproval('0', true); + expect(result.ok).toBe(true); + // The response on the wire must carry the ORIGINAL numeric id back. + expect(client.responses[0].id).toBe(0); + expect(signals.listPendingApprovals()).toEqual([]); + }); + test('a closed thread stops producing signals', () => { const { client, signals } = source(); client.emitNotification('thread/status/changed', { threadId: 't1', status: { type: 'idle' } }); From 39661fa1f70fd3b02a28c28ee9708c6bf1ff552d Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Sun, 26 Jul 2026 23:07:44 +1000 Subject: [PATCH 35/69] fix(atlas): machine-local config must not sync with the registry MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by a real two-machine end-to-end run: atlas.config.json (remote URL, audiences with machine-local output paths) lived inside the synced registry, so any two machines whose configs differ add/add-conflicted on B's very first sync — 'Registry pull conflicted. Resolve it by hand'. The existing multi-machine tests passed only because both fake machines wrote byte-identical configs. Ignore the config in the registry .gitignore, upgrade pre-fix .gitignores in place, and untrack an already-committed config on the next sync (--cached, so the local file survives). Regression tests drive the exact live failure plus the self-heal path through real git. Co-Authored-By: Claude Fable 5 --- server/atlas/atlasSync.js | 17 ++++++++++++- tests/unit/repoAtlasSync.test.js | 42 ++++++++++++++++++++++++++++++++ 2 files changed, 58 insertions(+), 1 deletion(-) diff --git a/server/atlas/atlasSync.js b/server/atlas/atlasSync.js index 7b0cb15b..d312b06a 100644 --- a/server/atlas/atlasSync.js +++ b/server/atlas/atlasSync.js @@ -25,6 +25,9 @@ discovery.json bundles/ subscriptions/ *.migrated +# Machine-local config: remote URL, audiences with local output paths. Syncing +# it would add/add-conflict between any two machines whose configs differ. +atlas.config.json `; function git(args, { cwd, timeout = GIT_TIMEOUT_MS } = {}) { @@ -69,7 +72,13 @@ function seedRepoFiles(dir) { const readme = path.join(dir, 'README.md'); if (!fs.existsSync(readme)) fs.writeFileSync(readme, REGISTRY_README, 'utf8'); const ignore = path.join(dir, '.gitignore'); - if (!fs.existsSync(ignore)) fs.writeFileSync(ignore, REGISTRY_GITIGNORE, 'utf8'); + if (!fs.existsSync(ignore)) { + fs.writeFileSync(ignore, REGISTRY_GITIGNORE, 'utf8'); + } else if (!fs.readFileSync(ignore, 'utf8').includes('atlas.config.json')) { + // Registries created before the config was ignored: upgrade in place so + // every machine converges on the same tracked .gitignore content. + fs.writeFileSync(ignore, REGISTRY_GITIGNORE, 'utf8'); + } fs.mkdirSync(path.join(dir, 'entries'), { recursive: true }); } @@ -106,6 +115,12 @@ async function syncRegistry({ remote = '', message = '' } = {}) { if (!prepared.ok) return { ok: false, error: prepared.error }; seedRepoFiles(dir); + // Self-heal registries that committed the machine-local config before it was + // ignored — leaving it tracked guarantees an add/add conflict with any other + // machine. --cached keeps the local file; --ignore-unmatch makes it a no-op + // once clean. + await git(['rm', '--cached', '--ignore-unmatch', '-q', 'atlas.config.json'], { cwd: dir }); + const steps = []; const localCommit = await commitAll(dir, message || `atlas: sync from ${require('os').hostname()}`); diff --git a/tests/unit/repoAtlasSync.test.js b/tests/unit/repoAtlasSync.test.js index 5d156aa7..e7348c2f 100644 --- a/tests/unit/repoAtlasSync.test.js +++ b/tests/unit/repoAtlasSync.test.js @@ -58,6 +58,48 @@ describe('Repo Atlas multi-machine sync', () => { expect(b.getEntry('only-on-a')).toBeNull(); }); + test('machine-local config never syncs, so differing configs cannot conflict', async () => { + // The live repro: A has an audience configured, B does not — their + // atlas.config.json contents differ, and when the config was tracked this + // add/add-conflicted on B's very first sync. + const a = machine('a'); + await a.setRemote(remote); + a.setAudience({ id: 'core-team', outputPath: '/machine/a/only/path.json' }); + a.addHighlight('repo-one', { topic: 'testing', quality: 5 }); + expect((await a.sync()).ok).toBe(true); + + const b = machine('b'); + await b.setRemote(remote); + const bSync = await b.sync(); + expect(bSync.ok).toBe(true); + expect(b.find('testing')).toHaveLength(1); + + // The config file must not exist anywhere in the shared history. + const tracked = execFileSync('git', ['ls-tree', '-r', '--name-only', 'main'], { cwd: remote, stdio: 'pipe' }).toString(); + expect(tracked).not.toMatch(/atlas\.config\.json/); + + // And B must not have inherited A's machine-local audience config. + expect(b.listAudiences().find((x) => x.id === 'core-team')).toBeUndefined(); + }); + + test('a registry that already committed its config self-heals on the next sync', async () => { + const a = machine('a'); + await a.setRemote(remote); + a.addHighlight('repo-one', { topic: 'testing', quality: 5 }); + expect((await a.sync()).ok).toBe(true); + + // Simulate the pre-fix state: force-track the config and commit it. + const dir = store.registryDir(); + execFileSync('git', ['add', '-f', 'atlas.config.json'], { cwd: dir, stdio: 'pipe' }); + execFileSync('git', ['commit', '-m', 'legacy: tracked config'], { cwd: dir, stdio: 'pipe' }); + + expect((await a.sync()).ok).toBe(true); + const tracked = execFileSync('git', ['ls-tree', '-r', '--name-only', 'main'], { cwd: remote, stdio: 'pipe' }).toString(); + expect(tracked).not.toMatch(/atlas\.config\.json/); + // The local file itself must survive (--cached). + expect(fs.existsSync(store.configPath())).toBe(true); + }); + test('a failing local commit reports itself, not a misleading push error', async () => { const a = machine('a'); await a.setRemote(remote); From 239fb9ebec4fbd828336f72b4d024993a70a16c8 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 07:45:04 +1000 Subject: [PATCH 36/69] fix(ui): clamp future timestamps so relative time never shows negative 'ago' Surfaced while recording the UI test video: a message whose timestamp is ahead of local time (clock skew between machines, or a client clock running fast) rendered as '-40717s ago'. Floor the elapsed seconds at 0. --- client/jarvis-panel.js | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/client/jarvis-panel.js b/client/jarvis-panel.js index 51b3895e..4b3750aa 100644 --- a/client/jarvis-panel.js +++ b/client/jarvis-panel.js @@ -33,7 +33,9 @@ const relativeTime = (iso) => { const ms = Date.parse(String(iso || '')); if (!Number.isFinite(ms)) return ''; - const seconds = Math.round((Date.now() - ms) / 1000); + // A future timestamp (clock skew between machines, or a message posted by a + // client whose clock is ahead) must not render as "-40717s ago". + const seconds = Math.max(0, Math.round((Date.now() - ms) / 1000)); if (seconds < 60) return `${seconds}s ago`; if (seconds < 3600) return `${Math.round(seconds / 60)}m ago`; if (seconds < 86400) return `${Math.round(seconds / 3600)}h ago`; From ee9538451d6648d3d387873863b80570c8d1251b Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 08:49:40 +1000 Subject: [PATCH 37/69] fix(voice): make the live speech transcript actually readable MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The transcript of what you just said rendered at 11px gray (#718096) on the dark header — the exact low-contrast gray-on-dark the project's UI rules forbid, truncated at 200px. Bump to 14px bold, show it as a bright cyan pill (only when non-empty) so you can see what was recognized. --- client/styles.css | 16 +++++++++++++--- 1 file changed, 13 insertions(+), 3 deletions(-) diff --git a/client/styles.css b/client/styles.css index 34dc342f..1fd4da11 100644 --- a/client/styles.css +++ b/client/styles.css @@ -11017,14 +11017,24 @@ body.dependency-onboarding-active #dependency-setup-modal { } .voice-transcript { - font-size: 11px; - color: #718096; - max-width: 200px; + font-size: 14px; + font-weight: 600; + color: #ffffff; + max-width: 340px; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; } +/* Only show the pill when there is something to read. */ +.voice-transcript:not(:empty) { + padding: 4px 10px; + border-radius: 999px; + background: #1a212b; + border: 1px solid #2f7a4c; + color: #a8d8f0; +} + /* Voice Backend Menu */ .voice-backend-menu { font-size: 13px; From e2723af0073f0c7563989fee7c0ef3e479147b5f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 08:52:52 +1000 Subject: [PATCH 38/69] docs: research local full-duplex voice models + swappable-provider plan PersonaPlex / Qwen3.5-Omni for full-duplex on the 5090, Parakeet+Kokoro pipeline for the swappable local lane, mapped onto the existing speechService/whisperService backends and a codex-style voice-provider registry. --- .../2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md | 92 +++++++++++++++++++ 1 file changed, 92 insertions(+) create mode 100644 PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md diff --git a/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md b/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md new file mode 100644 index 00000000..7a4c604e --- /dev/null +++ b/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md @@ -0,0 +1,92 @@ +# Local real-time voice models — research + integration plan (2026-07-27) + +Goal: full-duplex, back-and-forth, **local** voice for JARVIS, with models swappable at +runtime. Target hardware: **RTX 5090, 32 GB** (the real deployment) and an **RTX 3080 +Laptop, 16 GB** (for testing now). + +The strongest turnkey options actually landed Jan–Mar 2026, not June/July — the mid-2026 +items (DyaPlex, ASPIRin, SoulX-Duplug) are research papers, not shippable servers yet. So +the recommendation below is built on what you can run this week. + +## Two architectures, pick per use + +**A. Single full-duplex "speech model"** — listens and talks at the same time, real +barge-in, ~persona/voice control. No reasoning, no tools. This is the "feels like talking +to a person" layer. + +**B. Pipeline (STT → LLM/agent → TTS)** — swappable parts, the LLM can be your existing +Claude/Codex agent, so it can actually *do things*. Higher latency than A, but this is how +JARVIS gets both a voice AND the whole orchestrator API. Matches the pluggable backends the +codebase already has (`speechService`, `whisperService`). + +The right answer is **both**: A for ambient chat, B when the utterance needs to become work. + +## Recommended models + +### Full-duplex (architecture A) + +| Model | Params | VRAM | Latency | License | Notes | +|---|---|---|---|---|---| +| **NVIDIA PersonaPlex** ⭐ | 7B | ~19 GB BF16 (RTX 3090); 8–16 GB quantized (moshi.cpp) | ~70 ms speaker-switch, ~170–257 ms response | Open weights (HF + `NVIDIA/personaplex`) | Built on Moshi + Mimi codec. Persona/voice control via prompt + sample. Local PyTorch web UI at `localhost:8998`, self-signed SSL for the mic. Limits: 4-min context window, repetition loops, **no tools/websearch/delegation**. | +| **Moshi** (Kyutai) | 7B | ~16 GB (moshi.cpp for less) | ~200 ms | Open | The foundation PersonaPlex fine-tunes. Use PersonaPlex unless you want the base. | + +→ **On the 5090:** run PersonaPlex at full BF16 (~19 GB) — best latency, true full-duplex. +→ **On the 16 GB laptop:** PersonaPlex quantized via moshi.cpp fits and is testable now. + +### Omni (architecture A½ — streaming speech out + reasoning, not true barge-in) + +| Model | Params | VRAM | License | Notes | +|---|---|---|---|---| +| **Qwen3.5-Omni-30B-A3B** ⭐ | 30B MoE / 3B active | ~24 GB at INT4 (fits the 5090) | Apache 2.0 | Thinker (text) + Talker (streaming audio) = text **and** speech from one pass, no separate TTS. Real reasoning. vLLM ≥ 0.17. This is the one that can voice AND think/act. | + +→ **On the 5090:** Qwen3.5-Omni at INT4 is the sweet spot if you want one model that reasons +and speaks. Won't fit the laptop comfortably. + +### Pipeline parts (architecture B — the swappable lane) + +- **STT (streaming):** **NVIDIA Parakeet TDT** (RNN-T, streaming, RTFx > 2000 — extremely + low latency) is the pick; Distil-Whisper or Moonshine (edge) as lighter fallbacks. Both + are tiny next to the LLM, so they run alongside anything on the 5090 and fine on the laptop. +- **TTS (streaming):** **Kokoro-82M** (Apache 2.0, tiny, fast — great default) or + **Chatterbox** (Resemble, real-time, permissive, voice cloning). **CosyVoice2-0.5B** / + **Fish S2 Pro** (sub-100 ms on vLLM) if you want the lowest latency. All run on the laptop. + +## How this plugs into JARVIS (the swappable design you asked for) + +The codebase is already 80% there — don't bolt on, extend the seams that exist: + +1. **`speechService` already has pluggable TTS backends** (browser/piper/say/SAPI/espeak). + Add `kokoro` and `chatterbox` as two more backends behind the same interface. Swap with + the existing `POST /api/speech/backend`. +2. **`whisperService` already abstracts STT.** Add a `parakeet` backend next to whisper.cpp + /openai-whisper. Same pattern. +3. **Full-duplex is a *realtime provider*, not a backend.** The PR already speaks a realtime + protocol (`thread/realtime/*`, currently Codex app-server). Model it as a **voice-provider + registry** exactly like the existing agent-provider registry: each provider declares + `{ stt, tts, duplex }` capability and a transport. `CODEX_APP_SERVER`-style env/flags pick + the active one: + - `codex` — remote Codex realtime (what ships today) + - `personaplex` — local full-duplex server at `localhost:8998` + - `pipeline` — parakeet + + kokoro + The `getSignalForSession`-returns-null degradation pattern already in the supervisor is the + same idea: "use the best available, fall back cleanly." +4. **Config-driven, hot-swappable:** one `config/voice-providers.json` (like + `config/custom-agents.example.json`) so adding a model is data, not code — which is the + "swap out voice models as needed" requirement, met the same way the agent registry meets it. + +## Suggested first step (cheap, high-signal) + +Wire the **pipeline** lane first (Parakeet STT backend + Kokoro TTS backend into the existing +services): it's the least risky, runs on the laptop today, makes the whole thing local, and +proves the swappable-provider registry. Then add **PersonaPlex** as a `duplex` provider for +the 5090 once the registry exists. Qwen3.5-Omni is the stretch goal for a single +voice-and-reason model. + +## Sources +- https://research.nvidia.com/labs/adlr/personaplex · https://github.com/NVIDIA/personaplex +- https://www.makeuseof.com/nvidia-personaplex-local-speech-model-8gb-vram/ +- https://www.kunalganglani.com/blog/nvidia-personaplex-full-duplex-voice-ai +- https://github.com/QwenLM/Qwen3-Omni · https://huggingface.co/Qwen/Qwen3-Omni-30B-A3B-Instruct +- https://northflank.com/blog/best-open-source-speech-to-text-stt-model-in-2026-benchmarks (Parakeet/STT) +- https://localaimaster.com/blog/best-local-tts-models · https://bentoml.com/blog/exploring-the-world-of-open-source-text-to-speech-models (Kokoro/Chatterbox/TTS) +- https://arxiv.org/pdf/2505.15670 (SALM-Duplex) · https://arxiv.org/pdf/2502.13472 (FlexDuo) · https://arxiv.org/pdf/2603.14877 (SoulX-Duplug) From 7dcfff2f3b9f7c3f7dfabdbdb631047dd57d778b Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 09:17:08 +1000 Subject: [PATCH 39/69] feat(voice): swappable voice-model registry + local Piper TTS MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A model is now DATA (config/voice-providers.json), not code — adding one is a config entry. VoiceProviderService loads the registry, health-checks each provider (command present? model server reachable?), and resolves the ONE active provider per capability (tts/stt/duplex); 'auto' = best-quality available, a broken pin falls back to auto so voice never silently mutes, everything degrades when a model is absent. - speechService gains a Kokoro/generic-CLI TTS backend and auto-discovers a Piper voice under ~/.local/share/piper-voices, so the local neural voice works out of the box once piper + a voice are installed. - The registry drives what speaks: setActive('tts', …) applies to speechService immediately (verified live: browser<->piper swap over the API). - /api/voice-providers/* : list with health, swap active per capability, reload. - Registry documents every option (PersonaPlex/X-Talk duplex, Parakeet/faster- whisper/Moonshine STT, Kokoro/Chatterbox/Piper TTS) with install hints so better models drop in on the 5090 without code. Verified end to end on this box: Piper auto-selected and spoke locally over WSLg. 845 unit tests green. Co-Authored-By: Claude Fable 5 --- config/voice-providers.json | 143 +++++++++++++ server/index.js | 14 ++ server/routes/voiceProviderRoutes.js | 52 +++++ server/speechService.js | 72 ++++++- server/voice/voiceProviderService.js | 257 ++++++++++++++++++++++++ tests/unit/voiceProviderService.test.js | 122 +++++++++++ 6 files changed, 659 insertions(+), 1 deletion(-) create mode 100644 config/voice-providers.json create mode 100644 server/routes/voiceProviderRoutes.js create mode 100644 server/voice/voiceProviderService.js create mode 100644 tests/unit/voiceProviderService.test.js diff --git a/config/voice-providers.json b/config/voice-providers.json new file mode 100644 index 00000000..419a10b7 --- /dev/null +++ b/config/voice-providers.json @@ -0,0 +1,143 @@ +{ + "$comment": "Swappable voice models. A model is DATA here, not code — add one by adding an entry. `kind`: tts (speak) | stt (listen) | duplex (full-duplex speak+listen at once). `active*` picks the current provider per capability; 'auto' = best available that passes its health check; 'none' = off. Override per-machine at ~/.agent-workspace/voice-providers.json.", + "schemaVersion": 1, + + "activeTts": "auto", + "activeStt": "auto", + "activeDuplex": "none", + + "providers": [ + { + "id": "browser", + "kind": "tts", + "label": "Browser speech synthesis", + "engine": "browser", + "local": false, + "quality": 2, + "notes": "Zero install — the page speaks via the Web Speech API. Works on a fresh clone; the fallback when nothing local is set up." + }, + { + "id": "piper", + "kind": "tts", + "label": "Piper (local neural TTS)", + "engine": "piper", + "local": true, + "quality": 3, + "requires": { "command": "piper", "env": "PIPER_MODEL" }, + "install": "pip install --user piper-tts && download a voice .onnx (e.g. en_US-amy-medium) then set PIPER_MODEL to its path", + "notes": "Small, fast, fully offline. Good default local voice on modest hardware." + }, + { + "id": "kokoro", + "kind": "tts", + "label": "Kokoro-82M (local, Apache-2.0)", + "engine": "kokoro", + "local": true, + "quality": 4, + "requires": { "command": "kokoro-tts" }, + "install": "pip install --user kokoro-onnx (or the `kokoro` package); ~300MB model. Runs on CPU or GPU.", + "notes": "Best lightweight open TTS of 2026 — natural voice at tiny compute. Recommended local default once installed." + }, + { + "id": "chatterbox", + "kind": "tts", + "label": "Chatterbox (Resemble, real-time + cloning)", + "engine": "kokoro", + "local": true, + "quality": 4, + "requires": { "command": "chatterbox" }, + "install": "pip install --user chatterbox-tts; needs a GPU for real-time.", + "notes": "Real-time generative audio with voice cloning, permissive license. Uses the same generic CLI adapter as kokoro." + }, + { + "id": "espeak", + "kind": "tts", + "label": "espeak-ng (robotic, always-there fallback)", + "engine": "espeak", + "local": true, + "quality": 1, + "requires": { "command": "espeak-ng" }, + "notes": "Last-resort offline voice; robotic but never fails." + }, + + { + "id": "whisper-cpp", + "kind": "stt", + "label": "whisper.cpp (local, GPU-accelerated)", + "engine": "whisper", + "local": true, + "quality": 4, + "requires": { "env": "WHISPER_CPP_PATH" }, + "notes": "The existing STT path. Accurate, fully offline." + }, + { + "id": "faster-whisper", + "kind": "stt", + "label": "faster-whisper (CTranslate2)", + "engine": "whisper", + "local": true, + "quality": 4, + "requires": { "command": "faster-whisper" }, + "install": "pip install --user faster-whisper; runs on GPU or CPU.", + "notes": "Faster Whisper reimplementation. Drop-in for the whisper engine." + }, + { + "id": "parakeet", + "kind": "stt", + "label": "NVIDIA Parakeet TDT (streaming, ultra-low latency)", + "engine": "parakeet", + "local": true, + "quality": 5, + "requires": { "command": "parakeet-stt" }, + "install": "NVIDIA NeMo + parakeet-tdt checkpoint; best on the 5090. RNN-T, RTFx > 2000, true streaming.", + "notes": "The pick for the streaming pipeline lane. Lights up when NeMo + the checkpoint are present." + }, + { + "id": "moonshine", + "kind": "stt", + "label": "Moonshine (edge/CPU)", + "engine": "whisper", + "local": true, + "quality": 3, + "requires": { "command": "moonshine" }, + "notes": "Smallest footprint; good for the laptop or CPU-only." + }, + + { + "id": "codex", + "kind": "duplex", + "label": "Codex app-server realtime (remote)", + "engine": "codex", + "local": false, + "quality": 4, + "requires": { "env": "CODEX_APP_SERVER" }, + "notes": "The full-duplex path that ships today — browser STT -> appendText -> Codex thread -> spoken back. Reasoning + tools, remote." + }, + { + "id": "personaplex", + "kind": "duplex", + "label": "NVIDIA PersonaPlex (full-duplex, LOCAL)", + "engine": "personaplex", + "local": true, + "quality": 5, + "transport": "websocket", + "endpoint": "http://localhost:8998", + "requires": { "server": "http://localhost:8998" }, + "install": "git clone github.com/NVIDIA/personaplex; ~19GB BF16 (fits the 5090), 8-16GB quantized via moshi.cpp (fits the 16GB laptop). Serves a web UI at localhost:8998.", + "notes": "Best OPEN local full-duplex: ~70ms speaker-switch. Voice-only (no tools). Lights up when the server is reachable." + }, + { + "id": "xtalk", + "kind": "duplex", + "label": "X-Talk (open full-duplex framework)", + "engine": "personaplex", + "local": true, + "quality": 4, + "transport": "websocket", + "endpoint": "http://localhost:8080", + "requires": { "server": "http://localhost:8080" }, + "install": "Pure-Python cascaded full-duplex framework (STT+LLM+TTS with barge-in). Lighter than PersonaPlex.", + "notes": "Newer (mid-2026) open framework; swap the endpoint and it works through the same adapter." + } + ] +} diff --git a/server/index.js b/server/index.js index bb48dff4..22b96127 100644 --- a/server/index.js +++ b/server/index.js @@ -116,6 +116,8 @@ const { SupervisorService } = require('./supervisorService'); const { createSupervisorRoutes } = require('./routes/supervisorRoutes'); const { SpeechService } = require('./speechService'); const { createSpeechRoutes } = require('./routes/speechRoutes'); +const { VoiceProviderService } = require('./voice/voiceProviderService'); +const { createVoiceProviderRoutes } = require('./routes/voiceProviderRoutes'); const { DiscordWatchService } = require('./discordWatchService'); const { createDiscordWatchRoutes } = require('./routes/discordWatchRoutes'); const { AppServerService } = require('./appServerService'); @@ -362,6 +364,11 @@ const recommendationsService = RecommendationsService.getInstance(); const repoAtlasService = RepoAtlasService.getInstance({ logger }); const speechService = SpeechService.getInstance({ logger }); speechService.setIO(io); +const voiceProviderService = VoiceProviderService.getInstance({ logger }); +voiceProviderService.init({ speechService }); +// Apply the persisted active TTS choice at boot so the registry and the speech +// service agree on which model speaks. +voiceProviderService.applyActiveTts().catch((error) => logger.warn('Voice provider apply failed', { error: error.message })); const supervisorService = SupervisorService.getInstance({ logger }); const discordWatchService = DiscordWatchService.getInstance({ logger }); const appServerService = AppServerService.getInstance({ logger }); @@ -1435,6 +1442,13 @@ app.use('/api/speech', createSpeechRoutes({ requireWrite: requirePolicyAction('write') })); +app.use('/api/voice-providers', createVoiceProviderRoutes({ + voiceProviderService, + logger, + requireRead: requirePolicyAction('read'), + requireWrite: requirePolicyAction('write') +})); + app.get('/api/app-info', (req, res) => { res.json(readAppInfo()); }); diff --git a/server/routes/voiceProviderRoutes.js b/server/routes/voiceProviderRoutes.js new file mode 100644 index 00000000..896dd1ad --- /dev/null +++ b/server/routes/voiceProviderRoutes.js @@ -0,0 +1,52 @@ +const express = require('express'); + +const passthrough = (req, res, next) => next(); + +/** + * REST surface for the swappable voice-model registry. Reads are policy-`read`; + * changing the active provider is policy-`write`. + */ +function createVoiceProviderRoutes({ voiceProviderService, logger = console, requireRead = passthrough, requireWrite = passthrough } = {}) { + const router = express.Router(); + + const handle = (label, handler) => async (req, res) => { + try { + await handler(req, res); + } catch (error) { + logger.error(`Voice providers: ${label} failed`, { error: error.message }); + res.status(400).json({ ok: false, error: error.message }); + } + }; + + // Everything: active selections + every provider with a live health check. + router.get('/', requireRead, handle('status', async (req, res) => { + res.json({ ok: true, ...(await voiceProviderService.getStatus()) }); + })); + + // Just the providers for one capability (tts|stt|duplex), with health. + router.get('/:kind', requireRead, handle('list by kind', async (req, res) => { + const kind = String(req.params.kind || '').toLowerCase(); + if (!['tts', 'stt', 'duplex'].includes(kind)) { + return res.status(400).json({ ok: false, error: `unknown capability "${kind}"` }); + } + res.json({ ok: true, kind, providers: await voiceProviderService.listWithHealth(kind) }); + })); + + // Swap the active provider for a capability. id may be a provider id, 'auto', or 'none'. + router.post('/:kind/active', requireWrite, handle('set active', async (req, res) => { + const kind = String(req.params.kind || '').toLowerCase(); + const id = String(req.body?.id || '').trim(); + if (!id) return res.status(400).json({ ok: false, error: 'id is required' }); + const result = voiceProviderService.setActive(kind, id); + res.json({ ok: true, ...result, resolved: (await voiceProviderService.resolveActive(kind))?.id || null }); + })); + + router.post('/reload', requireWrite, handle('reload', async (req, res) => { + voiceProviderService.invalidate(); + res.json({ ok: true, ...(await voiceProviderService.getStatus()) }); + })); + + return router; +} + +module.exports = { createVoiceProviderRoutes }; diff --git a/server/speechService.js b/server/speechService.js index 17b45b9c..f276fd4b 100644 --- a/server/speechService.js +++ b/server/speechService.js @@ -1,4 +1,6 @@ const os = require('os'); +const fs = require('fs'); +const path = require('path'); const { spawn, spawnSync } = require('child_process'); const { augmentProcessEnv, getHiddenProcessOptions } = require('./utils/processUtils'); @@ -7,6 +9,25 @@ const MAX_SPOKEN_CHARS = 400; const REPEAT_WINDOW_MS = 30_000; const HISTORY_LIMIT = 50; +// Where `piper` voices land after `piper.download_voices` or a manual fetch — +// checked when PIPER_MODEL is unset so the local voice works out of the box. +const PIPER_VOICE_DIRS = [ + path.join(os.homedir(), '.local', 'share', 'piper-voices'), + path.join(os.homedir(), '.local', 'share', 'piper') +]; + +function discoverPiperModel() { + for (const dir of PIPER_VOICE_DIRS) { + try { + const onnx = fs.readdirSync(dir).find((f) => f.endsWith('.onnx')); + if (onnx) return path.join(dir, onnx); + } catch { + // Directory absent — try the next. + } + } + return ''; +} + /** * Anything spoken aloud is short, plain, and free of shell metacharacters. * Terminal output is full of escape sequences and punctuation that no @@ -45,8 +66,12 @@ class SpeechService { this.io = null; this.enabled = String(process.env.SPEECH_ENABLED || 'true').toLowerCase() !== 'false'; this.preferredBackend = String(process.env.SPEECH_BACKEND || '').trim().toLowerCase(); - this.piperModel = String(process.env.PIPER_MODEL || '').trim(); + this.piperModel = String(process.env.PIPER_MODEL || '').trim() || discoverPiperModel(); this.voice = String(process.env.SPEECH_VOICE || '').trim(); + // A generic local TTS CLI (kokoro-tts, chatterbox, …): reads text on stdin, + // writes raw s16le PCM on stdout. Set by the provider registry when one of + // those is the active engine; empty otherwise. + this.cliEngine = String(process.env.SPEECH_CLI_ENGINE || '').trim(); this.history = []; this.lastSpokenAt = new Map(); this.backendCache = null; @@ -75,6 +100,7 @@ class SpeechService { const backends = [ { id: 'browser', label: 'Browser speech synthesis', available: true, local: false }, { id: 'piper', label: 'Piper (local neural TTS)', available: commandExists('piper') && Boolean(this.piperModel), local: true }, + { id: 'kokoro', label: 'Kokoro / generic local CLI TTS', available: Boolean(this.cliEngine) && commandExists(this.cliEngine), local: true }, { id: 'say', label: 'macOS say', available: os.platform() === 'darwin' && commandExists('say'), local: true }, { id: 'sapi', label: 'Windows SAPI', available: os.platform() === 'win32', local: true }, { id: 'espeak', label: 'espeak-ng', available: commandExists('espeak-ng') || commandExists('espeak'), local: true } @@ -104,6 +130,21 @@ class SpeechService { return backendId; } + /** + * Apply a voice-provider registry choice. Unlike setBackend this never throws + * and re-detects afterward — the registry is the source of truth for which + * model speaks, and a generic-CLI engine (kokoro/chatterbox) changes what is + * available. `engine` is the provider's engine; `command` its CLI binary. + */ + setActiveEngine(engine, { command = '' } = {}) { + const map = { browser: 'browser', piper: 'piper', espeak: 'espeak', say: 'say', sapi: 'sapi', kokoro: 'kokoro' }; + const backend = map[engine] || 'browser'; + if (backend === 'kokoro' && command) this.cliEngine = command; + this.preferredBackend = backend; + this.backendCache = null; + return this.resolveBackend(); + } + isRepeat(text) { const last = this.lastSpokenAt.get(text); return Boolean(last && Date.now() - last < REPEAT_WINDOW_MS); @@ -163,6 +204,34 @@ class SpeechService { } } + /** + * A generic local neural TTS CLI (kokoro-tts, chatterbox, …). Same contract + * as piper: text in on stdin, raw s16le PCM out on stdout, piped to a player + * so the text never touches a shell. + */ + speakViaCli(engine, text) { + if (!engine) return { spoken: false, reason: 'no local CLI TTS engine configured' }; + const player = ['paplay', 'aplay'].find((candidate) => commandExists(candidate)); + if (!player) return { spoken: false, reason: `${engine} is installed but no audio player was found` }; + + const playerArgs = player === 'aplay' + ? ['-q', '-r', '24000', '-f', 'S16_LE', '-t', 'raw', '-'] + : ['--raw', '--rate=24000', '--format=s16le', '--channels=1']; + + try { + const env = augmentProcessEnv(process.env); + const tts = spawn(engine, ['--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); + const playback = spawn(player, playerArgs, { stdio: ['pipe', 'ignore', 'ignore'], env }); + tts.on('error', (error) => this.logger.warn?.(`${engine} failed`, { error: error.message })); + playback.on('error', (error) => this.logger.warn?.('Audio playback failed', { player, error: error.message })); + tts.stdout.pipe(playback.stdin); + tts.stdin.end(`${text}\n`); + return { spoken: true }; + } catch (error) { + return { spoken: false, reason: error.message }; + } + } + speakLocally(backendId, text) { if (backendId === 'say') { return this.spawnQuiet('say', this.voice ? ['-v', this.voice, text] : [text]); @@ -172,6 +241,7 @@ class SpeechService { return this.spawnQuiet(binary, [text]); } if (backendId === 'piper') return this.speakViaPiper(text); + if (backendId === 'kokoro') return this.speakViaCli(this.cliEngine, text); if (backendId === 'sapi') { // Text is already sanitized to printable ASCII with no shell metacharacters; // single quotes are doubled because PowerShell escapes them that way. diff --git a/server/voice/voiceProviderService.js b/server/voice/voiceProviderService.js new file mode 100644 index 00000000..a46f313e --- /dev/null +++ b/server/voice/voiceProviderService.js @@ -0,0 +1,257 @@ +const fs = require('fs'); +const path = require('path'); +const http = require('http'); +const { spawnSync } = require('child_process'); + +const { getAgentWorkspaceDir } = require('../utils/pathUtils'); + +const KINDS = ['tts', 'stt', 'duplex']; +const DEFAULT_CONFIG_PATH = path.join(__dirname, '..', '..', 'config', 'voice-providers.json'); + +function overrideConfigPath() { + return path.join(getAgentWorkspaceDir(), 'voice-providers.json'); +} + +function commandExists(command) { + if (!command) return false; + try { + const probe = process.platform === 'win32' ? 'where.exe' : 'which'; + return spawnSync(probe, [command], { stdio: 'ignore', windowsHide: true }).status === 0; + } catch { + return false; + } +} + +function envSet(name) { + if (!name) return false; + const value = String(process.env[name] || '').trim().toLowerCase(); + return value !== '' && value !== 'false' && value !== '0'; +} + +/** + * Reach a local model server (PersonaPlex/X-Talk) with a short timeout. Any + * response at all — even a 404 — means something is listening, which is all we + * need to know the provider is available. + */ +function serverReachable(endpoint, timeoutMs = 800) { + return new Promise((resolve) => { + let url; + try { url = new URL(endpoint); } catch { return resolve(false); } + const req = http.request( + { method: 'HEAD', hostname: url.hostname, port: url.port || 80, path: url.pathname || '/', timeout: timeoutMs }, + (res) => { res.resume(); resolve(true); } + ); + req.on('error', () => resolve(false)); + req.on('timeout', () => { req.destroy(); resolve(false); }); + req.end(); + }); +} + +function readJson(filePath) { + try { + return JSON.parse(fs.readFileSync(filePath, 'utf8')); + } catch { + return null; + } +} + +/** + * The swappable voice-model registry. + * + * A model is DATA (`config/voice-providers.json`), so adding one is a config + * entry, never code. This service loads that registry, health-checks each + * provider (is the command / model server actually present?), and resolves the + * ONE active provider per capability — where 'auto' means "the best-quality one + * that passes its health check". Everything degrades: a provider whose model + * isn't installed simply reports unavailable and is skipped. + */ +class VoiceProviderService { + constructor({ logger = console } = {}) { + this.logger = logger; + this.config = null; + } + + static getInstance(options = {}) { + if (!VoiceProviderService.instance) { + VoiceProviderService.instance = new VoiceProviderService(options); + } + return VoiceProviderService.instance; + } + + init({ speechService } = {}) { + this.speechService = speechService || this.speechService; + return this; + } + + /** + * Push the resolved active TTS provider into the speech service so a swap + * actually changes what speaks. No-op if speech isn't wired. + */ + async applyActiveTts() { + if (!this.speechService?.setActiveEngine) return null; + const provider = await this.resolveActive('tts'); + if (provider) this.speechService.setActiveEngine(provider.engine, { command: provider.requires?.command || '' }); + return provider; + } + + configPath() { + return readJson(overrideConfigPath()) ? overrideConfigPath() : DEFAULT_CONFIG_PATH; + } + + load({ force = false } = {}) { + if (this.config && !force) return this.config; + const raw = readJson(this.configPath()) || {}; + const providers = (Array.isArray(raw.providers) ? raw.providers : []) + .filter((p) => p && p.id && KINDS.includes(p.kind)) + .map((p) => ({ + id: String(p.id), + kind: p.kind, + label: String(p.label || p.id), + engine: String(p.engine || p.id), + local: p.local === true, + quality: Number.isFinite(Number(p.quality)) ? Number(p.quality) : 3, + requires: p.requires && typeof p.requires === 'object' ? p.requires : {}, + endpoint: String(p.endpoint || ''), + transport: String(p.transport || ''), + install: String(p.install || ''), + notes: String(p.notes || '') + })); + + this.config = { + source: this.configPath(), + activeTts: String(raw.activeTts || 'auto'), + activeStt: String(raw.activeStt || 'auto'), + activeDuplex: String(raw.activeDuplex || 'none'), + providers + }; + return this.config; + } + + invalidate() { + this.config = null; + } + + list(kind = null) { + const { providers } = this.load(); + return kind ? providers.filter((p) => p.kind === kind) : providers; + } + + get(id) { + return this.load().providers.find((p) => p.id === id) || null; + } + + /** + * Is this provider usable right now? A missing command / unset env / no model + * server means "not available" — never an error. + */ + async checkAvailability(provider) { + if (!provider) return { available: false, reason: 'unknown provider' }; + const req = provider.requires || {}; + + // TTS engines the speech service owns: defer to its real detection (which + // includes piper voice-model auto-discovery) so the registry and the thing + // that actually speaks never disagree. + const nativeTts = { browser: 'browser', piper: 'piper', espeak: 'espeak', say: 'say', sapi: 'sapi' }; + if (provider.kind === 'tts' && nativeTts[provider.engine] && this.speechService?.detectBackends) { + const backend = this.speechService.detectBackends({ force: true }).find((b) => b.id === nativeTts[provider.engine]); + if (backend) { + return backend.available + ? { available: true } + : { available: false, reason: `${provider.engine} not ready`, install: provider.install }; + } + } + + if (req.command && !commandExists(req.command)) { + return { available: false, reason: `command "${req.command}" not found`, install: provider.install }; + } + if (req.env && !envSet(req.env)) { + return { available: false, reason: `env ${req.env} not set` }; + } + if (req.server) { + const up = await serverReachable(provider.endpoint || req.server); + if (!up) return { available: false, reason: `no server at ${provider.endpoint || req.server}`, install: provider.install }; + } + // The browser TTS backend has no local requirement and is always usable. + return { available: true }; + } + + async listWithHealth(kind = null) { + const providers = this.list(kind); + return Promise.all(providers.map(async (p) => ({ ...p, ...(await this.checkAvailability(p)) }))); + } + + activeKey(kind) { + const cfg = this.load(); + return { tts: cfg.activeTts, stt: cfg.activeStt, duplex: cfg.activeDuplex }[kind]; + } + + /** + * The active provider for a capability. 'none' -> null; 'auto' -> the highest + * quality available one; an explicit id -> that provider if available, else + * fall back to auto so a broken pin never silently disables voice. + */ + async resolveActive(kind) { + const key = this.activeKey(kind); + if (key === 'none') return null; + + const withHealth = await this.listWithHealth(kind); + const available = withHealth.filter((p) => p.available); + + if (key && key !== 'auto') { + const pinned = withHealth.find((p) => p.id === key); + if (pinned?.available) return pinned; + // fall through to auto + } + return available.sort((a, b) => b.quality - a.quality)[0] || null; + } + + /** + * Persist the active provider for a capability into the machine-local + * override file, so it survives restarts without editing the shipped config. + */ + setActive(kind, id) { + if (!KINDS.includes(kind)) throw new Error(`Unknown capability "${kind}" (expected ${KINDS.join('|')})`); + if (id !== 'auto' && id !== 'none') { + const provider = this.get(id); + if (!provider) throw new Error(`Unknown voice provider "${id}"`); + if (provider.kind !== kind) throw new Error(`Provider "${id}" is a ${provider.kind}, not a ${kind}`); + } + + const target = overrideConfigPath(); + const current = readJson(target) || readJson(DEFAULT_CONFIG_PATH) || {}; + const field = { tts: 'activeTts', stt: 'activeStt', duplex: 'activeDuplex' }[kind]; + current[field] = id; + + fs.mkdirSync(path.dirname(target), { recursive: true }); + const tmp = `${target}.${process.pid}.tmp`; + fs.writeFileSync(tmp, `${JSON.stringify(current, null, 2)}\n`, 'utf8'); + fs.renameSync(tmp, target); + this.invalidate(); + // A TTS swap takes effect immediately. + if (kind === 'tts') this.applyActiveTts().catch(() => {}); + return { kind, active: id }; + } + + async getStatus() { + const [tts, stt, duplex] = await Promise.all([ + this.resolveActive('tts'), + this.resolveActive('stt'), + this.resolveActive('duplex') + ]); + return { + source: this.load().source, + active: { + tts: { selected: this.activeKey('tts'), resolved: tts?.id || null, label: tts?.label || null }, + stt: { selected: this.activeKey('stt'), resolved: stt?.id || null, label: stt?.label || null }, + duplex: { selected: this.activeKey('duplex'), resolved: duplex?.id || null, label: duplex?.label || null } + }, + providers: await this.listWithHealth() + }; + } +} + +module.exports = VoiceProviderService; +module.exports.VoiceProviderService = VoiceProviderService; +module.exports.KINDS = KINDS; +module.exports.DEFAULT_CONFIG_PATH = DEFAULT_CONFIG_PATH; +module.exports.overrideConfigPath = overrideConfigPath; diff --git a/tests/unit/voiceProviderService.test.js b/tests/unit/voiceProviderService.test.js new file mode 100644 index 00000000..c6b394da --- /dev/null +++ b/tests/unit/voiceProviderService.test.js @@ -0,0 +1,122 @@ +const fs = require('fs'); +const os = require('os'); +const path = require('path'); + +const { VoiceProviderService, DEFAULT_CONFIG_PATH } = require('../../server/voice/voiceProviderService'); + +// A speech service stand-in whose backend availability the tests control. +const fakeSpeech = (available = {}) => ({ + applied: [], + detectBackends() { + return [ + { id: 'browser', available: true }, + { id: 'piper', available: available.piper === true }, + { id: 'espeak', available: available.espeak === true } + ]; + }, + setActiveEngine(engine, opts) { this.applied.push({ engine, ...opts }); return engine; } +}); + +const service = (over = {}) => { + const s = new VoiceProviderService({ logger: { warn() {}, error() {} } }); + if (over.env) process.env.AGENT_WORKSPACE_DIR = over.env; + return s; +}; + +afterEach(() => { delete process.env.AGENT_WORKSPACE_DIR; }); + +describe('VoiceProviderService', () => { + test('loads the shipped registry with tts/stt/duplex providers', () => { + const cfg = service().load(); + expect(cfg.providers.length).toBeGreaterThan(0); + expect(service().list('tts').length).toBeGreaterThan(0); + expect(service().list('stt').length).toBeGreaterThan(0); + expect(service().list('duplex').length).toBeGreaterThan(0); + expect(cfg.source).toBe(DEFAULT_CONFIG_PATH); + }); + + test('a provider whose model is absent reports unavailable, never throws', async () => { + const s = service(); + const health = await s.checkAvailability(s.get('personaplex')); + expect(health.available).toBe(false); + expect(health.reason).toMatch(/no server/); + expect(health.install).toBeTruthy(); + }); + + test('browser TTS is always available (no local requirement)', async () => { + const s = service(); + expect((await s.checkAvailability(s.get('browser'))).available).toBe(true); + }); + + test('TTS health defers to the speech service so they never disagree', async () => { + const s = service(); + s.init({ speechService: fakeSpeech({ piper: true }) }); + expect((await s.checkAvailability(s.get('piper'))).available).toBe(true); + + const s2 = service(); + s2.init({ speechService: fakeSpeech({ piper: false }) }); + expect((await s2.checkAvailability(s2.get('piper'))).available).toBe(false); + }); + + test('auto resolves to the highest-quality AVAILABLE provider', async () => { + const s = service(); + s.init({ speechService: fakeSpeech({ piper: true }) }); + // piper (quality 3) beats browser (quality 2) when it is available. + expect((await s.resolveActive('tts'))?.id).toBe('piper'); + + const s2 = service(); + s2.init({ speechService: fakeSpeech({ piper: false }) }); + // With piper unavailable it falls back to browser rather than nothing. + expect((await s2.resolveActive('tts'))?.id).toBe('browser'); + }); + + test('a pin to an unavailable provider falls back to auto, never silently mutes', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vp-')); + fs.writeFileSync(path.join(dir, 'voice-providers.json'), JSON.stringify({ + activeTts: 'personaplex', // duplex id pinned as tts + unavailable anyway + providers: JSON.parse(fs.readFileSync(DEFAULT_CONFIG_PATH, 'utf8')).providers + })); + const s = service({ env: dir }); + s.init({ speechService: fakeSpeech({ piper: false }) }); + const resolved = await s.resolveActive('tts'); + expect(resolved?.id).toBe('browser'); // fell back to the best available + fs.rmSync(dir, { recursive: true, force: true }); + }); + + test('duplex defaults to none (off) until a model is explicitly chosen', async () => { + const s = service(); + expect(s.activeKey('duplex')).toBe('none'); + expect(await s.resolveActive('duplex')).toBeNull(); + }); + + test('setActive persists to the override and applies TTS immediately', async () => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vp-set-')); + const s = service({ env: dir }); + const speech = fakeSpeech({ piper: true }); + s.init({ speechService: speech }); + + s.setActive('tts', 'browser'); + const written = JSON.parse(fs.readFileSync(path.join(dir, 'voice-providers.json'), 'utf8')); + expect(written.activeTts).toBe('browser'); + await new Promise((r) => setTimeout(r, 5)); // applyActiveTts is async + expect(speech.applied.some((a) => a.engine === 'browser')).toBe(true); + + fs.rmSync(dir, { recursive: true, force: true }); + }); + + test('setActive rejects an unknown id and a cross-capability id', () => { + const s = service({ env: fs.mkdtempSync(path.join(os.tmpdir(), 'vp-rej-')) }); + expect(() => s.setActive('tts', 'not-a-provider')).toThrow(/Unknown voice provider/); + expect(() => s.setActive('tts', 'personaplex')).toThrow(/is a duplex, not a tts/); + expect(() => s.setActive('bogus', 'auto')).toThrow(/Unknown capability/); + }); + + test('getStatus reports selected + resolved per capability', async () => { + const s = service(); + s.init({ speechService: fakeSpeech({ piper: true }) }); + const status = await s.getStatus(); + expect(status.active.tts.resolved).toBe('piper'); + expect(status.active.duplex.selected).toBe('none'); + expect(status.providers.every((p) => 'available' in p)).toBe(true); + }); +}); From 8e9105101741e541bc01838bbd86422241cadf72 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 09:20:06 +1000 Subject: [PATCH 40/69] docs: full voice-model catalogue + swappable-registry usage; codebase map Documents every option found (GPT-Live closed SOTA reference, PersonaPlex/Moshi/ X-Talk duplex, Qwen omni, Parakeet/faster-whisper/Moonshine STT, Kokoro/Chatterbox/ Piper TTS + more) and the shipped add/swap-a-model workflow. Records the new voiceProviderService/routes/config in CODEBASE_DOCUMENTATION. --- CODEBASE_DOCUMENTATION.md | 16 +++- .../2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md | 80 +++++++++++++++++-- 2 files changed, 90 insertions(+), 6 deletions(-) diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index b281f4e0..7e42c585 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -153,7 +153,16 @@ server/speechService.js - Speech output with degrading backends server/routes/speechRoutes.js - `/api/speech/*` (say, backend, enabled, spoken fleet briefing) client/speech-output.js - Web Speech API listener for the browser backend (`window.SpeechOutput`) server/voiceCommandService.js - (existing) rule/LLM voice parsing, now with `setCommanderForwarder()`: unmatched speech is handed to the Commander agent instead of dead-ending -tests/unit/speechService.test.js - Sanitization, repeat suppression, backend resolution +server/voice/voiceProviderService.js - Swappable voice-model registry (a model is DATA, not code) +├─ Loads `config/voice-providers.json` (override `~/.agent-workspace/voice-providers.json`) +├─ Health-checks each provider (command present? env set? model server reachable?) — all degrade, never throw +├─ Resolves the ONE active provider per capability (tts/stt/duplex); `auto` = best-quality available, `none` = off; a broken pin falls back to auto so voice never silently mutes +├─ `setActive(kind, id)` persists to the override and applies TTS to speechService immediately +└─ speechService gained a Kokoro/generic-CLI backend + Piper voice auto-discovery (`~/.local/share/piper-voices`) +server/routes/voiceProviderRoutes.js - `/api/voice-providers/*` (list+health, swap active per capability, reload) +config/voice-providers.json - The model catalogue: TTS (browser/piper/kokoro/chatterbox/espeak), STT (whisper-cpp/faster-whisper/parakeet/moonshine), duplex (codex/personaplex/xtalk) with install hints +tests/unit/speechService.test.js, voiceProviderService.test.js - Sanitization/backends; registry load, health, auto-resolution, live swap +PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md - Full catalogue + how to add/swap a model on the 5090 server/appServerService.js - Codex app-server bridge: structured signals + realtime voice ├─ Protocol: JSON-RPC 2.0 over stdio to `codex app-server` (Apache 2.0, ships in the CLI); `initialize` handshake required first @@ -784,6 +793,11 @@ POST /api/speech/say - Speak text (`pri POST /api/speech/backend - Choose a backend POST /api/speech/enabled - Mute/unmute POST /api/speech/briefing - Speak the supervisor briefing + +GET /api/voice-providers - Every voice model with a live availability check + active tts/stt/duplex +GET /api/voice-providers/:kind - Providers for one capability (tts|stt|duplex), with health +POST /api/voice-providers/:kind/active - Swap the active model (id | 'auto' | 'none'); applies immediately +POST /api/voice-providers/reload - Re-read the registry from disk ``` ### WebSocket Events diff --git a/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md b/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md index 7a4c604e..46e31d1b 100644 --- a/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md +++ b/PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md @@ -4,9 +4,19 @@ Goal: full-duplex, back-and-forth, **local** voice for JARVIS, with models swapp runtime. Target hardware: **RTX 5090, 32 GB** (the real deployment) and an **RTX 3080 Laptop, 16 GB** (for testing now). -The strongest turnkey options actually landed Jan–Mar 2026, not June/July — the mid-2026 -items (DyaPlex, ASPIRin, SoulX-Duplug) are research papers, not shippable servers yet. So -the recommendation below is built on what you can run this week. +> **STATUS: the swappable framework SHIPPED (2026-07-27).** `config/voice-providers.json` +> + `server/voice/voiceProviderService.js` + `/api/voice-providers/*`. A model is now a +> config entry, not code. **Piper** is installed and running as the local default on the +> laptop (verified speaking over WSLg). Everything below is the catalogue of what plugs into +> that registry — **model choice is now a data edit**, so "try a better one on the PC later" +> is `atlas`-style: add an entry, point it at the model, `POST /api/voice-providers//active`. +> The heavy full-duplex models (PersonaPlex ~19 GB, Qwen 24 GB) are registered but light up +> only on the 5090 / when their server is running — they degrade cleanly here. + +Model churn is fast (GPT-Live shipped July 8; new open models monthly), which is exactly why +the registry — not any single model — is the deliverable. The strongest *turnkey open* options +landed Jan–Mar 2026; the true SOTA (GPT-Live) is closed. Newer open frameworks (X-Talk) and +research (DyaPlex, ASPIRin, SoulX-Duplug, mid-2026) are noted below but not yet turnkey. ## Two architectures, pick per use @@ -21,6 +31,46 @@ codebase already has (`speechService`, `whisperService`). The right answer is **both**: A for ambient chat, B when the utterance needs to become work. +## Full catalogue of options (everything found — nothing wasted) + +### Full-duplex speech-to-speech (single model, real barge-in) + +| Model | Open? | Params | VRAM | Latency | License | Notes | +|---|---|---|---|---|---|---| +| **GPT-Live-1 / -mini** (OpenAI, Jul 8 2026) | ❌ closed | — | cloud | best-in-class | proprietary | The current SOTA; full-duplex, natural turn-taking, live translation. **API not out yet.** Reference target, not self-hostable. | +| **NVIDIA PersonaPlex** ⭐ (Jan 2026) | ✅ | 7B | ~19 GB BF16 / 8–16 GB quant | ~70 ms switch, ~170 ms resp | open weights | Best open local. Moshi + Mimi. Persona/voice control. Voice-only (no tools), 4-min context. `github.com/NVIDIA/personaplex`. | +| **Moshi** (Kyutai, 2024) | ✅ | 7B | ~16 GB | ~200 ms | open | The base PersonaPlex fine-tunes. `moshi.cpp` for quant/CPU. | +| **X-Talk** (mid-2026) | ✅ | cascaded | light | low, interruptible | open | Pure-Python full-duplex *framework* (STT+LLM+TTS with barge-in). Newer, lighter than PersonaPlex. Registered in the registry (`xtalk`). | +| DyaPlex / ASPIRin / SoulX-Duplug | ✅ papers | — | — | — | research | Mid-2026 arXiv; plug-and-play duplex state prediction. Not turnkey yet — watch these. | +| FlexDuo / SALM-Duplex | ✅ papers | — | — | — | research | Pluggable duplex modules; ideas to steal, not servers. | + +### Omni (one model reasons + streams speech; not true barge-in) + +| Model | Params | VRAM | License | Notes | +|---|---|---|---|---| +| **Qwen3.5-Omni-30B-A3B** (Mar 2026) | 30B MoE / 3B active | ~24 GB INT4 (fits 5090) | Apache 2.0 | Thinker+Talker = text **and** speech in one pass, no separate TTS. Reasons + tools. vLLM ≥ 0.17. Older now — expect newer omni models; swap when they land. | + +### STT (listen) — pipeline lane + +| Model | VRAM | Notes | +|---|---|---| +| **Parakeet TDT** ⭐ (NVIDIA) | small | RNN-T, streaming, RTFx > 2000 — lowest latency. Best on the 5090. Registry id `parakeet`. | +| **faster-whisper** (CTranslate2) | small | Fast Whisper reimpl, GPU/CPU. Drop-in. Registry id `faster-whisper`. | +| **whisper.cpp** | small | The existing path. Accurate, offline. Registry id `whisper-cpp`. | +| **Distil-Whisper / Moonshine** | tiny | Lightest; Moonshine for edge/CPU (laptop). Registry id `moonshine`. | + +### TTS (speak) — pipeline lane + +| Model | VRAM | License | Notes | +|---|---|---|---| +| **Kokoro-82M** ⭐ | tiny | Apache 2.0 | Best lightweight open TTS 2026. Natural, tiny compute. Registry id `kokoro`. | +| **Chatterbox** (Resemble) | small (GPU for RT) | permissive | Real-time + voice cloning. Registry id `chatterbox`. | +| **Piper** ✅ installed | tiny (CPU) | MIT | **Running now as the local default.** Fully offline, fast. Registry id `piper`. | +| **CosyVoice2-0.5B** | small | — | Ultra-low-latency streaming. | +| **Fish S2 Pro** | — | — | Sub-100 ms on vLLM, 3B Llama-style decoder. | +| **Orpheus / Higgs Audio V2 / Dia2 / XTTS-v2 / F5-TTS** | varies | varies | Strong alternatives; clone quality fools listeners 70–85% on a 4060 Ti 16 GB or 3060. | +| **espeak-ng** | none | GPL | Robotic last-resort, never fails. Registry id `espeak`. | + ## Recommended models ### Full-duplex (architecture A) @@ -51,9 +101,28 @@ and speaks. Won't fit the laptop comfortably. **Chatterbox** (Resemble, real-time, permissive, voice cloning). **CosyVoice2-0.5B** / **Fish S2 Pro** (sub-100 ms on vLLM) if you want the lowest latency. All run on the laptop. -## How this plugs into JARVIS (the swappable design you asked for) +## How to add / swap a model (the shipped workflow) + +Adding "a better one on the PC later" is a **config edit + one API call**, no code: + +1. Add an entry to `config/voice-providers.json` (or the per-machine override + `~/.agent-workspace/voice-providers.json`): `{ id, kind: tts|stt|duplex, engine, + requires: {command|env|server}, endpoint, quality, install, notes }`. +2. Install the model per its `install` hint. +3. Make it active: `POST /api/voice-providers//active {"id":""}` — or set + `activeTts/activeStt/activeDuplex` to `auto` and it wins automatically if it's the + highest-quality one that passes its health check. +4. `GET /api/voice-providers` shows every provider with a live availability check, so you + can see what's installed vs what needs setup. + +On the **5090**: register PersonaPlex (`duplex`) pointing at its `localhost:8998` server and +`POST .../duplex/active {"id":"personaplex"}`; add `kokoro`/`parakeet` for the pipeline lane. +A provider whose model isn't present just shows `available:false` with its install hint — it +never breaks anything. + +## How this plugs into JARVIS (the seams it extends) -The codebase is already 80% there — don't bolt on, extend the seams that exist: +The codebase was already 80% there — this extended the seams rather than bolting on: 1. **`speechService` already has pluggable TTS backends** (browser/piper/say/SAPI/espeak). Add `kokoro` and `chatterbox` as two more backends behind the same interface. Swap with @@ -83,6 +152,7 @@ the 5090 once the registry exists. Qwen3.5-Omni is the stretch goal for a single voice-and-reason model. ## Sources +- **GPT-Live (OpenAI, Jul 8 2026, closed SOTA reference):** https://openai.com/index/introducing-gpt-live/ · https://techcrunch.com/2026/07/08/openai-releases-new-voice-models-for-more-natural-live-conversations/ - https://research.nvidia.com/labs/adlr/personaplex · https://github.com/NVIDIA/personaplex - https://www.makeuseof.com/nvidia-personaplex-local-speech-model-8gb-vram/ - https://www.kunalganglani.com/blog/nvidia-personaplex-full-duplex-voice-ai From 6cecd4c492153331c7d743c31947cd1284b5e56a Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 09:34:47 +1000 Subject: [PATCH 41/69] =?UTF-8?q?feat(voice):=20voice=20brain=20=E2=80=94?= =?UTF-8?q?=20Commander=20extension=20with=20fast=20fact=20answers?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Voice is no longer a fixed phrasebook. An utterance routes through three lanes, fastest first: 1. COMMAND — a semantic command in the registry, run instantly (kept). 2. FACT — a question answerable from live orchestrator state (sessions, supervisor briefing, queue, discord, workspace) answered straight from a context snapshot: an API shortcut, no LLM turn, spoken back in ~ms. 3. AGENT — anything else handed to the Commander (full API, can do anything), acknowledged aloud. Every lane speaks. An action phrasing ('open the queue') is never hijacked by the fact lane. Built on the existing commanderContextService snapshot + command registry, so the voice has the same visibility the Commander does. Verified live: 'how many agents working', 'what needs me', 'what workspace', 'what can you do' answered instantly from real state and spoken via local Piper. Co-Authored-By: Claude Fable 5 --- server/index.js | 17 +++ server/voice/voiceBrainService.js | 196 +++++++++++++++++++++++++++ server/voiceCommandService.js | 27 +++- tests/unit/voiceBrainService.test.js | 104 ++++++++++++++ 4 files changed, 343 insertions(+), 1 deletion(-) create mode 100644 server/voice/voiceBrainService.js create mode 100644 tests/unit/voiceBrainService.test.js diff --git a/server/index.js b/server/index.js index 22b96127..27ac7aa4 100644 --- a/server/index.js +++ b/server/index.js @@ -118,6 +118,7 @@ const { SpeechService } = require('./speechService'); const { createSpeechRoutes } = require('./routes/speechRoutes'); const { VoiceProviderService } = require('./voice/voiceProviderService'); const { createVoiceProviderRoutes } = require('./routes/voiceProviderRoutes'); +const { VoiceBrainService } = require('./voice/voiceBrainService'); const { DiscordWatchService } = require('./discordWatchService'); const { createDiscordWatchRoutes } = require('./routes/discordWatchRoutes'); const { AppServerService } = require('./appServerService'); @@ -494,6 +495,22 @@ if (discordWatchStarted.running) logger.info('Discord watch', discordWatchStarte // active Commander so the fallback is an agent, not an error. voiceCommandService.setCommanderForwarder(sendToCommander); +// The voice brain routes unmatched speech: a fast fact answer from live +// orchestrator state, else the Commander agent — both spoken. This is what +// makes voice an extension of Commander rather than a fixed phrasebook. +const voiceBrainService = VoiceBrainService.getInstance({ logger }); +voiceBrainService.init({ + voiceCommandService, + speechService, + commanderContextService, + workspaceManager, + commanderService, + commandRegistry, + supervisorService, + discordWatchService, + commanderForwarder: sendToCommander +}); + const loadPlugins = async () => { const status = await pluginLoaderService.loadAll({ app, diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js new file mode 100644 index 00000000..ad9e79f4 --- /dev/null +++ b/server/voice/voiceBrainService.js @@ -0,0 +1,196 @@ +/** + * The voice brain — what makes talking to JARVIS more than a phrasebook. + * + * An utterance is routed through three lanes, fastest first: + * + * 1. COMMAND — a semantic command in the registry ("open the queue", + * "focus work one"). Matched by voiceCommandService, run instantly. Zero + * tokens, near-zero latency. Phrases ARE good; they stay the fast path. + * + * 2. FACT — a question answerable from live orchestrator state ("how many + * agents are working?", "what needs me?", "anything from Discord?"). Answered + * straight from a context snapshot — an API shortcut, no LLM turn — so it + * comes back as fast as a command. This is the "full visibility" lane. + * + * 3. AGENT — anything else ("spin up a reviewer for PR 12 and tell me when + * it's done"). Handed to the Commander, a full agent with the entire + * orchestrator API, so voice can ask for *anything*, not just listed verbs. + * + * Every lane speaks its result back. The brain is the extension of Commander the + * voice layer needed: fast where a shortcut exists, an agent where it doesn't. + */ +class VoiceBrainService { + constructor({ logger = console } = {}) { + this.logger = logger; + this.deps = {}; + } + + static getInstance(options = {}) { + if (!VoiceBrainService.instance) { + VoiceBrainService.instance = new VoiceBrainService(options); + } + return VoiceBrainService.instance; + } + + init(deps = {}) { + this.deps = { ...this.deps, ...deps }; + // The voice command service forwards its unmatched utterances here. + if (this.deps.voiceCommandService?.setBrain) this.deps.voiceCommandService.setBrain(this); + return this; + } + + /** + * A compact, live snapshot of everything the brain can see — the same + * visibility the Commander has, assembled from the services directly so a + * fact answer never needs a round trip. + */ + buildContext() { + const d = this.deps; + const ctx = { sessions: [], workspace: null, queue: [], supervisor: null, discord: [], capabilities: 0 }; + + try { + const snap = d.commanderContextService?.getSnapshot?.({ + workspaceManager: d.workspaceManager, + commanderService: d.commanderService, + commandRegistry: d.commandRegistry + }) || {}; + ctx.sessions = snap.computed?.sessions || []; + ctx.workspace = snap.computed?.activeWorkspace?.name || null; + ctx.queue = snap.context?.queueSummary || []; + ctx.capabilities = snap.computed?.capabilitiesSummary?.commandCount || 0; + } catch (error) { + this.logger.warn?.('voice brain: context snapshot failed', { error: error.message }); + } + + try { ctx.supervisor = d.supervisorService?.getBriefing?.({ limit: 5 }) || null; } catch { /* optional */ } + try { ctx.discord = d.discordWatchService?.getUntracked?.() || []; } catch { /* optional */ } + + return ctx; + } + + countSessions(sessions) { + const tally = { busy: 0, waiting: 0, idle: 0, total: 0 }; + for (const s of Array.isArray(sessions) ? sessions : []) { + const status = String(s?.status || '').toLowerCase(); + if (!s?.sessionId && !s?.id) continue; + tally.total += 1; + if (status === 'busy') tally.busy += 1; + else if (status === 'waiting') tally.waiting += 1; + else tally.idle += 1; + } + return tally; + } + + /** + * Answer a question straight from the snapshot. Returns the spoken answer, or + * null if this isn't a question the state can answer — in which case the + * utterance falls through to the agent. + */ + answerFromContext(transcript, ctx = this.buildContext()) { + const t = String(transcript || '').toLowerCase().trim(); + if (!t) return null; + + // An action request ("open the queue", "start a reviewer") is never a fact + // to read back — it belongs to the command lane or the agent. Only answer + // questions here, so the fact lane can't hijack a thing you asked it to DO. + if (/^(open|show|hide|close|focus|switch|go to|goto|start|stop|run|create|make|spawn|launch|kill|delete|remove|add|set|move|approve|reject|merge|push|pull|commit|clear|refresh|reload|new)\b/.test(t)) { + return null; + } + + // What needs me / what's wrong / status of the fleet + if (/(needs?|need)\s+(me|my|your)|attention|anything (wrong|broken|stuck|urgent)|what.*(should i|do i need)/.test(t)) { + return ctx.supervisor?.spoken || 'Nothing needs you right now. Everything else was handled.'; + } + + // How many agents / sessions working + if (/(how many|number of).*(agent|session|running|working|busy)|what.*(agents|sessions).*(doing|status)|fleet status|are (they|the agents) (busy|working)/.test(t)) { + const c = this.countSessions(ctx.sessions); + if (!c.total) return 'No agent sessions are open right now.'; + const parts = []; + if (c.busy) parts.push(`${c.busy} working`); + if (c.waiting) parts.push(`${c.waiting} waiting on input`); + if (c.idle) parts.push(`${c.idle} idle`); + return `You have ${c.total} agent${c.total === 1 ? '' : 's'}: ${parts.join(', ')}.`; + } + + // The queue / what's next + if (/\bqueue\b|what.*(next|to do|on the list)|what.*work.*(left|remaining)/.test(t)) { + const q = Array.isArray(ctx.queue) ? ctx.queue : []; + if (!q.length) return 'The queue is empty.'; + const top = q.slice(0, 3).map((x) => String(x?.title || x?.id || '').trim()).filter(Boolean); + return `${q.length} item${q.length === 1 ? '' : 's'} in the queue. Top: ${top.join('; ')}.`; + } + + // Discord / what was asked for + if (/\b(discord|chat)\b|asked (for|me)|anyone (need|ask)|untracked/.test(t)) { + const items = Array.isArray(ctx.discord) ? ctx.discord : []; + if (!items.length) return 'Nothing outstanding from chat.'; + const first = items[0]; + return `${items.length} thing${items.length === 1 ? '' : 's'} asked for but not started. Most urgent: ${String(first?.summary || first?.text || '').slice(0, 120)}.`; + } + + // Which workspace + if (/what.*(workspace|project).*(in|on|open)|which workspace|where am i/.test(t)) { + return ctx.workspace ? `You're in the ${ctx.workspace} workspace.` : 'No workspace is open right now.'; + } + + // What can you do + if (/what can you do|what commands|help me|what.*(you|can i) (say|ask)/.test(t)) { + return `I can run about ${ctx.capabilities || 'a set of'} orchestrator commands directly, answer questions about your fleet, and hand anything else to the Commander to work on.`; + } + + return null; + } + + speak(text, { priority = 'normal' } = {}) { + if (!text) return { spoken: false }; + try { + return this.deps.speechService?.speak?.(text, { priority }) || { spoken: false }; + } catch (error) { + this.logger.warn?.('voice brain: speak failed', { error: error.message }); + return { spoken: false, reason: error.message }; + } + } + + /** + * The unmatched-utterance entry point voiceCommandService calls: try a fast + * fact answer, otherwise hand the whole thing to the Commander agent. Either + * way, speak. + */ + async handleUnmatched(transcript) { + const ctx = this.buildContext(); + const fact = this.answerFromContext(transcript, ctx); + if (fact) { + this.speak(fact); + return { handled: true, route: 'fact', spoken: fact }; + } + + // Agent lane: the Commander has the whole API and can do anything. + const forwarder = this.deps.commanderForwarder; + if (typeof forwarder === 'function') { + const brief = `[voice] ${String(transcript || '').trim()}\n\nAnswer or act using the orchestrator API (GET /api/commander/context, /capabilities, POST /execute). Keep any spoken reply to one or two sentences.`; + let delivered = false; + try { delivered = (await forwarder(brief)) !== false; } catch (error) { + this.logger.warn?.('voice brain: commander forward failed', { error: error.message }); + } + const ack = delivered ? 'On it — working on that now.' : 'No Commander is running to take that.'; + this.speak(ack); + return { handled: delivered, route: 'commander', spoken: ack }; + } + + const miss = "I couldn't do that myself and there's no Commander running to hand it to."; + this.speak(miss); + return { handled: false, route: 'none', spoken: miss }; + } + + /** A short spoken confirmation after a command lane hit. */ + confirmCommand(command) { + const label = String(command || '').replace(/[-_]/g, ' ').trim(); + const say = label ? `Done — ${label}.` : 'Done.'; + this.speak(say); + return say; + } +} + +module.exports = VoiceBrainService; +module.exports.VoiceBrainService = VoiceBrainService; diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 04412de8..a0131e68 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -1580,10 +1580,32 @@ JSON:`; * `forwardUnmatched` defaults on for spoken input: if no rule and no LLM can * turn the utterance into a command, the words themselves are still useful. */ + setBrain(brain) { + this.brain = brain || null; + return Boolean(this.brain); + } + async processVoiceCommand(transcript, { forwardUnmatched = true } = {}) { const parsed = await this.parseCommand(transcript); if (!parsed.success) { + // No command matched. If the brain is wired, let it try a fast fact answer + // from live orchestrator state, then fall back to the Commander agent — + // both spoken. This is what makes voice more than a fixed phrasebook. + if (this.brain?.handleUnmatched) { + const outcome = await this.brain.handleUnmatched(transcript); + return { + success: outcome.handled, + method: outcome.route, + command: null, + params: {}, + transcript, + executed: outcome.handled, + spoken: outcome.spoken, + forwardedToCommander: outcome.route === 'commander' + }; + } + if (!forwardUnmatched || !this.commanderForwarder) return parsed; const forward = await this.forwardToCommander(transcript); @@ -1602,11 +1624,14 @@ JSON:`; } const result = await this.executeCommand(parsed.command, parsed.params); + // Speak a short confirmation so the fast command lane talks back too. + const spoken = this.brain?.confirmCommand ? this.brain.confirmCommand(parsed.command) : undefined; return { ...parsed, executed: true, - result + result, + spoken }; } diff --git a/tests/unit/voiceBrainService.test.js b/tests/unit/voiceBrainService.test.js new file mode 100644 index 00000000..31f6063e --- /dev/null +++ b/tests/unit/voiceBrainService.test.js @@ -0,0 +1,104 @@ +const { VoiceBrainService } = require('../../server/voice/voiceBrainService'); + +function brain(over = {}) { + const spoken = []; + const forwarded = []; + const b = new VoiceBrainService({ logger: { warn() {} } }); + b.init({ + speechService: { speak: (t) => { spoken.push(t); return { spoken: true }; } }, + commanderContextService: { + getSnapshot: () => ({ + computed: { + sessions: over.sessions || [], + activeWorkspace: over.workspace ? { name: over.workspace } : null, + capabilitiesSummary: { commandCount: 42 } + }, + context: { queueSummary: over.queue || [] } + }) + }, + supervisorService: { getBriefing: () => over.briefing || { spoken: 'Nothing needs you right now. Everything else was handled.' } }, + discordWatchService: { getUntracked: () => over.discord || [] }, + commanderForwarder: over.forwarder || (async (text) => { forwarded.push(text); return over.forwarderReturns ?? true; }) + }); + return { b, spoken, forwarded }; +} + +describe('VoiceBrainService — fact lane', () => { + test('"what needs me" reads the supervisor briefing', () => { + const { b } = brain({ briefing: { spoken: 'work3 has been waiting on a permission for four minutes.' } }); + expect(b.answerFromContext('hey what needs me right now')).toMatch(/waiting on a permission/); + }); + + test('"how many agents are working" counts live sessions', () => { + const { b } = brain({ sessions: [ + { sessionId: 'a', status: 'busy' }, { sessionId: 'b', status: 'busy' }, + { sessionId: 'c', status: 'waiting' }, { sessionId: 'd', status: 'idle' } + ] }); + const answer = b.answerFromContext('how many agents are working'); + expect(answer).toMatch(/4 agents/); + expect(answer).toMatch(/2 working/); + expect(answer).toMatch(/1 waiting/); + }); + + test('queue question summarizes the top items', () => { + const { b } = brain({ queue: [{ id: '1', title: 'fix the crash' }, { id: '2', title: 'add leaderboard' }] }); + expect(b.answerFromContext('what is on the queue')).toMatch(/2 items.*fix the crash/); + }); + + test('discord question surfaces the most urgent untracked ask', () => { + const { b } = brain({ discord: [{ summary: 'fix the save crash urgently' }] }); + expect(b.answerFromContext('anything from discord')).toMatch(/1 thing.*save crash/); + }); + + test('workspace question names the active workspace', () => { + const { b } = brain({ workspace: 'Zoo Game' }); + expect(b.answerFromContext('what workspace am i in')).toMatch(/Zoo Game/); + }); + + test('an open-ended request is NOT a fact and falls through', () => { + const { b } = brain(); + expect(b.answerFromContext('spin up a reviewer for PR 12 and ping me when done')).toBeNull(); + }); + + test('an action phrasing is never hijacked by the fact lane', () => { + const { b } = brain({ queue: [{ id: '1', title: 'x' }] }); + // "open the queue" is a command/action, not a "how big is the queue" question. + expect(b.answerFromContext('open the queue')).toBeNull(); + expect(b.answerFromContext('start a reviewer on the queue')).toBeNull(); + }); +}); + +describe('VoiceBrainService — routing', () => { + test('a fact question is answered and spoken, never forwarded', async () => { + const { b, spoken, forwarded } = brain({ sessions: [{ sessionId: 'a', status: 'busy' }] }); + const out = await b.handleUnmatched('how many agents are running'); + expect(out.route).toBe('fact'); + expect(out.handled).toBe(true); + expect(spoken[0]).toMatch(/1 agent/); + expect(forwarded).toHaveLength(0); + }); + + test('an open-ended request goes to the Commander and is acknowledged aloud', async () => { + const { b, spoken, forwarded } = brain(); + const out = await b.handleUnmatched('create a new worktree and start a reviewer on PR 12'); + expect(out.route).toBe('commander'); + expect(out.handled).toBe(true); + expect(forwarded[0]).toMatch(/create a new worktree/); + expect(spoken[0]).toMatch(/on it/i); + }); + + test('with no Commander running, it says so instead of failing silently', async () => { + const b = new VoiceBrainService({ logger: { warn() {} } }); + const spoken = []; + b.init({ speechService: { speak: (t) => spoken.push(t) }, commanderContextService: { getSnapshot: () => ({}) } }); + const out = await b.handleUnmatched('do something open ended'); + expect(out.route).toBe('none'); + expect(spoken[0]).toMatch(/no Commander/i); + }); + + test('confirmCommand speaks a short confirmation for the fast command lane', () => { + const { b, spoken } = brain(); + b.confirmCommand('open-queue'); + expect(spoken[0]).toMatch(/done.*open queue/i); + }); +}); From 5dc2e2197c99c4604b8c0bbd14efa037311bc558 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 09:35:12 +1000 Subject: [PATCH 42/69] docs: record the voice brain in the codebase map --- CODEBASE_DOCUMENTATION.md | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/CODEBASE_DOCUMENTATION.md b/CODEBASE_DOCUMENTATION.md index 7e42c585..63cee551 100644 --- a/CODEBASE_DOCUMENTATION.md +++ b/CODEBASE_DOCUMENTATION.md @@ -159,6 +159,11 @@ server/voice/voiceProviderService.js - Swappable voice-model registry (a model i ├─ Resolves the ONE active provider per capability (tts/stt/duplex); `auto` = best-quality available, `none` = off; a broken pin falls back to auto so voice never silently mutes ├─ `setActive(kind, id)` persists to the override and applies TTS to speechService immediately └─ speechService gained a Kokoro/generic-CLI backend + Piper voice auto-discovery (`~/.local/share/piper-voices`) +server/voice/voiceBrainService.js - The voice "brain": routes an utterance through three lanes, fastest first +├─ COMMAND — a semantic command in the registry, run instantly (voiceCommandService) +├─ FACT — a question answerable from live orchestrator state (sessions/supervisor briefing/queue/discord/workspace) answered straight from a commanderContextService snapshot: an API shortcut, no LLM turn, spoken in ~ms +├─ AGENT — anything else handed to the Commander (full API, can do anything), acknowledged aloud +└─ An action phrasing ("open the queue") never gets hijacked by the fact lane; wired via voiceCommandService.setBrain() server/routes/voiceProviderRoutes.js - `/api/voice-providers/*` (list+health, swap active per capability, reload) config/voice-providers.json - The model catalogue: TTS (browser/piper/kokoro/chatterbox/espeak), STT (whisper-cpp/faster-whisper/parakeet/moonshine), duplex (codex/personaplex/xtalk) with install hints tests/unit/speechService.test.js, voiceProviderService.test.js - Sanitization/backends; registry load, health, auto-resolution, live swap From ad209a219361d264b615dd43f32a043c204008bc Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 10:17:13 +1000 Subject: [PATCH 43/69] =?UTF-8?q?feat(voice):=20speak=20the=20Commander's?= =?UTF-8?q?=20reply=20back=20=E2=80=94=20full=20two-way=20loop?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The agent lane now completes the conversation. On an open-ended request the brain acks immediately ('On it.'), then in the BACKGROUND watches the Commander's PTY buffer until its output settles, extracts the assistant's actual prose out of the Claude Code TUI (strips ANSI, box-drawing, spinners, chrome, the echoed request), and speaks it aloud. The request returns instantly; the answer arrives when the agent is done — a real back-and-forth, not a dead-end ack. extractAssistantReply is a pure, tested function (noisy-TUI-in, clean-prose-out, null when there's no prose so it never speaks garbage). Co-Authored-By: Claude Fable 5 --- server/index.js | 2 +- server/voice/voiceBrainService.js | 70 ++++++++++++++++++++++++++-- tests/unit/voiceBrainService.test.js | 39 ++++++++++++++++ 3 files changed, 106 insertions(+), 5 deletions(-) diff --git a/server/index.js b/server/index.js index 27ac7aa4..bb0db2bc 100644 --- a/server/index.js +++ b/server/index.js @@ -504,7 +504,7 @@ voiceBrainService.init({ speechService, commanderContextService, workspaceManager, - commanderService, + commanderService, // used to read the Commander's PTY buffer for spoken replies commandRegistry, supervisorService, discordWatchService, diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index ad9e79f4..70d45f74 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -142,6 +142,54 @@ class VoiceBrainService { return null; } + /** + * Pull the Commander's actual reply out of its PTY buffer and reduce it to + * something speakable. Claude Code renders a full-screen TUI, so this strips + * ANSI + box-drawing + chrome and keeps the last few lines of real prose. + */ + extractAssistantReply(fullText, beforeText = '') { + let text = String(fullText || ''); + // Only the output produced AFTER the request was sent. + if (beforeText && text.startsWith(beforeText)) text = text.slice(beforeText.length); + + const cleaned = text + .replace(/\x1b\][^\x07\x1b]*(\x07|\x1b\\)/g, '') // OSC sequences + .replace(/\x1b\[[0-9;?]*[ -/]*[@-~]/g, '') // CSI (colour/cursor) + .replace(/\x1b[()][AB0]/g, '') // charset selects + .replace(/[│┃─━╭╮╰╯├┤┬┴┼█▀▄▌▐░▒▓·]/g, ' ') // box-drawing/blocks + .replace(/\r/g, '\n'); + + const noise = /^(>|\?|·|✢|✳|✻|✽|\*|╭|╰|\||\s*esc |\s*⏵|\s*⎿|\s*⧉|tokens|context|auto-|\/|shift\+|ctrl\+|\d+ tokens|✔|✳️)/i; + const lines = cleaned.split('\n') + .map((l) => l.trim()) + .filter((l) => l.length >= 12 && /[a-z]{4,}/i.test(l) && !noise.test(l)) + // Drop obvious UI/status strings that slip through. Match whole + // status phrases, not bare words like "working" that appear in real prose. + .filter((l) => !/dangerously|skip permissions|welcome to claude|bypassing permissions|esc to interrupt|press up|for shortcuts|^\s*(thinking|working|processing)[.…\s]*$/i.test(l)); + + if (!lines.length) return null; + // The final assistant answer is at the tail; take the last couple of prose lines. + return lines.slice(-2).join(' ').slice(0, 360); + } + + async captureCommanderReply(beforeText, { maxWaitMs = 25000, settleMs = 2500, pollMs = 1000 } = {}) { + const cs = this.deps.commanderService; + if (!cs?.getRecentOutput) return null; + + const sleep = (ms) => new Promise((r) => setTimeout(r, ms)); + const startedAt = Date.now(); + let last = ''; + let lastChangeAt = Date.now(); + + while (Date.now() - startedAt < maxWaitMs) { + await sleep(pollMs); + const now = cs.getRecentOutput(150) || ''; + if (now !== last) { last = now; lastChangeAt = Date.now(); } + else if (last && Date.now() - lastChangeAt >= settleMs) break; // output settled + } + return this.extractAssistantReply(last, beforeText); + } + speak(text, { priority = 'normal' } = {}) { if (!text) return { spoken: false }; try { @@ -168,14 +216,28 @@ class VoiceBrainService { // Agent lane: the Commander has the whole API and can do anything. const forwarder = this.deps.commanderForwarder; if (typeof forwarder === 'function') { - const brief = `[voice] ${String(transcript || '').trim()}\n\nAnswer or act using the orchestrator API (GET /api/commander/context, /capabilities, POST /execute). Keep any spoken reply to one or two sentences.`; + const brief = `[voice] ${String(transcript || '').trim()}\n\nAnswer or act using the orchestrator API (GET /api/commander/context, /capabilities, POST /execute). Keep your reply to one or two sentences of plain prose so it can be read aloud.`; + const before = this.deps.commanderService?.getRecentOutput?.(150) || ''; let delivered = false; try { delivered = (await forwarder(brief)) !== false; } catch (error) { this.logger.warn?.('voice brain: commander forward failed', { error: error.message }); } - const ack = delivered ? 'On it — working on that now.' : 'No Commander is running to take that.'; - this.speak(ack); - return { handled: delivered, route: 'commander', spoken: ack }; + + if (!delivered) { + const miss = 'No Commander is running to take that.'; + this.speak(miss); + return { handled: false, route: 'commander', spoken: miss }; + } + + // Immediate ack, then speak the Commander's actual reply once it settles — + // in the background, so the request returns now and the answer arrives when + // the agent is done. This is the two-way loop. + this.speak('On it.'); + this.captureCommanderReply(before) + .then((reply) => { if (reply) this.speak(reply, { priority: 'high' }); }) + .catch((error) => this.logger.warn?.('voice brain: reply capture failed', { error: error.message })); + + return { handled: true, route: 'commander', spoken: 'On it.' }; } const miss = "I couldn't do that myself and there's no Commander running to hand it to."; diff --git a/tests/unit/voiceBrainService.test.js b/tests/unit/voiceBrainService.test.js index 31f6063e..c3c09c07 100644 --- a/tests/unit/voiceBrainService.test.js +++ b/tests/unit/voiceBrainService.test.js @@ -102,3 +102,42 @@ describe('VoiceBrainService — routing', () => { expect(spoken[0]).toMatch(/done.*open queue/i); }); }); + +describe('VoiceBrainService — Commander reply capture', () => { + test('extracts the assistant prose out of a noisy Claude Code TUI buffer', () => { + const { b } = brain(); + // A realistic-ish PTY buffer: ANSI colour, box-drawing chrome, a spinner + // line, and the actual answer at the tail. + const buf = [ + '\x1b[2m╭──────────────────────────────────────╮\x1b[0m', + '\x1b[2m│ > [voice] how is the fleet doing │\x1b[0m', + '\x1b[2m╰──────────────────────────────────────╯\x1b[0m', + '\x1b[33m✻ Thinking…\x1b[0m', + '⏵⏵ bypassing permissions', + '\x1b[1mThree agents are working and one is waiting on a permission prompt.\x1b[0m', + 'Nothing needs you right now.', + '\x1b[2m esc to interrupt · ⏵ for shortcuts\x1b[0m' + ].join('\n'); + const reply = b.extractAssistantReply(buf); + expect(reply).toMatch(/Three agents are working/); + expect(reply).toMatch(/Nothing needs you/); + expect(reply).not.toMatch(/esc to interrupt|bypassing|Thinking|╭|│/); + }); + + test('a buffer with no prose yields null rather than speaking garbage', () => { + const { b } = brain(); + expect(b.extractAssistantReply('\x1b[2m╭───╮\x1b[0m\n│ > │\n╰───╯\n✻ Thinking…')).toBeNull(); + }); + + test('open-ended request acks immediately and captures the reply in the background', async () => { + const { b, spoken } = brain(); + // Buffer is empty when the request is sent, then the answer appears and settles. + let calls = 0; + b.deps.commanderService = { getRecentOutput: () => (calls++ === 0 ? '' : 'The build passed and the PR is open.') }; + const out = await b.handleUnmatched('run the tests and tell me if they pass'); + expect(out.route).toBe('commander'); + expect(spoken[0]).toMatch(/on it/i); // immediate ack returned synchronously + await new Promise((r) => setTimeout(r, 5200)); // poll(1s) + settle(2.5s) -> reply ~4s later + expect(spoken.some((s) => /build passed/i.test(s))).toBe(true); // reply spoken later + }, 30000); +}); From 830c4cbaa09513493dda36101f21ea21c87f47e9 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 10:35:21 +1000 Subject: [PATCH 44/69] fix(voice): force JSON output from the local LLM so small models classify reliably parseWithOllama sent a free-form prompt; llama3.2:1b rambled prose and every fuzzy phrasing fell through to 'no match'. Add Ollama's format:'json' (constrains generation to the {"command":...} grammar at the API level) and widen the timeout. Verified with llama3.2:3b: 'pull up the queue for me' -> open-queue, 'open the settings please' -> open-settings, etc. --- server/voiceCommandService.js | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index a0131e68..66523ffc 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -1482,12 +1482,15 @@ JSON:`; model: this.ollamaModel, prompt, stream: false, + // Constrain generation to valid JSON at the API level so even a small + // local model can't ramble prose instead of the {"command":...} shape. + format: 'json', options: { temperature: 0.1, num_predict: 100 } }), - signal: AbortSignal.timeout(5000) + signal: AbortSignal.timeout(8000) }); if (!response.ok) return null; From c0cd9538bbd711b92d2c7dfb0563f4d175b3c6ef Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 10:38:54 +1000 Subject: [PATCH 45/69] =?UTF-8?q?docs:=20full=20session=20handoff=20?= =?UTF-8?q?=E2=80=94=20PR=20#1029=20fixes=20+=20voice=20system,=20how=20to?= =?UTF-8?q?=20run/test?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- PLANS/2026-07-27/HANDOFF.md | 176 ++++++++++++++++++++++++++++++++++++ 1 file changed, 176 insertions(+) create mode 100644 PLANS/2026-07-27/HANDOFF.md diff --git a/PLANS/2026-07-27/HANDOFF.md b/PLANS/2026-07-27/HANDOFF.md new file mode 100644 index 00000000..0fac4881 --- /dev/null +++ b/PLANS/2026-07-27/HANDOFF.md @@ -0,0 +1,176 @@ +# Handoff — PR #1029 review + voice system (2026-07-27) + +For the next agent. Everything done this session, why, how it's wired, how to run and +test it, what's verified, and what's left. Branch: `feature/autopilot-voice-and-repo-atlas` +(PR #1029). Worktree: `~/GitHub/tools/automation/agent-workspace/work1`. + +## TL;DR of what this session did + +1. **Reviewed PR #1029 and fixed ~20 real bugs** (crashes, a public-repo privacy leak, an + RCE path, correctness/durability) — all committed, tested, pushed. See "Part A". +2. **Built a swappable local voice system** on top of the PR's JARVIS/voice work — the user's + ask: talk to JARVIS naturally, full orchestrator visibility, fast API shortcuts, agent + fallback, and models you can swap. See "Part B". This is the newer, less-battle-tested work. +3. **Installed a real local stack on this laptop**: Piper (TTS) + Ollama `llama3.2:3b` (fuzzy + command LLM). Both verified working end to end. + +Tests: **859 unit tests green, 119 suites** (`npm run test:unit`). Was 652 on main. + +--- + +## Environment / running state (as left) + +- **A live JARVIS instance is running** for manual testing: + - Port **5857**, open at `http://localhost:5857` (localhost is required for the browser + mic/speech — a secure context; the Windows browser reaches WSL over localhost, verified). + - Launched with real `HOME` (so agents log in) but an **isolated data dir** + `AGENT_WORKSPACE_DIR=~/.agent-workspace-jarvis-test` so it can't disturb the other + orchestrator running on port 4000 (a different checkout). PID in `/tmp/pr1029-jarvis-server.pid`. + - Env: `CODEX_APP_SERVER=true SUPERVISOR_AUTONOMY=observe OLLAMA_MODEL=llama3.2:3b`. + - Relaunch command (from the worktree root): + ```bash + AGENT_WORKSPACE_DIR=~/.agent-workspace-jarvis-test CODEX_APP_SERVER=true \ + SUPERVISOR_AUTONOMY=observe OLLAMA_MODEL=llama3.2:3b ORCHESTRATOR_PORT=5857 \ + node server/index.js + ``` +- **Ollama is running** (local LLM for fuzzy voice commands): PID in `/tmp/ollama.pid`, + API at `http://localhost:11434`. Binary at `~/.local/ollama/bin/ollama`, wrapper on PATH at + `~/.local/bin/ollama` (sets `LD_LIBRARY_PATH` to `~/.local/ollama/lib`). Models: `llama3.2:3b` + (used) + `llama3.2:1b`. To restart: `~/.local/bin/ollama serve &` then it's ready. +- **Piper (local TTS)** installed via `pip3 install --user --break-system-packages piper-tts`; + voice model at `~/.local/share/piper-voices/en_US-amy-medium.onnx`. speechService + auto-discovers it. Plays to Windows speakers over WSLg (`paplay` → RDPSink — verified). +- ⚠️ **These are NOT the production orchestrator.** Production is `master/` (port 3000, currently + down); a dev instance runs on **4000** from `claude-orchestrator-dev` (a different repo). Do NOT + edit `master/`. Everything here is the `work1` worktree on the feature branch. + +## The user's vision (verbatim intent) + +Voice must NOT be hardcoded phrases only. It should: talk naturally; have full visibility of +what they're working on (like Commander); reply as fast as possible using **API shortcuts** +through the orchestrator for facts, and **LLM/agent** for open-ended things; keep fast commands +for efficiency; and let them ask the agent to do *anything*. "An extension of Commander Claude." +Models must be swappable. They have an **RTX 5090 (32GB)** on their PC (real target) and this +**RTX 3080 laptop (16GB)** for testing. + +--- + +## Part A — PR #1029 review fixes (battle-tested) + +~20 bugs found via a read-only scout swarm + live end-to-end testing, each fixed with a +regression test. Highlights (see `git log`): app-server `error`-notification crash that took +down the whole orchestrator; broken respawn; a **private-repo-name leak into this PUBLIC repo** +(scrubbed the template + skill); a **supervisor auto-approve RCE path** (path-based deny +patterns); atlas quality-null, subscription re-share, non-atomic writes, machine-local config +sync conflict; discord config-merge/NaN/negation/backfill; voice realtime thread filter; several +app-server lifecycle races found by driving a real Codex thread (stop→start "Not initialized", +numeric approval id). All verified live. Full detail is in the PR description on GitHub. + +--- + +## Part B — the voice system (newer; the focus of the last stretch) + +Three layers. Read the files; they're commented. + +### 1. Swappable voice-model registry — a model is DATA, not code +- `config/voice-providers.json` — the catalogue. Each provider: `{ id, kind: tts|stt|duplex, + engine, requires: {command|env|server}, endpoint, quality, install, notes }`. +- `server/voice/voiceProviderService.js` — loads it, **health-checks** each provider (command + present? model server reachable?), resolves the ONE active provider per capability. `auto` = + best-quality available; `none` = off; a broken pin falls back to auto so voice never silently + mutes. TTS health defers to speechService so they never disagree. +- `server/routes/voiceProviderRoutes.js` — `GET /api/voice-providers` (all + health), + `GET /api/voice-providers/:kind`, `POST /api/voice-providers/:kind/active {id}`, `POST .../reload`. +- `server/speechService.js` — gained a Kokoro/generic-CLI TTS backend + Piper voice + auto-discovery. `setActiveEngine()` is the bridge the registry calls so a swap takes effect live. +- **To add a model on the 5090:** add a config entry, install it, `POST + /api/voice-providers//active {"id":"..."}`. PersonaPlex/Qwen/Kokoro/Parakeet are already + registered and light up when present. Full catalogue + install hints: + `PLANS/2026-07-27/LOCAL_VOICE_MODELS_RESEARCH.md`. + +### 2. The voice brain — routing (the "extension of Commander") +`server/voice/voiceBrainService.js`. An utterance goes through three lanes, fastest first +(wired into `voiceCommandService.processVoiceCommand` via `setBrain()`): +- **COMMAND** — a semantic command in the registry, run instantly. Natural phrasing is mapped to + a command by the local LLM (Ollama) — e.g. "pull up the queue" → `open-queue`. Speaks a short + confirmation. +- **FACT** — a question answerable from live orchestrator state (sessions, supervisor briefing, + queue, discord, workspace) answered straight from a `commanderContextService` snapshot — no LLM + turn, spoken in ms. `answerFromContext()`. An **action phrasing** ("open the queue") is guarded + out of this lane so it isn't mistaken for a question. +- **AGENT** — anything else → forwarded to the **Commander** (full orchestrator API, can do + anything). Acks "On it." immediately, then in the BACKGROUND watches the Commander's PTY buffer + until it settles, extracts the assistant's prose (`extractAssistantReply` strips ANSI/TUI + chrome), and **speaks the reply** — the two-way loop. + +### 3. Local models installed + wired +- **Ollama `llama3.2:3b`** with `format:'json'` (forced JSON so a small model classifies + reliably — this was the key fix; 1b rambled). voiceCommandService auto-detects Ollama on boot. +- **Piper** TTS as the active local voice (registry auto-selected it; swap to browser/kokoro/etc). + +--- + +## HOW TO TEST (do this) + +### Automated +```bash +cd ~/GitHub/tools/automation/agent-workspace/work1 +npm run test:unit # 859 tests; voice: voiceProviderService/voiceBrainService/voiceCommandService/speechService.test.js +node --check server/index.js +``` +E2E (`npm run test:e2e:safe`) is **broken in this WSL env on main too** — Playwright+socket.io +never reports `connected`; NOT caused by this branch (proven against origin/main). Noted in +`~/.claude/projects/.../memory/MEMORY.md`. Don't chase it. + +### Manual — the voice system live (instance already up on 5857) +```bash +P=5857 +# fast FACT lane (instant, from live state, spoken): +curl -s -XPOST localhost:$P/api/voice/command -H 'Content-Type: application/json' -d '{"transcript":"how many agents are working"}' +curl -s -XPOST localhost:$P/api/voice/command -H 'Content-Type: application/json' -d '{"transcript":"what needs my attention"}' +# fuzzy COMMAND lane (local LLM maps phrasing -> command): +curl -s -XPOST localhost:$P/api/voice/command -H 'Content-Type: application/json' -d '{"transcript":"can you pull up the queue"}' # -> open-queue +# swap the voice model live: +curl -s localhost:$P/api/voice-providers # see all + health + active +curl -s -XPOST localhost:$P/api/voice-providers/tts/active -d '{"id":"browser"}' -H 'Content-Type: application/json' +# make it SPEAK (plays to Windows speakers via WSLg, backend=piper): +curl -s -XPOST localhost:$P/api/speech/say -H 'Content-Type: application/json' -d '{"text":"handoff test","force":true}' +``` +In the browser (`http://localhost:5857`, Chrome/Edge): press **Alt+J** for the JARVIS panel; +hold **V** and speak (grant mic). The transcript pill shows what you said. + +### AGENT lane needs a Commander running +The "do anything + spoken reply" lane forwards to the Commander. Start one: +```bash +curl -s -XPOST localhost:5857/api/commander/start -d '{}' -H 'Content-Type: application/json' +sleep 2 +curl -s -XPOST localhost:5857/api/commander/start-claude -d '{"mode":"fresh","yolo":true}' -H 'Content-Type: application/json' +# accept the trust prompt: send "1" then "\r" via /api/commander/input +``` +Then an open-ended voice request ("summarise the fleet and tell me when done") gets acked "On it." +and the Commander's reply is spoken ~4s after its output settles. + +--- + +## WHAT'S LEFT / KNOWN LIMITS + +- **The 5090 models** (PersonaPlex full-duplex, Qwen omni, Kokoro/Parakeet) are registered + + documented but not installed here (too big for 16GB). On the PC: install per the `install` + hint in `config/voice-providers.json`, then `POST /api/voice-providers//active`. +- **True full-duplex** (barge-in, talk over it) is the PersonaPlex/`duplex` provider — the current + loop is turn-based (STT → route → TTS). Duplex adapter is registered (`personaplex`/`xtalk`, + websocket to a local model server) but not yet driving audio; that's the next real build. +- **extractAssistantReply is heuristic** — it scrapes Claude's TUI buffer. Works in tests + simple + cases; a cleaner path would be a structured output channel (the Codex app-server already gives + `turn/completed` + transcript — using Codex as the voice agent would remove the scraping). +- **Ollama must be running** for fuzzy command matching; if it's down, voice falls back to exact + rule phrasings only (still works, just less forgiving). Restart: `~/.local/bin/ollama serve &`. +- **Persistence:** Ollama + Piper are user-local installs; they don't auto-start on reboot. If the + user wants them always-on, add a WSL startup hook (out of scope this session). +- The **app-server thread↔session linkage** is still dormant (documented in the design doc) — the + supervisor runs on PTY signals; structured Codex signals need a session→threadId link. + +## Cleanup note +Temp files under `/tmp/pr1029-*` and `/tmp/ollama-*` are throwaway. The isolated data dir +`~/.agent-workspace-jarvis-test` and the Ollama/Piper installs under `~/.local` are intentional +(the working local voice stack) — keep them. From 22bba41a8fbebfb6a185f521800fb1641311f83f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 12:59:22 +1000 Subject: [PATCH 46/69] fix(voice): stop the local LLM jamming chit-chat into commands Live testing exposed 'hello can you hear me' -> switch-workspace and 'thanks that is cool' -> open-project-chats: a small model forced to emit JSON picks a command for everything. Two guards: - Prompt: explicit 'return {command:null} for greetings/questions/small talk', with examples. - Grounding: reject a classified command unless the utterance actually shares a keyword with it (isGrounded), so a hallucinated command flows to the brain's greeting/fact/agent lanes instead of firing. Plus a greeting/presence handler in the brain ('are you there' -> 'Yes, I'm here...'). Verified: chit-chat -> spoken reply, real commands still fire. Also fixed the piper wrapper (dangling venv symlink -> python -m piper), so the local neural voice actually plays instead of falling back to the robotic browser voice. Co-Authored-By: Claude Fable 5 --- server/voice/voiceBrainService.js | 8 +++++++ server/voiceCommandService.js | 39 ++++++++++++++++++++++++++++--- 2 files changed, 44 insertions(+), 3 deletions(-) diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index 70d45f74..148e605b 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -97,6 +97,14 @@ class VoiceBrainService { return null; } + // Greetings / presence checks — answer as a person would, not with a command. + if (/^(hi|hey|hello|yo|howdy|greetings)\b|are you (there|awake|up|listening)|can you hear me|you there/.test(t)) { + const c = this.countSessions(ctx.sessions); + return c.total + ? `Yes, I'm here. ${c.busy} agent${c.busy === 1 ? '' : 's'} working right now. What do you need?` + : "Yes, I'm here and listening. What do you need?"; + } + // What needs me / what's wrong / status of the fleet if (/(needs?|need)\s+(me|my|your)|attention|anything (wrong|broken|stuck|urgent)|what.*(should i|do i need)/.test(t)) { return ctx.supervisor?.spoken || 'Nothing needs you right now. Everything else was handled.'; diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 66523ffc..54a615f7 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -1201,7 +1201,7 @@ class VoiceCommandService { // Try Ollama first (local, private) if (this.useOllama) { const ollamaResult = await this.parseWithOllama(text); - if (ollamaResult) { + if (ollamaResult && this.isGrounded(ollamaResult.command, text)) { return { success: true, method: 'ollama', @@ -1213,7 +1213,7 @@ class VoiceCommandService { // Try Claude API as fallback (fast, cheap) if (this.useClaude) { const claudeResult = await this.parseWithClaude(text); - if (claudeResult) { + if (claudeResult && this.isGrounded(claudeResult.command, text)) { return { success: true, method: 'claude', @@ -1229,6 +1229,29 @@ class VoiceCommandService { }; } + /** + * Grounding guard against LLM hallucination. A small local model, forced to + * emit JSON, will sometimes pick a command for input that isn't one ("thanks + * that is cool" -> open-project-chats). Only trust a classified command if the + * utterance actually shares a keyword with it — otherwise treat it as no match + * so it flows to the brain's fact/greeting/agent lanes instead of firing a + * random command. + */ + isGrounded(command, text) { + if (!command) return false; + const t = String(text || '').toLowerCase(); + const synonyms = { + queue: ['queue'], workspace: ['workspace', 'project', 'switch'], settings: ['setting', 'config', 'preference'], + tasks: ['task', 'todo'], commander: ['commander'], worktree: ['worktree', 'work'], focus: ['focus', 'show'], + pager: ['pager', 'ping'], advice: ['advice', 'recommend', 'suggest'], chats: ['chat', 'message'], + project: ['project'], claude: ['claude'], all: ['all', 'everything'], mode: ['mode'], tier: ['tier'], + new: ['new', 'create', 'start'], open: ['open', 'show', 'pull up', 'bring up', 'go to'], start: ['start', 'launch', 'run'], + stop: ['stop', 'kill', 'end'], status: ['status', 'state'] + }; + const parts = String(command).split(/[-_]/).filter((w) => w.length > 2); + return parts.some((w) => t.includes(w) || (synonyms[w] || []).some((s) => t.includes(s))); + } + /** * Rule-based command parsing */ @@ -1441,7 +1464,17 @@ Worktree matching: - Match partial names: "zoo" could match "zoo-game" Return JSON: {"command": "command-name", "params": {"key": "value"}} -Return {"command": null} if unclear. + +CRITICAL: Only return a command when the user is CLEARLY asking to perform one of the +actions above. If the input is a greeting, a question, small talk, a status query, or +anything that is not obviously one of the listed commands, you MUST return {"command": null}. +Never guess. When in doubt, return {"command": null}. + +Examples of {"command": null}: +- "hello", "hey jarvis", "can you hear me", "are you there" (greetings) +- "how many agents are working", "what needs my attention", "what's the status" (questions) +- "thanks", "cool", "never mind", "what can you do" (chit-chat) +- "write a note summarising the fleet" (a task, not a listed command) JSON:`; } From 21638e257a5da660ac89ee56de1fffe398351d9c Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 13:01:07 +1000 Subject: [PATCH 47/69] fix(voice): order fact lanes before the greeting fallback 'hey what needs me' was swallowed by the greeting matcher. Greetings now run last, so a real question opening with 'hey' hits its specific lane first. --- server/voice/voiceBrainService.js | 17 +++++++++-------- tests/unit/voiceBrainService.test.js | 8 ++++++++ 2 files changed, 17 insertions(+), 8 deletions(-) diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index 148e605b..c7a7d768 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -97,14 +97,6 @@ class VoiceBrainService { return null; } - // Greetings / presence checks — answer as a person would, not with a command. - if (/^(hi|hey|hello|yo|howdy|greetings)\b|are you (there|awake|up|listening)|can you hear me|you there/.test(t)) { - const c = this.countSessions(ctx.sessions); - return c.total - ? `Yes, I'm here. ${c.busy} agent${c.busy === 1 ? '' : 's'} working right now. What do you need?` - : "Yes, I'm here and listening. What do you need?"; - } - // What needs me / what's wrong / status of the fleet if (/(needs?|need)\s+(me|my|your)|attention|anything (wrong|broken|stuck|urgent)|what.*(should i|do i need)/.test(t)) { return ctx.supervisor?.spoken || 'Nothing needs you right now. Everything else was handled.'; @@ -147,6 +139,15 @@ class VoiceBrainService { return `I can run about ${ctx.capabilities || 'a set of'} orchestrator commands directly, answer questions about your fleet, and hand anything else to the Commander to work on.`; } + // Greetings / presence checks — LAST, so a real question that merely opens + // with "hey" ("hey what needs me") is matched by the specific lanes first. + if (/^(hi|hey|hello|yo|howdy|greetings|good (morning|afternoon|evening))\b|are you (there|awake|up|listening|around)|can you hear me|you (there|up)/.test(t)) { + const c = this.countSessions(ctx.sessions); + return c.total + ? `Yes, I'm here. ${c.busy} agent${c.busy === 1 ? '' : 's'} working right now. What do you need?` + : "Yes, I'm here and listening. What do you need?"; + } + return null; } diff --git a/tests/unit/voiceBrainService.test.js b/tests/unit/voiceBrainService.test.js index c3c09c07..b1723263 100644 --- a/tests/unit/voiceBrainService.test.js +++ b/tests/unit/voiceBrainService.test.js @@ -60,6 +60,14 @@ describe('VoiceBrainService — fact lane', () => { expect(b.answerFromContext('spin up a reviewer for PR 12 and ping me when done')).toBeNull(); }); + test('a pure greeting gets a friendly reply, but "hey " still hits the question lane', () => { + const { b } = brain({ sessions: [{ sessionId: 'a', status: 'busy' }], briefing: { spoken: 'work3 is waiting on a permission.' } }); + expect(b.answerFromContext('hello can you hear me')).toMatch(/i'm here/i); + expect(b.answerFromContext('are you there')).toMatch(/i'm here/i); + // A real question that merely opens with "hey" must not be swallowed as a greeting. + expect(b.answerFromContext('hey what needs me right now')).toMatch(/waiting on a permission/); + }); + test('an action phrasing is never hijacked by the fact lane', () => { const { b } = brain({ queue: [{ id: '1', title: 'x' }] }); // "open the queue" is a command/action, not a "how big is the queue" question. From 3315f3a2b6939975ed13f6833e56a9af4e49af75 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 13:10:42 +1000 Subject: [PATCH 48/69] fix(voice): stream the neural voice to the browser so it's actually audible on WSL Root cause of 'I don't hear anything': server-side PulseAudio (WSLg) doesn't reach the user's Windows speakers, but browser audio does (the browser TTS was audible). So when a browser is connected, synthesize with piper server-side and STREAM the WAV over the socket ('speech-audio'); the client plays it (Audio element, high-priority clips interrupt). Falls back to paplay only when nothing is listening in a browser. Reads the model's real sample rate from its .json. Verified: valid 126KB RIFF/WAVE emitted; 2 clients receive it. Co-Authored-By: Claude Fable 5 --- client/speech-output.js | 24 +++++++++++++++- server/speechService.js | 63 ++++++++++++++++++++++++++++++++++++++++- 2 files changed, 85 insertions(+), 2 deletions(-) diff --git a/client/speech-output.js b/client/speech-output.js index 6e536dc1..8a9b93c0 100644 --- a/client/speech-output.js +++ b/client/speech-output.js @@ -12,7 +12,8 @@ const state = { enabled: localStorage.getItem('speechOutputEnabled') !== 'false', voiceName: localStorage.getItem('speechOutputVoice') || '', - rate: Number(localStorage.getItem('speechOutputRate')) || 1.05 + rate: Number(localStorage.getItem('speechOutputRate')) || 1.05, + currentAudio: null }; function pickVoice() { @@ -42,10 +43,31 @@ return true; } + // Play server-synthesized neural audio (piper/kokoro) streamed as a WAV. On + // WSL the server can't reach the speakers, so it hands the bytes to us — and + // browser audio always reaches the user. A high-priority clip interrupts. + function playAudio(payload) { + const b64 = payload?.wav; + if (!b64) return false; + try { + if (payload.priority === 'high' && state.currentAudio) { + state.currentAudio.pause(); + state.currentAudio = null; + } + const audio = new Audio(`data:audio/wav;base64,${b64}`); + state.currentAudio = audio; + audio.play().catch(() => { /* autoplay blocked until a user gesture */ }); + return true; + } catch { + return false; + } + } + function attach(socket) { if (!socket || socket.__speechOutputAttached) return; socket.__speechOutputAttached = true; socket.on('speech-speak', (payload) => speak(payload?.text, { priority: payload?.priority })); + socket.on('speech-audio', (payload) => playAudio(payload)); } window.SpeechOutput = { diff --git a/server/speechService.js b/server/speechService.js index f276fd4b..e9ac896a 100644 --- a/server/speechService.js +++ b/server/speechService.js @@ -67,6 +67,14 @@ class SpeechService { this.enabled = String(process.env.SPEECH_ENABLED || 'true').toLowerCase() !== 'false'; this.preferredBackend = String(process.env.SPEECH_BACKEND || '').trim().toLowerCase(); this.piperModel = String(process.env.PIPER_MODEL || '').trim() || discoverPiperModel(); + // Piper models declare their sample rate in the companion .onnx.json. + this.piperSampleRate = 22050; + try { + if (this.piperModel && fs.existsSync(`${this.piperModel}.json`)) { + const cfg = JSON.parse(fs.readFileSync(`${this.piperModel}.json`, 'utf8')); + this.piperSampleRate = Number(cfg?.audio?.sample_rate) || 22050; + } + } catch { /* keep the default */ } this.voice = String(process.env.SPEECH_VOICE || '').trim(); // A generic local TTS CLI (kokoro-tts, chatterbox, …): reads text on stdin, // writes raw s16le PCM on stdout. Set by the provider registry when one of @@ -162,6 +170,53 @@ class SpeechService { return { spoken: true }; } + /** Wrap raw s16le mono PCM in a minimal WAV container for browser playback. */ + pcmToWav(pcm, sampleRate = 22050) { + const header = Buffer.alloc(44); + header.write('RIFF', 0); + header.writeUInt32LE(36 + pcm.length, 4); + header.write('WAVE', 8); + header.write('fmt ', 12); + header.writeUInt32LE(16, 16); // PCM chunk size + header.writeUInt16LE(1, 20); // format = PCM + header.writeUInt16LE(1, 22); // mono + header.writeUInt32LE(sampleRate, 24); + header.writeUInt32LE(sampleRate * 2, 28); // byte rate (16-bit mono) + header.writeUInt16LE(2, 32); // block align + header.writeUInt16LE(16, 34); // bits per sample + header.write('data', 36); + header.writeUInt32LE(pcm.length, 40); + return Buffer.concat([header, pcm]); + } + + /** + * Synthesize with piper and STREAM the audio to the browser to play, instead + * of server-side paplay. On WSL, server-side PulseAudio (WSLg) often doesn't + * reach the user's speakers, but browser audio always does — so this is the + * reliable way to hear a local neural voice. Falls back to nothing if piper + * or the model is missing (resolveBackend won't pick this backend then). + */ + speakViaPiperBrowser(text, priority) { + if (!this.io) return { spoken: false, reason: 'no socket connection to a client' }; + try { + const env = augmentProcessEnv(process.env); + const piper = spawn('piper', ['--model', this.piperModel, '--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); + const chunks = []; + piper.stdout.on('data', (d) => chunks.push(d)); + piper.on('error', (error) => this.logger.warn?.('Piper (browser) failed', { error: error.message })); + piper.on('close', () => { + const pcm = Buffer.concat(chunks); + if (!pcm.length) return; + const wav = this.pcmToWav(pcm, this.piperSampleRate || 22050).toString('base64'); + this.io.emit('speech-audio', { wav, priority, at: new Date().toISOString() }); + }); + piper.stdin.end(`${text}\n`); + return { spoken: true }; + } catch (error) { + return { spoken: false, reason: error.message }; + } + } + spawnQuiet(command, args) { try { const child = spawn(command, args, { @@ -240,7 +295,13 @@ class SpeechService { const binary = commandExists('espeak-ng') ? 'espeak-ng' : 'espeak'; return this.spawnQuiet(binary, [text]); } - if (backendId === 'piper') return this.speakViaPiper(text); + // On WSL, server-side PulseAudio usually can't reach the speakers, so when a + // browser is connected, stream piper's audio there (reliably audible). Only + // fall back to server-side paplay when nothing is listening in a browser. + if (backendId === 'piper') { + const hasClient = Number(this.io?.engine?.clientsCount ?? 0) > 0; + return hasClient ? this.speakViaPiperBrowser(text) : this.speakViaPiper(text); + } if (backendId === 'kokoro') return this.speakViaCli(this.cliEngine, text); if (backendId === 'sapi') { // Text is already sanitized to printable ASCII with no shell metacharacters; From 35b707773d13448c9488f5fc897983ee90542e5b Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 13:11:20 +1000 Subject: [PATCH 49/69] =?UTF-8?q?docs:=20handoff=20=E2=80=94=20WSL=20brows?= =?UTF-8?q?er-audio=20streaming=20for=20the=20local=20voice?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- PLANS/2026-07-27/HANDOFF.md | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/PLANS/2026-07-27/HANDOFF.md b/PLANS/2026-07-27/HANDOFF.md index 0fac4881..ab07ac65 100644 --- a/PLANS/2026-07-27/HANDOFF.md +++ b/PLANS/2026-07-27/HANDOFF.md @@ -152,6 +152,18 @@ and the Commander's reply is spoken ~4s after its output settles. --- +## AUDIO ON WSL (important — the "I don't hear anything" fix) + +Server-side PulseAudio (WSLg → RDPSink) reliably plays a test tone but often does **not** +reach the user's Windows speakers; **browser audio always does**. So the local neural voice +(piper) is synthesized server-side and the **WAV is streamed over the socket** (`speech-audio` +event) for the browser to play — see `speechService.speakViaPiperBrowser` + +`client/speech-output.js playAudio`. It only falls back to server-side `paplay` when no browser +client is connected. Requires the browser tab open + one prior user gesture (autoplay policy). +Caveat: the piper CLI here is `python3 -m piper` (wrapper at `~/.local/bin/piper`), which +cold-starts ~4-9s per utterance — functional but not snappy. **Responsiveness fix for later:** +keep a warm piper process, or use the piper C++ binary, or move to Kokoro/PersonaPlex. + ## WHAT'S LEFT / KNOWN LIMITS - **The 5090 models** (PersonaPlex full-duplex, Qwen omni, Kokoro/Parakeet) are registered + From 5a596436154abee4e05c9705498dda9b1e37c9c7 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 13:26:20 +1000 Subject: [PATCH 50/69] perf(voice): fact lane before the LLM + prewarm + auto-start Commander MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Questions ('how many agents working') were paying the multi-second LLM command-classifier BEFORE reaching the instant fact lane — 8034ms observed, worst case an 8s cold-model timeout. Answer facts from the snapshot FIRST; measured 15-24ms now. The LLM classifier only runs for genuine command phrasings. - Prewarm the Ollama model on boot + keep_alive 30m so the first real command doesn't eat the cold load; warm classification ~1.7s. - sendToCommander auto-starts the Commander if none is running (launch queue buffers the request through boot) instead of dead-ending on 'no Commander'. - Log each utterance: heard / route / command / reply / ms. Co-Authored-By: Claude Fable 5 --- server/index.js | 34 +++++++++++++++++++++++++++++++ server/voice/voiceBrainService.js | 2 +- server/voiceCommandService.js | 33 ++++++++++++++++++++++++++++++ 3 files changed, 68 insertions(+), 1 deletion(-) diff --git a/server/index.js b/server/index.js index bb0db2bc..af2f9acf 100644 --- a/server/index.js +++ b/server/index.js @@ -447,9 +447,33 @@ threadService.init({ workspaceManager, sessionManager }); intentHaikuService.setSessionManager(sessionManager); serviceStackRuntimeService.init({ workspaceManager, sessionManager, configPromoterService, io }); auditExportService.init({ activityFeed, schedulerService, userSettingsService }); +// Self-healing: if no Commander is running, start one automatically rather than +// dead-ending. The launch queue buffers input during boot and flushes it once +// Claude is interactive (trust prompt auto-accepted), so the request lands even +// on a cold start — the assistant just does what it needs instead of reporting +// a missing Commander. +let commanderStarting = null; +const ensureCommander = async () => { + try { + if (!commanderService?.start) return false; + if (!commanderService.session) { + if (!commanderStarting) commanderStarting = commanderService.start().finally(() => { commanderStarting = null; }); + await commanderStarting; + } + if (!commanderService.claudeStarted) { + await commanderService.startClaude('fresh', true); + } + return true; + } catch (error) { + logger.warn('ensureCommander failed', { error: error.message }); + return false; + } +}; + // Two writes: agent CLIs treat "text\r" in one chunk as a bracketed paste. const sendToCommander = async (text) => { if (!commanderService?.sendInput) return false; + await ensureCommander(); if (commanderService.sendInput(text) === false) return false; await new Promise((resolve) => setTimeout(resolve, 300)); commanderService.sendInput('\r'); @@ -8401,7 +8425,17 @@ app.post('/api/voice/command', async (req, res) => { if (!transcript) { return res.status(400).json({ error: 'transcript is required' }); } + const startedAt = Date.now(); const result = await voiceCommandService.processVoiceCommand(transcript); + // Log what was heard, how it routed, what JARVIS said back, and how long it + // took — so the whole conversation + latency is visible in the log. + logger.info('Voice', { + heard: transcript, + route: result.method || (result.success ? 'command' : 'unmatched'), + command: result.command || null, + reply: result.spoken || null, + ms: Date.now() - startedAt + }); res.json(result); } catch (error) { logger.error('Failed to process voice command', { error: error.message }); diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index c7a7d768..8f97eed5 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -181,7 +181,7 @@ class VoiceBrainService { return lines.slice(-2).join(' ').slice(0, 360); } - async captureCommanderReply(beforeText, { maxWaitMs = 25000, settleMs = 2500, pollMs = 1000 } = {}) { + async captureCommanderReply(beforeText, { maxWaitMs = 45000, settleMs = 2500, pollMs = 1000 } = {}) { const cs = this.deps.commanderService; if (!cs?.getRecentOutput) return null; diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 54a615f7..0606c8ba 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -1168,6 +1168,21 @@ class VoiceCommandService { if (!this.useOllama && !this.useClaude) { console.log('[Voice] No LLM available - using rule-based parsing only'); } + + // Pre-warm the model so the FIRST real command doesn't eat the ~8s cold + // model-load. Fire-and-forget; keep_alive holds it in VRAM afterwards. + if (this.useOllama) this.warmUpOllama(); + } + + warmUpOllama() { + if (this._warmed) return; + this._warmed = true; + fetch(`${this.ollamaUrl}/api/generate`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ model: this.ollamaModel, prompt: 'ok', stream: false, keep_alive: '30m', options: { num_predict: 1 } }), + signal: AbortSignal.timeout(60000) + }).then(() => console.log('[Voice] Ollama model warmed:', this.ollamaModel)).catch(() => { this._warmed = false; }); } /** @@ -1198,6 +1213,17 @@ class VoiceCommandService { }; } + // Fast fact lane BEFORE the LLM classifier: a question about live state + // ("how many agents are working") should be answered instantly from a + // snapshot, not pay the multi-second LLM command-classification cost first. + // Action phrasings ("open the queue") return null here and fall through. + if (!options.skipFact && this.brain?.answerFromContext) { + try { + const fact = this.brain.answerFromContext(text); + if (fact) return { success: false, fact, transcript: text }; + } catch { /* fall through to the classifier */ } + } + // Try Ollama first (local, private) if (this.useOllama) { const ollamaResult = await this.parseWithOllama(text); @@ -1518,6 +1544,7 @@ JSON:`; // Constrain generation to valid JSON at the API level so even a small // local model can't ramble prose instead of the {"command":...} shape. format: 'json', + keep_alive: '30m', // keep the model warm between commands options: { temperature: 0.1, num_predict: 100 @@ -1624,6 +1651,12 @@ JSON:`; async processVoiceCommand(transcript, { forwardUnmatched = true } = {}) { const parsed = await this.parseCommand(transcript); + // Fast fact/greeting answer (matched before the LLM classifier) — speak and done. + if (parsed.fact) { + this.brain?.speak?.(parsed.fact); + return { success: true, method: 'fact', command: null, params: {}, transcript, executed: true, spoken: parsed.fact }; + } + if (!parsed.success) { // No command matched. If the brain is wired, let it try a fast fact answer // from live orchestrator state, then fall back to the Commander agent — From 9ce84862a0b11e9297201a8b0391781709cd7db6 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 13:30:31 +1000 Subject: [PATCH 51/69] perf(voice): warm piper HTTP server -> ~0.2s synth instead of ~5s The felt delay was piper cold-starting (python -m piper reloads the model every call, ~5s). Run piper.http_server once (model stays loaded) and POST /synthesize to it (~0.2s), streaming the WAV to the browser. Falls back to spawning piper if the server is down. PIPER_HTTP_URL (default 127.0.0.1:5959). End-to-end synth measured at 242ms. Co-Authored-By: Claude Fable 5 --- server/speechService.js | 69 ++++++++++++++++++++++++++++++----------- 1 file changed, 51 insertions(+), 18 deletions(-) diff --git a/server/speechService.js b/server/speechService.js index e9ac896a..4d79f86c 100644 --- a/server/speechService.js +++ b/server/speechService.js @@ -80,6 +80,8 @@ class SpeechService { // writes raw s16le PCM on stdout. Set by the provider registry when one of // those is the active engine; empty otherwise. this.cliEngine = String(process.env.SPEECH_CLI_ENGINE || '').trim(); + // A warm piper HTTP server (model kept loaded) — ~0.2s synth vs ~5s cold. + this.piperHttpUrl = String(process.env.PIPER_HTTP_URL || 'http://127.0.0.1:5959').replace(/\/$/, ''); this.history = []; this.lastSpokenAt = new Map(); this.backendCache = null; @@ -193,28 +195,59 @@ class SpeechService { * Synthesize with piper and STREAM the audio to the browser to play, instead * of server-side paplay. On WSL, server-side PulseAudio (WSLg) often doesn't * reach the user's speakers, but browser audio always does — so this is the - * reliable way to hear a local neural voice. Falls back to nothing if piper - * or the model is missing (resolveBackend won't pick this backend then). + * reliable way to hear a local neural voice. + * + * Returns {spoken:true} optimistically and does the synth in the background + * (like the spawn path), emitting `speech-audio` when the WAV is ready. */ speakViaPiperBrowser(text, priority) { if (!this.io) return { spoken: false, reason: 'no socket connection to a client' }; - try { - const env = augmentProcessEnv(process.env); - const piper = spawn('piper', ['--model', this.piperModel, '--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); - const chunks = []; - piper.stdout.on('data', (d) => chunks.push(d)); - piper.on('error', (error) => this.logger.warn?.('Piper (browser) failed', { error: error.message })); - piper.on('close', () => { - const pcm = Buffer.concat(chunks); - if (!pcm.length) return; - const wav = this.pcmToWav(pcm, this.piperSampleRate || 22050).toString('base64'); - this.io.emit('speech-audio', { wav, priority, at: new Date().toISOString() }); - }); - piper.stdin.end(`${text}\n`); - return { spoken: true }; - } catch (error) { - return { spoken: false, reason: error.message }; + this.synthAndEmit(text, priority).catch((error) => this.logger.warn?.('Piper synth failed', { error: error.message })); + return { spoken: true }; + } + + emitAudio(wavBuffer, priority) { + if (!wavBuffer?.length) return; + this.io?.emit('speech-audio', { wav: wavBuffer.toString('base64'), priority, at: new Date().toISOString() }); + } + + async synthAndEmit(text, priority) { + // Fast path: a warm piper HTTP server keeps the model loaded (~0.2s synth + // vs ~5s for a cold `python -m piper` per call). Try it first. + if (this.piperHttpUrl) { + try { + const resp = await fetch(`${this.piperHttpUrl}/synthesize`, { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ text }), + signal: AbortSignal.timeout(8000) + }); + if (resp.ok) { + this.emitAudio(Buffer.from(await resp.arrayBuffer()), priority); + return; + } + } catch { + // Server down/unreachable — fall back to spawning piper. + } } + + // Fallback: spawn piper once (cold, slower) and wrap its raw PCM as WAV. + await new Promise((resolve) => { + try { + const env = augmentProcessEnv(process.env); + const piper = spawn('piper', ['--model', this.piperModel, '--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); + const chunks = []; + piper.stdout.on('data', (d) => chunks.push(d)); + piper.on('error', () => resolve()); + piper.on('close', () => { + this.emitAudio(this.pcmToWav(Buffer.concat(chunks), this.piperSampleRate || 22050), priority); + resolve(); + }); + piper.stdin.end(`${text}\n`); + } catch { + resolve(); + } + }); } spawnQuiet(command, args) { From 5fdc51fb31d6983515708e44e75a2e11d123220a Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 13:31:50 +1000 Subject: [PATCH 52/69] =?UTF-8?q?docs:=20handoff=20=E2=80=94=20voice=20lat?= =?UTF-8?q?ency=20stack,=20warm=20piper,=20auto-start=20Commander,=20start?= =?UTF-8?q?=20script?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- PLANS/2026-07-27/HANDOFF.md | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) diff --git a/PLANS/2026-07-27/HANDOFF.md b/PLANS/2026-07-27/HANDOFF.md index ab07ac65..f2ef611e 100644 --- a/PLANS/2026-07-27/HANDOFF.md +++ b/PLANS/2026-07-27/HANDOFF.md @@ -152,6 +152,31 @@ and the Commander's reply is spoken ~4s after its output settles. --- +## LATENCY + VOICE STACK (added late in the session) + +- **Restart the voice backends** with `~/.local/bin/start-voice-stack.sh` — starts Ollama + (`:11434`, fuzzy commands) and a **warm Piper HTTP server** (`:5959`, fast TTS). Launch + JARVIS with `PIPER_HTTP_URL=http://127.0.0.1:5959 OLLAMA_MODEL=llama3.2:3b` (see the relaunch + command above). +- **Routing is now fast-lane-first:** rules → **fact lane** (instant, from a live snapshot) → + LLM command classifier → agent. Measured: fact questions **~20ms** (were 8034ms — they used + to pay the LLM cost first), fuzzy commands ~1.2s (warm model), TTS synth **~0.24s** (warm + piper HTTP server; was ~5s spawning `python -m piper` per call). Each utterance is logged: + `heard / route / command / reply / ms`. +- **Commander auto-starts** (`ensureCommander` in index.js) — an open-ended request no longer + dead-ends on "no Commander"; the launch queue buffers the request through boot. +- **Commander DOES speak back:** the brain captures its PTY reply once output settles + (`captureCommanderReply` + `extractAssistantReply`) and speaks it (~4s+ after a cold boot). + +### Remaining latency/quality items +- **Command accuracy:** the 3B model sometimes picks a WRONG command ("pull up the queue" -> + a different queue command). The fact lane is reliable; the fuzzy *command* lane is model- + limited. Fix: a better model (e.g. `qwen2.5:7b-instruct`, fits the 16GB laptop) — pull it and + set `OLLAMA_MODEL`. The grounding guard (`isGrounded`) blocks unrelated commands but can't + distinguish two commands in the same family. +- **Kokoro / PersonaPlex** for a nicer / full-duplex voice — registered in the config, not + installed. PersonaPlex serves its own browser audio (sidesteps the WSLg issue entirely). + ## AUDIO ON WSL (important — the "I don't hear anything" fix) Server-side PulseAudio (WSLg → RDPSink) reliably plays a test tone but often does **not** From d3f6e9ad348b35a09dbda72b63596cd566c6678a Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:11:45 +1000 Subject: [PATCH 53/69] fix(voice): answer identity questions instantly, not via the Commander 'what is your name' was routing to the Commander agent (8s -> 'On it.'). Add an identity handler to the fact lane: 'who/what are you', 'your name' -> instant 'I'm JARVIS...' reply. --- server/voice/voiceBrainService.js | 5 +++++ tests/unit/voiceBrainService.test.js | 7 +++++++ 2 files changed, 12 insertions(+) diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index 8f97eed5..24ea5a7d 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -134,6 +134,11 @@ class VoiceBrainService { return ctx.workspace ? `You're in the ${ctx.workspace} workspace.` : 'No workspace is open right now.'; } + // Identity — instant, not a job for the Commander agent. + if (/what.?s? your name|who are you|what are you|your name|introduce yourself/.test(t)) { + return "I'm JARVIS, your fleet supervisor. I keep an eye on your agents, answer questions about what's going on, run commands, and hand bigger jobs to the Commander."; + } + // What can you do if (/what can you do|what commands|help me|what.*(you|can i) (say|ask)/.test(t)) { return `I can run about ${ctx.capabilities || 'a set of'} orchestrator commands directly, answer questions about your fleet, and hand anything else to the Commander to work on.`; diff --git a/tests/unit/voiceBrainService.test.js b/tests/unit/voiceBrainService.test.js index b1723263..17ced4c2 100644 --- a/tests/unit/voiceBrainService.test.js +++ b/tests/unit/voiceBrainService.test.js @@ -55,6 +55,13 @@ describe('VoiceBrainService — fact lane', () => { expect(b.answerFromContext('what workspace am i in')).toMatch(/Zoo Game/); }); + test('identity questions answer instantly instead of going to the Commander', () => { + const { b } = brain(); + expect(b.answerFromContext('what is your name')).toMatch(/JARVIS/); + expect(b.answerFromContext("what's your name")).toMatch(/JARVIS/); + expect(b.answerFromContext('who are you')).toMatch(/JARVIS/); + }); + test('an open-ended request is NOT a fact and falls through', () => { const { b } = brain(); expect(b.answerFromContext('spin up a reviewer for PR 12 and ping me when done')).toBeNull(); From a66e2e571462d65cd3cd6962445589c8a1d4f27e Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:31:41 +1000 Subject: [PATCH 54/69] feat(voice): Kokoro natural neural TTS via warm HTTP server Adds a natural-sounding local voice (Kokoro) as the highest-quality TTS provider, served from a warm HTTP server (~2s CPU synth, faster on GPU) and streamed to the browser like piper so it is reliably audible on WSL. Registry health now checks the kokoro server, and speakLocally routes kokoro through the browser when a client is connected. Co-Authored-By: Claude Opus 4.8 --- config/voice-providers.json | 11 +++++----- server/speechService.js | 42 +++++++++++++++++++++++-------------- 2 files changed, 32 insertions(+), 21 deletions(-) diff --git a/config/voice-providers.json b/config/voice-providers.json index 419a10b7..241ecbad 100644 --- a/config/voice-providers.json +++ b/config/voice-providers.json @@ -30,13 +30,14 @@ { "id": "kokoro", "kind": "tts", - "label": "Kokoro-82M (local, Apache-2.0)", + "label": "Kokoro (local neural, natural — warm HTTP server)", "engine": "kokoro", "local": true, - "quality": 4, - "requires": { "command": "kokoro-tts" }, - "install": "pip install --user kokoro-onnx (or the `kokoro` package); ~300MB model. Runs on CPU or GPU.", - "notes": "Best lightweight open TTS of 2026 — natural voice at tiny compute. Recommended local default once installed." + "quality": 5, + "endpoint": "http://127.0.0.1:5960", + "requires": { "server": "http://127.0.0.1:5960" }, + "install": "pip install --user kokoro-onnx soundfile flask; model+voices in ~/.local/share/kokoro; run ~/.local/bin/kokoro-server.py (started by start-voice-stack.sh).", + "notes": "Natural neural voice; ~2s synth on CPU (faster on GPU). Streamed to the browser like piper. The nicer-sounding local default." }, { "id": "chatterbox", diff --git a/server/speechService.js b/server/speechService.js index 4d79f86c..68c83452 100644 --- a/server/speechService.js +++ b/server/speechService.js @@ -80,8 +80,10 @@ class SpeechService { // writes raw s16le PCM on stdout. Set by the provider registry when one of // those is the active engine; empty otherwise. this.cliEngine = String(process.env.SPEECH_CLI_ENGINE || '').trim(); - // A warm piper HTTP server (model kept loaded) — ~0.2s synth vs ~5s cold. + // Warm neural TTS HTTP servers (model kept loaded, POST /synthesize -> WAV). + // piper: fast (~0.2s), decent. kokoro: natural (~2s CPU), the nicer voice. this.piperHttpUrl = String(process.env.PIPER_HTTP_URL || 'http://127.0.0.1:5959').replace(/\/$/, ''); + this.kokoroHttpUrl = String(process.env.KOKORO_HTTP_URL || 'http://127.0.0.1:5960').replace(/\/$/, ''); this.history = []; this.lastSpokenAt = new Map(); this.backendCache = null; @@ -110,7 +112,7 @@ class SpeechService { const backends = [ { id: 'browser', label: 'Browser speech synthesis', available: true, local: false }, { id: 'piper', label: 'Piper (local neural TTS)', available: commandExists('piper') && Boolean(this.piperModel), local: true }, - { id: 'kokoro', label: 'Kokoro / generic local CLI TTS', available: Boolean(this.cliEngine) && commandExists(this.cliEngine), local: true }, + { id: 'kokoro', label: 'Kokoro (local neural, natural)', available: Boolean(this.kokoroHttpUrl) || (Boolean(this.cliEngine) && commandExists(this.cliEngine)), local: true }, { id: 'say', label: 'macOS say', available: os.platform() === 'darwin' && commandExists('say'), local: true }, { id: 'sapi', label: 'Windows SAPI', available: os.platform() === 'win32', local: true }, { id: 'espeak', label: 'espeak-ng', available: commandExists('espeak-ng') || commandExists('espeak'), local: true } @@ -200,9 +202,12 @@ class SpeechService { * Returns {spoken:true} optimistically and does the synth in the background * (like the spawn path), emitting `speech-audio` when the WAV is ready. */ - speakViaPiperBrowser(text, priority) { + speakViaNeuralBrowser(engine, text, priority) { if (!this.io) return { spoken: false, reason: 'no socket connection to a client' }; - this.synthAndEmit(text, priority).catch((error) => this.logger.warn?.('Piper synth failed', { error: error.message })); + const httpUrl = engine === 'kokoro' ? this.kokoroHttpUrl : this.piperHttpUrl; + // Only piper has a local spawn fallback; kokoro is HTTP-only. + this.synthAndEmit(text, priority, httpUrl, engine !== 'kokoro') + .catch((error) => this.logger.warn?.(`${engine} synth failed`, { error: error.message })); return { spoken: true }; } @@ -211,26 +216,28 @@ class SpeechService { this.io?.emit('speech-audio', { wav: wavBuffer.toString('base64'), priority, at: new Date().toISOString() }); } - async synthAndEmit(text, priority) { - // Fast path: a warm piper HTTP server keeps the model loaded (~0.2s synth - // vs ~5s for a cold `python -m piper` per call). Try it first. - if (this.piperHttpUrl) { + async synthAndEmit(text, priority, httpUrl = this.piperHttpUrl, allowSpawnFallback = true) { + // Fast path: a warm HTTP TTS server keeps the model loaded, so synth is + // ~0.2s (piper) / ~2s (kokoro) instead of a multi-second cold start. + if (httpUrl) { try { - const resp = await fetch(`${this.piperHttpUrl}/synthesize`, { + const resp = await fetch(`${httpUrl}/synthesize`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ text }), - signal: AbortSignal.timeout(8000) + signal: AbortSignal.timeout(20000) }); if (resp.ok) { this.emitAudio(Buffer.from(await resp.arrayBuffer()), priority); return; } } catch { - // Server down/unreachable — fall back to spawning piper. + // Server down/unreachable — fall back to spawning piper (if allowed). } } + if (!allowSpawnFallback) return; + // Fallback: spawn piper once (cold, slower) and wrap its raw PCM as WAV. await new Promise((resolve) => { try { @@ -329,13 +336,16 @@ class SpeechService { return this.spawnQuiet(binary, [text]); } // On WSL, server-side PulseAudio usually can't reach the speakers, so when a - // browser is connected, stream piper's audio there (reliably audible). Only - // fall back to server-side paplay when nothing is listening in a browser. + // browser is connected, stream the neural audio there (reliably audible). + const hasClient = Number(this.io?.engine?.clientsCount ?? 0) > 0; if (backendId === 'piper') { - const hasClient = Number(this.io?.engine?.clientsCount ?? 0) > 0; - return hasClient ? this.speakViaPiperBrowser(text) : this.speakViaPiper(text); + return hasClient ? this.speakViaNeuralBrowser('piper', text) : this.speakViaPiper(text); + } + if (backendId === 'kokoro') { + // kokoro is a warm HTTP neural server streamed to the browser. + if (hasClient) return this.speakViaNeuralBrowser('kokoro', text); + return this.speakViaCli(this.cliEngine, text); // headless fallback if a CLI engine is set } - if (backendId === 'kokoro') return this.speakViaCli(this.cliEngine, text); if (backendId === 'sapi') { // Text is already sanitized to printable ASCII with no shell metacharacters; // single quotes are doubled because PowerShell escapes them that way. From 10c66ea59be71ea79a971efd154e2a9b580289cc Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:31:48 +1000 Subject: [PATCH 55/69] fix(voice): never obey a negated command; ack dismissals instantly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Two routing bugs found in stress testing: - "don't open the queue" was classified by the LLM into a queue command — doing the opposite of what was said. parseCommand now short-circuits a leading negation and hands it to the Commander (which understands "don't") instead of the classifier. - "never mind" / "actually never mind" fell through to the Commander because the action/negation guard swallowed the leading "never". The dismissal ack now runs before that guard and tolerates a leading filler word, so it answers instantly with "Okay, forget it." Co-Authored-By: Claude Opus 4.8 --- server/voice/voiceBrainService.js | 40 +++++++++++++++++++++----- server/voiceCommandService.js | 11 +++++++ tests/unit/voiceBrainService.test.js | 22 ++++++++++++++ tests/unit/voiceCommandService.test.js | 15 ++++++++++ 4 files changed, 81 insertions(+), 7 deletions(-) diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index 24ea5a7d..fbb6a438 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -90,13 +90,28 @@ class VoiceBrainService { const t = String(transcript || '').toLowerCase().trim(); if (!t) return null; + // A dismissal ("never mind", "forget it", "actually never mind") is a quick + // ack, not a Commander job. This MUST run before the action/negation guard + // below — "never mind" starts with "never", so the guard would otherwise + // swallow it as a negated action and send it to the Commander. + if (/^(ok(ay)?|actually|uh+|um+|well|hmm|so|yeah|nah|no)?[\s,]*(never ?mind|forget it|forget about it)\b/.test(t)) { + return 'Okay, forget it.'; + } + // An action request ("open the queue", "start a reviewer") is never a fact // to read back — it belongs to the command lane or the agent. Only answer // questions here, so the fact lane can't hijack a thing you asked it to DO. - if (/^(open|show|hide|close|focus|switch|go to|goto|start|stop|run|create|make|spawn|launch|kill|delete|remove|add|set|move|approve|reject|merge|push|pull|commit|clear|refresh|reload|new)\b/.test(t)) { + // Negations ("don't open the queue") aren't facts either. + if (/^(open|show|hide|close|focus|switch|go to|goto|start|stop|run|create|make|spawn|launch|kill|delete|remove|add|set|move|approve|reject|merge|push|pull|commit|clear|refresh|reload|new)\b/.test(t) + || /^(don'?t|do not|never)\b/.test(t)) { return null; } + // Thanks / acknowledgement — a quick reply, never a Commander job. + if (/^(thanks|thank you|thankyou|cheers|ta|much appreciated|nice one|good (job|work|stuff)|awesome|great|cool|ok|okay|kk|got it|sounds good|perfect|no worries)\b/.test(t)) { + return "You're welcome."; + } + // What needs me / what's wrong / status of the fleet if (/(needs?|need)\s+(me|my|your)|attention|anything (wrong|broken|stuck|urgent)|what.*(should i|do i need)/.test(t)) { return ctx.supervisor?.spoken || 'Nothing needs you right now. Everything else was handled.'; @@ -135,18 +150,21 @@ class VoiceBrainService { } // Identity — instant, not a job for the Commander agent. - if (/what.?s? your name|who are you|what are you|your name|introduce yourself/.test(t)) { + if (/what.?s? your name|who are you|what are you|your name|introduce yourself|are you (an? )?(ai|bot|robot|real|human|person)/.test(t)) { return "I'm JARVIS, your fleet supervisor. I keep an eye on your agents, answer questions about what's going on, run commands, and hand bigger jobs to the Commander."; } - // What can you do - if (/what can you do|what commands|help me|what.*(you|can i) (say|ask)/.test(t)) { + // What can you do (not bare "help me" — that's a real request, let it flow on) + if (/what can you do|what commands|what.*(you|can i) (say|ask)|how do (i|you) work/.test(t)) { return `I can run about ${ctx.capabilities || 'a set of'} orchestrator commands directly, answer questions about your fleet, and hand anything else to the Commander to work on.`; } - // Greetings / presence checks — LAST, so a real question that merely opens - // with "hey" ("hey what needs me") is matched by the specific lanes first. - if (/^(hi|hey|hello|yo|howdy|greetings|good (morning|afternoon|evening))\b|are you (there|awake|up|listening|around)|can you hear me|you (there|up)/.test(t)) { + // Greetings / presence checks — LAST, and only when it's actually a greeting, + // not "hey " (which should hit the command/agent lane). A request + // verb anywhere in the utterance disqualifies it as a pure greeting. + const hasRequestVerb = /\b(show|open|tell|give|create|start|stop|switch|run|make|find|set|pull|bring|list|check|fix|build|write|add|do)\b/.test(t); + if (!hasRequestVerb + && (/^(hi|hey|hello|yo|howdy|greetings|good (morning|afternoon|evening))\b|are you (there|awake|up|listening|around)|can you hear me|you (there|up)/.test(t))) { const c = this.countSessions(ctx.sessions); return c.total ? `Yes, I'm here. ${c.busy} agent${c.busy === 1 ? '' : 's'} working right now. What do you need?` @@ -227,6 +245,14 @@ class VoiceBrainService { return { handled: true, route: 'fact', spoken: fact }; } + // A single stray word ("uh", "hmm") isn't a request — don't wake the + // Commander for it. + if (String(transcript || '').trim().split(/\s+/).filter(Boolean).length <= 1) { + const miss = "Sorry, I didn't catch that."; + this.speak(miss); + return { handled: false, route: 'unclear', spoken: miss }; + } + // Agent lane: the Commander has the whole API and can do anything. const forwarder = this.deps.commanderForwarder; if (typeof forwarder === 'function') { diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 0606c8ba..17f1ab02 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -1217,6 +1217,9 @@ class VoiceCommandService { // ("how many agents are working") should be answered instantly from a // snapshot, not pay the multi-second LLM command-classification cost first. // Action phrasings ("open the queue") return null here and fall through. + // The brain also owns the ack lane ("never mind" -> "Okay, forget it"), so + // this MUST run before the negation guard below or that ack regresses to a + // Commander round-trip. if (!options.skipFact && this.brain?.answerFromContext) { try { const fact = this.brain.answerFromContext(text); @@ -1224,6 +1227,14 @@ class VoiceCommandService { } catch { /* fall through to the classifier */ } } + // A negation ("don't open the queue", "cancel that") must never be turned + // into the command it negates. The fact lane already handled bare acks + // ("never mind"); anything still negating here skips the classifier and is + // handed to the Commander, which understands "don't". + if (/^(don'?t\b|do not\b|never\b|stop\b|cancel\b|no,?\s|nope\b)/.test(text)) { + return { success: false, error: 'negation is not a command', transcript: text }; + } + // Try Ollama first (local, private) if (this.useOllama) { const ollamaResult = await this.parseWithOllama(text); diff --git a/tests/unit/voiceBrainService.test.js b/tests/unit/voiceBrainService.test.js index 17ced4c2..8bc3931b 100644 --- a/tests/unit/voiceBrainService.test.js +++ b/tests/unit/voiceBrainService.test.js @@ -81,6 +81,28 @@ describe('VoiceBrainService — fact lane', () => { expect(b.answerFromContext('open the queue')).toBeNull(); expect(b.answerFromContext('start a reviewer on the queue')).toBeNull(); }); + + test('a dismissal is acked instantly, even with a leading filler word', () => { + const { b } = brain(); + expect(b.answerFromContext('never mind')).toMatch(/forget it/i); + expect(b.answerFromContext('actually never mind')).toMatch(/forget it/i); + expect(b.answerFromContext('ok forget it')).toMatch(/forget it/i); + expect(b.answerFromContext('forget about it')).toMatch(/forget it/i); + }); + + test('thanks gets a friendly ack, not a Commander round-trip', () => { + const { b } = brain(); + expect(b.answerFromContext('thanks so much')).toMatch(/welcome/i); + expect(b.answerFromContext('cheers')).toMatch(/welcome/i); + }); + + test('a negated action ("don\'t open the queue") is not a fact and falls through', () => { + const { b } = brain({ queue: [{ id: '1', title: 'x' }] }); + // The fact lane must not answer it — it falls through so the command layer's + // negation guard can hand it to the Commander instead of opening the queue. + expect(b.answerFromContext("don't open the queue")).toBeNull(); + expect(b.answerFromContext('do not merge that pr')).toBeNull(); + }); }); describe('VoiceBrainService — routing', () => { diff --git a/tests/unit/voiceCommandService.test.js b/tests/unit/voiceCommandService.test.js index ad8882bf..22d0e64a 100644 --- a/tests/unit/voiceCommandService.test.js +++ b/tests/unit/voiceCommandService.test.js @@ -373,4 +373,19 @@ describe('VoiceCommandService (free-form routing)', () => { expect(result.success).toBe(false); expect(result.forwardedToCommander).toBeUndefined(); }); + + test('a negated action is never turned into the command it negates', async () => { + // "don't open the queue" must not resolve to a queue command; the classifier + // is short-circuited so it falls through to the Commander (which understands + // "don't") rather than doing the opposite of what was said. + const parsed = await voiceCommandService.parseCommand("don't open the queue"); + expect(parsed.success).toBe(false); + expect(parsed.error).toMatch(/negation/i); + + const forwarded = []; + voiceCommandService.setCommanderForwarder(async (text) => { forwarded.push(text); return 'sent'; }); + const result = await voiceCommandService.processVoiceCommand("don't open the queue"); + expect(result.method).toBe('commander'); + expect(forwarded).toEqual(["don't open the queue"]); + }); }); From 23ca05679c1774d10709ccff10c3887b5e6413b8 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:31:55 +1000 Subject: [PATCH 56/69] perf(voice): resolve a pinned provider without probing every server resolveActive now health-checks only the pinned provider first and returns it if available, instead of HTTP-probing every model server just to confirm the one already chosen. Falls back to auto only when the pin is gone/unavailable, so a broken pin still never mutes voice. Tests made hermetic (fixed provider set) so resolution assertions don't depend on which real model servers happen to be up on the test machine. Co-Authored-By: Claude Opus 4.8 --- server/voice/voiceProviderService.js | 18 ++++++----- tests/unit/voiceProviderService.test.js | 41 ++++++++++++++++++------- 2 files changed, 41 insertions(+), 18 deletions(-) diff --git a/server/voice/voiceProviderService.js b/server/voice/voiceProviderService.js index a46f313e..c3f7201b 100644 --- a/server/voice/voiceProviderService.js +++ b/server/voice/voiceProviderService.js @@ -194,15 +194,19 @@ class VoiceProviderService { const key = this.activeKey(kind); if (key === 'none') return null; - const withHealth = await this.listWithHealth(kind); - const available = withHealth.filter((p) => p.available); - + // A pin only needs its own health check — cheap, and avoids HTTP-probing + // every other model server just to confirm the one you chose. if (key && key !== 'auto') { - const pinned = withHealth.find((p) => p.id === key); - if (pinned?.available) return pinned; - // fall through to auto + const pinned = this.get(key); + if (pinned && pinned.kind === kind) { + const health = await this.checkAvailability(pinned); + if (health.available) return { ...pinned, ...health }; + } + // pinned is gone/unavailable — fall back to auto rather than muting. } - return available.sort((a, b) => b.quality - a.quality)[0] || null; + + const withHealth = await this.listWithHealth(kind); + return withHealth.filter((p) => p.available).sort((a, b) => b.quality - a.quality)[0] || null; } /** diff --git a/tests/unit/voiceProviderService.test.js b/tests/unit/voiceProviderService.test.js index c6b394da..5300c9ff 100644 --- a/tests/unit/voiceProviderService.test.js +++ b/tests/unit/voiceProviderService.test.js @@ -23,7 +23,31 @@ const service = (over = {}) => { return s; }; -afterEach(() => { delete process.env.AGENT_WORKSPACE_DIR; }); +// A hermetic service whose provider set is fixed, so resolution assertions +// don't depend on which real model servers (kokoro/piper HTTP) happen to be up +// on the machine running the tests. +const tmpDirs = []; +const controlledService = (providers, actives = {}) => { + const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vp-ctl-')); + tmpDirs.push(dir); + fs.writeFileSync(path.join(dir, 'voice-providers.json'), JSON.stringify({ + activeTts: actives.tts || 'auto', activeStt: actives.stt || 'auto', activeDuplex: actives.duplex || 'none', + providers + })); + process.env.AGENT_WORKSPACE_DIR = dir; + return new VoiceProviderService({ logger: { warn() {}, error() {} } }); +}; + +// Two TTS providers whose availability the fake speech service controls. +const TTS_SET = [ + { id: 'browser', kind: 'tts', engine: 'browser', quality: 2 }, + { id: 'piper', kind: 'tts', engine: 'piper', quality: 3 } +]; + +afterEach(() => { + delete process.env.AGENT_WORKSPACE_DIR; + while (tmpDirs.length) fs.rmSync(tmpDirs.pop(), { recursive: true, force: true }); +}); describe('VoiceProviderService', () => { test('loads the shipped registry with tts/stt/duplex providers', () => { @@ -59,28 +83,23 @@ describe('VoiceProviderService', () => { }); test('auto resolves to the highest-quality AVAILABLE provider', async () => { - const s = service(); + const s = controlledService(TTS_SET); s.init({ speechService: fakeSpeech({ piper: true }) }); // piper (quality 3) beats browser (quality 2) when it is available. expect((await s.resolveActive('tts'))?.id).toBe('piper'); - const s2 = service(); + const s2 = controlledService(TTS_SET); s2.init({ speechService: fakeSpeech({ piper: false }) }); // With piper unavailable it falls back to browser rather than nothing. expect((await s2.resolveActive('tts'))?.id).toBe('browser'); }); test('a pin to an unavailable provider falls back to auto, never silently mutes', async () => { - const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'vp-')); - fs.writeFileSync(path.join(dir, 'voice-providers.json'), JSON.stringify({ - activeTts: 'personaplex', // duplex id pinned as tts + unavailable anyway - providers: JSON.parse(fs.readFileSync(DEFAULT_CONFIG_PATH, 'utf8')).providers - })); - const s = service({ env: dir }); + // Pin a nonexistent provider; only browser+piper exist and piper is down. + const s = controlledService(TTS_SET, { tts: 'no-such-model' }); s.init({ speechService: fakeSpeech({ piper: false }) }); const resolved = await s.resolveActive('tts'); expect(resolved?.id).toBe('browser'); // fell back to the best available - fs.rmSync(dir, { recursive: true, force: true }); }); test('duplex defaults to none (off) until a model is explicitly chosen', async () => { @@ -112,7 +131,7 @@ describe('VoiceProviderService', () => { }); test('getStatus reports selected + resolved per capability', async () => { - const s = service(); + const s = controlledService(TTS_SET); s.init({ speechService: fakeSpeech({ piper: true }) }); const status = await s.getStatus(); expect(status.active.tts.resolved).toBe('piper'); From 3ccca0c8ad72e77116bb5eab075c756ccc369fa6 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:36:46 +1000 Subject: [PATCH 57/69] fix(voice): use the 3b model by default; skip the LLM for stray single words - The classifier was silently running on llama3.2:1b even when the tuned 3b was installed: the model-preference check was satisfied by ANY llama3.2 tag, so it never upgraded off the 1b that rambles and misfiles chit-chat as commands. Default to 3b and pick the best installed model (qwen/3b before 1b/phi), degrading gracefully. - A single mis-heard word ("uh") matched no rule and no fact but still paid a ~900ms LLM round-trip only to fail. Short-circuit it so it falls through instantly to "didn't catch that". Co-Authored-By: Claude Opus 4.8 --- server/voiceCommandService.js | 35 ++++++++++++++++++-------- tests/unit/voiceCommandService.test.js | 8 ++++++ 2 files changed, 33 insertions(+), 10 deletions(-) diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 17f1ab02..302ce3e5 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -17,7 +17,9 @@ class VoiceCommandService { constructor() { // Ollama config (local LLM) this.ollamaUrl = process.env.OLLAMA_URL || 'http://localhost:11434'; - this.ollamaModel = process.env.OLLAMA_MODEL || 'llama3.2:1b'; // Small, fast model + // 3b is the accuracy sweet spot for command classification — the 1b rambles + // and jams chit-chat into commands. Still fast, and kept warm in VRAM. + this.ollamaModel = process.env.OLLAMA_MODEL || 'llama3.2:3b'; this.useOllama = false; // Claude API config (external, fast) @@ -1143,15 +1145,20 @@ class VoiceCommandService { console.log('[Voice] Ollama available with models:', models); this.useOllama = true; - // Check if our preferred model is available - const hasPreferred = models.some(m => m.startsWith(this.ollamaModel.split(':')[0])); - if (!hasPreferred && models.length > 0) { - // Use first available small model - const smallModel = models.find(m => - m.includes('llama3.2:1b') || m.includes('phi') || m.includes('qwen') - ) || models[0]; - console.log(`[Voice] Using model: ${smallModel}`); - this.ollamaModel = smallModel; + // Use the configured model if it's actually installed; otherwise pick + // the best available. Bigger llama3.2 / qwen beats the tiny 1b for + // command accuracy, so prefer those before degrading to 1b/phi. + if (!models.includes(this.ollamaModel) && models.length > 0) { + const preference = [ + (m) => /qwen2\.5.*(7b|3b)/.test(m), + (m) => m.startsWith('llama3.2:3b'), + (m) => m.includes('qwen'), + (m) => m.startsWith('llama3.2'), + (m) => m.includes('phi') + ]; + const chosen = preference.map((pick) => models.find(pick)).find(Boolean) || models[0]; + console.log(`[Voice] Configured model not installed; using: ${chosen}`); + this.ollamaModel = chosen; } } } catch (err) { @@ -1235,6 +1242,14 @@ class VoiceCommandService { return { success: false, error: 'negation is not a command', transcript: text }; } + // A single stray word that matched no rule and no fact is almost always + // mis-heard speech ("uh", "hmm"), never a fuzzy command — the command + // vocabulary is already covered by rules. Skip the ~1s LLM round-trip and + // let it fall through fast to "didn't catch that". + if (text.split(/\s+/).filter(Boolean).length <= 1) { + return { success: false, error: 'too short to classify', transcript: text }; + } + // Try Ollama first (local, private) if (this.useOllama) { const ollamaResult = await this.parseWithOllama(text); diff --git a/tests/unit/voiceCommandService.test.js b/tests/unit/voiceCommandService.test.js index 22d0e64a..d67ddff9 100644 --- a/tests/unit/voiceCommandService.test.js +++ b/tests/unit/voiceCommandService.test.js @@ -388,4 +388,12 @@ describe('VoiceCommandService (free-form routing)', () => { expect(result.method).toBe('commander'); expect(forwarded).toEqual(["don't open the queue"]); }); + + test('a single stray word skips the LLM classifier (mis-heard speech)', async () => { + // "uh" matches no rule and no fact; it must not pay an LLM round-trip only + // to fail — it falls through fast so the brain says "didn't catch that". + const parsed = await voiceCommandService.parseCommand('uh'); + expect(parsed.success).toBe(false); + expect(parsed.error).toMatch(/too short/i); + }); }); From 79e65d950e85b921a9468e3b4a490df7be9eea6f Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:42:47 +1000 Subject: [PATCH 58/69] fix(voice): rule-match 'open the ' so the LLM can't misfile it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Live testing showed "open the queue" being classified by the 3b model as queue-select-by-pr-ref — the wrong command within the queue family. The auto-parser only matched the exact command name ("open queue"), so the natural "open THE queue" / "pull up the queue" / "show me the queue" phrasings fell through to the fuzzy LLM. Add deterministic rules for the open-queue / open-tasks / open-advice / open-settings panels; each requires its object noun so it never shadows the specific queue rules ("open blockers", "triage queue", "open next review"). Co-Authored-By: Claude Opus 4.8 --- server/voiceCommandService.js | 33 ++++++++++++++++++++++++++ tests/unit/voiceCommandService.test.js | 13 ++++++++++ 2 files changed, 46 insertions(+) diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 302ce3e5..c3b65ff9 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -79,6 +79,39 @@ class VoiceCommandService { return { behavior: 'always' }; } }, + // Open a panel by name. These are fixed phrases with an exact intent, so + // match them deterministically here instead of letting the fuzzy LLM + // misfile "open the queue" as a specific queue action (e.g. select-by-pr). + // Each requires its object noun, so they never shadow the more specific + // queue rules below ("open blockers", "triage queue", "open next review"). + { + patterns: [ + /^(?:open|show(?:\s+me)?|pull\s+up|bring\s+up|go\s+to|jump\s+to)\s+(?:the\s+)?(?:review\s+|pr\s+)?queue\b/i, + ], + command: 'open-queue', + extractParams: () => ({}) + }, + { + patterns: [ + /^(?:open|show(?:\s+me)?|pull\s+up|bring\s+up|go\s+to)\s+(?:the\s+)?(?:task\s+list|tasks?|to-?dos?)\b/i, + ], + command: 'open-tasks', + extractParams: () => ({}) + }, + { + patterns: [ + /^(?:open|show(?:\s+me)?|pull\s+up|bring\s+up|go\s+to)\s+(?:the\s+)?(?:advice|recommendations?|suggestions?)\b/i, + ], + command: 'open-advice', + extractParams: () => ({}) + }, + { + patterns: [ + /^(?:open|show(?:\s+me)?|pull\s+up|bring\s+up|go\s+to)\s+(?:the\s+)?settings\b/i, + ], + command: 'open-settings', + extractParams: () => ({}) + }, // Open Queue { patterns: [ diff --git a/tests/unit/voiceCommandService.test.js b/tests/unit/voiceCommandService.test.js index d67ddff9..acfbe97c 100644 --- a/tests/unit/voiceCommandService.test.js +++ b/tests/unit/voiceCommandService.test.js @@ -36,6 +36,19 @@ describe('VoiceCommandService (rule parsing)', () => { expect(always.params).toEqual({ behavior: 'always' }); }); + test('natural "open the ..." phrasings resolve to a panel, never the LLM', () => { + for (const phrase of ['open the queue', 'pull up the queue', 'show me the queue', 'open the review queue', 'go to the pr queue']) { + expect(voiceCommandService.parseWithRules(phrase)?.command).toBe('open-queue'); + } + expect(voiceCommandService.parseWithRules('open the tasks')?.command).toBe('open-tasks'); + expect(voiceCommandService.parseWithRules('show me the recommendations')?.command).toBe('open-advice'); + expect(voiceCommandService.parseWithRules('open the settings')?.command).toBe('open-settings'); + // Must NOT shadow the more specific queue rules. + expect(voiceCommandService.parseWithRules('show blockers')?.command).toBe('queue-blockers'); + expect(voiceCommandService.parseWithRules('triage queue')?.command).toBe('queue-triage'); + expect(voiceCommandService.parseWithRules('start next review')?.command).toBe('queue-next'); + }); + test('parses open process panels', () => { const queue = voiceCommandService.parseWithRules('open queue'); expect(queue.command).toBe('open-queue'); From 0a457ca9049408a6cae6f6bddfda14466bff0c28 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:45:35 +1000 Subject: [PATCH 59/69] fix(voice): answer natural fleet-status questions instantly "how's the fleet doing" fell through to the Commander (a ~1.2s LLM round-trip) instead of the instant fact lane, because the regex only matched "how many agents" / "fleet status". Broaden it to catch "how's the fleet", "how are the agents doing", etc. Co-Authored-By: Claude Opus 4.8 --- server/voice/voiceBrainService.js | 4 ++-- tests/unit/voiceBrainService.test.js | 7 +++++++ 2 files changed, 9 insertions(+), 2 deletions(-) diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index fbb6a438..1c203d6f 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -117,8 +117,8 @@ class VoiceBrainService { return ctx.supervisor?.spoken || 'Nothing needs you right now. Everything else was handled.'; } - // How many agents / sessions working - if (/(how many|number of).*(agent|session|running|working|busy)|what.*(agents|sessions).*(doing|status)|fleet status|are (they|the agents) (busy|working)/.test(t)) { + // How many agents / sessions working / how the fleet is doing + if (/(how many|number of).*(agent|session|running|working|busy)|what.*(agents|sessions).*(doing|status)|\bfleet\b|how('?s| is| are)\b.*\b(agents?|sessions?)\b|are (they|the agents) (busy|working)/.test(t)) { const c = this.countSessions(ctx.sessions); if (!c.total) return 'No agent sessions are open right now.'; const parts = []; diff --git a/tests/unit/voiceBrainService.test.js b/tests/unit/voiceBrainService.test.js index 8bc3931b..b91ecada 100644 --- a/tests/unit/voiceBrainService.test.js +++ b/tests/unit/voiceBrainService.test.js @@ -40,6 +40,13 @@ describe('VoiceBrainService — fact lane', () => { expect(answer).toMatch(/1 waiting/); }); + test('natural fleet-status phrasings answer instantly instead of going to the Commander', () => { + const { b } = brain({ sessions: [{ sessionId: 'a', status: 'busy' }, { sessionId: 'b', status: 'idle' }] }); + for (const phrase of ["how's the fleet doing", 'how is the fleet', 'how are the agents doing', "how's it going with the sessions"]) { + expect(b.answerFromContext(phrase)).toMatch(/2 agents/); + } + }); + test('queue question summarizes the top items', () => { const { b } = brain({ queue: [{ id: '1', title: 'fix the crash' }, { id: '2', title: 'add leaderboard' }] }); expect(b.answerFromContext('what is on the queue')).toMatch(/2 items.*fix the crash/); From 593c66a1900a19195a842c5922fe91d632fbb8aa Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Mon, 27 Jul 2026 15:47:11 +1000 Subject: [PATCH 60/69] docs(voice): record natural-voice activation + routing hardening in handoff Co-Authored-By: Claude Opus 4.8 --- PLANS/2026-07-27/HANDOFF.md | 48 +++++++++++++++++++++++++++++++++++++ 1 file changed, 48 insertions(+) diff --git a/PLANS/2026-07-27/HANDOFF.md b/PLANS/2026-07-27/HANDOFF.md index f2ef611e..7263706a 100644 --- a/PLANS/2026-07-27/HANDOFF.md +++ b/PLANS/2026-07-27/HANDOFF.md @@ -211,3 +211,51 @@ keep a warm piper process, or use the piper C++ binary, or move to Kokoro/Person Temp files under `/tmp/pr1029-*` and `/tmp/ollama-*` are throwaway. The isolated data dir `~/.agent-workspace-jarvis-test` and the Ollama/Piper installs under `~/.local` are intentional (the working local voice stack) — keep them. + +--- + +## SESSION 2 UPDATE (2026-07-27, later) — natural voice + routing hardening + +Continued the same branch. **Tests now 868 green / 120 suites.** All changes committed + pushed. + +### 1. Kokoro natural voice — INSTALLED and ACTIVE (supersedes the "not installed" note above) +- `kokoro-onnx` installed; a warm Flask HTTP server at **:5960** (`~/.local/bin/kokoro-server.py`) + keeps the model loaded (POST `/synthesize` → WAV, voice `af_sarah`, ~2s CPU synth). +- `config/voice-providers.json` kokoro is now `engine:kokoro, quality:5, requires.server:5960`. + `speechService` streams its WAV to the browser exactly like piper (WSL audio fix). Kokoro is the + resolved active TTS (verified: `GET /api/voice-providers` → `active.tts.resolved = kokoro`). +- **Piper is also warmed** now: HTTP server at **:5959** (`python3 -m piper.http_server`), ~0.2s + synth vs the old ~5s cold `python -m piper`. `synthAndEmit` hits the warm server first. +- **Start the whole voice stack** (ollama + warm piper + warm kokoro), idempotent: + `bash ~/.local/bin/start-voice-stack.sh` + +### 2. Routing hardening — bugs found by live stress-testing, all fixed + unit-tested +Verified live via `POST /api/voice/parse` (classifier only, no side effects). All now 50-72ms: +- **Negations no longer do the opposite.** `"don't open the queue"` used to be classified into a + queue command; now short-circuited to the Commander (`voiceCommandService.parseCommand`). +- **Dismissals ack instantly.** `"never mind"` / `"actually never mind"` → "Okay, forget it." + (was a Commander round-trip; the action guard had been swallowing the leading "never"). +- **Stray single words skip the LLM.** `"uh"` → "didn't catch that" instantly (was ~900ms). +- **`open the ` is rule-matched**, not LLM-guessed. `"open the queue"` was misfiling as + `queue-select-by-pr-ref` on the 3b model; added deterministic rules for open-queue / open-tasks + / open-advice / open-settings (they require their object noun, so they don't shadow the specific + queue rules like "open blockers" / "triage queue"). +- **Natural fleet-status questions** (`"how's the fleet doing"`) now hit the instant fact lane + instead of the Commander. +- **Default LLM is now `llama3.2:3b`** (was silently using 1b even when 3b was installed — the + model-preference check matched any llama3.2 tag). Selection degrades gracefully to the best + installed model (qwen/3b before 1b/phi). + +### 3. Live server as left +- **JARVIS running on :5857** from this worktree (`work1`), env: `ORCHESTRATOR_PORT=5857 + AGENT_WORKSPACE_DIR=~/.agent-workspace-jarvis-test OLLAMA_MODEL=llama3.2:3b + PIPER_HTTP_URL=http://127.0.0.1:5959 KOKORO_HTTP_URL=http://127.0.0.1:5960`. It serves its own + UI at `http://localhost:5857`. It is a **static node process** (not nodemon) — code changes + need a manual relaunch (kill the pid on :5857, re-run with the same env). Log: `/tmp/jarvis-server.log`. + +### Still open (unchanged, lower priority) +- Genuine gibberish (`"purple monkey dishwasher"`) still routes to the Commander (~0.8s) — the + Commander just says it doesn't understand. Reliable gibberish detection isn't worth the + false-reject risk on real requests. +- Multi-part commands (`"open the queue AND approve everything"`) execute the first clause only. +- True full-duplex (PersonaPlex on the 5090) is still the next real build — see above. From dd820ed765c83fce6638e013595651f7b31b6db0 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:19:43 +1000 Subject: [PATCH 61/69] fix(codex): contain stdio stream errors, real crash-loop backoff, undelivered approvals stay pending MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - stdio pipes had no error listeners: an async EPIPE from a half-dead child (or any other stream error code) threw uncaught — only the literal EPIPE happened to be swallowed by the global handler; anything else took the whole orchestrator down. A stream error now retires that child so the normal exit/restart path takes over. - restartAttempts was zeroed at every spawn, so a binary that spawns fine but dies instantly read attempt 0 every cycle and was hammered at the 1s floor forever. Only sustained uptime (30s) resets the ladder now. - answerApproval deleted the pending entry even when the wire write failed (app-server mid-restart), making the approval vanish from the UI while the codex thread stayed blocked. An undelivered answer now keeps the approval pending and retryable. Co-Authored-By: Claude Fable 5 --- server/agents/appServerClient.js | 32 +++++++++- server/agents/appServerSignals.js | 8 ++- tests/unit/appServerClientLifecycle.test.js | 66 ++++++++++++++++++++- tests/unit/appServerService.test.js | 21 +++++++ 4 files changed, 123 insertions(+), 4 deletions(-) diff --git a/server/agents/appServerClient.js b/server/agents/appServerClient.js index 3ca20df8..f44d641a 100644 --- a/server/agents/appServerClient.js +++ b/server/agents/appServerClient.js @@ -6,6 +6,10 @@ const { augmentProcessEnv, getHiddenProcessOptions } = require('../utils/process const DEFAULT_REQUEST_TIMEOUT_MS = 30_000; const RESTART_BACKOFF_MS = [1_000, 2_000, 5_000, 15_000, 30_000]; const MAX_LINE_BYTES = 8 * 1024 * 1024; +// Uptime below this is a crash, not a run — only sustained uptime resets the +// restart-backoff counter, otherwise a spawn-then-die-immediately binary would +// read attempt 0 on every cycle and be hammered at the shortest delay forever. +const STABLE_UPTIME_MS = 30_000; /** * JSON-RPC client for `codex app-server`. @@ -73,6 +77,7 @@ class AppServerClient extends EventEmitter { // A partial line left over from a previous process must not prefix the // new stream — it would corrupt the first frame the new child sends. this.buffer = ''; + const spawnedAtMs = Date.now(); // Every handler below is bound to THIS child and bails if a newer one // has replaced it. A SIGTERM'd child's 'exit' event arrives on a later @@ -89,6 +94,26 @@ class AppServerClient extends EventEmitter { if (text) this.logger.debug?.('[app-server]', text); }); + // The stdio pipes surface their own errors (an async EPIPE from writing + // to a child that closed its stdin, a destroyed pipe). Without listeners + // those throw uncaught — and any code other than the literal EPIPE the + // global handler swallows would take the whole orchestrator down. A + // stream error means this child is broken: kill it so the normal + // exit/restart path takes over instead of pending requests timing out. + const onStreamError = (error) => { + if (this.child !== child) return; + this.lastError = error.message; + this.logger.warn?.('[app-server] stdio stream error', { error: error.message }); + try { + child.kill('SIGKILL'); + } catch { + // Already gone. + } + }; + child.stdin?.on?.('error', onStreamError); + child.stdout?.on?.('error', onStreamError); + child.stderr?.on?.('error', onStreamError); + child.on('error', (error) => { if (this.child !== child) return; this.lastError = error.message; @@ -99,12 +124,15 @@ class AppServerClient extends EventEmitter { if (this.child !== child) return; this.rejectAllPending(new Error(`app-server exited (code ${code}, signal ${signal})`)); this.child = null; + // Sustained uptime is what proves the binary works, so the backoff + // counter resets here — never at spawn time, where a crash-looping + // binary would clear it right before every death. + if (Date.now() - spawnedAtMs >= STABLE_UPTIME_MS) this.restartAttempts = 0; this.emit('exit', { code, signal }); if (!this.stopped && this.autoRestart) this.scheduleRestart(); }); - this.startedAt = new Date().toISOString(); - this.restartAttempts = 0; + this.startedAt = new Date(spawnedAtMs).toISOString(); this.emit('started', { pid: child.pid }); resolve({ running: true, pid: child.pid }); }); diff --git a/server/agents/appServerSignals.js b/server/agents/appServerSignals.js index 04560a37..24bc5efa 100644 --- a/server/agents/appServerSignals.js +++ b/server/agents/appServerSignals.js @@ -182,6 +182,12 @@ class AppServerSignalSource extends EventEmitter { if (!entry) return { ok: false, error: `no pending approval "${requestId}"` }; const sent = this.client?.respond(entry.requestId, { decision: approved ? 'approved' : 'denied', note }); + // An undelivered answer (app-server mid-restart) must keep the approval + // pending and retryable — deleting it here made the request vanish from + // the UI while the real codex thread stayed blocked on it forever. + if (!sent) { + return { ok: false, error: 'app-server is not running — answer not delivered, approval still pending', approved, threadId: entry.threadId }; + } this.pendingApprovals.delete(key); const state = this.threads.get(entry.threadId); @@ -189,7 +195,7 @@ class AppServerSignalSource extends EventEmitter { state.status = 'busy'; state.activeFlags = []; } - return { ok: Boolean(sent), approved, threadId: entry.threadId }; + return { ok: true, approved, threadId: entry.threadId }; } listPendingApprovals() { diff --git a/tests/unit/appServerClientLifecycle.test.js b/tests/unit/appServerClientLifecycle.test.js index c0510a6d..1d96b410 100644 --- a/tests/unit/appServerClientLifecycle.test.js +++ b/tests/unit/appServerClientLifecycle.test.js @@ -16,7 +16,7 @@ function makeChild(pid = 4242) { child.stdout.setEncoding = () => {}; child.stderr = new EventEmitter(); child.stderr.setEncoding = () => {}; - child.stdin = { write: jest.fn() }; + child.stdin = Object.assign(new EventEmitter(), { write: jest.fn() }); child.kill = jest.fn(() => { child.killed = true; }); return child; } @@ -118,6 +118,70 @@ describe('AppServerClient lifecycle', () => { expect(client.buffer).toBe(''); }); + test('a stdio stream error is contained and retires the child instead of throwing uncaught', async () => { + const child = makeChild(); + mockSpawn.mockReturnValue(child); + const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); + await client.start(); + + // An async EPIPE from a half-dead child arrives on the STREAM, not the + // process object — before the fix nothing listened and it threw uncaught. + expect(() => child.stdin.emit('error', new Error('write EPIPE'))).not.toThrow(); + expect(client.lastError).toMatch(/EPIPE/); + expect(child.kill).toHaveBeenCalledWith('SIGKILL'); + }); + + test('immediate crash-exits climb the restart backoff ladder instead of resetting it', async () => { + jest.useFakeTimers(); + try { + mockSpawn.mockImplementation(() => makeChild()); + const client = new AppServerClient({ autoRestart: true, logger: quietLogger }); + await client.start(); + + // The binary spawns fine but dies instantly, over and over. Before the + // fix, restartAttempts was zeroed at every spawn, so every cycle read + // the shortest delay and hammered the broken binary once a second. + client.child.emit('exit', 1, null); + expect(client.restartAttempts).toBe(1); + + await jest.advanceTimersByTimeAsync(1_000); + expect(client.isRunning()).toBe(true); + client.child.emit('exit', 1, null); + expect(client.restartAttempts).toBe(2); + + await jest.advanceTimersByTimeAsync(2_000); + client.child.emit('exit', 1, null); + expect(client.restartAttempts).toBe(3); + + client.stop(); + } finally { + jest.useRealTimers(); + } + }); + + test('sustained uptime is what resets the backoff ladder', async () => { + jest.useFakeTimers(); + try { + mockSpawn.mockImplementation(() => makeChild()); + const client = new AppServerClient({ autoRestart: true, logger: quietLogger }); + await client.start(); + + client.child.emit('exit', 1, null); + expect(client.restartAttempts).toBe(1); + + // The replacement stays alive well past the stability window before + // dying — that proves the binary works, so the ladder starts over. + await jest.advanceTimersByTimeAsync(1_000); + await jest.advanceTimersByTimeAsync(31_000); + client.child.emit('exit', 1, null); + expect(client.restartAttempts).toBe(1); + + client.stop(); + } finally { + jest.useRealTimers(); + } + }); + test('a spawn failure resolves cleanly and leaves start() retryable', async () => { mockSpawn.mockImplementationOnce(() => { throw new Error('ENOENT'); }); const client = new AppServerClient({ autoRestart: false, logger: quietLogger }); diff --git a/tests/unit/appServerService.test.js b/tests/unit/appServerService.test.js index a205afdf..c3be1658 100644 --- a/tests/unit/appServerService.test.js +++ b/tests/unit/appServerService.test.js @@ -152,6 +152,27 @@ describe('AppServerSignalSource', () => { expect(signals.listPendingApprovals()).toEqual([]); }); + test('an approval answer that cannot be delivered keeps the approval pending', () => { + const { client, signals } = source(); + client.emitRequest(5, 'item/commandExecution/requestApproval', { threadId: 't1', command: 'npm test' }); + expect(signals.listPendingApprovals()).toHaveLength(1); + + // The app-server is mid-restart: the wire write fails. + client.respond = () => false; + const failed = signals.answerApproval(5, true); + expect(failed.ok).toBe(false); + // Before the fix the entry was deleted anyway — the approval vanished from + // the UI while the real codex thread stayed blocked on it forever. + expect(signals.listPendingApprovals()).toHaveLength(1); + + // Once the server is back, the SAME approval is still answerable. + client.respond = (id, result) => { client.responses.push({ id, result }); return true; }; + const retried = signals.answerApproval(5, true); + expect(retried.ok).toBe(true); + expect(client.responses[0].result.decision).toBe('approved'); + expect(signals.listPendingApprovals()).toEqual([]); + }); + test('a closed thread stops producing signals', () => { const { client, signals } = source(); client.emitNotification('thread/status/changed', { threadId: 't1', status: { type: 'idle' } }); From 856483a8269ce2df2cf017dcee8d669a2de9bc87 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:22:26 +1000 Subject: [PATCH 62/69] =?UTF-8?q?fix(atlas):=20compile=20bundles=20from=20?= =?UTF-8?q?your=20own=20layers=20only=20=E2=80=94=20cloning=20a=20shared?= =?UTF-8?q?=20repo=20no=20longer=20republishes=20the=20sharer's=20judgemen?= =?UTF-8?q?t?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - compile() filtered on the `foreign` flag, which clears the moment discovery also knows a subscribed repo (you cloned it / gh lists it). From then on the teammate's highlights/summary rode into YOUR bundles, unattributed, to audiences they never authorized. Bundles now merge from discovery/manifest/registry only (getOwnEntries) — subscription content structurally cannot be re-shared, while a repo you genuinely have stays shareable with just your own fields. - writeManifest only knew the master/ worktree convention; for main/ layouts `atlas init` wrote .repo-atlas.json into the non-repo parent dir where it could never be committed or synced. - atlas CLI: a value-less --topic parsed as boolean true, slipped past the usage check, and was recorded as the literal topic "true" (--quality already had this guard; note/avoid/propose now do too). - qualityFloor treats bare booleans as "no floor" like the schema does, instead of coercing false to a floor of 0. Co-Authored-By: Claude Fable 5 --- scripts/atlas.js | 20 +++++++++++------ server/atlas/atlasQuery.js | 6 +++--- server/atlas/atlasStore.js | 11 ++++++---- server/repoAtlasService.js | 37 ++++++++++++++++++++++++++++++-- tests/unit/repoAtlasSync.test.js | 27 +++++++++++++++++++++++ 5 files changed, 86 insertions(+), 15 deletions(-) diff --git a/scripts/atlas.js b/scripts/atlas.js index b9788d85..6ee57f6e 100755 --- a/scripts/atlas.js +++ b/scripts/atlas.js @@ -70,6 +70,11 @@ const qualityFlag = (value) => { return Number.isFinite(num) ? num : QUALITY_INVALID; }; +// A value-less `--topic` parses as boolean `true`, which passes a truthiness +// usage check and would be recorded as the literal topic "true". Only a +// non-empty string counts as a given topic. +const topicFlag = (value) => (typeof value === 'string' && value.trim() ? value : null); + const out = (text) => process.stdout.write(`${text}\n`); const fail = (message) => { process.stderr.write(`atlas: ${message}\n`); @@ -213,11 +218,12 @@ const commands = { note(positionals, flags) { const id = positionals[0]; - if (!id || !flags.topic) return fail('usage: atlas note --topic [--quality 1-5] [--paths a,b] [--notes "..."]'); + const topic = topicFlag(flags.topic); + if (!id || !topic) return fail('usage: atlas note --topic [--quality 1-5] [--paths a,b] [--notes "..."]'); const quality = qualityFlag(flags.quality); if (quality === QUALITY_INVALID) return fail('--quality needs a number 1-5'); const saved = atlas.addHighlight(id, { - topic: flags.topic, + topic, quality, paths: listFlag(flags.paths), notes: flags.notes === true ? '' : String(flags.notes || '') @@ -227,8 +233,9 @@ const commands = { avoid(positionals, flags) { const id = positionals[0]; - if (!id || !flags.topic) return fail('usage: atlas avoid --topic --reason "..."'); - const saved = atlas.addAvoid(id, { topic: flags.topic, reason: flags.reason === true ? '' : flags.reason }); + const topic = topicFlag(flags.topic); + if (!id || !topic) return fail('usage: atlas avoid --topic --reason "..."'); + const saved = atlas.addAvoid(id, { topic, reason: flags.reason === true ? '' : flags.reason }); return out(`Marked do-not-copy: ${saved.id} → ${(saved.avoid || []).map((a) => a.topic).join(', ')}`); }, @@ -285,14 +292,15 @@ const commands = { propose(positionals, flags) { const id = positionals[0]; - if (!id || !flags.topic) { + const topic = topicFlag(flags.topic); + if (!id || !topic) { return fail('usage: atlas propose --topic [--quality 1-5] [--paths a,b] [--notes "..."] [--evidence "why"] [--avoid]'); } const quality = qualityFlag(flags.quality); if (quality === QUALITY_INVALID) return fail('--quality needs a number 1-5'); const proposal = atlas.proposeHighlight({ repoId: id, - topic: flags.topic, + topic, kind: flags.avoid === true ? 'avoid' : 'highlight', quality, paths: listFlag(flags.paths), diff --git a/server/atlas/atlasQuery.js b/server/atlas/atlasQuery.js index fabf2daf..8690a737 100644 --- a/server/atlas/atlasQuery.js +++ b/server/atlas/atlasQuery.js @@ -2,10 +2,10 @@ const { normalizeTopic, kebab } = require('./atlasSchema'); const STALE_AFTER_DAYS = 365; -// `null`, `''` and `undefined` all mean "no floor" — Number() turns the first -// two into 0, which would silently drop every uncurated repo. +// `null`, `''`, `undefined` and bare booleans all mean "no floor" — Number() +// turns them into 0, which would silently drop every uncurated repo. function qualityFloor(value) { - if (value === null || value === undefined || value === '' || value === true) return null; + if (value === null || value === undefined || value === '' || typeof value === 'boolean') return null; const num = Number(value); return Number.isFinite(num) ? num : null; } diff --git a/server/atlas/atlasStore.js b/server/atlas/atlasStore.js index 81112033..2210c919 100644 --- a/server/atlas/atlasStore.js +++ b/server/atlas/atlasStore.js @@ -260,10 +260,13 @@ function loadManifest(projectRoot) { } function writeManifest(projectRoot, entry) { - const target = fs.existsSync(path.join(projectRoot, 'master')) - ? path.join(projectRoot, 'master') - : projectRoot; - return writeJson(path.join(target, MANIFEST_FILENAME), entry); + // Worktree layouts collapse to the parent dir during discovery, but the + // manifest must land inside the checkout (master/ or main/) — the parent is + // not a git repo, so a manifest written there could never be committed. + const checkout = ['master', 'main'] + .map((dir) => path.join(projectRoot, dir)) + .find((candidate) => fs.existsSync(candidate)); + return writeJson(path.join(checkout || projectRoot, MANIFEST_FILENAME), entry); } function saveBundle(audienceId, bundle, outputPath = '') { diff --git a/server/repoAtlasService.js b/server/repoAtlasService.js index dc0550a7..c277a380 100644 --- a/server/repoAtlasService.js +++ b/server/repoAtlasService.js @@ -158,6 +158,36 @@ class RepoAtlasService { return entries; } + /** + * Entries as YOU may share them: merged WITHOUT the subscription layer, so a + * teammate's highlights/summary can never ride into a bundle you compile. + * The `foreign` flag alone is not enough — it clears the moment discovery + * also knows the repo (you cloned it, or `gh` can list it), and cloning a + * repo someone shared with you must not declassify THEIR judgement of it. + * Entries whose existence you only know from a subscription (even if you + * annotated them locally) are skipped outright. + */ + getOwnEntries() { + const { byId, registry } = this.loadLayers(); + const entries = []; + + for (const [id, layers] of byId.entries()) { + if (layers.subscription && !layers.discovery && !layers.manifest) continue; + const merged = schema.mergeEntries( + { id, visibility: registry.defaults?.visibility, groups: registry.defaults?.groups, __source: 'defaults' }, + layers.discovery, + layers.manifest, + layers.registry + ); + merged.id = id; + merged.sources = (merged.sources || []).filter((s) => s !== 'defaults'); + entries.push(merged); + } + + entries.sort((a, b) => a.id.localeCompare(b.id)); + return entries; + } + getEntry(id, options = {}) { const key = schema.kebab(id); return this.getEntries(options).find((entry) => entry.id === key) || null; @@ -254,8 +284,11 @@ class RepoAtlasService { compile(audience, { write = true } = {}) { const meta = this.listAudiences().find((a) => a.id === schema.kebab(audience)) || {}; // Never re-share what someone else shared with you — attribution and - // permission both belong to whoever published it. - const own = this.getEntries().filter((entry) => entry.foreign !== true); + // permission both belong to whoever published it. getOwnEntries() merges + // without the subscription layer, so this holds even for a subscribed + // repo you later cloned (the old `foreign` filter alone let that case + // republish the teammate's fields as if they were yours). + const own = this.getOwnEntries().filter((entry) => entry.foreign !== true); const result = compiler.compileBundle(own, { audience, label: meta.label, diff --git a/tests/unit/repoAtlasSync.test.js b/tests/unit/repoAtlasSync.test.js index e7348c2f..710b930b 100644 --- a/tests/unit/repoAtlasSync.test.js +++ b/tests/unit/repoAtlasSync.test.js @@ -243,6 +243,33 @@ describe('Repo Atlas multi-machine sync', () => { expect(compiled.bundle.entries.map((e) => e.id)).not.toContain('shared-repo'); }); + test('cloning a repo a teammate shared must not republish their judgement of it', async () => { + const shared = publishBundle(root, [{ + id: 'shared-repo', + name: 'shared-repo', + summary: 'their description', + visibility: 'public', + highlights: [{ topic: 'auth', quality: 4, notes: 'their private assessment' }] + }]); + + const me = machine('me'); + await me.subscribe({ name: 'them', source: shared }); + // Later the repo shows up in local discovery too (you cloned it, or gh + // can list it). That used to clear `foreign` — and compile() would then + // republish the teammate's summary/highlights as if they were yours. + store.saveDiscoveryCache([{ id: 'shared-repo', name: 'shared-repo', localPath: '/machine/me/shared-repo', cloned: true }]); + me.invalidate(); + me.setEntry('shared-repo', { visibility: 'public' }); + + const compiled = me.compile('anyone', { write: false }); + const entry = compiled.bundle.entries.find((e) => e.id === 'shared-repo'); + // The repo itself is yours to describe now (you actually have it)… + expect(entry).toBeDefined(); + // …but nothing the teammate authored may ride along into your bundle. + expect(entry.summary || '').not.toBe('their description'); + expect((entry.highlights || []).map((h) => h.topic)).not.toContain('auth'); + }); + test('subscribing to something that is not a bundle fails loudly', async () => { const notABundle = path.join(root, 'nope.json'); store.writeJson(notABundle, { hello: 'world' }); From 50aa6ba529e956e57d0be68d9c13067964b87f17 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:26:45 +1000 Subject: [PATCH 63/69] fix(speech): dead neural backends can no longer mute the voice silently MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - kokoro reported 'available' unconditionally (its HTTP URL has a default, so Boolean(url) was always true). A dead kokoro was selectable, synth failures were swallowed with no fallback, and speak() recorded success while nothing was ever heard — voice went permanently dark until a restart. Availability now needs an explicit opt-in signal (env var set, CLI engine present, or a provider-registry health check that actually probed the server), and a failed neural synth falls back to browser speech with a warning instead of silence. - The warm-piper HTTP path now counts toward piper availability (PIPER_HTTP_URL set), instead of demanding the CLI + model that the warm server replaces. - Selecting TTS 'none' in the provider registry now actually mutes TTS — applyActiveTts previously did nothing on null, leaving the old backend speaking. - priority now survives speakLocally() into the streamed-audio path, so a high-priority clip can interrupt again. - client: streamed audio respects the mute switch, an autoplay-blocked clip is replayed on the first user gesture instead of being lost, and consecutive clips queue instead of talking over each other. Co-Authored-By: Claude Fable 5 --- client/speech-output.js | 82 +++++++++++++++++++---- server/speechService.js | 97 +++++++++++++++++++++------- server/voice/voiceProviderService.js | 10 ++- tests/unit/speechService.test.js | 66 +++++++++++++++++++ 4 files changed, 218 insertions(+), 37 deletions(-) diff --git a/client/speech-output.js b/client/speech-output.js index 8a9b93c0..2d282470 100644 --- a/client/speech-output.js +++ b/client/speech-output.js @@ -13,7 +13,10 @@ enabled: localStorage.getItem('speechOutputEnabled') !== 'false', voiceName: localStorage.getItem('speechOutputVoice') || '', rate: Number(localStorage.getItem('speechOutputRate')) || 1.05, - currentAudio: null + currentAudio: null, + audioQueue: [], + pendingGesture: null, + gestureArmed: false }; function pickVoice() { @@ -43,20 +46,72 @@ return true; } + // Autoplay is blocked until the page has seen a user gesture, and losing the + // very first spoken reply to that policy is a silent failure. Hold the most + // recent blocked clip and replay it on the first interaction. + function armGestureRetry() { + if (state.gestureArmed) return; + state.gestureArmed = true; + const retry = () => { + state.gestureArmed = false; + const pending = state.pendingGesture; + state.pendingGesture = null; + if (pending) playAudio(pending); + }; + window.addEventListener('pointerdown', retry, { once: true, capture: true }); + window.addEventListener('keydown', retry, { once: true, capture: true }); + } + + function stopAudio() { + state.audioQueue.length = 0; + if (state.currentAudio) { + state.currentAudio.pause(); + state.currentAudio = null; + } + } + + function startClip(payload) { + const audio = new Audio(`data:audio/wav;base64,${payload.wav}`); + state.currentAudio = audio; + const advance = () => { + if (state.currentAudio !== audio) return; + state.currentAudio = null; + const next = state.audioQueue.shift(); + if (next) startClip(next); + }; + audio.addEventListener('ended', advance); + audio.addEventListener('error', advance); + audio.play().catch(() => { + // Autoplay blocked — everything queued would fail the same way, so keep + // only the newest utterance and replay it on the first user gesture. + state.pendingGesture = state.audioQueue.pop() || payload; + state.audioQueue.length = 0; + if (state.currentAudio === audio) state.currentAudio = null; + armGestureRetry(); + }); + } + // Play server-synthesized neural audio (piper/kokoro) streamed as a WAV. On // WSL the server can't reach the speakers, so it hands the bytes to us — and - // browser audio always reaches the user. A high-priority clip interrupts. + // browser audio always reaches the user. A high-priority clip interrupts; + // normal clips queue behind whatever is already playing instead of talking + // over it. function playAudio(payload) { - const b64 = payload?.wav; - if (!b64) return false; + if (!payload?.wav || !state.enabled) return false; try { - if (payload.priority === 'high' && state.currentAudio) { - state.currentAudio.pause(); - state.currentAudio = null; + if (payload.priority === 'high') { + stopAudio(); + if (synth?.speaking) synth.cancel(); + startClip(payload); + return true; } - const audio = new Audio(`data:audio/wav;base64,${b64}`); - state.currentAudio = audio; - audio.play().catch(() => { /* autoplay blocked until a user gesture */ }); + if (state.currentAudio) { + state.audioQueue.push(payload); + // A stale backlog reads like a haunted radio — keep it short. + if (state.audioQueue.length > 5) state.audioQueue.shift(); + return true; + } + startClip(payload); return true; } catch { return false; @@ -78,7 +133,12 @@ setEnabled(enabled) { state.enabled = enabled !== false; localStorage.setItem('speechOutputEnabled', String(state.enabled)); - if (!state.enabled && synth?.speaking) synth.cancel(); + if (!state.enabled) { + if (synth?.speaking) synth.cancel(); + // Muting must silence the streamed neural audio too, not just Web Speech. + stopAudio(); + state.pendingGesture = null; + } return state.enabled; }, setVoice(name) { diff --git a/server/speechService.js b/server/speechService.js index 68c83452..3a3d383c 100644 --- a/server/speechService.js +++ b/server/speechService.js @@ -87,6 +87,9 @@ class SpeechService { this.history = []; this.lastSpokenAt = new Map(); this.backendCache = null; + // Backends the provider registry has actually health-checked (its server + // probe is async and lives there) — sync detection below can trust these. + this.verifiedBackends = new Set(); } static getInstance(options = {}) { @@ -109,10 +112,30 @@ class SpeechService { detectBackends({ force = false } = {}) { if (this.backendCache && !force) return this.backendCache; + // The warm HTTP servers can't be probed synchronously here, so they count + // as available only on an explicit opt-in signal: the env var was set, or + // the provider registry health-checked the server (verifiedBackends). The + // old `Boolean(this.kokoroHttpUrl)` was ALWAYS true (the URL has a + // default), which reported kokoro available on every machine and made a + // dead kokoro selectable — with no fallback, that was total silence. const backends = [ { id: 'browser', label: 'Browser speech synthesis', available: true, local: false }, - { id: 'piper', label: 'Piper (local neural TTS)', available: commandExists('piper') && Boolean(this.piperModel), local: true }, - { id: 'kokoro', label: 'Kokoro (local neural, natural)', available: Boolean(this.kokoroHttpUrl) || (Boolean(this.cliEngine) && commandExists(this.cliEngine)), local: true }, + { + id: 'piper', + label: 'Piper (local neural TTS)', + available: (commandExists('piper') && Boolean(this.piperModel)) + || Boolean(process.env.PIPER_HTTP_URL) + || this.verifiedBackends.has('piper'), + local: true + }, + { + id: 'kokoro', + label: 'Kokoro (local neural, natural)', + available: Boolean(process.env.KOKORO_HTTP_URL) + || this.verifiedBackends.has('kokoro') + || (Boolean(this.cliEngine) && commandExists(this.cliEngine)), + local: true + }, { id: 'say', label: 'macOS say', available: os.platform() === 'darwin' && commandExists('say'), local: true }, { id: 'sapi', label: 'Windows SAPI', available: os.platform() === 'win32', local: true }, { id: 'espeak', label: 'espeak-ng', available: commandExists('espeak-ng') || commandExists('espeak'), local: true } @@ -123,6 +146,9 @@ class SpeechService { } resolveBackend() { + // 'none' is an explicit "voice off" from the provider registry, not a + // backend to fall back from. + if (this.preferredBackend === 'none') return 'none'; const backends = this.detectBackends(); if (this.preferredBackend) { const preferred = backends.find((b) => b.id === this.preferredBackend); @@ -148,10 +174,21 @@ class SpeechService { * model speaks, and a generic-CLI engine (kokoro/chatterbox) changes what is * available. `engine` is the provider's engine; `command` its CLI binary. */ - setActiveEngine(engine, { command = '' } = {}) { + setActiveEngine(engine, { command = '', verified = false } = {}) { + if (engine === 'none') { + // The registry chose "voice off" — that must actually silence TTS, not + // leave whatever backend was previously active still speaking. + this.preferredBackend = 'none'; + this.backendCache = null; + return 'none'; + } const map = { browser: 'browser', piper: 'piper', espeak: 'espeak', say: 'say', sapi: 'sapi', kokoro: 'kokoro' }; const backend = map[engine] || 'browser'; if (backend === 'kokoro' && command) this.cliEngine = command; + // The registry health-checks its providers (including HTTP-server + // reachability) before applying one — remember that so the sync + // availability detection above doesn't veto a probe that already passed. + if (verified) this.verifiedBackends.add(backend); this.preferredBackend = backend; this.backendCache = null; return this.resolveBackend(); @@ -232,29 +269,36 @@ class SpeechService { return; } } catch { - // Server down/unreachable — fall back to spawning piper (if allowed). + // Server down/unreachable — fall through to the next option. } } - if (!allowSpawnFallback) return; - // Fallback: spawn piper once (cold, slower) and wrap its raw PCM as WAV. - await new Promise((resolve) => { - try { - const env = augmentProcessEnv(process.env); - const piper = spawn('piper', ['--model', this.piperModel, '--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); - const chunks = []; - piper.stdout.on('data', (d) => chunks.push(d)); - piper.on('error', () => resolve()); - piper.on('close', () => { - this.emitAudio(this.pcmToWav(Buffer.concat(chunks), this.piperSampleRate || 22050), priority); - resolve(); - }); - piper.stdin.end(`${text}\n`); - } catch { - resolve(); + if (allowSpawnFallback && this.piperModel && commandExists('piper')) { + const pcm = await new Promise((resolve) => { + try { + const env = augmentProcessEnv(process.env); + const piper = spawn('piper', ['--model', this.piperModel, '--output-raw'], { stdio: ['pipe', 'pipe', 'ignore'], env }); + const chunks = []; + piper.stdout.on('data', (d) => chunks.push(d)); + piper.on('error', () => resolve(Buffer.alloc(0))); + piper.on('close', () => resolve(Buffer.concat(chunks))); + piper.stdin.end(`${text}\n`); + } catch { + resolve(Buffer.alloc(0)); + } + }); + if (pcm.length) { + this.emitAudio(this.pcmToWav(pcm, this.piperSampleRate || 22050), priority); + return; } - }); + } + + // Neural synthesis failed outright (server down, spawn produced nothing). + // Never go silent — hand the text to the browser's own speech synthesis so + // the utterance is still heard, and leave a trace of why. + this.logger.warn?.('Neural TTS failed — falling back to browser speech', { httpUrl }); + this.io?.emit('speech-speak', { text, priority, at: new Date().toISOString() }); } spawnQuiet(command, args) { @@ -327,7 +371,7 @@ class SpeechService { } } - speakLocally(backendId, text) { + speakLocally(backendId, text, priority) { if (backendId === 'say') { return this.spawnQuiet('say', this.voice ? ['-v', this.voice, text] : [text]); } @@ -339,11 +383,11 @@ class SpeechService { // browser is connected, stream the neural audio there (reliably audible). const hasClient = Number(this.io?.engine?.clientsCount ?? 0) > 0; if (backendId === 'piper') { - return hasClient ? this.speakViaNeuralBrowser('piper', text) : this.speakViaPiper(text); + return hasClient ? this.speakViaNeuralBrowser('piper', text, priority) : this.speakViaPiper(text); } if (backendId === 'kokoro') { // kokoro is a warm HTTP neural server streamed to the browser. - if (hasClient) return this.speakViaNeuralBrowser('kokoro', text); + if (hasClient) return this.speakViaNeuralBrowser('kokoro', text, priority); return this.speakViaCli(this.cliEngine, text); // headless fallback if a CLI engine is set } if (backendId === 'sapi') { @@ -371,9 +415,12 @@ class SpeechService { } const backend = this.resolveBackend(); + if (backend === 'none') { + return this.record({ text, backend, at: new Date().toISOString(), spoken: false, reason: 'tts provider set to none' }); + } let result; try { - result = backend === 'browser' ? this.speakViaBrowser(text, priority) : this.speakLocally(backend, text); + result = backend === 'browser' ? this.speakViaBrowser(text, priority) : this.speakLocally(backend, text, priority); } catch (error) { result = { spoken: false, reason: error.message }; } diff --git a/server/voice/voiceProviderService.js b/server/voice/voiceProviderService.js index c3f7201b..f275cd86 100644 --- a/server/voice/voiceProviderService.js +++ b/server/voice/voiceProviderService.js @@ -90,7 +90,15 @@ class VoiceProviderService { async applyActiveTts() { if (!this.speechService?.setActiveEngine) return null; const provider = await this.resolveActive('tts'); - if (provider) this.speechService.setActiveEngine(provider.engine, { command: provider.requires?.command || '' }); + if (provider) { + // resolveActive just health-checked this provider (including any HTTP + // model server), so the speech service can trust it as available. + this.speechService.setActiveEngine(provider.engine, { command: provider.requires?.command || '', verified: true }); + } else if (this.activeKey('tts') === 'none') { + // An explicit "none" must actually mute TTS — doing nothing here left + // the previously active backend still speaking. + this.speechService.setActiveEngine('none'); + } return provider; } diff --git a/tests/unit/speechService.test.js b/tests/unit/speechService.test.js index a1a58633..2ee1d601 100644 --- a/tests/unit/speechService.test.js +++ b/tests/unit/speechService.test.js @@ -91,4 +91,70 @@ describe('SpeechService', () => { expect(status.connectedClients).toBe(2); expect(status.recent[0].text).toBe('one'); }); + + test('kokoro is not "available" merely because its URL has a default', () => { + const withoutEnv = { ...process.env }; + delete withoutEnv.KOKORO_HTTP_URL; + const original = process.env; + process.env = withoutEnv; + try { + const service = new SpeechService({ logger: { warn: () => {} } }); + service.cliEngine = ''; + const kokoro = service.detectBackends({ force: true }).find((b) => b.id === 'kokoro'); + // Before the fix this was unconditionally true (the URL always defaults), + // so a dead kokoro was selectable and speech went permanently silent. + expect(kokoro.available).toBe(false); + expect(() => service.setBackend('kokoro')).toThrow(/not available/); + } finally { + process.env = original; + } + }); + + test('a registry-verified engine counts as available for sync detection', () => { + const service = new SpeechService({ logger: { warn: () => {} } }); + service.cliEngine = ''; + service.setActiveEngine('kokoro', { verified: true }); + const kokoro = service.detectBackends({ force: true }).find((b) => b.id === 'kokoro'); + expect(kokoro.available).toBe(true); + expect(service.resolveBackend()).toBe('kokoro'); + }); + + test('the "none" engine actually mutes TTS instead of leaving the old backend live', () => { + const { service, emitted } = browserSpeech(); + expect(service.speak('audible').spoken).toBe(true); + + service.setActiveEngine('none'); + const result = service.speak('should be silent', { force: true }); + expect(result.spoken).toBe(false); + expect(result.backend).toBe('none'); + expect(emitted).toHaveLength(1); + }); + + test('a failed neural synth falls back to browser speech instead of silence', async () => { + const emitted = []; + const service = new SpeechService({ logger: { warn: () => {} } }); + service.setIO({ emit: (event, payload) => emitted.push({ event, payload }) }); + service.piperModel = ''; + + // Nothing listens on this port and there is no spawn fallback — before the + // fix this returned silently while speak() had already reported success. + await service.synthAndEmit('still audible', 'high', 'http://127.0.0.1:1', false); + + expect(emitted).toHaveLength(1); + expect(emitted[0].event).toBe('speech-speak'); + expect(emitted[0].payload.text).toBe('still audible'); + expect(emitted[0].payload.priority).toBe('high'); + }); + + test('priority survives the local neural path so a high clip can interrupt', () => { + const service = new SpeechService({ logger: { warn: () => {} } }); + service.setIO({ emit: () => {}, engine: { clientsCount: 1 } }); + const seen = []; + service.speakViaNeuralBrowser = (engine, text, priority) => { + seen.push({ engine, priority }); + return { spoken: true }; + }; + service.speakLocally('kokoro', 'urgent thing', 'high'); + expect(seen).toEqual([{ engine: 'kokoro', priority: 'high' }]); + }); }); From e1ec3a087835d5dc3737526bf8e1218c4bc9c4d5 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:30:03 +1000 Subject: [PATCH 64/69] docs: scrub remaining private repo names from files this PR adds to a public repo MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The earlier scrub covered the example manifest and the public skill, but this PR still shipped real private repo names (with quality judgements attached) in the new design doc, the new Commander-doc examples, and the atlas test fixtures. All replaced with the placeholder vocabulary the scrubbed files already use (acme-*, physics-kit, puzzle-proto, example-engine). Pre-existing references on main (docs/COMMANDER_CLAUDE.md worktree examples, older PLANS/) are deliberately untouched — that exposure predates this PR and is a separate decision. Note the scrubbed names do remain in this branch's earlier commit history. Co-Authored-By: Claude Fable 5 --- ...OMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md | 22 +++++----- docs/COMMANDER_CLAUDE.md | 10 ++--- tests/unit/repoAtlasProposals.test.js | 30 ++++++------- tests/unit/repoAtlasQuery.test.js | 32 +++++++------- tests/unit/repoAtlasSchema.test.js | 8 ++-- tests/unit/repoAtlasService.test.js | 42 +++++++++---------- tests/unit/repoAtlasSync.test.js | 8 ++-- 7 files changed, 76 insertions(+), 76 deletions(-) diff --git a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md index 3606f061..0256a6f1 100644 --- a/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md +++ b/PLANS/2026-07-26/AUTONOMOUS_ORCHESTRATOR_VOICE_AND_REPO_ATLAS.md @@ -110,7 +110,7 @@ Hard invariants regardless of level: ### Why this is the Iron Man bit -The Jarvis experience isn't a nicer chat box — it's that **the assistant noticed first**. "Sir, `work3` has been waiting on a file-write permission for four minutes, and `zoo-game/work1` pushed eleven minutes ago without opening a PR." That is entirely a sensors-and-rules problem, and it is now solved with zero tokens. +The Jarvis experience isn't a nicer chat box — it's that **the assistant noticed first**. "Sir, `work3` has been waiting on a file-write permission for four minutes, and `acme-tycoon/work1` pushed eleven minutes ago without opening a PR." That is entirely a sensors-and-rules problem, and it is now solved with zero tokens. --- @@ -138,16 +138,16 @@ Plus: `briefing` (spoken fleet summary assembled from supervisor findings + advi ### Three properties that make it work -**1. Cloned-ness is irrelevant.** An entry describes a repo whether or not it's on this disk. `drain-the-lake` can be a first-class breadcrumb with a clone hint attached. This is the whole point — the map must cover the territory, not the local cache of it. +**1. Cloned-ness is irrelevant.** An entry describes a repo whether or not it's on this disk. `puzzle-proto` can be a first-class breadcrumb with a clone hint attached. This is the whole point — the map must cover the territory, not the local cache of it. -**2. Quality is a first-class field, per-topic.** The ask is explicit and correct: Epic Survivors and HyFire2 are early work but *fully functioning*, and Drain the Lake is a rough prototype that might still have the best testing setup you've written. So quality is not a repo-level star rating; it's **per highlight**: +**2. Quality is a first-class field, per-topic.** The ask is explicit and correct: Acme Shooter and Acme Arena are early work but *fully functioning*, and Puzzle Proto is a rough prototype that might still have the best testing setup you've written. So quality is not a repo-level star rating; it's **per highlight**: ```jsonc "highlights": [ { "topic": "testing", "quality": 5, "paths": ["tests/"], "notes": "best harness we have" }, { "topic": "worldgen", "quality": 2, "notes": "prototype spaghetti — read for ideas, not patterns" } ], -"avoid": [ { "topic": "ui", "reason": "hand-rolled, superseded by roblox-game-kit" } ] +"avoid": [ { "topic": "ui", "reason": "hand-rolled, superseded by game-kit" } ] ``` A repo can be simultaneously "don't copy this" and "copy exactly this one thing", which is the truth about real codebases and something a flat rating cannot express. @@ -172,7 +172,7 @@ Each entry carries `visibility` (`private|team|public`) and `groups: [...]`. A b Layer 1 — **in-repo manifest**, `.repo-atlas.json`, committed. The repo describes itself; it travels with the code; the agent working in that repo maintains it (same discipline as `CODEBASE_DOCUMENTATION.md`). -Layer 2 — **central registry**, `~/.agent-workspace/atlas/registry.json`. Curated entries for repos with no manifest (forks, references, archived, never-cloned). This is where you write "drain-the-lake: rough, but the testing is worth reading." +Layer 2 — **central registry**, `~/.agent-workspace/atlas/registry.json`. Curated entries for repos with no manifest (forks, references, archived, never-cloned). This is where you write "puzzle-proto: rough, but the testing is worth reading." Layer 3 — **auto-discovery**, zero-effort baseline. Scan `~/GitHub` for git repos + `gh repo list` for the rest; infer kind, language, activity, fork/archive status. Produces a draft you curate rather than a blank page. Curated fields always win over inferred ones. @@ -182,12 +182,12 @@ The token-efficiency argument is the point, so the primary interface is a **dige ``` $ atlas digest --topics -roblox/luau box2d-luau(physics:5, testing:5) roblox-game-kit(mechanics:4) sabot-fps(fps-net:3) -hytopia zoo-game(data-compression:5, worldgen:4) hyfire2(matchmaking:3 ⚠ old) -monogame/c# epic-survivors(save-system:4 ⚠ old) beat-em-up-engine(input:4) +roblox/luau physics-kit(physics:5, testing:5) game-kit(mechanics:4) acme-fps(fps-net:3) +hytopia acme-tycoon(data-compression:5, worldgen:4) acme-arena(matchmaking:3 ⚠ old) +monogame/c# acme-shooter(save-system:4 ⚠ old) example-engine(input:4) ``` -Paste that into a prompt (or a `CLAUDE.md`) and the agent *has the map* — it never needs to search to know that `box2d-luau` is where the good tests live. Then `atlas show box2d-luau` for detail and `atlas find testing --min-quality 4` when it needs to look sideways. +Paste that into a prompt (or a `CLAUDE.md`) and the agent *has the map* — it never needs to search to know that `physics-kit` is where the good tests live. Then `atlas show physics-kit` for detail and `atlas find testing --min-quality 4` when it needs to look sideways. Surfaces shipped: standalone CLI (`scripts/atlas.js`, no server required — symlink into `~/.claude/scripts/`), orchestrator REST API (`/api/atlas/*`), and an agent skill so any Claude/Codex session can query it without being told how. @@ -223,8 +223,8 @@ confirming differential redaction, no private entries, and no local paths in the ### Seeded atlas state -Only highlights with actual evidence behind them were recorded — `box2d-luau` (physics, testing) and -`roblox-mechanics-encyclopedia` (architecture), all sourced from the repos' own descriptions. **No +Only highlights with actual evidence behind them were recorded — `physics-kit` (physics, testing) and +`mechanics-encyclopedia` (architecture), all sourced from the repos' own descriptions. **No quality scores were invented for the other 230.** The map is built; the judgement is deliberately left to you, because a fabricated 4/5 is worse than a blank field — it sends agents somewhere on a false promise. diff --git a/docs/COMMANDER_CLAUDE.md b/docs/COMMANDER_CLAUDE.md index 926d0a87..7ae16661 100644 --- a/docs/COMMANDER_CLAUDE.md +++ b/docs/COMMANDER_CLAUDE.md @@ -189,11 +189,11 @@ curl -sS "$BASE_URL/api/atlas/find?topic=data-compression" -H "X-Auth-Token: $AU # Compact map worth pasting into a prompt curl -sS "$BASE_URL/api/atlas/digest" -H "X-Auth-Token: $AUTH_TOKEN" | jq -r .digest -curl -sS "$BASE_URL/api/atlas/entries/zoo-game" -H "X-Auth-Token: $AUTH_TOKEN" | jq -r .description +curl -sS "$BASE_URL/api/atlas/entries/acme-tycoon" -H "X-Auth-Token: $AUTH_TOKEN" | jq -r .description curl -sS "$BASE_URL/api/atlas/topics" -H "X-Auth-Token: $AUTH_TOKEN" | jq # Record what a repo turned out to be good at -curl -sS -X POST "$BASE_URL/api/atlas/entries/zoo-game/highlights" \ +curl -sS -X POST "$BASE_URL/api/atlas/entries/acme-tycoon/highlights" \ -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ -d '{"topic": "data-compression", "quality": 5, "paths": ["src/data/"], "notes": "bitpacked saves"}' ``` @@ -231,10 +231,10 @@ After substantial work, propose what you learned. You cannot write to the map di ```bash curl -sS -X POST "$BASE_URL/api/atlas/proposals" \ -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ - -d '{"repoId":"zoo-game","topic":"data-compression","quality":5, + -d '{"repoId":"acme-tycoon","topic":"data-compression","quality":5, "paths":["src/data/"],"notes":"bitpacked saves", "evidence":"12x smaller than the JSON it replaced, benchmarked", - "proposedBy":"zoo-game-work1-claude"}' + "proposedBy":"acme-tycoon-work1-claude"}' ``` Always include `evidence`. Proposals without it get rejected, and rightly so. @@ -253,7 +253,7 @@ curl -sS "$BASE_URL/api/discord-watch/status" -H "X-Auth-Token: $AUTH_TOKEN" | j # Bind a work item to the session doing it — this is what makes agent status visible to the team curl -sS -X POST "$BASE_URL/api/discord-watch/items/discord:123/link" \ -H "X-Auth-Token: $AUTH_TOKEN" -H "Content-Type: application/json" \ - -d '{"sessionId": "zoo-game-work1-claude"}' + -d '{"sessionId": "acme-tycoon-work1-claude"}' ``` Link a work item whenever you start a session for one — an unlinked item looks untouched to everyone else. Work item tiers come from how urgently the message was phrased, and they flow into the task record, so linking also sets the session's tier correctly. diff --git a/tests/unit/repoAtlasProposals.test.js b/tests/unit/repoAtlasProposals.test.js index b6fd0e85..32d60292 100644 --- a/tests/unit/repoAtlasProposals.test.js +++ b/tests/unit/repoAtlasProposals.test.js @@ -20,15 +20,15 @@ describe('Atlas write-back', () => { }); test('a proposal does not touch the map until it is approved', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5, proposedBy: 'work1-claude' }); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 5, proposedBy: 'work1-claude' }); expect(atlas.find('testing')).toEqual([]); expect(atlas.listProposals()).toHaveLength(1); }); test('approving writes through the same path manual curation uses', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5, notes: 'good harness' }); - const result = atlas.approveProposal('zoo-game:testing'); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 5, notes: 'good harness' }); + const result = atlas.approveProposal('acme-tycoon:testing'); expect(result.ok).toBe(true); const hits = atlas.find('testing'); @@ -38,8 +38,8 @@ describe('Atlas write-back', () => { }); test('rejecting leaves the map untouched and clears the queue', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5 }); - expect(atlas.rejectProposal('zoo-game:testing').ok).toBe(true); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 5 }); + expect(atlas.rejectProposal('acme-tycoon:testing').ok).toBe(true); expect(atlas.find('testing')).toEqual([]); expect(atlas.listProposals()).toEqual([]); @@ -47,8 +47,8 @@ describe('Atlas write-back', () => { }); test('a second proposal for the same topic supersedes the first rather than stacking', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 3 }); - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 5 }); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 3 }); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 5 }); const pending = atlas.listProposals(); expect(pending).toHaveLength(1); @@ -57,25 +57,25 @@ describe('Atlas write-back', () => { }); test('topic aliases are normalized so proposals do not fragment the vocabulary', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'unit-tests', quality: 4 }); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'unit-tests', quality: 4 }); expect(atlas.listProposals()[0].topic).toBe('testing'); }); test('an avoid proposal records a do-not-copy note when approved', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'ui', kind: 'avoid', notes: 'hand-rolled, superseded' }); - atlas.approveProposal('zoo-game:ui'); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'ui', kind: 'avoid', notes: 'hand-rolled, superseded' }); + atlas.approveProposal('acme-tycoon:ui'); - expect(atlas.getEntry('zoo-game').avoid).toEqual([{ topic: 'ui', reason: 'hand-rolled, superseded' }]); + expect(atlas.getEntry('acme-tycoon').avoid).toEqual([{ topic: 'ui', reason: 'hand-rolled, superseded' }]); }); test('quality is clamped, so an over-eager agent cannot invent a 9/5', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 9 }); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 9 }); expect(atlas.listProposals()[0].quality).toBe(5); }); test('proposals without a repo or topic are refused', () => { expect(() => atlas.proposeHighlight({ topic: 'testing' })).toThrow(/repo id/); - expect(() => atlas.proposeHighlight({ repoId: 'zoo-game' })).toThrow(/topic/); + expect(() => atlas.proposeHighlight({ repoId: 'acme-tycoon' })).toThrow(/topic/); }); test('deciding an unknown proposal reports it instead of failing silently', () => { @@ -85,7 +85,7 @@ describe('Atlas write-back', () => { test('evidence travels with the proposal so review takes seconds', () => { atlas.proposeHighlight({ - repoId: 'zoo-game', + repoId: 'acme-tycoon', topic: 'testing', quality: 4, evidence: 'added 40 tests in tests/unit, all green', @@ -107,7 +107,7 @@ describe('Atlas write-back', () => { }); test('proposal stats surface in atlas status', () => { - atlas.proposeHighlight({ repoId: 'zoo-game', topic: 'testing', quality: 4 }); + atlas.proposeHighlight({ repoId: 'acme-tycoon', topic: 'testing', quality: 4 }); expect(atlas.getStatus().proposals.pending).toBe(1); }); }); diff --git a/tests/unit/repoAtlasQuery.test.js b/tests/unit/repoAtlasQuery.test.js index 5805c4e4..2bb6cd24 100644 --- a/tests/unit/repoAtlasQuery.test.js +++ b/tests/unit/repoAtlasQuery.test.js @@ -6,12 +6,12 @@ const daysAgo = (days) => new Date(Date.now() - days * DAY_MS).toISOString(); const entries = [ normalizeEntry({ - id: 'box2d-luau', + id: 'physics-kit', kind: 'library', platforms: ['roblox'], languages: ['Luau'], cloned: true, - localPath: '/repos/box2d-luau', + localPath: '/repos/physics-kit', lastActivity: daysAgo(2), highlights: [ { topic: 'physics', quality: 5, paths: ['src/'], notes: 'faithful port' }, @@ -19,7 +19,7 @@ const entries = [ ] }, { strict: true }), normalizeEntry({ - id: 'drain-the-lake', + id: 'puzzle-proto', kind: 'game', platforms: ['roblox'], cloned: true, @@ -28,7 +28,7 @@ const entries = [ avoid: [{ topic: 'architecture', reason: 'prototype spaghetti' }] }, { strict: true }), normalizeEntry({ - id: 'epic-survivors', + id: 'acme-shooter', kind: 'game', platforms: ['monogame'], languages: ['C#'], @@ -55,53 +55,53 @@ describe('atlasQuery', () => { test('minQuality filters on the best highlight a repo has', () => { const ids = filterEntries(entries, { minQuality: 5 }).map((e) => e.id); - expect(ids).toEqual(['box2d-luau']); + expect(ids).toEqual(['physics-kit']); }); test('filters compose across kind, platform and fork state', () => { expect(filterEntries(entries, { platform: 'roblox' }).map((e) => e.id)) - .toEqual(['box2d-luau', 'drain-the-lake']); + .toEqual(['physics-kit', 'puzzle-proto']); expect(filterEntries(entries, { includeForks: false }).map((e) => e.id)).not.toContain('some-fork'); - expect(filterEntries(entries, { includeArchived: false }).map((e) => e.id)).not.toContain('epic-survivors'); + expect(filterEntries(entries, { includeArchived: false }).map((e) => e.id)).not.toContain('acme-shooter'); }); test('text search reaches highlight notes, not just names', () => { - expect(filterEntries(entries, { query: 'best harness' }).map((e) => e.id)).toEqual(['box2d-luau']); + expect(filterEntries(entries, { query: 'best harness' }).map((e) => e.id)).toEqual(['physics-kit']); }); test('findByTopic ranks by quality and resolves topic aliases', () => { const hits = findByTopic(entries, 'unit-tests'); - expect(hits.map((h) => h.id)).toEqual(['box2d-luau', 'drain-the-lake']); + expect(hits.map((h) => h.id)).toEqual(['physics-kit', 'puzzle-proto']); expect(hits[0].quality).toBe(5); }); test('findByTopic marks long-untouched repos as stale', () => { const hits = findByTopic(entries, 'testing'); - expect(hits.find((h) => h.id === 'drain-the-lake').stale).toBe(true); - expect(hits.find((h) => h.id === 'box2d-luau').stale).toBe(false); + expect(hits.find((h) => h.id === 'puzzle-proto').stale).toBe(true); + expect(hits.find((h) => h.id === 'physics-kit').stale).toBe(false); }); test('findByTopic honours a quality floor', () => { - expect(findByTopic(entries, 'testing', { minQuality: 4 }).map((h) => h.id)).toEqual(['box2d-luau']); + expect(findByTopic(entries, 'testing', { minQuality: 4 }).map((h) => h.id)).toEqual(['physics-kit']); }); test('an avoid entry hides that repo for that topic only', () => { expect(findByTopic(entries, 'architecture')).toEqual([]); - expect(findByTopic(entries, 'testing').map((h) => h.id)).toContain('drain-the-lake'); + expect(findByTopic(entries, 'testing').map((h) => h.id)).toContain('puzzle-proto'); }); test('topicIndex summarizes who has what', () => { const index = topicIndex(entries); const testing = index.find((row) => row.topic === 'testing'); expect(testing.count).toBe(2); - expect(testing.repos[0]).toBe('box2d-luau'); + expect(testing.repos[0]).toBe('physics-kit'); }); test('digest groups by platform and flags stale repos', () => { const digest = buildDigest(entries, { groupBy: 'platform' }); expect(digest).toMatch(/roblox/); - expect(digest).toMatch(/box2d-luau\(physics:5, testing:5\)/); - expect(digest).toMatch(/drain-the-lake\(testing:3 ⚠old\)/); + expect(digest).toMatch(/physics-kit\(physics:5, testing:5\)/); + expect(digest).toMatch(/puzzle-proto\(testing:3 ⚠old\)/); expect(digest).not.toMatch(/some-fork/); }); diff --git a/tests/unit/repoAtlasSchema.test.js b/tests/unit/repoAtlasSchema.test.js index 57d48b0c..856cd2db 100644 --- a/tests/unit/repoAtlasSchema.test.js +++ b/tests/unit/repoAtlasSchema.test.js @@ -23,10 +23,10 @@ describe('atlasSchema', () => { }); test('normalizeEntry keeps only supplied keys unless strict', () => { - const partial = normalizeEntry({ id: 'zoo-game', quality: 9 }); - expect(partial).toEqual({ id: 'zoo-game', quality: 5 }); + const partial = normalizeEntry({ id: 'acme-tycoon', quality: 9 }); + expect(partial).toEqual({ id: 'acme-tycoon', quality: 5 }); - const strict = normalizeEntry({ id: 'zoo-game' }, { strict: true }); + const strict = normalizeEntry({ id: 'acme-tycoon' }, { strict: true }); expect(strict.visibility).toBe('private'); expect(strict.highlights).toEqual([]); }); @@ -59,7 +59,7 @@ describe('atlasSchema', () => { test('mergeEntries lets later layers win per field without wiping earlier ones', () => { const merged = mergeEntries( - { __source: 'discovery', id: 'zoo-game', name: 'zoo-game', kind: 'game', languages: ['TypeScript'] }, + { __source: 'discovery', id: 'acme-tycoon', name: 'acme-tycoon', kind: 'game', languages: ['TypeScript'] }, { __source: 'manifest', summary: 'Multiplayer zoo tycoon', highlights: [{ topic: 'networking', quality: 3 }] }, { __source: 'registry', visibility: 'team', groups: ['core-team'] } ); diff --git a/tests/unit/repoAtlasService.test.js b/tests/unit/repoAtlasService.test.js index 70cc5c00..e9ac4f7c 100644 --- a/tests/unit/repoAtlasService.test.js +++ b/tests/unit/repoAtlasService.test.js @@ -12,16 +12,16 @@ describe('RepoAtlasService', () => { beforeEach(() => { tmpDir = fs.mkdtempSync(path.join(os.tmpdir(), 'atlas-test-')); - repoDir = path.join(tmpDir, 'repos', 'zoo-game'); + repoDir = path.join(tmpDir, 'repos', 'acme-tycoon'); fs.mkdirSync(repoDir, { recursive: true }); process.env.AGENT_WORKSPACE_ATLAS_DIR = path.join(tmpDir, 'atlas'); atlas = new RepoAtlasService(); store.saveDiscoveryCache([{ __source: 'discovery', - id: 'zoo-game', - name: 'zoo-game', - repo: 'owner/zoo-game', + id: 'acme-tycoon', + name: 'acme-tycoon', + repo: 'owner/acme-tycoon', kind: 'game', languages: ['TypeScript'], localPath: repoDir, @@ -36,7 +36,7 @@ describe('RepoAtlasService', () => { }); test('discovery alone produces a usable entry', () => { - const entry = atlas.getEntry('zoo-game'); + const entry = atlas.getEntry('acme-tycoon'); expect(entry.kind).toBe('game'); expect(entry.visibility).toBe('private'); expect(entry.sources).toEqual(['discovery']); @@ -44,13 +44,13 @@ describe('RepoAtlasService', () => { test('an in-repo manifest layers over discovery', () => { fs.writeFileSync(path.join(repoDir, '.repo-atlas.json'), JSON.stringify({ - id: 'zoo-game', + id: 'acme-tycoon', summary: 'Multiplayer zoo tycoon', highlights: [{ topic: 'data-compression', quality: 5, notes: 'bitpacked saves' }] })); atlas.invalidate(); - const entry = atlas.getEntry('zoo-game'); + const entry = atlas.getEntry('acme-tycoon'); expect(entry.summary).toBe('Multiplayer zoo tycoon'); expect(entry.kind).toBe('game'); expect(entry.highlights[0].topic).toBe('data-compression'); @@ -59,31 +59,31 @@ describe('RepoAtlasService', () => { test('the registry overrides the manifest — your opinion wins', () => { fs.writeFileSync(path.join(repoDir, '.repo-atlas.json'), JSON.stringify({ - id: 'zoo-game', + id: 'acme-tycoon', summary: 'From the repo', maturity: 'production' })); - atlas.setEntry('zoo-game', { summary: 'From you', maturity: 'prototype' }); + atlas.setEntry('acme-tycoon', { summary: 'From you', maturity: 'prototype' }); - const entry = atlas.getEntry('zoo-game'); + const entry = atlas.getEntry('acme-tycoon'); expect(entry.summary).toBe('From you'); expect(entry.maturity).toBe('prototype'); expect(entry.sources).toEqual(['discovery', 'manifest', 'registry']); }); test('addHighlight persists and replaces the same topic', () => { - atlas.addHighlight('zoo-game', { topic: 'multiplayer', quality: 3, notes: 'chatty' }); - atlas.addHighlight('zoo-game', { topic: 'networking', quality: 5, paths: ['src/net'], notes: 'rewritten' }); + atlas.addHighlight('acme-tycoon', { topic: 'multiplayer', quality: 3, notes: 'chatty' }); + atlas.addHighlight('acme-tycoon', { topic: 'networking', quality: 5, paths: ['src/net'], notes: 'rewritten' }); - const entry = new RepoAtlasService().getEntry('zoo-game'); + const entry = new RepoAtlasService().getEntry('acme-tycoon'); expect(entry.highlights).toEqual([ { topic: 'networking', quality: 5, paths: ['src/net'], notes: 'rewritten' } ]); }); test('addAvoid records a do-not-copy note without removing the repo', () => { - atlas.addAvoid('zoo-game', { topic: 'ui', reason: 'hand-rolled' }); - const entry = atlas.getEntry('zoo-game'); + atlas.addAvoid('acme-tycoon', { topic: 'ui', reason: 'hand-rolled' }); + const entry = atlas.getEntry('acme-tycoon'); expect(entry.avoid).toEqual([{ topic: 'ui', reason: 'hand-rolled' }]); expect(atlas.find('ui')).toEqual([]); }); @@ -104,20 +104,20 @@ describe('RepoAtlasService', () => { test('compile writes an audience bundle and withholds private entries', () => { atlas.setAudience({ id: 'core-team', label: 'Core team' }); - atlas.setEntry('zoo-game', { visibility: 'team', groups: ['core-team'] }); + atlas.setEntry('acme-tycoon', { visibility: 'team', groups: ['core-team'] }); atlas.setEntry('secret-thing', { name: 'secret', visibility: 'private' }); const result = atlas.compile('core-team'); expect(result.counts.included).toBe(1); - expect(result.bundle.entries[0].id).toBe('zoo-game'); + expect(result.bundle.entries[0].id).toBe('acme-tycoon'); const written = JSON.parse(fs.readFileSync(result.written[0], 'utf8')); - expect(written.entries.map((e) => e.id)).toEqual(['zoo-game']); + expect(written.entries.map((e) => e.id)).toEqual(['acme-tycoon']); expect(JSON.stringify(written)).not.toContain(repoDir); }); test('compile --dry-run writes nothing', () => { - atlas.setEntry('zoo-game', { visibility: 'public' }); + atlas.setEntry('acme-tycoon', { visibility: 'public' }); const result = atlas.compile('core-team', { write: false }); expect(result.written).toEqual([]); expect(fs.existsSync(store.bundlesDir())).toBe(false); @@ -126,7 +126,7 @@ describe('RepoAtlasService', () => { test('initManifest seeds a manifest from what is already known', () => { const { path: manifestPath, entry } = atlas.initManifest(repoDir); expect(fs.existsSync(manifestPath)).toBe(true); - expect(entry.id).toBe('zoo-game'); + expect(entry.id).toBe('acme-tycoon'); expect(entry.kind).toBe('game'); expect(entry.visibility).toBe('private'); }); @@ -138,7 +138,7 @@ describe('RepoAtlasService', () => { }); test('getStatus reports where data lives and how much is curated', () => { - atlas.addHighlight('zoo-game', { topic: 'testing', quality: 4 }); + atlas.addHighlight('acme-tycoon', { topic: 'testing', quality: 4 }); const status = atlas.getStatus(); expect(status.entryCount).toBe(1); expect(status.clonedCount).toBe(1); diff --git a/tests/unit/repoAtlasSync.test.js b/tests/unit/repoAtlasSync.test.js index 710b930b..b1b9d317 100644 --- a/tests/unit/repoAtlasSync.test.js +++ b/tests/unit/repoAtlasSync.test.js @@ -32,16 +32,16 @@ describe('Repo Atlas multi-machine sync', () => { test('curated entries are one file per repo so machines cannot conflict', () => { const atlas = machine('a'); - atlas.addHighlight('box2d-luau', { topic: 'testing', quality: 5 }); - atlas.addHighlight('zoo-game', { topic: 'networking', quality: 3 }); + atlas.addHighlight('physics-kit', { topic: 'testing', quality: 5 }); + atlas.addHighlight('acme-tycoon', { topic: 'networking', quality: 3 }); - expect(fs.readdirSync(store.entriesDir()).sort()).toEqual(['box2d-luau.json', 'zoo-game.json']); + expect(fs.readdirSync(store.entriesDir()).sort()).toEqual(['acme-tycoon.json', 'physics-kit.json']); }); test('judgement travels between machines; local discovery does not', async () => { const a = machine('a'); await a.setRemote(remote); - a.addHighlight('box2d-luau', { topic: 'testing', quality: 5, notes: 'best harness we have' }); + a.addHighlight('physics-kit', { topic: 'testing', quality: 5, notes: 'best harness we have' }); store.saveDiscoveryCache([{ id: 'only-on-a', name: 'only-on-a', localPath: '/machine/a/only-on-a', cloned: true }]); expect((await a.sync()).ok).toBe(true); From 829fa7dffcc3f0ed6f4938f8a4f86895e9506cc4 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:31:34 +1000 Subject: [PATCH 65/69] fix(voice): imperative 'stop/cancel ' is a command, not a negation The negation guard matched any utterance starting with stop/cancel, so a valid command that missed an exact rule phrasing ('stop the server') was short-circuited away from the LLM classifier and misrouted to the Commander. Only the dismissal forms ('stop that', 'cancel it', 'forget that') short-circuit now; don't/never/no/nope still guard as before. Co-Authored-By: Claude Fable 5 --- server/voiceCommandService.js | 8 ++++++-- tests/unit/voiceCommandService.test.js | 15 +++++++++++++++ 2 files changed, 21 insertions(+), 2 deletions(-) diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index c3b65ff9..5d062e6e 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -1270,8 +1270,12 @@ class VoiceCommandService { // A negation ("don't open the queue", "cancel that") must never be turned // into the command it negates. The fact lane already handled bare acks // ("never mind"); anything still negating here skips the classifier and is - // handed to the Commander, which understands "don't". - if (/^(don'?t\b|do not\b|never\b|stop\b|cancel\b|no,?\s|nope\b)/.test(text)) { + // handed to the Commander, which understands "don't". Bare "stop …" and + // "cancel …" are NOT negations though — they are imperative commands + // ("stop the server") that just missed an exact rule phrasing, so only + // their dismissal forms ("stop that", "cancel it") short-circuit. + if (/^(don'?t\b|do not\b|never\b|no,?\s|nope\b)/.test(text) + || /^(stop|cancel|forget)\s+(that|it)\b/.test(text)) { return { success: false, error: 'negation is not a command', transcript: text }; } diff --git a/tests/unit/voiceCommandService.test.js b/tests/unit/voiceCommandService.test.js index acfbe97c..953e42e5 100644 --- a/tests/unit/voiceCommandService.test.js +++ b/tests/unit/voiceCommandService.test.js @@ -409,4 +409,19 @@ describe('VoiceCommandService (free-form routing)', () => { expect(parsed.success).toBe(false); expect(parsed.error).toMatch(/too short/i); }); + + test('an imperative "stop/cancel " is not swallowed as a negation', async () => { + // "stop the server" is a command that merely missed an exact rule phrasing. + // The old guard treated any leading stop/cancel as negation, so it was + // short-circuited away from the classifier and misrouted to the Commander. + const parsed = await voiceCommandService.parseCommand('stop the server'); + expect(parsed.error).not.toMatch(/negation/i); + + // The dismissal forms still short-circuit. + for (const dismissal of ['stop that', 'cancel it', 'forget that']) { + const dismissed = await voiceCommandService.parseCommand(dismissal); + expect(dismissed.success).toBe(false); + expect(dismissed.error).toMatch(/negation/i); + } + }); }); From 3756f6cb972f90405d7db622ee0f814ecd240c8c Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:37:00 +1000 Subject: [PATCH 66/69] fix(voice): negated destructive phrases can no longer execute; harden grounding + capture MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - CRITICAL: rule matching ran BEFORE the negation guard, and many rule patterns are unanchored substrings — verified live, "don't stop all claudes" matched stop-all-claudes and executed it fleet-wide with no confirmation gate ('don't kill work 3' likewise). Negation is now detected first and skips rule matching entirely; the utterance goes to the Commander, which understands "don't". - /what.*sessions/ was greedy enough to steal natural fact questions ("what are my sessions doing") from the fact lane, speaking a useless "Done — list sessions." instead of the real fleet status. Only enumerative phrasings match the rule now. - isGrounded accepted a hallucinated destructive command if ANY token matched — "is my session about to time out" grounded kill-session via the word 'session'. A destructive command now requires its verb to have actually been heard. - Commander-reply captures are serialized: two concurrent polls over the one shared PTY buffer could speak one utterance's answer for the other. Co-Authored-By: Claude Fable 5 --- server/voice/voiceBrainService.js | 7 +++- server/voiceCommandService.js | 57 +++++++++++++++++--------- tests/unit/voiceCommandService.test.js | 28 +++++++++++++ 3 files changed, 72 insertions(+), 20 deletions(-) diff --git a/server/voice/voiceBrainService.js b/server/voice/voiceBrainService.js index 1c203d6f..1b6aefbf 100644 --- a/server/voice/voiceBrainService.js +++ b/server/voice/voiceBrainService.js @@ -23,6 +23,10 @@ class VoiceBrainService { constructor({ logger = console } = {}) { this.logger = logger; this.deps = {}; + // Reply captures poll ONE shared Commander buffer, so they must run one + // at a time — two interleaved polling loops would each take the newest + // tail and speak one utterance's answer for the other. + this.captureChain = Promise.resolve(); } static getInstance(options = {}) { @@ -273,7 +277,8 @@ class VoiceBrainService { // in the background, so the request returns now and the answer arrives when // the agent is done. This is the two-way loop. this.speak('On it.'); - this.captureCommanderReply(before) + this.captureChain = this.captureChain + .then(() => this.captureCommanderReply(before)) .then((reply) => { if (reply) this.speak(reply, { priority: 'high' }); }) .catch((error) => this.logger.warn?.('voice brain: reply capture failed', { error: error.message })); diff --git a/server/voiceCommandService.js b/server/voiceCommandService.js index 5d062e6e..91430f35 100644 --- a/server/voiceCommandService.js +++ b/server/voiceCommandService.js @@ -849,12 +849,15 @@ class VoiceCommandService { return { worktreeId: `work${num}` }; } }, - // List sessions + // List sessions. The old /what.*sessions/ variant was greedy enough to + // steal natural fact questions ("what are my sessions doing") from the + // fact lane, which answers them properly — so only the enumerative + // phrasings match here. { patterns: [ /list\s+sessions/i, /show\s+sessions/i, - /what.*sessions/i, + /^what\s+sessions\b/i, ], command: 'list-sessions', extractParams: () => ({}) @@ -1232,14 +1235,25 @@ class VoiceCommandService { // Clean up transcript const text = transcript.toLowerCase().trim(); + // A negation must be detected BEFORE rule matching, not after: many rule + // patterns are unanchored substrings, so "don't stop all claudes" contains + // a perfectly matching "stop all claudes" — and with no confirmation gate + // between parse and execute, the spoken sentence would do the exact + // opposite of what was said. Bare "stop …"/"cancel …" are imperative + // commands, not negations; only their dismissal forms short-circuit. + const negated = /^(don'?t\b|do not\b|never\b|no,?\s|nope\b)/.test(text) + || /^(stop|cancel|forget)\s+(that|it)\b/.test(text); + // Try rule-based parsing first (instant, free) - const ruleResult = this.parseWithRules(text); - if (ruleResult) { - return { - success: true, - method: 'rules', - ...ruleResult - }; + if (!negated) { + const ruleResult = this.parseWithRules(text); + if (ruleResult) { + return { + success: true, + method: 'rules', + ...ruleResult + }; + } } // Typed input (e.g. Commander panel slash commands) wants a fast, @@ -1267,15 +1281,10 @@ class VoiceCommandService { } catch { /* fall through to the classifier */ } } - // A negation ("don't open the queue", "cancel that") must never be turned - // into the command it negates. The fact lane already handled bare acks - // ("never mind"); anything still negating here skips the classifier and is - // handed to the Commander, which understands "don't". Bare "stop …" and - // "cancel …" are NOT negations though — they are imperative commands - // ("stop the server") that just missed an exact rule phrasing, so only - // their dismissal forms ("stop that", "cancel it") short-circuit. - if (/^(don'?t\b|do not\b|never\b|no,?\s|nope\b)/.test(text) - || /^(stop|cancel|forget)\s+(that|it)\b/.test(text)) { + // A negation that wasn't a bare ack (the fact lane above owns "never + // mind") skips the classifier and is handed to the Commander, which + // understands "don't". + if (negated) { return { success: false, error: 'negation is not a command', transcript: text }; } @@ -1335,9 +1344,19 @@ class VoiceCommandService { pager: ['pager', 'ping'], advice: ['advice', 'recommend', 'suggest'], chats: ['chat', 'message'], project: ['project'], claude: ['claude'], all: ['all', 'everything'], mode: ['mode'], tier: ['tier'], new: ['new', 'create', 'start'], open: ['open', 'show', 'pull up', 'bring up', 'go to'], start: ['start', 'launch', 'run'], - stop: ['stop', 'kill', 'end'], status: ['status', 'state'] + stop: ['stop', 'kill', 'end'], status: ['status', 'state'], + kill: ['kill', 'stop', 'end', 'terminate'], destroy: ['destroy', 'delete', 'remove'], + remove: ['remove', 'delete', 'drop'], close: ['close', 'shut'], merge: ['merge'], approve: ['approve', 'accept'] }; const parts = String(command).split(/[-_]/).filter((w) => w.length > 2); + // A destructive command must be grounded by its VERB, not by an incidental + // noun — "is my session about to time out?" mentions a session but must + // never ground a hallucinated kill-session from the small local model. + const destructive = ['kill', 'stop', 'destroy', 'remove', 'delete', 'close', 'merge', 'approve']; + if (destructive.includes(parts[0])) { + const verbHeard = t.includes(parts[0]) || (synonyms[parts[0]] || []).some((s) => t.includes(s)); + if (!verbHeard) return false; + } return parts.some((w) => t.includes(w) || (synonyms[w] || []).some((s) => t.includes(s))); } diff --git a/tests/unit/voiceCommandService.test.js b/tests/unit/voiceCommandService.test.js index 953e42e5..04b2357b 100644 --- a/tests/unit/voiceCommandService.test.js +++ b/tests/unit/voiceCommandService.test.js @@ -410,6 +410,34 @@ describe('VoiceCommandService (free-form routing)', () => { expect(parsed.error).toMatch(/too short/i); }); + test('a negated destructive phrase never executes the rule it contains', async () => { + // Many rule patterns are unanchored substrings, so "don't stop all + // claudes" CONTAINS a perfect "stop all claudes" match. Before the fix, + // rules ran ahead of the negation guard and these executed for real — + // fleet-wide, with no confirmation gate. + for (const phrase of ["don't stop all claudes", "don't kill work 3", "don't destroy work 3", 'never stop work 2']) { + const parsed = await voiceCommandService.parseCommand(phrase); + expect(parsed.success).toBe(false); + expect(parsed.error).toMatch(/negation/i); + } + }); + + test('natural session-status questions are left for the fact lane', () => { + // /what.*sessions/ was greedy enough to steal these from the fact lane, + // answering a real question with a spoken "Done — list sessions." + expect(voiceCommandService.parseWithRules('what are my sessions doing')).toBeFalsy(); + // Enumerative phrasings still resolve to the command. + expect(voiceCommandService.parseWithRules('list sessions')?.command).toBe('list-sessions'); + expect(voiceCommandService.parseWithRules('what sessions do i have')?.command).toBe('list-sessions'); + }); + + test('a destructive command must be grounded by its verb, not an incidental noun', () => { + expect(voiceCommandService.isGrounded('kill-session', 'is my session about to time out')).toBe(false); + expect(voiceCommandService.isGrounded('kill-session', 'kill the session on work 3')).toBe(true); + expect(voiceCommandService.isGrounded('stop-server', 'is the server up')).toBe(false); + expect(voiceCommandService.isGrounded('stop-server', 'shut down the server, kill it')).toBe(true); + }); + test('an imperative "stop/cancel " is not swallowed as a negation', async () => { // "stop the server" is a command that merely missed an exact rule phrasing. // The old guard treated any leading stop/cancel as negation, so it was From 6baec23e00852e8f5b656beb24e8623b2ca63794 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:41:23 +1000 Subject: [PATCH 67/69] fix(supervisor): close permission-classifier bypasses; clear healed cooldowns; audit rule reloads MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The auto-approve deny list was bypassable three ways, all reproduced against the live loaded rules: - Fully case-sensitive matching: Write(.ENV) and lowercase SQL (drop table / delete from / truncate) evaded denies written as \.env and \bDROP\b. Deny patterns now compile case-insensitive (fail-closed direction only — allow patterns stay exact so nothing NEW auto-approves). - The credentials deny only knew the ~/ form; Write(/home/x/.ssh/...) — the way agents actually render out-of-repo paths — was auto-approved. Absolute /home, /Users and /root forms are now denied too. - Quoted paths (Write("package.json")) slipped the exec-on-next-op anchors; quote/space/equals now count as boundaries, and git push -f is denied like --force. Also: forgetHealed() never cleared cooldowns, so a stale timestamp from an already-healed occurrence silently blocked action on a genuinely new recurrence (and the map grew unbounded); reload-rules now writes an audit entry like every other config-changing action. Co-Authored-By: Claude Fable 5 --- config/supervisor-rules.json | 10 +++++----- server/supervisor/supervisorRules.js | 10 +++++++--- server/supervisorService.js | 10 ++++++++++ tests/unit/supervisorActions.test.js | 25 +++++++++++++++++++++++++ tests/unit/supervisorService.test.js | 16 ++++++++++++++++ 5 files changed, 63 insertions(+), 8 deletions(-) diff --git a/config/supervisor-rules.json b/config/supervisor-rules.json index dad5bc09..51af3b2b 100644 --- a/config/supervisor-rules.json +++ b/config/supervisor-rules.json @@ -33,15 +33,15 @@ ], "$comment_deny_paths": "A permission prompt to Edit/Write is auto-approvable in general, but NOT for targets that execute on the next ordinary operation — a poisoned build script, git hook, CI workflow or shell rc turns an approved edit plus an already-allowlisted `npm run build`/`git commit` into arbitrary code execution. Any write to these fails closed to a human.", "permissionDenyPatterns": [ - "\\brm\\s+-[rf]", "\\bsudo\\b", "\\bgit (push --force|reset --hard|clean)\\b", + "\\brm\\s+-[rf]", "\\bsudo\\b", "\\bgit (push\\s+(--force|-f\\b)|reset --hard|clean)\\b", "\\bgh (pr merge|release|repo delete|repo edit)\\b", "\\bDROP\\b", "\\bTRUNCATE\\b", "\\bDELETE FROM\\b", "\\bcurl\\b[^\\n]*\\|\\s*(sh|bash)", "\\bchmod\\s+777\\b", "\\bkill(all)?\\b", "\\bnpm publish\\b", "\\bdd\\s+if=", "\\bmkfs\\b", - "~/\\.(ssh|aws|config|codex|claude)\\b", "\\.env\\b", + "(~|/home/[^/\\s\"']+|/Users/[^/\\s\"']+|/root)/\\.(ssh|aws|config|codex|claude)\\b", "\\.env\\b", "\\.git/(hooks|config)", "\\.github/(workflows|actions)\\b", - "(^|[/(])(package(-lock)?\\.json|pnpm-lock\\.yaml|yarn\\.lock)\\b", - "(^|[/(])(Makefile|Rakefile|Gemfile|Cargo\\.toml|pyproject\\.toml|setup\\.py|build\\.gradle|pom\\.xml)\\b", - "(^|[/(.])(bashrc|zshrc|bash_profile|zshenv|profile|gitconfig|npmrc|pre-commit-config\\.yaml)\\b", + "(^|[/(\"'\\s=])(package(-lock)?\\.json|pnpm-lock\\.yaml|yarn\\.lock)\\b", + "(^|[/(\"'\\s=])(Makefile|Rakefile|Gemfile|Cargo\\.toml|pyproject\\.toml|setup\\.py|build\\.gradle|pom\\.xml)\\b", + "(^|[/(.\"'\\s=])(bashrc|zshrc|bash_profile|zshenv|profile|gitconfig|npmrc|pre-commit-config\\.yaml)\\b", "(^|[\\s(])/etc/" ] }, diff --git a/server/supervisor/supervisorRules.js b/server/supervisor/supervisorRules.js index a641f00b..cfad7232 100644 --- a/server/supervisor/supervisorRules.js +++ b/server/supervisor/supervisorRules.js @@ -25,11 +25,11 @@ function overrideRulesPath() { return path.join(getAgentWorkspaceDir(), 'supervisor-rules.json'); } -function compilePatterns(patterns) { +function compilePatterns(patterns, flags = '') { const out = []; for (const pattern of Array.isArray(patterns) ? patterns : []) { try { - out.push(new RegExp(String(pattern))); + out.push(new RegExp(String(pattern), flags)); } catch { // A bad pattern must not take the whole supervisor down with it. } @@ -125,7 +125,11 @@ function loadRules({ rulesPath = null } = {}) { safety: { allowedHandlers: Array.isArray(safety.allowedHandlers) ? safety.allowedHandlers.map(String) : [], permissionAllowPatterns: compilePatterns(safety.permissionAllowPatterns), - permissionDenyPatterns: compilePatterns(safety.permissionDenyPatterns) + // Deny patterns are case-insensitive on purpose (fail closed): the + // case-sensitive form let `Write(.ENV)` and lowercase `drop table` + // sail past denies written as `\.env` and `\bDROP\b`. Allow patterns + // stay exact — widening what auto-approves is the wrong direction. + permissionDenyPatterns: compilePatterns(safety.permissionDenyPatterns, 'i') }, conditions: (Array.isArray(raw.conditions) ? raw.conditions : []).map(normalizeCondition).filter(Boolean) }; diff --git a/server/supervisorService.js b/server/supervisorService.js index 1c3b748d..8330ef5c 100644 --- a/server/supervisorService.js +++ b/server/supervisorService.js @@ -123,6 +123,9 @@ class SupervisorService { this.budget.setPolicy(this.rules.interruption); this.digest.setInterval(this.rules.interruption.digestIntervalMinutes); if (this.running) this.restartTimer(); + // A reload can swap autonomy and the entire safety table — that must be + // as traceable as setAutonomy() is. + this.appendAudit({ event: 'rules-reloaded', source: this.rules.source, autonomy: this.rules.autonomy }); return this.rules; } @@ -239,6 +242,13 @@ class SupervisorService { this.appendAudit({ event: 'self-healed', id }); } } + // Cooldowns are cleared separately: they are keyed like attempts but can + // exist without an attempts entry (interrupt path). A stale cooldown from + // an already-healed occurrence silently blocked action on a genuinely new + // recurrence of the same finding — and grew this map without bound. + for (const id of [...this.cooldowns.keys()]) { + if (!activeFindingIds.has(id)) this.cooldowns.delete(id); + } } /** diff --git a/tests/unit/supervisorActions.test.js b/tests/unit/supervisorActions.test.js index 55bf922a..ff827acd 100644 --- a/tests/unit/supervisorActions.test.js +++ b/tests/unit/supervisorActions.test.js @@ -88,6 +88,31 @@ describe('permission classification', () => { test('an empty prompt is not safe', () => { expect(classifyPermissionPrompt('', rules.safety).safe).toBe(false); }); + + test('case cannot be used to slip past the deny list', () => { + // The classifier used to be fully case-sensitive: Write(.ENV) and + // lowercase SQL sailed past denies written as \.env and \bDROP\b. + expect(classifyPermissionPrompt('Write(.ENV)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Bash(psql -c "drop table users")', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Bash(mysql -e "delete from accounts")', rules.safety).safe).toBe(false); + }); + + test('credential paths are denied in absolute form, not only the ~/ form', () => { + // Claude/Codex render out-of-repo paths absolute — that is the realistic + // rendering, and it used to auto-approve. + expect(classifyPermissionPrompt('Write(/home/someone/.ssh/authorized_keys)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Edit(/Users/someone/.aws/credentials)', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt('Read(~/.ssh/id_rsa)', rules.safety).safe).toBe(false); + }); + + test('quoting a build-file path does not evade the exec-on-next-op denies', () => { + expect(classifyPermissionPrompt('Write("package.json")', rules.safety).safe).toBe(false); + expect(classifyPermissionPrompt("Edit('Makefile')", rules.safety).safe).toBe(false); + }); + + test('git push -f is denied like git push --force', () => { + expect(classifyPermissionPrompt('Bash(git push -f origin main)', rules.safety).safe).toBe(false); + }); }); describe('parseResetTime', () => { diff --git a/tests/unit/supervisorService.test.js b/tests/unit/supervisorService.test.js index 8fc63ac6..293229e4 100644 --- a/tests/unit/supervisorService.test.js +++ b/tests/unit/supervisorService.test.js @@ -299,4 +299,20 @@ describe('SupervisorService', () => { supervisor.stop(); expect(supervisor.resumeTimers.size).toBe(0); }); + + test('a healed finding releases its cooldown so a fresh recurrence can be acted on', () => { + const { supervisor } = harness(); + supervisor.attempts.set('f1', 2); + supervisor.cooldowns.set('f1', Date.now()); + // A cooldown can exist without an attempts entry (interrupt path). + supervisor.cooldowns.set('f2', Date.now()); + + supervisor.forgetHealed(new Set()); + + expect(supervisor.attempts.size).toBe(0); + // Before the fix these stale timestamps survived healing, silently + // classifying a genuinely new recurrence as 'cooling-down' for up to the + // full window — and the map grew without bound. + expect(supervisor.cooldowns.size).toBe(0); + }); }); From ff3b172486dcb0be23c1b814d840dfb3f0fcc5da Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:47:12 +1000 Subject: [PATCH 68/69] fix(discord): real guild permalinks, backward backfill paging, persisted channels, awaited task records MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Permalinks always rendered the DM-only /channels/@me/... form: REST message objects don't carry guild_id and nothing looked it up. The guild id is now fetched once per channel (cached in state), so links into guild channels — the primary use case — actually work. - A first-sight backfill paged FORWARD from the newest message, so any backfillMessages > 100 silently capped at 100 (the second page asked for messages after the newest and got nothing). Backfills now walk before= into history and keep the most recent N. - addChannel/removeChannel only mutated memory; a channel added over the API vanished on reload-config or restart. Both now persist to the override config file the reload reads. - linkSession's task-record upsert is async but wasn't awaited — its try/catch was dead code and a failure became an unhandled rejection. - numberOr(null|''|boolean) now falls back instead of coercing to 0 (a null minLength disabled the spam filter), and an explicit tier: 0 is honored instead of ||-coerced to 3. Co-Authored-By: Claude Fable 5 --- server/discord/discordClient.js | 49 ++++++++++++--- server/discord/workExtractor.js | 12 ++-- server/discordWatchService.js | 57 ++++++++++++++++- tests/unit/discordWatchService.test.js | 86 ++++++++++++++++++++++++++ 4 files changed, 188 insertions(+), 16 deletions(-) diff --git a/server/discord/discordClient.js b/server/discord/discordClient.js index ccb07063..119c3174 100644 --- a/server/discord/discordClient.js +++ b/server/discord/discordClient.js @@ -69,18 +69,19 @@ class DiscordClient { * so pages are reversed into chronological order — work items should be * created in the order the conversation actually happened. * - * With no cursor (first-sight backfill) there is nothing to page forward from, - * so a single newest-first page is taken and trimmed to the most recent N. - * Trimming the oldest N instead would silently skip the newest messages — - * exactly the ones a backfill is meant to catch. + * With no cursor (first-sight backfill) there is nothing to page forward + * from, and `after` pages can never reach past the newest 100 — history is + * gathered by paging BACKWARD instead (fetchBackfill). */ async fetchMessagesAfter(channelId, afterId, { maxMessages = 400 } = {}) { + if (!afterId) return this.fetchBackfill(channelId, maxMessages); + const collected = []; let cursor = afterId; while (collected.length < maxMessages) { const query = new URLSearchParams({ limit: String(MAX_PAGE) }); - if (cursor) query.set('after', cursor); + query.set('after', cursor); const result = await this.request(`/channels/${channelId}/messages?${query}`); if (!result.ok) return { ok: false, error: result.error, status: result.status, messages: collected }; @@ -93,10 +94,40 @@ class DiscordClient { if (page.length < MAX_PAGE) break; } - // Forward paging keeps the oldest N after the cursor; a backfill keeps the - // most recent N (the tail of the chronological list). - const messages = afterId ? collected.slice(0, maxMessages) : collected.slice(-maxMessages); - return { ok: true, messages, cursor: messages.length ? messages[messages.length - 1].id : (cursor || afterId) }; + const messages = collected.slice(0, maxMessages); + return { ok: true, messages, cursor: messages.length ? messages[messages.length - 1].id : afterId }; + } + + /** + * First-sight backfill: keep the most recent N. Trimming the oldest N + * instead would silently skip the newest messages — exactly the ones a + * backfill is meant to catch. Beyond one page this must walk `before=` into + * history; the old forward walk re-queried after the NEWEST message and got + * an empty page back, silently capping every backfill at 100. + */ + async fetchBackfill(channelId, maxMessages) { + const newestFirst = []; + let before = null; + + while (newestFirst.length < maxMessages) { + const query = new URLSearchParams({ limit: String(MAX_PAGE) }); + if (before) query.set('before', before); + + const result = await this.request(`/channels/${channelId}/messages?${query}`); + if (!result.ok) { + return { ok: false, error: result.error, status: result.status, messages: newestFirst.slice(0, maxMessages).reverse() }; + } + + const page = Array.isArray(result.data) ? result.data : []; // newest-first + if (!page.length) break; + + newestFirst.push(...page); + before = page[page.length - 1].id; + if (page.length < MAX_PAGE) break; + } + + const messages = newestFirst.slice(0, maxMessages).reverse(); + return { ok: true, messages, cursor: messages.length ? messages[messages.length - 1].id : null }; } async getLatestMessageId(channelId) { diff --git a/server/discord/workExtractor.js b/server/discord/workExtractor.js index 2c400d06..868c6769 100644 --- a/server/discord/workExtractor.js +++ b/server/discord/workExtractor.js @@ -31,9 +31,13 @@ function readJson(filePath) { } // `Number(undefined)` is NaN and `NaN ?? fallback` is still NaN (?? only catches -// null/undefined), so a missing numeric field must be caught explicitly. An -// explicit 0 (e.g. backfillMessages: 0 = "never look back") is preserved. +// null/undefined), so a missing numeric field must be caught explicitly. And +// `Number(null)`/`Number('')` are 0, not NaN — an explicit null/empty in an +// override must fall back, not silently zero the setting (a null minLength +// would disable the spam filter entirely). An explicit 0 (e.g. +// backfillMessages: 0 = "never look back") is preserved. function numberOr(value, fallback) { + if (value === null || value === undefined || value === '' || typeof value === 'boolean') return fallback; const num = Number(value); return Number.isFinite(num) ? num : fallback; } @@ -57,12 +61,12 @@ function loadConfig({ configPath = null } = {}) { publishStatus: raw.publishStatus !== false, priority: (Array.isArray(raw.priority) ? raw.priority : []).map((row) => ({ level: String(row.level || 'normal'), - tier: Number(row.tier) || 3, + tier: numberOr(row.tier, 3), patterns: compile(row.patterns) })), defaultPriority: { level: String(raw.defaultPriority?.level || 'normal'), - tier: Number(raw.defaultPriority?.tier) || 3 + tier: numberOr(raw.defaultPriority?.tier, 3) }, kinds: (Array.isArray(raw.kinds) ? raw.kinds : []).map((row) => ({ kind: String(row.kind || 'fyi'), diff --git a/server/discordWatchService.js b/server/discordWatchService.js index 353add20..51ed2e8f 100644 --- a/server/discordWatchService.js +++ b/server/discordWatchService.js @@ -67,10 +67,11 @@ class DiscordWatchService { return { cursors: raw.cursors && typeof raw.cursors === 'object' ? raw.cursors : {}, items: Array.isArray(raw.items) ? raw.items : [], - memberNames: raw.memberNames && typeof raw.memberNames === 'object' ? raw.memberNames : {} + memberNames: raw.memberNames && typeof raw.memberNames === 'object' ? raw.memberNames : {}, + channelGuilds: raw.channelGuilds && typeof raw.channelGuilds === 'object' ? raw.channelGuilds : {} }; } catch { - return { cursors: {}, items: [], memberNames: {} }; + return { cursors: {}, items: [], memberNames: {}, channelGuilds: {} }; } } @@ -101,6 +102,7 @@ class DiscordWatchService { const id = String(channelId || '').trim(); if (!/^\d+$/.test(id)) throw new Error('A Discord channel id is a numeric snowflake'); if (!this.config.channels.includes(id)) this.config.channels.push(id); + this.persistChannels(); return this.config.channels; } @@ -109,9 +111,36 @@ class DiscordWatchService { this.config.channels = this.config.channels.filter((existing) => existing !== id); delete this.state.cursors[id]; this.saveState(); + this.persistChannels(); return this.config.channels; } + /** + * Channels changed over the API must land in the override config file — + * in-memory-only meant a channel added via POST vanished on the next + * reload-config or restart, contradicting the config's own comment that + * the two paths are equivalent. + */ + persistChannels() { + try { + const target = extractor.overrideConfigPath(); + let existing = {}; + try { + existing = JSON.parse(fs.readFileSync(target, 'utf8')) || {}; + } catch { + // No override yet — channels become its first key; loadConfig layers + // the override on the shipped defaults, so nothing else is lost. + } + existing.channels = [...this.config.channels]; + fs.mkdirSync(path.dirname(target), { recursive: true }); + const tmp = `${target}.${process.pid}.tmp`; + fs.writeFileSync(tmp, `${JSON.stringify(existing, null, 2)}\n`, 'utf8'); + fs.renameSync(tmp, target); + } catch (error) { + this.logger.warn?.('Discord watch could not persist channel config', { error: error.message }); + } + } + findItemByMessage(messageId) { return this.state.items.find((item) => item.messageId === messageId) || null; } @@ -217,6 +246,7 @@ class DiscordWatchService { }); if (!fetched.ok) return { channelId, ok: false, error: fetched.error }; + const guildId = await this.resolveGuildId(channelId); const created = []; const updated = []; for (const message of fetched.messages) { @@ -225,6 +255,7 @@ class DiscordWatchService { } const extraction = extractor.extractFromMessage(message, { config: this.config, + guildId, memberNames: this.state.memberNames }); const result = this.applyExtraction(extraction, { channelId }); @@ -241,6 +272,23 @@ class DiscordWatchService { return { channelId, ok: true, messages: fetched.messages.length, created: created.length, updated: updated.length, items: created }; } + /** + * REST message objects do not carry guild_id (only gateway events do), so + * it is looked up once per channel and cached — without it every permalink + * rendered as the DM-only `/channels/@me/...` form, which is a dead link + * for guild channels, the primary use case. + */ + async resolveGuildId(channelId) { + if (Object.prototype.hasOwnProperty.call(this.state.channelGuilds, channelId)) { + return this.state.channelGuilds[channelId]; + } + const info = await this.client.getChannel?.(channelId); + // Only cache on success — a transient lookup failure retries next poll. + if (!info?.ok) return ''; + this.state.channelGuilds[channelId] = String(info.channel?.guild_id || ''); + return this.state.channelGuilds[channelId]; + } + async poll() { if (this.polling) return { skipped: 'already polling' }; this.polling = true; @@ -288,7 +336,10 @@ class DiscordWatchService { this.saveState(); try { - this.taskRecordService?.upsert?.(`session:${sessionId}`, { + // upsert is async — without the await, a rejection skipped this catch + // entirely (unhandled rejection) and the record wasn't guaranteed + // written before this method returned. + await this.taskRecordService?.upsert?.(`session:${sessionId}`, { tier: item.tier, ticketProvider: 'discord', ticketCardId: item.messageId, diff --git a/tests/unit/discordWatchService.test.js b/tests/unit/discordWatchService.test.js index 84017481..061bfeef 100644 --- a/tests/unit/discordWatchService.test.js +++ b/tests/unit/discordWatchService.test.js @@ -112,6 +112,27 @@ describe('workExtractor loadConfig', () => { expect(cfg.backfillMessages).toBe(0); }); }); + + test('an explicit null in an override falls back instead of silently becoming 0', () => { + // Number(null) === 0, so a null minLength used to disable the + // short-message spam filter entirely rather than defaulting to 12. + withConfig({ enabled: true, channels: ['123'], minLength: null, backfillMessages: null }, (cfg) => { + expect(cfg.minLength).toBe(12); + expect(cfg.backfillMessages).toBe(50); + }); + }); + + test('an explicit tier of 0 is honored, not coerced to 3', () => { + withConfig({ + enabled: true, + channels: ['123'], + priority: [{ level: 'now', tier: 0, patterns: ['\\bnow\\b'] }], + defaultPriority: { level: 'normal', tier: 0 } + }, (cfg) => { + expect(cfg.priority[0].tier).toBe(0); + expect(cfg.defaultPriority.tier).toBe(0); + }); + }); }); describe('DiscordClient', () => { @@ -157,6 +178,29 @@ describe('DiscordClient', () => { expect(result.messages[0].id).toBe('101'); }); + test('a backfill larger than one page walks BACKWARD through history', async () => { + // 150 messages, newest-first (ids 150..1), served in URL-aware pages. + const all = Array.from({ length: 150 }, (_, i) => ({ id: String(150 - i) })); + const seen = []; + const client = new DiscordClient({ + token: 't', + fetchImpl: async (url) => { + seen.push(url); + const before = new URL(url).searchParams.get('before'); + const start = before ? all.findIndex((m) => m.id === before) + 1 : 0; + return { ok: true, status: 200, headers: { get: () => null }, text: async () => JSON.stringify(all.slice(start, start + 100)) }; + } + }); + + const result = await client.fetchMessagesAfter('chan1', null, { maxMessages: 150 }); + // The old forward walk re-queried after the NEWEST message, got an empty + // page, and silently capped every backfill at 100. + expect(result.messages).toHaveLength(150); + expect(result.messages[0].id).toBe('1'); + expect(result.messages[149].id).toBe('150'); + expect(seen[1]).toContain('before=51'); + }); + test('a 429 backs off instead of hammering', async () => { const client = new DiscordClient({ token: 't', @@ -332,4 +376,46 @@ describe('DiscordWatchService', () => { expect(result.channels[0].ok).toBe(false); expect(service.getStatus().lastError).toBe('HTTP 503'); }); + + test('guild-channel permalinks carry the real guild id, not the DM-only @me', async () => { + const { service } = harness({ messages: [message({ id: '10', content: '<@2002> please fix the crash on level three' })] }); + let lookups = 0; + service.client.getChannel = async () => { lookups += 1; return { ok: true, channel: { guild_id: 'g777' } }; }; + + await service.poll(); + expect(service.getItems()[0].permalink).toContain('/channels/g777/chan1/10'); + expect(service.getItems()[0].permalink).not.toContain('@me'); + + // The lookup is cached — a second poll must not re-fetch channel info. + service.client.fetchMessagesAfter = async () => ({ ok: true, messages: [] }); + await service.poll(); + expect(lookups).toBe(1); + }); + + test('a failing task-record write is logged, not an unhandled rejection', async () => { + const warnings = []; + const { service } = harness({ messages: [message({ id: '10', content: '<@2002> please fix the crash on level three' })] }); + service.logger = { warn: (msg) => warnings.push(msg), error: () => {}, info: () => {} }; + service.init({ taskRecordService: { upsert: async () => { throw new Error('disk full'); } } }); + await service.poll(); + + // Without the await, the rejection skipped the catch entirely. + const item = await service.linkSession('discord:10', 'work1-claude', { announce: false }); + expect(item.status).toBe('in-progress'); + expect(warnings.some((w) => /task record/i.test(w))).toBe(true); + }); + + test('channels added or removed over the API survive a config reload', async () => { + const { service } = harness(); + service.addChannel('424242424242'); + + // reload-config rebuilds this.config from disk — the in-memory-only + // version of addChannel lost the channel right here. + service.reloadConfig(); + expect(service.getChannels()).toContain('424242424242'); + + service.removeChannel('424242424242'); + service.reloadConfig(); + expect(service.getChannels()).not.toContain('424242424242'); + }); }); From 30fec9df4e823d6ba6252c5e3ee2701c36a37a64 Mon Sep 17 00:00:00 2001 From: web3dev1337 Date: Wed, 5 Aug 2026 13:48:36 +1000 Subject: [PATCH 69/69] docs: record the 2026-08-05 full-PR review pass in the handoff Co-Authored-By: Claude Fable 5 --- PLANS/2026-07-27/HANDOFF.md | 50 +++++++++++++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) diff --git a/PLANS/2026-07-27/HANDOFF.md b/PLANS/2026-07-27/HANDOFF.md index 7263706a..a333e778 100644 --- a/PLANS/2026-07-27/HANDOFF.md +++ b/PLANS/2026-07-27/HANDOFF.md @@ -259,3 +259,53 @@ Verified live via `POST /api/voice/parse` (classifier only, no side effects). Al false-reject risk on real requests. - Multi-part commands (`"open the queue AND approve everything"`) execute the first clause only. - True full-duplex (PersonaPlex on the 5090) is still the next real build — see above. + +--- + +## SESSION 3 UPDATE (2026-08-05) — full-PR review pass (Fable) + fixes + +Re-reviewed the whole PR: 7 read-only subsystem scouts + an independent Codex pass over the +unreviewed 07-27 voice commits; every finding verified against the actual code before fixing. +**Tests now 893 green / 120 suites** (was 868). Nine fix commits, all pushed. + +Fixed, worst first: +- **voice CRITICAL** — rules matched BEFORE the negation guard and destructive rule patterns are + unanchored: `"don't stop all claudes"` actually executed `stop-all-claudes` (verified live). + Negation now short-circuits rule matching. Also: "stop the server" no longer swallowed as a + negation; `/what.*sessions/` no longer steals fact questions; `isGrounded` requires the + destructive VERB, not an incidental noun; Commander-reply captures serialized (shared-buffer race). +- **supervisor CRITICAL** — auto-approve deny list was case-sensitive (`Write(.ENV)`, lowercase SQL + passed) and only knew `~/`-style credential paths (`Write(/home/x/.ssh/...)` auto-approved). + Deny patterns compile case-insensitive now; absolute `/home|/Users|/root` credential paths, + quoted build-file paths and `git push -f` denied. `forgetHealed` clears cooldowns (stale cooldown + blocked action on fresh recurrences + unbounded map); `reload-rules` writes an audit entry. +- **speech** — kokoro was hardcoded "available" (URL has a default) with a silent no-op failure + path: a dead kokoro server muted the voice permanently while `speak()` reported success. + Availability is now an explicit signal (env set / registry-verified); failed neural synth falls + back to browser speech; TTS `none` actually mutes; priority reaches the streamed-audio path; + client queues clips instead of overlapping, respects mute, and replays autoplay-blocked audio on + the first user gesture. +- **atlas** — `compile()` re-shared a teammate's subscribed highlights once the repo also appeared + in local discovery (cloning it declassified their content). Bundles now merge WITHOUT the + subscription layer (`getOwnEntries`). Also: `writeManifest` knows `main/` checkouts; CLI + value-less `--topic` rejected; `qualityFloor` boolean guard. +- **app-server** — stdio pipes had no error listeners (a non-EPIPE stream error would take down the + whole orchestrator); restart backoff reset at spawn, so a spawn-then-die binary was hammered at + the 1s floor forever (resets only after 30s uptime now); an undelivered approval answer keeps the + approval pending instead of vanishing. +- **discord** — permalinks always rendered the DM-only `/channels/@me/...` form (guild id now + fetched once per channel + cached); backfills >100 silently capped at 100 (now page `before=` + backward); channels added over the API now persist to the override config; `linkSession` awaits + its task-record write (its catch was dead code); `numberOr(null)` falls back instead of becoming + 0; explicit `tier: 0` honored. +- **privacy** — files NEW in this PR (the design doc, COMMANDER_CLAUDE.md examples, atlas test + fixtures) shipped real private repo names into this public repo; scrubbed to the established + `acme-*` placeholder vocabulary. Names remain in this branch's earlier commit history; + pre-existing references on main are left as a separate decision. + +Consciously left as-is: `POST /api/atlas/proposals` stays read-level (agents must be able to +propose; approving is what writes and stays write-gated); `speech-speak`/`speech-audio` are +fleet-wide broadcasts by design (two open browser windows will both narrate); +`extractAssistantReply` remains heuristic (the structured replacement is the Codex app-server +path); `package.json` `engines >=16` predates this PR even though server code now uses fetch — +Node 18+ is the real floor.