From 60a814997fb084e694da62af13eb7747ead41395 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 07:32:15 +0000 Subject: [PATCH 01/12] chore(harness): bootstrap Systemfsoftware agent harness and strategy Single-source AGENTS.md across all product trees with machine-native @AGENTS.md CLAUDE.md pointers. Establish Systemfsoftware positioning and refactor roadmap in STRATEGY.md, rewrite server/foundry architecture docs to reflect current runtime, and add deterministic docs terminology gate. --- CLAUDE.md | 85 +----- STRATEGY.md | 63 +++++ docs/deploy/boxlite.mdx | 2 +- docs/processes.mdx | 10 +- docs/sdk-overview.mdx | 2 +- examples/AGENTS.md | 12 + examples/CLAUDE.md | 23 +- foundry/ARCHITECTURE.md | 54 ++++ foundry/CLAUDE.md | 392 +------------------------- foundry/packages/backend/AGENTS.md | 34 +++ foundry/packages/backend/CLAUDE.md | 339 +---------------------- frontend/CLAUDE.md | 5 +- scripts/check-docs.ts | 32 +++ sdks/AGENTS.md | 36 +++ sdks/CLAUDE.md | 38 +-- server/ARCHITECTURE.md | 427 ++++------------------------- server/CLAUDE.md | 94 +------ 17 files changed, 299 insertions(+), 1349 deletions(-) create mode 100644 STRATEGY.md create mode 100644 examples/AGENTS.md create mode 100644 foundry/ARCHITECTURE.md create mode 100644 foundry/packages/backend/AGENTS.md create mode 100755 scripts/check-docs.ts create mode 100644 sdks/AGENTS.md diff --git a/CLAUDE.md b/CLAUDE.md index 34bea894..43c994c2 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -1,84 +1 @@ -# Instructions - -## Naming and Ownership - -- This repository/product is **Sandbox Agent**. -- **Gigacode** is a separate user-facing UI/client, not the server product name. -- Gigacode integrates with Sandbox Agent via the OpenCode-compatible surface (`/opencode/*`) when that compatibility layer is enabled. -- Canonical extension namespace/domain string is `sandboxagent.dev` (no hyphen). -- Canonical custom ACP extension method prefix is `_sandboxagent/...` (no hyphen). - -## Docs Terminology - -- Never mention "ACP" in user-facing docs (`docs/**/*.mdx`) except in docs that are specifically about ACP itself (e.g. `docs/acp-http-client.mdx`). -- Never expose underlying protocol method names (e.g. `session/request_permission`, `session/create`, `_sandboxagent/session/detach`) in non-ACP docs. Describe the behavior in user-facing terms instead. -- Do not describe the underlying protocol implementation in docs. Only document the SDK surface (methods, types, options). ACP protocol details belong exclusively in ACP-specific pages. -- Do not use em dashes (`—`) in docs. Use commas, periods, or parentheses instead. - -### Docs Source Of Truth (HTTP/CLI) - -- For HTTP/CLI docs/examples, source of truth is: - - `server/packages/sandbox-agent/src/router.rs` - - `server/packages/sandbox-agent/src/cli.rs` -- Keep docs aligned to implemented endpoints/commands only (for example ACP under `/v1/acp`, not legacy session REST APIs). - -## Docs Styling - -- Docs styling is owned by the shared **`@rivet-dev/docs-theme`** repo (`github.com/rivet-dev/docs-theme`), consumed via `github:rivet-dev/docs-theme#` in `frontend/packages/website`. To change any docs styling (palette, header, sidebar, code blocks, fonts), edit that repo and follow its CLAUDE.md release workflow — never restyle docs here. This site owns only docs content + `frontend/packages/website/docs.config.mjs` (sidebar icons via each item's `attrs['data-icon']`). - -## Change Tracking - -- If the user asks to "push" changes, treat that as permission to commit and push all current workspace changes, not a hand-picked subset, unless the user explicitly scopes the push. -- Keep CLI subcommands and HTTP endpoints in sync. -- Update `docs/cli.mdx` when CLI behavior changes. -- Regenerate `docs/openapi.json` when HTTP contracts change. -- Keep `docs/inspector.mdx` and `docs/sdks/typescript.mdx` aligned with implementation. -- Append blockers/decisions to `research/acp/friction.md` during ACP work. -- `docs/agent-capabilities.mdx` lists models/modes/thought levels per agent. Update it when adding a new agent or changing `fallback_config_options`. If its "Last updated" date is >2 weeks old, re-run `cd scripts/agent-configs && npx tsx dump.ts` and update the doc to match. Source data: `scripts/agent-configs/resources/*.json` and hardcoded entries in `server/packages/sandbox-agent/src/router/support.rs` (`fallback_config_options`). -- Some agent models are gated by subscription (e.g. Claude `opus`). The live report only shows models available to the current credentials. The static doc and JSON resource files should list all known models regardless of subscription tier. - -## Docker Test Image - -- Docker-backed Rust and TypeScript tests build `docker/test-agent/Dockerfile` directly in-process and cache the image tag only in memory (`OnceLock` in Rust, module-level variable in TypeScript). -- Do not add cross-process image-build scripts unless there is a concrete need for them. - -## Common Software Sync - -- These three files must stay in sync: - - `docs/common-software.mdx` (user-facing documentation) - - `docker/test-common-software/Dockerfile` (packages installed in the test image) - - `server/packages/sandbox-agent/tests/common_software.rs` (test assertions) -- When adding or removing software from `docs/common-software.mdx`, also add/remove the corresponding `apt-get install` line in the Dockerfile and add/remove the test in `common_software.rs`. -- Run `cargo test -p sandbox-agent --test common_software` to verify. - -## Install Version References - -- Channel policy: - - Sandbox Agent install/version references use a pinned minor channel `0.N.x` (for curl URLs and `sandbox-agent` / `@sandbox-agent/cli` npm/bun installs). - - Gigacode install/version references use `latest` (for `@sandbox-agent/gigacode` install/run commands and `gigacode-install.*` release promotion). - - Release promotion policy: `latest` releases must still update `latest`; when a release is `latest`, Sandbox Agent must also be promoted to the matching minor channel `0.N.x`. -- Keep every install-version reference below in sync whenever versions/channels change: - - `README.md` - - `docs/acp-http-client.mdx` - - `docs/cli.mdx` - - `docs/quickstart.mdx` - - `docs/sdk-overview.mdx` - - `docs/react-components.mdx` - - `docs/session-persistence.mdx` - - `docs/deploy/local.mdx` - - `docs/deploy/cloudflare.mdx` - - `docs/deploy/vercel.mdx` - - `docs/deploy/daytona.mdx` - - `docs/deploy/e2b.mdx` - - `docs/deploy/docker.mdx` - - `frontend/packages/website/src/components/GetStarted.tsx` - - `.claude/commands/post-release-testing.md` - - `examples/cloudflare/Dockerfile` - - `examples/daytona/src/index.ts` - - `examples/shared/src/docker.ts` - - `examples/docker/src/index.ts` - - `examples/e2b/src/index.ts` - - `examples/vercel/src/index.ts` - - `scripts/release/main.ts` - - `scripts/release/promote-artifacts.ts` - - `scripts/release/sdk.ts` +@AGENTS.md diff --git a/STRATEGY.md b/STRATEGY.md new file mode 100644 index 00000000..e958502f --- /dev/null +++ b/STRATEGY.md @@ -0,0 +1,63 @@ +--- +name: Systemfsoftware +last_updated: 2026-08-29 +--- + +# Systemfsoftware Strategy + +## Purpose + +Developers embedding coding agents (Claude Code, Codex, OpenCode, Cursor, Amp, Pi) into products hit the same wall: every agent speaks a different protocol, sessions die with the sandbox, and remote control over HTTP has to be rebuilt per agent. Systemfsoftware exists to remove that wall: one control plane that runs in the sandbox and speaks to every agent. + +## Positioning + +L'État, c'est moi. Systemfsoftware is not a competitor in the agent-tooling market; it is the lawgiver of it: the Effect-TS idiom set, the oxlint rulebook, and the sandbox control plane are the standard other products conform to. A neighboring product can integrate with Systemfsoftware; it cannot truthfully claim to define the standard. + +## Users + +**Primary:** Platform engineer embedding agent control. They're hiring Systemfsoftware to run any coding agent inside a sandbox and control it over HTTP without per-agent rewrites. + +**Secondary:** Foundry operator. They're hiring the Foundry tree to run multi-repo agent tasks with GitHub-backed state and realtime UI. + +## Boundaries + +- No session storage on disk; consumers stream events to their own storage. +- No direct LLM wrappers; the Vercel AI SDK stays complementary. +- No sandbox-provider API layer; providers get guides, not abstractions. +- No polling, retry loops, or manual-refresh flows in Foundry; push-based progression only. + +_Resist a change when:_ a provider asks for a custom API surface; guides and the existing HTTP contracts come first. + +## Key metrics + +- **Green CI with lint gates** - `ci.yaml` passes with oxlint (`@systemfsoftware/all`) in the chain; measured on GitHub Actions. +- **Rename completeness** - zero new occurrences of `rivet-dev` org refs, `@sandbox-agent` scope, or `sandboxagent.dev` outside migration notes; checked by `git grep` count. +- **Effect coverage of TS surfaces** - sdks, frontend, and foundry-backend packages compile under Effect v4 with per-package `oxlint.config.ts`; checked by `pnpm typecheck` and `pnpm lint`. + +Flip to product metrics (installs, session success rate) after the refactor ships. + +## Tracks + +### Banner migration + +Repo, org, npm scopes, crates, domain, and product display name become systemfsoftware / Systemfsoftware. + +_Why it serves the approach:_ one name that is the standard is the precondition for the positioning. + +### Effect v4 adoption + +All TS surfaces move to Effect v4 via pnpm catalogs; Foundry's RivetKit actor runtime is replaced by Effect's actor model. + +_Why it serves the approach:_ the platform's law is Effect; running products on a different runtime contradicts it. + +### Lint sovereignty + +Every package lints under `@systemfsoftware/all` through per-package `oxlint.config.ts`, with gates defined once in the check chain. + +_Why it serves the approach:_ law without deterministic enforcement is prose. + +## Brand + +**One-liner:** Systemfsoftware is the state of coding-agent infrastructure: one control plane, one type system, one rulebook. + +**Key message:** Other tools ask which agent framework you picked. Systemfsoftware is the framework the agents answer to. diff --git a/docs/deploy/boxlite.mdx b/docs/deploy/boxlite.mdx index 8c02bb48..f34b192a 100644 --- a/docs/deploy/boxlite.mdx +++ b/docs/deploy/boxlite.mdx @@ -3,7 +3,7 @@ title: "BoxLite" description: "Run Sandbox Agent inside a BoxLite micro-VM." --- -BoxLite is a local-first micro-VM sandbox — no cloud account needed. +BoxLite is a local-first micro-VM sandbox, no cloud account needed. See [BoxLite docs](https://docs.boxlite.ai) for platform requirements (KVM on Linux, Apple Silicon on macOS). ## Prerequisites diff --git a/docs/processes.mdx b/docs/processes.mdx index 282c0f14..31d7d6fb 100644 --- a/docs/processes.mdx +++ b/docs/processes.mdx @@ -7,11 +7,11 @@ icon: "terminal" The process API supports: -- **One-shot execution** — run a command to completion and capture stdout, stderr, and exit code -- **Managed processes** — spawn, list, stop, kill, and delete long-lived processes -- **Log streaming** — fetch buffered logs or follow live output -- **Terminals** — full PTY support with bidirectional WebSocket I/O -- **Configurable limits** — control concurrency, timeouts, and buffer sizes per runtime +- **One-shot execution**: run a command to completion and capture stdout, stderr, and exit code +- **Managed processes**: spawn, list, stop, kill, and delete long-lived processes +- **Log streaming**: fetch buffered logs or follow live output +- **Terminals**: full PTY support with bidirectional WebSocket I/O +- **Configurable limits**: control concurrency, timeouts, and buffer sizes per runtime ## Run a command diff --git a/docs/sdk-overview.mdx b/docs/sdk-overview.mdx index c3d5d2ae..6b23dbd9 100644 --- a/docs/sdk-overview.mdx +++ b/docs/sdk-overview.mdx @@ -85,7 +85,7 @@ const sdk = await SandboxAgent.start({ sandbox: local(), }); -// sdk.sandboxId — prefixed provider ID (e.g. "local/127.0.0.1:2468") +// sdk.sandboxId: prefixed provider ID (e.g. "local/127.0.0.1:2468") await sdk.destroySandbox(); // provider-defined cleanup + disposes client ``` diff --git a/examples/AGENTS.md b/examples/AGENTS.md new file mode 100644 index 00000000..3e9eb7cf --- /dev/null +++ b/examples/AGENTS.md @@ -0,0 +1,12 @@ +# Examples Instructions + +## Docker isolation + +- Docker examples behave as standalone sandboxes: no host bind mounts of files or directories. +- Needed tools, skills, or MCP servers are installed inside the container during setup. + +## Validation + +- Validate examples against the implemented server surface (`server/packages/sandbox-agent/src/router.rs`), not against narrated flows in docs. +- Bleeding-edge local testing: `SANDBOX_AGENT_DEV=1 pnpm start` builds `docker/runtime/Dockerfile.full` from local source. +- Drive the ACP surface with the `mock` agent (initialize, then session/new, session/prompt; SSE for events) exactly as the SDK does; keep any example README instructions aligned with `server/ARCHITECTURE.md`. diff --git a/examples/CLAUDE.md b/examples/CLAUDE.md index 048312f1..43c994c2 100644 --- a/examples/CLAUDE.md +++ b/examples/CLAUDE.md @@ -1,22 +1 @@ -# Examples Instructions - -## Docker Isolation - -- Docker examples must behave like standalone sandboxes. -- Do not bind mount host files or host directories into Docker example containers. -- If an example needs tools, skills, or MCP servers, install them inside the container during setup. - -## Testing Examples (ACP v2) - -Examples should be validated against v2 endpoints: - -1. Start the example: `SANDBOX_AGENT_DEV=1 pnpm start` -2. Create an ACP client by POSTing `initialize` to `/v2/rpc` with `x-acp-agent: mock` (or another installed agent). -3. Capture `x-acp-connection-id` from the response headers. -4. Open SSE stream: `GET /v2/rpc` with `x-acp-connection-id`. -5. Send `session/new` then `session/prompt` via `POST /v2/rpc` with the same connection id. -6. Close connection via `DELETE /v2/rpc` with `x-acp-connection-id`. - -v1 reminder: - -- `/v1/*` is removed and returns `410 Gone`. +@AGENTS.md diff --git a/foundry/ARCHITECTURE.md b/foundry/ARCHITECTURE.md new file mode 100644 index 00000000..2e4da24f --- /dev/null +++ b/foundry/ARCHITECTURE.md @@ -0,0 +1,54 @@ +# Architecture + +last-validated-against-decisions: 2026-08-29 (no ADR log; see root `STRATEGY.md`) + +Foundry is a workspace product for multi-repo agent tasks: a Vite frontend, a Bun backend built on RivetKit actors (migrating to Effect's actor model), a realtime client library, shared types, and a Tauri desktop shell. + +## Component map + +| Component | Location | Role | +|---|---|---| +| Frontend | `foundry/packages/frontend/` | Vite + BaseUI/Styletron app | +| Backend | `foundry/packages/backend/` | RivetKit actors, SQLite per actor, GitHub sync, sandbox orchestration | +| Client | `foundry/packages/client/` | Subscription manager; the only permitted `rivetkit` import site | +| Shared | `foundry/packages/shared/` | Types and the mock workspace implementation (UI testing reference) | +| Desktop | `foundry/packages/desktop/` | Tauri shell (excluded from the cargo workspace) | +| CLI | `foundry/packages/cli/` | Disabled; excluded from workspace resolution | + +## Actor hierarchy + +```mermaid +flowchart TD + Org["OrganizationActor"] --> Audit["AuditLogActor"] + Org --> GH["GithubDataActor"] + Org --> Task["TaskActor(task)"] + Task --> Sess["taskSessions (sessions + transcripts)"] + Task --> SBX["taskSandboxes"] + SBX --> SI["SandboxInstanceActor x N"] +``` + +- Coordinators keep index tables (org: `taskIndex`, `taskSummaries`, auth session/email/account indexes; task: `taskWorkspaceSessions`, `taskSandboxes`), create/destroy their children, and route lookups. Children push updates to their direct coordinator only; read paths use local index tables, never fan out to children. +- Everything is scoped to an organization; actor keys are prefixed `["org", organizationId, ...]`. +- Task actors are created lazily on first user interaction; PR sync writes virtual task entries to org-local tables and spawns no actors. + +## Realtime data flow + +1. The client connects to an actor over WebSocket, fetches the materialized snapshot with one action call, then subscribes. +2. Events carry full replacement payloads of the changed entity - not patches, not empty notifications. +3. The subscription manager (`packages/client`) deduplicates topics, keeps connections warm for 30 seconds after the last subscriber leaves, and exposes `useSubscription(topic, params)`. +4. Topics: `app`, `organization`, `task`, `session` (session topic rides the task actor connection, filtered by session id), `sandboxProcesses` - at most 4 actor connections at a time. + +## Sandbox and git model + +- One active sandbox per task; the structure allows more, features assume one. +- The task owner's GitHub OAuth token is injected into the sandbox for git operations (credential file `chmod 600`, overwritten on owner swap); git never prompts interactively, and git errors surface in the UI with context. +- The backend stores zero git state: no clones, refs, or working trees. Repository metadata comes from GitHub API data and webhooks; working-tree operations run inside the sandbox via `executeInSandbox()`. Backend drivers: `GithubDriver` and `TmuxDriver` only. +- `GithubDataActor` syncs repositories and pull requests (not branches); PRs materialize as virtual tasks in the sidebar until a user opens one. + +## Persistence + +- SQLite per actor instance through RivetKit `c.db` with Drizzle migrations; no backend-global databases and no namespacing columns for actor-owned rows. Simple metadata lives in `c.state`; anything with query or relational needs lives in `c.db`. + +## Migration note + +The Effect-actor migration (root `STRATEGY.md`) replaces the RivetKit runtime: actor coordination, queues, and workflows move onto Effect primitives, and the per-actor law in `foundry/packages/backend/AGENTS.md` is deleted module by module as each migrates. This file re-validates when the first migrated actor lands and its ADR is written. diff --git a/foundry/CLAUDE.md b/foundry/CLAUDE.md index 2d9bcbb9..43c994c2 100644 --- a/foundry/CLAUDE.md +++ b/foundry/CLAUDE.md @@ -1,391 +1 @@ -# Project Instructions - -## Language Policy - -Use TypeScript for all source code. - -- Never add raw JavaScript source files (`.js`, `.mjs`, `.cjs`). -- Prefer `.ts`/`.tsx` for runtime code, scripts, tests, and tooling. -- If touching old JavaScript, migrate it to TypeScript instead of extending it. - -## Monorepo + Tooling - -Use `pnpm` workspaces and Turborepo. - -- Repository root uses `pnpm-workspace.yaml` and `turbo.json`. -- Packages live in `packages/*`. -- `core` is renamed to `shared`. -- `packages/cli` is disabled and excluded from active monorepo validation. -- Integrations and providers live under `packages/backend/src/{integrations,providers}`. - -## CLI Status - -- `packages/cli` is fully disabled for active development. -- Do not implement new behavior in `packages/cli` unless explicitly requested. -- Frontend is the primary product surface; prioritize `packages/frontend` + supporting `packages/client`/`packages/backend`. -- Monorepo `build`, `typecheck`, and `test` intentionally exclude `@sandbox-agent/foundry-cli`. -- `pnpm-workspace.yaml` excludes `packages/cli` from monorepo package resolution. - -## Common Commands - -- Foundry is the canonical name for this product tree. Do not introduce or preserve legacy pre-Foundry naming in code, docs, commands, or runtime paths. -- Install deps: `pnpm install` -- Full active-monorepo validation: `pnpm -w typecheck`, `pnpm -w build`, `pnpm -w test` -- Start the full dev stack (real backend + frontend): `just foundry-dev` — frontend on **port 4173**, backend on **port 7741** (Docker via `compose.dev.yaml`) -- Start the mock frontend stack (no backend): `just foundry-mock` — mock frontend on **port 4174** (Docker via `compose.mock.yaml`) -- Start the local production-build preview stack: `just foundry-preview` -- Start only the backend locally: `just foundry-backend-start` -- Start only the frontend locally: `pnpm --filter @sandbox-agent/foundry-frontend dev` -- Start the mock frontend locally (no Docker): `just foundry-dev-mock` — mock frontend on **port 4174** -- Dev and mock stacks can run simultaneously on different ports (4173 and 4174). -- Stop the compose dev stack: `just foundry-dev-down` -- Tail compose dev logs: `just foundry-dev-logs` -- Stop the mock stack: `just foundry-mock-down` -- Tail mock logs: `just foundry-mock-logs` -- Stop the preview stack: `just foundry-preview-down` -- Tail preview logs: `just foundry-preview-logs` - -## Dev Environment Setup - -- `compose.dev.yaml` loads `foundry/.env` (optional) for credentials needed by the backend (GitHub OAuth, Stripe, Daytona, API keys, etc.). -- The canonical source for these credentials is `~/misc/the-foundry.env`. If `foundry/.env` does not exist, copy it: `cp ~/misc/the-foundry.env foundry/.env` -- `foundry/.env` is gitignored and must never be committed. -- If your changes affect the dev server, mock server, frontend runtime, backend runtime, Vite wiring, compose files, or other server-startup/runtime behavior, you must start or restart the relevant stack before finishing the task. -- Use the matching stack for verification: - - real backend + frontend changes: `just foundry-dev` or restart with `just foundry-dev-down && just foundry-dev` - - mock frontend changes: `just foundry-mock` or restart with `just foundry-mock-down && just foundry-mock` - - local frontend-only work outside Docker: restart `pnpm --filter @sandbox-agent/foundry-frontend dev` or `just foundry-dev-mock` as appropriate -- The backend does **not** hot reload. Bun's `--hot` flag causes the server to re-bind on a different port (e.g. 6421 instead of 6420), breaking all client connections while the container still exposes the original port. After backend code changes, restart the backend container: `just foundry-dev-down && just foundry-dev`. -- The dev server has debug logging enabled by default (`RIVET_LOG_LEVEL=debug`, `FOUNDRY_LOG_LEVEL=debug`) via `compose.dev.yaml`. Error stacks and timestamps are also enabled. -- The frontend client uses JSON encoding for RivetKit in development (`import.meta.env.DEV`) for easier debugging. Production uses the default encoding. - -## Foundry Base Sandbox Image - -Local Docker sandboxes use the `rivetdev/sandbox-agent:foundry-base-latest` image by default. This image extends the sandbox-agent runtime with sudo, git, neovim, gh, node, bun, chromium, and agent-browser. - -- **Dockerfile:** `docker/foundry-base.Dockerfile` (builds sandbox-agent from source, x86_64 only) -- **Publish script:** `scripts/publish-foundry-base.sh` (builds and pushes to Docker Hub `rivetdev/sandbox-agent`) -- **Tags:** `foundry-base-TZ` (timestamped) + `foundry-base-latest` (rolling) -- **Build from repo root:** `./foundry/scripts/publish-foundry-base.sh` (or `--dry-run` to skip push) -- **Override image in dev:** set `HF_LOCAL_SANDBOX_IMAGE` in `foundry/.env` or environment. The env var is passed through `compose.dev.yaml` to the backend. -- **Resolution order:** `config.sandboxProviders.local.image` (config.toml) > `HF_LOCAL_SANDBOX_IMAGE` (env var) > `DEFAULT_LOCAL_SANDBOX_IMAGE` constant in `packages/backend/src/actors/sandbox/index.ts`. -- The image must be built with `--platform linux/amd64`. The Rust build is memory-intensive; Docker Desktop needs at least 8GB RAM allocated. -- When updating the base image contents (new system packages, agent versions), rebuild and push with the publish script, then update the `foundry-base-latest` tag. - -## Production GitHub App + OAuth App - -Foundry uses two separate GitHub entities in production: - -- **OAuth App** (`GITHUB_CLIENT_ID` / `GITHUB_CLIENT_SECRET`) — handles "Sign in with GitHub" via Better Auth. This is a standard OAuth App. -- **GitHub App** (`GITHUB_APP_ID` / `GITHUB_APP_CLIENT_ID` / `GITHUB_APP_CLIENT_SECRET` / `GITHUB_APP_PRIVATE_KEY`) — handles webhooks, installation tokens for repo access, and GitHub API sync (repos, PRs). Must be manually installed on each org. - -Key env vars and where they connect: - -- `GITHUB_REDIRECT_URI` — OAuth callback, must point to `https://api.sandboxagent.dev/v1/auth/callback/github` -- `GITHUB_WEBHOOK_SECRET` — must match the secret configured on the GitHub App's Webhook settings page exactly. Mismatches cause silent 500s on webhook delivery (signature verification fails inside the actor, surfaced as a generic RivetKit `internal_error`). -- `BETTER_AUTH_URL` — must be the **API** URL (`https://api.sandboxagent.dev`), not the frontend URL. Better Auth uses this internally for sign-out and session management calls. -- `APP_URL` — the **frontend** URL (`https://foundry.sandboxagent.dev`). - -Troubleshooting: - -- **"GitHub App not installed"** — The GitHub App must be manually installed on each org. Sign-in does not auto-install it. Go to the GitHub App settings → Install App tab. The sign-in flow can only detect existing installations, not create them. -- **Webhooks not arriving** — Check the GitHub App → Advanced tab for delivery history. If deliveries show 500, the webhook secret likely doesn't match `GITHUB_WEBHOOK_SECRET`. Test with: `echo -n '{"test":true}' | openssl dgst -sha256 -hmac "$SECRET"` and curl the endpoint with the computed signature. -- **Deleting all actors wipes GitHub App installation state.** After a full actor reset, you must trigger a webhook (e.g. redeliver from GitHub App Advanced tab, or re-install the app) to repopulate installation records. - -## Railway Logs - -- Production Foundry Railway logs can be read from a linked checkout with `railway logs --deployment --lines 200` or `railway logs --deployment --lines 200`. -- Production deploys should go through `git push` to the deployment branch/workflow. Do not use `railway up` for Foundry deploys. -- If Railway logs fail because the checkout is not linked to the correct Railway project/service/environment, run: - `railway link --project 33e3e2df-32c5-41c5-a4af-dca8654acb1d --environment cf387142-61fd-4668-8cf7-b3559e0983cb --service 91c7e450-d6d2-481a-b2a4-0a916f4160fc` -- That links this directory to the `sandbox-agent` project, `production` environment, and `foundry-api` service. -- Production proxy chain: `api.sandboxagent.dev` routes through Cloudflare → Fastly/Varnish → Railway. When debugging request duplication, timeouts, or retry behavior, check headers like `cf-ray`, `x-varnish`, `x-railway-edge`, and `cdn-loop` to identify which layer is involved. - -## Frontend + Client Boundary - -- Keep a browser-friendly GUI implementation aligned with the TUI interaction model wherever possible. -- Do not import `rivetkit` directly in CLI or GUI packages. RivetKit client access must stay isolated inside `packages/client`. -- All backend interaction (actor calls, metadata/health checks, backend HTTP endpoint access) must go through the dedicated client library in `packages/client`. -- Outside `packages/client`, do not call backend endpoints directly (for example `fetch(.../v1/rivet...)`), except in black-box E2E tests that intentionally exercise raw transport behavior. -- GUI state should update in realtime (no manual refresh buttons). Prefer RivetKit push reactivity and actor-driven events; do not add polling/refetch for normal product flows. -- Keep the mock workspace types and mock client in `packages/shared` + `packages/client` up to date with the frontend contract. The mock is the UI testing reference implementation while backend functionality catches up. -- Keep frontend route/state coverage current in code and tests; there is no separate page-inventory doc to maintain. -- If Foundry uses a shared component from `@sandbox-agent/react`, make changes in `sdks/react` instead of copying or forking that component into Foundry. -- When changing shared React components in `sdks/react` for Foundry, verify they still work in the Sandbox Agent Inspector before finishing. -- When making UI changes, verify the live flow with the Chrome DevTools MCP or `agent-browser`, take screenshots of the updated UI, and offer to open those screenshots in Preview when you finish. -- When asked for screenshots, capture all relevant affected screens and modal states, not just a single viewport. Include empty, populated, success, and blocked/error states when they are part of the changed flow. -- If a screenshot catches a transition frame, blank modal, or otherwise misleading state, retake it before reporting it. -- When verifying UI in the browser, attempt to sign in by navigating to `/signin` and clicking "Continue with GitHub". If the browser lands on the GitHub login page (github.com/login) and you don't have credentials, stop and ask the user to complete the sign-in. Do not assume the session is invalid just because you see the Foundry sign-in page — always attempt the OAuth flow first. - -## Realtime Data Architecture - -### Core pattern: fetch initial state + subscribe to deltas - -All client data flows follow the same pattern: - -1. **Connect** to the actor via WebSocket. -2. **Fetch initial state** via an action call to get the current materialized snapshot. -3. **Subscribe to events** on the connection. Events carry **full replacement payloads** for the changed entity (not empty notifications, not patches — the complete new state of the thing that changed). -4. **Unsubscribe** after a 30-second grace period when interest ends (screen navigation, component unmount). The grace period prevents thrashing during screen transitions and React double-renders. - -Do not use polling (`refetchInterval`), empty "go re-fetch" broadcast events, or full-snapshot re-fetches on every mutation. Every mutation broadcasts the new absolute state of the changed entity to connected clients. - -### Materialized state in coordinator actors - -- **Organization actor** materializes sidebar-level data in its own SQLite: repo catalog, task summaries (title, status, branch, PR, updatedAt), repo summaries (overview/branch state), and session summaries (id, name, status, unread, model — no transcript). Task actors push summary changes to the organization actor when they mutate. The organization actor broadcasts the updated entity to connected clients. `getOrganizationSummary` reads from local tables only — no fan-out to child actors. -- **Task actor** materializes its own detail state (session summaries, sandbox info, diffs, file tree). `getTaskDetail` reads from the task actor's own SQLite. The task actor broadcasts updates directly to clients connected to it. -- **Session data** lives on the task actor but is a separate subscription topic. The task topic includes `sessions_summary` (list without content). The `session` topic provides full transcript and draft state. Clients subscribe to the `session` topic for whichever session is active, and filter `sessionUpdated` events by session ID (ignoring events for other sessions on the same actor). -- There is no fan-out on the read path. The organization actor owns all task summaries locally. - -### Subscription manager - -The subscription manager (`packages/client`) is a global singleton that manages WebSocket connections, cached state, and subscriptions for all topics. It: - -- **Deduplicates** — multiple subscribers to the same topic share one connection and one cached state. -- **Grace period (30s)** — when the last subscriber leaves, the connection and state stay alive for 30 seconds before teardown. This keeps data warm for back-navigation and prevents thrashing. -- **Exposes a single hook** — `useSubscription(topicKey, params)` returns `{ data, status, error }`. Null params = no subscription (conditional subscription). -- **Shared harness, separate implementations** — the `SubscriptionManager` interface is shared between mock and remote implementations. The mock implementation uses in-memory state. The remote implementation uses WebSocket connections. The API/client exposure is identical for both. - -### Topics - -Each topic maps to one actor connection and one event stream: - -| Topic | Actor | Event | Data | -|---|---|---|---| -| `app` | Organization `"app"` | `appUpdated` | Auth, orgs, onboarding | -| `organization` | Organization `{organizationId}` | `organizationUpdated` | Repo catalog, task summaries, repo summaries | -| `task` | Task `{organizationId, repoId, taskId}` | `taskUpdated` | Session summaries, sandbox info, diffs, file tree | -| `session` | Task `{organizationId, repoId, taskId}` (filtered by sessionId) | `sessionUpdated` | Transcript, draft state | -| `sandboxProcesses` | SandboxInstance | `processesUpdated` | Process list | - -The client subscribes to `app` always, `organization` when entering an organization, `task` when viewing a task, and `session` when viewing a specific session. At most 4 actor connections at a time (app + organization + task + sandbox if terminal is open). The `session` topic reuses the task actor connection and filters by session ID. - -### Rules - -- Do not add `useQuery` with `refetchInterval` for data that should be push-based. -- Do not broadcast empty notification events. Events must carry the full new state of the changed entity. -- Do not re-fetch full snapshots after mutations. The mutation triggers a server-side broadcast with the new entity state; the client replaces it in local state. -- All event subscriptions go through the subscription manager. Do not create ad-hoc `handle.connect()` + `conn.on()` patterns. -- Backend mutations that affect sidebar data (task title, status, branch, PR state) must push the updated summary to the parent organization actor, which broadcasts to organization subscribers. -- Comment architecture-related code: add doc comments explaining the materialized state pattern, why deltas flow the way they do, and the relationship between parent/child actor broadcasts. New contributors should understand the data flow from comments alone. - -## Sandbox Architecture - -- Structurally, the system supports multiple sandboxes per task, but in practice there is exactly one active sandbox per task. Design features assuming one sandbox per task. If multi-sandbox is needed in the future, extend at that time. -- Each task has a **primary user** (owner) whose GitHub OAuth credentials are injected into the sandbox for git operations. The owner swaps when a different user sends a message. See `.context/proposal-task-owner-git-auth.md` for the full design. -- **Security: OAuth token scope.** The user's GitHub OAuth token has `repo` scope, granting full control of all private repositories the user has access to. When the user is the active task owner, their token is injected into the sandbox. This means the agent can read/write ANY repo the user has access to, not just the task's target repo. This is the standard trade-off for OAuth-based git integrations (same as GitHub Codespaces, Gitpod). The user consents to `repo` scope at sign-in time. Credential files in the sandbox are `chmod 600` and overwritten on owner swap. -- All git operations in the sandbox must be auto-authenticated. Never configure git to prompt for credentials (no interactive `GIT_ASKPASS` prompts). Use a credential store file that is pre-populated with the active owner's token. -- All git operation errors (push 401, clone failure, branch protection rejection) must surface in the UI with actionable context. Never silently swallow git errors. - -## Git State Policy - -- The backend stores zero git state. No local clones, no refs, no working trees, and no git-spice. -- Repository metadata (branches, default branch, pull requests) comes from GitHub API data and webhook events already flowing into the system. -- All git operations that require a working tree run inside the task's sandbox via `executeInSandbox()`. -- Do not add backend git clone paths, `git fetch`, `git for-each-ref`, or direct backend git CLI calls. If you need git data, either read stored GitHub metadata or run the command inside a sandbox. -- The `BackendDriver` has no `GitDriver` or `StackDriver`. Only `GithubDriver` and `TmuxDriver` remain. - -## React Hook Dependency Safety - -- **Never use unstable references as `useEffect`/`useMemo`/`useCallback` dependencies.** React compares dependencies by reference, not value. Expressions like `?? []`, `?? {}`, `.map(...)`, `.filter(...)`, or object/array literals create new references every render, causing infinite re-render loops when used as dependencies. -- If the upstream value may be `undefined`/`null` and you need a fallback, either: - - Use the raw upstream value as the dependency and apply the fallback inside the effect body: `useEffect(() => { doThing(value ?? []); }, [value]);` - - Derive a stable primitive key: `const key = JSON.stringify(value ?? []);` then depend on `key` - - Memoize: `const stable = useMemo(() => value ?? [], [value]);` -- When reviewing code, treat any `?? []`, `?? {}`, or inline `.map()/.filter()` in a dependency array as a bug. - -## UI System - -- Foundry's base UI system is `BaseUI` with `Styletron`, plus Foundry-specific theme/tokens on top. Treat that as the default UI foundation. -- The full `BaseUI` reference for available components and guidance on animations, customization, composition, and forms is at `https://base-ui.com/llms.txt`. -- Prefer existing `BaseUI` components and composition patterns whenever possible instead of building custom controls from scratch. -- Reuse the established Foundry theme/token layer for colors, typography, spacing, and surfaces instead of introducing ad hoc visual values. -- If the same UI pattern is shared with the Inspector or other consumers, prefer extracting or reusing it through `@sandbox-agent/react` rather than duplicating it in Foundry. -- If a requested UI cannot be implemented cleanly with an existing `BaseUI` component, stop and ask the user whether they are sure they want to diverge from the system. -- In that case, recommend the closest existing `BaseUI` components or compositions that could satisfy the need before proposing custom UI work. -- Only introduce custom UI primitives when `BaseUI` and existing Foundry patterns are not sufficient, or when the user explicitly confirms they want the divergence. -- **Styletron atomic CSS rule:** Never mix CSS shorthand properties with their longhand equivalents in the same style object (including nested pseudo-selectors like `:hover`), or in a base styled component whose consumers override with longhand via `$style`. This includes `padding`/`paddingLeft`, `margin`/`marginTop`, `background`/`backgroundColor`, `border`/`borderLeft`, etc. Styletron generates independent atomic classes for shorthand and longhand, so they conflict unpredictably. Use `backgroundColor: "transparent"` instead of `background: "none"` for button resets. Always use longhand properties when any side may be overridden individually. - -## Runtime Policy - -- Runtime is Bun-native. -- Use Bun for CLI/backend execution paths and process spawning. -- Do not add Node compatibility fallbacks for OpenTUI/runtime execution. - -## Defensive Error Handling - -- Write code defensively: validate assumptions at boundaries and state transitions. -- If the system reaches an unexpected state, raise an explicit error with actionable context. -- Do not fail silently, swallow errors, or auto-ignore inconsistent data. -- Prefer fail-fast behavior over hidden degradation when correctness is uncertain. -- **Never use bare `catch {}` or `catch { }` blocks.** Every catch must at minimum log the error with `logActorWarning` or `console.warn`. Silent catches hide bugs and make debugging impossible. If a catch is intentionally degrading (e.g. returning empty data when a sandbox is expired), it must still log so operators can see what happened. Use `catch (error) { logActorWarning(..., { error: resolveErrorMessage(error) }); }` or equivalent. - -## RivetKit Dependency Policy - -For all Rivet/RivetKit implementation: - -1. Use SQLite + Drizzle for persistent state. -2. SQLite is **per actor instance** (per actor key), not a shared backend-global database: - - Each actor instance gets its own SQLite DB. - - Schema design should assume a single actor instance owns the entire DB. - - Do not add `organizationId`/`repoId`/`taskId` columns just to "namespace" rows for a given actor instance; use actor state and/or the actor key instead. - - Example: the `task` actor instance already represents `(organizationId, repoId, taskId)`, so its SQLite tables should not need those columns for primary keys. -3. Do not use backend-global SQLite singletons; database access must go through actor `db` providers (`c.db`). -4. The default dependency source for RivetKit is the published `rivetkit` package so monorepo installs and CI remain self-contained. - -## Rivet Routing - -- Mount RivetKit directly on `/v1/rivet` via `registry.handler(c.req.raw)`. -- Do not add an extra proxy or manager-specific route layer in the backend. -- Let RivetKit own metadata/public endpoint behavior for `/v1/rivet`. - -## Organization + Actor Rules - -- Everything is scoped to an organization. -- Organization resolution order: `--organization` flag -> config default -> `"default"`. -- `ControlPlaneActor` is replaced by `OrganizationActor` (organization coordinator). -- Every actor key must be prefixed with organization namespace (`["org", organizationId, ...]`). -- CLI/TUI/GUI must use `@sandbox-agent/foundry-client` (`packages/client`) for backend access; `rivetkit/client` imports are only allowed inside `packages/client`. -- Do not add custom backend REST endpoints (no `/v1/*` shim layer). -- We own the sandbox-agent project; treat sandbox-agent defects as first-party bugs and fix them instead of working around them. -- Keep strict single-writer ownership: each table/row has exactly one actor writer. -- Parent actors (`organization`, `task`, `sandbox-instance`) use command-only loops with no timeout. -- Periodic syncing lives in dedicated child actors with one timeout cadence each. -- **Task actors must be created lazily** — never during sync or bulk operations. PR sync writes virtual entries to the org's local `taskIndex`/`taskSummaries` tables. The task actor is created on first user interaction via `getOrCreate`. See `packages/backend/CLAUDE.md` "Lazy Task Actor Creation" for details. -- Do not build blocking flows that wait on external systems to become ready or complete. Prefer push-based progression driven by actor messages, events, webhooks, or queue/workflow state changes. -- Use workflows/background commands for any repo sync, sandbox provisioning, agent install, branch restack/rebase, or other multi-step external work. Do not keep user-facing actions/requests open while that work runs. -- `send` policy: always `await` the `send(...)` call itself so enqueue failures surface immediately, but default to `wait: false`. -- Never self-send with `wait: true` from inside a workflow handler — the workflow processes one message at a time, so the handler would deadlock waiting for the new message to be dequeued. -- Read paths must not force refresh/sync work inline. Serve the latest cached projection, mark staleness explicitly, and trigger background refresh separately when needed. -- If a workflow needs to resume after some external work completes, model that as workflow state plus follow-up messages/events instead of holding the original request open. -- No retries: never add retry loops (`withRetries`, `setTimeout` retry, exponential backoff) anywhere in the codebase. If an operation fails, surface the error immediately. If a dependency is not ready yet, model that explicitly with workflow state and resume from a push/event instead of polling or retry loops. -- Never throw errors that expect the caller to retry (e.g. `throw new Error("... retry shortly")`). If a dependency is not ready, write the current state to the DB with an appropriate pending status, enqueue the async work, and return successfully. Let the client observe the pending → ready transition via push events. -- Action return contract: every action that creates a resource must write the resource record to the DB before returning, so the client can immediately query/render it. The record may have a pending status, but it must exist. Never return an ID that doesn't yet have a corresponding DB row. - -### Action handler responsiveness - -Action handlers must return fast. The pattern: - -1. **Creating an entity** — `wait: true` is fine. Do the DB write, return the ID/record. The caller needs the ID to proceed. The record may have a pending status; that's expected. -2. **Enqueuing work** (sending a message, triggering a sandbox operation, starting a sync) — `wait: false`. Write any precondition state to the DB synchronously, enqueue the work, and return. The client observes progress via push events on the relevant topic (session status, task status, etc.). -3. **Validating preconditions** — check state synchronously in the action handler *before* enqueuing. If a precondition isn't met (e.g. session not ready, task not initialized), throw an error immediately. Do not implicitly provision missing dependencies or poll for readiness inside the action handler. It is the client's responsibility to ensure preconditions are met before calling the action. - -Examples: -- `createTask` → `wait: true` (returns `{ taskId }`), then enqueue provisioning with `wait: false`. Client sees task appear immediately with pending status, observes `ready` via organization events. -- `sendWorkspaceMessage` → validate session is `ready` (throw if not), enqueue with `wait: false`. Client observes session transition to `running` → `idle` via session events. -- `createWorkspaceSession` → `wait: true` (returns `{ sessionId }`), enqueue sandbox provisioning with `wait: false`. Client observes `pending_provision` → `ready` via task events. - -Never use `wait: true` for operations that depend on external readiness, sandbox I/O, agent responses, git network operations, polling loops, or long-running queue drains. Never hold an action open while waiting for an external system to become ready — that is a polling/retry loop in disguise. - -### Timeout policy - -All `wait: true` sends must have an explicit `timeout`. Maximum timeout for any `wait: true` send is **10 seconds** (`10_000`). If an operation cannot reliably complete within 10 seconds, it must be restructured: write the initial record to the DB, return it to the caller, and continue the work asynchronously with `wait: false`. The client observes completion via push events. - -`wait: false` sends do not need a timeout (the enqueue is instant; the work runs in the workflow loop with its own step-level timeouts). - -### Task creation: resolve metadata before creating the actor - -When creating a task, all deterministic metadata (title, branch name) must be resolved synchronously in the organization actor *before* the task actor is created. The task actor must never be created with null `branchName` or `title`. - -- Title is derived from the task description via `deriveFallbackTitle()` — pure string manipulation, no external I/O. -- Branch name is derived from the title via `sanitizeBranchName()` + conflict checking against the repository's task index. -- The organization actor owns the task index and reads GitHub-backed default branch metadata from the github-data actor. Resolve the branch name there without local git fetches. -- Do not defer naming to a background provision workflow. Do not poll for names to become available. -- The `onBranch` path (attaching to an existing branch) and the new-task path should both produce a fully-named task record on return. -- Actor handle policy: -- Prefer explicit `get` or explicit `create` based on workflow intent; do not default to `getOrCreate`. -- Use `get`/`getForId` when the actor is expected to already exist; if missing, surface an explicit `Actor not found` error with recovery context. -- Use create semantics only on explicit provisioning/create paths where creating a new actor instance is intended. -- `getOrCreate` is a last resort for create paths when an explicit create API is unavailable; never use it in read/command paths. -- For long-lived cross-actor links (for example sandbox/session runtime access), persist actor identity (`actorId`) and keep a fallback lookup path by actor id. -- RivetKit actor `c.state` is durable, but in Docker it is stored under `/root/.local/share/rivetkit`. If that path is not persisted, actor state-derived indexes can be lost after container recreation even when other data still exists. -- Workflow history divergence policy: -- Production: never auto-delete actor state to resolve `HistoryDivergedError`; ship explicit workflow migrations (`ctx.removed(...)`, step compatibility). -- Development: manual local state reset is allowed as an operator recovery path when migrations are not yet available. -- Storage rule of thumb: -- Put simple metadata in `c.state` (KV state): small scalars and identifiers like `{ taskId }`, `{ repoId }`, booleans, counters, timestamps, status strings. -- If it grows beyond trivial (arrays, maps, histories, query/filter needs, relational consistency), use SQLite + Drizzle in `c.db`. - -## Testing Policy - -- Never use vitest mocks (`vi.mock`, `vi.spyOn`, `vi.fn`). Instead, define driver interfaces for external I/O and pass test implementations via the actor runtime context. -- All external service calls (git CLI, GitHub CLI, sandbox-agent HTTP, tmux) must go through the `BackendDriver` interface on the runtime context. -- Integration tests use `setupTest()` from `rivetkit/test` and are gated behind `HF_ENABLE_ACTOR_INTEGRATION_TESTS=1`. -- End-to-end testing must run against the dev backend started via `docker compose -f compose.dev.yaml up` (host -> container). Do not run E2E against an in-process test runtime. - - E2E tests should talk to the backend over HTTP (default `http://127.0.0.1:7741/v1/rivet`) and use real GitHub repos/PRs. - - For Foundry live verification, use `rivet-dev/sandbox-agent-testing` as the default testing repo unless the task explicitly says otherwise. - - Secrets (e.g. `OPENAI_API_KEY`, `GITHUB_TOKEN`/`GH_TOKEN`) must be provided via environment variables, never hardcoded in the repo. - - `~/misc/env.txt` and `~/misc/the-foundry.env` contain the expected local OpenAI + GitHub OAuth/App config for dev. - - For local GitHub webhook development, use the configured Smee proxy (`SMEE_URL`) to forward deliveries into `POST /v1/webhooks/github`. Check `.env` / `foundry/.env` if you need the current channel URL. - - If GitHub repos, PRs, or install state are not showing up, verify that the GitHub App is installed for the organization and that webhook delivery is enabled and healthy. Foundry depends on webhook events for GitHub-backed state; missing webhooks means the product will appear broken. - - Do not assume `gh auth token` is sufficient for Foundry task provisioning against private repos. Sandbox/bootstrap git clone, push, and PR flows require a repo-capable `GITHUB_TOKEN`/`GH_TOKEN` in the backend container. - - Preferred product behavior for organizations is to mint a GitHub App installation token from the organization installation and inject it into backend/sandbox git operations. Do not rely on an operator's ambient CLI auth as the long-term solution. -- Treat client E2E tests in `packages/client/test` as the primary end-to-end source of truth for product behavior. -- Keep backend tests small and targeted. Only retain backend-only tests for invariants or persistence rules that are not well-covered through client E2E. -- Do not keep large browser E2E suites around in a broken state. If a frontend browser E2E is not maintained and producing signal, remove it until it can be replaced with a reliable test. - -## Config - -- Keep config path at `~/.config/foundry/config.toml`. -- Evolve properties in place; do not move config location. - -## Project Guidance - -Project-specific guidance lives in `README.md`, `CONTRIBUTING.md`, and the relevant files under `research/`. - -Keep those updated when: - -- Commands change -- Configuration options change -- Architecture changes -- Plugins/providers change -- Actor ownership changes - -## Friction Logs - -Track friction at: - -- `research/friction/rivet.mdx` -- `research/friction/sandbox-agent.mdx` -- `research/friction/sandboxes.mdx` -- `research/friction/general.mdx` - -Category mapping: - -- `rivet`: Rivet/RivetKit runtime, actor model, queues, keys -- `sandbox-agent`: sandbox-agent SDK/API behavior -- `sandboxes`: provider implementations (worktree/daytona/etc) -- `general`: everything else - -Each entry must include: - -- Date (`YYYY-MM-DD`) -- Commit SHA (or `uncommitted`) -- What you were implementing -- Friction/issue -- Attempted fix/workaround and outcome - -## Audit Log Events - -Log notable workflow changes to `events` so the audit log remains complete: - -- create -- attach -- push/sync/merge -- archive/kill -- status transitions -- PR state transitions - -When adding new task/workspace commands, always add a corresponding audit log event. - -## Validation After Changes - -Always run and fix failures: - -```bash -pnpm -w typecheck -pnpm -w build -pnpm -w test -``` - -After making code changes, always update the dev server before declaring the work complete. If the dev stack is running through Docker Compose, restart or recreate the relevant dev services so the running app reflects the latest code. +@AGENTS.md diff --git a/foundry/packages/backend/AGENTS.md b/foundry/packages/backend/AGENTS.md new file mode 100644 index 00000000..2865c0b1 --- /dev/null +++ b/foundry/packages/backend/AGENTS.md @@ -0,0 +1,34 @@ +# Backend Instructions (current RivetKit runtime) + +Supersession: this runtime migrates to Effect's actor model (root `STRATEGY.md`). The law below binds until a module migrates; delete each rule as its module migrates. Topology: [../../ARCHITECTURE.md](../../ARCHITECTURE.md). + +## Actor law + +- Coordinator pattern: children push state changes up to their direct coordinator only; read paths use the coordinator's local index tables. No fan-out reads; no "enriched" read actions chaining coordinator to child to sibling - fix the write path to push instead. +- Only a coordinator creates or destroys its children; every actor key is prefixed with the organization namespace. +- Never create task actors during GitHub sync, webhooks, or bulk operations - that OOM-crashed the backend (186 actors during one PR sync). `getOrCreateTask` is allowed only in: `createTaskMutation`, user-initiated `requireWorkspaceTask` paths, the org `getTask` action, and the `backend-client` lazy materialization. +- Workflow primitives (`c.step()`, `c.loop()`, `c.sleep()`, `c.queue.next()`) appear only at the top level of a workflow function or inside a `loop` callback - never inside a step's `run` (runtime crash: "Cannot start a new workflow entry while another is in progress"). Mutations called from steps do plain DB/API work only. +- Queue for multi-step external I/O and must-not-lose messages; direct action for pure RPCs. `wait: true` only when the caller needs the return value, capped at 10 seconds with an explicit timeout; enqueues default to `wait: false`. Never self-send with `wait: true` from inside a workflow handler (deadlock). +- No retry loops and no "retry shortly" throws: write pending state, enqueue the work, let clients observe the transition via push events. +- Every action that creates a resource writes its DB row before returning. +- Task metadata (title via `deriveFallbackTitle()`, branch via `sanitizeBranchName()` + conflict check) resolves synchronously before the task actor exists; branch names are immutable after creation. +- Read paths serve the latest cached projection and trigger background refresh separately; never force sync work inline. + +## Persistence + +- SQLite is per actor instance via `c.db`; no backend-global singletons; no organization/repo/task namespacing columns inside an actor's own tables. +- Drizzle: after changing an actor's `db/schema.ts`, generate the migration and regenerate the compiled migrations, then verify NOT NULL coverage. + Gates (from `packages/backend`): `npx drizzle-kit generate --config=./src/actors//db/drizzle.config.ts`; then from `foundry/`: `npx tsx packages/backend/src/actors/_scripts/generate-actor-migrations.ts`. +- Single-row tables use an integer primary key with `CHECK (id = 1)`; row id is fixed as `1` in code. +- Actors with drizzle migrations: `organization`, `audit-log`, `task`; `user` and `github-data` use inline migrations. +- After migration changes in dev, reset RivetKit state: `docker compose -f compose.dev.yaml down && docker volume rm foundry_foundry_rivetkit_storage && docker compose -f compose.dev.yaml up -d`. + +## State ownership + +- Per-user UI state (active session tab, unread counts, drafts) lives on the user actor keyed by `(userId, taskId, sessionId)`; task-global state (transcript, model, runtime status, sandbox identity, branch, PR state) lives on the task actor. If two users looking at the same task should see different values, it belongs on the user actor. +- User actor owns the task feed (`userTaskFeed` bumps) and notifications (`userNotifications`); task/org actors resolve the task owner and enqueue `user.bump_task` / `user.notify`. + +## Debugging + +- RivetKit inspector at `http://localhost:6420/ui/` is the reliable surface; the `/inspector/workflow-history` API returns empty `{}` even with entries - use the UI's Workflow tab. `GET /inspector/queue` and `/inspector/state` are reliable. +- Find actor ids: `curl -s 'http://127.0.0.1:6420/actors?name=organization'`. Query an actor DB from inside the container with `bun:sqlite` (readonly) at `/root/.local/share/foundry/rivetkit/databases/.db`. diff --git a/foundry/packages/backend/CLAUDE.md b/foundry/packages/backend/CLAUDE.md index f7e054d4..43c994c2 100644 --- a/foundry/packages/backend/CLAUDE.md +++ b/foundry/packages/backend/CLAUDE.md @@ -1,338 +1 @@ -# Backend Notes - -## Actor Hierarchy - -Keep the backend actor tree aligned with this shape unless we explicitly decide to change it: - -```text -OrganizationActor (direct coordinator for tasks) -├─ AuditLogActor (organization-scoped global feed) -├─ GithubDataActor -├─ TaskActor(task) -│ ├─ taskSessions → session metadata/transcripts -│ └─ taskSandboxes → sandbox instance index -└─ SandboxInstanceActor(sandboxProviderId, sandboxId) × N -``` - -## Coordinator Pattern - -Actors follow a coordinator pattern where each coordinator is responsible for: -1. **Index tables** — keeping a local SQLite index/summary of its child actors' data -2. **Create/destroy** — handling lifecycle of child actors -3. **Routing** — resolving lookups to the correct child actor - -Children push updates **up** to their direct coordinator only. Coordinators broadcast changes to connected clients. This keeps the read path local (no fan-out to children). - -### Coordinator hierarchy and index tables - -```text -OrganizationActor (coordinator for tasks + auth users) -│ -│ Index tables: -│ ├─ taskIndex → TaskActor index (taskId → repoId + branchName) -│ ├─ taskSummaries → TaskActor materialized sidebar projection -│ ├─ authSessionIndex → UserActor index (session token → userId) -│ ├─ authEmailIndex → UserActor index (email → userId) -│ └─ authAccountIndex → UserActor index (OAuth account → userId) -│ -├─ TaskActor (coordinator for sessions + sandboxes) -│ │ -│ │ Index tables: -│ │ ├─ taskWorkspaceSessions → Session index (session metadata + transcript) -│ │ └─ taskSandboxes → SandboxInstanceActor index (sandbox history) -│ │ -│ └─ SandboxInstanceActor (leaf) -│ -├─ AuditLogActor (organization-scoped audit log, not a coordinator) -└─ GithubDataActor (GitHub API cache, not a coordinator) -``` - -When adding a new index table, annotate it in the schema file with a doc comment identifying it as a coordinator index and which child actor it indexes (see existing examples). - -## GitHub Sync Data Model - -The GithubDataActor syncs **repositories** and **pull requests** from GitHub, not branches. We only need repos (to know which repos exist and their metadata) and PRs (to lazily populate virtual tasks in the sidebar). Branch data is not synced because we only create tasks from PRs or fresh user-initiated creation, never from bare branches. Generated branch names for new tasks are treated as unique enough to skip conflict detection against remote branches. - -Tasks are either: -1. **Created fresh** by the user (no PR yet, branch name generated from task description) -2. **Lazily populated from pull requests** during PR sync (virtual task entries in org tables, no actor spawned) - -## Lazy Task Actor Creation — CRITICAL - -**Task actors must NEVER be created during GitHub sync or bulk operations.** Creating hundreds of task actors simultaneously causes OOM crashes. An org can have 200+ PRs; spawning an actor per PR kills the process. - -### The two creation points - -There are exactly **two** places that may create a task actor: - -1. **`createTaskMutation`** in `task-mutations.ts` — the only backend code that calls `getOrCreateTask`. Triggered by explicit user action ("New Task" button). One actor at a time. - -2. **`backend-client.ts` client helper** — calls `client.task.getOrCreate(...)`. This is the lazy materialization point: when a user clicks a virtual task in the sidebar, the client creates the actor, and it self-initializes in `getCurrentRecord()` (`workflow/common.ts`) by reading branch/title from the org's `getTaskIndexEntry` action. - -### The rule - -### The rule - -**Never use `getOrCreateTask` inside a sync loop, webhook handler, or any bulk operation.** That's what caused the OOM — 186 actors spawned simultaneously during PR sync. - -`getOrCreateTask` IS allowed in: -- `createTaskMutation` — explicit user "New Task" action -- `requireWorkspaceTask` — user-initiated actions (createSession, sendMessage, etc.) that may hit a virtual task -- `getTask` action on the org — called by sandbox actor and client, needs to materialize virtual tasks -- `backend-client.ts` client helper — lazy materialization when user views a task - -### Virtual tasks (PR-driven) - -During PR sync, `refreshTaskSummaryForBranchMutation` is called for every changed PR (via github-data's `emitPullRequestChangeEvents`). It writes **virtual task entries** to the org actor's local `taskIndex` + `taskSummaries` tables only. No task actor is spawned. No cross-actor calls to task actors. - -When the user interacts with a virtual task (clicks it, creates a session): -1. Client or org actor calls `getOrCreate` on the task actor key → actor is created with empty DB -2. Any action on the actor calls `getCurrentRecord()` → sees empty DB → reads branch/title from org's `getTaskIndexEntry` → calls `initBootstrapDbActivity` + `initCompleteActivity` → task is now real - -### Call sites to watch - -- `refreshTaskSummaryForBranchMutation` — called in bulk during sync. Must ONLY write to org local tables. Never create task actors or call task actor actions. -- `emitPullRequestChangeEvents` in github-data — iterates all changed PRs. Must remain fire-and-forget with no actor fan-out. - -## Queue vs Action Decision Framework - -The default is a direct action. Use a queue only if the answer to one or more of these questions is **yes**. - -Actions are pure RPCs with no DB overhead on send — fast, but if the call fails the operation is lost. Queues persist the message to the database on send, guaranteeing it will be processed even if the target actor is busy, slow, or recovering. The tradeoff: queues add write overhead and serialize processing. - -### 1. Does this operation coordinate multi-step work? - -Does it involve external I/O (sandbox API, GitHub API, agent process management) or state machine transitions where interleaving would corrupt state? This is different from database-level serialization — a simple read-then-write on SQLite can use a transaction. The queue is for ordering operations that span DB writes + external I/O. - -**Queue examples:** -- `workspace.send_message` — sends to sandbox agent, writes session status, does owner-swap. Multi-step with external I/O. -- `push` / `sync` / `merge` — git operations in sandbox that must not interleave. -- `createTask` — read-then-write across task index + actor creation. Returns result, so `wait: true`. - -**Action examples:** -- `billing.stripe_customer.apply` — single column upsert, no external I/O. -- `workspace.update_draft` — writes draft text, no coordination with sandbox ops. -- `workspace.rename_task` — updates title column, queue handlers don't touch title. - -### 2. Must this message be processed no matter what? - -Is this a cross-actor fire-and-forget where the caller won't retry and data loss is unacceptable? A queue persists the message — if the target is down, it waits. An action RPC that fails is gone. - -**Queue examples:** -- `audit.append` — caller must never be affected by audit failures, and audit entries must not be lost. -- `applyTaskSummaryUpdate` — task actor pushes summary to org and moves on. Won't retry if org is busy. -- `refreshTaskSummaryForBranch` — webhook-driven, won't be redelivered for the same event. - -**Action examples:** -- `billing.invoice.upsert` — Stripe retries handle failures externally. No durability need on our side. -- `workspace.mark_unread` — UI convenience state. Acceptable to lose on transient failure. -- `github.webhook_receipt.record` — timestamp columns with no downstream effects. - -### Once on a queue: wait or fire-and-forget? - -If the caller needs a return value, use `wait: true`. If the UI updates via push events, use `wait: false`. - -Full migration plan: `QUEUE_TO_ACTION_MIGRATION.md`. - -## Ownership Rules - -- `OrganizationActor` is the organization coordinator, direct coordinator for tasks, and lookup/index owner. It owns the task index, task summaries, and repo catalog. -- `AuditLogActor` is organization-scoped. There is one organization-level audit log feed. -- `TaskActor` is one branch. Treat `1 task = 1 branch` once branch assignment is finalized. -- `TaskActor` can have many sessions. -- `TaskActor` can reference many sandbox instances historically, but should have only one active sandbox/session at a time. -- Session unread state and draft prompts are backend-owned workspace state, not frontend-local state. -- Branch names are immutable after task creation. Do not implement branch-rename flows. -- `SandboxInstanceActor` stays separate from `TaskActor`; tasks/sessions reference it by identity. -- The backend stores no local git state. No clones, no refs, no working trees, and no git-spice. Repository metadata comes from GitHub API data and webhook events. Any working-tree git operation runs inside a sandbox via `executeInSandbox()`. -- When a backend request path must aggregate multiple independent actor calls or reads, prefer bounded parallelism over sequential fan-out when correctness permits. Do not serialize independent work by default. -- Only a coordinator creates/destroys its children. Do not create child actors from outside the coordinator. -- Children push state changes up to their direct coordinator only. Task actors push summary updates directly to the organization actor. -- Read paths must use the coordinator's local index tables. Do not fan out to child actors on the hot read path. -- Never build "enriched" read actions that chain through multiple actors (e.g., coordinator → child actor → sibling actor). If data from multiple actors is needed for a read, it should already be materialized in the coordinator's index tables via push updates. If it's not there, fix the write path to push it — do not add a fan-out read path. - -## Drizzle Migration Maintenance - -After changing any actor's `db/schema.ts`, you **must** regenerate the corresponding migration so the runtime creates the tables that match the schema. Forgetting this step causes `no such table` errors at runtime. - -1. **Generate a new drizzle migration.** Run from `packages/backend`: - ```bash - npx drizzle-kit generate --config=./src/actors//db/drizzle.config.ts - ``` - If the interactive prompt is unavailable (e.g. in a non-TTY), manually create a new `.sql` file under `./src/actors//db/drizzle/` and add the corresponding entry to `meta/_journal.json`. - -2. **Regenerate the compiled `migrations.ts`.** Run from the foundry root: - ```bash - npx tsx packages/backend/src/actors/_scripts/generate-actor-migrations.ts - ``` - -3. **Verify insert/upsert calls.** Every column with `.notNull()` (and no `.default(...)`) must be provided a value in all `insert()` and `onConflictDoUpdate()` calls. Missing a NOT NULL column causes a runtime constraint violation, not a type error. - -4. **Nuke RivetKit state in dev** after migration changes to start fresh: - ```bash - docker compose -f compose.dev.yaml down - docker volume rm foundry_foundry_rivetkit_storage - docker compose -f compose.dev.yaml up -d - ``` - -Actors with drizzle migrations: `organization`, `audit-log`, `task`. Other actors (`user`, `github-data`) use inline migrations without drizzle. - -## Workflow Step Nesting — FORBIDDEN - -**Never call `c.step()` / `ctx.step()` from inside another step's `run` callback.** RivetKit workflow steps cannot be nested. Doing so causes the runtime error: *"Cannot start a new workflow entry while another is in progress."* - -This means: -- Functions called from within a step `run` callback must NOT use `c.step()`, `c.loop()`, `c.sleep()`, or `c.queue.next()`. -- If a mutation function needs to be called both from a step and standalone, it must only do plain DB/API work — no workflow primitives. The workflow step wrapping belongs in the workflow file, not in the mutation. -- Helper wrappers that conditionally call `c.step()` (like a `runSyncStep` pattern) are dangerous — if the caller is already inside a step, the nested `c.step()` will crash at runtime with no compile-time warning. - -**Rule of thumb:** Workflow primitives (`step`, `loop`, `sleep`, `queue.next`) may only appear at the top level of a workflow function or inside a `loop` callback — never inside a step's `run`. - -## SQLite Constraints - -- Single-row tables must use an integer primary key with `CHECK (id = 1)` to enforce the singleton invariant at the database level. -- Follow the task actor pattern for metadata/profile rows and keep the fixed row id in code as `1`, not a string sentinel. - -## Multiplayer Correctness - -Per-user UI state must live on the user actor, not on shared task/session actors. This is critical for multiplayer — multiple users may view the same task simultaneously with different active sessions, unread states, and in-progress drafts. - -**Per-user state (user actor):** active session tab, unread counts, draft text, draft attachments. Keyed by `(userId, taskId, sessionId)`. - -**Task-global state (task actor):** session transcript, session model, session runtime status, sandbox identity, task status, branch name, PR state. These are shared across all users viewing the task — that is correct behavior. - -Do not store per-user preferences, selections, or ephemeral UI state on shared actors. If a field's value should differ between two users looking at the same task, it belongs on the user actor. - -## Audit Log Maintenance - -Every new action or command handler that represents a user-visible or workflow-significant event must append to the audit log actor. The audit log must remain a comprehensive record of significant operations. - -## Debugging Actors - -### RivetKit Inspector UI - -The RivetKit inspector UI at `http://localhost:6420/ui/` is the most reliable way to debug actor state in local development. The inspector HTTP API (`/inspector/workflow-history`) has a known bug where it returns empty `{}` even when the workflow has entries — always cross-check with the UI. - -**Useful inspector URL pattern:** -``` -http://localhost:6420/ui/?u=http%3A%2F%2F127.0.0.1%3A6420&ns=default&r=default&n=[%22%22]&actorId=&tab= -``` - -Tabs: `workflow`, `database`, `state`, `queue`, `connections`, `metadata`. - -**To find actor IDs:** -```bash -curl -s 'http://127.0.0.1:6420/actors?name=organization' -``` - -**To query actor DB via bun (inside container):** -```bash -docker compose -f compose.dev.yaml exec -T backend bun -e ' - var Database = require("bun:sqlite"); - var db = new Database("/root/.local/share/foundry/rivetkit/databases/.db", { readonly: true }); - console.log(JSON.stringify(db.query("SELECT name FROM sqlite_master WHERE type=?").all("table"))); -' -``` - -**To call actor actions via inspector:** -```bash -curl -s -X POST 'http://127.0.0.1:6420/gateway//inspector/action/' \ - -H 'Content-Type: application/json' -d '{"args":[{}]}' -``` - -### Known inspector API bugs - -- `GET /inspector/workflow-history` may return `{"history":{}}` even when workflow has run. Use the UI's Workflow tab instead. -- `GET /inspector/queue` is reliable for checking pending messages. -- `GET /inspector/state` is reliable for checking actor state. - -## Inbox & Notification System - -The user actor owns two per-user systems: a **task feed** (sidebar ordering) and **notifications** (discrete events). These are distinct concepts that share a common "bump" mechanism. - -### Core distinction: bumps vs. notifications - -A **bump** updates the task's position in the user's sidebar feed. A **notification** is a discrete event entry shown in the notification panel. Every notification also triggers a bump, but not every bump creates a notification. - -| Event | Bumps task? | Creates notification? | -|-------|-------------|----------------------| -| User sends a message | Yes | No | -| User opens/clicks a task | Yes | No | -| User creates a session | Yes | No | -| Agent finishes responding | Yes | Yes | -| PR review requested | Yes | Yes | -| PR merged | Yes | Yes | -| PR comment added | Yes | Yes | -| Agent error/needs input | Yes | Yes | - -### Recipient resolution - -Notifications and bumps go to the **task owner** only. Each task has exactly one owner at a time (the user who last sent a message or explicitly took ownership). This is an acceptable race condition — it rarely makes sense for two users to work on the same task simultaneously, and ownership transfer is explicit. - -The system supports multiplayer (multiple users can view the same task), but the notification/bump target is always the single current owner. Each user has their own independent notification and unread state on their own user actor. - -### Tables (on user actor) - -Two new tables: - -- **`userTaskFeed`** — one row per task. Tracks `bumpedAtMs` and `bumpReason` for sidebar sort order. Does NOT denormalize task content (title, repo, etc.) — the frontend queries the org actor for task content and uses the feed only for ordering/filtering. -- **`userNotifications`** — discrete notification entries with `type`, `message`, `read` state, and optional `sessionId`. Retention: notifications are retained for a configurable number of days after being marked read, then cleaned up. - -### Queue commands (user actor workflow) - -- `user.bump_task` — upserts `userTaskFeed` row, no notification created. Used for user-initiated actions (send message, open task, create session). -- `user.notify` — inserts `userNotifications` row AND upserts `userTaskFeed` (auto-bump). Used for system events (agent finished, PR review requested). -- `user.mark_read` — marks notifications read for a given `(taskId, sessionId?)`. Also updates `userTaskState.unread` for the session. - -### Data flow - -Task actor (or org actor) resolves the current task owner, then sends to the owner's user actor queue: -1. `user.notify(...)` for notification-worthy events (auto-bumps the feed) -2. `user.bump_task(...)` for non-notification bumps (send message, open task) - -The user actor processes the queue message, writes to its local tables, and broadcasts a `userFeedUpdated` event to connected clients. - -### Sidebar architecture change - -The left sidebar changes from showing the repo/PR tree to showing **recent tasks** ordered by `userTaskFeed.bumpedAtMs`. Two new buttons at the top of the sidebar: -- **All Repositories** — navigates to a page showing the current repo + PR list (preserving existing functionality) -- **Notifications** — navigates to a page showing the full notification list - -The sidebar reads from two sources: -- **User actor** (`userTaskFeed`) — provides sort order and "which tasks are relevant to this user" -- **Org actor** (`taskSummaries`) — provides task content (title, status, branch, PR state, session summaries) - -The frontend merges these: org snapshot gives task data, user feed gives sort order. Uses the existing subscription system (`useSubscription`) for both initial state fetch and streaming updates. - -### `updatedAtMs` column semantics - -The org actor's `taskSummaries.updatedAtMs` and the user actor's `userTaskFeed.bumpedAtMs` serve different purposes: -- `taskSummaries.updatedAtMs` — updated by task actor push. Reflects the last time the task's global state changed (any mutation, any user). Used for "All Repositories" / "All Tasks" views. -- `userTaskFeed.bumpedAtMs` — updated by bump/notify commands. Reflects the last time this specific user's attention was drawn to this task. Used for the per-user sidebar sort. - -Add doc comments on both columns clarifying the update source. - -### Unread semantics - -Each user has independent unread state. The existing `userTaskState` table tracks per-`(taskId, sessionId)` unread state. When the user clicks a session: -1. `userTaskState.unread` is set to 0 for that session -2. All `userNotifications` rows matching `(taskId, sessionId)` are marked `read = 1` - -These two unread systems must stay in sync via the `user.mark_read` queue command. - -## Better Auth: Actions, Not Queues - -All Better Auth adapter operations (verification CRUD, session/email/account index mutations, and user-actor auth record mutations) are exposed as **actions**, not queue commands. This is an intentional exception to the normal pattern of using queues for mutations. - -**Why:** The org actor's workflow queue is shared with GitHub sync, webhook processing, task mutations, and billing — 20+ queue names processed sequentially. During the OAuth callback, Better Auth needs to read/write verification records and upsert session/account indexes. If any long-running queue handler (e.g., a GitHub sync step) is ahead in the queue, auth operations time out (10s), `expectQueueResponse` throws a regular `Error`, and Better Auth's `parseState` catches it as a non-`StateError` → redirects to `?error=please_restart_the_process`. - -**Why it's safe:** Auth operations are simple SQLite reads/writes scoped to a single actor instance with no cross-actor side effects. They don't need workflow replay semantics or sequential ordering guarantees relative to other queue commands. - -**Rule:** Never move Better Auth operations back to queue commands. If new auth-related mutations are added, expose them as actions on the relevant actor. - -## Maintenance - -- Keep this file up to date whenever actor ownership, hierarchy, or lifecycle responsibilities change. -- If the real actor tree diverges from this document, update this document in the same change. -- When adding, removing, or renaming coordinator index tables, update the hierarchy diagram above in the same change. -- When adding a new coordinator index table in a schema file, add a doc comment identifying which child actor it indexes (pattern: `/** Coordinator index of {ChildActor} instances. ... */`). +@AGENTS.md diff --git a/frontend/CLAUDE.md b/frontend/CLAUDE.md index c4515dc0..43c994c2 100644 --- a/frontend/CLAUDE.md +++ b/frontend/CLAUDE.md @@ -1,4 +1 @@ -# Frontend Instructions - -- When the user asks for UI changes, capture screenshots of the updated UI after implementation and verification. -- At the end, offer to open those screenshots for the user and provide absolute filesystem paths to the screenshot files. +@AGENTS.md diff --git a/scripts/check-docs.ts b/scripts/check-docs.ts new file mode 100755 index 00000000..04b25ce7 --- /dev/null +++ b/scripts/check-docs.ts @@ -0,0 +1,32 @@ +#!/usr/bin/env tsx +// Docs law gate: em dashes and ACP mentions in user-facing docs (docs/**/*.mdx). +// Product output: violations on stderr with exit 1, otherwise "docs law: clean". +import { readdirSync, readFileSync } from "node:fs"; +import { join } from "node:path"; + +const isAcpPage = (path: string): boolean => /acp/i.test(path); + +const findings: string[] = []; + +function walk(dir: string): void { + for (const entry of readdirSync(dir, { withFileTypes: true })) { + const p = join(dir, entry.name); + if (entry.isDirectory()) { + walk(p); + } else if (entry.name.endsWith(".mdx")) { + const lines = readFileSync(p, "utf8").split("\n"); + lines.forEach((line, i) => { + if (line.includes("—")) findings.push(`${p}:${i + 1}: em dash in docs`); + if (!isAcpPage(p) && /\bACP\b/.test(line)) findings.push(`${p}:${i + 1}: ACP mention outside an ACP page`); + }); + } + } +} + +walk("docs"); + +if (findings.length > 0) { + console.error(`docs law violations (${findings.length}):\n${findings.join("\n")}`); + process.exit(1); +} +console.log("docs law: clean"); diff --git a/sdks/AGENTS.md b/sdks/AGENTS.md new file mode 100644 index 00000000..1a22f7aa --- /dev/null +++ b/sdks/AGENTS.md @@ -0,0 +1,36 @@ +# SDK Instructions + +## TypeScript SDK architecture + +- `sdks/acp-http-client`: protocol-pure ACP-over-HTTP client, no Sandbox-specific helpers. +- `sdks/typescript`: the `SandboxAgent` SDK combining ACP session operations with control-plane and filesystem helpers. +- `SandboxAgent` entry points: `SandboxAgent.connect(...)` (server mode) and `SandboxAgent.start(...)` (embedded mode, spawns the daemon). +- Stable session methods: `createSession`, `resumeSession`, `resumeOrCreateSession`, `destroySession`, `rawSendSessionMethod`, `onSessionEvent`, `setSessionMode`, `setSessionModel`, `setSessionThoughtLevel`, `setSessionConfigOption`, `getSessionConfigOptions`, `getSessionModes`, `respondPermission`, `rawRespondPermission`, `onPermissionRequest`. +- Cleanup is `sdk.dispose()`. + +## Naming conventions + +- `respond(id, reply)` for methods answering agent-initiated JSON-RPC requests (e.g. `respondPermission`). +- `raw` prefix for protocol-level escape hatches bypassing SDK abstractions (`rawRespondPermission`, `rawSend`). + +## React components (`sdks/react`) + +- Shared React UI goes here only when reusable outside the Inspector; when a pattern is shared between Inspector and Foundry, extract here rather than forking. +- Keep shared components unstyled: behavior in the package, styling in the consumer via `className`, slot `classNames`, render overrides, `data-*` hooks. +- Keep `docs/react-components.mdx` aligned with `sdks/react/src/index.ts`. + +## Effect v4 target state + +- All TS packages adopt Effect v4 via the workspace catalog (`effect: "catalog:"`). New modules use `Effect.gen`/`yield*`, `Context.Service` services, and Schema declarations in `*.schema.ts`. +- Gate: per-package `oxlint.config.ts` extending `@systemfsoftware/all` (effect plugins fire on Schema/service conventions), plus `pnpm typecheck`. +- Property-test pure cores with `@effect/vitest it.prop`. + +## Tests + +- Tests run against a real server/runtime over real HTTP with the real `mock` agent for determinism. +- Do not use fetch/transport mocks to simulate server functionality. +- Gate: `pnpm --dir sdks/typescript test` (also in CI). + +## Docs source of truth + +- TypeScript docs/examples ground in `sdks/typescript/src/client.ts`, `sdks/typescript/src/index.ts`, `sdks/acp-http-client/src/index.ts`; document only exported, implemented APIs. diff --git a/sdks/CLAUDE.md b/sdks/CLAUDE.md index a71eac4d..43c994c2 100644 --- a/sdks/CLAUDE.md +++ b/sdks/CLAUDE.md @@ -1,37 +1 @@ -# SDK Instructions - -## TypeScript SDK Architecture - -- TypeScript clients are split into: - - `acp-http-client`: protocol-pure ACP-over-HTTP (`/v1/acp`) with no Sandbox-specific HTTP helpers. - - `sandbox-agent`: `SandboxAgent` SDK wrapper that combines ACP session operations with Sandbox control-plane and filesystem helpers. -- `SandboxAgent` entry points are `SandboxAgent.connect(...)` and `SandboxAgent.start(...)`. -- Stable Sandbox session methods are `createSession`, `resumeSession`, `resumeOrCreateSession`, `destroySession`, `rawSendSessionMethod`, `onSessionEvent`, `setSessionMode`, `setSessionModel`, `setSessionThoughtLevel`, `setSessionConfigOption`, `getSessionConfigOptions`, `getSessionModes`, `respondPermission`, `rawRespondPermission`, and `onPermissionRequest`. -- `Session` helpers are `prompt(...)`, `rawSend(...)`, `onEvent(...)`, `setMode(...)`, `setModel(...)`, `setThoughtLevel(...)`, `setConfigOption(...)`, `getConfigOptions()`, `getModes()`, `respondPermission(...)`, `rawRespondPermission(...)`, and `onPermissionRequest(...)`. -- Cleanup is `sdk.dispose()`. - -### React Component Methodology - -- Shared React UI belongs in `sdks/react` only when it is reusable outside the Inspector. -- If the same UI pattern is shared between the Sandbox Agent Inspector and Foundry, prefer extracting it into `sdks/react` instead of maintaining parallel implementations. -- Keep shared components unstyled by default: behavior in the package, styling in the consumer via `className`, slot-level `classNames`, render overrides, and `data-*` hooks. -- Prefer extracting reusable pieces such as transcript, composer, and conversation surfaces. Keep Inspector-specific shells such as session selection, session headers, and control-plane actions in `frontend/packages/inspector/`. -- Document all shared React components in `docs/react-components.mdx`, and keep that page aligned with the exported surface in `sdks/react/src/index.ts`. - -### TypeScript SDK Naming Conventions - -- Use `respond(id, reply)` for SDK methods that reply to an agent-initiated request (e.g. `respondPermission`). This is the standard pattern for answering any inbound JSON-RPC request from the agent. -- Prefix raw/low-level escape hatches with `raw` (e.g. `rawRespondPermission`, `rawSend`). These accept protocol-level types directly and bypass SDK abstractions. - -### Docs Source Of Truth - -- For TypeScript docs/examples, source of truth is implementation in: - - `sdks/typescript/src/client.ts` - - `sdks/typescript/src/index.ts` - - `sdks/acp-http-client/src/index.ts` -- Do not document TypeScript APIs unless they are exported and implemented in those files. - -## Tests - -- TypeScript SDK tests should run against a real running server/runtime over real `/v1` HTTP APIs, typically using the real `mock` agent for deterministic behavior. -- Do not use Vitest fetch/transport mocks to simulate server functionality in TypeScript SDK tests. +@AGENTS.md diff --git a/server/ARCHITECTURE.md b/server/ARCHITECTURE.md index 4ac7de8a..c771744b 100644 --- a/server/ARCHITECTURE.md +++ b/server/ARCHITECTURE.md @@ -1,375 +1,56 @@ # Architecture -How the daemon, schemas, and agents fit together. - -Sandbox Agent SDK is built around a single daemon that runs inside the sandbox and exposes a universal HTTP API. Clients use the API (or the TypeScript SDK / CLI) to create sessions, send messages, and stream events. - -## Components - -- **Daemon**: Rust HTTP server that manages agent processes and streaming. -- **Universal schema**: Shared input/output types for messages and events. -- **SDKs & CLI**: Convenience wrappers around the HTTP API. - -## Agent Schema Pipeline - -The schema pipeline extracts type definitions from AI coding agents and converts them to a universal format. - -### Schema Extraction - -TypeScript extractors in `resources/agent-schemas/src/` pull schemas from each agent: - -| Agent | Source | Extractor | -|-------|--------|-----------| -| Claude | `claude --output-format json --json-schema` | `claude.ts` | -| Codex | `codex app-server generate-json-schema` | `codex.ts` | -| OpenCode | GitHub OpenAPI spec | `opencode.ts` | -| Amp | Scrapes ampcode.com docs | `amp.ts` | - -All extractors include fallback schemas for when CLIs or URLs are unavailable. - -**Output:** JSON schemas written to `resources/agent-schemas/artifacts/json-schema/` - -### Rust Type Generation - -The `server/packages/extracted-agent-schemas/` package generates Rust types at build time: - -- `build.rs` reads JSON schemas and uses the `typify` crate to generate Rust structs -- Generated code is written to `$OUT_DIR/{agent}.rs` -- Types are exposed via `include!()` macros in `src/lib.rs` - -``` -resources/agent-schemas/artifacts/json-schema/*.json - ↓ (build.rs + typify) -$OUT_DIR/{claude,codex,opencode,amp}.rs - ↓ (include!) -extracted_agent_schemas::{claude,codex,opencode,amp}::* -``` - -### Universal Schema - -The `server/packages/universal-agent-schema/` package defines agent-agnostic types: - -**Core types** (`src/lib.rs`): -- `UniversalEvent` - Wrapper with id, timestamp, session_id, agent, data -- `UniversalEventData` - Enum: Message, Started, Error, QuestionAsked, PermissionAsked, Unknown -- `UniversalMessage` - Parsed (role, parts, metadata) or Unparsed (raw JSON) -- `UniversalMessagePart` - Text, ToolCall, ToolResult, FunctionCall, FunctionResult, File, Image, Error, Unknown - -**Converters** (`src/agents/{claude,codex,opencode,amp}.rs`): -- Each agent has a converter module that transforms native events to universal format -- Conversions are best-effort; unparseable data preserved in `Unparsed` or `Unknown` variants - -## Session Management - -Sessions track agent conversations with in-memory state. - -### Session Model - -- **Session ID**: Client-provided primary session identifier. -- **Agent session ID**: Underlying ID from the agent (thread/session). This is surfaced in events but is not the primary key. - -### Storage - -Sessions are stored in an in-memory `HashMap` inside `SessionManager`: - -```rust -struct SessionManager { - sessions: Mutex>, - // ... -} -``` - -There is no disk persistence. Sessions are ephemeral and lost on server restart. - -### SessionState - -Each session tracks: - -| Field | Purpose | -|-------|---------| -| `session_id` | Client-provided identifier | -| `agent` | Agent type (Claude, Codex, OpenCode, Amp) | -| `agent_mode` | Operating mode (build, plan, custom) | -| `permission_mode` | Permission handling (default, plan, bypass) | -| `model` | Optional model override | -| `events: Vec` | Full event history | -| `pending_questions` | Question IDs awaiting reply | -| `pending_permissions` | Permission IDs awaiting reply | -| `broadcaster` | Tokio broadcast channel for SSE streaming | -| `ended` | Whether agent process has terminated | - -### Lifecycle - -``` -POST /v1/acp/{serverId}?agent=... initialize ACP server, auto-install agent - ↓ -POST /v1/acp/{serverId} session/new -POST /v1/acp/{serverId} session/prompt - ↓ -GET /v1/acp/{serverId} Subscribe to ACP SSE stream - ↓ -JSON-RPC response envelopes Answer questions / reply to permissions - ↓ -DELETE /v1/acp/{serverId} Close ACP server -``` - -### Event Streaming - -- ACP envelopes are stored in memory per server and assigned a monotonically increasing SSE `id`. -- `GET /v1/acp/{serverId}` replays buffered envelopes and then streams live updates. -- Clients continue turns by POSTing ACP JSON-RPC requests to the same server id. - -When a message is sent: - -1. `send_message()` spawns the agent CLI as a subprocess -2. `consume_spawn()` reads stdout/stderr line by line -3. Each JSON line is parsed and converted via `parse_agent_line()` -4. Events are recorded via `record_event()` which: - - Assigns incrementing event ID - - Appends to `events` vector - - Broadcasts to SSE subscribers - -## Agent Execution - -Each agent has a different execution model and communication pattern. There are two main architectural patterns: - -### Architecture Patterns - -**Subprocess Model (Claude, Amp):** -- New process spawned per message/turn -- Process terminates after turn completes -- Multi-turn via CLI resume flags (`--resume`, `--continue`) -- Simple but has process spawn overhead - -**Client/Server Model (OpenCode, Codex):** -- Single long-running server process -- Multiple sessions/threads multiplexed via RPC -- Multi-turn via server-side thread persistence -- More efficient for repeated interactions - -### Overview - -| Agent | Architecture | Binary Source | Multi-Turn Method | -|-------|--------------|---------------|-------------------| -| Claude Code | Subprocess (per-turn) | GCS (Anthropic) | `--resume` flag | -| Codex | **Shared Server (JSON-RPC)** | GitHub releases | **Thread persistence** | -| OpenCode | HTTP Server (SSE) | GitHub releases | Server-side sessions | -| Amp | Subprocess (per-turn) | GCS (Amp) | `--continue` flag | - -### Claude Code - -Spawned as a subprocess with JSONL streaming: - -```bash -claude --print --output-format stream-json --verbose \ - [--model MODEL] [--resume SESSION_ID] \ - [--permission-mode plan | --dangerously-skip-permissions] \ - PROMPT -``` - -- Streams JSON events to stdout, one per line -- Supports session resumption via `--resume` -- Permission modes: `--permission-mode plan` for approval workflow, `--dangerously-skip-permissions` for bypass - -### Codex - -Uses a **shared app-server process** that handles multiple sessions via JSON-RPC over stdio: - -```bash -codex app-server -``` - -**Daemon flow:** -1. First Codex session triggers `codex app-server` spawn -2. Performs `initialize` / `initialized` handshake -3. Each session creation sends `thread/start` → receives `thread_id` -4. Messages sent via `turn/start` with `thread_id` -5. Notifications routed back to session by `thread_id` - -**Key characteristics:** -- Single process handles all Codex sessions -- JSON-RPC over stdio (JSONL format) -- Thread IDs map to daemon session IDs -- Approval requests arrive as server-to-client JSON-RPC requests -- Process lifetime matches daemon lifetime (not per-turn) - -### OpenCode - -Unique architecture - runs as a **persistent HTTP server** rather than per-message subprocess: - -```bash -opencode serve --port {4200-4300} -``` - -Then communicates via HTTP endpoints: - -| Endpoint | Purpose | -|----------|---------| -| `POST /session` | Create new session | -| `POST /session/{id}/prompt` | Send message | -| `GET /event/subscribe` | SSE event stream | -| `POST /question/reply` | Answer HITL question | -| `POST /permission/reply` | Grant/deny permission | - -The server is started once and reused across sessions. Events are received via Server-Sent Events (SSE) subscription. - -### Amp - -Spawned as a subprocess with dynamic flag detection: - -```bash -amp [--execute|--print] [--output-format stream-json] \ - [--model MODEL] [--continue SESSION_ID] \ - [--dangerously-skip-permissions] PROMPT -``` - -- **Dynamic flag detection**: Probes `--help` output to determine which flags the installed version supports -- **Fallback strategy**: If execution fails, retries with progressively simpler flag combinations -- Streams JSON events to stdout -- Supports session continuation via `--continue` - -### Communication Patterns - -**Per-turn subprocess agents (Claude, Amp):** -1. Agent CLI spawned with appropriate flags -2. Stdout/stderr read line-by-line -3. Each line parsed as JSON -4. Events converted via `parse_agent_line()` → agent-specific converter -5. Universal events recorded and broadcast to SSE subscribers -6. Process terminated on turn completion - -**Shared stdio server agent (Codex):** -1. Single `codex app-server` process started on first session -2. `initialize`/`initialized` handshake performed once -3. New sessions send `thread/start`, receive `thread_id` -4. Messages sent via `turn/start` with `thread_id` -5. Notifications read from stdout, routed by `thread_id` -6. Process persists across sessions and turns - -**HTTP server agent (OpenCode):** -1. Server started on available port (if not running) -2. Session created via HTTP POST -3. Prompts sent via HTTP POST -4. Events received via SSE subscription -5. HITL responses forwarded via HTTP POST - -### Credential Handling - -All agents receive API keys via environment variables: - -| Agent | Environment Variables | -|-------|----------------------| -| Claude | `ANTHROPIC_API_KEY`, `CLAUDE_API_KEY`, `CLAUDE_CODE_OAUTH_TOKEN`, `ANTHROPIC_AUTH_TOKEN` | -| Codex | `OPENAI_API_KEY`, `CODEX_API_KEY` | -| OpenCode | `OPENAI_API_KEY` | -| Amp | `ANTHROPIC_API_KEY` | - -## Human-in-the-Loop - -Questions and permission prompts are normalized into the universal schema: - -- Question events surface as `questionAsked` with selectable options. -- Permission events surface as `permissionAsked` with `reply: once | always | reject`. -- Claude plan approval is normalized into a question event (approve/reject). - -## SDK Modes - -The TypeScript SDK supports two connection modes. - -### Embedded Mode - -Defined in `sdks/typescript/src/spawn.ts`: - -1. **Binary resolution**: Checks `SANDBOX_AGENT_BIN` env, then platform-specific npm package, then `PATH` -2. **Port selection**: Uses provided port or finds a free one via `net.createServer()` -3. **Token generation**: Uses provided token or generates random 24-byte hex string -4. **Spawn**: Launches `sandbox-agent server --host --port --token ` -5. **Health wait**: Polls `GET /v1/health` until server is ready (up to 15s timeout) -6. **Cleanup**: On dispose, sends SIGTERM then SIGKILL if needed; also registers process exit handlers - -```typescript -const handle = await spawnSandboxAgent({ log: "inherit" }); -// handle.baseUrl = "http://127.0.0.1:" -// handle.token = "" -// handle.dispose() to cleanup -``` - -### Server Mode - -Defined in `sdks/typescript/src/client.ts`: - -- Direct HTTP client to a remote `sandbox-agent` server -- Uses provided `baseUrl` and optional `token` -- No subprocess management - -```typescript -const client = await SandboxAgent.connect({ - baseUrl: "http://remote-server:8080", - token: "secret", -}); -``` - -### Auto-Detection - -Sandbox Agent provides two factory methods: - -```typescript -// Connect to existing server -const client = await SandboxAgent.connect({ - baseUrl: "http://remote:8080", -}); - -// Start embedded subprocess -const client = await SandboxAgent.start(); - -// With options -const client = await SandboxAgent.start({ - spawn: { port: 9000 }, -}); -``` - -The `spawn` option can be: -- `true` / `false` - Enable/disable embedded mode -- `SandboxAgentSpawnOptions` - Fine-grained control over host, port, token, binary path, timeout, logging - -## Authentication - -The daemon uses a **global token** configured at startup. All HTTP and CLI operations reuse the same token and are validated against the `Authorization` header (`Bearer` or `Token`). - -## Key Files - -| Component | Path | -|-----------|------| -| Agent spawn/install | `server/packages/agent-management/src/agents.rs` | -| Session routing | `server/packages/sandbox-agent/src/router.rs` | -| Event converters | `server/packages/universal-agent-schema/src/agents/*.rs` | -| Schema extractors | `resources/agent-schemas/src/*.ts` | -| TypeScript SDK | `sdks/typescript/src/` | - ---- - -# Agent Compatibility - -Supported agents, install methods, and streaming formats. - -## Compatibility Matrix - -| Agent | Provider | Binary | Install method | Session ID | Streaming format | -|-------|----------|--------|----------------|------------|------------------| -| Claude Code | Anthropic | `claude` | curl raw binary from GCS | `session_id` | JSONL via stdout | -| Codex | OpenAI | `codex` | curl tarball from GitHub releases | `thread_id` | JSON-RPC over stdio | -| OpenCode | Multi-provider | `opencode` | curl tarball from GitHub releases | `session_id` | SSE or JSONL | -| Amp | Sourcegraph | `amp` | curl raw binary from GCS | `session_id` | JSONL via stdout | -| Mock | Built-in | — | bundled | `mock-*` | daemon-generated | - -## Agent Modes - -- **OpenCode**: discovered via the server API. -- **Claude Code / Codex / Amp**: hardcoded modes (typically `build`, `plan`, or `custom`). - -## Capability Notes - -- **Questions / permissions**: OpenCode natively supports these workflows. Claude plan approval is normalized into a question event (tests do not currently exercise Claude question/permission flows). -- **Streaming**: all agents stream events; OpenCode uses SSE, Codex uses JSON-RPC over stdio, others use JSONL. Codex is currently normalized to thread/turn starts plus user/assistant completed items (deltas and tool/reasoning items are not emitted yet). -- **User messages**: Claude CLI output does not include explicit user-message events in our snapshots, so only assistant messages are surfaced for Claude today. -- **Files and images**: normalized via `UniversalMessagePart` with `File` and `Image` parts. +last-validated-against-decisions: 2026-08-29 (no ADR log yet; first records expected with the Effect migration, see root `STRATEGY.md`) + +The server is a single Rust daemon (`sandbox-agent`) that runs inside a sandbox and exposes one HTTP surface for controlling coding agents: Claude Code, Codex, OpenCode, Cursor, Amp, Pi. Clients speak ACP JSON-RPC over streamable HTTP; the daemon hosts agent processes and translates their behavior onto that surface. + +## Component map + +```mermaid +flowchart LR + Client["TS SDK / CLI / Inspector / Foundry"] -->|ACP JSON-RPC over HTTP| Daemon + subgraph Daemon["sandbox-agent daemon (axum)"] + Router["router.rs"] + ACP["acp_proxy_runtime.rs"] + Proc["process_runtime.rs"] + Desk["desktop_runtime / desktop_streaming / desktop_recording"] + OE["opencode_compat.rs"] + UE["universal_events.rs"] + end + Daemon -->|spawn + stdio JSON-RPC| Agent["Agent process (claude, codex, opencode, amp, pi)"] + Client -->|fs / processes / desktop REST| Daemon +``` + +| Component | Location | Role | +|---|---|---| +| HTTP surface | `server/packages/sandbox-agent/src/router.rs` | Every route: ACP, control plane, fs, processes, desktop, OpenCode compat | +| ACP proxy runtime | `server/packages/sandbox-agent/src/acp_proxy_runtime.rs` | Per-server-id session state, envelope buffering, SSE replay | +| Agent install | `server/packages/agent-management/` | Native and ACP agent-process install, lazy install on first use | +| Desktop services | `server/packages/sandbox-agent/src/desktop_runtime.rs`, `desktop_streaming.rs`, `desktop_recording.rs` | VM desktop control, WebRTC signaling, recordings | +| Process services | `server/packages/sandbox-agent/src/process_runtime.rs` | Long-lived processes, terminals, log streaming | +| OpenCode compat | `server/packages/sandbox-agent/src/opencode_compat.rs`, `server/packages/opencode-adapter/`, `server/packages/opencode-server-manager/` | `/opencode/*` compatibility surface | +| Universal events | `server/packages/sandbox-agent/src/universal_events.rs` | Normalized event schema; unparseable data preserved via Unknown variants | +| Credentials | `server/packages/agent-credentials/` | Agent API-key extraction into environment | +| Errors | `server/packages/error/` | Shared error type, `application/problem+json` mapping | +| OpenAPI | `server/packages/openapi-gen/` | Generates `docs/openapi.json` from utoipa annotations | +| Inspector UI | `frontend/packages/inspector/`, served by `server/packages/sandbox-agent/src/ui.rs` at `/ui/` | Session/event debugging UI | +| Gigacode CLI | `gigacode/` | OpenCode TUI client for any agent | + +## Session data flow + +1. Client creates an ACP server (`/v1/acp`): the daemon resolves and lazily installs the agent, spawns its process, and performs the ACP initialize handshake. +2. Prompts travel as ACP JSON-RPC envelopes (`POST /v1/acp/{server_id}`); the daemon bridges them to the agent process. +3. Agent responses are normalized through `universal_events.rs` into ACP envelopes, buffered per server with monotonically increasing SSE ids. +4. Clients consume state via `GET /v1/acp/{server_id}` (SSE: buffered replay, then live) and close with `DELETE`. + +## Key boundaries + +- ACP carries agent/session behavior; plain HTTP carries host/runtime services (fs, terminals, processes, desktop). Rationale: binary transfer volume and cross-agent consistency exceed what ACP envelopes are suited to stream, and ACP-native `fs/*`/`terminal/*` client implementations are inconsistent across agents. +- Sessions live in daemon memory (per ACP server id). Persistence is the consumer's job: stream events out to Postgres, ClickHouse, or Rivet. +- Each agent has an adapter surface; the daemon never embeds agent-specific protocol details outside its adapter path. + +For the SDK-side view (embedded vs server mode), see `sdks/AGENTS.md` and `docs/sdk-overview.mdx`. + +## Decision correspondence + +No `docs/decisions/` log exists yet. First ADRs are expected with the Effect-actor migration of Foundry and the banner rename; this file re-validates when they land. diff --git a/server/CLAUDE.md b/server/CLAUDE.md index 88f4f0ad..43c994c2 100644 --- a/server/CLAUDE.md +++ b/server/CLAUDE.md @@ -1,93 +1 @@ -# Server Instructions - -## ACP v1 Baseline - -- v1 is ACP-native. -- `/v1/*` is removed and returns `410 Gone` (`application/problem+json`). -- `/opencode/*` is disabled during ACP core phases and returns `503`. -- Prompt/session traffic is ACP JSON-RPC over streamable HTTP on `/v1/rpc`: - - `POST /v1/rpc` - - `GET /v1/rpc` (SSE) - - `DELETE /v1/rpc` -- Control-plane endpoints: - - `GET /v1/health` - - `GET /v1/agents` - - `POST /v1/agents/{agent}/install` -- Binary filesystem transfer endpoints (intentionally HTTP, not ACP extension methods): - - `GET /v1/fs/file` - - `PUT /v1/fs/file` - - `POST /v1/fs/upload-batch` -- Sandbox Agent ACP extension method naming: - - Custom ACP methods use `_sandboxagent/...` (not `_sandboxagent/v1/...`). - - Session detach method is `_sandboxagent/session/detach`. - -## API Scope - -- ACP is the primary protocol for agent/session behavior and all functionality that talks directly to the agent. -- ACP extensions may be used for gaps (for example `skills`, `models`, and related metadata), but the default is that agent-facing behavior is implemented by the agent through ACP. -- Custom HTTP APIs are for non-agent/session platform services (for example filesystem, terminals, and other host/runtime capabilities). -- Filesystem and terminal APIs remain Sandbox Agent-specific HTTP contracts and are not ACP. - - Do not make Sandbox Agent core flows depend on ACP client implementations of `fs/*` or `terminal/*`; in practice those client-side capabilities are often incomplete or inconsistent. - - ACP-native filesystem and terminal methods are also too limited for Sandbox Agent host/runtime needs, so prefer the native HTTP APIs for richer behavior. -- Keep `GET /v1/fs/file`, `PUT /v1/fs/file`, and `POST /v1/fs/upload-batch` on HTTP: - - These are Sandbox Agent host/runtime operations with cross-agent-consistent behavior. - - They may involve very large binary transfers that ACP JSON-RPC envelopes are not suited to stream. - - This is intentionally separate from ACP native `fs/read_text_file` and `fs/write_text_file`. - - ACP extension variants may exist in parallel, but SDK defaults should prefer HTTP for these binary transfer operations. - -## Architecture - -- HTTP contract and problem/error mapping: `server/packages/sandbox-agent/src/router.rs` -- ACP proxy runtime: `server/packages/sandbox-agent/src/acp_proxy_runtime.rs` -- ACP client runtime and agent process bridge: `server/packages/sandbox-agent/src/acp_runtime/mod.rs` -- Agent install logic (native + ACP agent process + lazy install): `server/packages/agent-management/` -- Inspector UI served at `/ui/` and bound to ACP over HTTP from `frontend/packages/inspector/` - -## API Contract Rules - -- Every `#[utoipa::path(...)]` handler needs a summary line + description lines in its doc comment. -- Every `responses(...)` entry must include `description`. -- Regenerate `docs/openapi.json` after endpoint contract changes. -- Keep CLI and HTTP endpoint behavior aligned (`docs/cli.mdx`). - -## ACP Protocol Compliance - -- Before adding any new ACP method, property, or config option category to the SDK, verify it exists in the ACP spec at `https://agentclientprotocol.com/llms-full.txt`. -- Valid `SessionConfigOptionCategory` values are: `mode`, `model`, `thought_level`, `other`, or custom categories prefixed with `_` (e.g. `_permission_mode`). -- Do not invent ACP properties or categories (e.g. `permission_mode` is not a valid ACP category — use `_permission_mode` if it's a custom extension, or use existing ACP mechanisms like `session/set_mode`). -- `NewSessionRequest` only has `_meta`, `cwd`, and `mcpServers`. Do not add non-ACP fields to it. -- Sandbox Agent SDK abstractions (like `SessionCreateRequest`) may add convenience properties, but must clearly map to real ACP methods internally and not send fabricated fields over the wire. - -## Source Documents - -- ACP protocol specification (full LLM-readable reference): `https://agentclientprotocol.com/llms-full.txt` -- `~/misc/acp-docs/schema/schema.json` -- `~/misc/acp-docs/schema/meta.json` -- `research/acp/spec.md` -- `research/acp/v1-schema-to-acp-mapping.md` -- `research/acp/friction.md` -- `research/acp/todo.md` - -## Tests - -Primary v1 integration coverage: -- `server/packages/sandbox-agent/tests/v1_api.rs` -- `server/packages/sandbox-agent/tests/v1_agent_process_matrix.rs` - -Run: -```bash -cargo test -p sandbox-agent --test v1_api -cargo test -p sandbox-agent --test v1_agent_process_matrix -``` - -## Migration Docs Sync - -- Keep `research/acp/spec.md` as the source spec. -- Update `research/acp/todo.md` when scope/status changes. -- Log blockers/decisions in `research/acp/friction.md`. - -## Docker Examples (Dev Testing) - -- When manually testing bleeding-edge (unreleased) versions of sandbox-agent in `examples/`, use `SANDBOX_AGENT_DEV=1` with the Docker-based examples. -- This triggers a local build of `docker/runtime/Dockerfile.full` which builds the server binary from local source and packages it into the Docker image. -- Example: `SANDBOX_AGENT_DEV=1 pnpm --filter @sandbox-agent/example-mcp start` +@AGENTS.md From 753a5dcdbd9b4e367ee67eb0acbb1c9cbef16a37 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 07:44:05 +0000 Subject: [PATCH 02/12] chore(harness): materialize symlink AGENTS.md files and deno-native docs gate Root, server, frontend, and foundry AGENTS.md were git symlinks to CLAUDE.md (30a66a7); writing through them lost the new law. Replace with real files. Convert scripts/check-docs.ts to deno with deno.jsonc and deno.lock in scripts/. --- AGENTS.md | 59 ++++++++++++++++++++++++++++++++++++++++++- foundry/AGENTS.md | 56 +++++++++++++++++++++++++++++++++++++++- frontend/AGENTS.md | 5 +++- scripts/check-docs.ts | 34 ++++++++++--------------- scripts/deno.jsonc | 6 +++++ scripts/deno.lock | 30 ++++++++++++++++++++++ server/AGENTS.md | 40 ++++++++++++++++++++++++++++- 7 files changed, 206 insertions(+), 24 deletions(-) mode change 120000 => 100644 AGENTS.md mode change 120000 => 100644 foundry/AGENTS.md mode change 120000 => 100644 frontend/AGENTS.md create mode 100644 scripts/deno.jsonc create mode 100644 scripts/deno.lock mode change 120000 => 100644 server/AGENTS.md diff --git a/AGENTS.md b/AGENTS.md deleted file mode 120000 index 681311eb..00000000 --- a/AGENTS.md +++ /dev/null @@ -1 +0,0 @@ -CLAUDE.md \ No newline at end of file diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 00000000..c3228e43 --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,58 @@ +# Instructions + +Single source of agent law for this repository. Every `CLAUDE.md` in this repo is a one-line pointer to the `AGENTS.md` beside it; never duplicate content between them. + +## Naming + +- Product display name: **Systemfsoftware**. Components keep their names: `sandbox-agent`, `foundry`, `gigacode`. +- Org: `systemfsoftware`. npm scope: `@systemfsoftware/*`. Crates keep component names (`sandbox-agent`, ...). +- Domain: `sandboxagent.systemfsoftware.com`. Do not add new `sandboxagent.dev` references (migration in progress). +- ACP extension method prefix stays `_sandboxagent/...` for wire compatibility; extension namespace string is `sandboxagent.systemfsoftware.com`. +- Foundry is the canonical name for the `foundry/` product tree; no pre-Foundry naming. +- If the user asks to "push", treat that as permission to commit and push all current workspace changes, not a hand-picked subset, unless they scope the push. + +## Verification gates + +Gates are defined once, in `package.json` scripts and `turbo.json`; workflows and leaves invoke them, never re-enumerate steps. Definition of done: `pnpm check:ci` green plus the tests below for touched surfaces. + +- Gate: `pnpm typecheck` - TypeScript across all workspaces. +- Gate: `pnpm check:docs` - docs law (see Docs). +- Gate: `pnpm check:ci` = typecheck + `check:docs`; CI runs exactly this. +- Rust gates: `cargo test -p sandbox-agent-agent-management`, `cargo test -p sandbox-agent --test v1_api`, `--test v1_agent_process_matrix`, `--lib`. +- SDK gate: `pnpm --dir sdks/typescript test`. +- Merge, deploy, release, force-push, and credentials are human-approved; ask first. + +## Lint ratchet (end state, not yet wired) + +- End state: every package lints under oxlint with a per-package `oxlint.config.ts` extending `@systemfsoftware/all`; `pnpm lint` joins `check:ci`. +- Today: lint is not wired (deps not installed; ~6,700 findings measured 2026-08-29 with oxlint 1.80.0 + `@systemfsoftware/all` 1.0.1 against the unliterated tree). Adoption is per package: add its `oxlint.config.ts`, fix its findings, then add `lint` to `check:ci` in the same commit. + +## Effect-first TypeScript + +- All TS surfaces adopt Effect-TS v4. Versions pin via the `catalogs.effect` block in `pnpm-workspace.yaml` when the first package adopts; the reference monorepo pins ^4.0.0-rc.112. +- Idioms (reference: `github.com/systemfsoftware/systemfsoftware`, e.g. `packages/core/effect/daemon-spec/src/`): `Effect.gen`/`yield*`, `Context.Service` class services, `Layer.succeed`/`Layer.effect`, Schema declarations in `*.schema.ts` (`Schema.TaggedError`, `decodeUnknownExit`), property tests via `@effect/vitest it.prop`. +- Foundry's actor runtime migrates from RivetKit to Effect's actor model (see `STRATEGY.md` Tracks). Until a module migrates, the law in `foundry/AGENTS.md` and `foundry/packages/backend/AGENTS.md` still binds; delete each rule as its module migrates. +- Gate: the oxlint effect plugins shipped in `@systemfsoftware/all` (activated per package at adoption), plus `pnpm typecheck`. + +## Docs law + +- HTTP truth: `server/packages/sandbox-agent/src/router.rs`. CLI truth: `server/packages/sandbox-agent/src/cli.rs`. Document implemented surface only. +- Never mention "ACP" in `docs/**/*.mdx` except pages that are about ACP; never expose protocol method names in non-ACP docs; document the SDK surface, not the protocol implementation; no em dashes in docs. + Gate: `pnpm check:docs` (`scripts/check-docs.ts`). +- Docs styling is owned by the `@rivet-dev/docs-theme` repo consumed by `frontend/packages/website`; never restyle docs here. This site owns only content and `frontend/packages/website/docs.config.mjs`. +- Keep `docs/cli.mdx` aligned with CLI behavior. Regenerate `docs/openapi.json` after HTTP contract changes. Gate: `pnpm generate`. +- `docs/agent-capabilities.mdx`: when its "Last updated" is >2 weeks old or `fallback_config_options` changes, re-run `cd scripts/agent-configs && npx tsx dump.ts` and update it. Static docs list all known models regardless of subscription gating. + +## Sync obligations + +- CLI subcommands and HTTP endpoints stay in sync with each other. +- Common-software trio stays in sync: `docs/common-software.mdx`, `docker/test-common-software/Dockerfile`, `server/packages/sandbox-agent/tests/common_software.rs`. Gate: `cargo test -p sandbox-agent --test common_software`. +- Install-version channel policy: `sandbox-agent` installs use pinned minor `0.N.x` (curl URLs, `sandbox-agent` / `@sandbox-agent/cli`); gigacode installs use `latest` (`@sandbox-agent/gigacode`, `gigacode-install.*` promotion); a `latest` release updates `latest` and promotes sandbox-agent to the matching `0.N.x`. Affected files: `README.md`, `docs/acp-http-client.mdx`, `docs/cli.mdx`, `docs/quickstart.mdx`, `docs/sdk-overview.mdx`, `docs/react-components.mdx`, `docs/session-persistence.mdx`, `docs/deploy/*.mdx`, `frontend/packages/website/src/components/GetStarted.tsx`, `.claude/commands/post-release-testing.md`, `examples/cloudflare/Dockerfile`, `examples/daytona/src/index.ts`, `examples/shared/src/docker.ts`, `examples/{docker,e2b,vercel}/src/index.ts`, `scripts/release/{main,promote-artifacts,sdk}.ts`. (Gate earned after the rename lands: a `git grep` count over this list.) + +## Docker test images + +- Docker-backed tests build `docker/test-agent/Dockerfile` in-process and cache the tag in memory only (OnceLock / module-level variable); no cross-process image-build scripts. + +## Shared component rule + +- React UI shared between Inspector and Foundry belongs in `sdks/react`, unstyled, behavior-only; consumers style via `className`/`classNames`/`data-*`. Changes there must be verified in both consumers. diff --git a/foundry/AGENTS.md b/foundry/AGENTS.md deleted file mode 120000 index 681311eb..00000000 --- a/foundry/AGENTS.md +++ /dev/null @@ -1 +0,0 @@ -CLAUDE.md \ No newline at end of file diff --git a/foundry/AGENTS.md b/foundry/AGENTS.md new file mode 100644 index 00000000..3ea3f251 --- /dev/null +++ b/foundry/AGENTS.md @@ -0,0 +1,55 @@ +# Foundry Instructions + +Product tree `foundry/`: the Foundry workspace app (frontend, backend, client, shared, desktop; `packages/cli` disabled). The backend actor runtime is migrating from RivetKit to Effect's actor model (root `STRATEGY.md`, Tracks). Until a module migrates, `foundry/packages/backend/AGENTS.md` binds; delete each rule there as its module migrates. + +## Language and runtime + +- TypeScript everywhere; never add `.js` / `.mjs` / `.cjs` sources; migrate old JavaScript on touch. Gate: `pnpm lint`. +- Bun is the runtime for CLI/backend execution and process spawning; no Node-compat fallbacks for OpenTUI/runtime execution. + +## Commands + +- Validation, always run and fix after changes: `pnpm -w typecheck && pnpm -w build && pnpm -w test`. +- Full dev stack (real backend + frontend; frontend 4173, backend 7741): `just foundry-dev`. Mock frontend (4174): `just foundry-mock`. The two stacks can run simultaneously. +- The backend does not hot reload (Bun `--hot` re-binds to a different port and breaks clients); after backend changes restart with `just foundry-dev-down && just foundry-dev`. +- Restart the matching stack before declaring work done. + +## Dev environment + +- `compose.dev.yaml` loads `foundry/.env`; canonical source `~/misc/the-foundry.env` (`cp ~/misc/the-foundry.env foundry/.env`). `foundry/.env` is gitignored; never commit it. +- Local sandboxes use `rivetdev/sandbox-agent:foundry-base-latest`; build/publish via `./foundry/scripts/publish-foundry-base.sh`, override with `HF_LOCAL_SANDBOX_IMAGE`. + +## Production access + +- Two GitHub entities: an OAuth App (sign-in) and a GitHub App (webhooks, installation tokens, repo sync). The webhook secret must match `GITHUB_WEBHOOK_SECRET` exactly; a mismatch surfaces as silent 500s on delivery. +- Production deploys go through `git push` to the deployment branch; never `railway up`. Logs: `railway logs --deployment --lines 200` (relink if needed: project `sandbox-agent`, environment `production`, service `foundry-api`). + +## Frontend + client boundary + +- `rivetkit` imports stay inside `packages/client` (until the Effect migration); all backend access goes through `packages/client`; no direct backend fetches outside black-box E2E. +- Push-based UI: no polling (`refetchInterval`), no manual-refresh buttons, no empty notification events; every mutation broadcasts the new absolute state of the changed entity. + +## UI system + +- BaseUI + Styletron with the Foundry token layer; prefer composition over custom primitives; ask before diverging from BaseUI. +- Styletron: never mix CSS shorthand and longhand properties in the same style object (or in a base component consumers override via `$style`), including nested pseudo-selectors; use longhand when either side may be overridden; use `backgroundColor: "transparent"` for resets. + +## React hook dependency safety + +- Never use unstable references as hook dependencies: `?? []`, `?? {}`, and inline `.map()` / `.filter()` create fresh references every render and loop effects. Depend on the raw value and apply fallbacks inside the effect body, or memoize a stable key. + +## Error handling + +- Fail fast with actionable context; never bare `catch {}` - every catch at minimum logs (`logActorWarning` / `console.warn`); intentional degradation still logs. Gate: `pnpm lint` correctness rules plus review. + +## Testing policy + +- No `vi.mock` / `vi.spyOn` / `vi.fn`: external I/O goes through driver interfaces with test implementations passed via the runtime context. Gate: `@systemfsoftware/all` test-hygiene plugin plus review. +- Integration tests use `setupTest()` from `rivetkit/test`, gated behind `HF_ENABLE_ACTOR_INTEGRATION_TESTS=1`. +- E2E runs against the compose dev backend over HTTP with real GitHub repos (default: `rivet-dev/sandbox-agent-testing`); secrets come from environment variables, never the repo. +- Client E2E in `packages/client/test` is the product source of truth; remove unmaintained browser E2E suites rather than keep them red. + +## Friction and audit + +- Friction logs: `foundry/research/friction/{rivet,sandbox-agent,sandboxes,general}.mdx`; each entry carries date, commit SHA (or `uncommitted`), what you were implementing, the friction, and the attempted fix with outcome. +- Every new task/workspace command ships with a matching audit-log event. diff --git a/frontend/AGENTS.md b/frontend/AGENTS.md deleted file mode 120000 index 681311eb..00000000 --- a/frontend/AGENTS.md +++ /dev/null @@ -1 +0,0 @@ -CLAUDE.md \ No newline at end of file diff --git a/frontend/AGENTS.md b/frontend/AGENTS.md new file mode 100644 index 00000000..1b5f39d4 --- /dev/null +++ b/frontend/AGENTS.md @@ -0,0 +1,4 @@ +# Frontend Instructions + +- When the user asks for UI changes, capture screenshots of the updated UI after implementation and verification, then offer to open them and provide absolute filesystem paths. +- Shared components live in `sdks/react` (see `sdks/AGENTS.md`); Inspector-specific shells stay in `packages/inspector/`. diff --git a/scripts/check-docs.ts b/scripts/check-docs.ts index 04b25ce7..bf0cee01 100755 --- a/scripts/check-docs.ts +++ b/scripts/check-docs.ts @@ -1,32 +1,26 @@ -#!/usr/bin/env tsx -// Docs law gate: em dashes and ACP mentions in user-facing docs (docs/**/*.mdx). -// Product output: violations on stderr with exit 1, otherwise "docs law: clean". -import { readdirSync, readFileSync } from "node:fs"; -import { join } from "node:path"; +#!/usr/bin/env -S deno run +import { walk } from "@std/fs/walk"; +const ROOT = new URL("../docs", import.meta.url).pathname; const isAcpPage = (path: string): boolean => /acp/i.test(path); const findings: string[] = []; -function walk(dir: string): void { - for (const entry of readdirSync(dir, { withFileTypes: true })) { - const p = join(dir, entry.name); - if (entry.isDirectory()) { - walk(p); - } else if (entry.name.endsWith(".mdx")) { - const lines = readFileSync(p, "utf8").split("\n"); - lines.forEach((line, i) => { - if (line.includes("—")) findings.push(`${p}:${i + 1}: em dash in docs`); - if (!isAcpPage(p) && /\bACP\b/.test(line)) findings.push(`${p}:${i + 1}: ACP mention outside an ACP page`); - }); +for await (const entry of walk(ROOT, { includeDirs: false, exts: [".mdx"] })) { + const text = await Deno.readTextFile(entry.path); + const lines = text.split("\n"); + lines.forEach((line, i) => { + if (line.includes("—")) { + findings.push(`${entry.path}:${i + 1}: em dash in docs`); } - } + if (!isAcpPage(entry.path) && /\bACP\b/.test(line)) { + findings.push(`${entry.path}:${i + 1}: ACP mention outside an ACP page`); + } + }); } -walk("docs"); - if (findings.length > 0) { console.error(`docs law violations (${findings.length}):\n${findings.join("\n")}`); - process.exit(1); + Deno.exit(1); } console.log("docs law: clean"); diff --git a/scripts/deno.jsonc b/scripts/deno.jsonc new file mode 100644 index 00000000..3187d987 --- /dev/null +++ b/scripts/deno.jsonc @@ -0,0 +1,6 @@ +{ + "lock": true, + "imports": { + "@std/fs": "jsr:@std/fs@^1.0.20" + } +} diff --git a/scripts/deno.lock b/scripts/deno.lock new file mode 100644 index 00000000..c4c2227d --- /dev/null +++ b/scripts/deno.lock @@ -0,0 +1,30 @@ +{ + "version": "5", + "specifiers": { + "jsr:@std/fs@^1.0.20": "1.0.24", + "jsr:@std/internal@^1.0.14": "1.0.14", + "jsr:@std/path@^1.1.5": "1.1.6" + }, + "jsr": { + "@std/fs@1.0.24": { + "integrity": "f3061b45b81673a2bece689da041df32d174be064c89eb6397fb5718d3fb7877", + "dependencies": [ + "jsr:@std/path" + ] + }, + "@std/internal@1.0.14": { + "integrity": "291516b3d4c35024d6ffbc0a9df5bf4c64116e05b50012cf846710152d2ffdf7" + }, + "@std/path@1.1.6": { + "integrity": "c68485c2a4dfbb5ae3cc74fae4e8c4e5d874cf8a8ed12927917235c758b46cbe", + "dependencies": [ + "jsr:@std/internal" + ] + } + }, + "workspace": { + "dependencies": [ + "jsr:@std/fs@^1.0.20" + ] + } +} diff --git a/server/AGENTS.md b/server/AGENTS.md deleted file mode 120000 index 681311eb..00000000 --- a/server/AGENTS.md +++ /dev/null @@ -1 +0,0 @@ -CLAUDE.md \ No newline at end of file diff --git a/server/AGENTS.md b/server/AGENTS.md new file mode 100644 index 00000000..9314b80b --- /dev/null +++ b/server/AGENTS.md @@ -0,0 +1,39 @@ +# Server Instructions + +Topology: [ARCHITECTURE.md](./ARCHITECTURE.md). Wire truth lives in code, not in this file: `packages/sandbox-agent/src/router.rs` (routes) and generated `docs/openapi.json`. Do not narrate endpoint inventories here; they drift. + +## API scope law + +- ACP is the protocol for agent/session behavior and everything that talks directly to the agent; ACP extensions fill gaps (skills, models, metadata). +- Custom HTTP is for host/runtime services: filesystem, terminals, processes, desktop. Core flows must not depend on ACP client `fs/*` / `terminal/*` implementations; they are inconsistent across agents and too limited for host needs. +- Binary filesystem transfers stay on HTTP (`/v1/fs/file`, `/v1/fs/upload-batch`): large payloads ACP JSON-RPC envelopes are not suited to stream. SDK defaults prefer HTTP for these. + +## ACP compliance + +- Before adding any ACP method, property, or config-option category, verify it against the spec: `https://agentclientprotocol.com/llms-full.txt`. +- Valid `SessionConfigOptionCategory` values: `mode`, `model`, `thought_level`, `other`, or `_`-prefixed custom categories (e.g. `_permission_mode`). +- `NewSessionRequest` carries only `_meta`, `cwd`, `mcpServers`; do not add non-ACP fields. +- SDK conveniences may wrap real ACP methods; never send fabricated fields over the wire. + +## API contract rules + +- Every `#[utoipa::path]` handler needs summary + description doc comments; every `responses(...)` entry needs a description. +- Gate: `pnpm generate` regenerates `docs/openapi.json` after contract changes; keep CLI behavior aligned (`docs/cli.mdx`). + +## Tests + +Primary integration coverage for the current surface: + +- Gate: `cargo test -p sandbox-agent --test v1_api` +- Gate: `cargo test -p sandbox-agent --test v1_agent_process_matrix` +- Gate: `cargo test -p sandbox-agent --lib` +- Gate: `cargo test -p sandbox-agent-agent-management` + +## ACP work protocol + +- Source spec and mapping notes: `research/acp/spec.md`, `research/acp/v1-schema-to-acp-mapping.md`. +- Append blockers/decisions to `research/acp/friction.md`; update `research/acp/todo.md` when scope or status changes. + +## Dev testing against examples + +- `SANDBOX_AGENT_DEV=1 pnpm --filter @sandbox-agent/example-mcp start` builds `docker/runtime/Dockerfile.full` from local source and packages the binary into the image. From e7c2e757903d61dd9e3d47fc8e4f512d1e51fa47 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 07:46:29 +0000 Subject: [PATCH 03/12] chore(harness): explicit --allow-read in docs gate shebang --- scripts/check-docs.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/check-docs.ts b/scripts/check-docs.ts index bf0cee01..4a812b9d 100755 --- a/scripts/check-docs.ts +++ b/scripts/check-docs.ts @@ -1,4 +1,4 @@ -#!/usr/bin/env -S deno run +#!/usr/bin/env -S deno run --allow-read import { walk } from "@std/fs/walk"; const ROOT = new URL("../docs", import.meta.url).pathname; From a7f351c32754b60cadcd164c2f67397f93bbf8c4 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 07:52:25 +0000 Subject: [PATCH 04/12] chore(harness): wire check:ci so gate law matches reality Judge sweep flagged G1/D4: AGENTS.md named check:ci and check:docs gates that existed nowhere, and described CI behavior that was false. Wire the Deno docs-gate chain into package.json, point CI at it, trim the shared-component echo trio to its sdks owner plus a root pointer. --- .github/workflows/ci.yaml | 6 ++++-- AGENTS.md | 8 ++++---- frontend/AGENTS.md | 1 - package.json | 4 +++- 4 files changed, 11 insertions(+), 8 deletions(-) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 85f828d9..64cc0b54 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -52,9 +52,11 @@ jobs: done pnpm exec lefthook run pre-commit --no-stage-fixed --fail-on-changes "${args[@]}" - - run: npm install -g tsx + - uses: denoland/setup-deno@v2 + with: + deno-version: v2.x - name: Run checks - run: ./scripts/release/main.ts --version 0.0.0 --only-steps run-ci-checks + run: pnpm check:ci - name: Run ACP v1 server tests run: | cargo test -p sandbox-agent-agent-management diff --git a/AGENTS.md b/AGENTS.md index c3228e43..0f842467 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -13,11 +13,11 @@ Single source of agent law for this repository. Every `CLAUDE.md` in this repo i ## Verification gates -Gates are defined once, in `package.json` scripts and `turbo.json`; workflows and leaves invoke them, never re-enumerate steps. Definition of done: `pnpm check:ci` green plus the tests below for touched surfaces. +Gates are defined once in `package.json` scripts; workflows and leaves invoke the scripts, never re-implement them. Rust and SDK gates are the named commands, and CI runs exactly these. Definition of done: `pnpm check:ci` green plus the tests below for touched surfaces. +- Gate: `pnpm check:ci` = `pnpm typecheck` + `pnpm check:docs`; CI runs this. - Gate: `pnpm typecheck` - TypeScript across all workspaces. -- Gate: `pnpm check:docs` - docs law (see Docs). -- Gate: `pnpm check:ci` = typecheck + `check:docs`; CI runs exactly this. +- Gate: `pnpm check:docs` - docs law via Deno (`scripts/check-docs.ts`). - Rust gates: `cargo test -p sandbox-agent-agent-management`, `cargo test -p sandbox-agent --test v1_api`, `--test v1_agent_process_matrix`, `--lib`. - SDK gate: `pnpm --dir sdks/typescript test`. - Merge, deploy, release, force-push, and credentials are human-approved; ask first. @@ -55,4 +55,4 @@ Gates are defined once, in `package.json` scripts and `turbo.json`; workflows an ## Shared component rule -- React UI shared between Inspector and Foundry belongs in `sdks/react`, unstyled, behavior-only; consumers style via `className`/`classNames`/`data-*`. Changes there must be verified in both consumers. +- React UI shared between Inspector and Foundry is governed by `sdks/AGENTS.md`: change it in `sdks/react`, never fork it. diff --git a/frontend/AGENTS.md b/frontend/AGENTS.md index 1b5f39d4..d610ec03 100644 --- a/frontend/AGENTS.md +++ b/frontend/AGENTS.md @@ -1,4 +1,3 @@ # Frontend Instructions - When the user asks for UI changes, capture screenshots of the updated UI after implementation and verification, then offer to open them and provide absolute filesystem paths. -- Shared components live in `sdks/react` (see `sdks/AGENTS.md`); Inspector-specific shells stay in `packages/inspector/`. diff --git a/package.json b/package.json index c10fd76b..1ea226ab 100644 --- a/package.json +++ b/package.json @@ -8,7 +8,9 @@ "dev": "turbo run dev --parallel", "generate": "turbo run generate", "prepare": "git rev-parse --git-dir >/dev/null 2>&1 && lefthook install || true", - "typecheck": "turbo run typecheck" + "typecheck": "turbo run typecheck", + "check:docs": "deno run --allow-read scripts/check-docs.ts", + "check:ci": "pnpm typecheck && pnpm check:docs" }, "devDependencies": { "@biomejs/biome": "^2.4.6", From 5d0ee615e5f156c958db540d160161fc30788f66 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 08:00:50 +0000 Subject: [PATCH 05/12] chore(ci): skip claude review without credentials, soften skill install --- .github/workflows/claude-code-review.yml | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-) diff --git a/.github/workflows/claude-code-review.yml b/.github/workflows/claude-code-review.yml index 80c880a6..c5c476b5 100644 --- a/.github/workflows/claude-code-review.yml +++ b/.github/workflows/claude-code-review.yml @@ -36,10 +36,17 @@ jobs: with: node-version: "20" + - name: Detect review credentials + id: creds + run: echo "has_token=${{ secrets.CLAUDE_CODE_OAUTH_TOKEN != '' }}" >> "$GITHUB_OUTPUT" + - name: Install sandbox-agent skill + if: steps.creds.outputs.has_token == 'true' + continue-on-error: true run: npx skills add rivet-dev/skills -s sandbox-agent --yes - name: Run Claude Code Review + if: steps.creds.outputs.has_token == 'true' id: claude-review uses: anthropics/claude-code-action@v1 with: @@ -49,4 +56,3 @@ jobs: prompt: '/code-review:code-review ${{ github.repository }}/pull/${{ github.event.pull_request.number }}' # See https://github.com/anthropics/claude-code-action/blob/main/docs/usage.md # or https://code.claude.com/docs/en/cli-reference for available options - From 21c1c701402401435e39f6e304d6a28660599b72 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 12:12:49 +0000 Subject: [PATCH 06/12] fix(agent-management): stage mock launcher stub in idempotent install test The test added Mock to the install list but not to the launcher setup loop, so install(Mock) always wrote the builtin launcher and reported already_installed=false. Mock is native_required=false like Pi and Cursor; stage its launcher stub alongside theirs. Red on main since June 19 (upstream run 27808704382). --- server/packages/agent-management/src/agents.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/server/packages/agent-management/src/agents.rs b/server/packages/agent-management/src/agents.rs index 4f906344..ae62fec1 100644 --- a/server/packages/agent-management/src/agents.rs +++ b/server/packages/agent-management/src/agents.rs @@ -1782,8 +1782,8 @@ exit 0 .expect("write agent process launcher"); } - // Pi and Cursor only need agent process launchers (native_required = false). - for agent in [AgentId::Pi, AgentId::Cursor] { + // Pi, Cursor, and Mock only need agent process launchers (native_required = false). + for agent in [AgentId::Pi, AgentId::Cursor, AgentId::Mock] { fs::write(manager.agent_process_path(agent), b"stub") .expect("write agent process launcher"); } From 50aecef252183a1b9d7f4802de79dfd653485996 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 12:26:20 +0000 Subject: [PATCH 07/12] fix(sandbox-agent): use signaling path in desktop stream websocket test The test connected to /v1/desktop/stream/ws, a path that no longer exists; the route, the TypeScript SDK, and both generated OpenAPI documents all serve /v1/desktop/stream/signaling. --- server/packages/sandbox-agent/tests/v1_api/desktop.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/server/packages/sandbox-agent/tests/v1_api/desktop.rs b/server/packages/sandbox-agent/tests/v1_api/desktop.rs index 76d9389a..9f74efd0 100644 --- a/server/packages/sandbox-agent/tests/v1_api/desktop.rs +++ b/server/packages/sandbox-agent/tests/v1_api/desktop.rs @@ -432,7 +432,7 @@ async fn v1_desktop_lifecycle_and_actions_work_with_real_runtime() { assert_eq!(status, StatusCode::OK); assert_eq!(parse_json(&body)["active"], true); - let (mut ws, _) = connect_async(test_app.app.ws_url("/v1/desktop/stream/ws")) + let (mut ws, _) = connect_async(test_app.app.ws_url("/v1/desktop/stream/signaling")) .await .expect("connect desktop stream websocket"); From 774fe1f9988cf97b1fbb15a37503b1185cc7437a Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 12:37:35 +0000 Subject: [PATCH 08/12] fix(sandbox-agent): assert signaling relay contract in desktop stream test The test asserted a ready frame, a JPEG video frame, and moveMouse input on the signaling websocket, all artifacts of the pre-neko in-house streaming design. The endpoint is a transparent relay: video and input travel over WebRTC. Assert the real contract: a non-empty relayed signaling frame arrives, and a client frame is accepted by the relay. --- .../sandbox-agent/tests/v1_api/desktop.rs | 41 ++++++------------- 1 file changed, 13 insertions(+), 28 deletions(-) diff --git a/server/packages/sandbox-agent/tests/v1_api/desktop.rs b/server/packages/sandbox-agent/tests/v1_api/desktop.rs index 9f74efd0..037e8aaa 100644 --- a/server/packages/sandbox-agent/tests/v1_api/desktop.rs +++ b/server/packages/sandbox-agent/tests/v1_api/desktop.rs @@ -436,34 +436,19 @@ async fn v1_desktop_lifecycle_and_actions_work_with_real_runtime() { .await .expect("connect desktop stream websocket"); - let ready = recv_ws_message(&mut ws).await; - match ready { - Message::Text(text) => { - let value: Value = serde_json::from_str(&text).expect("desktop stream ready frame"); - assert_eq!(value["type"], "ready"); - assert_eq!(value["width"], 1440); - assert_eq!(value["height"], 900); - } - other => panic!("expected text ready frame, got {other:?}"), - } - - let frame = recv_ws_message(&mut ws).await; - match frame { - Message::Binary(bytes) => assert!(bytes.starts_with(&[0xff, 0xd8, 0xff])), - other => panic!("expected binary jpeg frame, got {other:?}"), - } - - ws.send(Message::Text( - json!({ - "type": "moveMouse", - "x": 320, - "y": 330 - }) - .to_string() - .into(), - )) - .await - .expect("send desktop stream mouse move"); + // The signaling endpoint is a transparent relay to the streaming + // process: the first frame is whatever the process sends on connect, + // and client messages are relayed without interpretation. Video and + // input travel over WebRTC, not this socket. + let first = recv_ws_message(&mut ws).await; + assert!( + matches!(&first, Message::Text(text) if !text.is_empty()), + "expected a relayed signaling frame, got {first:?}" + ); + + ws.send(Message::Text(json!({ "type": "ping" }).to_string().into())) + .await + .expect("send signaling frame to streaming process"); let _ = ws.close(None).await; let (status, _, body) = send_request( From a0da906bba2dff2081095bdd6a06ff44f2f09928 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 12:53:14 +0000 Subject: [PATCH 09/12] fix(sdk): allow cold docker image build in sdk test hook timeout The suite's beforeEach builds the docker test-agent image in-process on every fresh CI runner, and a cold build of the common-software stack exceeds vitest's default 120s hookTimeout before the server health wait (bounded separately at 30s) ever starts. Give the hook an explicit 15 minute budget. --- sdks/typescript/tests/integration.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/typescript/tests/integration.test.ts b/sdks/typescript/tests/integration.test.ts index d5ae278a..84f029c9 100644 --- a/sdks/typescript/tests/integration.test.ts +++ b/sdks/typescript/tests/integration.test.ts @@ -219,7 +219,7 @@ describe("Integration: TypeScript SDK flat session API", () => { }); baseUrl = handle.baseUrl; token = handle.token; - }); + }, 900_000); afterEach(async () => { await handle?.dispose?.(); From e20a4564fb48cfb01e8a8de1cfa2a2b08dba78b2 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 13:23:21 +0000 Subject: [PATCH 10/12] fix(sdk): raise docker image build ceiling to 45 minutes The in-process cold build compiles the whole Rust workspace in release mode inside docker on a cache-empty runner; 15 minutes measured short. Build progress is invisible because ensureImage pipes docker build stderr only, so the ceiling stays a ceiling until layer caching lands. --- sdks/typescript/tests/integration.test.ts | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/sdks/typescript/tests/integration.test.ts b/sdks/typescript/tests/integration.test.ts index 84f029c9..8308b459 100644 --- a/sdks/typescript/tests/integration.test.ts +++ b/sdks/typescript/tests/integration.test.ts @@ -219,7 +219,7 @@ describe("Integration: TypeScript SDK flat session API", () => { }); baseUrl = handle.baseUrl; token = handle.token; - }, 900_000); + }, 2_700_000); afterEach(async () => { await handle?.dispose?.(); From 56292382d812a2d057c9081f0f53ae3567d96050 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 14:22:04 +0000 Subject: [PATCH 11/12] fix(sdk): stream docker build progress in sdk tests Two ceilings maxed (120s, 900s, 2700s) with zero build output in the log: ensureImage swallowed docker build stdout and piped stderr to nowhere, so a stall in the hub pull, apt layer, or cargo release build was invisible. Inherit stdio and force plain progress so the next run names where the time goes. --- sdks/typescript/tests/helpers/docker.ts | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/sdks/typescript/tests/helpers/docker.ts b/sdks/typescript/tests/helpers/docker.ts index c15c03c6..f3a07d21 100644 --- a/sdks/typescript/tests/helpers/docker.ts +++ b/sdks/typescript/tests/helpers/docker.ts @@ -156,9 +156,9 @@ function ensureImage(): string { } cachedImage = process.env.SANDBOX_AGENT_TEST_IMAGE ?? DEFAULT_IMAGE_TAG; - execFileSync("docker", ["build", "--tag", cachedImage, "--file", resolve(REPO_ROOT, "docker/test-agent/Dockerfile"), REPO_ROOT], { + execFileSync("docker", ["build", "--progress", "plain", "--tag", cachedImage, "--file", resolve(REPO_ROOT, "docker/test-agent/Dockerfile"), REPO_ROOT], { cwd: REPO_ROOT, - stdio: ["ignore", "ignore", "pipe"], + stdio: "inherit", }); return cachedImage; } From 198e828bae026454e2b01f4ef109da42889b59f9 Mon Sep 17 00:00:00 2001 From: Ryan Lee Date: Sat, 29 Aug 2026 15:21:00 +0000 Subject: [PATCH 12/12] fix(ci): free disk before sdk docker image export The sdk suite's docker build completed fully cached in under a second, then the image export stalled for the entire 45 minute hook budget. Best-supported cause is disk pressure: the runner holds node_modules, the debug target tree, toolchains, base images, and buildkit cache mounts when the exporter writes the final image. Drop the dead target tree and dangling layers after the cargo suites, and print usage so the next run either clears or names the stall. --- .github/workflows/ci.yaml | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml index 64cc0b54..116ef9a7 100644 --- a/.github/workflows/ci.yaml +++ b/.github/workflows/ci.yaml @@ -63,5 +63,12 @@ jobs: cargo test -p sandbox-agent --test v1_api cargo test -p sandbox-agent --test v1_agent_process_matrix cargo test -p sandbox-agent --lib + - name: Free disk for docker image export + run: | + df -h / + docker system df + rm -rf target + docker image prune -f + df -h / - name: Run SDK tests run: pnpm --dir sdks/typescript test