diff --git a/.changeset/ai-harness-cli.md b/.changeset/ai-harness-cli.md new file mode 100644 index 0000000000..479bf06e29 --- /dev/null +++ b/.changeset/ai-harness-cli.md @@ -0,0 +1,12 @@ +--- +'@tanstack/ai-harness-cli': minor +--- + +Add `@tanstack/ai-harness-cli`: `runCli(harness)` runs a harness from the terminal. + +- Line mode reads one message or `/command` per line, hides a secret answer while the user types, and sends `@path` files. In a terminal, `/cancel` and a message typed while a turn runs act at once: the message joins the running turn. When a turn waits, each next line answers one wait: `y` or `n` for an approval (`once`, `always`, or `reject` for an approval of `permissions()`), and an answer read with the schema for a question. Pass `ui` to show your own screen on a session view instead. +- `-p "prompt"` runs one prompt for scripts and CI, and `--output ndjson` prints every AG-UI event. `-p` exits with code 3 (`EXIT.needsAction`) at the first wait of the turn: an approval, a question, or a sign-in. +- `--serve` serves the session protocol over HTTP behind a bearer token. It signs media URLs with its own key, never with the token: `--media-key` or `HARNESS_MEDIA_KEY` (at least 32 bytes), or a new random key at each start. Without a set key, signed media URLs stop working after a restart. On a host without `access`, it also passes `openAccess`: the `user` of `runCli`, or `true` without one. So each request with the token runs as the user of the CLI. +- `--acp` and `--mcp` load `@tanstack/ai-acp` and `@tanstack/ai-mcp` only when you use the flag. A missing package, or one without harness support, fails that flag with a clear message. Any other error while the package loads is reported, not hidden. +- In every mode, the first SIGINT or SIGTERM cancels the running turn. A second one, or one while no turn runs, stops the CLI, and `runCli` resolves to 130 (SIGINT) or 143 (SIGTERM). The CLI closes a host that it made with `recoverable: true`, or releases its session handle on your own host and does not wait for a turn that ignores the cancel. The `signals` option takes another signal source, for example an `EventEmitter` in a test. +- `host` and `user` options let the CLI use your durable host and run as your user. Every mode opens its sessions as that user. On a host with `access`, `--serve` reads the user of each request with `getUser`, and `--mcp` does not start, because stdio has no login. diff --git a/.changeset/ai-harness-coding-tools.md b/.changeset/ai-harness-coding-tools.md new file mode 100644 index 0000000000..216f8f9008 --- /dev/null +++ b/.changeset/ai-harness-coding-tools.md @@ -0,0 +1,20 @@ +--- +'@tanstack/ai-harness': minor +--- + +Add coding tools in the new Node entry `@tanstack/ai-harness/plugins/coding`: + +- `workspaceTools({ root })` gives `read_file` (text pages, images, PDFs), `write_file`, `edit_file` or `patch` (`editStyle`), `list_files`, `grep`, `bash` (with `background`), `webfetch`, and `websearch` (with a `SearchProvider`). With `permissions()`, edits, `bash`, and `webfetch` ask first, and the rules also apply to the real path of a link. `outside: 'ask'` makes a path outside `root` need an approval. `permissions()` decides outside paths and the real paths of links before the call runs, so a tool never asks while it runs. The `cwd` thread setting gives each thread its own working folder. +- `workspaceTools({ workspace: { bind, reattach? } })` gives each thread its own workspace. The root of the workspace is the boundary of every tool. The session log keeps where the workspace is (`harness.workspace.bound`). On another host, `reattach` gives the workspace back (for example a resumed sandbox), or `'pinned'` or `'gone'`. A gone workspace sends `tanstack/workspace-tools:stranded`. With this plugin first, `snapshots()` and `formatter()` use the workspace of each thread, and their `root` is optional. +- `createHarnessHost({ ownership: { hostId } })` gives a host a stable id. Then a thread that uses a folder on that host is pinned to it, and other hosts refuse its tool calls. Without `hostId`, threads are not pinned. A host with `hostId` sends a heartbeat to the log store, and its pins lapse when no heartbeat came for `ownership.expiresAfterMs`: then another host reattaches its threads. `host.unpin({ harness, threadId })` unpins a thread from a host that does not come back, once its ownership expired. +- `workspaceRoot(base, ...parts)` gives a folder in `base` for a tenant and a thread id. It encodes each part, so a thread id from a client cannot leave `base` or reach another thread. +- Commands, `rg`, `git`, and the formatters get only a short list of variables of the server process, plus the `env` of the backend call. Git runs without the hooks and the `fsmonitor` of the project. +- The tools resolve links with `realpath` under the file lock, and fail when a backend cannot resolve a path. A write fails when the file changed after the tool read it in the same chat run, so a lead and its subagents do not lose each other's edits. The file locks are shared by the sessions of a host, by the sandbox (or host) and the real path of the file. +- Background jobs: at most 8 per session (`maxJobs`), the newest 1 MiB of output per job, and the events `tanstack/workspace-tools:job-started`, `job-output`, and `job-ended`. A running command that `session.moveToolCallToBackground()` moves counts as a job, and its process id goes to the log. A job that an earlier host left running is stopped at the next tool call, with a note to the model. When the backend cannot check the job (for example with no `ps`), the log keeps it, and the note says that it can still run. The wake turn of a job runs as the user who started it. `session.jobs()` and `jobs()` of a `host.observe` view list the jobs that did not end (`JobInfo`: the job id, the thread, the command or tool name, who moved it, when it started, and `running` on this host). The `bash` spill file has `[redacted]` in place of each secret of the session. +- `read_file` sends images and PDFs up to 256 KB. `webfetch` gives the model the first 256 KB of a page. +- A `WorkspaceBackend` runs the tools. `hostBackend` is the default. `realpath` is required. `spawn` can give a `jobId`, `exec` can give one with `onStart`, and `killJob`, `binding`, and `close` are optional. `killJob` rejects when it cannot check that the process is the job. On Windows, commands do not run a program from the working folder by its name. +- `snapshots()` adds `/undo` and `/redo`, which restore only the files of the turn (git 2.26 or newer), file restore for `session.revert()`, and a diff for each model step. Each thread of each tenant has its own shadow repository, which never stores `.env` files. A step keeps only the files of its own run. The steps and the undo point are in the session log, so another host with the same repository can undo. A revert uses the last 500 steps. `host.sessions.delete` removes the repository of the thread, and `purgeSnapshots()` does it from your own code. A failed snapshot sends `tanstack/snapshots:failed`. +- `formatter()` formats each written file with the formatter of the project. With `permissions()`, the rules decide the formatter with the call that writes, before that call runs: one approval covers both, and the formatter runs after the write. An `always` answer is tied to a hash of the formatter config files. A write of a formatter config below the root, or under `node_modules`, asks each time, and an `always` answer saves no rule for that formatter run. +- `WorkspaceHooks` lets other plugins run code after a read or a write. + +`@vscode/ripgrep` and `turndown` are optional peer dependencies. diff --git a/.changeset/ai-harness-core.md b/.changeset/ai-harness-core.md new file mode 100644 index 0000000000..a5b3493049 --- /dev/null +++ b/.changeset/ai-harness-core.md @@ -0,0 +1,55 @@ +--- +'@tanstack/ai-harness': minor +--- + +Add `@tanstack/ai-harness`: build an agent harness on TanStack AI. A harness keeps one conversation open across many turns. + +- `defineHarness()` names an agent: its adapter, tools, system prompts, middleware, agents, subagents, routing, plugins, turn hooks, and durability options. +- `createHarnessHost()` opens sessions. Each `host.open` gives its own handle to the live session of the thread. `session.close()`, or `await using`, releases the handle. A dropped handle is released by the garbage collector, with one warning. After `close()`, each call on the handle throws. When the session closes under a handle that is not released (its log failed, or a delete ran), the next call on the handle opens it again. +- `lifecycle` on `createHarnessHost()` sets how sessions live. A session that no handle holds closes after it was idle for `closeIdleSessionsAfterMs` (default 60 000). `maxOpenSessions` (default 500) first closes the least recently used idle sessions. A session is not idle while a background job of this host runs. `retention` (`keepSettledOperations`, `keepSettledAgentRuns`, `maxEventBytesInMemory`) bounds the settled operations, agent runs, receipts, events, and log tail that each session keeps in memory. The log keeps the `MESSAGES_SNAPSHOT` that core sends when a run pauses at an interrupt, so a rejoin from the log rebuilds the paused run. `host.close({ recoverable })` stops the work of every session and refuses later opens. With `recoverable: true`, the host gives each log that has work back with a wake, so a worker takes it at once. +- `host.observe(harness, { threadId, user })` reads a thread with no session: its events, transcript, snapshot, media, usage, and entry. It sets up no plugins, writes nothing, and never takes the log, so a read works while another host owns the thread. Its snapshot follows the same rules as `session.snapshot()`, without what only an open session knows (a running turn, operations, queues). `view.logState()` reads the `durability.logState` fold. +- `host.sessions.delete` closes the sessions of a thread and of its child threads. Then it deletes the log of the session tree (with the settings, config, plugin state, interrupted turn, runs, interrupts, usage, revert data, and the inputs that wait), the index entries, the uploaded media, and the generated artifacts. The first-party plugins also end the thread: `workspaceTools()` closes an owned sandbox with `'closed'`, and `snapshots()` removes the shadow repository. This needs the index entry, and the harness on that host: a host that opened it, or one with it in the `harnesses` option of `createHarnessHost()`. Else the host warns. A child thread is ended in the log of its root. While the delete runs, an open of the thread throws. With `access`, a thread with no index entry throws `AccessDeniedError`. +- A session takes `send` (a prompt, a steer into the running turn, a follow-up, or a message to an agent run with `to`), `continueWithoutMessage`, `reset`, `resolve`, `cancel`, and `moveToolCallToBackground` inputs. Each returns an operation with a receipt, and the session streams typed events. `host.isOpen(harness, { threadId })` says whether this host holds that session open. +- The session methods take one object: `cancelInput({ inputId, user? })`, `revert({ messageId, user? })`, `background({ toolCallId?, user? })`, `getMedia({ id })`, `mediaUrl({ id })`, and `agentTranscript({ subagentRunId })` or `agentTranscript({ runId })`. `AgentRun.subagentRunId` and `AgentRunInfo.subagentRunId` name the child id of an agent run, so the transcript of a run that code or a background call starts is found too. +- `session.appendRecords({ records, inputId?, user? })` returns a `Receipt`. A second append with the same `inputId` writes nothing. Without `inputId`, the append gets a new input id, and the log keeps it as an input. The `harness.revert` record keeps the user of `revert`, and the `harness.job` record keeps the user of `background`. +- `createHarnessHost({ access })` decides who can use each session. `getUser({ request })` reads the user of a request, and `isAllowed({ user, action, session, run })` decides each action. The user type comes from what `getUser` returns. The host keeps each tenant apart, needs a `user` on each call, and `host.open` returns a handle that acts only as its user. A refused action throws `AccessDeniedError`. The operations and the agent runs that a handle returns check access too. A child thread is checked through its root, also on `host.open`, so a stranger does not learn its root. When one user answers the turn of another user, the rest of the turn runs as the user who sent it. Without `access`, one user runs the host, with no checks. With `access`, keyed adapters and provider keys read env vars only when `access.allowServerEnvKeys` is `true`. +- `membersOnly()` gives ready access rules with the roles owner, member, operator, and viewer. `access` is `{ getUser, rules, getGuestUser?, allowServerEnvKeys? }`, and `membersOnly()` returns rules: `{ getUser, rules: membersOnly() }`. Rules are `{ isAllowed, filterSessionList?, members? }`: `filterSessionList` is the store filter of `host.sessions.list`, and `members` turns on `host.sessions.share` and `unshare` (`members.fork` sets the members of a fork). Your own rules are `rules: { isAllowed }`. `host.sessions.share` and `host.sessions.unshare` change the members, and `host.sessions.fork` takes `members: 'self' | 'all'`. The owner is the member with the role `owner`. For apps with their own access tables, `host.revoke` takes the access of a user to a thread, and returns a `Promise`. `host.revoke` and `unshare` cancel the running and queued work of the user on every host, and remove the schedules of the user at once. When that work cannot be stopped, they reject, and the access is gone already. A running `events()` read of a handle or a view ends with `AccessDeniedError`, and later view calls throw. `host.getUserOfRequest` reads the user of a request for your own routes. Each `host.sessions` method takes one object. +- A user with a `tenantId` keeps its sessions in its tenant: one thread id in two tenants is two sessions, with two index entries and two logs. `logIdOf({ tenantId, logId })` gives the id of a log in the store. +- Every run has a run context with its user, and that user's credentials and provider keys: `ctx.context.run` in tools and middleware (type them with `HarnessRunContext`), `ctx.turn.run` in a turn plugin, and `run` in a command. The `ctx.credentials` and `ctx.keys` of a session plugin have no user, so each call throws. `ctx.agents.start(agent, input, { user })` starts an agent for a user. `AgentRun.send` takes `user?`, and the `send` and `cancel` of a run act as the user of the run. `session.agents..run` honors `user`. The log keeps the sender of every input. The run context also has `epoch`, the ownership epoch of the log, and `idempotencyKey(step, scope?)`, a key for one side effect that is the same when the run runs again on any host. +- `definePlugin()` adds tools, prompts, middleware, agents, commands, config options, state, and extension points to a session or to one turn. With `lifetime: 'host'`, the host sets a plugin object up once per harness and shares it with every session that uses the same object. It is cleaned up when the last session that uses it closes, or at `host.close()`. A prompt with `placement: 'context'` goes in one system prompt after all the others, made again at each model call, so text that changes often (the todo list, the goal status, plan mode) leaves the system prompts before it in the provider prompt cache. The default `placement: 'system'` keeps a prompt in the system prompts. `prepareTools` gets `run`. `onTurnStart(turn)` runs once at the start of each turn, before the prompts, so a plugin can refresh the state that its prompts read. A plugin's `ctx.session.send` and `note` take `user?`, and default to the session owner. A thread that removes a plugin cannot run the agents of that plugin by name. +- `routing.router` sends a turn to a root agent. The log keeps the active agent, and the router reads it as `activeAgent`. A router that returns `undefined` gives the turn to the active agent, else to the main model. One agent name makes that agent active, `'main'` clears it, and `{ handoff, note }` is a handoff from the router. `routing.afterAgentsAnswer: 'end-turn' | 'main-answers'` (default `'end-turn'`) says whether the main model answers after the routed agents. +- Handoffs move a conversation to another root agent, and the target answers in the same turn. With `routing`, the main model and each root agent get a `handoff` tool. Code hands off with `ctx.harness.handoff({ to, note })` in a root agent, and with `ctx.context.run.handoff(...)` (`RunContext.handoff`) in a tool of the main model. Each handoff writes a `harness.handoff` record and streams a `harness.handoff` event (`from`, `to`, `note`, `by`, `runId`). A message that an agent writes has `metadata.harness.agent`. A turn hands off 10 times at most. An app tool named `handoff` fails the turn. `routing.giveModelsHandoffTool: false` gives no model the `handoff` tool, and the router and code can still hand off. A handoff stops only the chat that made it. When the part that handed off also stops for interrupts, the resolve turn runs the target (or pauses for a person). The router writes a handoff to the main model once when the active agent is gone. A handoff from the router or from code adds a `handoff` tool call and its result to the transcript before the target answers, so the target sees why it got the conversation. With `routing.giveModelsHandoffTool: false`, it adds one assistant note. +- With `routing.allowHandoffToHuman: true`, a handoff to `{ human: true }` opens a `human` wait with no default expiry, and the turn ends `interrupted`. User messages during the pause go in the transcript, and no model answers them. `session.send(text, { as: 'operator' })` adds an operator reply. It needs the `operator` access action, and it is rejected with `no_human_wait` when no handoff to a person is open. The operator ends the pause with an answer to the `human` wait: `{ resume: true }` or `{ handBack: '' | 'main' }`, also to a root agent of a turn plugin. When the `human` wait ends in any other way (a resolve, a cancel, an expiry), the agent from before the handoff answers again. An operator reply with files keeps its text in the assistant message, and the files in a user message after it with `metadata.harness.operatorAttachment`. A view shows the files with the reply, live and after a reload. +- `host.transfer({ from, to, envelope, user, transferId })` moves a conversation to another thread, also of another harness, in the same tenant. Only the envelope moves: a note, then no message, a summary that the source model writes, or the settled transcript with its media. The user needs `transfer` on the source and `send` on a target that exists. A new target has the user as its only owner. Both logs get a `harness.transferred` record, the index entries link the threads in `metadata`, `host.events()` yields a `transferred` event, and a retry with the same `transferId` is safe. A source thread that does not exist throws. A fork stays in its harness: `host.fork` with a thread of another harness throws. +- Agents run from code with `session.agents..run()` and `.start()`. `agentRun.receipt` resolves. In agent code, `ctx.harness` gives `step.do()` (a finished step returns its stored value when the run runs again), `step.sleep({ name, ms })`, `agents`, `state`, and `handoff` (in a root agent). `ctx.harness` is optional: a run that a plain `chat()` starts has none. +- `ctx.harness.agents` in agent code and `ctx.agents` in a plugin have one type, `AgentActions`: `start(agent, input, options?)`, `run(agent, input, options?)`, and `group({ concurrency, onFailure, key }, (g) => [g.start(...)])`. The input type comes from the input schema of the agent, and the result type from its output. `group` resolves to the results in order. With `onFailure: 'collect'`, each entry is an `AgentSettled` (`{ ok, value }` or `{ ok, error }`). The children of a group over its `concurrency` wait as `queued`. +- Each start takes `key`, `limits: { timeoutMs, maxAttempts }`, `budget: { maxTokens, maxCost }`, and `onModelError`. A run over its budget fails with the code `budget`, after the first model call over it. A run that runs again after a host stop counts the model use of the runs before. A run past `limits.timeoutMs` fails with the code `timeout`. A child never has more budget than its parent has left. `onModelError` is asked when a model call fails before it streams, and `'retry'` calls only that model call again (at most 3 calls). As data, `onModelError: { retries: 1 | 2, transientOnly? }` retries up to `retries` times (by default only a transient error), and the log keeps it, so a run that another host takes over or resumes retries too. A function is not in the log, so a run that another host takes over runs without it. A `ctx.agents` start in a plugin tool that an agent run calls is a child of that run, so it takes no `maxConcurrent` slot and the run can await it. `parent: ctx.context.run` names the run, and `parent: null` starts a root run. A `parent` that names no running agent run of this session throws, so a mistake does not start a root run. A running turn of the session gives a root run. +- The log keeps every run of a session as a `RunNode` of one run tree (`turn`, `agent`, `tool`, or `command`), with its parent, its user, its limits, its budget, and its result. `agentRuns()` reads the tree, and `AgentRunInfo` has `runId`, `kind`, `budget`, `result`, and `error`. Each `harness.usage` record has the `runId` of its run, and an agent run that ends keeps the usage of the run and its children. +- `maxDepth` and `maxCalls` count per tree, and `maxConcurrent` counts the root agent runs of the session. Over `maxConcurrent`, a root start waits as `queued`. The log keeps the counts, so a restart does not reset them. A `queued` start survives a host stop: the next host starts it with the same run id, in order, when a slot is free. +- A child run gets the user, the keys, and the signal of its parent: a cancel, a timeout, or a budget stop of the parent stops the child. A cancel of a turn also cancels the background runs that it started. A background run that a turn starts keeps the turn plugins until it settles. An agent that the model calls as a tool, and a routed agent, get `ctx.harness` too. +- A run with `runAgainAfterCrash: true` that runs again after a host stop gets the children that it started before, with their stored results, and starts only the rest. A background child that the model starts (`background: true`) runs again after a host stop as a run with `runAgainAfterCrash: true`, with the same `subagentRunId`. `durability.resumeBackgroundAgents: false` fails it instead. +- `ctx.harness.state({ key, initial })` gives the agents of a session a shared state by key, with `get` and `update`. The log keeps it, and a fork copies it. `STATE_SNAPSHOT` has `{ plugins, shared }`. +- A routed agent takes steers at its next model call, and late joins in the same turn, and `turn.beforeFinish` and `turn.onModelError` run for it. +- Every host keeps each session tree in one log, `stores.log` from `@tanstack/ai-persistence`. `memoryPersistence()` has a memory log, and a host with no `persistence` uses it. The log holds the inputs, events, transcripts, runs, interrupts, settings, config, plugin state, agent results, schedules, and tool steps, and the host folds them when a thread opens. A root thread and its child threads (agent runs and subagents) write to the log of the root. `session.logState()` reads the `durability.logState` fold of the whole tree. A harness with another `durability.addRecordsToModelContext` or `durability.logState` than the one that folds an open log throws when it opens that log. With a log store that survives a restart, the next host recovers a turn that a crash stopped. +- One host owns a log at a time: the first write takes it, one timer per host renews it, and a host that loses it stops its work on it. A host gives a log back when it has no work. Another host recovers the work of a log only after its ownership expires. A read never takes the log. An input that reaches another host waits in the input lane of the log, and the owner runs it (receipt `queued`) with its JSON overrides (`model`, `modelOptions`, `reasoning`, and `promptCache`). A command that reaches another host ends as it ends on the owner: with its result, its error, or `InputRejectedError`. An input to a thread that was deleted is refused: the receipt is `rejected` with the reason `deleted`, and an awaited operation rejects with `InputRejectedError` ("The thread was deleted."). `ownership: { expiresAfterMs, renewEveryMs }` sets the timing. `renewEveryMs` must be less than `expiresAfterMs`, else `createHarnessHost` throws a `RangeError`. With only `expiresAfterMs`, `renewEveryMs` is a third of it. +- When the owner refuses an offered input (busy, or an id that the log has with another payload), `session.settled()` on the host that offered it rejects with `InputRejectedError`. The log gets a `harness.offer.rejected` record. An offer that comes while the owner appends is not missed. +- A host that does not own a thread opens it while the owner runs a background job. Settings, config values, and the interrupted turn follow the changes of other hosts. `updateThreadSettings` merges into the latest stored settings. +- The owner writes a snapshot of the folded log to the log store after `log.saveSnapshotAfter.bytes` of new records (default 1 000 000) or `log.saveSnapshotAfter.ms` (default 300 000), and a new host opens from it. With `lifecycle.retention.keepLogRecords: 'since-snapshot'` (default `'all'`), the owner truncates the records before each snapshot. A reader whose cursor is in the dropped part gets one `harness.gap` event (`EventGap`), then the stream goes on. With `'since-snapshot'`, a thread whose snapshot the host cannot use (a changed `reduce` version or `project`) cannot open, and rejects with `LogTruncatedError`. `session.revert()` works with `'since-snapshot'`: the snapshot keeps the last 500 file steps. A plugin sets which of its records the snapshot keeps with `keepRecords: { [type]: count }` on its definition, and reads them, oldest first, with `ctx.session.keptRecords(type)`. The fold keeps the last 100 ended runs, run nodes, interrupts, and agent results, and drops the turn overrides of a settled input, so a long thread does not grow its fold and its snapshot with each turn. +- `host.submit` stores an input for a thread and runs nothing. `session.schedule({ at, input, scheduleId, user })` runs an input later, and `session.cancelSchedule({ scheduleId })` stops it. `createHarnessWorker({ host, harnesses, concurrency, signal, pollMs })` runs the threads that have work and no live host, and returns `{ stop }`. `host.resumePending` does one sweep. A swept thread that fails to open goes back with a wake in 30 s. A worker skips a thread that fails to open for a time that doubles after each failure in a row, up to 5 minutes, so the other threads still run. `host.submit` throws for a thread of another harness, and a later offer never changes the harness of a log. +- `durableTool()` keeps the result of each step. A crash can cut a step, and then it runs again, so `step.do` passes `{ idempotencyKey, epoch }` to `fn`. Give the key to your system, so it refuses the second side effect. `step.ask({ name, message, schema })` asks the user: the log keeps the question, any host can take the answer, and a run again after a stop returns the logged answer. A `step.do` value must be JSON: a `Date`, a `Map`, or a class instance is a type error. `step.sleep({ name, ms })` waits in the process, and the log keeps the wake time, so a worker continues the call after a stop. A sleep longer than 30 s on a thread that only a worker runs gives the thread back with the wake, so it keeps no worker slot. `durability.truncatedToolResult` sets the text of a tool call cut at the output limit. Without it, a cut answer ends the turn. +- `turn.onModelError`, `retryTransientErrors()`, `turn.beforeFinish`, and `turn.canJoin` control how a turn retries, continues, and ends. +- Every approval, question, sign-in, client tool, and human takeover that a run waits for is a `Wait` in the session log (`waitId`, `kind`, `runId`, `schema`, `expiresAt`, `message`). `WaitKind` is `approval`, `question`, `sign-in`, `client-tool`, or `human`. `waitKind`, `isApproval`, `waitFromInterrupt`, and `waitFromQuestion` come from the root and from `./view`. `snapshot().pendingWaits` lists the open waits: each one with its `interrupt` when a chat run stopped for it, and `turn: true` for a wait of the stopped turn. It is the one wait view of a snapshot. and `session.waitForRun({ runId, signal })` resolves with the next one. A wait ends `answered`, `cancelled`, or `expired`. A wait with `expiresAt` expires on time. Only `sign-in` has a default limit: 15 minutes. +- A wait that only the log holds keeps no host busy: the session goes idle, and an answer on any host continues the run. `resolve` answers the waits with an `interrupt`: of a stopped turn, or of one background agent run per call. `answer` answers the waits with no `interrupt` (a `step.ask` question, an MCP question, the `human` wait of a handoff), checks the answer against the JSON Schema of the wait, and keeps a secret out of the log. A `resolve` of such a wait is refused with `use_answer`, and a second answer to a wait that one answer ends is refused. `host.submit` checks an answer against the wait that it answers. +- A run waits for a person only at fixed points: `permissions()` before a tool call, the `question` tool, `step.ask` in a durable tool, a command (`CommandContext.step`), or an agent run, and the live waits of MCP elicitation and OAuth sign-in. `ctx.session.ask` throws anywhere else, with an error that names `step.ask`, the `question` tool, and `PermissionResources`. A command that called `step.ask` runs again from the top after a host stop. `Question` takes `expiresAt` and `signal`. +- A background agent run stops with status `waiting` on a wait, for example an approval of `permissions()`, and continues after the answer, also after a restart. An agent that the model calls as a tool inside a turn waits with the turn: its approvals, questions, sign-ins, app interrupts, client tools, and `ctx.harness.step.ask` questions are waits of the turn in `snapshot().pendingWaits`, with `agent` and `subagentRunId`. After the resolve, also on another host or after a restart, the agent goes on from where it stopped, and its result goes to the model as the tool result. Its `ctx.harness.step.do` values are in the log. With `durability.agentToolWaits: false`, the agent does not wait: a call of it that needs an approval is refused. +- The `operator` access action answers a `human` wait of any run, also with a resolve. A `human` wait takes only `operator`. `membersOnly()` gives it only to the role `operator`, so the user who waits for a person cannot act as that person. `membersOnly({ operator: { roles } })` sets the roles that have it. `isAllowed` gets the `wait` (`waitId`, `kind`) of an `answer` or `operator` action. +- `session.settled({ inputId, signal })` takes one object. `signal` stops the wait for the settlement. +- `session.onQuestion`, `session.onAuthRequired`, `session.onMedia`, and `session.onUsage` call a listener for the events that a user answers or sees, with a typed value. +- Sessions also have thread settings, fork and revert, usage totals, media files, the transcript of each child agent (`session.agentTranscript`), and `harnessText()` / `harnessAgent()` to use a harness as a text adapter or an agent. In a session, a `harnessAgent` child is a child run in the log of the root, with the user, keys, limits, and budget of the run that calls it. Its usage and its waits show in the parent session, on every host and in `host.observe`. An answer on another host goes to the host that runs the child. The base64 cache of media files is shared by all the sessions of a host. +- `harnessText()` uses the user of each call. The inner thread is `/inner/` on every host, and `host.open` refuses that id from app code. When a host with access runs the outer thread too, the inner session is checked through the outer thread, so every user of a shared outer thread can use it. The index entry of the inner thread keeps the link to its outer thread. An abort on a remote harness sends a cancel. The usage of the inner run reaches the outer `RUN_FINISHED`. Its SSE reader takes multi-line `data:` and CRLF. `harnessText()` and `harnessAgent()` forward events on one shared path. +- `createHarnessHost({ observability: { tracer, onOperation } })` traces runs. `tracer` takes an OpenTelemetry `Tracer`. `@opentelemetry/api` is an optional peer dependency: the host loads it only when it has a tracer. The host writes `harness.operation` spans (turns, agent runs, commands), and `harness.recovery`, `harness.wait`, and `harness.transfer` spans. A child run on the same host is a child span: the host passes `trace.setSpan(context.active(), parent)` as the third argument of `startSpan`. A tracer that throws is logged and fails no run. `SendOptions.traceparent` stores a W3C `traceparent` with the input, and the run span on the host that runs it gets a link to it. +- `onOperation` gets a `started` report and a `settled` report (`OperationReport`) for each operation that this host runs, with `status`, `durationMs`, `error`, and `usage`. A listener that throws gets a `harness.plugin.warning` event, and the work goes on. +- More lifecycle events in the session stream: `harness.input.queued`, `harness.ownership.changed` (`reason: 'taken_over'` when the store says that the old owner expired, else `'acquired'`), `harness.stranded`, `harness.transferred`, `harness.wait.opened`, and `harness.wait.settled`. `HarnessEventValues` maps each `harness.*` event name to the type of its value. +- `onEvent`, `onQuestion`, `onUsage`, and `onMedia` also get the events that another host writes to the log of the session. An event fires once. The status of `host.events()` counts that work too, for the sessions that are open on the host. +- The router gets `model`: `model.chat({ adapter, options, label })` in `routing.router` (`HarnessRouterContext.model`) calls a model with the keys of the sender, and a cancel of the turn stops it. +- `session.usage()` has `bySource`: the model calls that are not a turn or an agent run, keyed by `plugin:`, `router`, or `transfer`. Each `UsageCall` and `harness.usage` event of such a call has `source`, and `label` when the call has one. The summary of `host.transfer` with `messages: 'summary'` counts in the source thread under `transfer`, with the label `summary`, and stops after 2 minutes. diff --git a/.changeset/ai-harness-goal.md b/.changeset/ai-harness-goal.md new file mode 100644 index 0000000000..52b3d593a8 --- /dev/null +++ b/.changeset/ai-harness-goal.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': minor +--- + +Add the `goal({ judge, maxRounds })` plugin to `@tanstack/ai-harness/plugins`. `/goal ` sets a goal and starts a turn. After each turn, the judge model checks the goal, and the plugin starts the next turn until the goal is met, the round limit is reached, a turn waits for approval or fails, or the user sends a message. `/goal`, `/goal stop`, and `/goal resume` show and control it. The `GoalMet` event fires when the goal is met, and `selectGoal(session.snapshot())` reads the goal for a UI. diff --git a/.changeset/ai-harness-plugins.md b/.changeset/ai-harness-plugins.md new file mode 100644 index 0000000000..0c2ca1a70d --- /dev/null +++ b/.changeset/ai-harness-plugins.md @@ -0,0 +1,18 @@ +--- +'@tanstack/ai-harness': minor +--- + +Add first-party plugins. `@tanstack/ai-harness/plugins` loads no Node module. The plugins that read files are in the Node entry `@tanstack/ai-harness/plugins/node`. + +- `permissions()`: rules for tools, paths, and commands, and the `default`, `plan`, `acceptEdits`, and `bypass` modes. It decides every call of a model message before the calls run, and opens one approval wait for each call that asks, so the turn stops and no process waits. An answer is `true` or `false`, `'once'`, `'always'`, or `'reject'`, or `{ answer, message? }`. It also has saved "always" rules with `/permissions` and `/permissions forget `, and `listSavedPermissions` / `deleteSavedPermission` for server code. The rules apply in subagent runs too. It decides in `onGuardToolCall`, on the final checked arguments. +- `agents()` (in `plugins/node`): agent profiles with their own prompt, tools, model, rules, and step limit, in code or in Markdown files, `/agent` to switch the primary agent, and the built-in `build`, `plan`, `general`, and `explore` agents. A subagent run uses the tools and the model of the turn that started it. A run whose turn is not in the memory of this host fails with a clear error. A run that code starts needs a profile `model`. +- `question()`: a `question` tool for the lead turn. Each question is a wait of the turn, answered with `resolve`. Agent runs do not get the tool. +- `todos()`, `title()`, `modelPicker()`, `compact()`, and `usage()` (with your own prices for calls that have no provider cost). +- `goal()` runs each turn as the user who set the goal. +- `projectInstructions()`, `fileCommands()`, and `boundToolOutput()`, in `plugins/node`. + +A plugin runs agents with `ctx.agents.start`, `ctx.agents.run`, and `ctx.agents.group({ concurrency }, (g) => [g.start(agent, input)])`, with the same types as `ctx.harness.agents` in agent code. + +`PermissionRules`, `PermissionResources`, `PermissionDecisionCapability`, and `decidePermission` are also exported from the package root. They load no Node module, so an edge-safe plugin can read the permission rules. + +`ctx.model.chat({ adapter, options, label, run, timeoutMs })` calls a model from a plugin, for example a judge or a title. It uses the keys of the user of the run, the budget of the run applies, and it stops when the run stops or after `timeoutMs` (default 2 minutes). Its usage counts in the session under `plugin:` in `session.usage().bySource`. `goal()`, `title()`, and `compact()` call their models this way, with the labels `judge`, `title`, and `compact`. diff --git a/.changeset/ai-harness-provider-keys.md b/.changeset/ai-harness-provider-keys.md new file mode 100644 index 0000000000..3539e664a5 --- /dev/null +++ b/.changeset/ai-harness-provider-keys.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': minor +--- + +Add the `providerKeys({ providers })` plugin to `@tanstack/ai-harness/plugins`. Users of a shipped harness connect their own model provider in the app, with no `.env` file. It adds `/connect:`, `/disconnect:`, and `/keys`. `/connect:` runs the provider's `signIn` when it has one, else it opens `keyUrl` when set and asks the user to paste the key with `step.ask` and `secret: true`. The log keeps the question as a wait, so a user on any host can answer it, and the log never keeps the key. The key is saved in the credential store of the user, and a `keyedAdapter(...)` for that provider uses it. `/keys` shows where each key of the user comes from: a saved key (the last 4 characters only), the env var, or no key. The plugin state shows only the env vars that the session reads, as every member of the thread reads it. On a host with access without `allowServerEnvKeys`, no env var counts. diff --git a/.changeset/ai-harness-transport.md b/.changeset/ai-harness-transport.md new file mode 100644 index 0000000000..c0cbab2f9c --- /dev/null +++ b/.changeset/ai-harness-transport.md @@ -0,0 +1,23 @@ +--- +'@tanstack/ai-harness': minor +--- + +Serve a harness session to clients. + +- `createHarnessHandler()` from `@tanstack/ai-harness/server` serves a session over HTTP, as the user that the host `access` reads from each request. `POST run` is standard AG-UI, so `useChat` with `fetchServerSentEvents` works on a harness, with approvals, client tools, and a reload that joins the running turn. `POST control`, `GET events`, `GET transcript`, `GET agent-transcript` (the messages of a child agent run), `GET snapshot`, and `GET describe` cover one session. `sessions` (with the `share` and `unshare` ops), `GET host-events`, and the media routes cover the rest. A `host.events()` reader listens from the call, and one that falls more than 1 000 changes behind stops, and reads again for the current statuses. A refused action answers 403. A read of a thread that does not exist answers 404 and starts no session. A signed media URL still needs a user: `getUser` reads the user of the request, and that user needs `read` on the thread, in the tenant of the URL. For a request with no login, `access.getGuestUser({ tenantId })` can give a guest, and `isAllowed` decides `read` for it. The guest exists only on the signed media route. `requestUsers` from `@tanstack/ai-harness/server` gives the user of each request with the same rule, for your own server. The read routes read a thread with `host.observe`, so they set up no plugins. `GET describe` opens the session. +- `handleHarnessSocket()`, from the same entry, serves the same session over a WebSocket. It takes the upgrade `request` and reads the user from it. It handles the frames of one connection in order. +- A `command` or an `agent` input over HTTP or the socket runs once for one `inputId`, and the `inputId` of its receipt is the id of the client. +- `expose` on `defineHarness()` names the agents, settings, config keys, and commands that a client can use. Any other input of those kinds gets `rejected` with `not_exposed`. +- `createHarnessClient()` from `@tanstack/ai-harness/client` is a typed, browser-safe client. It sends inputs, reads the session, and streams events that resume from a cursor. Without `from`, `client.events()` starts at the newest event (the cursor of the snapshot), so it does not send the whole history first. This is the default: a reader that needs the history (to rebuild a transcript, for example) must pass `from: '0'` or a saved cursor. When the snapshot answer has no string `cursor`, `client.events()` throws and does not retry. `client.events()` connects again after `idleTimeoutMs` (default 60 000) with no bytes. The `prompt` input of the protocol and `client.send` take `as: 'operator'` for the reply of an operator after a handoff to a person. +- The client methods take one object: `cancelInput`, `revert`, `agentTranscript`, `mediaUrl`, `appendRecords`, and `putMedia`. `client.sessions.list`, `rename`, `delete`, `update`, `fork`, `share`, and `unshare` manage the sessions. `HarnessErrorCode`, the type of `HarnessProtocolError.code`, comes from the main, `/client`, and `/server` entries. +- `oauthConnector()` and the OAuth helpers from `@tanstack/ai-harness/oauth` (`loopbackLogin`, `deviceLogin`, `exchangeCode`, `refreshCredential`, and more) keep a credential per user. A tool calls `token({ run: ctx.context.run })` for the token of the user of its run. `token({ run, wait: true })` stops the turn with an `auth_required` interrupt until the user signs in. `deviceLogin`, `loopbackLogin`, `startLoopbackReceiver`, `exchangeCode`, and `refreshCredential(config, credential, { fetch, signal })` stop at once when their `signal` aborts, so a cancel or a session close stops a running `/connect`. `OAuthConfig.issuer` makes `loopbackLogin` refuse a code from another issuer (RFC 9207). The loopback deadline (`timeoutMs`) does not keep the process alive. +- `run.credentials.refresh(id, update, { signal })` refreshes a saved credential once for all callers on a host, keyed by tenant, user, and id. It saves with `setIf` when the store has `getVersioned` and `setIf`, and uses the other host's token on a conflict. Without that pair, it does a plain `set`. It saves to the scope where it found the credential, so a tenant token stays the tenant's. `update` gets a signal that only `host.close()` aborts. An abort of `signal` rejects only that call. `oauthConnector` refreshes this way, and `refreshCredential` keeps the `client` of the old credential. +- The session stream speaks protocol version 2 on every transport: `GET events`, the socket, and the stdio worker. Each event is an AG-UI wire chunk in a `harness.event` frame, with `cursor`, `threadId`, and `operationId`. The AG-UI `RUN_STARTED` of a child agent run has `parentRunId`, so a client can rebuild the run tree. +- The first frame of a stream is `harness.hello` with the version `v`. A client asks for a version with `?v=` or with `v` in `harness.subscribe`. Another version gets a `harness.error` frame with the code `unsupported_version`, and the stream closes (socket code 4400). +- A `harness.error` frame has a `code` (`unauthorized`, `forbidden`, `invalid_frame`, `unsupported_version`, `slow_consumer`, or `internal`) and a `requestId`. On the socket and the stdio worker, the `requestId` is the id of the frame that failed. Every answer of the handler has an `X-Request-Id` header: the id that the client sent when it is a short safe id, else a new UUID. +- Streams stay open and protect the server. SSE streams send a `: ping` comment and the socket sends a `harness.ping` frame when idle. SSE reads the log only as fast as the client reads, and the socket waits while its `bufferedAmount` is over the limit. A client that does not read in time gets a `slow_consumer` error frame, and the stream closes (socket code 4408). A slow client loses no events: it connects again from its last cursor. `createHarnessClient` does this in `events()` and `hostEvents()`, and ends the read with a `HarnessProtocolError` (with its `code`) for any other error frame. +- `presence: { onConnect, onDisconnect }` on `createHarnessHost` reports each client stream of `GET events` and of the socket (after its subscribe), as a `PresenceEvent` with a new `connectionId`, the user, the thread, the transport, and `resumed`. `onDisconnect` runs once per stream, with the reason `closed`, `revoked`, or `error`. The host does not wait for a hook, and a hook that fails only warns. +- `stream: { heartbeatMs, slowConsumerMs, sseQueueBytes, socketBufferBytes }` on `createHarnessHandler` and `handleHarnessSocket` changes these values. The defaults are 20 s, 30 s, 1 MiB, and 8 MiB. A value that is not a finite number above 0 throws at the start. `HarnessStreamOptions` is the type. +- A reader whose events are gone gets one `harness.gap` event, with `reason: 'truncated'` (a truncate of the log) or `'evicted'` (the memory feed of a session dropped them). The stream then goes on from `resumeAt`. +- A W3C `traceparent` goes with a message: as the `traceparent` header of `POST control` and `POST run`, or as the `traceparent` field of a `harness.input` frame. `ClientSendOptions.traceparent` makes `client.send` send the header. +- A `cancel` input stops a turn from any host that reads the same log, so `client.cancel({ operationId })` works behind a load balancer. A closed connection still never cancels work. To stop a `useChat` turn, call `client.cancel` with the `runId` of `useChat` in the stop handler. diff --git a/.changeset/ai-harness-view.md b/.changeset/ai-harness-view.md new file mode 100644 index 0000000000..0fdf013d0d --- /dev/null +++ b/.changeset/ai-harness-view.md @@ -0,0 +1,14 @@ +--- +'@tanstack/ai-harness': minor +--- + +Add `createSessionView()` in `@tanstack/ai-harness/view`. It turns a session (local, or remote through `createHarnessClient()`) into one live TanStack Store state that works with any UI library. + +- The state: messages with their parts, tool calls, approvals, client tools, other interrupts, questions, sign-ins, `waits` (every open wait, with its kind), child agents with their results and a `status` of `running` or `waiting`, media with signed URLs, waiting inputs, usage, models, settings, commands, and plugin state. +- The waits of background agent runs show in `approvals`, `clientTools`, and `interrupts`. The view answers each one at once: with `resolve`, or with `answer` for a `human` wait that no turn holds. +- Actions: `send` (a prompt, a steer, or a `/command`), `followUp`, `continueWithoutMessage`, `putMedia`, `cancel`, `cancelInput`, `changeInputDelivery`, `updateThreadSettings`, `reset`, `revert`, `loadAgent`, and the answers to approvals (with edited arguments), client tools, interrupts, and questions. `cancelInput`, `revert`, `approve({ id, ...options })`, `reject({ id, payload? })`, `notice({ text })`, and `putMedia` take one object, and so does the `reject({ payload? })` of an approval item. +- Typed events: `view.on('approval' | 'interrupt' | 'turnEnd' | ..., handler)` and plugin events. +- In a harness with `routing`, an assistant message has `agent`, the root agent that wrote it. After a handoff, the next agent writes a new message in the same turn. +- `view.send(text, { as: 'operator' })` sends the reply of an operator while a handoff to a person is open. The reply shows in each open view at once. +- A view that opens while a turn runs shows the turn from its start. After a reload, each child agent shows its card, and `view.loadAgent({ id })` fills it with its own messages. +- When the stream sends a `harness.gap` event, the view stops its read, loads the snapshot and the transcript again, then reads on from the new cursor. No turn shows twice. diff --git a/.changeset/ai-mcp-harness.md b/.changeset/ai-mcp-harness.md new file mode 100644 index 0000000000..38e67ecbc9 --- /dev/null +++ b/.changeset/ai-mcp-harness.md @@ -0,0 +1,14 @@ +--- +'@tanstack/ai-mcp': minor +--- + +Use MCP in a TanStack AI harness. + +- `@tanstack/ai-mcp/harness` adds `createHarnessMcpServer`. It serves a harness as an MCP server, so Claude Code, Claude Desktop, Cursor, or another agent can use it. The tools are `chat`, `steer`, `cancel`, `approve`, `reject`, `resolve`, `answer`, and `status`, plus `agent_` for each exposed agent and `command_` for each exposed command. Every tool takes an optional `threadId`. With `approvals: 'ask'` (the default), the client asks the user about each approval when it supports elicitation. Otherwise the approvals come back in the result. `approvals: 'auto'` approves every tool call. `resolve` answers every interrupt of a stopped turn in one call, or the waits of one background agent run. It finds the wait in the log, so it also answers a wait that another server, or a server before a restart, opened. A question takes `answer`. Every result lists the open waits of its work in `waits`, each with a `kind` from `waitKind` (`approval`, `question`, `sign-in`, `client-tool`, or `human`). The `status` tool lists every wait, and each background agent with `status: 'running'` or `'waiting'`. +- On a host with `access`, `createHarnessMcpServer` reads the user of each HTTP request with `getUser`. The host checks each tool call of that user. A host without `access` needs `openAccess: true` (no user) or `openAccess: { user }` (one local user), else the server throws at the start. It uses `requestUsers`, the same rule as the HTTP handler. Each tool call releases its session handle when it ends, so the session closes on idle. The `status` tool and the media reads read the thread with `host.observe`, so they set up no plugins. The server forgets a wait when its operation settles. +- `@tanstack/ai-mcp/harness` also adds the `mcp()` harness plugin. It connects stdio and HTTP servers by name, gives the model their tools as `_`, and adds a `/mcp` status command. A stdio server starts once per host for each `mcp()` object, and every session that uses that object shares it. `oauth: true` signs in to a server with `/connect `, for each user. +- `mcp()` connects again with backoff when a server closes, and lists the tools again on `list_changed`. A server that fails its first connect also tries again with backoff. `mcpConnector` and `oauth: true` servers connect again at the next turn, and list their tools again. Each server takes `needsApproval` and `toolName`. The backoff starts again only after a connection stayed open for 30 s, so a server that crashes right after it connects waits longer each time. When two servers give a tool with the same name, `mcp()` warns once. With `onDuplicateTool: 'error'`, the later server fails instead, with an error that names both servers and the tool. +- `@tanstack/ai-mcp/connector` adds `mcpConnector`: one OAuth MCP server as a harness plugin, with `needsApproval`, `prefix`, `scopes`, and `clientName`. The sign-in passes the `iss` of the callback on (RFC 9207). A cancel or a session close stops a running sign-in at once: the loopback stops listening, and the OAuth requests abort. When the MCP SDK refreshes a token, the connector saves it with `credentials.refresh` of the harness: to the scope of the sign-in, and with compare-and-set when the store has it. Each session still sends its own refresh request. When the server refuses a token, the connector deletes the saved sign-in only if it is the token that this session used. When the user signs out while the SDK refreshes, the connector saves no new token, and the call asks for a sign-in. +- `mcp()` and `mcpConnector` answer the `elicitation/create` requests of MCP spec 2025 servers, and the input requests of spec 2026-07-28 servers, as session questions. One tool call can ask up to 5 times. Each question is a live wait of the run that made the tool call: it ends `cancelled` when that run stops. `mcp()` tracks the live questions for each server, so each question goes to the run that called the tool. A spec 2025 request goes to the user only when all the running calls of that server belong to one run. Else the server gets `decline`. + +`@tanstack/ai-harness` is an optional peer dependency. Only the `./harness` and `./connector` entries need it. diff --git a/.changeset/ai-sandbox-harness.md b/.changeset/ai-sandbox-harness.md new file mode 100644 index 0000000000..a837603ec3 --- /dev/null +++ b/.changeset/ai-sandbox-harness.md @@ -0,0 +1,11 @@ +--- +'@tanstack/ai-sandbox': minor +--- + +Add `@tanstack/ai-sandbox/harness` for `@tanstack/ai-harness`: + +- `codingAgents({ sandbox, agents, workspace? })` is a harness plugin. The lead model gets one tool per coding agent (Claude Code, Codex, Grok Build, or any ACP agent). Each agent works in the sandbox, keeps its own session per thread (also after a restart), and starts read-only in the `plan` mode of `permissions()`. `/fresh [agent]` starts a new session. `workspace: 'shared'` (default) runs the agents of a thread one at a time in one sandbox. `'per-agent'` gives each agent its own sandbox. +- `codingAgents` runs agents only on the host that owns the log of the thread, so `'shared'` runs one agent at a time also with many hosts. +- `sandboxWorkspaceBackend(sandbox, { owned? })` runs the coding tools of `@tanstack/ai-harness/plugins/coding` (`workspaceTools()`, `snapshots()`, `formatter()`) in a sandbox. It resolves links in the sandbox, reads one page of a file in the sandbox, and stops background jobs by their id (the pid and its start time), also from another host. It never stops another process that got the pid of a job. `binding()` gives the provider and the sandbox id, so another host can resume the sandbox. `close()` stops the jobs of the backend. A session close closes it with `'moved'`, so an owned sandbox stays for the next host. `host.sessions.delete` closes it with `'closed'`, and with `owned: true` that destroys the sandbox. + +`@tanstack/ai-harness` is an optional peer dependency. diff --git a/.changeset/ai-subagent-binding-interrupts.md b/.changeset/ai-subagent-binding-interrupts.md new file mode 100644 index 0000000000..b7d9a69386 --- /dev/null +++ b/.changeset/ai-subagent-binding-interrupts.md @@ -0,0 +1,16 @@ +--- +'@tanstack/ai': minor +--- + +Add `interrupts` to `SubagentBinding`. + +- A host sets `binding.interrupts` to give interrupt definitions to every `ctx.chat` call of an agent run. Then the host middleware in `chatMiddleware` can stop the child with these interrupts. +- The own `interrupts` of the call win. When a host definition has the same id as one of the call, the host definition is not added. +- A child's `ctx.chat({ subagents })` passes the definitions on to its own children. +- A plain `chat()` call does not get them. + +Add `lastChatMiddleware` to `SubagentBinding`. It runs after the own middleware of each `ctx.chat` call of an agent run, so it sees the final model context and tool results. A child's `ctx.chat({ subagents })` passes it on, after the child's own. + +Add `validateJsonSchema(schema, value)`. It checks a value against a JSON Schema with `@cfworker/json-schema`, as a tool input is checked, but with no repair. The result has the Standard Schema shape: `{ value }` or `{ issues }`. The root also exports the `StandardSchemaV1` and `StandardJSONSchemaV1` types. + +An agent `run` that throws the shape of `MCPInputRequiredError` (`name`, `kind`, `request`) stops the child as `suspended`, as a tool does, and the parent run stops with its interrupt. The interrupt has a generic binding and no tool call. Its id is the `interruptId` of the thrown value, else `mcp_input_`, and its message is the error message. The resume runs the child again, and `ctx.resume` has the answer. diff --git a/.changeset/ai-tool-guard.md b/.changeset/ai-tool-guard.md new file mode 100644 index 0000000000..30087006ea --- /dev/null +++ b/.changeset/ai-tool-guard.md @@ -0,0 +1,11 @@ +--- +'@tanstack/ai': minor +--- + +Add the `onGuardToolCall` middleware hook for checks that must always run, such as permissions. + +- It runs for every tool call, after all `onBeforeToolCall` hooks and after the `inputSchema` check. `hookCtx.args` holds the final checked arguments that the tool gets. +- The guard of every middleware runs, in order, until one returns a decision. An earlier `onBeforeToolCall` decision does not skip it. +- Return `{ type: 'skip', result }` to deny the call. The tool does not run, and the model gets `result`, the same as an `onBeforeToolCall` skip. Return `{ type: 'abort', reason }` to abort the run. +- It runs before a server tool runs and before a client tool goes to the client. A guard cannot change the arguments. +- The new `GuardToolCallDecision` type is exported. diff --git a/.changeset/ai-utils-command-env.md b/.changeset/ai-utils-command-env.md new file mode 100644 index 0000000000..86516cd423 --- /dev/null +++ b/.changeset/ai-utils-command-env.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-utils': minor +--- + +Add `commandEnv(env, platform, passEnv?)`, the short env of a command that a process starts (no provider keys or other secrets of the server), and `createOutputTail(maxBytes)`, the newest output of a command. `@tanstack/ai-harness`, `@tanstack/ai-sandbox`, and `@tanstack/ai-sandbox-local-process` use them. diff --git a/.changeset/ai-websocket-buffered-amount.md b/.changeset/ai-websocket-buffered-amount.md new file mode 100644 index 0000000000..5391cc79e7 --- /dev/null +++ b/.changeset/ai-websocket-buffered-amount.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai': patch +--- + +`WebSocketLike` has an optional `bufferedAmount`. A server can read it to stop sending to a slow socket. A socket without it works as before. diff --git a/.changeset/approval-raw-edits-resume.md b/.changeset/approval-raw-edits-resume.md new file mode 100644 index 0000000000..305d549292 --- /dev/null +++ b/.changeset/approval-raw-edits-resume.md @@ -0,0 +1,9 @@ +--- +'@tanstack/ai': minor +'@tanstack/ai-persistence': patch +--- + +Approved edits to tool arguments stay raw until the final input check. + +- `@tanstack/ai`: a resume no longer rejects approved `editedArgs` that do not fit the tool's `inputSchema`. The edit goes through `onBeforeToolCall` and the final input check, like the model's arguments. If it still does not fit, the approval is used, the tool does not run, and the model gets the input error as the tool result. The transcript keeps the approved edit as it was sent, but only when the call passed the check. +- `@tanstack/ai-persistence`: `withPersistence` binds the client-tool phase of an approved tool call to its approval. After a restart, the client result resumes with the approval that came before it, and a stale or missing approval fails the resume. diff --git a/.changeset/byok-credential-store.md b/.changeset/byok-credential-store.md new file mode 100644 index 0000000000..2b3af2222c --- /dev/null +++ b/.changeset/byok-credential-store.md @@ -0,0 +1,20 @@ +--- +'@tanstack/ai': minor +--- + +BYOK keys can now live on the server. `@tanstack/ai/byok` exports the `CredentialStore` contract, the `Credential` and `CredentialScope` types, and `defineCredentialStore`. A `CredentialScope` has `tenantId` and `userId`. Its `threadId` is optional, and a store ignores it. `@tanstack/ai/byok/server` adds: + +- `storedProviderKeys(store, scope)`: provider keys from the `api_key` credentials of a user, then of the tenant, then the provider's `env` names. Pass `keys.adapter(...)` to `chat()` and `keys.get` to `subagents.keys`. +- `readCredential(store, scope, id)`: one credential, the user's own first, then the tenant's. A user with no tenant reads only their own. +- `tenantScopeOf(scope)`: the tenant scope of a user scope, or `undefined` when the scope has no user or no tenant. +- `memoryCredentialStore()`: a store in memory, for development and tests. +- `envKeyEntry(provider)`: the first `provider.env` name that is set, with its value. +- `readCredentialEntry(store, scope, id)`: reads a credential like `readCredential`. It also tells you which scope had the credential (the user or the tenant), and gives its revision when the store has `getVersioned`. + +`CredentialStore` has two optional methods for compare-and-set writes, `getVersioned` and `setIf`, with the same shape as the pair on `MetadataStore`. A store without them still works. `memoryCredentialStore` has both. + +Keep the store in your own database or in a secrets manager, and encrypt the secrets at rest. An `oauth` credential can keep the OAuth `client` it was issued to. `list` never returns secret values. + +`scrubSecrets` from `@tanstack/ai/byok` now also removes the JSON-escaped and URL-encoded forms of each secret. It skips a secret that is shorter than 6 characters, so a short value does not remove common words. + +`Credential` has a new type, `secret_answer` (`{ value, expiresAt }`). The harness keeps a secret answer to a question there for a short time and deletes it once read. Leave this type out of a list of connected accounts. `memoryCredentialStore().list` returns its `expiresAt`. diff --git a/.changeset/chat-resume-should-continue.md b/.changeset/chat-resume-should-continue.md new file mode 100644 index 0000000000..9179d7e862 --- /dev/null +++ b/.changeset/chat-resume-should-continue.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai': patch +--- + +`chat()` asks the middleware `onShouldContinue` after it runs the pending tool calls of a resume, before the next model call. A middleware that returns `false` there ends the run with no more model calls. The loop strategy is not asked at that point. diff --git a/.changeset/code-mode-harness.md b/.changeset/code-mode-harness.md new file mode 100644 index 0000000000..d4d780073b --- /dev/null +++ b/.changeset/code-mode-harness.md @@ -0,0 +1,14 @@ +--- +'@tanstack/ai-code-mode': minor +--- + +Add `@tanstack/ai-code-mode/harness` with `codeMode({ driver })`: a harness plugin that gives the model one `execute_typescript` tool. The model writes a program that calls several tools, and the program runs in the isolate of any `@tanstack/ai-isolate-*` driver. + +- By default, only tools that are safe to run without a question move into code mode. These are server tools with no `needsApproval` and no permission resources, which the permission rules allow in plan mode. A safe tool with `metadata.codeMode: true` always moves. +- `include` picks more tools, also tools with permission checks, such as the coding tools `read_file`, `write_file`, `edit_file`, `patch`, `list_files`, `grep`, `bash`, and `webfetch`. Each call of such a tool in a program gets the same decision from `permissions()` as a direct call. The call runs, or it throws an error with the reason, or the program waits for the user to approve it. The approval is a question wait that `session.answer` answers with `once`, `always`, or `reject`. A program that can wait gets a timeout of 10 minutes by default. +- The session log keeps the result of each checked call. After a stop, the next host runs the program again, and a call that finished before the stop does not run twice. A call that the stop cut, and that is not safe to run again, throws in the program. +- `include` never moves a tool that needs approval, or a tool that the rules deny in the current permission mode. The rules decide again at each turn, so in plan mode the tools that edit stay normal tool calls. +- `directCalls: 'keep'` keeps the moved tools in the model's tool list too. The session setting `directCalls` changes it while the session runs. +- `lazy: true` lists only the tool names and adds `discover_tools`. A tool name that is not a JavaScript identifier gets a safe name inside the program. + +`@tanstack/ai-harness` is an optional peer dependency. diff --git a/.changeset/code-mode-result-shape.md b/.changeset/code-mode-result-shape.md new file mode 100644 index 0000000000..98673986c6 --- /dev/null +++ b/.changeset/code-mode-result-shape.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-code-mode': patch +--- + +The `execute_typescript` tool description and the code-mode system prompt now tell the model that the tool returns `{ success, result, logs, error }`, where `result` is the value its code returned. diff --git a/.changeset/computer-use-harness.md b/.changeset/computer-use-harness.md new file mode 100644 index 0000000000..e283670bcb --- /dev/null +++ b/.changeset/computer-use-harness.md @@ -0,0 +1,15 @@ +--- +'@tanstack/ai-computer-use': minor +'@tanstack/ai-harness': minor +--- + +Add `computerUse()` in `@tanstack/ai-computer-use/harness`: computer use as a harness plugin. + +- Each screenshot goes to the media store of the session. The transcript and the log keep a small `harness-media:` reference. +- The model gets only the newest `keepScreenshots` screenshots (3 by default). An older one goes to the model as a 1x1 image. +- A call with OpenAI safety checks asks the user through `permissions()`, each time. Without `permissions()`, the call is refused. +- The `plan` mode of `permissions()` refuses the `computer` tool. + +`@tanstack/ai-harness` is an optional peer dependency of `@tanstack/ai-computer-use`. + +A plugin can now store a file with `ctx.session.putMedia(upload)`, and put `mediaPart(record)` in a tool result. diff --git a/.changeset/docker-local-source.md b/.changeset/docker-local-source.md new file mode 100644 index 0000000000..38e9db057d --- /dev/null +++ b/.changeset/docker-local-source.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-sandbox-docker': minor +--- + +`dockerSandbox()` now copies a `localSource(dir)` folder into a new container: the files that git lists, their uncommitted changes, and the history of the current branch. The folder must be a Git repository, and the image must have git (for example `node:24`). diff --git a/.changeset/docker-resume-errors.md b/.changeset/docker-resume-errors.md new file mode 100644 index 0000000000..354b453875 --- /dev/null +++ b/.changeset/docker-resume-errors.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-sandbox-docker': patch +--- + +`resume()` of `dockerSandbox()` gives `null` only for a container that Docker does not have. Any other error, such as Docker not running, now goes to the caller, so a user sees "Docker is not running" and not "the workspace is gone". diff --git a/.changeset/elevenlabs-sound-v3.md b/.changeset/elevenlabs-sound-v3.md new file mode 100644 index 0000000000..659c1aab53 --- /dev/null +++ b/.changeset/elevenlabs-sound-v3.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-elevenlabs': patch +--- + +The audio adapter accepts `eleven_text_to_sound_v3`, a sound model that the ElevenLabs SDK types do not list yet. diff --git a/.changeset/harness-acp-agent.md b/.changeset/harness-acp-agent.md new file mode 100644 index 0000000000..e8684a7229 --- /dev/null +++ b/.changeset/harness-acp-agent.md @@ -0,0 +1,7 @@ +--- +'@tanstack/ai-acp': minor +--- + +`@tanstack/ai-acp/agent` adds `createAcpAgent` and `serveAcp`, which serve a harness as an ACP v2 agent (experimental, like the draft protocol). Each wait of a turn goes to the editor by its kind: an approval becomes a permission request (with `allow_always` only when the answer schema takes `always`), a question or a `human` wait becomes a form elicitation, and a sign-in becomes a URL elicitation, when the editor declares them in `initialize`. A wait that the editor cannot answer is cancelled. Cancel cancels the running turn, and image, audio, and embedded file blocks go to the media store of the session. A prompt takes its place in the queue before its media is saved, and gives the place back when the session is idle. `session/list` lists the sessions of the harness from the session index of the host. The `user` option is the user of the editor: each session opens as that user, and the list shows the sessions of that user. It is required on a host with `access`. The agent holds one session handle per ACP session. `session/close` releases it, and the close of the connection releases them all (a session whose prompt runs keeps its handle until that prompt ends), so the host closes each session on idle. `@tanstack/ai-acp` now uses `@agentclientprotocol/sdk` 1.8, pinned to `~1.8.0` because the agent uses its experimental v2 entry point. The existing ACP client adapters keep working. + +One agent app serves one connection, and reads the editor capabilities from that connection. A second connection gets a clear error. For more connections, make one app for each: `new AcpServer({ createAgent: () => createAcpAgent(options) })`. A session that the agent made or opened in its connection takes prompts after the agent released its handle, also on a host with no `stores.sessions`. diff --git a/.changeset/harness-build-worker.md b/.changeset/harness-build-worker.md new file mode 100644 index 0000000000..490e722096 --- /dev/null +++ b/.changeset/harness-build-worker.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': minor +--- + +`@tanstack/ai-harness/build` and `@tanstack/ai-harness/worker`: `buildHarness` bundles a harness with Bun into a worker artifact (`harness.js` and a `harness.manifest.json` with the sha256 digest), and can also build a single executable. `artifactText(dir)` checks the digest and uses the artifact as the model of a `chat()` call, with one worker process per thread. `runHarnessWorker` runs a harness on stdin and stdout with the session frames of the harness protocol. diff --git a/.changeset/harness-credentials-option.md b/.changeset/harness-credentials-option.md new file mode 100644 index 0000000000..4e0ddb7ef9 --- /dev/null +++ b/.changeset/harness-credentials-option.md @@ -0,0 +1,6 @@ +--- +'@tanstack/ai-harness': minor +'@tanstack/ai-harness-cli': minor +--- + +`createHarnessHost({ credentials })` takes the `CredentialStore` (from `@tanstack/ai/byok`) where users' API keys and OAuth sign-ins are saved. Without it, credentials stay in memory until the process stops. `runCli` takes the same `credentials` option. diff --git a/.changeset/harness-define-plugin-keys.md b/.changeset/harness-define-plugin-keys.md new file mode 100644 index 0000000000..91b6ef15d7 --- /dev/null +++ b/.changeset/harness-define-plugin-keys.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': patch +--- + +`definePlugin` refuses a `setup()` contribution, such as `adapter` or `tools`, at the top level of a definition, where it would be ignored. Return it from `setup()`. diff --git a/.changeset/harness-file-changed.md b/.changeset/harness-file-changed.md new file mode 100644 index 0000000000..12806d03a9 --- /dev/null +++ b/.changeset/harness-file-changed.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': minor +--- + +`write_file`, `edit_file`, and `patch` send `FileChanged` (`tanstack/workspace-tools:file-changed`) with a unified patch and its line counts for each file that they change. At the end of each model step, `snapshots()` sends it for the files that other tools, like `bash`, changed: at most 100 for each step. `/undo`, `/redo`, `session.revert()`, and `session.unrevert()` also send it for each file that they put back. diff --git a/.changeset/harness-join-any-instance.md b/.changeset/harness-join-any-instance.md new file mode 100644 index 0000000000..dc47ad6053 --- /dev/null +++ b/.changeset/harness-join-any-instance.md @@ -0,0 +1,9 @@ +--- +'@tanstack/ai-harness': patch +--- + +Any server instance that reads the same log store can serve a run. + +- `POST run` sends the log cursor of each event as its SSE `id`. A client that is cut off sends the same request with `Last-Event-ID`, and any instance continues the stream from there. The turn does not run again. +- `GET run?runId=&threadId=` finds a turn that another instance runs. It streams the turn from the log of `threadId`, from its start or from `Last-Event-ID`, until the turn ends. It reads the log from the start of the turn, not from the start of the log. A turn rebuilt after a restart (a retried `POST run`) also streams from its start. Without `threadId`, only the instance that runs the turn finds it, and another instance answers `400`. The join request of a `ChatClient` sends only the run id, so put `threadId` in the URL of the connection. +- `GET run?threadId=` shows a turn that another instance runs as `activeRun`, so a reloaded `useChat` joins it. diff --git a/.changeset/harness-log-newer-records.md b/.changeset/harness-log-newer-records.md new file mode 100644 index 0000000000..671dc6ca55 --- /dev/null +++ b/.changeset/harness-log-newer-records.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': patch +--- + +A host that folds a `harness.*` log record type it does not know (a newer harness wrote it, for example during a rolling deploy) reads the log but refuses to write it, with `LogVersionError`. A log snapshot from a newer harness is not used. diff --git a/.changeset/harness-logger.md b/.changeset/harness-logger.md new file mode 100644 index 0000000000..68f354055b --- /dev/null +++ b/.changeset/harness-logger.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': minor +--- + +`createHarnessHost({ logger })` takes a `Logger` from `@tanstack/ai` for the warnings and errors of the harness and its `chat()` calls (default: `ConsoleLogger`). Plugins get it as `ctx.logger`, and a logger that throws does not stop the harness. diff --git a/.changeset/harness-permission-call.md b/.changeset/harness-permission-call.md new file mode 100644 index 0000000000..51ed715dd8 --- /dev/null +++ b/.changeset/harness-permission-call.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': minor +--- + +`PermissionCallCapability` gives a plugin that runs tool calls itself, like code mode, the decision that `permissions()` makes for one call: `allow`, `deny` with the error that the model gets, or `ask` with the approval question. It uses the current mode, the saved `always` answers, and the resources of the call args, and `settle(reply)` applies the user's answer as a direct approval does. diff --git a/.changeset/harness-plugin-logger.md b/.changeset/harness-plugin-logger.md new file mode 100644 index 0000000000..5ed7539ea9 --- /dev/null +++ b/.changeset/harness-plugin-logger.md @@ -0,0 +1,7 @@ +--- +'@tanstack/ai-mcp': patch +'@tanstack/ai-code-mode': patch +'@tanstack/ai-skills': patch +--- + +The `mcp()`, `codeMode()`, and `skills()` harness plugins send their warnings to the logger of the host (`createHarnessHost({ logger })`). diff --git a/.changeset/harness-redaction.md b/.changeset/harness-redaction.md new file mode 100644 index 0000000000..3db1857a62 --- /dev/null +++ b/.changeset/harness-redaction.md @@ -0,0 +1,12 @@ +--- +'@tanstack/ai-harness': minor +--- + +Keep the secrets of a session out of what it keeps and sends. + +- Each session replaces every secret that it gave out with `[redacted]`. These are the values of `run.keys` (also from env vars) and `run.credentials`, connector credentials and OAuth tokens, and secret answers (each string field of an answer with fields). It scrubs log records, feed events, plugin state (also in a `Map` or a `Set`), notes, the model context, and tool results, also after the own middleware of an agent run. It also scrubs the error frames of the HTTP handler, the WebSocket, and the stdio worker. It finds the raw, JSON-escaped, and URL-encoded forms, and skips values shorter than 6 characters. A secret split across two streaming deltas stays in those two events, and a secret that is JSON-escaped two times is not found. +- `scrubSecrets` from `@tanstack/ai-harness` is the one from `@tanstack/ai/byok`. +- A `step.do` value that holds a secret of the session throws, on the first run and on a replay, so the log never keeps a secret. Read the secret outside `step.do`. +- A retry of an input with a secret in it is a duplicate, not a conflict: the check compares the input without the secrets, as the log keeps it. +- The log keeps the final result of each tool call: the result after every `onAfterToolCall` hook (also the agent's own middleware in an agent run), scrubbed. A replacement from a hook wins, also on an error. Recovery after a crash gives the model that result. +- An answer to a `secret: true` question that reaches a host that does not own the log goes through the `credentials` store of the host, as a one-time reference with the credential type `secret_answer` and an `expiresAt` 24 hours later. Only the reference goes into the input lane, and the owner reads and deletes it. The references that nobody read go when the question ends. A kept answer for a run again goes when the run ends. Without `createHarnessHost({ credentials })`, the answer is rejected with `secret_answer_needs_credentials`. A used, expired, or unknown reference is rejected with `unknown_secret_answer`. `credentials.list` does not show the `secret_answer` type. diff --git a/.changeset/harness-round5-asks.md b/.changeset/harness-round5-asks.md new file mode 100644 index 0000000000..5bf51455c8 --- /dev/null +++ b/.changeset/harness-round5-asks.md @@ -0,0 +1,23 @@ +--- +'@tanstack/ai': minor +'@tanstack/ai-compaction': minor +'@tanstack/ai-harness': minor +'@tanstack/ai-persistence': minor +'@tanstack/ai-sandbox': minor +--- + +Add the harness asks from the fifth findings list. + +- `chat({ outputSchema, stream: true })` now checks the full result against a Standard Schema. A failure ends with `RUN_ERROR` `structured-output-validation-failed`, with the model text in `metadata.tanstack.rawText`. +- `session.send(prompt, { outputSchema })` gives the turn a typed `structured` value. It is per send only, and the HTTP handler does not take it. +- A server tool reads run capabilities with `context.get(capability)` and `context.getOptional(capability)`, for example the sandbox under `withSandbox`. +- Readers keep the full history after a durable compaction. `session.transcript()`, `GET transcript`, and the chat hydration show every message. The model still gets the compacted context. +- `createHarnessHost({ acceptInput })` sees each input after the access check and before the log keeps it. It can change the input or refuse it. +- A plugin `adapter()` pick with `override: true` wins over the stored thread model. +- `onModelError` can return `{ retry: true, adapter }` to retry on another model. `ModelErrorContext` has `adapter` and `model`. +- Middleware reads `ctx.context.run.source` and `label` to tell a turn call from a plugin, router, or transfer call. +- `host.sessions.archive()` and `unarchive()`. Lists hide archived threads unless you pass `withArchived: true`. +- `host.sessions.eraseUser({ harness, threadId, userId, replies })` replaces one person's messages with `[removed]`, deletes their media, and cuts the log before a new snapshot. The new `erase` access action guards it. +- `HarnessRecord` is exported, for a typed `LogStore` wrapper. +- Coding-tool file calls pass their abort signal through `WorkspaceBackend` to the sandbox. +- `withCompaction` limits (`maxTokens`, `contextWindow`, `background.atTokens`) can be a function of `{ model }`. diff --git a/.changeset/harness-round5-fixes.md b/.changeset/harness-round5-fixes.md new file mode 100644 index 0000000000..0829bfd354 --- /dev/null +++ b/.changeset/harness-round5-fixes.md @@ -0,0 +1,43 @@ +--- +'@tanstack/ai': patch +'@tanstack/ai-anthropic': patch +'@tanstack/ai-client': patch +'@tanstack/ai-cloudflare': patch +'@tanstack/ai-compaction': patch +'@tanstack/ai-event-client': patch +'@tanstack/ai-harness': patch +'@tanstack/ai-persistence': patch +'@tanstack/ai-sandbox': patch +'@tanstack/ai-sandbox-blaxel': patch +'@tanstack/ai-sandbox-boxd': patch +'@tanstack/ai-sandbox-cloudflare': patch +'@tanstack/ai-sandbox-daytona': patch +'@tanstack/ai-sandbox-docker': patch +'@tanstack/ai-sandbox-e2b': patch +'@tanstack/ai-sandbox-local-process': patch +'@tanstack/ai-sandbox-railway': patch +'@tanstack/ai-sandbox-sprites': patch +'@tanstack/ai-sandbox-upstash-box': patch +'@tanstack/ai-sandbox-vercel': patch +'@tanstack/openai-base': patch +--- + +Fix recovery, cancel, steer, and provider bugs, and fill small harness gaps. + +- Recovery: a resolve that another host recovers runs its continuation. Before, it settled `completed` with no answer. +- Approvals: a tool that a middleware adds in `onConfig` can now be approved. `chat()` checks a durable resume after the whole `onConfig` chain. +- Cancel: a cancelled turn keeps the text it streamed, and ends with `TEXT_MESSAGE_END` and `RUN_FINISHED` (`outcome: cancelled`). A turn that times out ends with `RUN_ERROR` (`code: 'timeout'`). +- Steer: a steer from another sender gets a `queued` receipt with its own operation. `host.events()` shows `idle` after a turn that a steer joined. +- The duplicate-input check no longer depends on the key order of the stored input. +- `TurnInfo` has `model`, `attempt`, and `sender`. The log keeps the sender's `name`. `harness.input.accepted` carries the message. The chat hydration has `pendingWaits`. `ObservedSession.settled()` reads a settlement with no open session. Another host's `snapshot().plugins` shows the saved plugin state. +- New options: `subagents.background: false`, `media.urlTtlMs`, `conversationSummarizer({ middleware })`, and the `beforeWrite` workspace hook, which can refuse a write. +- Plugin model calls (`ctx.model.chat`) run the harness middleware. +- Exports: `SHUTDOWN_REASON`, `BUDGET_REASON`, and `RUN_CANCEL_REASON` from `@tanstack/ai-harness`, `HARNESS_EVENTS` from `@tanstack/ai-harness/client`, and `ChatHydrateOptions` and `ChatHydrationResult` from `@tanstack/ai-client`. +- Compaction drops a background summary that another model prepared. +- Sandbox file calls take an optional `{ signal }`. +- Anthropic: `outputSchema` moves keywords that Anthropic rejects into `description`, usage reports 1-hour cache writes as `cacheWrite1hTokens` again, and `reasoning` on a model with no reasoning entry warns once. +- OpenAI Responses: a message whose model call sent a reasoning item is marked with `metadata.tanstack.reasoningItem`. A replay of that message without its reasoning item no longer sends its `msg_` and `fc_` ids. +- Cloudflare: a `data:` line with no `choices` no longer throws, and the SSE fix-up accepts `data:` with no space, CRLF line ends, and any case in the content type. +- Structured output: a schema failure carries `rawText`, and a parse failure has the code `structured-output-parse-failed`. +- `ChatClient.addToolResult` answers the interrupt when the tool's schemas differ, so the result is sent. +- The persistence conformance suite no longer fails a correct store on a slow runner. diff --git a/.changeset/harness-secure-defaults.md b/.changeset/harness-secure-defaults.md new file mode 100644 index 0000000000..6f187ee8e7 --- /dev/null +++ b/.changeset/harness-secure-defaults.md @@ -0,0 +1,10 @@ +--- +'@tanstack/ai-harness': patch +--- + +Make the transport defaults safe for a server with many instances. + +- `createHarnessHandler` and `handleHarnessSocket` read the user of each request with the host `access`. On a host without `access`, pass `openAccess: true` (no user) or `openAccess: { user }` (one local user), else they throw at the start. So a server never serves every thread to everyone by mistake. +- `media: { secret }` signs the media URLs of `createHarnessHandler`. Set the same secret on every instance, so a media URL that one instance signs works on every instance. Without it, the handler signs with a random key of its own, and a URL works only on that instance. A host with `ownership` options needs `media`, else the handler throws at the start. On another host without `media`, the first handler logs one warning. A signed URL names its harness and its user, and the handler checks that user again when it serves the file. A URL of a deleted thread answers 404. +- `media: { keys: [newKey, oldKey] }` rotates the key: the first key signs, and every key checks. Each key must be at least 32 bytes, else the handler throws at the start. +- `oauthConnector` needs `login`. Use `'device'` on a server, and `'loopback'` only for a harness on the user's own machine, such as a CLI. diff --git a/.changeset/harness-skills.md b/.changeset/harness-skills.md new file mode 100644 index 0000000000..365b90e0c1 --- /dev/null +++ b/.changeset/harness-skills.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-skills': minor +--- + +`@tanstack/ai-skills/harness`: the new `skills({ dirs })` plugin gives the harness model the skills of a list of folders, and makes each skill a `/` command that starts a turn with that skill. A skill with the name of another command is `/skill:`, and `/skills` lists them. The plugin watches the folders, so a skill that you add or remove while the session runs changes the commands at once. The watcher lives as long as the host, every session that uses the same `skills()` object shares it, and a change makes each session read the folders again. A skill that does not load gets a fallback message. diff --git a/.changeset/harness-tool-ops.md b/.changeset/harness-tool-ops.md new file mode 100644 index 0000000000..2ddc4d7494 --- /dev/null +++ b/.changeset/harness-tool-ops.md @@ -0,0 +1,35 @@ +--- +'@tanstack/ai': minor +'@tanstack/ai-harness': minor +'@tanstack/ai-mcp': minor +--- + +Run a harness tool with no model turn, add host records from a client, and add a per-run system preamble. + +- `session.runTool({ name, args?, meta?, inputId?, user? })` runs one harness tool with no model + turn, for example from a schedule or a webhook. The call goes through the + same middleware, input check, and `permissions()` rules as a model call, and + the call and its result join the transcript. A tool that asks for approval + stops the turn with an interrupt. While interrupts wait, the call is refused + with `pending_interrupts`. `meta` is echoed as a `tanstack.injection` custom + event. `args` is typed from the input schema of a harness tool. +- `expose.tools` lists the harness tools a client may run with a + `{ op: 'tool', name, args?, meta? }` input, or with the new + `client.runTool({ name, args?, meta?, inputId? })`. The list is typed from the harness + `tools`, and `defineHarness` throws for a name that is not in `tools`. Any + other tool is refused with `not_exposed`. The capabilities document marks + each tool with `exposed`. +- `expose.records` lists the host record types a client may add with an + `{ op: 'append', records }` input, or with the new `client.appendRecords({ records })`. + A record of another type, or with a `thread` field, is refused with + `not_exposed`. The session log keeps the records. +- `createHarnessMcpServer` adds a `tool_` tool for each tool in + `expose.tools`, and an `append` tool when `expose.records` has types. A tool + input with its own `threadId` field throws when the server is created, + because MCP uses `threadId` to pick the thread. +- `session.send(message, { systemPreamble })` prepends per-run, server-only + system prompts ahead of the harness's own system prompts (for example + host-provided memory). A client input cannot set them. +- A tool call in a harness turn gets `threadId` and `runId` in its execution + context. `ToolExecutionContext` in `@tanstack/ai` has the two new optional + fields for a host to set. diff --git a/.changeset/harness-turn-and-body-limits.md b/.changeset/harness-turn-and-body-limits.md new file mode 100644 index 0000000000..e1c3938cc5 --- /dev/null +++ b/.changeset/harness-turn-and-body-limits.md @@ -0,0 +1,14 @@ +--- +'@tanstack/ai-harness': patch +--- + +How a turn takes steers, settings, and plugin removals, and the size limit of client requests. + +- A steer that arrives during a `session.runTool()` turn waits, and then runs as its own turn. +- A turn keeps the config values that it started with. A config change while it runs, for example `/mode bypass`, applies from the next turn. +- `configure({ settings: { plugins: { remove } } })` removes everything that those plugins give the thread: also their agents, subagents, commands, agent and generation middleware, and the items that they add for other plugins (for example the permission rules of `agents()`). A remove that leaves a plugin (of any lifetime) or a harness middleware without a capability that it needs is rejected. +- When two plugins give a tool with the same name from `discoverTools`, the first one stays, and clients get one `harness.plugin.warning` event that names both plugins. +- `createHarnessHandler` reads each JSON body up to `media.maxBodyBytes` (default 32 MiB). A bigger body answers 413. `handleHarnessSocket` closes the socket with code 1009 on a bigger frame. The WebSocket server reads the whole frame first, so also set its own frame limit (for example `maxPayload` of `ws`). +- Client media is checked part by part before any part is stored, so a refused input stores nothing. A retried input with the same `inputId` stores its media no second time, and gets the first receipt. +- A client input with a provider file handle (`source.type: 'file'`) is refused with `media_url_not_allowed` unless the harness sets `media.allowProviderFiles: true`. Server code can still send file handles. +- `POST run` moves the inline bytes of the last user message to the media store, as `POST control` does, so the transcript keeps a `harness-media:` reference. A refused part answers 413 (`media_too_large`), 415 (`media_not_allowed`), or 400 (`media_invalid`, `media_url_not_allowed`). diff --git a/.changeset/harness-view-experimental.md b/.changeset/harness-view-experimental.md new file mode 100644 index 0000000000..700a912b5c --- /dev/null +++ b/.changeset/harness-view-experimental.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-harness': patch +--- + +Mark `createSessionView`, `SessionViewState`, and `SessionViewEvents` as `@experimental`. Document that event cursors are opaque strings: positions in the session log. diff --git a/.changeset/nested-subagent-resume.md b/.changeset/nested-subagent-resume.md new file mode 100644 index 0000000000..a948686be9 --- /dev/null +++ b/.changeset/nested-subagent-resume.md @@ -0,0 +1,6 @@ +--- +'@tanstack/ai': patch +'@tanstack/ai-persistence': patch +--- + +A nested subagent that waits on an interrupt now continues after the resume under `withPersistence`. Before, it ran again from its prompt. Core rebuilds the card of the nested child from the store and takes only its id from the client card. `LoadChild` takes an optional `parentSubagentRunId`, and returns `interruptIds` and `parentToolCallId`. diff --git a/.changeset/persistence-routed-host-after-tool-call.md b/.changeset/persistence-routed-host-after-tool-call.md new file mode 100644 index 0000000000..b176911c9b --- /dev/null +++ b/.changeset/persistence-routed-host-after-tool-call.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-persistence': patch +--- + +Keep the answer of a routed agent in its own message when the turn resumes after an interrupt and a finished tool call (for example a handoff) comes before that answer. Before, the stored answer moved onto the message of the tool call. diff --git a/.changeset/persistence-session-stores.md b/.changeset/persistence-session-stores.md new file mode 100644 index 0000000000..470f8f1c16 --- /dev/null +++ b/.changeset/persistence-session-stores.md @@ -0,0 +1,14 @@ +--- +'@tanstack/ai-persistence': minor +--- + +Add opt-in session stores. The chat and generation middleware work as before without them. + +- `log` (`LogStore`): an append-only log for each session tree, with 15 methods, each with one object argument. Records: `append`, `read`, and `subscribe`. Ownership: `acquireOwnership`, `renewOwnership`, `releaseOwnership`, `pinToWorkers`, `markHostAlive`, and `listLogsWithoutOwner`. A pin (`pinToWorkers`) counts only while its host is alive: `acquireOwnership` and `listLogsWithoutOwner` ignore the affinity of a host with no live `heartbeat({ hostId, ttlMs })`, so the logs of a host that stopped go to other hosts. A host that is alive keeps its pins, also on the logs that it released. The input lane: `offerInput` and `listOfferedInputs`, consumed in the same write with `append({ consumes })`. The owner or the first offer names the harness of a log for `listLogsWithoutOwner`: a later offer does not change it. Snapshots and lifecycle: `getSnapshot`, `putSnapshot`, `truncate`, and `deleteLog`. `acquireOwnership` returns `expired: true` when it takes the log from another owner whose ownership expired with no release. Each fenced write carries the epoch from `acquireOwnership`, and a write with an old epoch rejects with `OwnershipLostError`. A writer that never calls `acquireOwnership` appends with `NO_OWNER_EPOCH`. `pinToWorkers`, `putSnapshot`, and `truncate` need the owner, so they refuse `NO_OWNER_EPOCH`. After `deleteLog`, every fenced write fails until the next `acquireOwnership`, and `offerInput` returns the new `OfferResult` `'deleted'` and stores nothing, so a late write or input cannot bring a deleted log back. `append` rejects with `LogConflictError` when another writer took the position, and `read` rejects with `LogTruncatedError` before the truncation point. `memoryLogStore()` is the reference store. `memoryLogStore({ state })` with one `memoryLogState()` lets two store instances share one log. The conformance kit needs `twin`, a second store set with a new log store instance on the same backend, for the `log` cases across instances. +- `sessions` (`SessionIndexStore`): one index entry per tenant and thread. The key is `{ tenantId?, threadId }` (`SessionIndexKey`), so one thread id in two tenants is two entries. An entry has `members` (`SessionMember`, with the role `owner`, `member`, `viewer`, or `operator`). The owner is the member with the role `owner`. The store has `get(key)`, `getVersioned(key)`, `upsert(entry)`, `upsertIf({ entry, revision })` for compare-and-set writes, `list`, and `delete(key)`, a hard delete. `list` reads one tenant, filters by member, parent thread, title `search`, `harness` name, and exact `metadata` values, and pages with a cursor. + +`defineLogStore` and `defineSessionIndexStore` type an adapter. `memoryPersistence()` now also has `log` and `sessions`. `runPersistenceConformance` runs the cases of each session store when the store is present, so an existing backend needs no `skip` entry. + +With `stores.sessions`, `withPersistence` writes one index entry for each subagent child (`subagent:`), in the tenant of its `tenantId` option. The entry has the parent thread, the tool call, and the `harness` name of the parent entry. It has no members: access to a child goes through its top-level thread. + +`loadChildTranscript(messages, subagentRunId, threadId)` reads the stored messages of a child agent run for a UI. It returns `undefined` for a child of another thread. diff --git a/.changeset/persistence-subagent-interrupts.md b/.changeset/persistence-subagent-interrupts.md new file mode 100644 index 0000000000..7af790d480 --- /dev/null +++ b/.changeset/persistence-subagent-interrupts.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-persistence': patch +--- + +`withPersistence` lets a subagent that a tool call started check the answers to its own interrupts. The parent run does not check them against its own tools and batch, so a child client tool, and two children that wait at once, resume. diff --git a/.changeset/sandbox-local-process-env.md b/.changeset/sandbox-local-process-env.md new file mode 100644 index 0000000000..ca17bde66c --- /dev/null +++ b/.changeset/sandbox-local-process-env.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai-sandbox-local-process': minor +--- + +A command in `localProcessSandbox` gets only a few variables of the host process: `PATH`, `HOME`, `LANG`, `LC_ALL`, `TMPDIR`, `TEMP`, `TMP`, and on Windows `SystemRoot`, `ComSpec`, `PATHEXT`, `WINDIR`, and `USERPROFILE`. The workspace secrets and the env of the call still reach it. Provider keys and other secrets of the server do not. If a command needs another variable of the host, set it as a workspace secret or in the env of the call. `passEnv` names more variables of the host for each command, for example `HTTPS_PROXY` or `CLAUDE_CONFIG_DIR`. `scrubEnv` still removes a variable from the env of each command. diff --git a/.changeset/subagent-binding-debug.md b/.changeset/subagent-binding-debug.md new file mode 100644 index 0000000000..89532d73b7 --- /dev/null +++ b/.changeset/subagent-binding-debug.md @@ -0,0 +1,5 @@ +--- +'@tanstack/ai': patch +--- + +Add `SubagentBinding.debug`, the `debug` option for the model calls of child agents that set no `debug` of their own. diff --git a/.changeset/subagent-context-extend.md b/.changeset/subagent-context-extend.md new file mode 100644 index 0000000000..34239c85b3 --- /dev/null +++ b/.changeset/subagent-context-extend.md @@ -0,0 +1,8 @@ +--- +'@tanstack/ai': minor +--- + +A host can add fields to the `ctx` an agent's `run` gets. + +- `subagents.binding.extend(ctx)` returns the fields to add. A child's `ctx.chat({ subagents })` passes it down, so the fields reach nested children too. +- `SubagentContextExtension` is an empty interface that `SubagentRunContext` includes. A package augments it to type its fields, for example `ctx.harness` from `@tanstack/ai-harness`. Make each field optional, because a run that a plain `chat()` starts has no host. diff --git a/docs/adapters/openai-advanced.md b/docs/adapters/openai-advanced.md index bc523ef842..6296c2932a 100644 --- a/docs/adapters/openai-advanced.md +++ b/docs/adapters/openai-advanced.md @@ -374,6 +374,8 @@ OpenAI gives each answer item an `id` and a `phase`: `commentary` for text befor When the same model gets that message again, for example on the next turn, the adapter sends each item back with its `id` and `phase`. Another model gets the plain text. +When the model call also sent a reasoning item, the adapter sets `metadata.tanstack.reasoningItem`. If a stored message lost that reasoning item, the adapter sends the text and tool calls without their item ids. OpenAI refuses a `msg_` or `fc_` item without the reasoning item that made it. + ## Transcription options ```typescript diff --git a/docs/adapters/openrouter.md b/docs/adapters/openrouter.md index 9367065d6d..7b1a14562a 100644 --- a/docs/adapters/openrouter.md +++ b/docs/adapters/openrouter.md @@ -323,7 +323,7 @@ Your users can sign in with their own OpenRouter account. Then the requests use ### Model options -Provider options go in `modelOptions`. This includes sampling options such as `temperature` and `maxCompletionTokens` (the token limit of the chat adapter). The type only allows the options that the selected model supports. For example, Claude Sonnet 5.5 does not take `topP`, so `topP` is a type error for that model: +Provider options go in `modelOptions`. This includes sampling options such as `temperature` and `maxCompletionTokens` (the token limit of the chat adapter). The type only allows the options that the selected model supports. For example, Claude Sonnet 5.5 does not take `temperature` or `topP`, so they are type errors for that model: ```typescript import { chat } from "@tanstack/ai"; @@ -334,8 +334,8 @@ const stream = chat({ messages: [{ role: "user", content: "Hello!" }], threadId: "thread-1", modelOptions: { - temperature: 0.7, maxCompletionTokens: 1024, + stop: ["END"], }, }); ``` diff --git a/docs/advanced/byok.md b/docs/advanced/byok.md index 945331b756..3cfe532da3 100644 --- a/docs/advanced/byok.md +++ b/docs/advanced/byok.md @@ -19,6 +19,8 @@ Each send puts the key on an `x-byok-*` header. This setup takes about 10 minutes. You need a chat that already works with `useChat`. +To save the key on your server instead, see [Keep Keys on the Server](./server-keys). + ### 1. Create a store Import BYOK from the `/byok` subpath of your framework package. Vue, Solid, Svelte, Preact, Angular, Octane, and Remix have the same subpath. diff --git a/docs/advanced/compaction-advanced.md b/docs/advanced/compaction-advanced.md index 4ec730afd0..0778221fb2 100644 --- a/docs/advanced/compaction-advanced.md +++ b/docs/advanced/compaction-advanced.md @@ -19,17 +19,17 @@ This page covers the details of compaction that you need only for special cases. | Option | Type | Default | Description | |--------|------|---------|-------------| -| `maxTokens` | `number` | - | **Required.** Compact when the token count of `messages` passes this. See `countTokens`. | +| `maxTokens` | `number \| (ctx: { model }) => number` | - | **Required.** Compact when the token count of `messages` passes this. See `countTokens`. A function gives the limit for the model of the call. See [A limit for each model](#a-limit-for-each-model). | | `strategy` | `CompactionStrategy` | `evictOldest()` | How to shrink the messages. | | `estimateTokens` | `(message: ModelMessage) => number` | characters / 4 | The token estimate for one message. Pass a real tokenizer to get exact counts. | | `strategyKey` | `string` | identity of the built-in strategy | A stable identity for checkpoints. Set it for custom strategies, custom estimators, or a custom eviction marker. Change it when your `summarize` function can change. | | `onCompact` | `(info: CompactionInfo) => void` | - | Runs after each compaction. See [onCompact](#oncompact). | | `countTokens` | `'estimate' \| 'usage'` | `'estimate'` | `'usage'` counts with the usage that the provider reported for the last call. See [Count with real usage](#count-with-real-usage). | | `auto` | `boolean` | `true` | `false` turns off compaction at `maxTokens`. `compactNext`, the overflow retry, and the check after a run still compact. | -| `contextWindow` | `number` | - | The context window of the model. See [Set the limit from the context window](#set-the-limit-from-the-context-window). | +| `contextWindow` | `number \| (ctx: { model }) => number` | - | The context window of the model. See [Set the limit from the context window](#set-the-limit-from-the-context-window). | | `durable` | `boolean` | `false` | On a host with a session log, write the result into the log. See [Keep compaction in a session log](#keep-compaction-in-a-session-log). | | `continueOnError` | `boolean` | `false` | When the strategy fails, send the full messages and continue. By default the run fails. | -| `background` | `{ atTokens: number }` | - | Prepare the summary when the count passes `atTokens`, and apply it at the next run. See [Prepare the summary in the background](#prepare-the-summary-in-the-background). | +| `background` | `{ atTokens: number \| (ctx: { model }) => number }` | - | Prepare the summary when the count passes `atTokens`, and apply it at the next run. See [Prepare the summary in the background](#prepare-the-summary-in-the-background). | | `native` | `{ compact?: (options) => Promise }` | - | The adapter of the `chat()` call. When it has `compact`, the provider compacts in place of `strategy`. See [Let the provider compact](./compaction#let-the-provider-compact). | ### Strategy options @@ -200,6 +200,27 @@ On a host with a [session log](#keep-compaction-in-a-session-log), compaction wi - The answer is not sent again. The only extra model calls are the summary calls. - Without a session log, there is no check after a run. The next request compacts before its first model call. +### A limit for each model + +A thread can change its model. To give each model its own limit, pass a function. It gets the model of the call and gives the limit: + +```ts +import { withCompaction } from '@tanstack/ai-compaction' + +const windows: Record = { 'gpt-6.1-sol': 1_050_000 } +const windowOf = ({ model }: { model: string }) => windows[model] ?? 200_000 + +const perModel = withCompaction({ + maxTokens: (ctx) => windowOf(ctx) - 20_000, + contextWindow: windowOf, + background: { atTokens: (ctx) => windowOf(ctx) - 50_000 }, +}) +``` + +- `maxTokens`, `contextWindow`, and `background.atTokens` each take a number or a function. +- The function runs once for each model call. +- The checkpoint key holds the value of `maxTokens` for the call, so each limit keeps its own checkpoint. + ## Prepare the summary in the background A `summarizeOldest` call takes time. When compaction runs at `maxTokens`, the user waits for that call before the model answers. Set `background` to prepare the summary earlier, while the chat continues: @@ -241,7 +262,7 @@ export async function POST(request: Request) { How the background summary runs: -- `atTokens` must be less than `maxTokens`. If it is not, `withCompaction` throws. +- `atTokens` must be less than `maxTokens`. If both are numbers and it is not, `withCompaction` throws. If a function gives an `atTokens` that is not less, no summary starts at that call. - When the count at a model call passes `atTokens`, the summary starts on a copy of the messages. The model call does not wait for it. - The ready summary waits in the [`metadata` store](#checkpoints), under the namespace `@tanstack/ai-compaction:background`. Without a store, it waits in memory. - The summary applies at the first model call of the next run on the same `threadId`, never in the middle of a run. @@ -250,6 +271,7 @@ How the background summary runs: Sometimes a summary does not apply: - A newer compaction cut past the message that the summary keeps. The summary is dropped and reported with `stale: true`. +- The next run calls another model than the one that started the summary. The summary is dropped and reported with `stale: true`. - The summary call failed. Nothing applies, and the failure is reported with `error`. The next model call past `atTokens` starts a new summary. `onCompact` always gets these reports. A `compaction:ended` event for a failed summary reaches the stream only while the run that started the summary still runs. @@ -308,6 +330,7 @@ const agentCompaction = withCompaction({ ``` - In the text that goes to the summarizer, each tool result is cut to 2,000 characters. Set `maxToolResultChars` to change this limit. +- The summary call runs only its own usage count. To let your cost ledger or traces see it, pass them in `middleware`: `conversationSummarizer({ adapter, middleware: [ledger] })`. - Your own `summarize` function gets a second argument, `summarize(messages, input)`: - `previousSummary` and `previousDetails`: the older summary and its details. - `turnPrefix`: `true` when the call summarizes the start of the current turn. diff --git a/docs/advanced/middleware-advanced.md b/docs/advanced/middleware-advanced.md index ba06d451d7..c69c4038fe 100644 --- a/docs/advanced/middleware-advanced.md +++ b/docs/advanced/middleware-advanced.md @@ -603,6 +603,38 @@ The `hookCtx` fields: | `toolName` | `string` | The tool name | | `toolCallId` | `string` | The tool call ID | +### onGuardToolCall + +A permission check in `onBeforeToolCall` can miss calls. An earlier middleware can return a decision first, and then your hook does not run. Put checks that must always run in `onGuardToolCall`: + +```typescript +import { type ChatMiddleware } from "@tanstack/ai"; + +const noDeletes: ChatMiddleware = { + name: "no-deletes", + onGuardToolCall: (ctx, hookCtx) => { + if (hookCtx.toolName === "deleteFile") { + return { type: "skip", result: { error: "Deleting files is not allowed." } }; + } + }, +}; +``` + +When the guard runs: + +- It runs for every tool call, after all `onBeforeToolCall` hooks and after the `inputSchema` check. +- `hookCtx.args` holds the final checked arguments. The tool gets these same arguments. +- The guard of every middleware runs, in array order, until one returns a decision. +- It runs just before a server tool runs, and before a client tool goes to the client. A call that waits for approval does not run it. + +| Decision | Effect | +|----------|--------| +| `void` / `undefined` | Allow the call. The next guard runs. | +| `{ type: 'skip', result }` | Deny the call. The tool does not run, and the model gets `result`, the same as an `onBeforeToolCall` skip. | +| `{ type: 'abort', reason? }` | Abort the entire chat run | + +A guard cannot change the arguments. To change them, use `onBeforeToolCall`. + ### onAfterToolCall `onAfterToolCall` runs after each tool runs (or is skipped). Every middleware runs, in array order. @@ -926,6 +958,7 @@ const stream = chat({ | `onStart` | Sequential | All run in order. | | `onChunk` | **Piped.** Chunks flow through each middleware. | If the first one drops a chunk, the later ones never see it. | | `onBeforeToolCall` | **First wins.** The first decision wins. | The earlier middleware has priority. | +| `onGuardToolCall` | Sequential. Every guard runs until one returns a decision. | An earlier `onBeforeToolCall` decision never skips a guard. | | `onShouldContinue` | **AND.** One `false` stops the loop. | The order changes only which middleware runs first. | | `onModelError` | Sequential. If any middleware calls `ctx.retry()` and the call has retries left, the model is called again. | All run in order, so each one sees the error. | | `onAfterToolCall` | **Piped.** Each gets the result of the previous `replaceResult`. | The last replacement wins. | diff --git a/docs/advanced/prompt-caching.md b/docs/advanced/prompt-caching.md index 60758f3dce..13b8006751 100644 --- a/docs/advanced/prompt-caching.md +++ b/docs/advanced/prompt-caching.md @@ -83,9 +83,10 @@ From the second call of a conversation, most of the input comes from the cache. 2. The `threadId` that you pass to `chat()`. 3. No key. Claude does not need one. OpenAI uses the key to route requests, so a stable `threadId` gives more cache hits. - A request reads the cache only up to the first token that is different. Text that changes on each call, such as the current time in the system prompt, ends the match there. -- `usage.promptTokens` is the full input. Two fields split it: +- `usage.promptTokens` is the full input. These fields split it: - `usage.promptTokensDetails.cachedTokens`: tokens read from the cache. - `usage.promptTokensDetails.cacheWriteTokens`: tokens written to the cache. Claude reports this field. + - `usage.promptTokensDetails.cacheWrite1hTokens`: the part of `cacheWriteTokens` that is kept for 1 hour. Claude reports this field. A 1-hour write costs more than a 5-minute write. ## Advanced diff --git a/docs/advanced/server-keys.md b/docs/advanced/server-keys.md new file mode 100644 index 0000000000..5918826a34 --- /dev/null +++ b/docs/advanced/server-keys.md @@ -0,0 +1,254 @@ +--- +title: Keep Keys on the Server +id: server-keys +order: 10 +description: "Save each user's provider keys on your server with a CredentialStore, and read them in your chat route with storedProviderKeys." +keywords: + - tanstack ai + - byok + - api key + - CredentialStore + - storedProviderKeys + - encryption +--- + +A key in the browser is gone when the user opens your app on a new device. A background job has no browser to send it. For these cases, save the key on your server, one key for each user and provider. + +Do these four steps: + +1. Make a credential store. +2. Add a route that saves the key. +3. Send the key from your form. +4. Read the key in the chat route. + +## 1. Make a credential store + +Start with the memory store: + +```typescript +import { memoryCredentialStore } from "@tanstack/ai/byok/server"; + +export const credentials = memoryCredentialStore(); +``` + +The memory store loses the keys when the process stops. For production, [write a store](#write-a-store-for-production) for your database or your secrets manager. + +## 2. Add a route that saves the key + +Get the user from your auth, not from the request body: + +```typescript +import { credentials } from "./credentials"; +import { getUser } from "./auth"; + +export async function POST(request: Request) { + const user = await getUser(request); + if (!user) return new Response("Unauthorized", { status: 401 }); + + const body: unknown = await request.json(); + if ( + typeof body !== "object" || + body === null || + !("key" in body) || + typeof body.key !== "string" + ) { + return new Response("Send { key }", { status: 400 }); + } + + await credentials.set({ + scope: { userId: user.id }, + id: "openai", + credential: { type: "api_key", value: body.key }, + }); + return new Response(null, { status: 204 }); +} +``` + +## 3. Send the key from your form + +```tsx +import { useState } from "react"; + +export function ServerKeyForm() { + const [value, setValue] = useState(""); + + return ( +
{ + event.preventDefault(); + void fetch("/api/keys", { + method: "POST", + headers: { "content-type": "application/json" }, + body: JSON.stringify({ key: value.trim() }), + }).then(() => setValue("")); + }} + > + setValue(event.target.value)} + /> + +
+ ); +} +``` + +## 4. Read the key in the chat route + +`storedProviderKeys` reads the key that the user saved: + +```typescript +import { + chat, + chatParamsFromRequest, + keyedAdapter, + toServerSentEventsResponse, +} from "@tanstack/ai"; +import { byokMissing, storedProviderKeys } from "@tanstack/ai/byok/server"; +import { createOpenaiChat } from "@tanstack/ai-openai"; +import { openaiByok } from "@tanstack/ai-openai/byok"; +import { credentials } from "./credentials"; +import { getUser } from "./auth"; + +const gpt = keyedAdapter(openaiByok, (key) => + createOpenaiChat("gpt-6.1-sol", key), +); + +export async function POST(request: Request) { + const user = await getUser(request); + if (!user) return new Response("Unauthorized", { status: 401 }); + + const keys = storedProviderKeys(credentials, { userId: user.id }); + const apiKey = await keys.get(openaiByok); + if (!apiKey) return byokMissing(openaiByok); + + const params = await chatParamsFromRequest(request); + const stream = chat({ + adapter: gpt.create(apiKey), + messages: params.messages, + threadId: params.threadId, + runId: params.runId, + }); + return toServerSentEventsResponse(stream); +} +``` + +How `storedProviderKeys` finds a key: + +- It reads the `api_key` credential that the user saved under the provider id. +- If you add `tenantId` to the scope, a user with no key gets the key of the tenant. A tenant key is saved with `tenantId` and no `userId`. +- If there is no saved key, it reads the env var of the provider, here `OPENAI_API_KEY`. +- To give the same keys to your [agents](./byok#keys-inside-agents), pass `keys.get` to `subagents.keys`. + +Your users now save a key once, and every device and job uses it. + +## Write a store for production + +A `CredentialStore` has four methods, and two optional ones for [compare-and-set](#add-compare-and-set). You decide where the secrets live: + +- A `credentials` table in the database that you already use, next to your chat tables. +- A secrets manager, for example AWS Secrets Manager or Vault. + +```ts +import type { CredentialScope } from "@tanstack/ai/byok"; + +type Credential = + | { type: "api_key"; value: string } + | { type: "secret_answer"; value: string; expiresAt: number } // epoch ms + | { + type: "oauth"; + accessToken: string; + refreshToken?: string; + expiresAt?: number; // epoch ms + scopes?: Array; + client?: { + clientId: string; + clientSecret?: string; + redirectUri?: string; + issuer?: string; + }; + }; + +interface CredentialStore { + get(key: { scope: CredentialScope; id: string }): Promise; + set(entry: { + scope: CredentialScope; + id: string; + credential: Credential; + }): Promise; + delete(key: { scope: CredentialScope; id: string }): Promise; + list( + scope: CredentialScope, + ): Promise>; + // Optional: compare-and-set. + getVersioned?(key: { + scope: CredentialScope; + id: string; + }): Promise<{ credential: Credential; revision: string } | null>; + setIf?(entry: { + scope: CredentialScope; + id: string; + credential: Credential; + expectedRevision: string | null; + }): Promise<{ ok: true; revision: string } | { ok: false; reason: "conflict" }>; +} +``` + +Type your store with `defineCredentialStore` from `@tanstack/ai/byok`. `CredentialScope` has `tenantId` and `userId`. It can also have a `threadId`, which the store ignores. + +Rules for the keys: + +- Key each credential by `scope.tenantId`, `scope.userId`, and `id`. Do not use `scope.threadId`. +- A user credential is not visible at the tenant scope. A tenant credential is not visible at the user scope. +- A scope with no tenant and no user is one bucket that every caller shares. Get the scope from your server-side auth, never from the request body. +- `client` is the OAuth client that the tokens belong to. A token refresh needs it. +- A `secret_answer` row is not a user credential. The harness keeps a secret answer there for a short time. Leave this type out of a list of connected accounts. You can delete a row after its `expiresAt`. + +Rules for the secrets (`value`, `accessToken`, `refreshToken`, `client.clientSecret`): + +- Encrypt them at rest. Keep the encryption key in a KMS or a server-only env var, not in the same database. +- `list` returns ids and types only. It never returns a secret, not even masked. +- `delete` removes the row. Do not keep a soft-deleted copy of the secret. +- Do not write a secret to a log, an error message, or a trace. To remove known secrets from a text, use `scrubSecrets(text, secrets)` from `@tanstack/ai/byok`. + +The [harness](../harness/provider-keys) uses the same store: pass it to `createHarnessHost({ credentials })`. + +### Add compare-and-set + +Two servers can refresh the same OAuth token at the same time. If the service rotates refresh tokens, the second save then keeps a token that no longer works. Add `getVersioned` and `setIf`, so the second save does not remove the first: + +- `getVersioned` returns the credential and its revision, or `null` when the key is absent. +- `setIf` writes only when the stored revision equals `expectedRevision`. Pass `null` to write only when the key is absent. Success returns `{ ok: true, revision }`. A different revision returns `{ ok: false, reason: "conflict" }` and does not write. +- Every write gives the key a new revision. A plain `set` does this too. After `delete`, the deleted revision is never used again. +- Add both methods or neither. `memoryCredentialStore()` has both. + +To refresh a token in your own route, read it with `readCredentialEntry`. It finds the credential like `readCredential` (the user first, then the tenant), and also returns the scope where it found it and its `revision`: + +```typescript +import { readCredentialEntry } from "@tanstack/ai/byok/server"; +import type { Credential, CredentialStore } from "@tanstack/ai/byok"; + +export async function refreshGithub( + store: CredentialStore, + scope: { tenantId: string; userId: string }, + refresh: (current: Credential) => Promise, +) { + const entry = await readCredentialEntry(store, scope, "github"); + if (!entry) return null; + const next = await refresh(entry.credential); + // Save to the scope where the credential was, also when it is the tenant's. + const key = { scope: entry.scope, id: "github", credential: next }; + if (!store.setIf || entry.revision === undefined) { + await store.set(key); + return next; + } + const saved = await store.setIf({ ...key, expectedRevision: entry.revision }); + if (saved.ok) return next; + // Another server saved first. Use its token, and do not refresh again. + return (await readCredentialEntry(store, scope, "github"))?.credential ?? null; +} +``` + +In a harness, `run.credentials.refresh` does this for you. See [Refresh a token](../harness/auth#refresh-a-token). diff --git a/docs/api/ai.md b/docs/api/ai.md index 0c47a91ceb..6fbeb450b8 100644 --- a/docs/api/ai.md +++ b/docs/api/ai.md @@ -585,7 +585,10 @@ maskKey("sk-abcdefghij"); // "ghij" ### `scrubSecrets` -`scrubSecrets(input, secrets)` replaces each secret in `input` with `[redacted]`. Use it before you log an error string. +`scrubSecrets(text, secrets)` replaces each secret in `text` with `[redacted]`. Use it before you log an error string. + +- It also replaces the JSON-escaped and the URL-encoded form of each secret. +- It skips a secret shorter than 6 characters, so a short value does not remove common words. ```typescript import { scrubSecrets } from "@tanstack/ai/byok"; diff --git a/docs/chat/stream-events.md b/docs/chat/stream-events.md index 60fdd562bc..77573bf418 100644 --- a/docs/chat/stream-events.md +++ b/docs/chat/stream-events.md @@ -246,6 +246,7 @@ You want to know what a run cost. `RUN_FINISHED.usage.promptTokens` is the full - `cachedTokens`: tokens read from the cache. - `cacheWriteTokens`: tokens written to the cache. +- `cacheWrite1hTokens`: the part of `cacheWriteTokens` that is kept for 1 hour. Claude reports this field. A 1-hour write costs more than a 5-minute write. To get the uncached part, subtract both: diff --git a/docs/chat/subagents-advanced.md b/docs/chat/subagents-advanced.md index 3208d84251..48284d7deb 100644 --- a/docs/chat/subagents-advanced.md +++ b/docs/chat/subagents-advanced.md @@ -193,7 +193,7 @@ Without a router, every agent tool takes the fields of the agent, plus two more: - The fields of `inputSchema`, for an agent with one. The tool checks them against the schema. - `prompt`: the task as text, for an agent without `inputSchema`. The child gets it as the last user message. - `sessionId`: the `subagentRunId` of an earlier result. The call continues that child. The tool has this field only when `withPersistence` is on the parent `chat()`. -- `background`: `true` starts the child and returns its `subagentRunId` at once. The tool has this field only when the host sets `subagents.binding.start`. The host returns `done`. `maxConcurrent` counts that child until `done` settles. +- `background`: `true` starts the child and returns its `subagentRunId` at once. The tool has this field only when a host that runs agents for you, such as a [harness](../harness/subagents), turns it on. `maxConcurrent` counts that child until it ends. When the run uses `sessionId` or `background`, that name is reserved. If an `inputSchema` has a field with that name, `chat()` throws before the run starts. diff --git a/docs/chat/subagents.md b/docs/chat/subagents.md index 327aa42b7a..b2a7585cdc 100644 --- a/docs/chat/subagents.md +++ b/docs/chat/subagents.md @@ -432,6 +432,8 @@ Put `withPersistence` on the parent `chat()` only. The parent stores each child: A reload then shows every card, and a resume continues a waiting child. See [Subagent cards on reload](../persistence/chat-persistence-advanced#subagent-cards-on-reload). +A resume continues a waiting child from its stored messages, not from the card that the client sends back. This is also true for a nested child, for example a child that an agent tool of another child started. The card gives only the child id. A changed card cannot change a tool call before its approval runs it. + > [!WARNING] > Do not add `withPersistence` to a child `chat()` too. The child does not know that it runs as a subagent, so it stores the same child again under its own run and thread ids. Its interrupt records then conflict with the records of the parent, and a reload can lose the pending approval. diff --git a/docs/code-mode/code-mode.md b/docs/code-mode/code-mode.md index d4fc7b757e..bb5cc168ce 100644 --- a/docs/code-mode/code-mode.md +++ b/docs/code-mode/code-mode.md @@ -131,6 +131,7 @@ The Node, QuickJS, QuickJS Bun, and Cloudflare drivers have no access to the hos - **Run somewhere other than Node:** [Isolate Drivers](./code-mode-isolates). - **Let the model save and reuse code:** [Code Mode with Snippets](./code-mode-with-snippets). - **Have many tools:** [Lazy Tools](./lazy-tools) keeps rarely used tools out of the prompt. +- **Run in a harness session:** [Code mode in a harness](../harness/code-mode) adds code mode as a plugin. ## Advanced diff --git a/docs/computer-use/computer-use.md b/docs/computer-use/computer-use.md index d299316093..8dff14c9d5 100644 --- a/docs/computer-use/computer-use.md +++ b/docs/computer-use/computer-use.md @@ -76,6 +76,8 @@ export async function POST(request: Request) { > [!WARNING] > Run the browser in a network where it cannot reach private services. The browser can open every address that your server can reach. See [Keep the browser away from private addresses](#keep-the-browser-away-from-private-addresses). +Does your agent run in a [harness](../harness/overview)? Use the `computerUse()` plugin with the same driver. It stores each screenshot in the media store of the session, and asks about safety checks through `permissions()`. See [Give your agent a computer](../harness/computer-use). + ## 3. Register the interrupt on the client Give `computerSafetyCheck` to `useChat`. It is the same object as `computerUse.interrupt`, and it does not load the driver or Playwright on the client. diff --git a/docs/config.json b/docs/config.json index 31171d144e..ba16ab89da 100644 --- a/docs/config.json +++ b/docs/config.json @@ -344,7 +344,8 @@ { "label": "Server Tools: Advanced", "to": "tools/server-tools-advanced", - "addedAt": "2026-10-21" + "addedAt": "2026-10-21", + "updatedAt": "2026-10-22" }, { "label": "Client Tools", @@ -439,7 +440,8 @@ { "label": "Compaction: Advanced", "to": "advanced/compaction-advanced", - "addedAt": "2026-10-21" + "addedAt": "2026-10-21", + "updatedAt": "2026-10-22" } ], "tab": "guides" @@ -1059,6 +1061,280 @@ ], "tab": "guides" }, + { + "label": "Harness", + "tab": "guides", + "children": [ + { + "label": "Build your first harness", + "to": "harness/overview", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Host options", + "to": "harness/host-options", + "addedAt": "2026-10-20", + "updatedAt": "2026-10-22" + }, + { + "label": "Send and show media", + "to": "harness/media", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Connect clients", + "to": "harness/connect", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Write a client in any language", + "to": "harness/protocol", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-22" + }, + { + "label": "Access and tenants", + "to": "harness/access", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-22" + }, + { + "label": "Run in the terminal", + "to": "harness/cli", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-19" + }, + { + "label": "Deploy a harness", + "to": "harness/deploy", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Send inputs safely", + "to": "harness/inputs", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Run a tool without a model", + "to": "harness/tool-runs", + "addedAt": "2026-10-15" + }, + { + "label": "Durable sessions", + "to": "harness/durable-sessions", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Run background work on workers", + "to": "harness/workers", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-19" + }, + { + "label": "List, rename, and fork sessions", + "to": "harness/sessions", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Lifecycle and retention", + "to": "harness/lifecycle", + "addedAt": "2026-10-17", + "updatedAt": "2026-10-22" + }, + { + "label": "Run side effects once", + "to": "harness/durable-tools", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Wait for a person", + "to": "harness/waits", + "addedAt": "2026-10-18", + "updatedAt": "2026-10-21" + }, + { + "label": "Build on the session log", + "to": "harness/session-log", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Control how a turn ends", + "to": "harness/turn-control", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Store settings per thread", + "to": "harness/thread-settings", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Compact a long session", + "to": "harness/compaction", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Fork and reset a thread", + "to": "harness/fork-and-reset", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "One log for a session tree", + "to": "harness/shared-logs", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Write a plugin", + "to": "harness/plugins", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Track usage and cost", + "to": "harness/usage", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "Ask before risky tool calls", + "to": "harness/permissions", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-21" + }, + { + "label": "Give your agent coding tools", + "to": "harness/coding-tools", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-21" + }, + { + "label": "Sandbox and extend the coding tools", + "to": "harness/coding-tools-backends", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Run the coding tools for many threads", + "to": "harness/coding-workspaces", + "addedAt": "2026-10-18", + "updatedAt": "2026-10-21" + }, + { + "label": "Undo a turn", + "to": "harness/snapshots", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Build a coding agent", + "to": "harness/coding-agent", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "Delegate to coding agents", + "to": "harness/coding-agents", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "Switch between agent profiles", + "to": "harness/agents", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "Auth and connectors", + "to": "harness/auth", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-19" + }, + { + "label": "Share a thread between people", + "to": "harness/shared-threads", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Connect model providers", + "to": "harness/provider-keys", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Build your own UI", + "to": "harness/custom-ui", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Code mode in a harness", + "to": "harness/code-mode", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-21" + }, + { + "label": "Work until a goal is met", + "to": "harness/goal", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "Run agents from a harness", + "to": "harness/subagents", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-22" + }, + { + "label": "Hand off a conversation", + "to": "harness/handoffs", + "addedAt": "2026-10-18", + "updatedAt": "2026-10-20" + }, + { + "label": "Add skills", + "to": "harness/skills", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-20" + }, + { + "label": "Use MCP servers", + "to": "harness/mcp", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-21" + }, + { + "label": "Give your agent a computer", + "to": "harness/computer-use", + "addedAt": "2026-10-21" + }, + { + "label": "Use from any MCP client", + "to": "harness/mcp-server", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + }, + { + "label": "Use a harness from an editor (ACP)", + "to": "harness/acp", + "addedAt": "2026-10-13", + "updatedAt": "2026-10-18" + } + ] + }, { "label": "Sandboxes", "tab": "guides", @@ -1305,6 +1581,12 @@ "addedAt": "2026-08-18", "updatedAt": "2026-10-22" }, + { + "label": "Keep Keys on the Server", + "to": "advanced/server-keys", + "addedAt": "2026-10-16", + "updatedAt": "2026-10-19" + }, { "label": "Debug Logging", "to": "advanced/debug-logging", @@ -1397,7 +1679,8 @@ { "label": "OpenAI: Advanced", "to": "adapters/openai-advanced", - "addedAt": "2026-10-21" + "addedAt": "2026-10-21", + "updatedAt": "2026-10-22" }, { "label": "Anthropic", diff --git a/docs/harness/access.md b/docs/harness/access.md new file mode 100644 index 0000000000..2bc840b411 --- /dev/null +++ b/docs/harness/access.md @@ -0,0 +1,296 @@ +--- +title: Access and tenants +id: harness-access +order: 3 +description: "Decide who can use each session of a harness. Read the user of each request, check each action, and keep the threads of each organization apart." +keywords: + - tanstack ai + - harness + - access + - getUser + - isAllowed + - tenantId + - membersOnly +--- + +Your harness serves many users. Ada must not read the threads of Bob, and the company Acme must never see a thread of the company Globex. Give the host `access`. Then the host reads the user of each request, asks you about each action, and keeps each organization apart. + +A user here is `{ id, name?, tenantId? }`: who a call acts for. Its type is `HarnessUser`. + +## 1. Turn on access + +Give `createHarnessHost` an `access` object and a `sessions` store. The store keeps the owner and the members of each session: + +```ts group=harness-access +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +// Your own login check: a session token to a user. +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme' }], + ['Bearer token-gus', { id: 'gus', tenantId: 'globex' }], +]) + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host, harness: assistant }) +``` + +- `getUser` reads the user of a request with your own login check. Return `null` to refuse the request with `401`. +- `rules` decides each action. `membersOnly()` gives ready rules: only the members of a session can use it. See [Share a thread between people](./shared-threads). +- `memoryPersistence()` has a `sessions` store. Without `stores.sessions`, `createHarnessHost` throws. + +The handler gets the user from `getUser`, so it needs no login option of its own. A new thread belongs to the user who opens it first. + +## 2. Write your own rule + +Give `access` your own `rules` with an `isAllowed` function. `isAllowed` gets the user, the action, and the session. Return `false` to refuse. The user has the type that `getUser` returns, so your own fields are typed too: + +```ts group=harness-access-rule +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +interface User { + id: string + tenantId: string + isAdmin: boolean +} + +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme', isAdmin: false }], + ['Bearer token-eve', { id: 'eve', tenantId: 'acme', isAdmin: true }], +]) + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: { isAllowed: ({ user, action, session }) => { + const isOwner = session.members.some( + (member) => member.userId === user.id && member.role === 'owner', + ) + if (isOwner || user.isAdmin) return true + // Everyone in the organization can read every thread of it. + return action === 'read' || action === 'list' + } }, + }, +}) +``` + +Each action covers these calls: + +| Action | What it covers | +| --- | --- | +| `read` | Open the session, and read its stream, snapshot, transcript, media, and child runs. | +| `list` | See the session in `host.sessions.list`. | +| `send` | Prompt, steer, follow up, run a command or a tool, start an agent, and upload media. | +| `answer` | Answer a question or an approval. | +| `operator` | Reply as an operator with `send(text, { as: 'operator' })`, and answer a `human` wait of any run. See [Hand off to a person](./handoffs#4-hand-off-to-a-person). | +| `cancel` | Stop a run, or drop an input that waits. | +| `configure` | Change the thread settings, the plugin config, the title, and the metadata. | +| `share` | Add or remove members. | +| `delete` | Remove the session from the index. | +| `erase` | Erase the messages of one user with `host.sessions.eraseUser`. See [Erase the messages of one person](./shared-threads#9-erase-the-messages-of-one-person). | +| `fork` | Copy the session into a new session. | +| `transfer` | Move the conversation to another thread, also of another harness, with `host.transfer`. See [Move a conversation to another harness](./handoffs#5-move-a-conversation-to-another-harness). | + +- For `answer` and `cancel`, `isAllowed` also gets `run`: the `runId` and the `userId` of the user who started that run. +- For `answer` and `operator`, `isAllowed` also gets `wait`: the `waitId` and the `kind` of the [wait](./waits) that the input answers, when the host knows it. A `human` wait takes only `operator`, also for a resolve that ends it. +- For `cancelSchedule`, `run` is the schedule: the `runId` is the schedule id, and the `userId` is the user who made it. +- `host.submit` and `schedule` check an answer or a cancel when you send it, with `run` from the log. A run that the log does not have is refused with `unknown_run`: `submit` returns a `rejected` receipt, and `schedule` throws `InputRejectedError`. +- A handle checks `read` once, when it opens. Each action that changes the session asks `isAllowed` again. This includes `cancel` and `send` on an operation or an agent run that the handle returns. +- `isAllowed` does not get the input. To check what an input says, or to replace its `context`, give the host `acceptInput`. It runs after `isAllowed`. See [Check an input on the server](./inputs#check-an-input-on-the-server). + +The rules have two more keys. `membersOnly()` sets both. Your own rules can leave them out: + +- `filterSessionList`: the filter that `host.sessions.list` gives the store, for example `{ memberId }`. Without it, the store reads every session of the tenant, and `isAllowed` drops the others, so a page can have fewer entries than `limit`. +- `members`: turns on `host.sessions.share` and `unshare`, and `members.fork` sets the members of a fork. Without it, `share` and `unshare` throw. + +## What the host checks for you + +`isAllowed` decides only the rule of your app. The host does these checks before it calls it: + +- The tenant. A user reaches only the sessions of their own tenant. `isAllowed` never gets a session of another tenant. +- A user on every call. With access, `host.open`, `host.fork`, `host.transfer`, and each `host.sessions` method need a `user`. A call without one throws. +- One user per handle. `host.open` returns a session that acts only as its `user`. A call on it that passes another `user` throws. +- Child threads. The host checks a subagent child as its root thread, so `session` is the root session. A user who cannot read the root gets `403` for the child, and does not learn its root. +- Missing threads. A read of a thread with no entry answers `404` and starts no session. `GET run?threadId=` answers an empty hydration instead, because `useChat` reads it before the first message. + +A refused action throws `AccessDeniedError`, with `code: 'forbidden'`, the `action`, and the `threadId`. The handler answers it with `403`. + +The host keeps only the `id` and the `tenantId` of a user in the log. So `isAllowed` can get a user with only these two fields. A rule that reads another field, such as `isAdmin` above, must refuse when the field is missing. + +A signed media URL is no login. The handler runs `getUser` for it too, and checks `read` for that user, in the tenant of the URL. A request with no user gets `401`. So load the file with a request that sends the login, for example an `` on the same site, which sends its cookies. + +### Serve media to a request with no login + +An `` on another site sends no cookies, so `getUser` finds no user. Give `access` a `getGuestUser` function. It returns a guest for that request, and `isAllowed` decides `read` for the guest: + +```ts group=harness-access-guest +import { createHarnessHost, membersOnly } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +interface User { + id: string + tenantId?: string + isGuest?: boolean +} + +const members = membersOnly() + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: () => null, + // `tenantId` is the tenant of the signed URL. + getGuestUser: ({ tenantId }) => ({ id: 'guest', tenantId, isGuest: true }), + rules: { + isAllowed: (input) => + input.user.isGuest === true ? input.action === 'read' : members.isAllowed(input), + filterSessionList: members.filterSessionList, + members: members.members, + }, + }, +}) +``` + +- The handler calls `getGuestUser` only for a signed media URL, after the signature checks out, and only when `getUser` finds no user. +- The guest gets nothing else. Every other route still needs a user from `getUser`. +- Without `getGuestUser`, or when it returns `null`, the request gets `401`. +- The guest needs the tenant of the URL, else the request gets `403`. + +## Keep tenants apart + +Give each user a `tenantId` in `getUser`. The tenant keeps each organization apart: + +- One `threadId` in two tenants is two sessions, with two index entries and two logs. +- `host.sessions.list` reads one tenant: the tenant of `user`. A user with no `tenantId` is in the default tenant. +- The log store keeps each session tree under `logIdOf({ tenantId, logId })`, for example `acme:thread-1`. The default tenant gives `:thread-1`. Use `logIdOf` when you read a log yourself. +- The credentials of a user are kept per tenant. See [Keep organizations apart](./shared-threads#7-keep-organizations-apart). + +## Call the host from code with no request + +A cron job, an admin tool, or a queue worker has no request, so `getUser` cannot run. Pass the user yourself. The host checks the action for that user, the same as for a request: + +```ts group=harness-access +// Your database knows the owner of each thread. +export async function nightlyReport(threadId: string, owner: { id: string; tenantId: string }) { + const session = await host.open(assistant, { threadId, user: owner }) + await session.send('Write the report of today.') +} +``` + +Your own routes and an MCP server have a request. Read its user with `host.getUserOfRequest`: + +```ts group=harness-access +export async function myThreads(request: Request) { + const user = await host.getUserOfRequest({ request }) + if (!user) return new Response('Unauthorized', { status: 401 }) + const page = await host.sessions.list({ user, limit: 20 }) + return Response.json(page.entries.map((entry) => entry.title ?? 'New chat')) +} +``` + +`host.getUserOfRequest` throws on a host with no `access`. + +## Use your own access tables + +Your app already keeps who can use each thread, for example a table of teams. Read it in `isAllowed`. When you take a user out of the table, call `host.revoke`: + +```ts group=harness-access-own-tables +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const users = new Map([ + ['Bearer token-ada', { id: 'ada', tenantId: 'acme' }], + ['Bearer token-bob', { id: 'bob', tenantId: 'acme' }], +]) + +// Your own table: the team of each thread. +const teams = new Map([['plan-q3', new Set(['ada', 'bob'])]]) + +export const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => users.get(request.headers.get('authorization') ?? '') ?? null, + rules: { isAllowed: ({ user, session }) => + session.members.some((member) => member.userId === user.id) || + (teams.get(session.threadId)?.has(user.id) ?? false) }, + }, +}) + +export async function removeFromTeam(threadId: string, user: { id: string; tenantId: string }) { + teams.get(threadId)?.delete(user.id) + await host.revoke({ threadId, user }) +} +``` + +When the promise of `host.revoke` resolves: + +- The running and queued work of the user on the thread is cancelled, on every host. The schedules of the user on the thread are removed. +- The streams of the user on the thread close. A socket gets a `forbidden` error frame. A running `events()` read of a handle or a view ends with `AccessDeniedError`. +- The open handles and views of the user on the thread throw `AccessDeniedError`. A new `host.open` asks `isAllowed` again. +- The signed media URLs stop working for the user. The handler checks the user of each request again when it serves the file. +- `host.events()` yields a `revoked` event. + +When the cancels cannot be sent (for example, the log store fails), the promise rejects. The access is gone already. Call `host.revoke` again to stop the work. `host.sessions.unshare` rejects the same way, after it removed the member. + +`host.sessions.share` and `unshare` need `members` in the rules, which `membersOnly()` has. With your own tables, change the tables, then call `host.revoke`. + +## Serve with no login + +A demo, or a tool on your own machine, has no login. Give the handler `openAccess` on a host with no `access`: + +```ts group=harness-access-open +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) + +// One user on this machine: every request runs as Ada. +export const handler = createHarnessHandler({ + host, + harness: assistant, + openAccess: { user: { id: 'ada' } }, +}) +``` + +- `openAccess: true`: every request runs with no user, in the default tenant. +- `openAccess: { user }`: every request runs as that one user. Their credentials and keys do the work. +- Without host `access` and without `openAccess`, `createHarnessHandler` throws at the start. So a server never serves every thread to everyone by mistake. +- On a host with `access`, `openAccess` throws. The host reads the user itself. +- `handleHarnessSocket` takes the same option. + +CAUTION: Do not use `openAccess` on a server that the internet can reach. Every caller can then use every thread. + +## Use the provider keys of the server + +With access, a keyed adapter and `run.keys` read only the keys that each user saved. They do not read the env vars of the server, so a user never spends the key of the server. To let users fall back to the server key, set `allowServerEnvKeys: true` in `access`, next to `getUser` and `policy`. See [Connect model providers](./provider-keys). + +## What you have now + +- A host that reads the user of each request, and checks each action. +- A rule of your own, typed from what `getUser` returns. +- Organizations that never see each other's threads. +- Cron jobs and admin tools that act as a user they name. +- A way to take access away from a user at once. diff --git a/docs/harness/acp.md b/docs/harness/acp.md new file mode 100644 index 0000000000..58d1afd50e --- /dev/null +++ b/docs/harness/acp.md @@ -0,0 +1,117 @@ +--- +title: Use a harness from an editor (ACP) +id: harness-acp +order: 6 +description: "Serve a harness as an ACP v2 agent, so an editor such as Zed can start it as a process and talk to it over stdio." +keywords: + - tanstack ai + - harness + - ACP + - agent client protocol + - editor +--- + +Your users want your agent in their editor. Editors such as Zed start an agent as a process and talk the Agent Client Protocol (ACP) over stdin and stdout. `serveAcp` makes your harness an ACP v2 agent. + +## 1. Install the package + + + +react: @tanstack/ai-acp +vue: @tanstack/ai-acp +solid: @tanstack/ai-acp +svelte: @tanstack/ai-acp +preact: @tanstack/ai-acp +angular: @tanstack/ai-acp +octane: @tanstack/ai-acp +vanilla: @tanstack/ai-acp + + + +## 2. Serve the harness + +Make a script that the editor starts: + +```ts group=harness-acp +import { serveAcp } from '@tanstack/ai-acp/agent' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) + +serveAcp({ host, harness: assistant, version: '1.0.0' }) +``` + +- Each ACP session is one harness session. The ACP session id is the thread id. +- `user` is the user of the editor, for example `serveAcp({ host, harness: assistant, user: { id: 'ada' } })`. Every session opens as that user, so `/connect` saves keys for them. On a host with [`access`](./access), `user` is required. +- The prompts of a session run one after another, in the order that the editor sent them. A prompt with a large file keeps its place while the file uploads. The harness `busy` setting does not apply. +- The agent holds a session only while a prompt runs or waits. An idle session closes after the host `lifecycle.closeIdleSessionsAfterMs`, and the next prompt opens it again. +- `version` is the version that `initialize` reports. Default: `'0.0.0'`. +- `cwd` is the working directory that the session list shows for each session. Default: `process.cwd()`. +- Use a persistent store instead of `memoryPersistence()` if a session must stay after the process stops. + +## 3. Add the agent to the editor + +Give the editor the command that runs the script, for example `node agent.js`. The editor then sends prompts, and the harness streams the answer back. + +## What the editor gets + +- The text and the reasoning of the main agent. Text from child agents stays out of the main message. +- Each tool call, of the main agent and of child agents, as a tool call update. +- Each [wait](./waits) of a turn, by its kind (see [Waits in the editor](#waits-in-the-editor)). +- Cancel stops the running turn, also while the editor asks for an answer. A turn that fails ends with the `error` stop reason and the message. +- Image blocks, audio blocks, and embedded files of a prompt go to the media store of the session. A file that the harness refuses fails the prompt with an invalid params error. See [Send and show media](./media). + +## Waits in the editor + +The agent sends each wait to the editor by its kind: + +| Kind | What the editor shows | +| --- | --- | +| `approval` | A permission request with the id of the real tool call. **Allow always** shows only for an approval of `permissions()`, because its answer schema takes `always`. | +| `question`, `human` | A form elicitation of the answer schema. | +| `sign-in` | A URL elicitation with the sign-in page. | +| `client-tool` | Nothing. The editor cannot run it, so the agent cancels it. | + +- The editor must declare form and URL elicitation in `initialize`. If it does not, the agent cancels that wait, and the model continues without the answer. +- A question that a tool asks while it runs (`step.ask`, an MCP server) is a form elicitation too. An answer that does not fit the schema asks again. +- If the editor cancels the prompt while it asks, the turn stops. Its waits stay open until the next prompt. + +## Open an old session + +The editor lists the sessions of the harness, newest first. The user picks one, and the next prompt continues that thread. + +- The list shows the top-level sessions of the harness that `user` can list. Sessions of child agents stay out. +- The list comes from the session index of the host. Use a persistent store, so that the list keeps the sessions after the process stops. +- A session id that the session index does not know, or that `user` cannot read, fails with a not found error. + +## Known limits + +- ACP v2 is still a draft, so this API is experimental. +- A client tool call, and a wait that the editor did not declare it can show, are cancelled. The model then continues without the answer. +- Each session in the list shows the same working directory: the `cwd` of the agent. If the editor asks for the sessions of another directory, the list is empty. +- An old session opens without its earlier messages. The editor shows only the new turns. The model still gets the full thread. +- A resource link in a prompt is only named. The harness does not read it. The model can open it with its own tools. + +## Use your own stream + +`serveAcp` uses stdin and stdout when you pass no `stream`. To use another transport, pass an ACP `Stream`, or call `createAcpAgent({ host, harness })` and connect the app yourself: + +```ts group=harness-acp +import { createAcpAgent } from '@tanstack/ai-acp/agent' + +export const app = createAcpAgent({ host, harness: assistant }) +``` + +One agent app serves one connection. It reads the editor capabilities from that connection, and a second connection to it fails with an error. To serve more editors, for example over HTTP, make one app for each connection. With the `AcpServer` of `@agentclientprotocol/sdk/experimental/server`, pass `createAgent: () => createAcpAgent({ host, harness: assistant })`. + +## What you have now + +- Your harness as an agent in any ACP v2 editor. +- Approvals, questions, and sign-ins in the editor, and cancel, media, and old sessions. diff --git a/docs/harness/agents.md b/docs/harness/agents.md new file mode 100644 index 0000000000..2afb1e488c --- /dev/null +++ b/docs/harness/agents.md @@ -0,0 +1,231 @@ +--- +title: Switch between agent profiles +id: harness-agents +order: 10 +description: "Give one harness several agents, such as a read-only planner and a builder. Each profile has its own prompt, tools, model, rules, and step limit, in code or in Markdown files." +keywords: + - tanstack ai + - harness + - agents + - agent profiles + - plan + - subagents +--- + +Your agent has two jobs. First it must plan a change without touching a file. Then it must build the change with every tool. The main model also needs a fast helper that searches the code. `agents()` gives the harness named agent profiles. Each profile has its own system prompt, tools, model, permission rules, and step limit. + +## 1. Add the plugin + +```ts group=harness-agents +import { readFile, writeFile } from 'node:fs/promises' +import { resolve } from 'node:path' +import { toolDefinition } from '@tanstack/ai' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { agents } from '@tanstack/ai-harness/plugins/node' +import { anthropicText } from '@tanstack/ai-anthropic' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { z } from 'zod' + +const root = process.cwd() +const sonnet = anthropicText('claude-sonnet-5-5') + +const readTool = toolDefinition({ + name: 'read_file', + description: 'Read a file in the project', + inputSchema: z.object({ path: z.string() }), +}).server(({ path }) => readFile(resolve(root, path), 'utf8')) + +const writeTool = toolDefinition({ + name: 'write_file', + description: 'Write a file in the project', + inputSchema: z.object({ path: z.string(), content: z.string() }), +}).server(async ({ path, content }) => { + await writeFile(resolve(root, path), content) + return `Wrote ${path}.` +}) + +const coder = defineHarness({ + name: 'acme/coder', + adapter: sonnet, + tools: [readTool, writeTool], + plugins: () => [ + permissions({ root, rules: [{ tool: 'write_file', decision: 'ask', kind: 'edit' }] }), + agents({ adapter: () => sonnet }), + ], +}) +``` + +`adapter` turns a model id into an adapter. Here every agent uses the same model. + +## 2. Plan, then build + +```ts group=harness-agents +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(coder, { threadId: 'dark-mode' }) + +await session.runCommand({ name: 'agent', input: 'plan' }) +const plan = await session.send('How do I add a dark mode?') +console.log(plan.text) + +await session.runCommand({ name: 'agent', input: 'build' }) +await session.send('Do step 1.') +``` + +1. The `plan` agent reads the code and gives numbered steps. It has no tool that changes a file. +2. The `build` agent changes the files, after your answer to each [permission question](./permissions). + +- `session.runCommand({ name: 'agent' })` with no input shows the current agent and the list: `Agent: build. Agents: build, plan.` +- A switch applies at the next turn. + +The same switch is the `agent` setting. `session.describe().config` lists its options for a menu: + +```ts group=harness-agents +const setting = session.describe().config.find((entry) => entry.key === 'agent') +if (setting?.option.type === 'select') { + console.log(setting.value, setting.option.options) // 'build' ['build', 'plan'] +} + +await session.setPluginSetting({ key: 'agent', value: 'plan' }) +``` + +A client can change the setting only when the harness exposes it. Add `expose: { config: ['agent'] }` to `defineHarness`. + +## The built-in agents + +| Agent | Mode | Tools | What it does | +| --- | --- | --- | --- | +| `build` | `primary`, the default | Every tool | Reads code, changes files, and runs commands. | +| `plan` | `primary` | The read tools and `todo_write` | Reads the code and gives a plan in numbered steps. | +| `general` | `subagent` | Every tool | Does a task in many steps and reports back. | +| `explore` | `subagent` | The read tools | Searches the code and reports the paths that it finds. | + +The read tools are `read_file`, `list_files`, `grep`, `webfetch`, `websearch`, and `question`. An agent gets only the tools that the harness has. + +The mode of a profile says who starts it: + +- `primary`: it answers the user's turns. Pick it with the `agent` command or setting. +- `subagent`: the main model starts it as a child. +- `all`: both. + +The `plan` agent and the `plan` mode of [permissions](./permissions#pick-a-mode) are separate. The agent has only the read tools and `todo_write`. The mode denies edits and commands for every agent. To drop the built-in agents, set `builtIns: false`. + +## Add your own agent + +Give `agents()` a profile. This one reviews the changes with a stronger model: + +```ts group=harness-agents +import type { AnyTextAdapter } from '@tanstack/ai' +import type { AgentProfile } from '@tanstack/ai-harness/plugins/node' + +const review: AgentProfile = { + name: 'review', + description: 'Reviews the changes and lists the risks. Does not change files.', + mode: 'all', + model: 'claude-opus-5-5', + system: 'You review code. Read the diff, then list each risk with its file and line.', + tools: ['read_file', 'bash'], + permissions: [{ tool: 'bash', resource: 'git diff*', decision: 'allow' }], + steps: 20, +} + +const models: Record = { + 'claude-opus-5-5': anthropicText('claude-opus-5-5'), +} + +export const withReview = agents({ + adapter: (model) => models[model] ?? sonnet, + agents: [review], +}) +``` + +`name`, `description`, and `mode` are required. The other fields: + +| Field | What it does | +| --- | --- | +| `model` | A model id. `adapter` turns it into an adapter. Without it, a primary agent keeps the harness model, and a subagent uses the model of the turn that started it. | +| `system` | Text for the system prompt of each turn of this agent. | +| `tools` | The tools that this agent can call, by name. A `*` matches any text, for example `todo_*`. Without it, the agent gets every tool. | +| `permissions` | [Permission rules](./permissions#write-rules) that count while this agent is the primary agent. At that time, they also apply in each subagent run. | +| `steps` | The step limit. See [Limit the steps](#limit-the-steps). | +| `hidden` | `true` leaves the agent out of the `agent` command and setting. The model can still start it as a subagent. | + +- A profile replaces an earlier profile with the same name. The order is the built-in agents, then `dirs`, then `agents`. So a profile named `plan` replaces the built-in `plan`. +- `default` names the primary agent of a new session. Without it, the first primary agent is the default, which is `build`. +- In the main turn, a model that the thread stored with `session.configure` wins over the profile `model`. See [What wins](./thread-settings#what-wins). +- A `tools` list also filters the subagent tools. To let the agent start subagents, add their tool names, for example `explore`. + +## Load agents from Markdown files + +Keep your agents next to your code as Markdown files. Save this file as `.agents/agents/review.md`: + +```md +--- +description: Reviews the changes and lists the risks. Does not change files. +mode: all +model: claude-opus-5-5 +tools: [read_file] +steps: 20 +--- + +You review code. Read the diff, then list each risk with its file and line. +``` + +Then give `agents()` the folder: + +```ts group=harness-agents +export const fromFiles = agents({ + adapter: (model) => models[model] ?? sonnet, + dirs: [`${root}/.agents/agents`], +}) +``` + +- The file name is the agent name, here `review`. +- The body is the system prompt. +- The frontmatter sets `description`, `mode`, `model`, `tools`, `steps`, and `hidden: true`. The default `mode` is `all`. +- An unknown `mode`, or a `steps` that is not a whole number, stops `host.open` with an error, for example `Agent file review.md: "mode" is not valid.` +- The plugin reads the folder when a session opens, and again at each `session.reloadPlugins()`. A folder that it cannot read gives no agents. + +## Limit the steps + +An agent can call tools for a long time. `steps` sets the most model calls with tools in one run. With `steps: 20`: + +1. The agent makes up to 20 model calls with tools. +2. Call 21 gets `toolChoice: 'none'` and a note: answer now, and say what is left to do. +3. Then the run stops. + +The limit counts again in each turn of a primary agent and in each run of a subagent. + +Some providers do not obey `'none'`. On Amazon Bedrock, `'none'` after a tool call goes to the model as `auto`. See [Provider notes](../tools/tools#provider-notes). If the model calls a tool on the last call anyway: + +- The tool does not run. +- The model gets this tool result: `{ "error": "Step limit reached. Answer without calling tools." }` +- The run stops. That turn can end with no text answer. + +## Let the model start a subagent + +A profile with the mode `subagent` or `all` is also a subagent. The main model gets a tool for each one, with the name of the agent, for example `explore`. Ask the build agent: `use explore to find where the theme is set`. + +A subagent run gets these parts of its profile: + +- The tools of the turn that started it, filtered by its `tools`. +- Its `system` prompt. +- Its `model`, or the model of the turn that started it. +- Its step limit. + +A subagent run stops after 50 model calls, or when an answer is cut at the output limit. The harness also limits the depth and the number of children. See [Stay within limits](./subagents#stay-within-limits). To give one run from code its own budget and time limit, see [Give one run a budget and a time limit](./subagents#give-one-run-a-budget-and-a-time-limit). + +## Known limits + +- A subagent run uses the `permissions` of the primary agent, not the rules of its own profile. For example, when `build` is the primary agent and starts `review`, the `build` rules apply and the `review` rules do not. Put rules that must hold in every run in `permissions({ rules })`. See [Rules in subagents](./permissions#rules-in-subagents). +- A Markdown agent cannot set `permissions`. Put those rules in a profile in code. +- On Amazon Bedrock, the model can still call a tool on the last call. The tool does not run, but the turn can end with no text answer. +- A subagent run gets the tools and the model of its turn from the memory of the host that ran that turn. On another host, or after a restart, the run fails with `Agent cannot go on here`. Start it again from a turn. +- A subagent run that your code starts, not a turn, has no tools of a turn. Give its profile a `model`. + +## What you have now + +- A read-only `plan` agent, and a `build` agent to do the work. +- Your own agents in code or in Markdown files, each with its own prompt, tools, model, and rules. +- A step limit that ends a run with a text answer. +- Subagents that the main model starts to search and to research. diff --git a/docs/harness/auth.md b/docs/harness/auth.md new file mode 100644 index 0000000000..7ffbce667a --- /dev/null +++ b/docs/harness/auth.md @@ -0,0 +1,332 @@ +--- +title: Auth and connectors +id: harness-auth +order: 7 +description: "Let users sign in to GitHub and other OAuth services from the harness. Tokens stay in your credential store and never reach the model." +keywords: + - tanstack ai + - harness + - oauth + - connectors + - credentials +--- + +Your agent needs to open a pull request, so it needs the user's GitHub token. The user should sign in once, in the browser, and the model should never see the token. `oauthConnector` does that: it adds `/connect github`, stores the token, and hands it to your tools. + +## 1. Add a connector + +```ts group=harness-auth +import { toolDefinition } from '@tanstack/ai' +import { defineHarness, oauthConnector } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +const github = oauthConnector({ + id: 'github', + label: 'GitHub', + // The harness runs on the user's machine (a CLI), so open a browser here. + login: 'loopback', + oauth: { + authorizationUrl: 'https://github.com/login/oauth/authorize', + tokenUrl: 'https://github.com/login/oauth/access_token', + deviceUrl: 'https://github.com/login/device/code', + clientId: 'your-github-oauth-app-client-id', + scopes: ['repo'], + }, + tools: (token) => [ + toolDefinition({ name: 'list_issues', description: 'List my open issues' }).server( + async (_args, ctx) => { + const response = await fetch('https://api.github.com/issues', { + headers: { authorization: `Bearer ${await token({ run: ctx.context.run })}` }, + }) + return response.json() + }, + ), + ], +}) + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [github], +}) +``` + +`token({ run })` returns a fresh access token of the user of that run. If the token is expired and the service gave a refresh token, `token` refreshes it first, one time for all the calls that need it. See [Refresh a token](#refresh-a-token). Pass the `run` of the tool call, `ctx.context.run`, so each user gets their own token. See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). + +## 2. Sign in + +- The user runs `/connect github`. +- With `login: 'loopback'`, the CLI opens the browser. The harness listens on `127.0.0.1` on a random port for one callback, with PKCE and a random `state`. +- With `oauth.issuer` set, the sign-in refuses a code that the service sends with another `iss` (RFC 9207). +- The token goes into the credential store. `/disconnect github` deletes it. +- A cancel of the command, or a close of the session, stops a sign-in that waits. The listener on `127.0.0.1` closes at once. + +`login` is required. Use `login: 'device'` for a harness on a server, and for SSH sessions, containers, and CI. The user then gets a code to enter on the service's page. Device login needs `oauth.deviceUrl`. + +If a tool runs before sign-in, it stops with a `harness.auth_required` event. The CLI shows which service to connect. In your own app, `session.onAuthRequired` tells you which service to show: + +```ts group=harness-auth-prompt +import type { HarnessSession } from '@tanstack/ai-harness' + +export function showSignIns(session: HarnessSession) { + // Call the returned function to stop the calls. + return session.onAuthRequired(({ connector, url, userCode }) => { + console.log(`Sign in to ${connector}: ${url ?? `run /connect ${connector}`}`) + if (userCode) console.log(`Enter this code on that page: ${userCode}`) + }) +} +``` + +## Wait for the sign-in + +A turn from a background agent or an email runs while nobody watches it. That turn can need a sign-in. Without a sign-in, the tool fails, and the work of that turn stops. Pass `wait: true` to `token`. Then the turn waits for the sign-in and continues after it. + +```ts group=harness-auth-wait +import { toolDefinition } from '@tanstack/ai' +import { defineHarness, oauthConnector } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +const github = oauthConnector({ + id: 'github', + label: 'GitHub', + // The server cannot open a browser for the user, so show a code. + login: 'device', + oauth: { + authorizationUrl: 'https://github.com/login/oauth/authorize', + tokenUrl: 'https://github.com/login/oauth/access_token', + deviceUrl: 'https://github.com/login/device/code', + clientId: 'your-github-oauth-app-client-id', + scopes: ['repo'], + }, + tools: (token) => [ + toolDefinition({ name: 'list_issues', description: 'List my open issues' }).server( + async (_args, ctx) => { + const response = await fetch('https://api.github.com/issues', { + headers: { authorization: `Bearer ${await token({ run: ctx.context.run, wait: true })}` }, + }) + return response.json() + }, + ), + ], +}) + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [github], + // A web app starts the sign-in with this command. + expose: { commands: ['connect:github'] }, +}) +``` + +In a tool of your own plugin, use `ctx.context.run.credentials.require('github', { wait: true })`. + +When the credential is missing, the turn stops with an interrupt: + +- The interrupt has `reason: 'auth_required'`. Its `metadata['tanstack:interruptPayload'].request.connector` is `'github'`. +- The sign-in belongs to the sender of the stopped turn. When that person saves the credential, the same turn continues as that person, and the tool runs again. The save can be `/connect github`, the `connect:github` command, or `run.credentials.set` in a command of your own. +- A credential that another person saves never continues the turn. +- To stop the wait, resolve the interrupt with `status: 'cancelled'`. Then the tool fails, the same as without `wait`. +- On a [durable host](./durable-sessions), the wait continues after a restart. +- `wait` is for a tool of the running chat turn. Do not pass it in a command, a background agent, or a hook. The credentials of a command never wait. +- Provider keys (`run.keys.require`) do not wait. + +In a web app, the wait shows in `state.signIns` of the session view, also after a reload. When the user presses your connect button, run the connector command. Each sign-in then has the `url` and the `userCode` to show. A client can run only the commands in `expose.commands`, so the harness above exposes `connect:github` (see [Choose what clients can change](./connect#choose-what-clients-can-change)): + +```ts group=harness-auth-wait-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import { createSessionView } from '@tanstack/ai-harness/view' + +const view = createSessionView( + createHarnessClient({ url: '/api/harness', threadId: 'user-1-thread' }), +) + +view.on('signIn', (signIn) => { + console.log(`Connect ${signIn.connector} to continue the turn.`) +}) + +export const connectGithub = () => view.runCommand({ name: 'connect:github' }) +``` + +After the sign-in, the turn finishes its work with no new message. + +## Keep credentials + +Pass a credential store to the host. The host reads credentials from it, keyed by the user of each run. Without one, credentials live in memory until the process stops. When several people write in one thread, see [Share a thread between people](./shared-threads). + +```ts group=harness-auth +import { defineCredentialStore } from '@tanstack/ai/byok' +import type { Credential } from '@tanstack/ai/byok' + +const saved = new Map() +const keyOf = (userId: string | undefined, id: string) => `${userId ?? 'tenant'}:${id}` + +export const credentials = defineCredentialStore({ + get: async ({ scope, id }) => saved.get(keyOf(scope.userId, id)) ?? null, + set: async ({ scope, id, credential }) => { + saved.set(keyOf(scope.userId, id), credential) + }, + delete: async ({ scope, id }) => { + saved.delete(keyOf(scope.userId, id)) + }, + list: async (scope) => + [...saved.entries()] + .filter(([key]) => key.startsWith(`${scope.userId ?? 'tenant'}:`)) + .map(([key, credential]) => ({ id: key.split(':')[1] ?? key, type: credential.type })), +}) +``` + +```ts group=harness-auth +import { createHarnessHost } from '@tanstack/ai-harness' + +export const host = createHarnessHost({ credentials }) +``` + +The store can be a table in your own database or a secrets manager. See the rules in [Write a store for production](../advanced/server-keys#write-a-store-for-production). + +- Encrypt tokens at rest in a real store. +- `list` returns ids and types only, never the secret values. +- The harness keeps a secret answer in the store for a short time, with the type `secret_answer`. Leave that type out of a list of connected accounts. See [Secret answers on another host](./durable-sessions#secret-answers-on-another-host). +- A credential saved without a `userId` belongs to the whole tenant. A user who has no credential of their own uses it. See [Share one credential with the whole organization](./shared-threads#share-one-credential-with-the-whole-organization). +- If more than one host uses the store, add `getVersioned` and `setIf`. Then two hosts do not both save a refreshed token. See [Add compare-and-set](../advanced/server-keys#add-compare-and-set). + +## Read credentials in your own plugin + +Your plugin reads the credentials of the user of each run: + +- In a tool or a middleware hook, `ctx.context.run.credentials.require('github')` returns the credential. Before the user signs in, it stops with `auth_required`. +- In a command, use `run.credentials` of the command context. +- The `ctx.credentials` of a session plugin have no user, so each call throws. + +See [Act as the user of a run](./plugins#act-as-the-user-of-a-run). + +## Keep tokens out of the log and the model + +A tool can show a token by mistake: an error message with the request URL, or a debug field in its result. The session replaces each secret that it gave out with `[redacted]` before it keeps or sends anything. + +The secrets that the session knows: + +- The provider keys of `run.keys`, also a key from an env var. +- The credentials of `run.credentials` and of the connectors: API keys, OAuth tokens, and the OAuth client secret. +- The answer to a question with `secret: true`. For an answer with fields, each string field. + +Where the session replaces them: + +- The session log: your records, the transcript, plugin state (also in a `Map` or a `Set`), notes, and tool results. +- The events that clients get. +- The model context, so the model never sees a secret. +- The error frames of the HTTP handler, the WebSocket, and the stdio worker. + +A [durable step](./durable-tools) value with a secret throws, so the log never keeps it. Read the secret outside `step.do`. + +The session finds the raw value, its JSON-escaped form, and its URL-encoded form. It does not find: + +- A value shorter than 6 characters. So a short value does not remove common words. +- A secret that a stream splits across two deltas, in those two events. The message in the transcript does not have the secret. +- A secret that is JSON-escaped two times. +- A secret in an instance of your own class, a `Date`, or a typed array in plugin state. + +To scrub a text of your own, use `scrubSecrets(text, secrets)` from `@tanstack/ai/byok`. It uses the same rules. + +## Write your own sign-in command + +`oauthConnector` covers most services. For a sign-in of your own, use the helpers from `@tanstack/ai-harness/oauth`. Pass the `signal` of the command, so a cancel stops the sign-in at once: + +```ts group=harness-auth-own +import { defineCommand, definePlugin } from '@tanstack/ai-harness' +import { deviceLogin, isExpired, refreshCredential } from '@tanstack/ai-harness/oauth' +import type { OAuthConfig } from '@tanstack/ai-harness/oauth' + +const gitlab: OAuthConfig = { + authorizationUrl: 'https://gitlab.com/oauth/authorize', + tokenUrl: 'https://gitlab.com/oauth/token', + deviceUrl: 'https://gitlab.com/oauth/authorize_device', + clientId: 'your-gitlab-app-client-id', + scopes: ['read_api'], +} + +export const gitlabLogin = definePlugin({ + name: 'acme/gitlab-login', + setup: (ctx) => ({ + commands: { + 'connect:gitlab': defineCommand({ + description: 'Sign in to GitLab', + run: async (_input, { credentials, signal }) => { + const credential = await deviceLogin(gitlab, { + onCode: ({ userCode, verificationUri }) => + ctx.session.authRequired({ connector: 'gitlab', url: verificationUri, userCode }), + signal, + }) + await credentials.set({ id: 'gitlab', credential }) + return 'Connected to GitLab.' + }, + }), + }, + }), +}) +``` + +- `deviceLogin`, `loopbackLogin`, `exchangeCode`, and `refreshCredential` take a `signal`. When it aborts, the call stops at once and rejects with the reason of the signal. +- The `signal` of a command aborts on a cancel of the command and on a close of the session. +- `loopbackLogin` takes `timeoutMs`. The default is 10 minutes. This timer does not keep the process alive. +- When another library runs the OAuth flow, `startLoopbackReceiver({ timeoutMs, signal })` only waits for the callback. `waitForCode(state)` gives the code and the `iss` of the callback. + +### Refresh a token + +An access token expires, and several tool calls can need it at the same time. Some services rotate the refresh token: after one refresh, the old refresh token stops working. Then a second refresh at the same time fails, and the user must sign in again. + +`run.credentials.refresh(id, update, { signal })` runs one refresh, and every caller gets its result: + +```ts group=harness-auth-own +import { toolDefinition } from '@tanstack/ai' +import type { HarnessRunContext } from '@tanstack/ai-harness' + +export const gitlabProjects = toolDefinition({ + name: 'gitlab_projects', + description: 'List my GitLab projects', +}).server(async (_args, ctx) => { + const { credentials, signal } = ctx.context.run + let credential = await credentials.require('gitlab') + if (isExpired(credential)) { + const fresh = await credentials.refresh( + 'gitlab', + // Another call can refresh first. Then keep its credential. + (current, hostSignal) => + isExpired(current) + ? refreshCredential(gitlab, current, { signal: hostSignal }) + : Promise.resolve(current), + { signal }, + ) + // The user disconnected meanwhile: ask for a sign-in. + credential = fresh ?? (await credentials.require('gitlab')) + } + const token = credential.type === 'oauth' ? credential.accessToken : credential.value + const response = await fetch('https://gitlab.com/api/v4/projects?membership=true', { + headers: { authorization: `Bearer ${token}` }, + signal, + }) + return response.json() +}) +``` + +`update` gets the saved credential and returns the new one. Return `current` to keep it. `refresh` resolves with the credential to use, or `null` when none is saved. `oauthConnector` refreshes with `refresh` too. + +What `refresh` does: + +- One host runs one `update` at a time for each saved credential. A call that comes while one runs waits for it, and gets the same credential. +- It saves the new credential to the scope where it found it. A tenant token stays the tenant's, so every user of the tenant gets the new token. +- If the store has `getVersioned` and `setIf`, the save is a compare-and-set. When another host saved first, the call uses that token and does not refresh again. +- Without that pair, the one-at-a-time rule is per host only. Two hosts can refresh at the same time, and the last save wins. +- `update` gets a signal of the host, which only `host.close()` aborts. An abort of your `signal` rejects only your call. The refresh goes on for the other calls. + +`refreshCredential` keeps the OAuth `client` of the old credential, so the next refresh has it too. + +## What you have now + +- Browser and device-code sign-in for any OAuth service, and sign-ins of your own that stop at once on a cancel. +- Tools that get a fresh token without the model seeing it. +- One refresh for each expired token, also on many hosts. +- A log, events, and a model context with `[redacted]` in place of each secret. +- Turns that wait for a sign-in and then finish their work. diff --git a/docs/harness/cli.md b/docs/harness/cli.md new file mode 100644 index 0000000000..d67ae84b6c --- /dev/null +++ b/docs/harness/cli.md @@ -0,0 +1,240 @@ +--- +title: Run a harness in the terminal +id: harness-cli +order: 3 +description: "Run your harness in a terminal with line mode or your own screen, a print mode for scripts and CI, NDJSON output, an ACP mode for editors, and an HTTP server." +keywords: + - tanstack ai + - harness + - cli + - terminal + - ink +--- + +You built a harness and want to use it like Claude Code: type in a terminal, watch it work, approve tools. You also want to run it in CI. `runCli` gives one harness all of these modes. + +## Install + + + +react: @tanstack/ai-harness-cli +vue: @tanstack/ai-harness-cli +solid: @tanstack/ai-harness-cli +svelte: @tanstack/ai-harness-cli +preact: @tanstack/ai-harness-cli +angular: @tanstack/ai-harness-cli +vanilla: @tanstack/ai-harness-cli +octane: @tanstack/ai-harness-cli + + + +## 1. Write the entry file + +```ts group=harness-cli +import { defineHarness } from '@tanstack/ai-harness' +import { runCli } from '@tanstack/ai-harness-cli' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +process.exitCode = await runCli(assistant) +``` + +## 2. Pick a mode + +To use the harness yourself: + +- No flags: line mode. Type a message and press Enter. With a `ui`, your own screen starts in its place (see [Run your own screen](#run-your-own-screen)). +- `-p "prompt"`: run one prompt, print the answer, and exit. +- `-p "prompt" --output ndjson`: print every AG-UI event as one JSON line. + +To use the harness from another program: + +- `--acp`: serve the harness as an ACP v2 agent over stdio, for editors. Needs `@tanstack/ai-acp` with harness support. +- `--mcp`: serve the harness as an MCP server over stdio, for Claude Code, Cursor, and other MCP clients. Needs `@tanstack/ai-mcp` with harness support. Add `--yes` to approve every tool call. +- `--serve`: serve the session protocol over HTTP on `127.0.0.1:8787`. Every request needs the bearer token. Pass `--token`, set `HARNESS_TOKEN`, or copy the token the CLI prints. With `@tanstack/ai-mcp` and its harness support, it also serves MCP at `/mcp`. One token, one user, every thread, so keep it on a network you trust (see [Serve over HTTP safely](#serve-over-http-safely)). + +`--mcp` and `--serve` run only the commands in `expose.commands` of the harness. `--serve` also sets only the config keys in `expose.config`. Add the names to `expose` in `defineHarness`, for example `expose: { commands: ['connect:github'] }`. Line mode and `-p` run every command. See [Choose what clients can change](./connect#choose-what-clients-can-change). + +Line mode reads one message or command per line. In a terminal, you can type while a turn runs: + +- `/cancel` cancels the running turn at once. +- A message joins the running turn as a steer. The model reads it at its next call. +- A message that you type when no turn runs starts a new turn. + +With piped input, each line waits for the turn that the line before it started. A script runs in order. In a terminal, line mode also opens sign-in links in the browser. + +## 3. Use it in CI + +`-p` exits with a code your script can check: + +| Code | Meaning | +|---|---| +| 0 | The turn finished. | +| 1 | The turn failed. | +| 3 | The turn waits for a person: an approval, an answer, or a sign-in. | +| 130 | The turn was cancelled, or SIGINT (Ctrl+C) stopped the CLI. | +| 143 | SIGTERM cancelled the turn or stopped the CLI. | + +`EXIT` from `@tanstack/ai-harness-cli` has these codes, for example `EXIT.needsAction` (3). `-p` exits 3 at the first [wait](./waits) of the turn, because nobody can answer in this mode. It writes what the turn waits for on stderr. On a durable host, the wait stays in the log, so a later run, or another server, can answer it. + +The codes 130 and 143 apply in every mode. See [Stop with Ctrl+C](#stop-with-ctrlc). + +## Stop with Ctrl+C + +Ctrl+C works the same in every mode: + +- The first Ctrl+C (SIGINT) or SIGTERM cancels the running turn. +- A second one, or one while no turn runs, stops the CLI. `runCli` resolves to 130 for SIGINT, and to 143 for SIGTERM. +- When the CLI made the host, it closes the host with `recoverable: true`. Another host then runs the cut work again. +- When you gave the CLI your own `host`, the CLI releases only its session handle. Your host stays open. + +The CLI hears the signals on `process`. To send them yourself, for example in a test, pass a `signals` source. An `EventEmitter` works: + +```ts group=harness-cli-signals +import { EventEmitter } from 'node:events' +import { defineHarness } from '@tanstack/ai-harness' +import { runCli } from '@tanstack/ai-harness-cli' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const signals = new EventEmitter() +const exit = runCli(assistant, { argv: ['-p', 'Write a long story.'], signals }) +// No turn runs yet, so this stops the CLI. +signals.emit('SIGTERM') +console.log(await exit) // 143 +``` + +## Serve over HTTP safely + +`--serve` is the quick way to put a harness on HTTP, so it makes the trust decisions for you: one bearer token, one user, every thread. Anyone with the token acts as that one user. They can open or drive any thread on the host, including the tools it runs (shell commands, file edits) on the machine the CLI runs on. Treat the token like a shell login. + +- It binds to `127.0.0.1:8787`, reachable only from the same machine. Change the bind with `--host` and `--port`. `--host 0.0.0.0` puts it on the network, so do that only behind TLS and a proxy you trust. +- The CLI signs media links with its own key, never with the token. By default it makes a new random key at each start, so media links stop working after a restart. To keep them working, pass `--media-key` or set `HARNESS_MEDIA_KEY`, with a key of at least 32 bytes. Change the key to stop every link a turn already handed out. + +Need per-user access, your own sign-in check, or a rule for which threads each user can open? Use [`createHarnessHandler`](./connect) on a host with [`access`](./access) instead. The host reads the user of each request and checks each action, so no thread is open by accident. + +## Use your own host and user + +By default, the CLI builds its own host in memory and opens sessions with no user. Your server already has a durable host with its own stores, and each person has their own credentials. Give the CLI that host and the user: + +```ts group=harness-cli-host +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { runCli } from '@tanstack/ai-harness-cli' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { memoryCredentialStore } from '@tanstack/ai/byok/server' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + credentials: memoryCredentialStore(), +}) + +process.exitCode = await runCli(assistant, { host, user: { id: 'ada' } }) +await host.close() +``` + +- You own the host, so the CLI does not close it. Close it yourself when `runCli` returns. Without `host`, the CLI builds a host and closes it. +- Give `host`, or `persistence` and `credentials`, not both. With both, `runCli` writes `Give runCli a host, or persistence and credentials, not both.` and returns code 1. +- Line mode, `-p`, your own `ui`, `--acp`, and `--mcp` open their sessions as `user`, so `/connect` saves keys for that user. +- `--serve` checks the token first. On a host without `access`, each request then runs as `user`. On a host with [`access`](./access), `getUser` reads the user of each request. +- On a host with `access`, give `user` too. `--mcp` does not start on such a host, because stdio has no login. + +## Commands in line mode + +For the session: + +- `/agents`: list the agents. +- `/agent {"json":"input"}`: run an agent in the background. When it is done, a new turn starts with its result. +- `/config`: show the settings. `/config ` changes one. +- `/connect ` and `/disconnect `: sign in to a connector, or out. + +For the running work: + +- `/cancel`: cancel the running turn. +- `/status`: show what runs and what waits. +- `/exit`: quit. + +Plugin commands (for example `/model` or `/todos`) show up in `/help`. + +When a turn waits for you, each next line answers one wait, in order. The turn continues after the last one: + +- An approval: `y` approves and `n` refuses. An approval of `permissions()` also takes `once`, `always`, or `reject`. +- A question: type the answer. The CLI reads it with the schema of the question. +- A sign-in: sign in, then type `y` to go on, or `n` to skip it. +- A client tool: the terminal cannot run it. Any line skips it. + +## Send files and save media + +To send a file with a message, write `@` and the path of the file: + +```text +what is wrong in @./bug.png +compare @"old logo.png" with @./new-logo.png +``` + +- The path is relative to the working folder. Put quotes around a path with spaces. +- `@path` works in line mode, with `-p`, with piped input, and in your own `ui`. +- A `@word` that is not a file stays in the text, for example `@types/node`. +- If the CLI does not know the type of the file, it does not send the message. To send the path as text, remove the `@`. + +The CLI saves the media that a turn makes in a folder. The default folder is `./-media` in the working folder, for example `./acme-assistant-media`. To use another folder, add `--media-dir `: + +```bash +npx tsx cli.ts -p "Draw a logo for acme" --media-dir ./out +``` + +- Line mode prints `[image saved: ]` for each file. +- `-p` prints the same line on stderr, so stdout keeps only the answer. +- `-p --output ndjson` adds the saved `path` to the value of the `harness.media` event. + +The CLI never replaces a file. If the name is taken, it adds `-1`, `-2`, and so on to the new name. + +## Run your own screen + +Line mode prints plain lines. For a full screen with your own layout, pass `ui` to `runCli`. It works with any TUI library, for example Ink, OpenTUI, or blessed. + +`ui` gets a ready [session view](./custom-ui) and resolves when the user quits. This entry file starts an Ink screen: + +```tsx ignore +import { render } from 'ink' +import { runCli } from '@tanstack/ai-harness-cli' +import { assistant } from './harness' +import { Screen } from './screen' + +process.exitCode = await runCli(assistant, { + ui: async (view) => { + await render().waitUntilExit() + }, +}) +``` + +- `ui` runs only in an interactive terminal. Piped input uses line mode. `-p`, `--acp`, `--mcp`, and `--serve` do not use `ui`. +- When `ui` resolves, the CLI disposes the view and `runCli` returns. +- In a reopened thread, a saved child agent shows only its card. To show what it wrote, call `view.loadAgent({ id })`, for example when the screen draws the card. See [Show subagents](./custom-ui#show-subagents). +- To write `Screen`, read [Build your own UI](./custom-ui). + +For an Ink screen with approvals, questions, sign-ins, and child agents, copy [`examples/harness-cli/src/tui.tsx`](https://github.com/TanStack/ai/blob/main/examples/harness-cli/src/tui.tsx). + +## What you have now + +- One entry file that runs your harness as a terminal app, a script step, an editor agent, an MCP server, or an HTTP server. +- A CLI that uses your durable host and runs as your user. +- A Ctrl+C that cancels the turn first, then stops with exit code 130. +- Exit code 3 from `-p` when a turn waits for a person, and one line for each answer in line mode. +- Your own terminal screen on the same session, with any TUI library. +- Files that you send with `@path`, and a folder with the media that your agents make. + +Next: keep long turns alive through crashes with [durable sessions](./durable-sessions). diff --git a/docs/harness/code-mode.md b/docs/harness/code-mode.md new file mode 100644 index 0000000000..ccd2a33b30 --- /dev/null +++ b/docs/harness/code-mode.md @@ -0,0 +1,235 @@ +--- +title: Code mode in a harness +id: harness-code-mode +order: 13 +description: "Let the harness model write one TypeScript program that calls many tools, also the coding tools under their permission rules, and run it in an isolate. Any TanStack AI isolate driver plugs in." +keywords: + - tanstack ai + - harness + - code mode + - isolate + - quickjs + - permissions +--- + +Your agent has many read-only tools, and it calls them one at a time. Each call is a model round trip, and every tool schema goes into every request. With code mode, the model writes one TypeScript program that calls the tools it needs, and the program runs in an isolate. The read-only tools leave the tool list and become functions in that program. The [coding tools](#run-the-coding-tools-in-a-program) can move too, and each of their calls keeps its permission rules. + +## 1. Add the plugin + + + +react: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +vue: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +solid: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +svelte: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +preact: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +angular: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +octane: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs +vanilla: @tanstack/ai-code-mode @tanstack/ai-isolate-quickjs + + + +```ts group=harness-code-mode +import { toolDefinition } from '@tanstack/ai' +import { codeMode } from '@tanstack/ai-code-mode/harness' +import { defineHarness } from '@tanstack/ai-harness' +import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs' +import { openaiText } from '@tanstack/ai-openai' +import { z } from 'zod' + +const issues = [{ id: 'ENG-1', title: 'Fix login', state: 'open' }] + +const listIssues = toolDefinition({ + name: 'list_issues', + description: 'List the issues of the team', + inputSchema: z.object({ state: z.enum(['open', 'closed']) }), +}).server(async ({ state }) => issues.filter((issue) => issue.state === state)) + +const getIssue = toolDefinition({ + name: 'get_issue', + description: 'Get one issue by id', + inputSchema: z.object({ id: z.string() }), +}).server(async ({ id }) => issues.find((issue) => issue.id === id) ?? null) + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), + tools: [listIssues, getIssue], + plugins: () => [codeMode({ driver: createQuickJSIsolateDriver() })], +}) +``` + +The model now has an `execute_typescript` tool. Inside the program, each moved tool is an `external_*` function, for example `external_list_issues({ state: 'open' })`. + +## 2. Pick the isolate + +`driver` takes any isolate driver. Change one line to run the programs somewhere else: + +| Driver | Package | Runs in | +| --- | --- | --- | +| `createQuickJSIsolateDriver()` | `@tanstack/ai-isolate-quickjs` | WebAssembly, on any runtime | +| `createQuickJSBunIsolateDriver()` | `@tanstack/ai-isolate-quickjs-bun` | Native QuickJS in Bun | +| `createNodeIsolateDriver()` | `@tanstack/ai-isolate-node` | A V8 isolate in Node | +| `createCloudflareIsolateDriver()` | `@tanstack/ai-isolate-cloudflare` | A Cloudflare Worker | +| `createDaytonaIsolateDriver()` | `@tanstack/ai-isolate-daytona` | A Daytona sandbox | +| `createE2BIsolateDriver()` | `@tanstack/ai-isolate-e2b` | An E2B sandbox | + +[Code mode isolates](../code-mode/code-mode-isolates) has the options of each driver. + +## 3. Choose which tools move + +By default, code mode moves only the tools whose calls need no question and no check: + +- The tool runs on the server and does not set `needsApproval`. +- The permission rules allow it in plan mode. A tool with an `edit` or `execute` rule stays. +- The tool does not declare `PermissionResources`, so the rules check no path or command of its calls. The coding tools stay. + +Where the permission rules come from: + +- With [`permissions()`](./permissions) in the plugin list, code mode asks it about each tool. The answer is the one a call gets in plan mode: your `rules` win, and a tool with no rule gets your `default`. So with `default: 'ask'`, a tool with no rule stays. +- Without `permissions()`, code mode reads the rules that tool plugins add. + +Every other tool stays a normal tool call, with its approvals. + +Code mode also adds `allow` rules for `execute_typescript` and `discover_tools`, so a program itself does not ask, also with `default: 'ask'`. Each call in the program gets the rules of its own tool. Your own `rules` still win: `{ tool: 'execute_typescript', decision: 'ask' }` asks before each program. With `getSnippetBindings`, code mode adds no rule, because a snippet runs code of its own. + +To pick the tools yourself, pass `include`: + +```ts group=harness-code-mode +export const onlyIssues = codeMode({ + driver: createQuickJSIsolateDriver(), + include: (tool) => tool.name.endsWith('_issue') || tool.name.endsWith('_issues'), +}) +``` + +- `include` can also pick a tool that asks or that declares `PermissionResources`, for example a [coding tool](#run-the-coding-tools-in-a-program). Each call of such a tool in a program gets the decision of a direct call. +- `include` never moves a tool with `needsApproval`, or a tool that the rules deny in the mode of the session. A `deny` rule with a `resource` also counts, for example `{ tool: 'read_file', resource: '.env', decision: 'deny' }`. Such a tool stays a normal tool call, and code mode writes one warning to the host `logger` (the console by default). +- The rules decide again at each turn, in the permission mode of the session. So in `plan` mode, a tool that edits or runs a command stays a normal tool call. +- A safe tool with `metadata.codeMode: true` moves even when `include` does not pick it. + +The other options of `createCodeMode` pass through: `timeout`, `memoryLimit`, `lazyToolsConfig`, and more. + +## 4. Send only the tool names + +The prompt of code mode lists each moved tool with its full TypeScript type. With 100 tools, that is a lot of tokens on every turn, and most turns use a few tools. Set `lazy: true`, and the prompt lists only the names: + +```ts group=harness-code-mode +export const lazyCodeMode = codeMode({ + driver: createQuickJSIsolateDriver(), + lazy: true, + lazyToolsConfig: { includeDescription: 'first-sentence' }, +}) +``` + +The model then works in three steps: + +1. It reads the tool names in the prompt. +2. It calls `discover_tools` with the names it needs, and gets their TypeScript signatures. +3. It calls those tools in `execute_typescript`. + +`includeDescription` sets what the prompt shows with each name: `'none'` (the default, the name only), `'first-sentence'`, or `'full'`. Tools that stay outside code mode keep their full schema. + +## Run the coding tools in a program + +To fix a bug, the model reads a file, edits it, and runs the tests. Each step is a tool call and a model round trip. Move the [coding tools](./coding-tools) into code mode, and one program does all the steps. Each call in the program still gets the [permission rules](./permissions) of its tool. + +```ts group=harness-code-mode-coding +import { codeMode } from '@tanstack/ai-code-mode/harness' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { workspaceTools } from '@tanstack/ai-harness/plugins/coding' +import { createQuickJSIsolateDriver } from '@tanstack/ai-isolate-quickjs' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const root = process.cwd() + +export const coder = defineHarness({ + name: 'acme/coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + permissions({ root }), + workspaceTools({ root }), + codeMode({ driver: createQuickJSIsolateDriver(), include: () => true }), + ], +}) +``` + +`include: () => true` picks every tool of this harness. Mount `codeMode()` after `workspaceTools()`. Then the program gets the same edit tool as the model: `editStyle` picks `edit_file` or `patch`. With the default `'auto'`, a GPT model gets `patch`. + +Each `external_*` call of a coding tool gets the decision of a direct call, for the paths and the commands of the call: + +- `allow`: the call runs. +- `deny`: the call throws in the program, with the reason. For example, in `plan` mode a read of `.env` throws `This tool is not allowed in plan mode.` +- `ask`: the program waits for the user. A yes runs the call, and a no makes the call throw. + +The prompt of code mode tells the model that these calls can wait and can throw. + +### Answer the question + +A call that asks opens a [wait](./waits) of kind `question`, with the schema of an approval. The turn does not stop while the program waits. Answer the question with `session.answer`: + +```ts group=harness-code-mode-coding +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(coder, { threadId: 'main' }) + +const turn = session.send('Run the tests, then fix the first failure.') +const wait = await session.waitForRun({ runId: turn.id }) +// wait.kind is 'question'. wait.message is, for example: +// Allow bash {"command":"pnpm test"}? Answer once, always, or reject. +await session.answer({ questionId: wait.waitId, value: 'once' }) +``` + +In a UI, the question is an item of `questions` in a [session view](./custom-ui#show-a-question). In the [CLI](./cli#commands-in-line-mode), type the answer. + +```ts group=harness-code-mode-coding +import { createSessionView } from '@tanstack/ai-harness/view' + +const view = createSessionView(session) + +// Your buttons call this with 'once', 'always', or 'reject'. +async function answerQuestion(answer: 'once' | 'always' | 'reject') { + const [question] = view.store.get().questions + await question?.answer(answer) +} +``` + +- `'once'` runs the call. `'always'` also saves an allow rule, as a direct approval does. +- `'reject'` rejects the call, so it throws. `{ answer: 'reject', message: 'Use pnpm, not npm.' }` also gives the program your message as the error. +- An answer that does not fit the schema, for example `'y'`, gets a `rejected` receipt, and the question stays open. +- The program continues after the answer. Each later call that asks opens a new question. +- In an agent that the model calls as a tool, a call that asks does not run. That agent cannot ask the user, so the call throws. + +### Time limits and host stops + +A program has 30 seconds by default. With a coding tool in it, the default is 10 minutes, because the wait for the user counts. Set `timeout` in milliseconds to change it. When the time runs out, the program stops, and its open question closes. + +With a coding tool, `execute_typescript` is a [durable tool](./durable-tools). The session log keeps the result of each finished call of a coding tool. After a host stop, the next host runs the program again: a finished call returns its kept result, and the same question waits. If the stop cut an edit or a command while it ran, that call throws and does not run again. + +## Keep direct calls + +A moved tool leaves the tool list of the model. Then the model needs a program even for one small call. Set `directCalls: 'keep'`, and the model can call each moved tool directly too: + +```ts group=harness-code-mode +export const keepDirect = codeMode({ + driver: createQuickJSIsolateDriver(), + directCalls: 'keep', +}) +``` + +- `'remove'` (default): a moved tool is only a function in the program. +- `'keep'`: a moved tool is also a normal tool. A direct call gets its approval, as any tool call does. +- `directCalls` is also a session setting: `session.setPluginSetting({ key: 'directCalls', value: 'keep' })`, or `/config directCalls keep` in the [CLI](./cli#commands-in-line-mode). A change applies from the next turn. +- A client can change it only with `directCalls` in `expose.config`. See [Choose what clients can change](./connect#choose-what-clients-can-change). + +## Tools that appear while the session runs + +The plugin chooses the tools again before every turn. A plugin that finds tools at run time with `discoverTools` (for example after a sign-in) adds them first, and code mode picks from them on the next turn. Tool names that are not JavaScript identifiers, for example `notion-search`, get a safe name inside the program: `external_notion_search`. When two tools get the same safe name, only the first one moves. The other one stays a normal tool call. + +## What you have now + +- One `execute_typescript` tool in place of many tool calls. +- Programs that read, edit, and run commands, under the same permission rules as a direct call. +- With `lazy: true`, a prompt that lists only the tool names. +- Programs that run in the isolate you choose. +- An approval for each call that asks, in a program or as a normal tool call. diff --git a/docs/harness/coding-agent.md b/docs/harness/coding-agent.md new file mode 100644 index 0000000000..81ffed8f91 --- /dev/null +++ b/docs/harness/coding-agent.md @@ -0,0 +1,200 @@ +--- +title: Build a coding agent +id: harness-coding-agent +order: 6 +description: "Turn a harness into a coding agent with file tools, permission modes, undo, a todo list, a model picker, project instructions, session titles, a cost count, and /compact." +keywords: + - tanstack ai + - harness + - coding agent + - permissions + - workspace tools + - project instructions + - undo +--- + +You want your own coding agent in the terminal. It reads and edits files, and runs commands after you approve them. When a turn goes wrong, you undo it. The first-party plugins in `@tanstack/ai-harness/plugins`, `@tanstack/ai-harness/plugins/node`, and `@tanstack/ai-harness/plugins/coding` give you those parts. You pick the model and the rules. + +## 1. Define the agent + +```ts group=harness-coding-agent +import { randomUUID } from 'node:crypto' +import { homedir } from 'node:os' +import { join } from 'node:path' +import { createInterface } from 'node:readline/promises' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import type { RunAgentResumeItem } from '@tanstack/ai' +import { + compact, + modelPicker, + permissions, + title, + todos, + usage, +} from '@tanstack/ai-harness/plugins' +import { fileCommands, projectInstructions } from '@tanstack/ai-harness/plugins/node' +import { snapshots, workspaceTools } from '@tanstack/ai-harness/plugins/coding' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const root = process.cwd() +const smart = openaiText('gpt-6.1-sol') +const fast = openaiText('gpt-6-luna') + +const coder = defineHarness({ + name: 'acme/coder', + adapter: smart, + systemPrompts: ['You are a careful coding agent. Read before you edit.'], + plugins: () => [ + permissions({ root }), + workspaceTools({ root }), + snapshots({ root, dataDir: join(homedir(), '.acme-coder', 'snapshots') }), + todos(), + modelPicker({ choices: { smart, fast }, default: 'smart' }), + fileCommands({ dir: `${root}/.claude/commands` }), + compact({ adapter: fast }), + title({ adapter: fast }), + usage(), + // Keep this plugin last. See "Project instructions" below. + projectInstructions({ root }), + ], +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(coder, { threadId: 'main' }) +const terminal = createInterface({ + input: process.stdin, + output: process.stdout, +}) + +// Print the text of every turn, also of a turn that continues after an answer. +const stopPrinting = session.onEvent('TEXT_MESSAGE_CONTENT', ({ event }) => { + process.stdout.write(event.delta) +}) + +// A risky call stops the turn before it runs. Ask about each approval: +// once, always, or reject. Then send all the answers in one resolve. +async function answerApprovals() { + let waits = session.snapshot().pendingWaits + while (waits.length > 0) { + const resume: Array = [] + for (const wait of waits) { + const answer = await terminal.question(`\n${wait.message ?? wait.kind} (once/always/reject) `) + resume.push({ interruptId: wait.waitId, status: 'resolved', payload: answer }) + } + const inputId = randomUUID() + await session.resolve({ resume, inputId }) + await session.settled({ inputId }) + waits = session.snapshot().pendingWaits + } +} + +// Type `exit` to stop. +let line = await terminal.question('\n> ') +while (line !== 'exit') { + if (line.startsWith('/')) { + // `/undo`, `/model fast`, `/mode plan`, and the other plugin commands. + const [name = '', ...args] = line.slice(1).split(' ') + console.log(await session.runCommand({ name, input: args.join(' ') || undefined })) + } else { + try { + await session.send(line) + await answerApprovals() + } catch (error) { + // A failed turn rejects. Show the error and keep the loop. + console.error(error) + } + } + line = await terminal.question('\n> ') +} +stopPrinting() +terminal.close() +await host.close() +``` + +The `plugins/node` and `plugins/coding` entries run only in Node. The coding tools use the file system and the shell of the machine. A line that starts with `/` runs a command of a plugin. Any other line is a prompt. + +## 2. Run it + +1. Run `npx tsx coder.ts` in your project. +2. Ask for a change. The agent reads files freely, and asks before it writes a file or runs a command. +3. Answer `once` to allow the call, `always` to allow it from now on, or `reject` to refuse it. +4. If you do not like the result, run `/undo`. The files go back to how they were before the turn. + +## What each plugin adds + +| Plugin | Adds | +|---|---| +| `permissions({ root })` | Asks before risky tool calls. `/mode` switches between `default`, `plan` (read-only), `acceptEdits` (edits run without asking), and `bypass`. See [Ask before risky tool calls](./permissions). | +| `workspaceTools({ root })` | Tools that read, search, and change the files in `root`, run commands, and read web pages. A path outside `root` is refused, or asks first with `outside: 'ask'`. See [Give your agent coding tools](./coding-tools). | +| `snapshots({ root, dataDir })` | `/undo` puts the files back as they were before the last turn, and `/redo` brings them back. It needs `git`. See [Undo a turn](./snapshots). | +| `todos()` | A `todo_write` tool the model uses for multi-step work, and `/todos`. | +| `modelPicker({ choices })` | `/model ` switches the model at the next turn. | +| `fileCommands({ dir })` | Each `.md` file becomes a slash command. `$ARGUMENTS` is replaced by what you type after it. | +| `compact({ adapter })` | `/compact` replaces a long conversation with a summary. | +| `title({ adapter })` | Names the session from its first message. | +| `usage()` | `/usage` shows the tokens and the cost of the session, with a line for each model. See [Track usage and cost](./usage). | +| `projectInstructions({ root })` | Adds `AGENTS.md`, `CLAUDE.md`, and facts about the environment to the system prompt. | + +To compact on its own before the context limit, see [Compact a harness session](./compaction). + +## Project instructions + +Your repository has an `AGENTS.md` or a `CLAUDE.md` with its rules. `projectInstructions({ root })` gives them to the model: + +- It reads `AGENTS.md` and `CLAUDE.md` in each folder from the repository root (the folder with `.git`) down to `root`. Set other file names with `files`. +- `global` adds files that come first, for example `['~/.config/AGENTS.md']`. +- When `read_file` reads a file in a subfolder, the model also gets the instruction files of the folders on the way, once each. +- An environment block gives the date, the platform, the working folder, and the git repository and branch. `env: false` turns it off. +- When you change an instruction file, the next turn sends the new text as a note. + +Put `projectInstructions()` last in the plugin list. Then a changed file goes into the conversation as a [mid-conversation change](../advanced/mid-conversation-changes), and the prompt cache stays. When the prompt of another plugin comes after it, a change builds the system prompt again, and the prompt cache starts again. + +## Session titles and cost + +`title({ adapter })` and `usage()` write to the session index, one entry per thread: + +- `title` names the session from its first message. The turn does not wait for it. +- When `usage` knows the cost of the session, it writes the cost. A cost that the provider reports is used as it is. +- To price the calls of a provider that reports no cost, pass `usage({ model })`. [Track usage and cost](./usage#show-usage-and-price-the-calls) shows how. + +A [sidebar of sessions](./sessions) reads the title and the cost from the index. The `usage()` state also has `contextTokens`, the size of the lead model's context at its latest call, for a context meter. + +## Add more + +- `agents({ adapter })`: named agents with their own prompt, tools, and model. `/agent plan` switches to a read-only agent. See [Switch between agent profiles](./agents). +- `question()`: a `question` tool. The model asks the user to pick an option, and waits for the answer. +- `formatter({ root })`: formats each file after a write. See [Format each file after a write](./coding-tools-backends#format-each-file-after-a-write). + +## Add your own rules + +Tool plugins add permission rules to the `PermissionRules` extension point. Add your own for any tool: + +```ts group=harness-coding-agent +import { PermissionRules } from '@tanstack/ai-harness/plugins' +import { definePlugin } from '@tanstack/ai-harness' + +export const noDeploys = definePlugin({ + name: 'acme/no-deploys', + setup: () => ({ + contribute: [PermissionRules.item({ tool: 'deploy_*', decision: 'deny' })], + }), +}) +``` + +A trailing `*` matches every tool that starts with the text. The last matching rule wins. [Ask before risky tool calls](./permissions) explains the rules, the modes, and the saved answers. + +The workspace tools run on your machine with your permissions. For code that you do not trust, [run the tools in a sandbox](./coding-tools-backends#run-the-tools-in-a-sandbox). + +## Hand work to Claude Code or Codex + +Your agent can also give tasks to coding agents you already use. [Delegate to coding agents](./coding-agents) shows how. + +## What you have now + +- A terminal coding agent with file tools, approvals, modes, a todo list, and a model picker. +- `/undo` and `/redo` for the file changes of a turn. +- The rules of your repository in every turn. A change to them keeps the prompt cache. +- A title and a cost for each session. + +Next: let the user undo more than one turn with [Undo a turn](./snapshots). diff --git a/docs/harness/coding-agents.md b/docs/harness/coding-agents.md new file mode 100644 index 0000000000..e264d7f401 --- /dev/null +++ b/docs/harness/coding-agents.md @@ -0,0 +1,131 @@ +--- +title: Delegate to coding agents +id: harness-coding-agents +order: 11 +description: "Let a harness hand coding work to Claude Code, Codex, Grok Build, or any ACP agent. Each one works in a sandbox and keeps its own session." +keywords: + - tanstack ai + - harness + - claude code + - codex + - grok build + - sandbox + - subagents +--- + +Your lead agent plans the work, but you want Claude Code or Codex to do the edits, because they are good at code and you already use them. `codingAgents` gives the lead model one tool per coding agent. Each agent works in a sandbox and keeps its own session between calls. + +## 1. Add the plugin + + + +react: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +vue: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +solid: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +svelte: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +preact: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +angular: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +octane: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex +vanilla: @tanstack/ai-harness @tanstack/ai-sandbox @tanstack/ai-sandbox-local-process @tanstack/ai-claude-code @tanstack/ai-codex + + + +```ts group=harness-coding-agents +import { defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { claudeCodeText } from '@tanstack/ai-claude-code' +import { codexText } from '@tanstack/ai-codex' +import { openaiText } from '@tanstack/ai-openai' +import { defineSandbox, defineWorkspace, localSource } from '@tanstack/ai-sandbox' +import { codingAgents } from '@tanstack/ai-sandbox/harness' +import { localProcessSandbox } from '@tanstack/ai-sandbox-local-process' + +const repo = '/path/to/your/repo' + +export const lead = defineHarness({ + name: 'acme/lead', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + permissions(), + codingAgents({ + sandbox: defineSandbox({ + id: 'repo', + provider: localProcessSandbox({ dir: repo }), + workspace: defineWorkspace({ source: localSource(repo) }), + }), + agents: { + claude_code: { + adapter: claudeCodeText('claude-opus-4-8', { permissionMode: 'acceptEdits' }), + description: 'Larger changes, refactors, and reviews', + }, + codex: { + adapter: codexText('gpt-5.3-codex', { sandboxMode: 'workspace-write' }), + description: 'Quick fixes and tests', + }, + }, + }), + ], +}) +``` + +`localProcessSandbox({ dir: repo })` works in your repo folder itself, so every thread and every agent shares that folder. Leave out `dir` to give each sandbox its own copy of the repo. + +The lead model now has a `claude_code` tool and a `codex` tool. Each tool takes one `task`: the whole job, in words. The agent sees only that text and the files in its sandbox. + +## 2. Ask for work + +Open a session, and give the lead a task for both agents: + +```ts group=harness-coding-agents +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(lead, { threadId: 'date-parser' }) + +const turn = await session.send( + 'Have claude_code add a test for the date parser, then have codex fix what fails.', +) +console.log(turn.text) +``` + +The lead calls `claude_code` with a task, and gets back its answer. Then it calls `codex`. Each agent works in the same sandbox, one after the other. + +## Sessions + +Each agent keeps its own session per harness thread. The next task for `claude_code` resumes the same Claude Code session, so it still knows the files it read. The session ids live in the plugin state, so they also survive a restart of the host. + +To start over, run `/fresh claude_code`, or `/fresh` for every agent. + +## Workspaces + +`workspace` chooses where the agents work: + +| Value | Where each agent works | At the same time | +| --- | --- | --- | +| `'shared'` (default) | One sandbox per harness thread | One agent at a time | +| `'per-agent'` | A sandbox per agent | Yes | + +Use `'shared'` when the agents build on each other's changes. Use `'per-agent'` when they work on separate copies and you merge the results. + +Only the host that owns the log of the thread runs its agents. So with many servers, `'shared'` still runs one agent at a time. + +## Plan mode + +When the `permissions()` plugin is in [`plan` mode](./permissions#pick-a-mode), the agents start read-only: + +- Claude Code gets `permissionMode: 'plan'`. +- Codex gets `sandboxMode: 'read-only'`. +- Any other agent gets its `planModelOptions`, for example `{ permissionMode: 'default' }` for an ACP agent that asks before each edit. +- An agent with no `planModelOptions` does not run in `plan` mode. Its tool call gets an error. + +## Other agents and sandboxes + +- `adapter` takes any coding-agent adapter: `grokBuildText` from `@tanstack/ai-grok-build`, or `acpCompatibleText` from `@tanstack/ai-acp` for any ACP agent. +- `sandbox` takes any sandbox provider. [Sandbox providers](../sandbox/providers) lists them. +- `modelOptions` on an agent is added to every call, for example a fixed `permissionMode`. + +## What you have now + +- A lead model that hands tasks to Claude Code and Codex. +- One sandbox per thread, and one saved session per agent. diff --git a/docs/harness/coding-tools-backends.md b/docs/harness/coding-tools-backends.md new file mode 100644 index 0000000000..6d579fc546 --- /dev/null +++ b/docs/harness/coding-tools-backends.md @@ -0,0 +1,347 @@ +--- +title: Sandbox and extend the coding tools +id: harness-coding-tools-backends +order: 7 +description: "Run the coding tools in a sandbox or on your own backend, format each written file, and keep tool output small." +keywords: + - tanstack ai + - harness + - coding agent + - workspace tools + - sandbox + - formatter + - workspace hooks +--- + +Your agent must work on files that are not on this machine, or on code that you do not trust. Or you want code to run after each write, like a formatter. The [coding tools](./coding-tools) send every read, write, and command through a backend, and they run hooks after each read and write. This page shows how to change both. + +## Run the tools in a sandbox + +By default, the tools run on your machine, with the rights of your user. For code that you do not trust, run them in a sandbox. `sandboxWorkspaceBackend()` sends every read, write, and command to the sandbox. + +1. Install the sandbox packages: + + + +react: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +vue: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +solid: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +svelte: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +preact: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +angular: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +octane: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker +vanilla: @tanstack/ai-sandbox @tanstack/ai-sandbox-docker + + + +2. Start a sandbox, and give its backend to `workspaceTools()`: + +```ts group=harness-coding-tools-sandbox +import { defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { workspaceTools } from '@tanstack/ai-harness/plugins/coding' +import { openaiText } from '@tanstack/ai-openai' +import { sandboxWorkspaceBackend } from '@tanstack/ai-sandbox/harness' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' + +const sandbox = await dockerSandbox({ image: 'node:22' }).create({}) +// Without `dir`, the clone goes to the workspace root, /workspace. +await sandbox.git.clone({ url: 'https://github.com/acme/web-app.git' }) + +const root = '/workspace' + +export const sandboxed = defineHarness({ + name: 'acme/sandboxed-coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + permissions({ root }), + workspaceTools({ root, backend: sandboxWorkspaceBackend(sandbox) }), + ], +}) +``` + +To give the agent the project folder on your machine, [copy it into the container](#copy-your-project-into-the-container). + +3. When you are done, stop the sandbox with `await sandbox.destroy()`. + +Every provider works. Get the handle from `create()`, or from `resume()` for a sandbox that runs already. See [Providers](../sandbox/providers). To create a sandbox for each thread, and resume it on another server, see [Run the coding tools for many threads](./coding-workspaces). + +In a sandbox, some things are different: + +- Paths are POSIX paths in the sandbox, like `/workspace/src/a.ts`, also on a Windows host. +- The tools check where links lead with `lstat` and `readlink` in the sandbox. A sandbox without `lstat` cannot find links, so the tools refuse every path. +- `background: true` needs a sandbox with `capabilities.backgroundProcesses`. +- A sandbox with `capabilities.killableProcesses` off cannot stop a command after the time limit. The call returns, and the command continues in the sandbox. +- `list_files` and `grep` use `rg` in the sandbox. The `@vscode/ripgrep` binary is a host file, so the sandbox does not use it. +- When the run stops, the file calls of a tool in the sandbox get the abort signal too, so they stop with the run. + +### Copy your project into the container + +A clone has only the commits that you pushed. To give the agent your project as it is now, copy its folder into the container. In step 2, replace the `create({})` and `git.clone()` lines with this code: + +```ts group=harness-coding-tools-local-source +import { defineWorkspace, localSource } from '@tanstack/ai-sandbox' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' + +// Run this script in the root folder of your repository. +const sandbox = await dockerSandbox({ image: 'node:24' }).create({ + workspace: defineWorkspace({ source: localSource(process.cwd()) }), +}) +``` + +The tools now work on your project in `/workspace`. + +- The copy has every file that git does not ignore, with its uncommitted changes. Ignored files, like `node_modules`, stay on your machine, so install the dependencies in the container. +- The copy has the current branch with its history, so `git status` in the container shows your uncommitted changes. Staged changes show as unstaged. +- The folder must be the root of a git repository or a git worktree, and not a shallow clone. Otherwise, `create()` throws before it starts a container. +- Your machine and the image both need `git`. `node:24` has `git`, and `alpine` does not. +- `create()` copies the folder once. Your folder does not get the changes that the agent makes in the container. + +## Write your own backend + +A `WorkspaceBackend` is where the tools read and write files and run commands. Write one for another place, for example a remote machine. Every path that the tools give it is absolute. + +The backend must have: + +| Method | What it does | +| --- | --- | +| `readFile(path)` | Gives the bytes of a file. Throws when there is no file. | +| `writeFile(path, data)` | Creates or replaces a file. Creates the missing parent folders. | +| `stat(path)` | Gives `{ type, size, mtimeMs }`, or `undefined` when nothing is at `path`. It does not read the file. | +| `realpath(path)` | Gives the path with every link resolved, or `undefined` when nothing is at `path`. Throws when it cannot resolve a path. | +| `readdir(path)` | Gives the entries of a folder. A link has the type `'link'`. | +| `exec(command, options)` | Runs a shell command and waits. Gives `{ exitCode, stdout, stderr }`. | + +- `exec` resolves with the exit code of a command that fails. It does not throw. +- After `options.timeoutMs`, `exec` stops the command with every process that it started, and gives exit code 124. `options.signal` stops it too. +- `options.env` holds the variables to add. Give a command only these and the few variables that it needs to start. Do not give it the variables of your server. +- When `realpath` throws, the tool call fails. So a path with a link that the backend cannot resolve is never used. +- Each file method gets an optional last argument, `{ signal }`. It is the abort signal of the tool call. Give it to your file calls, so a stopped run stops them too. `readLines` gets `signal` in its options. + +The backend can also have: + +| Member | What it does | Without it | +| --- | --- | --- | +| `shell` | `'sh'` or `'cmd'`. Sets the path style and how arguments are quoted. | `sh` quotes, and the path style of this machine. | +| `remove(path)` | Removes one file. | `patch` cannot delete or move files. | +| `readLines(path, { offset, limit })` | Gives one page of a file: `{ head, size, lines, total }`. `head` is the first 8 KiB. | `read_file` reads the whole file for each page. | + +For background jobs and for threads that move to another server: + +| Member | What it does | Without it | +| --- | --- | --- | +| `spawn(command, options)` | Starts a command in the background. Gives `{ wait, kill, output, jobId? }`. `jobId` resolves with the id of the process, for `killJob`. | `background: true` gives an error. | +| `killJob(jobId)` | Stops a job that `spawn` started, also on an earlier process or machine. Resolves `false` when no such job runs. Rejects when it cannot check that the process is the job: then the log keeps the job. | A job that a crash left running is not stopped. | +| `exec` option `onStart(jobId)` | Gives the id of a foreground command when it starts, so a command that moves to the background can be stopped by a later host too. | A moved command has no id in the log. | +| `binding()` | Gives `{ kind: 'sandbox', provider, sandboxId }` for a sandbox that another machine can resume. | The workspace is a folder on this host. | +| `close({ reason })` | Releases the backend. `'closed'`: the session closed. `'moved'`: the session continues on another machine, so keep what it can resume. | Nothing to release. | + +`close` must never destroy a sandbox that the backend did not create. + +The fastest start is to wrap `hostBackend`. This backend logs each command before it runs: + +```ts group=harness-coding-tools-backends +import { hostBackend, workspaceTools } from '@tanstack/ai-harness/plugins/coding' +import type { WorkspaceBackend } from '@tanstack/ai-harness/plugins/coding' + +const root = process.cwd() + +const logged: WorkspaceBackend = { + ...hostBackend, + exec: (command, options) => { + console.log(`$ ${command}`) + return hostBackend.exec(command, options) + }, +} + +export const loggedTools = workspaceTools({ root, backend: logged }) +``` + +Only `hostBackend` itself uses the `@vscode/ripgrep` binary. A wrapped backend uses `rg` on the `PATH`. + +## Run code after a read or a write + +Another plugin can add code that runs after the tools read or write a file. Add a `WorkspaceHooks` item: + +```ts group=harness-coding-tools-backends +import { definePlugin } from '@tanstack/ai-harness' +import { WorkspaceHooks } from '@tanstack/ai-harness/plugins/coding' + +export const generatedFiles = definePlugin({ + name: 'acme/generated-files', + setup: () => ({ + contribute: [ + WorkspaceHooks.item({ + afterRead: async (path) => + path.endsWith('.gen.ts') + ? 'This file is generated. Change its source, not this file.' + : undefined, + afterWrite: async (path) => { + console.log(`changed ${path}`) + }, + }), + ], + }), +}) +``` + +- `afterRead` gets the absolute path. The text that it gives is added after the lines that `read_file` shows. +- `afterWrite` runs after `write_file`, `edit_file`, and `patch`. +- A hook runs while the tool holds the lock on that path. A hook that throws fails the tool call. + +Add the plugin next to the tools: `plugins: () => [workspaceTools({ root }), generatedFiles]`. + +## Refuse a write + +Add a `beforeWrite` hook to stop a write before it changes a file. Return `{ refuse }`, and the tool call fails with that text: + +```ts group=harness-coding-tools-refuse-write +import { definePlugin } from '@tanstack/ai-harness' +import { WorkspaceHooks } from '@tanstack/ai-harness/plugins/coding' + +export const protectGenerated = definePlugin({ + name: 'acme/protect-generated', + setup: () => ({ + contribute: [ + WorkspaceHooks.item({ + beforeWrite: async (path) => + path.endsWith('.gen.ts') + ? { refuse: `${path} is generated. Change its source.` } + : undefined, + }), + ], + }), +}) +``` + +- `beforeWrite` gets the absolute path. It runs before `write_file`, `edit_file`, and `patch` change the file. +- A `patch` asks for each file that it changes before its first write. One refusal fails the whole patch, and no file changes. +- The hook runs before the tool takes the lock on the path. + +## Format each file after a write + +The model writes code that your formatter changes later, and the diff gets noisy. `formatter()` formats each file that the tools write: + +```ts group=harness-coding-tools-backends +import { defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { formatter } from '@tanstack/ai-harness/plugins/coding' +import { openaiText } from '@tanstack/ai-openai' + +export const formatted = defineHarness({ + name: 'acme/formatted-coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + permissions({ root }), + workspaceTools({ root }), + formatter({ root }), + ], +}) +``` + +At the first write of a session, `formatter()` looks at the files in `root`. For each file, the first formatter that the project uses for that file type runs. + +For JavaScript, TypeScript, JSON, CSS, and other web files: + +| Formatter | Runs when `root` has | +| --- | --- | +| prettier | `.prettierrc*`, `prettier.config.*`, or a `prettier` field in `package.json` | +| biome | `biome.json` or `biome.jsonc` | +| oxfmt | `.oxfmtrc*`, or `oxfmt` in the `devDependencies` | + +These three run with `npx --no-install`, so the project must have them installed. + +For other languages: + +| Formatter | Files | Runs when `root` has | +| --- | --- | --- | +| ruff | `.py`, `.pyi` | `ruff.toml`, `.ruff.toml`, or `[tool.ruff]` in `pyproject.toml` | +| gofmt | `.go` | `go.mod` | +| rustfmt | `.rs` | `Cargo.toml` | + +Add your own formatter. Your formatters come before the built-in ones: + +```ts group=harness-coding-tools-backends +export const withToml = formatter({ + root, + formatters: [ + { + name: 'taplo', + extensions: ['.toml'], + command: (file) => `taplo fmt ${file}`, + }, + ], +}) +``` + +- `command` gets the absolute path, already quoted for the shell. +- `when(project)` limits a formatter to some projects. Without it, the formatter is used in every project. +- `builtins: false` turns the built-in formatters off. +- `timeoutMs` limits each run. The default is 20 seconds. +- In a sandbox or on your own backend, give `formatter()` the same `backend` as `workspaceTools()`. +- With [`workspaceTools({ workspace: { bind } })`](./coding-workspaces) before it, `formatter()` runs in the workspace of each thread. Leave out `root` and `backend`. + +A formatter that fails, stops at the time limit, or is not installed does not fail the tool call. The file stays as the tool wrote it, and the plugin sends `FormatFailed`. A formatter that changes the file sends `FormatterRewrote` with `{ path, formatter }`: + +```ts group=harness-coding-tools-backends +import { FormatFailed } from '@tanstack/ai-harness/plugins/coding' + +export const formatAlerts = definePlugin({ + name: 'acme/format-alerts', + setup: (ctx) => { + ctx.on(FormatFailed, (failure) => + console.warn(`${failure.path}: ${failure.message}`), + ) + }, +}) +``` + +Clients get the same events as a `harness.plugin.event` with the name `tanstack/formatter:failed` or `tanstack/formatter:rewrote`. + +### One approval covers the write and its formatter + +A formatter runs code from the project, like `prettier.config.js`. The model can change that code with an edit. So with `permissions()`, the rules decide the formatter as a part of the call that writes. The rule names the run `formatter:prettier`, `formatter:biome`, and so on. + +The rules decide before the call runs, and the formatter runs after the write. The call never stops in the middle to ask. + +- When the formatter asks, the call asks one time for the write and the formatter, also in `acceptEdits` mode. Answer that [approval](./permissions#answer-an-approval) with `resolve`. +- A `reject` stops the call. Nothing is written, and the formatter does not run. +- Answer `always` once, and the formatter runs without a question in this project. The answer is tied to a hash of the formatter config files, like `prettier.config.js` and `package.json`. When they change, the next write asks again. +- A rule that denies the formatter keeps the write. The tool result says that the formatter did not run. +- A rule can allow a formatter: `{ tool: 'formatter:prettier', decision: 'allow' }`. Without `permissions()`, the formatter runs without a question. + +[Formatters](./permissions#formatters) shows what each mode does. + +## Keep tool output small + +One tool result with a full log or a big JSON file fills the context of the model. `boundToolOutput()` cuts every tool result that is too long: + +```ts group=harness-coding-tools-backends +import { boundToolOutput } from '@tanstack/ai-harness/plugins/node' + +export const bounded = defineHarness({ + name: 'acme/bounded-coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + workspaceTools({ root }), + boundToolOutput({ dir: '.agent/tool-output' }), + ], +}) +``` + +- A result over 2000 lines or 50 KiB keeps its first lines and gets a note. Set other limits with `maxLines` and `maxBytes`. +- With `dir`, the full output goes to a file in `dir`, and the note gives the path. Files older than `retentionDays` (default 7) are removed. +- It cuts the result of every tool, MCP tools too, in the lead turn and in agent runs. +- An error stays an error. Images and other content parts do not change. + +`read_file` and `bash` keep their output inside the default limits, so `boundToolOutput()` does not cut them again. + +## What you have now + +- The coding tools in a sandbox, with one option. +- Your project in a Docker container, with its uncommitted changes. +- A backend of your own for another place. +- Code that runs after each read and each write. +- Formatted files after each write, and tool results that fit the context. + +Next: let the user undo the changes of a turn with [snapshots](./snapshots). diff --git a/docs/harness/coding-tools.md b/docs/harness/coding-tools.md new file mode 100644 index 0000000000..b48f292fa8 --- /dev/null +++ b/docs/harness/coding-tools.md @@ -0,0 +1,319 @@ +--- +title: Give your agent coding tools +id: harness-coding-tools +order: 6 +description: "Let a harness agent read, change, and search files, run commands, and read web pages, with a question before each risky call." +keywords: + - tanstack ai + - harness + - coding agent + - workspace tools + - permissions + - formatter +--- + +Your agent must read and change the files of a project, search them, and run commands like `pnpm test`. It must stay in that project, and it must ask before it changes a file. `workspaceTools()` gives the model these tools for one folder. `permissions()` asks the user before each risky call. + +## 1. Install + + + +react: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +vue: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +solid: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +svelte: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +preact: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +angular: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +octane: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai +vanilla: @tanstack/ai @tanstack/ai-harness @tanstack/ai-persistence @tanstack/ai-openai + + + +Two optional packages make the tools better: + + + +react: @vscode/ripgrep turndown +vue: @vscode/ripgrep turndown +solid: @vscode/ripgrep turndown +svelte: @vscode/ripgrep turndown +preact: @vscode/ripgrep turndown +angular: @vscode/ripgrep turndown +octane: @vscode/ripgrep turndown +vanilla: @vscode/ripgrep turndown + + + +- `@vscode/ripgrep`: a fast `rg` binary for `list_files` and `grep`. Without it, the tools use `rg` on the `PATH`. Without `rg`, they use `git ls-files`, then a walk through the folders. +- `turndown`: `webfetch` gives HTML pages as Markdown. Without it, HTML pages come back as plain text. + +## 2. Add the tools + +```ts group=harness-coding-tools +import { randomUUID } from 'node:crypto' +import { createInterface } from 'node:readline/promises' +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import type { RunAgentResumeItem } from '@tanstack/ai' +import { permissions } from '@tanstack/ai-harness/plugins' +import { workspaceTools } from '@tanstack/ai-harness/plugins/coding' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const root = process.cwd() + +const coder = defineHarness({ + name: 'acme/coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [permissions({ root }), workspaceTools({ root })], +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(coder, { threadId: 'main' }) +const terminal = createInterface({ + input: process.stdin, + output: process.stdout, +}) + +// Print the text of every turn, also of a turn that continues after an answer. +const stopPrinting = session.onEvent('TEXT_MESSAGE_CONTENT', ({ event }) => { + process.stdout.write(event.delta) +}) + +// A risky call stops the turn before it runs. Ask about each approval of +// the turn, then send all the answers in one resolve. +async function answerApprovals() { + let waits = session.snapshot().pendingWaits + while (waits.length > 0) { + const resume: Array = [] + for (const wait of waits) { + const answer = await terminal.question(`\n${wait.message ?? wait.kind} (once/always/reject) `) + resume.push({ interruptId: wait.waitId, status: 'resolved', payload: answer }) + } + const inputId = randomUUID() + await session.resolve({ resume, inputId }) + await session.settled({ inputId }) + waits = session.snapshot().pendingWaits + } +} + +// Send each line as a prompt. Type `exit` to stop. +let line = await terminal.question('\n> ') +while (line !== 'exit') { + try { + await session.send(line) + await answerApprovals() + } catch (error) { + // A failed turn rejects. Show the error and keep the loop. + console.error(error) + } + line = await terminal.question('\n> ') +} +stopPrinting() +terminal.close() +await host.close() +``` + +`@tanstack/ai-harness/plugins/coding` runs only in Node. The tools use the file system and the shell of the machine. For code that you do not trust, [run the tools in a sandbox](./coding-tools-backends#run-the-tools-in-a-sandbox). + +## 3. Run it + +1. Run `npx tsx coder.ts` in your project. +2. Ask: `find the failing test and fix it`. +3. When the agent wants to change a file or run a command, the turn stops, and the script asks in the terminal. Answer `once`, `always`, or `reject`. + +The agent reads and searches without a question. For a full coding agent with undo, a todo list, and a model picker, see [Build a coding agent](./coding-agent). + +## The tools + +### Read and search + +These tools run without a question. + +| Tool | What it does | +| --- | --- | +| `read_file` | Reads a text file as numbered lines, at most 2000 lines or 50 KiB a page. The model reads more with `offset` and `limit`. | +| `list_files` | Lists the files in a folder. An optional glob, like `src/**/*.ts`, filters them. | +| `grep` | Searches the file contents with a regular expression. Gives `file:line: text`. | + +- `read_file` gives images (PNG, JPEG, GIF, WebP) and PDFs up to 256 KB to the model as media. For a bigger file, the model gets an error that tells it to make a smaller copy. +- `read_file` refuses a binary file with a short note. For a missing file, it names up to 3 files with a close name. +- `list_files` and `grep` skip the files that `.gitignore` names, the `.git` and `node_modules` folders, and links. +- `list_files` shows at most 1000 files. `grep` shows at most 200 matches. + +### Change files + +These tools ask first. + +| Tool | What it does | +| --- | --- | +| `write_file` | Creates or replaces a file. | +| `edit_file` | Replaces text in a file. The old text must be in the file once, unless the model sets `replaceAll`. | +| `patch` | Applies a patch that adds, deletes, updates, or moves files. | + +A model gets `edit_file` or `patch`, not both. Every model gets `write_file`. The `editStyle` option picks the edit tool: + +- `'auto'` (default): GPT models (ids that start with `gpt-`, or `o` and a digit, like `o3`) get `patch`. Other models get `edit_file`. +- `'edit'` or `'patch'`: every model gets that tool. + +`edit_file` keeps the line endings and the byte order mark of the file. It also finds text that is different only in Unicode form or in the spaces at the end of a line. + +When a file changed after the model read it, for example by another agent, the three tools fail. The model reads the file again, then edits it. + +To let the user undo the changes of a turn, add [snapshots](./snapshots). + +### Commands and the web + +| Tool | What it does | Asks first | +| --- | --- | --- | +| `bash` | Runs a shell command in the working folder. | Yes | +| `webfetch` | Reads a web page by its URL. | Yes | +| `websearch` | Searches the web. It is there only with a search provider. | No | + +For `bash`: + +- The model gets the exit code and the last 1000 lines (20 KiB) of the output. +- A command stops after 2 minutes, with exit code 124. Every process that it started stops too. Set another limit with `bashTimeoutMs`. The model can ask for up to 10 minutes. +- With `spillDir`, long output goes to a file in that folder, and the model gets the path. `spillDir` is relative to `root`, or absolute. The file has `[redacted]` in place of each secret of the session, the same as the tool result. See [Keep tokens out of the log and the model](./auth#keep-tokens-out-of-the-log-and-the-model). +- A command gets only a few variables of your process, like `PATH` and `HOME`. Your provider keys are not in its env. On the host this is not isolation: a command can still read the files of your server. See [Keep server secrets out of commands](./coding-workspaces#2-keep-server-secrets-out-of-commands). + +With `background: true`, the call returns at once with a job id. When the job ends, the model gets a note with the output, and a new turn starts. That turn runs as the user whose run started the job. + +- The time limit does not apply to a background job. +- At most 8 background jobs run at a time in a session. Set another limit with `maxJobs`. Each job keeps the newest 1 MiB of its output. +- When the session closes, the background jobs stop. +- The log keeps each background job until it ends. When a crash stops a job, the next host adds the note "Background job bash-1 stopped when the host restarted." + +On Windows, the default backend runs commands in `cmd.exe`. A program name like `rg` or `gradlew` comes from the `PATH` only, not from the working folder. So a cloned repo cannot put its own `rg.cmd` in place of `rg`. To run a script in the working folder, give its path: `.\gradlew build`. + +For `webfetch`: + +- It reads `http` and `https` URLs only, and follows at most 5 redirects. +- It refuses `localhost`, private network addresses, and cloud metadata addresses. It checks the DNS answer of each connection. +- HTML comes back as Markdown, or as text without `turndown`. The model can ask for `format: 'html'`. Text and JSON come back as they are. +- The model gets the first 256 KB of a page, with a note when there is more. It reads at most 5 MB. +- It stops after 30 seconds, and the model can ask for up to 120 seconds. + +Change the web tools with the `web` option: + +- `web: false`: no web tools. +- `web: { allowPrivateHosts: true }`: `webfetch` can read `localhost` and your private network. Set it only for a model that is allowed to reach your network. +- `web: { search }`: adds `websearch`. + +There is no built-in search engine. Give `websearch` a `SearchProvider`: + +```ts group=harness-coding-tools +import type { SearchProvider } from '@tanstack/ai-harness/plugins/coding' +import { searchTheWeb } from './search' + +const search: SearchProvider = { + search: (query, { limit, signal }) => searchTheWeb(query, { limit, signal }), +} + +export const withSearch = workspaceTools({ root, web: { search } }) +``` + +The provider gives at most `limit` results. Each result is `{ title, url, snippet? }`. The model reads a result with `webfetch`. + +## Permissions and approvals + +Each tool tells `permissions()` what kind of call it makes: + +| Tools | `default` mode | `plan` mode | +| --- | --- | --- | +| `read_file`, `list_files`, `grep`, `websearch` | Runs | Runs | +| `write_file`, `edit_file`, `patch` | Asks | Denied | +| `bash` | Asks | Denied | +| `webfetch` | Asks | Denied | + +- `acceptEdits` mode runs the edits without a question. `bash` still asks. +- `bypass` mode runs every call. +- `read_file` asks for a `.env` file. +- `grep` skips each file that `read_file` asks about or denies, also in `plan` mode. A note at the end gives the number of skipped files. +- `list_files` still shows the names of these files. A name is not the contents. + +Let `webfetch` run without a question with one rule: + +```ts group=harness-coding-tools +export const trustTheWeb = permissions({ + root, + rules: [{ tool: 'webfetch', decision: 'allow' }], +}) +``` + +In `plan` mode, a call that asks is denied. This rule makes `webfetch` an allow, so `webfetch` runs in `plan` mode too. + +Each call also tells `permissions()` the paths or the commands that it touches. So a rule with a `resource` can allow one folder or one command: + +```ts group=harness-coding-tools +export const trustTests = permissions({ + root, + rules: [ + { tool: 'edit_file', resource: 'src/**', decision: 'allow' }, + { tool: 'bash', resource: 'pnpm test*', decision: 'allow' }, + ], +}) +``` + +- Paths match from `root`. Give `permissions()` and `workspaceTools()` the same `root`. +- A rule also matches the real path of a link, and the stricter decision wins. So a link `notes` to `.env` asks, and a `deny` for `secrets/**` also stops a link to `secrets`. +- The rules decide every path before the call runs, so a tool never asks while it runs. In a [code mode program](./code-mode#run-the-coding-tools-in-a-program), a call that asks makes the program wait, and the turn does not stop. +- `bash` splits a command at `&&`, `||`, `;`, `|`, and new lines. Every part must be allowed. +- A command with `$(`, a backtick, or a heredoc never gets an allow. + +[Permissions](./permissions) explains the rules, the modes, and the saved answers. + +## Outside paths and the working folder + +### A path outside `root` + +The tools refuse a path outside `root`. To let the user approve it, set `outside: 'ask'`: + +```ts group=harness-coding-tools +export const askOutside = workspaceTools({ root, outside: 'ask' }) +``` + +`permissions()` decides the outside path before the call runs. The path is a part of the one [approval](./permissions#answer-an-approval) of the call, so the tool never stops in the middle of a call to ask. For a direct call, the turn stops, and you answer with `resolve`. In a [code mode program](./code-mode#answer-the-question), the program waits, and you answer with `session.answer`. + +- `once` allows this call. `always` also saves an allow rule for the path. +- `reject` stops the call. The call does not run, so nothing is read or written. +- A link in `root` that leads out of `root` counts as outside. The rules decide its real path. +- `bypass` mode allows every path without a question. Without `permissions()`, nobody can approve the path, so the tool refuses it. +- With `permissions()`, `grep` skips the files outside `root`, because `read_file` asks for them. A `read_file` rule for that folder, for example `{ tool: 'read_file', resource: '/shared/**', decision: 'allow' }`, lets `grep` show them. + +With a [workspace for each thread](./coding-workspaces), the root is not known before the call, so no approval can cover an outside path. Then only a rule that allows the path, or `bypass` mode, lets the call use it. + +### A working folder for each thread + +Each thread can work in its own folder in `root`. Set the `cwd` thread setting: + +```ts group=harness-coding-tools-cwd +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { workspaceTools } from '@tanstack/ai-harness/plugins/coding' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const repos = defineHarness({ + name: 'acme/repos', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [workspaceTools({ root: '/srv/repos' })], +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(repos, { threadId: 'fix-login' }) +await session.updateThreadSettings({ settings: { cwd: 'web-app' } }) +``` + +- Relative paths start in `/srv/repos/web-app`, and `bash` runs there. +- The system prompt tells the model the working folder. +- Permission rules still match paths from `root`, for example `web-app/src/**`. +- `cwd` is relative to `root`. The tools refuse a `cwd` outside `root`, also with `outside: 'ask'`. + +[Store settings per thread](./thread-settings) shows how a client sets `cwd`. + +## What you have now + +- An agent that reads, searches, and changes the files of one folder, and runs commands in it. +- A question before each edit, command, and web fetch. +- A working folder for each thread, and a question before a path outside `root`. + +Next: run the same tools in a sandbox, or format each file after a write. See [Sandbox and extend the coding tools](./coding-tools-backends). To give each thread its own folder on a server, see [Run the coding tools for many threads](./coding-workspaces). diff --git a/docs/harness/coding-workspaces.md b/docs/harness/coding-workspaces.md new file mode 100644 index 0000000000..1d53e9fa5e --- /dev/null +++ b/docs/harness/coding-workspaces.md @@ -0,0 +1,279 @@ +--- +title: Run the coding tools for many threads +id: harness-coding-workspaces +order: 7 +description: "Give each thread of a coding agent its own folder or sandbox, keep server secrets out of commands, let agents share files safely, watch background jobs, and move a thread to another server." +keywords: + - tanstack ai + - harness + - coding agent + - workspace tools + - sandbox + - multi-tenant + - background jobs +--- + +Your server runs coding agents for many users. Each thread must work in its own folder or sandbox. A command must not see the API keys of your server. Two agents in one folder must not lose each other's edits. And a thread must continue when another server takes it. + +`workspace: { bind }` gives each thread its own workspace. The session log keeps where that workspace is, so the next server finds it again. + +## 1. Give each thread its own workspace + +Pass `workspace` to `workspaceTools()` in place of `root`. `bind` gets the run, and gives a backend with its `root`: + +```ts group=harness-coding-workspaces +import { mkdir } from 'node:fs/promises' +import { defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { + hostBackend, + workspaceRoot, + workspaceTools, +} from '@tanstack/ai-harness/plugins/coding' +import { openaiText } from '@tanstack/ai-openai' + +export const coder = defineHarness({ + name: 'acme/coder', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [ + permissions(), + workspaceTools({ + workspace: { + bind: async ({ run }) => { + const root = workspaceRoot('/srv/workspaces', run.tenantId ?? 'default', run.threadId) + await mkdir(root, { recursive: true }) + return { ...hostBackend, root } + }, + }, + }), + ], +}) +``` + +- The tools call `bind` once for each thread, before the first turn or at the first tool call. If `bind` throws, the next turn calls it again. +- A client sends the thread id. Do not put it in a path with `join` or a template string: an id like `../other-tenant/t-1` leads into another folder. `workspaceRoot` encodes each part, so each id gets its own folder in `/srv/workspaces`, and it throws for a path that leaves it. +- The `root` that `bind` gives is the boundary of the file tools. A path outside it is refused. With `outside: 'ask'`, a rule that allows the path, or `bypass` mode, lets a call use it. No approval can cover it, because the root is not known before the call. The `cwd` thread setting must stay inside it. +- The tools resolve every link with `realpath` before they read or write, while they hold the lock on the file. So a link in one thread cannot lead into the folder of another thread. +- The session log keeps a `harness.workspace.bound` record with the place of the workspace. [Move a thread to another server](#5-move-a-thread-to-another-server) uses it. +- Put `workspaceTools()` before [`snapshots()`](./snapshots) and [`formatter()`](./coding-tools-backends#format-each-file-after-a-write). Then they work in the workspace of each thread, and their `root` is optional. + +A folder on the server is not isolation: `bash` runs as the user of your server (see [Keep server secrets out of commands](#2-keep-server-secrets-out-of-commands)). When many users share the server, give each thread a sandbox. Create it in `bind`, and give the backend `owned: true`. Then the thread owns the sandbox: + +```ts group=harness-coding-workspaces +import { sandboxWorkspaceBackend } from '@tanstack/ai-sandbox/harness' +import { dockerSandbox } from '@tanstack/ai-sandbox-docker' + +const docker = dockerSandbox({ image: 'node:22' }) + +export const sandboxed = workspaceTools({ + workspace: { + bind: async () => ({ + ...sandboxWorkspaceBackend(await docker.create({}), { owned: true }), + root: '/workspace', + }), + }, +}) +``` + +- A session close (an idle close, a deploy, or a move to another server) closes the backend with `'moved'`. The sandbox stays, so the next server can [reattach](#a-sandbox) it. +- [`host.sessions.delete`](./lifecycle#delete-a-thread-for-good) ends the thread. The host reattaches the sandbox and closes it with `'closed'`, so it is destroyed. The host that deletes must know the harness: it opened the harness, or it has the harness in its [`harnesses`](./lifecycle#delete-a-thread-for-good) option. Else it warns, and the sandbox stays. + +## 2. Keep server secrets out of commands + +A command gets only a few variables of the server process: + +- `PATH`, `HOME`, `LANG`, `LC_ALL`, `TMPDIR`, `TEMP`, and `TMP`. +- On Windows also `SystemRoot`, `ComSpec`, `PATHEXT`, `WINDIR`, and `USERPROFILE`. +- `AGENT=1`, so a tool can see that an agent runs it. + +Provider keys, `DATABASE_URL`, and the other variables of the server are not in the env of a command. The same rule applies to `bash`, `rg`, `git`, and the formatters. Git also runs without the hooks and the `fsmonitor` of the project. + +With `hostBackend`, and with the local-process sandbox, this short env and the `root` are defense in depth. They are not isolation: + +- A command runs as the user of your server. `bash` can read every file that this user can read, for example the `.env` file of the server, or the folder of another thread. +- `bash` can also read the env of each process of that user, for example `/proc//environ` of the server. + +For users that you do not trust, use [a sandbox for each thread](#1-give-each-thread-its-own-workspace). + +To give commands a variable, add it to the `env` of the backend calls: + +```ts group=harness-coding-workspaces +import type { WorkspaceBackend } from '@tanstack/ai-harness/plugins/coding' + +const extra = { NPM_TOKEN: 'npm_read_only_token' } +const { spawn } = hostBackend + +const withToken: WorkspaceBackend = { + ...hostBackend, + exec: (command, options) => + hostBackend.exec(command, { ...options, env: { ...options?.env, ...extra } }), + spawn: + spawn && + ((command, options) => + spawn(command, { ...options, env: { ...options?.env, ...extra } })), +} + +export const withTokenTools = workspaceTools({ root: '/srv/repo', backend: withToken }) +``` + +`exec` runs the foreground commands. `spawn` runs the background commands. + +## 3. Let agents share a folder + +Two agents, or two sessions, can work in the same folder. No edit is lost: + +- A write holds a lock on the file. The lock is shared by every session on this server, and it is keyed by the sandbox (or this server) and the real path of the file. Two roots that lead to one folder share the lock. Two sandboxes with the same root do not. +- `write_file`, `edit_file`, and `patch` fail when the file changed after the tool read it in the same agent run. A subagent and its lead each check their own read. The model gets `src/app.ts changed since it was read. Read it again before you edit it.`, reads the file again, and tries again. +- The check compares the size and the time of the last change. A sandbox gives no change time, so there only a change of size is seen. + +## 4. Watch background jobs + +With `background: true`, `bash` returns at once with a job id like `bash-1`. Two limits keep jobs small: + +- At most 8 background jobs run at a time in a session. One more gives the model a tool error. Set another limit with `maxJobs`. A running command that `session.moveToolCallToBackground()` moves counts too, and at the limit it does not move. +- Each job keeps the newest 1 MiB of its output. + +When a job ends, the model gets a note, and the wake turn runs as the user whose run started the job. + +To show the jobs that run now, for example in a status bar, call `session.jobs()`. A [view](./lifecycle#read-a-thread-without-a-session) of `host.observe` has `jobs()` too: + +```ts group=harness-coding-workspaces +import type { HarnessSession } from '@tanstack/ai-harness' + +export function listJobs(session: HarnessSession) { + for (const job of session.jobs()) { + console.log(`${job.jobId}: ${job.label ?? 'job'}, started by ${job.user?.id ?? 'the agent'}`) + } +} +``` + +Each entry has: + +- `jobId` and `threadId`: the job, and the thread of the workspace that runs it. +- `label`: the command, or the name of the tool that `session.moveToolCallToBackground()` moved. +- `user` and `startedAt`: who moved the call to the background, and when the job started. +- `running`: `true` when this host runs the job. A job of another host, or one that a crash stopped, has `false`. +- `backendId`: the id of the process in the workspace backend, when it has one. + +`session.jobs()` reads the log, so it also lists the jobs of other hosts. After a restart, it lists a job until the host notes that the job stopped. + +Each job sends events. Listen for them in a plugin: + +```ts group=harness-coding-workspaces +import { definePlugin } from '@tanstack/ai-harness' +import { JobEnded, JobOutput, JobStarted } from '@tanstack/ai-harness/plugins/coding' + +export const jobLog = definePlugin({ + name: 'acme/job-log', + setup: (ctx) => { + ctx.on(JobStarted, (job) => console.log(`${job.jobId} started: ${job.command}`)) + ctx.on(JobOutput, (job) => process.stdout.write(job.output)) + ctx.on(JobEnded, (job) => console.log(`${job.jobId} ended: ${job.exitCode}`)) + }, +}) +``` + +A client gets each event as a `CUSTOM` event with the name `harness.plugin.event`. Its value is `{ plugin, name, value }`: + +```ts group=harness-coding-workspaces-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { coder } from './harness' + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'fix-login', +}) + +for await (const { event } of client.events()) { + if (event.type !== 'CUSTOM' || event.name !== 'harness.plugin.event') continue + const { value } = event + if (typeof value !== 'object' || value === null) continue + if (!('name' in value) || value.name !== 'tanstack/workspace-tools:job-output') continue + if ('value' in value) console.log(value.value) +} +``` + +The coding plugins send these events: + +| Name | When | +| --- | --- | +| `tanstack/workspace-tools:job-started` | A background job started. | +| `tanstack/workspace-tools:job-output` | New output of a job. At most one event each 250 ms, with at most the newest 64 KiB. | +| `tanstack/workspace-tools:job-ended` | A job ended, after its last output event. | +| `tanstack/workspace-tools:stranded` | The workspace of the thread is gone. See below. | +| `tanstack/workspace-tools:file-changed` | A file changed. The value has a unified diff of the change. See [Show live diffs](./custom-ui#show-live-diffs). | +| `tanstack/snapshots:failed` | A snapshot failed. The turn goes on. | + +The formatter also sends `tanstack/formatter:rewrote` and `tanstack/formatter:failed`. See [Format each file after a write](./coding-tools-backends#format-each-file-after-a-write). + +## 5. Move a thread to another server + +One server owns a thread at a time. When it stops, another server takes the thread (see [One host drives a thread](./durable-sessions#one-host-drives-a-thread)). What happens to the workspace depends on where it is. + +### A sandbox + +The log keeps the provider and the id of the sandbox. The next server calls `reattach` in place of `bind`. Resume the sandbox there: + +```ts group=harness-coding-workspaces +export const movable = workspaceTools({ + workspace: { + bind: async () => ({ + ...sandboxWorkspaceBackend(await docker.create({}), { owned: true }), + root: '/workspace', + }), + reattach: async ({ binding }) => { + if (binding.kind === 'host') return 'pinned' + if (binding.provider !== 'docker') return 'gone' + const handle = await docker.resume({ id: binding.sandboxId }) + if (!handle) return 'gone' + return { ...sandboxWorkspaceBackend(handle, { owned: true }), root: binding.root } + }, + }, +}) +``` + +- Give the workspace back to continue in the same files. +- Give `'gone'` when the sandbox is gone. Then each tool call of the thread fails with `The workspace of this thread, /workspace in the docker sandbox , is gone, so the tools cannot work on it.`, and the tools send `tanstack/workspace-tools:stranded`. The session also sends a `harness.stranded` event, for clients that do not know the plugin. See the [event catalog](./protocol#event-catalog). +- Without `reattach`, a sandbox counts as `'gone'`. + +### A folder on one server + +A folder on the disk of one server cannot move. Give each server a stable id: + +```ts group=harness-coding-workspaces +import { createHarnessHost } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' + +const host = createHarnessHost({ + persistence: memoryPersistence(), + ownership: { hostId: process.env.POD_NAME ?? 'web-1' }, +}) +``` + +- With `hostId`, a thread that uses a local folder is pinned to this server. Workers on other servers do not take it, also while the thread is idle. +- Another server that opens the thread refuses its tool calls with `The workspace of this thread is on the host web-1. Only that host can continue the thread.` +- Keep the id the same after a restart. On Kubernetes, use the pod name of a StatefulSet. Do not use `HOSTNAME` of a Deployment pod: it changes at each restart, and the threads of the old pod stay pinned to it. +- Without `hostId`, threads are not pinned, and every server calls `bind` again. Use this when every server sees the same folder, for example on a shared disk. + +The server sends a heartbeat to the log store each `ownership.renewEveryMs`. When no heartbeat came for `ownership.expiresAfterMs` (default 30 seconds), its pins do not count. Then the next server that gets work for the thread calls `reattach` with the old binding. Give a new workspace there, for example a fresh clone, to move the thread. Keep a restart shorter than `ttlMs`, or raise `ttlMs`. + +To move the threads of a server at once, unpin them with `host.unpin`. Its ownership must have expired first: + +```ts group=harness-coding-workspaces +await host.unpin({ harness: coder, threadId: 'fix-login' }) +``` + +### After a move + +- Background jobs do not move. At the next tool call of the thread, the new server stops each job that the old server left running. The model gets the note `Background job bash-1 was stopped: it ran on a host before the session moved to this one.` +- A job that the backend cannot check stays in the log, and the model gets `Background job bash-1 may still run: it ran on a host before this one, and this host cannot stop it.` For example, `hostBackend` needs `ps` to check a job. +- `/undo`, `/redo`, and a revert work on the new server when it reaches the same snapshot repository. See [Undo a turn](./snapshots#on-more-than-one-server). +- [Coding agents](./coding-agents) run only on the server that owns the log. So two servers never run the agents of one shared workspace at the same time. + +## What you have now + +- A folder or a sandbox for each thread, with its root as the boundary of every tool. +- Commands that get only a short env, and a sandbox for users that you do not trust. +- Agents that share a folder and never lose an edit. +- Events for every background job, and a limit on how many run. +- Threads that move to another server with their sandbox, or stay on the server that has their folder. diff --git a/docs/harness/compaction.md b/docs/harness/compaction.md new file mode 100644 index 0000000000..537002c2ff --- /dev/null +++ b/docs/harness/compaction.md @@ -0,0 +1,133 @@ +--- +title: Compact a harness session +id: harness-compaction +order: 4 +description: "Keep a long harness session under the context limit. The compaction goes into the session log, so a session that opens again sees the same context." +keywords: + - tanstack ai + - harness + - compaction + - context window + - projectCompaction + - compactNext + - session log +--- + +A harness session can run for days. Its transcript grows until a model call passes the context limit. [Compaction](../advanced/compaction) shrinks what the model sees. In a durable harness, the result must also survive a restart. A session that opens again must see the same context as before. + +Set `durable: true`. Compaction then writes a `tanstack.compaction` record into the session log. The host folds that record into the messages with `projectCompaction`, live and when the session opens again. + +## Add compaction to a harness + +1. Create the compaction middleware with `durable: true`. +2. Add it to `middleware` in `defineHarness`. +3. Give the harness `projectCompaction` in its `durability.addRecordsToModelContext` option. + +```ts group=harness-compaction +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' +import { + conversationSummarizer, + projectCompaction, + summarizeOldest, + withCompaction, +} from '@tanstack/ai-compaction' + +const compaction = withCompaction({ + maxTokens: 180_000, + contextWindow: 200_000, + countTokens: 'usage', + durable: true, + strategy: summarizeOldest({ + cut: 'turn', + keepRecentTokens: 8_000, + summarize: conversationSummarizer({ adapter: openaiText('gpt-6.1-sol') }), + }), +}) + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), + middleware: [compaction], + durability: { addRecordsToModelContext: { version: 'v1', record: projectCompaction } }, +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +``` + +- Compaction runs before a model call when the count passes `maxTokens`. If the usage of the last model call of a turn passed `maxTokens` or `contextWindow`, compaction also runs right after that call. This check needs `countTokens: 'usage'`. +- The log only grows, so the original messages stay in the store. The fold that the model gets starts with the summary. +- Readers still get every message. `session.transcript()`, `GET transcript`, and the chat hydration show the full history, not the summary. +- A thread can change its model. To give each model its own limit, pass a function, like `maxTokens: ({ model }) => ...`. See [A limit for each model](../advanced/compaction-advanced#a-limit-for-each-model). + +## Prepare the summary in the background + +A summary call takes time. When compaction runs at `maxTokens`, the user waits for that call before the model answers. Set `background` to prepare the summary earlier, while the session goes on: + +```ts group=harness-compaction +const earlyCompaction = withCompaction({ + maxTokens: 180_000, + contextWindow: 200_000, + countTokens: 'usage', + durable: true, + background: { atTokens: 150_000 }, + strategy: summarizeOldest({ + cut: 'turn', + keepRecentTokens: 8_000, + summarize: conversationSummarizer({ adapter: openaiText('gpt-6.1-sol') }), + }), +}) +``` + +- `atTokens` must be below `maxTokens`. If it is not, `withCompaction` throws. +- When the count at a model call passes `atTokens`, the summary starts beside the turn. The turn does not wait for it. +- The summary applies at the first model call of the next run, never in the middle of a run. It goes into the log as a `tanstack.compaction` record with `reason: 'background'`. +- A model call over `maxTokens` while the summary runs waits for it and applies it. It compacts inline only when the messages are still over `maxTokens`. +- If you cancel the run during that wait, the wait stops. The summary continues and applies at the next run. + +Sometimes a summary does not apply: + +- A newer compaction cut past the message that the summary keeps. The summary is dropped. `onCompact` and `compaction:ended` report it with `stale: true`. +- The next run calls another model than the one that started the summary. The summary is dropped and reported with `stale: true`. +- The summary call failed. Nothing applies. `onCompact` and `compaction:ended` report it with `error`. The next model call past `atTokens` starts a new summary. + +## Fold your own records too + +If your harness already folds records of its own, call `projectCompaction` first: + +```ts group=harness-compaction +import type { ProjectOptions } from '@tanstack/ai-harness' + +const project: ProjectOptions = { + version: 'v2', + record: (input) => { + const compacted = projectCompaction(input) + if (compacted) return compacted + const { record, messages } = input + if (record.type === 'app.signal' && typeof record.text === 'string') { + return [...messages, { role: 'user', content: record.text }] + } + return undefined + }, +} +``` + +Change `version` when you change the function. A log snapshot with another version is not used, so the host folds the log again from the first record. + +## Retry after a context overflow + +A model call can still fail because the context is too long. `withCompaction` handles it for you. When a call fails with `CONTEXT_OVERFLOW_CODE` (`'context_overflow'`), the middleware compacts and calls the model again, inside the `chat()` call of the turn. That compaction also writes its record to the log. + +- It retries once per model call. This also works with `auto: false`. +- `turn.onModelError` sees the overflow only when the compacted call is still too long. Then `error.code` is `'context_overflow'`, and the turn fails unless your hook returns `'retry'`. +- To compact the next call yourself, for example before you switch the thread to a model with a smaller context window, call `compaction.compactNext(session.threadId)`. + +For a strategy that always finds something to cut on the retry, see [Give the retry something to cut](../advanced/compaction#give-the-retry-something-to-cut). + +## What you have now + +- A session that compacts before it passes the context limit, counted with the real usage of each call. +- The same context after a restart, because the compaction is in the log. +- With `background`, a summary that is ready before the session reaches `maxTokens`. +- One retry after an overflow, with a compaction first, and no code of your own. diff --git a/docs/harness/computer-use.md b/docs/harness/computer-use.md new file mode 100644 index 0000000000..60f96142f1 --- /dev/null +++ b/docs/harness/computer-use.md @@ -0,0 +1,125 @@ +--- +title: Give your agent a computer +id: harness-computer-use +order: 8 +description: "Let a harness agent use a web page. Each thread gets its own browser page. Screenshots go to the media store of the session, and OpenAI safety checks ask the user." +keywords: + - tanstack ai + - harness + - computer use + - computerUse + - browser agent + - playwright + - screenshots +--- + +Your agent must do a task on a web page that has no API, for example fill in a form or find a price. `computerUse()` gives the agent a screen, a mouse, and a keyboard. Each thread gets its own browser page. Each screenshot goes to the media store of the session, so the log stays small. + +To use computer use in a `chat()` route with no harness, read [Computer Use](../computer-use/computer-use). + +## 1. Add the plugin + + + +react: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +vue: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +solid: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +svelte: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +preact: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +angular: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +octane: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod +vanilla: @tanstack/ai-harness @tanstack/ai-computer-use @tanstack/ai-computer-playwright playwright-core zod + + + +Then download Chromium one time: + +```bash +npx playwright-core install chromium +``` + +```ts group=harness-computer-use +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { permissions } from '@tanstack/ai-harness/plugins' +import { computerUse } from '@tanstack/ai-computer-use/harness' +import { playwrightComputer } from '@tanstack/ai-computer-playwright' +import { openaiText } from '@tanstack/ai-openai' +import { memoryPersistence } from '@tanstack/ai-persistence' + +// One time, outside `plugins`, so the sessions share one Chromium. +const browser = computerUse({ + driver: playwrightComputer({ startUrl: 'https://tanstack.com' }), +}) + +export const agent = defineHarness({ + name: 'acme/browser-agent', + adapter: openaiText('gpt-6.1-sol'), + plugins: () => [permissions(), browser], +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(agent, { threadId: 'browser-1' }) +await session.send('Go to https://tanstack.com/ai and tell me the title of the page.') +``` + +- The model gets the `computer` tool. With it, the model clicks, types, scrolls, and takes screenshots. +- The model also gets the `navigate` tool when the driver can open a URL. `playwrightComputer()` can. +- Chromium starts on the first action. Each thread gets a new page, and the page opens `startUrl`. + +To use Anthropic, change only the adapter, for example to `anthropicText('claude-opus-5-5')`. + +## 2. Answer a safety check + +OpenAI can mark an action as risky, for example when the page tells the model to do something else. Before that action runs, `permissions()` asks the user. The turn stops, and you answer the approval: + +```ts group=harness-computer-use +// wait.message is, for example: Allow computer {"action":{"type":"click",...},"pending_safety_checks":[...]}? +const waits = session.snapshot().pendingWaits +await session.resolve({ + inputId: 'approve-1', + resume: waits.map((wait) => ({ + interruptId: wait.waitId, + status: 'resolved', + payload: 'once', + })), +}) +await session.settled({ inputId: 'approve-1' }) +``` + +- `once` runs the action. The model gets the screenshot, and OpenAI gets the acknowledged check. +- `always` runs the action too, but it saves no rule. The next safety check asks again. +- `reject` does not run the action. The model gets an error result. + +The other actions run with no question. Anthropic sends no safety checks, so its actions never ask. + +In the `plan` mode, the `computer` tool does not run. To ask before every action, add a rule: `permissions({ rules: [{ tool: 'computer', decision: 'ask' }] })`. + +CAUTION: Keep `permissions()` in the plugins. Without it, an action with a safety check does not run. In the `bypass` mode, it runs with no question. See [Pick a mode](./permissions#pick-a-mode). + +## 3. Choose how many screenshots the model sees + +Each action returns a screenshot. The plugin stores it in the media store of the session. The transcript and the log keep a small `harness-media:` reference to each one. + +The model gets only the 3 newest screenshots. An older screenshot goes to the model as a 1x1 image, so a long task does not send every old screenshot again. To change the number, set `keepScreenshots`: + +```ts group=harness-computer-use +export const browserWithHistory = computerUse({ + driver: playwrightComputer({ startUrl: 'https://tanstack.com' }), + keepScreenshots: 5, +}) +``` + +The media store keeps every screenshot, so a client can still show all of them. To keep the files after a restart, give the host a blob store. See [Keep media across restarts](./media#keep-media-across-restarts). + +## When the pages close + +- All the sessions of a host that use the same `computerUse()` object share one Chromium. +- A thread that stays idle for `idleTimeout` loses its page. The default is 10 minutes. The next action of that thread opens a new page at `startUrl`. +- `host.close()` closes every page, then Chromium. +- The pages live in the memory of the host. When another host opens the thread, the thread gets a new page there. + +The browser can open every address that your server can reach. See [Keep the browser away from private addresses](../computer-use/computer-use#keep-the-browser-away-from-private-addresses). To use a Linux desktop or your own computer, pass another driver from [Drivers](../computer-use/drivers). + +## What you have now + +Run the session from step 1. The model calls `navigate`, reads the screenshot, and answers with the title of the page. Your harness agent now uses a browser, and each screenshot is in the media store of the session. diff --git a/docs/harness/connect.md b/docs/harness/connect.md new file mode 100644 index 0000000000..23362c964b --- /dev/null +++ b/docs/harness/connect.md @@ -0,0 +1,514 @@ +--- +title: Connect clients to a harness +id: harness-connect +order: 2 +description: "Serve a harness session over HTTP, SSE, or WebSocket. Talk to it from a web app or another chat() call." +keywords: + - tanstack ai + - harness + - AG-UI + - websocket + - harnessText +--- + +Your harness runs on a server, but the people who use it are in a browser or another agent. `createHarnessHandler` serves a session over HTTP. A web app talks to it with `createHarnessClient`, and another `chat()` call with `harnessText`. + +## Serve the session over HTTP + +Mount one fetch handler on a route. The host reads the user of each request with its `access`, so no endpoint is open by accident. + +```ts group=harness-connect +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +export const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host, harness: assistant }) +``` + +- `getUser` reads the user of each request. `null` answers `401`. +- `isAllowed` decides each action. A refused action answers `403`. `membersOnly()` lets a user use the threads they opened and the threads shared with them. +- A host with no `access` needs `openAccess` on the handler, else `createHarnessHandler` throws at the start. See [Serve with no login](./access#serve-with-no-login). + +[Access and tenants](./access) has the full rules. + +When more than one instance serves the same URL, also set `media: { secret }`. The secret signs media URLs, so every instance must use the same value. Without it, the handler signs with a random key of its own, so a signed URL works only on this instance, until it restarts. + +On a host with `ownership` options, `createHarnessHandler` throws at the start when `media` is not set. On another host without `media`, the first handler logs one warning. + +```ts group=harness-connect +const mediaKey = process.env.MEDIA_KEY +const oldMediaKey = process.env.OLD_MEDIA_KEY +if (!mediaKey || !oldMediaKey) { + throw new Error('Set MEDIA_KEY and OLD_MEDIA_KEY.') +} + +export const rotatingHandler = createHarnessHandler({ + host, + harness: assistant, + // The first key signs. Every key checks the URLs. + media: { keys: [mediaKey, oldMediaKey] }, +}) +``` + +- Each key must be at least 32 bytes. A shorter key throws at the start. +- To change the key, put the new key first in `keys` and keep the old key after it. A URL that the old key signed works until it expires (1 hour). Then remove the old key. +- A signed URL works for 1 hour. To change that, set `media.urlTtlMs`, for example `{ secret, urlTtlMs: 5 * 60_000 }` for 5 minutes on private files. +- A signed URL works only for this harness, in the tenant of the user who got it. Each request still needs a user from `getUser` that can read the thread, so send the login with it. For a request with no login, see [Serve media to a request with no login](./access#serve-media-to-a-request-with-no-login). When the thread is deleted, its URLs answer `404`. + +A request body over `media.maxBodyBytes` (default 32 MiB) answers `413`. See [Files inside a client message](./media#files-inside-a-client-message). + +The handler answers these paths under your route. For chat and events: + +- `GET capabilities`: the AG-UI capabilities, with the agents in `expose.agents`. +- `POST run`: standard AG-UI. One request runs one prompt and streams it as SSE. The prompt keeps every content part of the last user message, for example an image. `useChat` works with it. See [Use it from useChat](#use-it-from-usechat). +- `GET run?threadId=`: the saved messages, the running turn, and the waiting approvals, for a `useChat` that loads the thread after a reload. `pendingWaits` has every wait of the thread, as `GET snapshot` has them, also a `step.ask` question. +- `GET run?runId=&threadId=`: the turn with that run id as SSE, from its first event. A reloaded `useChat` joins a running turn with it. Another instance finds the turn only with `threadId`. See [Join a turn after a reload](#join-a-turn-after-a-reload). +- `GET events?threadId=`: every event of the session as SSE. Each event id is a cursor, so a reconnect with `Last-Event-ID` continues where it stopped. + +For one session: + +- `POST control`: send `{ threadId, input }`, for example `{ op: 'prompt', message }`. You get a receipt back. Agents, settings, config keys, and commands need `expose`. See [Choose what clients can change](#choose-what-clients-can-change). +- `GET snapshot?threadId=`: the status, running operations, and waiting approvals. +- `GET transcript?threadId=` and `GET describe?threadId=`: the saved messages, and the commands and settings that a client can use. +- `GET agent-transcript?threadId=&subagentRunId=`: the saved messages of one child agent of the thread. See [Show what a child wrote after a reload](./subagents#show-what-a-child-wrote-after-a-reload). +- `POST media`, `GET media-url`, and `GET media`: upload a file, get a signed URL, and read the bytes. A signed URL works for 1 hour (`media.urlTtlMs`), and only for a user who can read the thread. See [Send and show media](./media). + +For a list of sessions: + +- `GET sessions`: one page of the sessions that this user can list, newest first. Add `withArchived=true` to also get the archived sessions. +- `POST sessions`: change a session with `{ op, threadId }`. The ops are `rename`, `archive`, `unarchive`, `update`, `delete`, `fork`, `share`, and `unshare`. `client.sessions` has one method for each op. A thread that is not there throws `404`. See [List, rename, and fork sessions](./sessions) and [Share the thread](./shared-threads#2-share-the-thread). +- `GET host-events`: the status changes and index changes of the sessions that this user can read, as SSE. + +A read of a thread that does not exist answers `404`. Only `GET run?threadId=` answers an empty thread, because `useChat` reads it before the first message. + +The read paths read the thread without a session, so they set up no plugins and claim nothing. `GET describe` opens the session, because the plugins give the commands and the settings. For a read route of your own, use `host.observe`. See [Read a thread without a session](./lifecycle#read-a-thread-without-a-session). + +## Cursors + +Each event id of `GET events` and `GET run`, and the `cursor` of each `harness.event` frame, is an opaque string. Send it back as `Last-Event-ID` or `from`, as it is. Do not parse it. + +A cursor is a position in the log of the thread. It works on every instance that reads the same log store, and after a restart when the store keeps the log. With the memory log of `memoryPersistence()`, only this process knows the log. + +To run more than one instance behind one URL, give every instance the same log store. + +Sometimes the events after a cursor are gone. A host with `keepLogRecords: 'since-snapshot'` drops old records (see [Drop the records before a snapshot](./lifecycle#drop-the-records-before-a-snapshot)), and before its log opens, a session keeps only its newest events in memory. A reader whose cursor is in the lost part gets one `CUSTOM` event first: + +- Its name is `harness.gap` (`HARNESS_EVENTS.gap`). +- Its value is an `EventGap`: `{ from, resumeAt, reason }`. `reason` is `truncated` or `evicted`. Its cursor is `resumeAt`. +- Then the stream goes on from `resumeAt`. + +The session view loads the snapshot and the transcript again when it sees this event. A custom client must do the same. See [When events are missing](./protocol#when-events-are-missing). + +## Keep the stream open + +Every transport sends the same frames: each event is an AG-UI wire chunk in a `harness.event` frame. A long stream stays usable through proxies and slow networks: + +- The first frame of `GET events` and of the socket is `harness.hello`, with the protocol version `v`. A client can ask for a version with `?v=2` or `v: 2`. +- When the stream is idle, the server sends a heartbeat every 20 s. Set the idle timeout of your proxy to more than that. +- A client that does not read for 30 s gets a `slow_consumer` error frame, and the stream closes. Connect again from the last cursor. +- Every answer of the handler has an `X-Request-Id` header, and every error frame has a `code` and a `requestId`. Log the id, to find the request in the server logs. +- To know who has a stream open, pass `presence: { onConnect, onDisconnect }` to `createHarnessHost`. See [Show who is here](./shared-threads#8-show-who-is-here). + +To change the heartbeat or the limits, pass `stream` to `createHarnessHandler` and `handleHarnessSocket`. [The harness wire protocol](./protocol) lists every frame, the error codes, and the `stream` values, for a client in any language. + +## Choose what clients can change + +A client can send a prompt and answer questions. Other inputs can change how the session works, so a client can use only what the harness exposes. Nothing is exposed by default. + +Name what a client can use in `expose`: + +```ts group=harness-connect +export const team = defineHarness({ + name: 'acme/team-assistant', + adapter: openaiText('gpt-6.1-sol'), + expose: { settings: ['instructions'], commands: ['undo'] }, +}) +``` + +| Field | What a client can do | +| --- | --- | +| `agents` | Run these agents, for example with `client.agents..start(input)`. | +| `settings` | Change these [thread settings](./thread-settings) with `client.updateThreadSettings({ settings })`. | +| `config` | Set these plugin config keys with `client.setPluginSetting({ key, value })`. | +| `commands` | Run these commands with `client.runCommand({ name })`, or with `/name` in a [session view](./custom-ui) on a client. | +| `tools` | Run these harness tools with no model turn, with `client.runTool({ name, args })`. See [Run a tool without a model](./tool-runs). | +| `records` | Add records of these types to the thread with `client.appendRecords({ records })`. See [Let a client add records](./session-log#let-a-client-add-records). | + +- Any other input of these kinds gets `{ status: 'rejected', reason: 'not_exposed' }`. +- To check what an input says, for example its `context`, use `acceptInput` on the host. See [Check an input on the server](./inputs#check-an-input-on-the-server). +- `GET describe` and `client.describe()` list only the exposed commands and config keys. A UI built from them shows only what a client can use. +- Server code that calls the session is not limited. `session.setPluginSetting()` and `session.runCommand()` work for every key and command, and `session.describe()` lists all of them. +- `defineHarness` checks the names in `agents` and `tools`, and TypeScript suggests them. Plugins add their config keys and commands when a session opens, so it does not check those names. + +## Talk to it from a web app + +`createHarnessClient` wraps those endpoints. Import the harness with `import type`, so no server code reaches the browser. + +```ts group=harness-connect-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { assistant } from './harness' + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'user-1-thread', + headers: { authorization: 'Bearer my-token' }, +}) + +const { cursor } = await client.snapshot() +await client.send('Summarize my open tickets.') + +for await (const entry of client.events({ from: cursor })) { + if (entry.event.type === 'TEXT_MESSAGE_CONTENT') { + console.log(entry.event.delta) + } +} +``` + +`events()` reconnects after a network error and continues from the last cursor. It also reconnects when no bytes come for 60 s, so a dead connection (a laptop that slept) does not stay `open`. To change that time, pass `idleTimeoutMs` to `createHarnessClient`. Keep it larger than the [heartbeat](./protocol#stream-values) of the server, or the client reconnects again and again. A second tab, a phone, or a reload all see the same session. + +Without `from`, `events()` starts at the newest event, so it sends no old events. That is why the example reads the snapshot cursor before it sends. + +- To read every event from the start of the session, pass `from: '0'`. Do this when you rebuild a transcript from the events. +- If the snapshot answer has no `cursor` (the URL is not a harness handler), `events()` throws and does not try again. + +To upload files and show the media that your agents make, read [Send and show media](./media). + +## Send a message to a running agent + +A background agent runs for minutes, and the user wants to correct it from the page, or ask for one more change after it ends. The browser sends the run a message with `client.send(message, { to: operationId })`. + +1. On the server, list the agent in `expose.agents`. A client can start and message only those agents: + +```ts group=harness-connect-agents +import { defineAgent } from '@tanstack/ai' +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const drafter = defineAgent({ + name: 'drafter', + description: 'Drafts a blog post', + run: (ctx) => ctx.chat({ adapter: openaiText('gpt-6.1-sol'), stream: false }), +}) + +export const studio = defineHarness({ + name: 'acme/studio', + adapter: openaiText('gpt-6.1-sol'), + agents: [drafter], + expose: { agents: ['drafter'] }, +}) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, +}) + +export const handler = createHarnessHandler({ host, harness: studio }) +``` + +2. In the browser, send the message to the id of the run: + +```ts group=harness-connect-agents-client +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { studio } from './harness' + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'user-1-thread', + headers: { authorization: 'Bearer my-token' }, +}) + +export async function correct(operationId: string) { + const receipt = await client.send('Cite papers, not blogs.', { + to: operationId, + }) + if (receipt.status === 'rejected') { + console.warn(receipt.reason) + } +} + +export function addTitle(operationId: string) { + return client.send('Add a title.', { to: operationId, mode: 'followUp' }) +} +``` + +The id of the run is the `operationId` in the receipt of `client.agents.drafter.start()`. A [session view](./custom-ui#3-act-on-the-session) also lists the running agents with their ids. + +- `mode: 'steer'` (the default): the message joins the next model call of the run. +- `mode: 'followUp'`: the agent runs again after the run ends. `receipt.operationId` is the id of the new run. + +`send` with `to` posts an `agentMessage` input to `POST control`. It runs as the user that `getUser` returned. The input looks like this: + +```json +{ "op": "agentMessage", "operationId": "op-agent-m1x2-3", "message": "Add a title.", "mode": "followUp" } +``` + +The session rejects the message with a `reason` in these cases: + +- `not_running`: the session does not know the run. Another host can run it, or the id is wrong. +- `not_exposed`: the agent of the run is not in `expose.agents`. + +For how steers and follow-ups run on the server, see [Message a running agent](./subagents#message-a-running-agent). + +## Use it from useChat + +Your app already uses `useChat`. You want the same chat screen on a harness, with approvals, tools that run in the browser, and the thread after a reload. Point `useChat` at the `run` path of the handler. + +1. Put the tools in a file that the server and the browser both import: + +```ts group=harness-connect-tools +import { toolDefinition } from '@tanstack/ai' +import { z } from 'zod' + +export const deploy = toolDefinition({ + name: 'deploy', + description: 'Deploy the app. A person must approve it.', + needsApproval: true, + inputSchema: z.object({ env: z.string() }), + outputSchema: z.object({ ok: z.boolean() }), +}) +``` + +2. On the server, give the harness the tool with its implementation: + +```ts group=harness-connect-tools-server +import { defineHarness } from '@tanstack/ai-harness' +import { openaiText } from '@tanstack/ai-openai' +import { deploy } from './tools' + +export const deployer = defineHarness({ + name: 'acme/deployer', + adapter: openaiText('gpt-6.1-sol'), + tools: [ + deploy.server(async () => { + // Your deploy code runs here. + return { ok: true } + }), + ], +}) +``` + +Serve `deployer` with `createHarnessHandler`, as in [Serve the session over HTTP](#serve-the-session-over-http). + +3. In the browser, give `useChat` the same tool definitions: + +```tsx group=harness-connect-tools-client +import { fetchServerSentEvents, useChat } from '@tanstack/ai-react' +import { deploy } from './tools' + +export function Chat() { + const { messages, interrupts, sendMessage } = useChat({ + threadId: 'user-1-thread', + connection: fetchServerSentEvents('/api/harness/run'), + tools: [deploy.client()] as const, + // The input's context: tools and plugins read it. + forwardedProps: { page: '/releases' }, + }) + + return ( + <> +

{messages.length} messages

+ {interrupts.map((interrupt) => + interrupt.kind === 'tool-approval' ? ( + + ) : null, + )} + + + ) +} +``` + +The approval reaches the harness, and the tool runs on the server. What else `POST run` does for `useChat`: + +- Each turn runs as the `runId` of its request. A retry with the same `runId` runs once. The same `runId` with another message gets `409`. +- A tool with only a `.client()` implementation runs in the browser, and the same turn gets its result. +- A question of `step.ask` or of an MCP server arrives on the stream of the turn that waits for it. See [Wait for a person](./waits). +- The `forwardedProps` of the request are the input's [context](./inputs#send-context-with-a-message). +- With `persistence: true`, `useChat` loads the thread from `GET run?threadId=`. +- If a turn still runs after a reload, `useChat` joins it with `GET run?runId=`. The turn shows from its first event, and its answer continues to stream. +- Each event of `POST run` has its log cursor as its id. If the connection drops, `useChat` sends the request again with `Last-Event-ID`. Any server continues the stream from there, and the turn does not run again. + +### Join a turn after a reload + +To keep the thread after a reload, set `persistence: true` in `useChat`. Put the `threadId` in the URL of the connection too: + +```tsx group=harness-connect-tools-client +export function ReloadableChat() { + const { messages } = useChat({ + threadId: 'user-1-thread', + connection: fetchServerSentEvents('/api/harness/run?threadId=user-1-thread'), + tools: [deploy.client()] as const, + persistence: true, + }) + return

{messages.length} messages

+} +``` + +The join has these rules: + +- The host checks that the user can read the thread, the same as for every other path. +- The join request of `useChat` sends only the run id. With `threadId` in the URL, any instance that reads the same log store can answer the join. It reads the turn from the log of that thread. +- Without `threadId`, only the instance that runs the turn finds it. Another instance answers `400`. +- A turn that ended gives the events that the session still keeps, and then the stream ends. + +### Stop a turn from useChat + +The `stop()` of `useChat` closes the request, but the turn continues on the server. A closed connection never cancels work, because another tab or a reload can still read the turn. To stop the turn too, send a `cancel` input for its run. + +1. On the server, serve the harness with one handler. `POST run` streams the turn to `useChat`, and `POST control` takes the cancel: + +```ts group=harness-connect-stop-server +import { createHarnessHost, defineHarness, membersOnly } from '@tanstack/ai-harness' +import { createHarnessHandler } from '@tanstack/ai-harness/server' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +export const writer = defineHarness({ + name: 'acme/writer', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ + persistence: memoryPersistence(), + access: { + getUser: ({ request }) => + request.headers.get('authorization') === `Bearer ${process.env.HARNESS_TOKEN}` + ? { id: 'user-1' } + : null, + rules: membersOnly(), + }, +}) + +// Mount it on `/api/harness/*`. +export const handler = createHarnessHandler({ host, harness: writer }) +``` + +2. In the browser, call `client.cancel` with the `runId` of `useChat` in your stop handler. The turn of `POST run` runs as that id: + +```tsx group=harness-connect-stop-client +import { fetchServerSentEvents, useChat } from '@tanstack/ai-react' +import { createHarnessClient } from '@tanstack/ai-harness/client' +import type { writer } from './harness' + +const headers = { authorization: 'Bearer my-token' } + +const client = createHarnessClient({ + url: '/api/harness', + threadId: 'user-1-thread', + headers, +}) + +export function StoppableChat() { + const { messages, runId, isLoading, stop, sendMessage } = useChat({ + threadId: 'user-1-thread', + connection: fetchServerSentEvents('/api/harness/run', { headers }), + }) + + const stopTurn = async () => { + stop() + if (!runId) return + const receipt = await client.cancel({ operationId: runId }) + if (receipt.status === 'rejected') console.warn(receipt.reason) + } + + return ( + <> +

{messages.length} messages

+ + {isLoading ? : null} + + ) +} +``` + +- The cancel works on any server that reads the same log store. The input goes to the server that runs the turn. +- If the turn already ended, the receipt can be `rejected` with the reason `not_running`. +- The turn ends with the status `cancelled`. + +## Use a WebSocket + +For one connection that carries events and inputs, hand the socket and its upgrade request to `handleHarnessSocket`: + +```ts group=harness-connect +import { handleHarnessSocket } from '@tanstack/ai-harness/server' +import type { WebSocketLike } from '@tanstack/ai' + +export function onUpgrade(socket: WebSocketLike, request: Request) { + handleHarnessSocket({ host, harness: assistant, socket, request }) +} +``` + +The client sends `{ type: 'harness.subscribe', threadId }` first, and the host answers with a `harness.hello` frame. After that, each `harness.input` frame gets a `harness.receipt`, and every event arrives as a `harness.event` frame with its cursor. + +- The host reads the user from the upgrade request with `getUser`, the same as for HTTP. Every input on the socket runs as that user. +- A user that `getUser` refuses gets an `unauthorized` error frame, and the socket closes. +- The socket handles the frames of one connection in the order they came. So a prompt with a large image reaches the session before the frame after it. +- A user who cannot read the thread, or who loses access later, gets a `forbidden` error frame, and the socket closes. +- A frame over `media.maxBodyBytes` (default 32 MiB) gets an error frame, and the socket closes with code `1009`. Your WebSocket server reads the whole frame before this check, so the check does not cap memory. Set the frame limit of your server too, for example `maxPayload` of `ws`. +- On a host with no `access`, pass `openAccess`, the same as for `createHarnessHandler`. +- Pass `socket: true` to `createHarnessHandler` too. Then `GET capabilities` says `transport.websocket: true`, so clients know that they can use the socket. The default is `false`. + +## Use it as the model of another chat + +`harnessText` turns a harness into a text adapter. The outer chat sends a message, and the harness runs a full turn with its own tools, plugins, and agents: + +```ts group=harness-connect +import { chat } from '@tanstack/ai' +import { harnessText } from '@tanstack/ai-harness' + +const stream = chat({ + adapter: harnessText(assistant, { host }), + messages: [{ role: 'user', content: 'Fix the failing test.' }], + threadId: 'outer-thread', +}) +``` + +Each outer thread gets its own inner session, so the harness keeps its own history. The inner session gets every content part of the last user message, so images and files go through too. + +- Who runs the inner turn: when the outer `chat()` runs inside a harness turn, the inner session opens as the user of that turn. Else it opens as `user`, for example `harnessText(assistant, { host, user: { id: 'ada' } })`. +- Access: the inner thread is `/inner/`. When `host` has `access` and also runs the outer thread, the inner session is checked through the outer thread, so every user who can read a shared outer thread can use its inner session. From app code, `host.open`, `host.submit`, `host.fork`, and `host.transfer` refuse any thread id with `/inner/`. +- Usage: the outer run gets the token usage of every inner model call, so the usage totals of the outer session count them. +- Cancel: an abort of the outer call cancels the inner turn. For a remote harness, `harnessText({ url, token })` also tells the remote host to cancel the turn. + +If a plugin picks the model of the harness, the harness has no adapter to read its input kinds from. Pass them with `harnessText(assistant, { host, inputModalities: ['text', 'image'] })`. + +## What you have now + +- One handler that serves standard AG-UI and the session stream, as the user of each request. +- Clients that can change only what you expose. +- A `useChat` screen on the harness, with approvals, tools that run in the browser, a reload that joins the running turn, and a stop button that cancels the turn. +- A typed web client that reconnects and resumes from a cursor. +- Messages from the browser to a running agent. +- A WebSocket, and a harness you can call from `chat()`. + +Next: run the same harness in a terminal with the [CLI](./cli). diff --git a/docs/harness/custom-ui.md b/docs/harness/custom-ui.md new file mode 100644 index 0000000000..9a75eb03e0 --- /dev/null +++ b/docs/harness/custom-ui.md @@ -0,0 +1,727 @@ +--- +title: Build your own UI +id: harness-custom-ui +order: 13 +description: "Show a harness session in your own terminal screen or web app. One live store holds the messages, tool calls, approvals, and plugin state, and works with any UI library." +keywords: + - tanstack ai + - harness + - session view + - tanstack store + - ink + - custom ui +--- + +You want your own screen for your harness: your own layout in a terminal, or a page in your web app. `session.events()` gives you raw AG-UI events. To show them, you must join text deltas, follow each tool call, and keep a list of open approvals. That is a reducer, and you must keep it correct. + +`createSessionView` does that work. It keeps one live state in a TanStack Store, and it gives you actions and typed events. You show the state with any UI library. + +At the end of this page you have a terminal screen in about 20 lines, and the same view in a web app. + +## Install + +Install the harness and the TanStack Store adapter for your UI library. With no UI library, the harness is enough. + + + +react: @tanstack/ai-harness @tanstack/react-store +vue: @tanstack/ai-harness @tanstack/vue-store +solid: @tanstack/ai-harness @tanstack/solid-store +svelte: @tanstack/ai-harness @tanstack/svelte-store +preact: @tanstack/ai-harness @tanstack/preact-store +angular: @tanstack/ai-harness @tanstack/angular-store +vanilla: @tanstack/ai-harness +octane: @tanstack/ai-harness + + + +The terminal screen on this page uses Ink, which needs Node 22 or later. Install Ink and React too: + + + +react: ink react + + + +## 1. Make a view + +Open a session, then give it to `createSessionView`: + +```ts group=harness-custom-ui +import { createHarnessHost, defineHarness } from '@tanstack/ai-harness' +import { createSessionView } from '@tanstack/ai-harness/view' +import { memoryPersistence } from '@tanstack/ai-persistence' +import { openaiText } from '@tanstack/ai-openai' + +const assistant = defineHarness({ + name: 'acme/assistant', + adapter: openaiText('gpt-6.1-sol'), +}) + +const host = createHarnessHost({ persistence: memoryPersistence() }) +const session = await host.open(assistant, { threadId: 'thread-1' }) + +const view = createSessionView(session) +await view.ready +``` + +`view.ready` resolves when the view has the saved messages, the status, and the commands and settings of the session. If you open a thread again, its history is in the view immediately. And if a turn is still running when you open the view, you see that turn from its start, not only from the moment you opened it. + +## 2. Show the state + +`view.store` is a TanStack Store. `useSelector` reads one part of the state, and your component updates only when that part changes. This Ink screen shows the messages and the status: + +```tsx ignore +import { Box, Text, render } from 'ink' +import { useSelector } from '@tanstack/react-store' +import type { SessionView } from '@tanstack/ai-harness/view' + +function Screen({ view }: { view: SessionView }) { + const messages = useSelector(view.store, (state) => state.messages) + const status = useSelector(view.store, (state) => state.status) + return ( + + {messages.map((message) => + message.role === 'assistant' ? ( + + {message.parts.map((part) => (part.type === 'text' ? part.text : '')).join('')} + + ) : ( + + {message.role === 'user' ? `> ${message.text}` : message.text} + + ), + )} + {status} + + ) +} + +render() +await view.send('Summarize the README.') +``` + +Put the code of steps 1 and 2 in one `.tsx` file, then run it. The screen shows your prompt, then the answer while it streams, then the status `idle`. Press Ctrl+C to quit. + +An Ink screen with approvals, questions, sign-ins, and child agents is in [`examples/harness-cli/src/tui.tsx`](https://github.com/TanStack/ai/blob/main/examples/harness-cli/src/tui.tsx). It runs from [`runCli({ ui })`](./cli#run-your-own-screen). + +Each TanStack Store adapter reads the store the same way. Vue, Solid, Svelte, and Preact have `useSelector`. Angular has `injectSelector`. With no UI library, subscribe to the store: + +```ts group=harness-custom-ui +const subscription = view.store.subscribe((state) => { + console.log(`${state.status}: ${state.messages.length} messages`) +}) +``` + +Call `subscription.unsubscribe()` to stop. + +### What the state holds + +The conversation: + +- `messages`: the lines to show. Each message has a `role`: `user`, `assistant`, or `notice`. +- `status`: `idle`, `running`, or `requires_action`. Only the host that owns the thread runs turns. On another host, `idle` does not mean that no turn runs. +- `connection`: `open`, `reconnecting`, or `closed`. + +What waits for the user. Each item is a [wait](./waits) of the session: + +- `approvals`: tool calls that wait for a yes or a no. +- `clientTools`: tool calls that run in your UI and wait for their output. See [Run a tool in the browser](#run-a-tool-in-the-browser). +- `interrupts`: everything else a turn can wait for, for example a value that a tool asks for. See [Answer other interrupts](#answer-other-interrupts). +- `questions`: questions of `step.ask` (in a command, a durable tool, or an agent run) and of an MCP server. See [Show a question](#show-a-question). +- `signIns`: connectors that need a sign-in, with a `url` and a `userCode` when the connector gives them. + +`waits` lists every open wait in one array, as `snapshot().pendingWaits` does, with the `kind` of each one. Use it for a badge or a count. Answer each wait through its item in the lists above. + +The session: + +- `threadId`: the id of the conversation. +- `agents`: the background agents that run now. Each one has an `id`, a `name`, a `status` (`running`, or `waiting` for an answer), `send`, and `cancel`. +- `queuedTurns` and `waitingInputs`: how many messages wait for their turn, and which ones, in the order they run. +- `plugins`: the saved state of each plugin, by plugin name. + +Settings and usage, for a settings panel or a status bar: + +- `commands`, `config`, and `tools`: what the session has, for a help screen. +- `models`: the model names from `defineHarness({ models })`, for a model menu. +- `settings`: the stored settings of the thread, for example the model it uses now. +- `usage`: the token usage of the thread. It updates after each model call. + +In a harness with `routing`, an assistant message also has `agent`: the name of the root agent that wrote it, or `undefined` for the main model. After a [handoff](./handoffs), the next agent writes a new message in the same turn. + +An assistant message has `parts`. Each part is one of these: + +- `text` or `reasoning`: the text, which grows while it streams. +- `tool-call`: a tool call with its `name`, `args`, and `status` (`running`, `done`, `failed`, or `needs-approval`). +- `agent`: a child agent, with its own `parts`, its `status`, and its `result` when it gives one. See [Show subagents](#show-subagents). +- `media`: a file that an agent made. See [Show media](#show-media). + +A notice has a `kind`: + +- `info`: the session continues a turn that a crash stopped, or a [reset](./fork-and-reset#reset-with-a-handoff-note) started a fresh context. +- `error`: a turn or an action failed. +- `rejected`: the session did not accept a message. +- `command`: the text result of a command. +- `ui`: a line that you added with `view.notice({ text })`. + +## 3. Act on the session + +Send work: + +- `view.send(text)`: sends a prompt. While a turn runs, the text steers that turn. Text that starts with `/` runs a command, for example `/goal all tests pass`. With `{ as: 'operator' }`, it sends the [reply of an operator](./handoffs#reply-as-the-operator), and no model runs. +- `view.followUp(text)`: the message waits, and runs as its own turn after the current work. +- `view.continueWithoutMessage()`: starts a turn from the saved messages, with no new message. +- `view.runCommand({ name, input })`: runs a command, for example from a button. +- `view.putMedia({ body, mimeType, name })`: stores a file. Send the record you get back with `view.send(text, [record])`. See [Show media](#show-media). +Change the session: + +- `view.setPluginSetting({ key, value })`: changes a config value, for example `view.setPluginSetting({ key: 'mode', value: 'plan' })`. +- `view.updateThreadSettings({ settings })`: changes the stored settings of the thread, for example `view.updateThreadSettings({ settings: { model: 'strong' } })`. +- `view.reset({ note })`: the model starts fresh from the next turn, with your `note` first. See [Reset with a handoff note](./fork-and-reset#reset-with-a-handoff-note). +- `view.revert({ messageId })` and `view.unrevert()`: go back to a message, and undo that. + +Stop work: + +- `view.cancel()`: cancels the running turn. `view.cancel({ operationId })` cancels one operation, for example a background agent. +- `view.cancelInput({ inputId })`: drops a message from `waitingInputs` before it runs. `view.changeInputDelivery({ inputId, delivery: 'steer' })` moves it into the running turn instead. +- `view.dispose()`: stops the view when your screen closes. After that, each action throws an error. + +On a view with a [`HarnessClient`](#a-ui-in-the-browser), two actions need `expose` in the harness: + +- `view.command`, and a `/command` in `view.send`, run only the commands in `expose.commands`. +- `view.setPluginSetting` sets only the keys in `expose.config`. + +For any other command or key, the view adds a `rejected` notice with the text `Not accepted: not_exposed`. It also calls your `'error'` handler with that text. On a client, `commands` and `config` in the state list only the exposed items. + +See [Choose what clients can change](./connect#choose-what-clients-can-change). + +Answer what waits: + +- `approval.approve()` and `approval.reject()`: answer one approval. `view.approve({ id })` and `view.reject({ id })` do the same by id. +- `view.approveAll()` and `view.rejectAll()`: answer all open approvals. +- `call.resolve(output)` and `call.fail(message)`: answer one client tool call. +- `request.resolve(value)` and `request.cancel()`: answer one item of `interrupts`. +- `question.answer(value)`: answers a question. + +Want the tool to run with different arguments than the model picked? Pass them to `approve`: + +```ts group=harness-custom-ui +const [waiting] = view.store.get().approvals +waiting?.approve({ editedArgs: { path: 'notes/draft.md' } }) +``` + +`reject({ payload })` sends data with a no, when the tool's approval schema takes some. + +An approval of [`permissions()`](./permissions#answer-an-approval) reads `payload` as the answer of the user: + +- `approve()` runs the call one time. +- `approve({ payload: 'always' })` runs the call, and the plugin saves an `allow` rule for it. See [What `always` saves](./permissions#what-always-saves). +- `reject({ payload: 'Use pnpm, not npm.' })` refuses the call. The model gets your text as the tool error. An object with a `message` works too. +- `approve({ editedArgs })` also refuses the call, because this approval cannot run changed arguments. The model gets a tool error that tells it why. + +The user types `once`, `always`, or a message for the model. This function sends the answer: + +```ts group=harness-custom-ui +import type { Approval } from '@tanstack/ai-harness/view' + +function answerPermission(approval: Approval, reply: string) { + if (reply === 'once') approval.approve() + else if (reply === 'always') approval.approve({ payload: 'always' }) + else approval.reject({ payload: reply }) +} +``` + +If a turn waits for more than one approval, client tool, or interrupt, the turn continues after you answer all of them. + +A background agent run can wait too, for example for an approval of `permissions()`. Its waits are in the same `approvals`, `clientTools`, and `interrupts` lists. The view sends the answer to each of them at once, and the run continues. See [When a background agent waits](./subagents#when-a-background-agent-waits). + +Message a background agent: + +- `agent.send(text, { mode })`: sends a message to one item of `state.agents`. `mode` is `'steer'` (the default) or `'followUp'`. See [Message a running agent](./subagents#message-a-running-agent). + +```ts group=harness-custom-ui +const [agent] = view.store.get().agents +if (agent) { + await agent.send('Add a title.', { mode: 'followUp' }) +} +``` + +`agent.cancel()` stops the run, and the runs it started. + +A session and a `HarnessClient` can do all of this. You can also give `createSessionView` your own `SessionViewSource`. Its methods take the same arguments as the session methods, for example `resolve({ resume })` and `cancel({ operationId })`. `send` takes `mode` for a steer or a follow-up, and `to` for a message to an agent run. + +A source must have these members: + +- Inputs: `send`, `resolve`, `cancel`, `answer`, `runCommand`, and `setPluginSetting`. +- Reads: `events`, `snapshot`, `transcript`, and `describe`. + +These members are optional: + +- `continueWithoutMessage`, `cancelInput`, `changeInputDelivery`, `updateThreadSettings`, `reset`, `revert`, and `unrevert`. Without one, its view action adds an `error` notice. +- `putMedia`. Without it, `view.putMedia` rejects. +- `mediaUrl` and `loadMedia`. Without `mediaUrl`, media parts have no `url`. Without `loadMedia`, `load()` rejects. +- `agentTranscript`. Without it, `view.loadAgent` does nothing, so a saved child shows only its card. + +## 4. Listen for events + +Some items need attention when they arrive. `view.on` calls your handler for each one. When a tool call waits for approval, this handler rings the terminal bell and adds a line to the screen: + +```ts group=harness-custom-ui +view.on('approval', (approval) => { + process.stdout.write('\u0007') + view.notice({ text: `${approval.tool} waits for your answer.` }) +}) +``` + +Items that wait for the user: + +- `'approval'`: an approval. Call `approve()` or `reject()` on it. +- `'clientTool'`: a client tool call. Call `resolve(output)` or `fail(message)` on it. +- `'interrupt'`: an item of `interrupts`. Call `resolve(value)` or `cancel()` on it. +- `'question'`: a question with a `message`, a `schema`, a `url` when the user must open a page, and `answer(value)`. +- `'signIn'`: a connector that needs a sign-in, with `connector`, `url`, and `userCode`. + +Progress: + +- `'toolCall'`: `{ id, name }`, when a tool call starts. +- `'agent'`: `{ id, name, status }`, when a child agent starts, finishes, or fails. +- `'error'`: the error text. The view also adds it to `messages` as a notice. +- `'turnEnd'`: `{ operationId }`, when a chat turn ends. A turn that stops for an approval has not ended, so it does not fire then. + +The view calls your handler after the item is in the state, so the handler can read `view.store.get()` and find the item. `view.on` returns a function that removes the handler. + +The view also keeps its state right on its own: + +- Each event from `client.events()` or `session.events()` has the `threadId` it belongs to. The `RUN_STARTED` event of a child run has the `parentRunId` of the run that started it. Use them to show the run tree in your own UI. +- When the server sends `harness.gap`, some events are gone. The view stops its read, loads the snapshot and the transcript again, then reads on from the new snapshot. So no turn shows twice after the gap. See [Write a client in any language](./protocol#when-events-are-missing). + +## Show a question + +A command, a durable tool, or an agent run can ask the user with `step.ask`. An MCP server can ask during a tool call. Each item of `state.questions` has: + +- `id` and `message`. +- `schema`: the JSON Schema of the answer, when the question has one. Without a schema, the answer is text. +- `secret`: the answer is a key or a password. Hide what the user types. +- `url`: a page that the user must open before they answer, for example a payment page. +- `answer(value)`: sends the answer. + +When a question has a `url`, show the link. The user opens it, then answers. The session does not open the page. This handler adds the link to the screen: + +```ts group=harness-custom-ui +view.on('question', (question) => { + if (question.url) view.notice({ text: `Open ${question.url}, then answer.` }) +}) +``` + +### Show a field only when it applies + +A schema can make a field apply only when another field has a value. JSON Schema writes this with `if` and `then`. In this schema, `address` applies only when `delivery` is `mail`: + +```json +{ + "type": "object", + "properties": { + "delivery": { "type": "string", "enum": ["pickup", "mail"] }, + "address": { "type": "string" } + }, + "required": ["delivery"], + "if": { "properties": { "delivery": { "const": "mail" } } }, + "then": { "required": ["address"] } +} +``` + +The view gives you the schema as is. Your UI reads the condition. This function returns the fields that `then` adds for the answer so far: + +```ts group=harness-custom-ui +function isRecord(value: unknown): value is Record { + return typeof value === 'object' && value !== null +} + +function fieldsThatApply(schema: unknown, answer: Record) { + if (!isRecord(schema) || !isRecord(schema.if) || !isRecord(schema.then)) { + return [] + } + const rules = isRecord(schema.if.properties) + ? Object.entries(schema.if.properties) + : [] + const matches = rules.every( + ([field, rule]) => isRecord(rule) && answer[field] === rule.const, + ) + const required = schema.then.required + return matches && Array.isArray(required) ? required : [] +} +``` + +Call it each time the user changes a field. Show a field from the result, and hide it again when the result no longer has it: + +```ts group=harness-custom-ui +const [question] = view.store.get().questions +const extra = fieldsThatApply(question?.schema, { delivery: 'mail' }) +// With the schema above, `extra` is ['address']. +``` + +## Answer other interrupts + +Some turns wait for something that is not a yes or a no. A tool can stop the turn to ask for a value, or your own code can stop it for a form. The view puts these in `state.interrupts`: the waits of kind `question` (for example a call of the `question` tool) and `human`. Each item has: + +- `id`, `reason`, and `message`: what the turn waits for. `reason` is the name the interrupt gave itself, for example `pick_region`. +- `payload`: the data the interrupt sent for your UI, for example the fields of a form. +- `responseSchema`: the JSON Schema of the answer, when it has one. [Show a field only when it applies](#show-a-field-only-when-it-applies) works for it too. +- `resolve(value)`: sends the answer. +- `cancel()`: answers with no value, and the turn continues without one. + +This handler shows each one, and answers the region picker right away: + +```ts group=harness-custom-ui +view.on('interrupt', (request) => { + view.notice({ text: request.message ?? `The turn waits for ${request.reason}.` }) + if (request.reason === 'pick_region') request.resolve({ region: 'eu-west-1' }) +}) +``` + +## Show media + +A user sends a screenshot, and an agent draws an image. The view gives you each file as a media part, with a URL that works in ``, `