feat: agent runners (Claude Code, Codex) ask mid-turn, store their tool rows, get their folder

docs/RUNNER-PROTOCOL.md

@@ -0,0 +1,102 @@
+# Agent runner protocol
+
+How the engine talks to an agent runner: a local OpenAI-compatible provider whose "model" is a coding
+CLI (Claude Code, Codex) that runs its own tools. The desktop app's shim
+(redrob-desktop `apps/desktop/node-host/harness.mjs`) is the one runner today. The shapes below are the
+ones redrob-desktop `apps/desktop/src/v2/features/HARNESS-NEEDS.md` uses; nothing was changed from it.
+
+## Final shapes
+
+### Provider options
+
+```jsonc
+"provider": {
+  "claude-code": {
+    "npm": "@ai-sdk/openai-compatible",
+    "options": {
+      "baseURL": "http://127.0.0.1:<port>/v1",
+      "providerExecutedTools": true, // read runner tool calls from the stream (below)
+      "agentRunner": true            // no title/summary run on this model
+    }
+  }
+}
+```
+
+Either flag marks the provider as a runner for `x-redrob-directory`.
+
+### Request headers (engine to runner)
+
+| Header | Value |
+| --- | --- |
+| `x-session-affinity`, `X-Session-Id` | engine session id |
+| `x-redrob-directory` | the session's folder, `encodeURIComponent`-encoded. Sent only to runner providers. |
+| `x-parent-session-id` | parent session id, for a subagent session |
+
+### Tools the runner ran (runner to engine, in the stream)
+
+One `tool_calls` delta per finished CLI tool call, in a normal chat-completions chunk:
+
+```json
+{
+  "choices": [{
+    "index": 0,
+    "delta": {
+      "tool_calls": [{
+        "index": 0,
+        "id": "runner_<cli id>",
+        "type": "function",
+        "function": { "name": "bash", "arguments": "{\"command\":\"ls\"}" },
+        "provider_executed": true,
+        "title": "ls",
+        "result": { "output": "a\nb\n" }
+      }]
+    }
+  }]
+}
+```
+
+- `result` is `{ "output": string }` or `{ "error": string }`.
+- `function.arguments` is a JSON string (an object is accepted too).
+- Each call needs its own `index` and a unique `id`; send the whole call in one delta.
+- Use `finish_reason: "stop"`, not `"tool_calls"`.
+- The engine stores each as a `tool` part of the assistant message (`state.status` `completed` or
+  `error`, `metadata.providerExecuted: true`) and runs nothing locally. A call shows once it has
+  finished.
+- Engine side: `packages/redrob/src/session/llm/runner-tools.ts`.
+
+Not replayed: on later turns the runner's own calls are left out of the prompt the engine sends back,
+so the history has no `tool_calls` without tool messages.
+
+### Approval mid-turn (runner to engine)
+
+```
+POST /session/:sessionID/permission
+{ "permission": "bash", "patterns": ["git push"], "metadata": { ... }, "always": ["git push"] }
+
+200 { "reply": "once" | "always" | "reject", "message"?: string }
+```
+
+- The engine finds the session's folder from the id in the path; no directory header is needed.
+- Blocks until the person answers the session's permission card.
+- `metadata` and `always` are optional. `always` defaults to `patterns`: the patterns an "always"
+  answer allows for the rest of the session.
+- The session's own permission rules and earlier "always" answers apply: an allowed pattern returns
+  `"always"` at once, a denied one returns `"reject"` with a `message`.
+- A reject with feedback typed by the person returns it as `message`.
+- Closing the HTTP request drops the pending request.
+- The request carries `metadata.runner: true` unless the runner sets `metadata.runner` itself.
+- Permission names are the engine's tool names (`bash`, `edit`, `write`, `read`, `webfetch`, ...);
+  patterns are the command or the file paths.
+
+### Titles and summaries
+
+- A chat on an `agentRunner` provider is titled after the first line of its first message; no model
+  call is made for it (unless the `title` agent names its own model).
+- Compaction is not skipped. It fires automatically only when the runner reports token usage that
+  crosses the threshold; a runner that reports none is compacted only when the person asks. That
+  request arrives without tools, and the shim answers it as a read-only side request.
+
+### Not done
+
+- `x-redrob-turn` (telling a retry from a new turn) is not sent; the shim does not need it for
+  correctness.

packages/redrob/src/permission/index.ts

@@ -100,8 +100,12 @@ const layer = Layer.effect(
       yield* events.publish(Event.Asked, info)
       return yield* Effect.ensuring(
         Deferred.await(deferred),
-        Effect.sync(() => {
+        // Still pending here means nobody answered: the asker went away (a cancelled turn, or a runner
+        // that dropped its HTTP call). Tell the clients, so an open card does not linger.
+        Effect.suspend(() => {
+          if (!pending.has(id)) return Effect.void
           pending.delete(id)
+          return events.publish(Event.Replied, { sessionID: info.sessionID, requestID: id, reply: "reject" }).pipe(Effect.ignore)
         }),
       )
     })

packages/redrob/src/plugin/index.ts

@@ -7,6 +7,7 @@ import type {
   WorkspaceAdapter as PluginWorkspaceAdapter,
 } from "@redrob-code/plugin"
 import { Config } from "@/config/config"
+import { pluginShell } from "./shell"
 import { createRedrobClient } from "@redrob-labs/sdk"
 import { ServerAuth } from "@/server/auth"
 import { Session } from "@/session/session"
@@ -156,8 +157,8 @@ const layer = Layer.effect(
           get serverUrl(): URL {
             return Server.url ?? new URL("http://localhost:4096")
           },
-          // @ts-expect-error
-          $: typeof Bun === "undefined" ? undefined : Bun.$,
+          // @ts-expect-error: undefined outside Bun, as before
+          $: pluginShell(),
         }
 
         for (const plugin of flags.disableDefaultPlugins ? [] : internalPlugins()) {

packages/redrob/src/plugin/shell.ts

@@ -0,0 +1,177 @@
+import type { PluginInput } from "@redrob-code/plugin"
+import { Shell } from "@redrob-code/core/shell"
+import { Process } from "@/util/process"
+
+type PluginShell = PluginInput["$"]
+type ShellPromise = ReturnType<PluginShell>
+type ShellOutput = Awaited<ShellPromise>
+type Expression = Parameters<PluginShell>[1]
+
+/**
+ * The `$` handed to plugins.
+ *
+ * On Windows `Bun.$` opens a console window for every command it runs, which flashes on top of the
+ * desktop app each time a plugin checks git or runs a tool. There the plugins get this shell instead:
+ * the same tagged-template API (`await $\`cmd\``, `.text()`, `.json()`, `.lines()`, `.quiet()`,
+ * `.nothrow()`, `.throws()`, `.cwd()`, `.env()`, `$.escape()`, `$.cwd()`, `$.env()`, `$.nothrow()`),
+ * run through the shared spawn helper with `windowsHide`. The command runs in Git Bash when Git for
+ * Windows is installed (the closest to Bun's bash-like syntax), otherwise in cmd.exe. Interpolated values
+ * are quoted for that shell. Output is buffered, never echoed, and `.stdin` is not available.
+ *
+ * Everywhere else plugins keep `Bun.$` itself.
+ */
+export function pluginShell(): PluginShell | undefined {
+  if (process.platform === "win32") return make({ throws: true })
+  return typeof Bun === "undefined" ? undefined : (Bun.$ as unknown as PluginShell)
+}
+
+type Defaults = {
+  cwd?: string
+  env?: Record<string, string | undefined>
+  throws: boolean
+}
+
+type Target = { file: string; posix: boolean }
+
+let target: Target | undefined
+function shellTarget(): Target {
+  if (target) return target
+  const bash = Shell.gitbash()
+  target = bash ? { file: bash, posix: true } : { file: process.env.ComSpec || "cmd.exe", posix: false }
+  return target
+}
+
+function quote(value: string): string {
+  if (value !== "" && /^[\w@%+=:,./\\-]+$/.test(value)) return value
+  if (shellTarget().posix) return `'${value.replace(/'/g, `'\\''`)}'`
+  return `"${value.replace(/"/g, '""')}"`
+}
+
+function render(strings: TemplateStringsArray, expressions: Expression[]): string {
+  let out = strings[0] ?? ""
+  for (let i = 0; i < expressions.length; i++) {
+    out += expression(expressions[i]) + (strings[i + 1] ?? "")
+  }
+  return out
+}
+
+function expression(value: unknown): string {
+  if (Array.isArray(value)) return value.map(expression).join(" ")
+  if (typeof value === "object" && value !== null && "raw" in value && typeof value.raw === "string") return value.raw
+  if (typeof ReadableStream !== "undefined" && value instanceof ReadableStream) {
+    throw new Error("The plugin shell on Windows does not take a stream as input")
+  }
+  return quote(String(value))
+}
+
+function make(defaults: Defaults): PluginShell {
+  const shell = ((strings: TemplateStringsArray, ...expressions: Expression[]) =>
+    command(render(strings, expressions), defaults)) as PluginShell
+  shell.braces = (pattern: string) => {
+    if (typeof Bun !== "undefined") return Bun.$.braces(pattern)
+    return [pattern]
+  }
+  shell.escape = quote
+  shell.env = (env?: Record<string, string | undefined>) => make({ ...defaults, env })
+  shell.cwd = (cwd?: string) => make({ ...defaults, cwd })
+  shell.nothrow = () => make({ ...defaults, throws: false })
+  shell.throws = (throws: boolean) => make({ ...defaults, throws })
+  return shell
+}
+
+function command(text: string, defaults: Defaults): ShellPromise {
+  const state = { ...defaults }
+  let started: Promise<ShellOutput> | undefined
+  const start = () => (started ??= execute(text, state))
+  const self = {
+    get stdin(): WritableStream {
+      throw new Error("The plugin shell on Windows has no stdin")
+    },
+    cwd(cwd: string) {
+      state.cwd = cwd
+      return self
+    },
+    env(env: Record<string, string> | undefined) {
+      state.env = env
+      return self
+    },
+    quiet() {
+      return self
+    },
+    nothrow() {
+      state.throws = false
+      return self
+    },
+    throws(throws: boolean) {
+      state.throws = throws
+      return self
+    },
+    async *lines() {
+      const output = await start()
+      for (const line of output.text().split(/\r?\n/)) {
+        if (line) yield line
+      }
+    },
+    text(encoding?: BufferEncoding) {
+      return start().then((output) => output.text(encoding))
+    },
+    json() {
+      return start().then((output) => output.json())
+    },
+    arrayBuffer() {
+      return start().then((output) => output.arrayBuffer())
+    },
+    blob() {
+      return start().then((output) => output.blob())
+    },
+    then<A = ShellOutput, B = never>(
+      onfulfilled?: ((value: ShellOutput) => A | PromiseLike<A>) | null,
+      onrejected?: ((reason: any) => B | PromiseLike<B>) | null,
+    ) {
+      return start().then(onfulfilled, onrejected)
+    },
+    catch<B = never>(onrejected?: ((reason: any) => B | PromiseLike<B>) | null) {
+      return start().catch(onrejected)
+    },
+    finally(onfinally?: (() => void) | null) {
+      return start().finally(onfinally)
+    },
+    [Symbol.toStringTag]: "ShellPromise",
+  }
+  // Bun.$ runs a command even when nobody awaits it; modifiers chained in the same tick still apply.
+  queueMicrotask(() => void start().catch(() => undefined))
+  return self as unknown as ShellPromise
+}
+
+async function execute(text: string, state: Defaults): Promise<ShellOutput> {
+  const shell = shellTarget()
+  const env = state.env
+    ? Object.fromEntries(Object.entries(state.env).filter((entry): entry is [string, string] => entry[1] !== undefined))
+    : undefined
+  const result = shell.posix
+    ? await Process.run([shell.file, "-c", text], { cwd: state.cwd, env, nothrow: true })
+    : await Process.run([text], { cwd: state.cwd, env, shell: true, nothrow: true })
+  const output = toOutput(result)
+  if (result.code !== 0 && state.throws) {
+    const error = new Error(
+      `Failed with exit code ${result.code}${result.stderr.length ? `\n${result.stderr.toString().trim()}` : ""}`,
+    )
+    error.name = "ShellError"
+    throw Object.assign(error, output)
+  }
+  return output
+}
+
+function toOutput(result: Process.Result): ShellOutput {
+  const stdout = result.stdout
+  return {
+    stdout,
+    stderr: result.stderr,
+    exitCode: result.code,
+    text: (encoding?: BufferEncoding) => stdout.toString(encoding),
+    json: () => JSON.parse(stdout.toString()),
+    arrayBuffer: () => stdout.buffer.slice(stdout.byteOffset, stdout.byteOffset + stdout.byteLength) as ArrayBuffer,
+    bytes: () => new Uint8Array(stdout),
+    blob: () => new Blob([new Uint8Array(stdout)]),
+  }
+}

packages/redrob/src/server/routes/instance/httpapi/groups/permission.ts

@@ -2,6 +2,7 @@ import { PermissionV1 } from "@redrob-code/core/v1/permission"
 import { Permission } from "@/permission"
 import { Schema } from "effect"
 import { HttpApi, HttpApiEndpoint, HttpApiError, HttpApiGroup, OpenApi } from "effect/unstable/httpapi"
+import { SessionID } from "@/session/schema"
 import { PermissionNotFoundError } from "../errors"
 import { Authorization } from "../middleware/authorization"
 import { InstanceContextMiddleware } from "../middleware/instance-context"
@@ -14,6 +15,21 @@ const ReplyPayload = Schema.Struct({
   message: Schema.optional(Schema.String),
 })
 
+/**
+ * A permission raised from outside a local tool call, by an agent runner (Claude Code, Codex) that
+ * runs its own tools. See docs/RUNNER-PROTOCOL.md.
+ */
+export const RunnerAskPayload = Schema.Struct({
+  permission: Schema.String,
+  patterns: Schema.Array(Schema.String),
+  metadata: Schema.optional(Schema.Record(Schema.String, Schema.Unknown)),
+  always: Schema.optional(Schema.Array(Schema.String)),
+})
+export const RunnerAskResult = Schema.Struct({
+  reply: PermissionV1.Reply,
+  message: Schema.optional(Schema.String),
+})
+
 export const PermissionApi = HttpApi.make("permission")
   .add(
     HttpApiGroup.make("permission")
@@ -41,6 +57,20 @@ export const PermissionApi = HttpApi.make("permission")
             description: "Approve or deny a permission request from the AI assistant.",
           }),
         ),
+        HttpApiEndpoint.post("ask", "/session/:sessionID/permission", {
+          params: { sessionID: SessionID },
+          query: WorkspaceRoutingQuery,
+          payload: RunnerAskPayload,
+          success: described(RunnerAskResult, "The person's answer"),
+          error: [HttpApiError.BadRequest],
+        }).annotateMerge(
+          OpenApi.annotations({
+            identifier: "permission.ask",
+            summary: "Ask the person for a permission",
+            description:
+              "Raise a permission request for a session from outside a tool call and wait for the answer. Used by agent runners that run their own tools.",
+          }),
+        ),
       )
       .annotateMerge(
         OpenApi.annotations({