diff --git a/src/content/docs/architecture/context-structure.md b/src/content/docs/architecture/context-structure.md index 40bad3a..2452e0a 100644 --- a/src/content/docs/architecture/context-structure.md +++ b/src/content/docs/architecture/context-structure.md @@ -1,77 +1,77 @@ --- -title: "Context Structure" -description: "How netclaw assembles the model's context each turn — the layers, their order, and how subagents inherit them." +title: "Context structure" +description: "How netclaw assembles the model's context each turn: the layers, their order, and how subagents inherit them." --- -Everything the model sees on a turn is assembled from layers, in a fixed order. Netclaw doesn't hand the model one blob of text — it stacks a persisted system prompt, a frozen block of static context, the conversation history, and a fresh per-turn nudge, then attaches the tool schemas separately on the request. +Everything the model sees on a turn is assembled from layers, in a fixed order. netclaw doesn't hand the model one blob of text. It stacks a persisted system prompt, a block of static context, the conversation history and a fresh per-turn nudge, then attaches the tool schemas separately on the request. -The order is deliberate, and it's built for [prompt caching](https://www.anthropic.com/news/prompt-caching). The stable material sits at the front so the provider can reuse the cached prefix across turns; the volatile, changes-every-turn material sits at the back near the model's generation point. That's why, for example, the current git state is injected late and your persona sits first. +The order is deliberate, and it's built for [prompt caching](https://www.anthropic.com/news/prompt-caching). The stable material sits at the front so the provider can reuse the cached prefix across turns, and the material that changes every turn sits at the back near the model's generation point. That's why the current git state is injected late and your persona sits first. -What each layer contains — and whether it appears at all — depends on the session's [audience](/security/security-model/#trust-audiences). A `Public` turn gets a deliberately thin stack; `Team` and `Personal` get the full one. +What each layer contains, and whether it appears at all, depends on the session's [audience](/security/security-model/#trust-audiences). A `Public` turn gets a deliberately thin stack. `Team` and `Personal` get the full one. ## The layers ```mermaid flowchart TD - subgraph TA["Tier A · Persisted system prompt (cached head)"] - A1["SOUL.md — persona, your name"] --> A2["AGENTS.md — operating rules"] --> A3["TOOLING.md — host capabilities"] --> A4["Project instructions"] - end - subgraph TB2["Tier B · Static context (frozen once at startup)"] - B1["Tool index"] --> B2["Skill index"] --> B3["Memory status"] --> B4["session id + dir"] - end - subgraph TC["Tier C · Conversation history (compacted as it grows)"] - C1["Prior turns"] - end - subgraph TD2["Tier D · Per-turn volatile nudge"] - D1["memory-recall"] --> D2["available-subagents"] --> D3["current-time"] --> D4["working-context — git + recent files"] --> D5["active-background-jobs"] - end - TA --> TB2 --> TC --> TD2 --> MODEL["Model generates the reply"] - TOOLS["Tool schemas — on the request, not prompt text"] -.-> MODEL + A["Tier A, persisted system prompt (cached head): SOUL.md, AGENTS.md, TOOLING.md, project instructions"] + B["Tier B, static context (first call, and again after compaction): tool index, skill index, memory status, session paths"] + C["Tier C, conversation history (compacted as it grows): prior turns, each with its nudge"] + D["Tier D, per-turn nudge (saved into history): memory-recall, skill hint, subagents, time, working-context, background jobs"] + M["Model generates the reply"] + T["Tool schemas, on the request and not in the prompt"] + A --> B --> C --> D --> M + T -.-> M ``` -### Tier A — the persisted system prompt +### Tier A: the persisted system prompt -Built once and stored as the head of the conversation, rebuilt only when its inputs change (byte-identical content is left alone to preserve the cache). Four identity layers, concatenated in order: +Built once and stored as the head of the conversation, then rebuilt only when its inputs change. Byte-identical content is left alone to preserve the cache. Four identity layers, concatenated in order: | Layer | What it is | Source | |-------|-----------|--------| -| `SOUL.md` | Persona, tone, your name | `~/.netclaw/SOUL.md` | -| `AGENTS.md` | Operating rules — the built-in core plus your deployment playbook | embedded in netclaw, plus `~/.netclaw/AGENTS.md` | -| `TOOLING.md` | What the host can actually do | `~/.netclaw/TOOLING.md` | +| `SOUL.md` | Persona, tone, your name | `~/.netclaw/identity/SOUL.md` | +| `AGENTS.md` | Operating rules: the built-in core plus your deployment playbook | embedded in netclaw, plus `~/.netclaw/identity/AGENTS.md` | +| `TOOLING.md` | What the host can actually do | `~/.netclaw/identity/TOOLING.md` | | Project instructions | Repo-specific rules | first of `.netclaw/AGENTS.md`, `CLAUDE.md`, `AGENTS.md`, `CONTEXT.md` in the project directory | -The `AGENTS.md` layer has two halves: an operating core compiled into netclaw (a stripped-down variant on `Public`) and the deployment playbook you write at `~/.netclaw/AGENTS.md`. Project instructions follow the same `AGENTS.md` / `CLAUDE.md` convention other coding tools use, but read from the project directory. +The `AGENTS.md` layer has two halves: an operating core compiled into netclaw (a stripped-down variant on `Public`) and the deployment playbook you write at `~/.netclaw/identity/AGENTS.md`. Project instructions follow the same `AGENTS.md` and `CLAUDE.md` convention other coding tools use, but are read from the project directory. -### Tier B — static context, frozen at startup +### Tier B: static context -Rendered once when the session starts and then held byte-stable, so it never breaks the cache prefix mid-conversation: +Layers that don't change over a session's life. They're rendered on the first model call and again right after a compaction, which resets the context, and otherwise they stay out of the way so they never churn the cached prefix: | Layer | What it is | Empty when | |-------|-----------|------------| -| Tool index | A compressed, audience-filtered catalog of available tools — for awareness, not invocation | — | +| Tool index | A compressed, audience-filtered catalog of available tools, for awareness and not invocation | - | | Skill index | The loaded skill inventory | `Public`, or skills disabled | -| Memory status | Whether the memory subsystem is healthy | `Public`, or memory disabled | -| `[session]` | Session id, plus the session directory for non-`Public` turns | — | -| `[attachment]` hint | How to handle attached files | `file_read` not granted | +| Memory status | Whether the memory subsystem is healthy, and how to use its tools | `Public`, or memory disabled | +| `[session]` | Session id plus five paths: `session_dir`, `temp_dir`, `artifact_dir`, `worktree_dir` and `log_path`, with short guidance on using them | id only on `Public` | +| `[attachments]` hint | How to handle attached files | `file_read` not granted | -The tool index is awareness only. The actual tool **schemas** ride on the request itself (as the provider's tool parameter), resolved and audience-filtered every turn — so a tool has two surfaces: a compressed name in the prompt for discovery, and its full schema on the wire. +The tool index is awareness only. The actual tool **schemas** ride on the request itself (as the provider's tool parameter), resolved and audience-filtered every turn. A tool has two surfaces: a compressed name in the prompt for discovery, and its full schema on the wire. -### Tier C — conversation history +### Tier C: conversation history -The turns so far, verbatim. As it grows past the model's budget it gets [compacted](/architecture/sessions/#compaction) into a summary. The durable working context (below) survives compaction; the raw transcript doesn't. +The turns so far, verbatim. When it grows past the budget it gets [compacted](/architecture/sessions/#compaction): old tool output is cleared, older messages are dropped, and a short block of compaction notes about what was dropped takes their place. The durable working context (below) survives compaction. The raw transcript doesn't. -### Tier D — the per-turn volatile nudge +### Tier D: the per-turn nudge -This is the freshest layer, rebuilt every turn. One structural quirk worth knowing: it isn't a trailing system message. Netclaw injects it as a `[system: …]` block wrapped in a **user-role message, placed just before your latest message** — which keeps it inside the cacheable history prefix and sidesteps the "a fresh user turn just started" confusion some models have with trailing system text. Order within it: +This is the freshest layer, rebuilt every turn. One quirk is worth knowing: it isn't a trailing system message. netclaw wraps it in a `[system: ...]` block inside a **user-role message placed just before your latest message** and saves it into history. That keeps it inside the cacheable prefix on later turns, and it sidesteps the "a fresh user turn just started" confusion some models have with trailing system text. The order inside it: | Layer | What it is | Suppressed on `Public` | |-------|-----------|:----------------------:| -| `[memory-recall]` | Memories recalled for this turn | — | -| Skill hint | A nudge toward a relevant skill | — | +| `[memory-recall]` | Memories recalled for this turn, if any cleared the relevance gate | ✓ | +| Skill hint | A nudge toward a relevant skill | - | | `[available-subagents]` | Subagents this session can spawn | ✓ | -| `[current-time]` | UTC, local, day, timezone | — | -| `[working-context]` | Git state + project dir + recent files | ✓ | -| `[active-background-jobs]` | Running background work | ✓ | +| `[current-time]` | UTC, local, day, timezone | - | +| `[working-context]` | Shell, git state, project dir and recent files | ✓ | +| `[active-background-jobs]` | Running or reaped background work | ✓ | +| Slash-command skill content | The body of a skill invoked with `/name` | - | +| Session prompt overlay | A per-session instruction added on top | - | +| Turn restart notice | Tells the model the daemon restarted and it's resuming | - | +| Interrupted-automation note | Carried by the first model call after a person ends an automation turn: the automated turn stopped unfinished because a person sent a message. Later turns don't carry it | - | + +Most turns carry no `[memory-recall]` block. The relevance gate drops weak matches, and when nothing survives the block is omitted. See the [memory model](/architecture/memory-model/#how-recall-works). ## Audience scoping @@ -80,67 +80,73 @@ This is the freshest layer, rebuilt every turn. One structural quirk worth knowi | | `Public` | `Team` / `Personal` | |---|:---:|:---:| | SOUL.md, AGENTS.md core, tool index, `[current-time]` | ✓ (public AGENTS variant) | ✓ | -| TOOLING.md, project instructions | — | ✓ | -| Skill index, memory, subagents, working-context, background jobs | — | ✓ | -| `[session]` | id only | id + directory | +| TOOLING.md, project instructions | - | ✓ | +| Skill index, memory, subagents, working-context, background jobs | - | ✓ | +| `[session]` | id only | id and paths | ## Git working context -On `Team` and `Personal` turns where a project directory is set, netclaw derives a live snapshot of your git state and drops it into the volatile nudge as a `[working-context]` block. It's how the model knows what branch you're on and what you've been touching without you spelling it out. +On `Team` and `Personal` turns where a project directory is set, netclaw derives a live snapshot of your git state and drops it into the nudge as a `[working-context]` block. It's how the model knows what branch you're on and what you've been touching without you spelling it out. -It's computed fresh at the start of every turn by shelling out to `git` — so `git` has to be on the daemon's PATH — under a hard **2-second budget**. Nothing is persisted; the snapshot is rebuilt each turn, and the durable half (project directory and recent files) is what survives compaction and restarts. +It's computed fresh at the start of every turn by shelling out to `git`, so `git` has to be on the daemon's PATH, under a hard **2-second budget**. Nothing is persisted. The snapshot is rebuilt each turn, and the durable half (project directory and recent files) is what survives compaction and restarts. -What the block actually contains: +What the block contains: -- `project_dir` — set via the `set_working_directory` tool. -- `recent_files` — the last 10 files read or written, most-recent first. -- `git:` — `worktree`, `branch` (or `(detached)`), `head`, and `upstream` / `ahead` / `behind` **only when the branch tracks an upstream**, then **counts** of staged, modified, and untracked files. +- `shell:` on `Personal` turns only: `platform`, `executable`, `grammar` and, for PowerShell, `dialect`. It tells the model which shell syntax to write. +- `project_dir`, set with the `set_working_directory` tool. +- `recent_files`: the last 10 files read or written, most recent first. +- `git:` with `worktree`, `common_dir`, `branch` (or `(detached)`), `head`, and `upstream`, `ahead` and `behind` **only when the branch tracks an upstream**, then **counts** of staged, modified and untracked files. :::caution Two things the block does **not** do, despite how it's easy to read the feature: -- **Changed files appear as counts, not names.** The model sees "3 modified", not which three. The individual paths are tracked internally (for subagent handoff) but never rendered into the prompt. -- **There's no per-turn config.** It's fully automatic — no on/off switch, no tunable limits. The 2-second budget, the 256 KB output ceiling, and the 10-file recent list are all fixed. +- **Changed files appear as counts, not names.** The model sees "3 modified", not which three. The paths are tracked internally for subagent handoff but never rendered into the prompt. +- **There's no per-turn config.** It's fully automatic, with no on/off switch and no tunable limits. The 2-second budget, the 256 KB output ceiling and the 10-file recent list are all fixed. ::: -If git isn't reachable, the run is slow, or the directory isn't a repo, the block degrades to a short `unavailable` note with a reason instead of failing the turn. +If git isn't reachable, the run is slow, or the directory isn't a repo, the block degrades to a short `unavailable` note with a reason (for example `git executable not found`) instead of failing the turn. ## Subagent context -A [subagent](/guides/custom-subagents/) doesn't get a copy of the parent's context. It runs a much thinner stack, assembled fresh for a one-shot task: +A [subagent](/skills/custom-subagents/) doesn't get a copy of the parent's context. It runs a much thinner stack, assembled fresh for a one-shot task: ``` AGENTS.md operating rules + project instructions + the subagent's own markdown body + + a compressed tool index + a headless execution contract ``` +Its first user message carries a `Context: ... Task: ...` pair, the working-context block, and its own `[session]` block. + What a subagent **inherits** from its parent at spawn time: -- The `AGENTS.md` operating rules and project instructions (audience-appropriate) — so a child plays by the same house rules as the parent. -- The project directory, the recent-files list, and the **same working directory** — there's no worktree isolation; a subagent edits the same files the parent would. -- The parent's [audience](/security/security-model/#trust-audiences), and a tool policy filtered to it — **minus `spawn_agent`**, so subagents can't recursively spawn. +- The `AGENTS.md` operating rules and project instructions (audience-appropriate), so a child plays by the same house rules as the parent. +- The project directory, the recent-files list, and the **same working directory**. A subagent edits the same files the parent would. +- The parent's [audience](/security/security-model/#trust-audiences) and a tool policy filtered to it, **minus `spawn_agent`, `check_agent_run` and `attach_file`**, so subagents can't spawn their own. - An initial git snapshot taken at spawn. -What it **doesn't** get: your SOUL.md persona, TOOLING.md, memory recall, the skill or memory indexes, subagent discovery, the `[current-time]` layer, background jobs, or the parent's conversation history. Its entire user message is `Context: … Task: …`. +Each child gets its own `temp_dir`, `artifact_dir` and `log_path`. On Linux and macOS the child's `temp_dir` is `/tmp/netclaw-//c<8 hex>`, and the other two sit under the parent's session folder. A successful `spawn_agent` call returns only the run id and `"state":"accepted"`. When the child ends, a notice names its `result.md`, which lists the log path and the artifact directory, and the parent reads what the child produced with `file_read`. + +What it **doesn't** get: your SOUL.md persona, TOOLING.md, memory recall, the skill and memory indexes, subagent discovery, `[current-time]`, background jobs, or the parent's conversation history. -When a subagent finishes, it hands back a working-context delta. The parent merges **only the files a successful child confirmed it wrote or edited** into its own recent-files list — a failed or cancelled run merges nothing, and files the child only read aren't merged. This is an in-memory recent-files handoff, not a git merge: since parent and child share one working directory, the edits are already on disk. The merge just keeps the parent's sense of "what we've been touching" current. +When a subagent finishes, it hands back a working-context delta. The parent merges **only the files a successful child confirmed it wrote or edited** into its own recent-files list. A failed or cancelled run merges nothing, and files the child only read aren't merged. This is an in-memory recent-files handoff and not a git merge: since parent and child share one working directory, the edits are already on disk. The merge just keeps the parent's sense of "what we've been touching" current. ## Limitations -- The git working context needs `git` on the daemon's PATH and a project directory set; without both, the `[working-context]` block is absent, not an error. -- Changed-file **names** are never exposed to the model — only counts, and only the recent-files list carries specific paths. -- None of the layer sizes are configurable. Recent files cap at 10, git inspection at a 2-second budget. -- Subagents share the parent's working directory. There is no filesystem sandbox between a parent and its subagent beyond the audience's tool policy. +- The git working context needs a project directory. Without `git` on the daemon's PATH the block is present with an `unavailable` note, and without a project directory (on `Team`) it's absent. +- Changed-file **names** are never exposed to the model. Only counts, and the recent-files list, carry paths. +- None of the layer sizes are configurable. Recent files cap at 10 and git inspection at a 2-second budget. +- Subagents share the parent's working directory. Nothing isolates a parent from its subagent beyond the audience's tool policy. Every session also has a managed `worktree_dir`, and the agent can create a git worktree there with `git worktree add` and switch into it with `set_working_directory`, but that's something it chooses to do, not a default. ## Related pages -- [Sessions & Input Model](/architecture/sessions/) — the turn lifecycle these layers feed, and compaction -- [Memory Model](/architecture/memory-model/) — how the memory layers (recall, index) are formed and scoped -- [Custom Subagents](/guides/custom-subagents/) — defining the subagents whose context this page describes -- [Security Model](/security/security-model/#trust-audiences) — the audiences that gate each layer +- [Sessions and input model](/architecture/sessions/) - the turn lifecycle these layers feed, and compaction +- [Memory model](/architecture/memory-model/) - how the memory layers (recall, index) are formed and scoped +- [Custom subagents](/skills/custom-subagents/) - defining the subagents whose context this page describes +- [Security model](/security/security-model/#trust-audiences) - the audiences that gate each layer ## Resources -- [Prompt caching (Anthropic)](https://www.anthropic.com/news/prompt-caching) — why the stable-prefix ordering matters -- [AGENTS.md](https://agents.md/) — the operating-rules file convention netclaw follows for project instructions +- [Prompt caching (Anthropic)](https://www.anthropic.com/news/prompt-caching) - why the stable-prefix ordering matters +- [AGENTS.md](https://agents.md/) - the operating-rules file convention netclaw follows for project instructions diff --git a/src/content/docs/architecture/design-philosophy.md b/src/content/docs/architecture/design-philosophy.md index cda913f..e7d1746 100644 --- a/src/content/docs/architecture/design-philosophy.md +++ b/src/content/docs/architecture/design-philosophy.md @@ -1,85 +1,89 @@ --- -title: "Design Philosophy" +title: "Design philosophy" description: "Why netclaw exists, what we believe about agents, and the tradeoffs we chose." --- -Netclaw exists because we disagreed with the direction every other open-source agent project was heading. [OpenClaw](https://openclaw.ai/) (the most popular open-source personal AI assistant, 360K+ GitHub stars, dozens of platform integrations) and [Hermes Agent](https://hermes-agent.nousresearch.com/) (Nous Research's autonomous agent with a self-improving skill system) proved the category and inspired this project. We respect the hell out of what they've built. But we think their philosophies are bad for their projects long-term, and that's why we built something new. +We built netclaw because we disagreed with the direction every other open-source agent project was heading. [OpenClaw](https://openclaw.ai/) (the most popular open-source personal AI assistant, 360K+ GitHub stars, dozens of platform integrations) and [Hermes Agent](https://hermes-agent.nousresearch.com/) (Nous Research's autonomous agent with a self-improving skill system) proved the category and inspired this project. We respect the hell out of what they've built. But we think their philosophies are bad for their projects long-term, and that's why we built something new. ## Simplicity is a feature -[Petabridge](https://petabridge.com/) has maintained [Akka.NET](https://getakka.net/) since 2014. Over a decade of shepherding a large open-source framework through breaking changes, security patches, and community demands. The lesson: **the biggest threat to any project is its own complexity.** +[Petabridge](https://petabridge.com/) has maintained [Akka.NET](https://getakka.net/) since 2014. A decade of shepherding a large open-source framework through breaking changes, security patches, and community demands taught us one thing: **the biggest threat to any project is its own complexity.** -Every feature you add is a feature you have to maintain, test, secure, and explain. The most productive thing a maintainer can do is say no. Most agent projects treat feature count and velocity as proof of progress — hundreds of PRs per release, thousands of community integrations, autonomous skill lifecycle management. We think that's antithetical to how you should build software. +Every feature you add is a feature you have to maintain, test, secure, and explain. The most productive thing a maintainer can do is say no. Most agent projects treat feature count and velocity as proof of progress: hundreds of PRs per release, thousands of community integrations, autonomous skill lifecycle management. We think that's antithetical to how you should build software. -Netclaw's codebase is deliberately small. So is the configuration footprint. When we're tempted to add a feature, the first question is always: can this live in an MCP server someone else maintains instead? +The codebase is deliberately small. So is the configuration footprint. When we're tempted to add a feature, the first question is always: can this live in an MCP server someone else maintains instead? ### What we don't build -You'll probably never see voice mode in netclaw. That's a property of the model you're using and something your operating system will handle natively. We don't build web dashboards for daily interaction — your chat app already has auth, threading, mobile apps, notifications, and search. We don't build an autonomous skill lifecycle that generates, archives, and self-improves skills without human oversight. +You'll probably never see voice mode in netclaw. That's a property of the model you're using and something your operating system will handle natively. We don't build web dashboards for daily interaction. Your chat app already has auth, threading, mobile apps, notifications, and search. We don't build an autonomous skill lifecycle that generates, scores, archives, and rewrites skills without human oversight. The agent has a tool to create and edit skills, but nothing in netclaw curates them behind your back. -Every feature we don't build is a feature that can't break, can't have a CVE, and can't confuse your deployment. +A feature we never build can't break, can't have a CVE, and can't confuse your deployment. ## Chat-first interaction -Your chat app is the interface. Slack, Discord, Mattermost — whatever your team already uses. Every thread you spin up with the agent is its own session: reentrant (pick up where you left off), persisted to disk, and recoverable across daemon restarts. The agent doesn't need its own UI because it lives where your team already works. +Your chat app is the interface. Slack, Discord, Mattermost, whatever your team already uses. Every thread you spin up with the agent is its own session: reentrant (pick up where you left off), persisted to disk, and recoverable across daemon restarts. The agent doesn't need its own UI because it lives where your team already works. -At the session layer, the source of a message is irrelevant. Slack, a webhook, a scheduled timer, a CLI chat — they all produce the same kind of message routed to the same kind of [actor](https://en.wikipedia.org/wiki/Actor_model). The [architecture overview](/architecture/overview/#everything-is-just-input) covers how this works mechanically, but the philosophy is simple: the agent shouldn't care how you talk to it, and you shouldn't have to learn a new interface to use it. +At the session layer, the source of a message is irrelevant. Slack, a webhook, a scheduled timer, a CLI chat: they all produce the same kind of message routed to the same kind of [actor](https://en.wikipedia.org/wiki/Actor_model). The [architecture overview](/architecture/overview/#everything-is-just-input) covers how this works mechanically, but the philosophy is simple. The agent shouldn't care how you talk to it, and you shouldn't have to learn a new interface to use it. -This means adding a new channel — Mattermost, Signal, a web UI for remote management — never touches session logic. The codebase stays small because the transport layer is someone else's problem. +This means adding a new channel, whether Signal or a web UI for remote management, never touches session logic. The codebase stays small because the transport layer is someone else's problem. ## Default-deny security -Giving an LLM shell access and tool-calling ability is giving it the keys to your infrastructure. The question isn't whether something will go wrong — it's what happens when it does. +Giving an LLM shell access and tool-calling ability is giving it the keys to your infrastructure. The question isn't whether something will go wrong, it's what happens when it does. -Most agent frameworks start permissive and add restrictions after security incidents. Netclaw works the other way around. During [`netclaw init`](/cli/init/), you choose an [audience disposition](/architecture/security-model/#audience-as-the-trust-primitive) — Personal, Team, or Public — and that choice sets the baseline for what the agent can do. Some common operations are allowed out of the box, but anything potentially destructive requires explicit approval. You can approve a tool call for the current session or persistently, and the agent makes the distinction clear with a structured approval prompt in the TUI. +Most agent frameworks start permissive and add restrictions after security incidents. This project works the other way around. During [`netclaw init`](/cli/init/), you choose an [audience](/architecture/security-model/#audience-as-the-trust-primitive) (Personal, Team, or Public) and that choice sets the baseline for what the agent can do. Some common operations are allowed by default, but anything potentially destructive requires explicit approval. You can approve a tool call for the current session or persistently, and the agent makes the distinction clear with a structured approval prompt in the TUI or in your chat channel. -![Security posture selection during netclaw init](/screenshots/output/init-02-security-posture.png) +![Netclaw Setup, Step 3 of 5, Security Posture, offering Personal, Team and Public](/screenshots/output/init-02-security-posture.png) + +The choice is made once, in the third step of setup. This is the [principle of least privilege](https://csrc.nist.gov/glossary/term/least_privilege) applied to an LLM agent. -When something fails or is misconfigured, it fails loudly. Fallbacks hide bugs, and on security-relevant paths they can silently escalate privileges. Inbound message checks run in the [gateway boundary](/architecture/overview/#three-boundaries) before messages ever reach the session layer. And the security model stacks four independent layers — hard denies, path restrictions, audience-scoped grants, and human approval gates — so no single check is the only thing between the agent and a destructive action. The [security architecture](/architecture/security-model/) covers the full stack. +When something fails or is misconfigured, it fails loudly. Fallbacks hide bugs, and on security-relevant paths they can silently escalate privileges. Inbound message checks run in the [gateway boundary](/architecture/overview/#three-boundaries) before messages ever reach the session layer. The security model stacks four independent layers (hard denies, path restrictions, audience-scoped grants, and human approval gates) so no single check is the only thing between the agent and a destructive action. The [security architecture](/architecture/security-model/) covers the full stack. ### Determinism boundaries Traditional service architecture thinks in terms of service boundaries. Agent architecture needs to think in terms of **determinism boundaries**: where is it safe for an LLM to operate non-deterministically, and where isn't it? -LLMs are inherently non-deterministic — same input, different output, every time. That's fine for research, summarization, and conversation. It's not fine for `git push --force` or deleting a database. +LLMs are inherently non-deterministic: same input, different output, every time. That's fine for research, summarization, and conversation. It's not fine for `git push --force` or deleting a database. -The operating principle is **reversibility**: if the agent is about to perform an action that can't be undone, that action needs a deterministic gate — an [approval prompt](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch) where a human confirms intent. If you want to turn those gates off, that's your decision. You live with the consequences. +The operating principle is **reversibility**. If the agent is about to perform an action that can't be undone, that action needs a deterministic gate, an [approval prompt](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch) where a human confirms intent. If you want to [turn those gates off](/guides/reduce-approval-prompts/), that's your decision. You live with the consequences. ## Lightweight access control without an identity provider -The real-world use case that drove this: a bot in a team Slack channel that can answer questions about live service status for anyone, but only operators — the IT folks — can trigger high-privilege actions like exporting log dumps or restarting services. +The real-world use case that drove this: a bot in a team Slack channel that can answer questions about live service status for anyone, but only operators (the IT folks) can trigger high-privilege actions like exporting log dumps or restarting services. -Traditional role-based access control solves this, but it requires a fully managed identity provider — something most self-hosted deployments don't have and shouldn't need. To keep netclaw simple and self-hostable, we distilled two lightweight primitives that are adjacent to RBAC without the infrastructure dependency: +Traditional role-based access control solves this, but it requires a fully managed identity provider, something most self-hosted deployments don't have and shouldn't need. To keep netclaw simple and self-hostable, we distilled two lightweight primitives that are adjacent to RBAC without the infrastructure dependency: -1. **Audience dispositions** classify trust by deployment topology and channel type. Personal (you at your machine), Team (your Slack workspace), or Public (untrusted sources). The disposition determines what tools are visible, what memory is accessible, and whether approval gates are active. +1. **Audience dispositions** classify trust by deployment topology and channel type. Personal (you at your machine), Team (your Slack, Discord or Mattermost workspace), or Public (untrusted sources). The disposition determines what tools are visible, what memory is accessible, and whether approval gates are active. 2. **Listing owners** are the specific user IDs in each channel who actually operate the bot. They get elevated trust within their audience tier. | Audience | Trust | Typical source | |----------|-------|----------------| | **Personal** | Full access | Local TUI, operator at the machine | -| **Team** | Scoped access | Slack workspace, Discord server | +| **Team** | Scoped access | Slack, Discord or Mattermost workspace | | **Public** | Minimal access | Unknown or untrusted channels | -During [`netclaw init`](/cli/init/), you assign each connected channel to an audience tier. The same Slack workspace can have DMs classified as Personal and a shared ops channel classified as Team: +In [`netclaw config`](/cli/config/), you assign each connected channel to an audience tier. The same Slack workspace can have DMs classified as Personal and a shared ops channel classified as Team: + +![netclaw config Channels & Permissions screen for Mattermost, listing the Assistant and Ops Room channels as Team, Lunch as Public and direct messages as Personal](/screenshots/output/config-channel-audiences.png) -![Per-channel audience configuration during netclaw init](/screenshots/output/init-04-channel-audiences.png) +The screen is Mattermost's, and Slack and Discord use the same layout. Left and right set a channel's audience, and a public lunch channel can sit next to a team ops room. -This isn't a better idea than RBAC. It's a pragmatic simplification that lets you deploy an agent with meaningful access control today, on your own hardware, without standing up Entra or Okta first. Every other security decision flows from the audience disposition. See [audience scoping](/architecture/security-model/#audience-as-the-trust-primitive) for how it works in practice. +We're not claiming this beats RBAC. It's a pragmatic simplification that lets you deploy an agent with meaningful access control today, on your own hardware, without standing up Entra or Okta first. Every other security decision flows from the audience. See [audience scoping](/architecture/security-model/#audience-as-the-trust-primitive) for how it works in practice. ## MCP-first tooling -LLMs calling command-line tools is powerful and composable — especially when those tools are in the model's training data or expose their own help documentation. But shell access is where all the security problems come from. Agents deleting production databases, sending unauthorized emails, running destructive commands — that's shell territory. +LLMs calling command-line tools is powerful and composable, especially when those tools are in the model's training data or expose their own help documentation. But shell access is where the security problems come from. Agents deleting production databases, sending unauthorized emails, running destructive commands: that's shell territory. -For operators and developers, shell access makes sense. Netclaw supports it with strong [approval gates](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch) that extract verb-chains from commands (so approving `git push` covers `git push origin main`) and make shell access as safe as possible. +For operators and developers, shell access makes sense. netclaw supports it with strong [approval gates](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch) that extract verb-chains from commands (so approving `git push` covers `git push origin main`) and make shell access as safe as possible. -But for everyone else in an organization? Shell access is a non-starter. The accounting team isn't going to install the Google Cloud CLI on a sandboxed VM over SSH. That's unrealistic. +But for everyone else in an organization? Shell access is a non-starter. The accounting team isn't going to install the Google Cloud CLI on a sandboxed VM over SSH, and shouldn't have to. -[MCP](https://modelcontextprotocol.io/) solves this. It lets you deploy agents that connect to remote tool servers with scoped permissions — read-only access to your CRM, delegated service accounts for cloud resources, controlled access to internal APIs. The agent gets purpose-built tools without shell exposure, and every tool call goes through the same [approval gates](/architecture/security-model/#mcp-security-two-gate-model) as shell commands. +[MCP](https://modelcontextprotocol.io/) solves this. It lets you deploy agents that connect to remote tool servers with scoped permissions: read-only access to your CRM, delegated service accounts for cloud resources, controlled access to internal APIs. The agent gets purpose-built tools without shell exposure, and every tool call goes through the same [approval gates](/architecture/security-model/#mcp-security-two-gate-model) as shell commands. -What makes MCP work at scale is [progressive tool disclosure](/architecture/overview/#tool-system). The agent sees server-level summaries — "browser tools," "memory tools," "CRM tools" — and loads specific tools on demand. This keeps the prompt small and prevents the model from misfiring on similarly-named tools across servers. +What makes MCP work at scale is [progressive tool disclosure](/architecture/overview/#tool-system). The agent sees server-level summaries (browser tools, CRM tools, and so on) and loads specific tools on demand. This keeps the prompt small and prevents the model from misfiring on similarly named tools across servers. If an MCP server definition changes and new tools appear, operators can set the default approval status for unknown tools: auto-deny, auto-approve, or require human confirmation. MCP provides a deterministic security boundary that shell calling fundamentally cannot. @@ -91,44 +95,46 @@ Most agent platforms assume frontier models. OpenClaw's documentation recommends We think that's the wrong default. If your agent is working deals in a CRM, responding to GitHub webhooks, generating invoices, or triaging support tickets, you don't need a frontier model at $10 per million input tokens. -Netclaw runs production workloads on [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B). We've tested with models as small as Qwen 3.5 9B and gotten reliable tool-calling results. The architecture is designed to make this work: +netclaw runs production workloads on [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B). We've tested with models as small as Qwen 3.5 9B and gotten reliable tool-calling results. The architecture is designed to make this work: -Progressive tool disclosure reduces the noise in the prompt — what the model has to sort through to pick the right tool — so smaller models can navigate tool graphs without misfiring. Compressed skill and tool indexes keep the system prompt lean, and server-first MCP discovery means the model picks "which server" before it ever sees individual tool definitions. Much simpler decision space. +Progressive tool disclosure cuts the noise in the prompt, so smaller models can navigate tool graphs without misfiring. Compressed skill and tool indexes keep the system prompt lean, and server-first MCP discovery means the model picks "which server" before it ever sees individual tool definitions. That's a much simpler decision space. -There's an economic argument here too. A $3,000 GPU with 64GB VRAM runs local inference indefinitely — bounded capital expenditure instead of unbounded API billing. Local inference also means your data never leaves your infrastructure, which matters when the agent handles contracts, financials, or internal communications. +There's an economic argument here too. A $3,000 GPU with 64GB VRAM runs local inference indefinitely: bounded capital expenditure instead of unbounded API billing. Local inference also means your data never leaves your infrastructure, which matters when the agent handles contracts, financials, or internal communications. [Self-hosted providers](/configuration/self-hosted-providers/) covers the setup. -For tasks that genuinely need frontier capabilities — complex agentic coding, multi-step research across large codebases — netclaw delegates to tools like [Claude Code](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview) or [OpenCode](https://opencode.ai/) in headless mode via skills. Use your existing subscriptions for frontier work. Use local models for everything else. +For tasks that genuinely need frontier capabilities, like complex agentic coding or multi-step research across large codebases, netclaw delegates to tools like [Claude Code](https://docs.anthropic.com/en/docs/agents-and-tools/claude-code/overview) or [OpenCode](https://opencode.ai/) in headless mode via skills. Use your existing subscriptions for frontier work. Use local models for everything else. ## Automatic memory formation Agents that forget everything between conversations aren't useful for real work. But agents that require users to manually save context don't get used either. -Netclaw forms memories automatically. A per-session observation sidecar watches the conversation and, when the session goes idle, distills key facts, decisions, and project context into durable memory — without an extra LLM call during your turn. A background curation pipeline validates proposals, deduplicates against existing memories, blocks secret content, and persists what survives to SQLite. +netclaw forms memories automatically. A per-session memory observer watches the conversation and, when the session goes idle or every few turns, distills key facts, decisions, and project context into durable memory. That's a background call that never sits inside your turn. A curation pipeline validates proposals, deduplicates against existing memories, blocks secret content, and persists what survives to SQLite. + +Before each turn, in any session, netclaw recalls the memories that bear on what you just said. It combines full-text search with semantic search that runs on your own CPU, on two small ONNX models (on by default since 0.27.0). A relevance gate throws out weak matches, so most turns inject nothing, and what does get through arrives as context the model sees before it starts generating, not as a tool call. There's no external vector database, no cloud embedding service, and no third party in the recall path. The one network touch is a model download from huggingface.co on first start, and you can [pre-provision the files](/configuration/memory/#run-offline) to skip it. The agent remembers that your project uses PostgreSQL, that you prefer Terraform over Pulumi, that the deploy target changed last Tuesday, because it observed those facts and stored them. -On the next turn — in any session — relevant memories are recalled automatically via deterministic full-text search and injected into the system prompt. No vector database, no embeddings, no external service. The agent remembers that your project uses PostgreSQL, that you prefer Terraform over Pulumi, that the deploy target changed last Tuesday — because it observed those facts and stored them. +The cost is RAM. With embeddings on, the daemon sits around 480 MB at idle instead of about 235 MB, and you can [turn them off](/configuration/memory/#turn-embeddings-off) if that matters more to you than semantic recall. -This is the kind of feature that belongs in core, not in a plugin. Memory touches the session lifecycle, the compaction pipeline, the security model (public sessions get no memory access), and the system prompt assembly. Pushing it to the edges would mean every integration reimplements it badly. The [memory architecture](/architecture/memory-model/) covers the full system. +This is the kind of feature that belongs in core, not in a plugin. Memory touches the session lifecycle, the compaction pipeline, the security model (public sessions get no memory access), and context assembly. Pushing it to the edges would mean every integration reimplements it badly. The [memory architecture](/architecture/memory-model/) covers the full system. ## The philosophy of no Open-source maintainers tend to say yes too much. Every accepted contribution is a maintenance commitment. Every feature that lands in core is a feature that can't be pushed to a plugin, an MCP server, or a skill that someone else maintains. -Netclaw takes the opposite approach: functionality belongs at the edges unless there's a strong reason to centralize it. Want voice mode? That's a model capability and an OS integration — not an agent framework feature. Want a custom skill marketplace? Run your own [skill feed](/skills/skill-feeds/) or [skill server](/skills/skill-server/). Want a specialized tool for your workflow? Build an MCP server and connect it. +netclaw takes the opposite approach: functionality belongs at the edges unless there's a strong reason to centralize it. Want voice mode? That's a model capability and an OS integration, not an agent framework feature. Want a custom skill marketplace? Run your own [skill feed](/skills/skill-feeds/) or [skill server](/skills/skill-server/). Want a specialized tool for your workflow? Build an MCP server and connect it. The core stays small, and so does the surface area for bugs, CVEs, and breaking changes. When your competitors' complexity is the source of their security vulnerabilities, your simplicity is your security story. -## Keep reading +## Related pages -- [Architecture Overview](/architecture/overview/) — how these principles become a running system -- [Security Architecture](/architecture/security-model/) — the four-layer security model in detail -- [Quickstart](/getting-started/quickstart/) — try it yourself -- [MCP Tool Permissions](/guides/mcp-tool-permissions/) — practical guide to scoping MCP access -- [Self-Hosted Providers](/configuration/self-hosted-providers/) — setting up local model inference +- [Architecture overview](/architecture/overview/) - how these principles become a running system +- [Security architecture](/architecture/security-model/) - the four-layer security model in detail +- [Memory model](/architecture/memory-model/) - the full memory system behind the section above +- [Quickstart](/getting-started/quickstart/) - try it yourself +- [MCP tool permissions](/guides/mcp-tool-permissions/) - practical guide to scoping MCP access ## Resources -- [Actor model](https://en.wikipedia.org/wiki/Actor_model) — the concurrency model behind netclaw's session architecture -- [Akka.NET](https://getakka.net/) — the actor framework that implements it, maintained by Petabridge since 2014 -- [Model Context Protocol specification](https://modelcontextprotocol.io/) — the open standard for agent tool integration -- [Principle of Least Privilege (NIST)](https://csrc.nist.gov/glossary/term/least_privilege) — the access control philosophy behind default-deny -- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) — the attack taxonomy netclaw's security model is designed against +- [Actor model](https://en.wikipedia.org/wiki/Actor_model) - the concurrency model behind netclaw's session architecture +- [Akka.NET](https://getakka.net/) - the actor framework that implements it, maintained by Petabridge since 2014 +- [Model Context Protocol specification](https://modelcontextprotocol.io/) - the open standard for agent tool integration +- [Principle of Least Privilege (NIST)](https://csrc.nist.gov/glossary/term/least_privilege) - the access control philosophy behind default-deny +- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) - the attack taxonomy netclaw's security model is designed against diff --git a/src/content/docs/architecture/memory-model.md b/src/content/docs/architecture/memory-model.md index 7e43c86..8151279 100644 --- a/src/content/docs/architecture/memory-model.md +++ b/src/content/docs/architecture/memory-model.md @@ -1,9 +1,37 @@ --- -title: "Memory Model" -description: "How netclaw forms, stores, and recalls memories across sessions without manual effort." +title: "Memory model" +description: "How netclaw forms, curates and recalls memories across sessions, with local semantic search on by default." --- -Agents that forget everything between conversations aren't useful for real work. Agents that require users to manually tag and save context don't get used. Netclaw's memory system sits between these extremes: it forms memories automatically from conversation, stores them durably in SQLite, and recalls relevant ones before each turn — no manual effort, no external services. If you're not familiar with how [sessions](/architecture/sessions/) work, start there. +Agents that forget everything between conversations aren't useful for real work, and agents that make you tag and save context don't get used. netclaw sits between the two. A background memory observer watches each conversation, a curation pipeline decides what's worth keeping, and before every turn a recall step injects the few memories that actually bear on what you just said. Everything lives in one SQLite file, and since 0.27.0 the semantic search runs on your own CPU with two small ONNX models. If you're not familiar with how [sessions](/architecture/sessions/) work, start there. + +The mental model is two paths that share one store. The write path is slow, asynchronous and uses an LLM sparingly. The read path is fast, local and gated hard, so most turns inject nothing at all. + +Writing happens in the background. The memory observer's LLM call fires after 90 seconds idle, every 5 turns, and at passivation. + +```mermaid +flowchart TD + T["Session transcript"] --> O["Memory observer
(LLM call)"] + O --> Q[("Checkpoint queue")] + X["store_memory tool"] --> Q + K["Compaction notes"] --> Q + Q --> C["Curate and embed
locally"] + C --> S[("Memory store")] + C -->|"near-duplicate
or ambiguous"| L["Curation model call"] + L --> S +``` + +Reading happens on every turn, inside a 300 ms budget: + +```mermaid +flowchart TD + M["Your message"] --> F["Full-text
candidates"] + M --> V["Embed query,
top 50 by cosine"] + F --> R["Fuse, recency decay,
similarity floor"] + V --> R + R --> G["Cross-encoder
relevance gate"] + G --> I["Up to 3 memories
into the turn"] +``` ## What memory looks like in practice @@ -21,15 +49,15 @@ netclaw: Here are some United flights from Houston Intercontinental to Amsterdam departing October 1–7... ``` -After this conversation ends, the memory sidecar (a companion actor running alongside the session) observes two things worth remembering: +Once the conversation goes quiet, the memory observer (a companion actor running alongside the session) proposes three things worth keeping: | What it stores | Memory class | Why | |----------------|-------------|-----| -| You're a United MileagePlus member | **Durable fact** | Airline preference — useful indefinitely | -| Your usual origin airport is IAH | **Durable fact** | Travel pattern — useful indefinitely | -| You need a flight to Amsterdam in early October | **Evidence** | Specific task — expires after 30 days | +| You're a United MileagePlus member | Durable fact | Airline preference, useful indefinitely | +| Your usual origin airport is IAH | Durable fact | Travel pattern, useful indefinitely | +| You need a flight to Amsterdam in early October | Evidence | Specific task, expires after 30 days | -The agent didn't ask you to save any of this. It observed, classified, and stored it. +You didn't ask it to save any of this. ### Conversation 2: A week later @@ -41,11 +69,11 @@ netclaw: Here are some United flights from IAH to Boston departing award availability too? ``` -The agent recalled your airline preference and home airport from durable memory. You didn't repeat yourself. The Amsterdam trip evidence is still in memory too, but it's a different task — the agent doesn't confuse the two. +The agent recalled your airline and home airport from durable memory without a tool call. The Amsterdam evidence is still stored, but it's a different task, so it didn't clear the relevance gate for a Boston question. ### Living documents: memory that updates over time -Not all memories are write-once. If you ask the agent to track RAM prices every week, it doesn't create a new memory each time. It updates the same document — appending the latest prices alongside the historical record: +Not all memories are write-once. If you ask the agent to track RAM prices every week, it doesn't create a new memory each time. It updates the same document and keeps the history: ``` You: Check DDR5 64GB kit prices on Newegg and update your @@ -56,133 +84,136 @@ netclaw: Updated the DDR5 price tracker. Current best: $142.99 data point to the tracking document. ``` -The memory system supports two update patterns: **merge-document** (revise and consolidate, like the price tracker) and **immutable-record** (append-only evidence — "completed the database migration on May 2nd" is a historical fact that shouldn't be revised, just recorded). The LLM picks the pattern based on the content. - -## How it works - -Memory formation happens outside the conversation turn — the user never waits for it. - -```text -┌────────────────────────┐ -│ Session Actor │◂──────────┐ -└───────────┬────────────┘ │ - │ │ - ▾ transcript stream │ -┌────────────────────────┐ │ -│ Observation Sidecar │ │ -└───────────┬────────────┘ │ - │ │ - ▾ idle timeout │ -┌────────────────────────┐ │ -│ Sidecar LLM Call │ │ -└───────────┬────────────┘ │ - │ │ - ▾ memory proposals │ -┌────────────────────────┐ │ -│ Checkpoint Queue │ (SQLite) │ -└───────────┬────────────┘ │ - │ │ - ▾ │ -┌────────────────────────┐ │ -│ Curation Pipeline │ │ -└───────────┬────────────┘ │ - │ │ - ▾ validated memories │ -┌────────────────────────┐ │ -│ Memory Store (SQLite) │───────────┘ -└────────────────────────┘ auto-recall -``` +Memory has two update patterns. **Merge-document** memories get revised and consolidated, like the price tracker. **Immutable-record** memories are append-only evidence, like "completed the database migration on May 2nd". The LLM picks the pattern from the content. -**Observation sidecar.** Each session has a companion [actor](/architecture/sessions/#session-identity) that accumulates the conversation transcript — user messages, assistant replies, tool calls. When the session goes idle, the sidecar makes a separate LLM call (not part of the conversation) to distill observations into memory proposals. +## How memories form -**Checkpoint queue.** Proposals land in a durable SQLite table (`memory_checkpoints`), not an in-memory queue. If the daemon crashes between observation and curation, pending proposals survive the restart. A background worker continuously leases and processes pending checkpoints with retry logic (up to 5 attempts per proposal). +Formation happens outside your turn, so you never wait for it. -**Curation pipeline.** The background worker applies rules-first extraction (pattern-matching for project facts, milestones, preferences), deduplicates against existing memories via fingerprinting, blocks anything that looks like a secret or credential, and persists what survives. +**Memory observer.** Each session has a companion actor that accumulates the transcript: your messages, assistant replies, tool call names, recalled memories and loaded skills. It makes a separate LLM call to distill that transcript into memory proposals when the session has been idle for 90 seconds (`Session.MemoryObserverIdleSeconds`), after every 5 completed turns (`Session.Tuning.MemoryDistillationTurnInterval`), and once more when an idle session shuts down. Information already shown as a recalled memory is excluded from new proposals. -**Automatic recall.** Before each user turn, a deterministic retrieval coordinator runs a lightweight [retrieval-augmented generation](https://en.wikipedia.org/wiki/Retrieval-augmented_generation) (RAG) pipeline. It tokenizes the user's message and recent context, builds a query plan, searches the memory store via full-text search (SQLite FTS5), and scores candidates using a composite of lexical match strength and memory class rank. The top results are injected directly into the system prompt — not as a tool call, but as context the model sees before it starts generating. +**Other entry points.** The agent's `store_memory` tool queues a proposal directly. After a [compaction](/architecture/sessions/#compaction), the session also queues a checkpoint carrying the compaction summary. Subagents can't write memory themselves. They return structured findings to the parent session, which reviews them and queues only the accepted ones. -That's why the agent already knows your airline preference in Conversation 2. It didn't call a tool — the retrieval coordinator found the memory, scored it as relevant, and injected it into the system prompt. +**Checkpoint queue.** Proposals land in the `memory_checkpoints` table, not in an in-memory queue. A background worker leases them, retries up to 5 times, and survives a daemon crash between observation and curation. -Retrieval is deterministic by design — no embeddings, no neural ranking, no non-determinism in the recall path. The same query against the same memory store produces the same results every time. Durable facts are heavily prioritized for auto-recall; evidence can also surface when it scores highly enough, but traces are excluded entirely. Recall runs on a 300ms budget so it never delays the conversation. +**Curation.** The worker applies rules first: pattern matching for project facts, milestones and preferences, fingerprint deduplication, and a block on anything that looks like a secret or credential. Then it embeds the proposal and looks up existing memories with similarity above 0.86 (up to 5). Similarity only nominates candidates. It never merges anything by itself. When there's a nominee, or the rules can't decide, a curation call to your Main model picks create, update, consolidate or skip. A merged body is validated against its sources before it's written, and if validation fails the new text is appended instead, so nothing gets dropped. A proposal that matches nothing already stored is created without an LLM call. -## Memory classes +Put together, the memory-related calls that reach your model provider are the memory observer call per trigger, the curation call when a near-duplicate turns up or the rules can't decide, and the compaction notes call. Embedding and relevance scoring run locally and cost no provider tokens. -The LLM classifies each memory into one of three classes based on how long the information is useful: +## Memory classes | Class | Lifespan | Auto-recalled | Example | |-------|----------|---------------|---------| -| **Durable fact** | Permanent | Yes (prioritized) | "Prefers TypeScript over JavaScript" | -| **Evidence** | 30 days | Sometimes (if score is high enough) | "Completed the database migration on May 2nd" | +| **Durable fact** | Permanent | Yes, if it clears the gate | "Prefers TypeScript over JavaScript" | +| **Evidence** | 30 days | Yes, if it clears the gate | "Completed the database migration on May 2nd" | | **Trace** | 72 hours | Never | Raw conversation logs | -Durable facts are what make the agent feel like it knows you. Evidence — work product, findings, completed tasks — matters for weeks but not forever. When something goes wrong with memory, traces are how you debug it. +Durable facts are what make the agent feel like it knows you. Evidence (work product, findings, completed tasks) matters for weeks but not forever. Traces are for debugging memory itself. -Expired evidence stays in the database but drops out of search results. The query layer filters expired records at read time — no garbage collection runs. +Expired evidence stays in the database but drops out of search results. The query layer filters it at read time, and no garbage collection runs. ## The knowledge graph -Memories are organized around **anchors** — entities like projects, people, preferences, workflows, and milestones. Anchors can relate to each other (`related_to`, `depends_on`), forming a lightweight knowledge graph. The LLM manages the graph structure behind the scenes as it creates and updates memories. +Memories hang off **anchors**: entities like projects, people, preferences, workflows and milestones. Anchors can be linked by edges, which makes a lightweight graph. The LLM manages it as it creates and updates memories. + +That's what connects the dots across conversations. When the agent stores "uses PostgreSQL" as a durable fact anchored to your backend project, and later you ask about that project's infrastructure, the PostgreSQL fact comes along because the anchor links them. + +## How recall works -This is what connects the dots across conversations. When the agent stores "uses PostgreSQL" as a durable fact anchored to your backend project, and later you ask about the backend project's infrastructure, the PostgreSQL preference comes along for the ride — because the anchor links them. +Before each user turn, a retrieval coordinator runs a small [retrieval-augmented generation](https://en.wikipedia.org/wiki/Retrieval-augmented_generation) pipeline, all inside the daemon. + +1. **Plan.** It tokenizes your message and recent context into a lexical query and anchor hints. +2. **Candidates.** SQLite FTS5 returns lexical matches. In parallel the query is embedded with the local model and up to 50 nearest vectors are fetched by cosine similarity. The two sets are merged. +3. **Rank.** Each candidate gets a fused score: 0.7 times cosine similarity, plus 0.3 times its squashed lexical score, plus a small prior for its memory class. That score is multiplied by a recency factor, `0.85 + 0.15 × 2^(-age/30 days)`, so old memories lose a little but never vanish on age alone. +4. **Floor.** A candidate whose vector is below the similarity floor (0.24 for the default model) is dropped. A memory with no vector yet, because it predates the backfill, can't be measured, so it skips the floor and competes on its lexical score. Records (the immutable-record class) have no embeddings by design, so they always compete on lexical score and don't count as a recall coverage gap. +5. **Gate.** The top candidates, at most 3 by default, go to a cross-encoder that reads the query and the memory together. Anything scoring below 0.02 is dropped. +6. **Inject.** What's left goes into the turn as a `[memory-recall]` block, capped at 2,000 characters. Zero survivors means no block at all, which is a healthy result and the common one. + +The whole thing runs on a 300 ms budget, with 150 ms for embedding the query and 120 ms for the gate. If a stage runs out of time or a model isn't loaded, recall degrades to the stage before it instead of delaying your turn. With embeddings off or unavailable, recall is full-text only: the same query against the same store gives the same results every time. + +The recalled block rides in the per-turn nudge, not the system prompt, so it doesn't break prompt caching. [Context structure](/architecture/context-structure/) shows where. + +### Why the gate exists + +The 0.27.0 release notes report this out-of-sample test on 450 real-traffic queries: + +| Metric | Hybrid floor only | With the cross-encoder gate | +|--------|------------------:|----------------------------:| +| Correct zero-injection | 7.3% | 86.8% | +| Relevant recall retained | n/a | 98.3% | +| Mean injected items per turn | 2.538 | 0.251 | + +The floor alone lets weakly related memories through nearly every turn. The gate cuts that to about one turn in four while keeping almost all the useful hits. + +### The models + +| | Embedder | Relevance gate | +|---|---|---| +| Model | `snowflake-arctic-embed-m-int8` | `ms-marco-minilm-l-6-v2` | +| Job | Turns text into a 768-dimension vector | Scores a query and a memory together | +| Size on disk | 110 MB | 23 MB | +| Runs | At write time, and on each recall query | Per candidate on recall | + +Both are downloaded from huggingface.co on first daemon start into `~/.netclaw/models/`, checked against pinned SHA-256 hashes, and loaded in-process through ONNX Runtime. Vectors live in the `memory_embeddings` table, and the daemon keeps a brute-force index in memory. Expect roughly 480 MB resident with them loaded and about 235 MB without. The [memory configuration page](/configuration/memory/) has measured numbers, the offline procedure, and the switch to turn it all off. ## Audience scoping -Netclaw classifies every session into a [trust tier](/architecture/design-philosophy/#lightweight-access-control-without-an-identity-provider) based on its source channel — Personal, Team, or Public. Memory respects these tiers: +netclaw classifies every session into a [trust tier](/architecture/design-philosophy/#lightweight-access-control-without-an-identity-provider) from its source channel: Personal, Team or Public. Memory respects the tiers: -- **Public** sessions have no memory access — no recall, no writes, no search. Memories authored by Public sessions are excluded from all audiences. +- **Public** sessions have no memory access. No recall, no writes, no search. Memories authored by Public sessions are excluded from all audiences. - **Team** and **Personal** sessions have full memory access, scoped by policy boundary. -Each memory carries the audience and boundary context of the session that created it. This prevents a memory formed in a private Personal session from surfacing in a shared Team channel unless the policy explicitly allows it. +Each memory carries the audience and boundary of the session that created it. A memory formed in a Personal session doesn't surface in a shared Team channel unless policy allows it. ## Compaction and memory -When a session's context window fills up and [compaction](/architecture/sessions/#compaction) kicks in, the memory system gets involved at Phase 2 — before the lossy summarization step. Key facts, decisions, and action items are extracted and checkpointed into the memory pipeline, ensuring they survive even if the summary misses them. - -This is the safety net: compaction can lose nuance, but durable memories persist independently of the conversation history. +When a session's context fills up, [compaction](/architecture/sessions/#compaction) shrinks the history and a model call writes compaction notes about what was discarded. After the compaction is saved, those notes are queued as a `compaction-boundary` checkpoint and go through the same curation pipeline as everything else. That's the safety net: compaction can lose nuance, but durable memories persist independently of the conversation history. ## Explicit tools -The agent also has four tools for direct memory management, available in Team and Personal sessions: +The agent has four tools for direct memory management, available in Team and Personal sessions: | Tool | Purpose | |------|---------| -| `find_memories` | Search durable facts and current evidence by keyword (includes evidence that auto-recall may not surface) | -| `get_memories` | Retrieve full content of specific memories by ID | -| `store_memory` | Explicitly save a memory (bypasses most curation filters) | -| `update_memory` | Correct or supersede an existing memory | +| `find_memories` | Search durable facts and current evidence, optionally including expired evidence | +| `get_memories` | Load the full content of specific memories by id | +| `store_memory` | Save a memory on request | +| `update_memory` | Edit a memory (find and replace, or full rewrite), supersede a record, or delete one | -These are the manual-control layer. Most of the time, automatic observation handles memory formation. Explicit tools are for when you want to correct something the agent got wrong or store something it wouldn't have observed on its own. If the agent recalls something incorrect, tell it — "that's wrong, I switched to Postgres last month" — and it will update or supersede the old memory. +Most of the time, automatic observation handles formation and automatic recall handles retrieval. The tools cover the gaps: correcting something the agent got wrong, or searching when recall injected nothing. If you say "that's wrong, I switched to Postgres last month", it finds the memory and edits it. To find, correct or remove a wrong memory yourself, see [fix a wrong memory](/configuration/memory/#fix-a-wrong-memory). ## Observability -Memory formation is invisible during normal use — the agent just gets smarter over time. To see what's happening: +Memory is invisible in normal use. To see what it's doing: -- **`netclaw stats`** shows memory recall counts and checkpoint queue depth -- **`netclaw doctor`** runs a health check on the checkpoint queue (warns if pending checkpoints exceed 25) -- **Debug logs** emit query plans, recall scores, and curation decisions for troubleshooting +- **`netclaw status`** shows `memory: sqlite (healthy)` or degraded. +- **`netclaw stats`** shows how many memories were formed and recalled this process, the store's anchor, document, record and edge counts, and pending checkpoints. +- **`netclaw doctor`** checks the checkpoint backlog (warns above 25 pending), curation model health, embedding coverage and the relevance gate. +- **The daemon log** carries a `memory_retrieval_final` line per turn with how many candidates the floor and the gate dropped, and `memory_recall_*_degraded` lines when a stage was skipped. ## Configuration -Memory is enabled by default. The key settings in `netclaw.json`: +Memory is on by default, and the defaults suit most installs. The settings that matter are `Memory.Enabled`, `Memory.Embeddings.Enabled`, `Memory.Embeddings.AutoDownload` and `Memory.AutoRecallMaxItems`. All of them are covered, with defaults, on [Memory and embeddings](/configuration/memory/). -| Setting | Default | What it controls | -|---------|---------|-----------------| -| `Memory.Enabled` | `true` | Master switch — disables all memory when `false` | -| `Memory.RecallTimeoutMs` | `300` | Time budget for auto-recall per turn | -| `Memory.AutoRecallMaxItems` | `3` | Maximum memories injected per turn | +## Limitations -SQLite is the only storage backend — no vector database, no external service, no API keys. Netclaw should work on a single machine with nothing beyond what ships in the binary. FTS5 handles the current recall workload; vector search may arrive later. +- One machine, one SQLite file. There's no external vector database, and no remote embedding service to point at. +- The first start with embeddings on needs to reach huggingface.co, or the model files copied in beforehand. +- There's no `netclaw memory` command to list, export or delete memories. The agent's tools and the SQLite file are the routes. +- A memory edited through the agent is re-embedded as part of the edit. One edited directly in the SQLite file loses its vector until the next daemon start or `netclaw memory backfill-embeddings`. -## Keep reading +## Related pages -- [Sessions & Input Model](/architecture/sessions/) — session lifecycle and compaction (where memory extraction fits) -- [Context Structure](/architecture/context-structure/) — how the recall and memory-index layers slot into each turn's context -- [Security Architecture](/architecture/security-model/) — audience scoping that governs memory access -- [Design Philosophy](/architecture/design-philosophy/#automatic-memory-formation) — why memory belongs in core +- [Sessions and input model](/architecture/sessions/) - the lifecycle and compaction that feed memory +- [Context structure](/architecture/context-structure/) - where recalled memories sit in each turn +- [Memory and embeddings](/configuration/memory/) - configuration, resource cost, offline use, fixing a wrong memory +- [`netclaw memory`](/cli/memory/) - backfill vectors after an upgrade +- [Design philosophy](/architecture/design-philosophy/#automatic-memory-formation) - why memory belongs in core ## Resources -- [SQLite FTS5](https://www.sqlite.org/fts5.html) — the full-text search engine behind memory recall -- [Actor model](https://en.wikipedia.org/wiki/Actor_model) — the concurrency model that makes the observation sidecar possible -- [Akka.NET Persistence](https://getakka.net/articles/persistence/event-sourcing.html) — journal and snapshot recovery for the checkpoint queue -- [Write-Ahead Logging](https://www.sqlite.org/wal.html) — how SQLite ensures memory durability +- [SQLite FTS5](https://www.sqlite.org/fts5.html) - the full-text engine behind lexical recall +- [ONNX Runtime](https://onnxruntime.ai/) - runs the embedding and relevance models +- [snowflake-arctic-embed-m](https://huggingface.co/Snowflake/snowflake-arctic-embed-m) - the embedding model netclaw ships as its default +- [Write-ahead logging](https://www.sqlite.org/wal.html) - how SQLite keeps the memory store durable +- [Akka.NET persistence](https://getakka.net/articles/persistence/event-sourcing.html) - journal and snapshot recovery behind the observer diff --git a/src/content/docs/architecture/overview.md b/src/content/docs/architecture/overview.md index 1ae13f0..5defdf4 100644 --- a/src/content/docs/architecture/overview.md +++ b/src/content/docs/architecture/overview.md @@ -1,195 +1,230 @@ --- -title: "Architecture Overview" -description: "Daemon + CLI architecture, Akka.NET, and design principles." +title: "Architecture overview" +description: "The daemon and CLI, the Akka.NET actor system, and how a message becomes a turn." --- -Netclaw is a single-process .NET 10 application that runs as an always-on autonomous agent. It takes input from Slack, Discord, webhooks, timers, and a local TUI, and treats all of them the same way: as messages routed into persistent conversation actors. - -Two binaries ship together: `netclawd` (the daemon) and `netclaw` (the CLI). The daemon does everything. The CLI is a thin client that renders the daemon's output in a terminal. If the daemon isn't running, the CLI can't do anything interesting. +netclaw is a single-process .NET 10 application that runs as an always-on agent. It takes input from Slack, Discord, Mattermost, webhooks, timers and a local TUI, and treats all of them the same way: as messages routed into persistent conversation actors. One daemon owns everything that matters, and the rest of this page maps its parts. + +Two binaries ship together: `netclawd` (the daemon) and `netclaw` (the CLI). The daemon holds the agent logic. The CLI is mostly a thin client that renders the daemon's output in a terminal, with a few commands (`memory`, most of `doctor`) that work straight on files, so they still run when the daemon is down. + +```mermaid +flowchart TB + subgraph IN["Inputs"] + direction LR + CLI["netclaw CLI / TUI"] + CH["Slack, Discord, Mattermost"] + WH["Webhooks"] + RM["Reminders"] + end + subgraph D["netclawd (one process)"] + direction TB + GW["Gateway: auth and policy checks"] + SM["SessionManager"] + SA["LlmSessionActor, one per conversation"] + OBS["Memory observer"] + SUB["Subagents"] + TOOLS["ToolAuthorizer, built-in tools, MCP"] + GW --> SM --> SA + SA --> OBS + SA --> SUB + SA --> TOOLS + end + subgraph EXT["Backing services"] + direction LR + LLM["LLM providers"] + DB[("netclaw.db, SQLite: journal, memory, reminders")] + ONNX["Local ONNX models"] + MCPS["MCP servers"] + end + CLI -->|"SignalR + REST"| GW + CH --> GW + WH --> GW + RM -->|"fires a session"| SM + SA --> LLM + SA --> DB + OBS --> DB + OBS -->|"curation embeds"| ONNX + TOOLS --> MCPS +``` ## Two binaries -### `netclawd` — the daemon - -The daemon is an ASP.NET Core application that owns all agent logic: Akka.NET actor system, LLM sessions, tool execution, channel adapters (Slack/Discord), SignalR hub, REST API, and SQLite persistence. +### `netclawd`: the daemon -It binds to `http://127.0.0.1:5199` (loopback only) by default. A PID file at `~/.netclaw/netclaw.pid` and a lock file enforce single-instance — one daemon per machine. +The daemon is an ASP.NET Core application that owns the [Akka.NET](https://getakka.net/) actor system, LLM sessions, tool execution, the channel adapters (Slack, Discord, Mattermost), the SignalR hub, the REST API, and SQLite persistence. It binds to `http://127.0.0.1:5199` by default, and a PID file at `~/.netclaw/netclaw.pid` plus a lock file enforce one daemon per machine. To listen beyond loopback, see [exposure modes](/deployment/exposure-modes/). -### `netclaw` — the CLI +### `netclaw`: the CLI -A Termina-based TUI. No actor system, no persistence, no tool execution. It connects to the daemon over SignalR and renders what comes back. +A Termina-based TUI with no actor system and no tool execution. It connects to the daemon over SignalR and renders what comes back. It also manages the daemon: `netclaw daemon start` and `netclaw daemon stop` run `netclawd` as a detached process. -The CLI binary includes a `daemon` subcommand (`netclaw daemon start`, `netclaw daemon stop`) for managing the `netclawd` process. Think of `netclaw` as both the management interface and the interactive shell for a daemon that runs independently. +The split matters. Restart the CLI without interrupting conversations. Keep the daemon in Docker and the CLI on your host, or run the CLI inside the container with `docker exec netclaw `, since the image ships both binaries. Talk to the same sessions from Slack instead of a terminal. -The split matters: restart the CLI without interrupting conversations. Run the daemon in Docker and the CLI on your host. Swap the CLI out entirely for the web UI or a channel adapter. +![netclaw status output: overall healthy, the daemon version, the active model, and connector lines for Discord, Mattermost, the demo-utilities MCP server, Slack and the TUI](/screenshots/output/status.png) -![Netclaw status output showing a running daemon](/screenshots/output/status.png) - -`netclaw status` showing daemon state, active connectors, uptime, and current model. +`netclaw status` reports daemon state, connectors, uptime and the current model. ## Communication -The CLI talks to the daemon over two protocols: +The CLI reaches the daemon over two protocols: | Protocol | Path | Purpose | |----------|------|---------| -| **SignalR** | `/hub/session` | Real-time session interaction: create sessions, send messages, receive streaming output | -| **REST** | `/api/*` | Non-session operations: health, stats, device pairing, MCP queries, shutdown | +| **SignalR** | `/hub/session` | Interactive sessions: create or attach to a session, send messages, answer approval prompts, stream output back | +| **REST** | `/api/*` | Everything without real-time delivery: health, stats, device pairing, MCP queries, shutdown | -SignalR carries the interactive traffic. The client sends commands (`CreateSession`, `SendMessage`, `CompactSession`) and receives typed `SessionOutputDto` events back as a stream. REST handles everything that doesn't need real-time delivery. +The hub's methods are `CreateSession`, `EnsureSession`, `AttachSession`, `SendMessage` and `RespondToInteraction`. Output comes back as typed events on a stream. Remote clients authenticate with a device bearer token issued through [pairing](/guides/pairing-remote-devices/). A loopback request counts as the operator's own only when its `Host` is `localhost` or a loopback address and any `Origin` matches. Otherwise it gets a 401. ## Akka.NET internals -[Akka.NET](https://getakka.net/) is the concurrency and persistence layer. Every conversation is an actor. Every channel adapter is an actor. Background jobs, reminders, tool approvals — all actors. +[Akka.NET](https://getakka.net/) is the concurrency and persistence layer. Every conversation is an actor, every channel adapter is an actor, and so are background jobs, reminders and tool approvals. ### Key actors | Actor | Role | |-------|------| -| `LlmSessionActor` | One per conversation thread, keyed by channel + thread identifier. Persistent, with three states: Ready, Processing, Compacting. | -| `SessionManager` | Routes messages to the right session actor via child-per-entity routing (no lookup table). | -| `ReminderManagerActor` | Scheduled tasks, backed by Akka.Reminders + SQLite. | -| `BackgroundJobManagerActor` | Async shell execution. | -| `SubAgentActor` | Child of a session actor. Handles delegated work (research, code analysis, summarization). | +| `LlmSessionActor` | One per conversation thread. Persistent, with five phases: Recovering, Ready, Processing, Compacting, Passivating. See [sessions](/architecture/sessions/). | +| `SessionManager` | Routes messages to the right session actor with child-per-entity routing (no lookup table). | +| `SessionMemoryObserverActor` | A persistent child of each session. Watches the transcript and proposes memories when the session goes quiet. | +| `MemoryCurationActor` | Validates and deduplicates memory proposals. A background worker applies the same logic to the durable checkpoint queue. | +| `SubAgentActor` | Child of a session actor. Runs delegated work (research, code analysis, summarization) in the background while the session carries on. | +| `ReminderManagerActor` | Scheduled tasks, backed by Akka.Reminders and SQLite. | +| `BackgroundJobManagerActor` | Detached shell jobs, up to 5 at a time. | +| `ToolApprovalActor` | Holds approval requests until a human answers. | +| `PairingActor` | Issues pairing codes and device tokens for remote clients. | | `ModelCapabilityActor` | Singleton that caches model capabilities at startup. | -| `ToolApprovalActor` | Approval gates for restricted tools. | +| Channel actors | A gateway, conversation and thread-binding actor for each of Slack, Discord and Mattermost. | -### Why actors? +### Why actors -Each conversation needs its own state, its own message queue, and crash isolation from other conversations. Actors give you that without thread management. A session actor processes one turn at a time, persists events to SQLite, and recovers its full history on restart. If one conversation crashes, the rest keep running. +Each conversation needs its own state, its own message queue and crash isolation from the others. Actors give you that without thread management. A session actor processes one turn at a time, journals its events to SQLite, and recovers its full history on restart. If one conversation crashes, the rest keep running. -Persistence uses [Akka.Persistence.Sql](https://getakka.net/articles/persistence/event-sourcing.html) with SQLite. Events: `TurnRecorded`, `SessionTitleSet`, `SessionCompacted`. Serialization is [Google Protobuf](https://protobuf.dev/) via the `NetclawProtobufSerializer`. +Persistence uses [Akka.Persistence.Sql](https://getakka.net/articles/persistence/event-sourcing.html) with SQLite and [Google Protobuf](https://protobuf.dev/) serialization through `NetclawProtobufSerializer`. The journal records turns, accepted input, tool batches and their calls, approval requests and decisions, compaction, title changes and background job reaping. It's an event log for the session, not just a transcript. ## Three boundaries -Three logical boundaries divide the daemon (from the [runtime spec](https://github.com/netclaw-dev/netclaw/blob/dev/docs/spec/SPEC-001-runtime-boundaries.md)): +Three logical boundaries divide the daemon, from the [runtime spec](https://github.com/netclaw-dev/netclaw/blob/dev/docs/spec/SPEC-001-runtime-boundaries.md). -At the edge, the **gateway** receives transport events (Slack message, webhook POST, CLI input), runs policy checks, and converts them to actor commands. Nothing reaches the session layer before policy runs. +At the edge, the **gateway** receives transport events (a Slack message, a webhook POST, CLI input), runs policy checks, and converts them to actor commands. Nothing reaches the session layer before policy runs. -The **session** layer owns conversation state, turn lifecycle, compaction, and persistence. Sessions communicate outward only by broadcasting messages — they have no idea whether they're talking to Slack, Discord, or the CLI. +The **session** layer owns conversation state, turn lifecycle, compaction and persistence. Sessions only broadcast outward. They don't know whether they're talking to Slack, Discord or the CLI. -**Subscribers** (channel adapters and UI clients) consume those broadcasts via pub/sub. They render output and nothing else. They can't read actor internals or influence model behavior. +**Subscribers** (channel adapters and UI clients) consume those broadcasts. They render output and nothing else, and they can't read actor internals or influence model behavior. -Data flows one direction: gateway sends commands to sessions, sessions broadcast to subscribers. Policy errors close the gate. +Data flows one way: the gateway sends commands to sessions, and sessions broadcast to subscribers. A policy error closes the gate. ## Everything is just input -Every input source — Slack message, webhook POST, timer, CLI chat, web UI — produces a message routed to a session actor with context-specific instructions. The actor doesn't know or care where it came from; turns process identically regardless of source. - -Adding a new channel adapter requires no changes to the session layer. +A Slack message, a webhook POST, a timer, a CLI chat: each produces a message routed to a session actor with context-specific instructions. The actor doesn't know where it came from, and turns run identically regardless of source. Adding a channel adapter requires no changes to the session layer. ## Channel adapters -Slack and Discord adapters run inside the daemon as [hosted services](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services). They're in-process actors that maintain persistent connections to the platform APIs, not external bridges. +Slack, Discord and Mattermost adapters run inside the daemon as [hosted services](https://learn.microsoft.com/en-us/aspnet/core/fundamentals/host/hosted-services). They're in-process actors holding persistent connections to the platform APIs, not external bridges. | Channel | Connection | Actors | |---------|-----------|--------| | **Slack** | [Socket Mode](https://api.slack.com/apis/socket-mode) via SlackNet | `SlackGatewayActor`, `SlackConversationActor`, `SlackThreadBindingActor` | | **Discord** | WebSocket via Discord.Net | `DiscordGatewayActor`, `DiscordConversationActor`, `DiscordSessionBindingActor` | +| **Mattermost** | WebSocket and REST, with interactive actions posted back to the daemon | `MattermostGatewayActor`, `MattermostConversationActor`, `MattermostSessionBindingActor` | -An [Akka.Streams](https://getakka.net/articles/streams/introduction.html) `SessionPipeline` bridges channel adapters to the actor system. +An [Akka.Streams](https://getakka.net/articles/streams/introduction.html) `SessionPipeline` bridges channel adapters to the actor system. Setup for each is in the [channels section](/channels/slack/). ## LLM providers -Netclaw uses [Microsoft.Extensions.AI](https://learn.microsoft.com/en-us/dotnet/ai/ai-extensions) (`IChatClient`) as the provider abstraction. Supported providers: +netclaw uses [Microsoft.Extensions.AI](https://learn.microsoft.com/en-us/dotnet/ai/ai-extensions) (`IChatClient`) as the provider abstraction. Supported providers: | Provider | Type key | Notes | |----------|----------|-------| -| **Ollama** | `ollama` | Code default. Native integration, queries `/api/show` for capabilities. | -| **OpenRouter** | `openrouter` | Recommended for cloud models. Widest model catalog. | +| **Ollama** | `ollama` | Native integration, queries `/api/show` for capabilities. | +| **OpenRouter** | `openrouter` | Widest model catalog. | | **Anthropic** | `anthropic` | Direct API. | | **OpenAI** | `openai` | Direct API, OAuth or API key. | | **GitHub Copilot** | `github-copilot` | OAuth device flow. Uses your Copilot subscription. | | **Venice.ai** | `veniceai` | Privacy-focused, OpenAI-compatible API. | -| **DeepSeek** | `deepseek` | 1M context window, thinking mode, text-only. | -| **OpenAI-compatible** | `openai-compatible` | Generic self-hosted endpoint. | +| **DeepSeek** | `deepseek` | `deepseek-v4-*` models get a 1M context window and text-only limits from netclaw. | +| **OpenAI-compatible** | `openai-compatible` | Any self-hosted endpoint (llama.cpp, vLLM). The API key is optional. | -At startup, the daemon queries the provider to detect context window size, vision support, and tool use capabilities. If the direct query fails, it falls back to the OpenRouter catalog, then HuggingFace metadata, then assumes text-only. - -See [Models](/configuration/models/) for configuration and [Managed Providers](/configuration/managed-providers/) / [Self-hosted Providers](/configuration/self-hosted-providers/) for provider setup. +At startup the daemon works out each model's context window and input and output modalities. It asks the provider first, then falls back through the OpenRouter catalog and HuggingFace metadata, filling each field from the first source that knows it, and assumes text-only if nothing answers. See [Models](/configuration/models/) for configuration and [managed](/configuration/managed-providers/) or [self-hosted](/configuration/self-hosted-providers/) providers for setup. ## Session lifecycle -Each conversation thread maps to one persistent actor. In Slack, the key is `{channelId}/{threadTs}`; Discord uses a similar channel + thread mapping. - -**States:** Ready → Processing → (threshold check) → Ready or Compacting → Ready. - -When a session's context grows past the compaction threshold, the actor compacts in phases: clear stale tool results (Phase 1), drop old messages via extractive reduction keeping the N most recent (Phase 2), then optionally generate observation notes from the discarded window via a best-effort LLM call (Phase 3). If the LLM call fails, extractive-only results stand. Context shrinks, and the session returns to Ready. +Each conversation thread maps to one persistent actor. The actor cycles between Ready and Processing, compacts when its context passes a threshold, and passivates when idle. [Sessions](/architecture/sessions/) covers the phases, the timeouts, compaction and what a session leaves on disk. ## Tool system -Tools are C# methods registered at compile time via a Roslyn incremental source generator. The generator produces JSON schema and typed deserializers — no runtime reflection. `ToolRegistry` holds the catalog; `IToolExecutor` runs them. +Tools are C# methods registered at compile time through a Roslyn incremental source generator, which produces the JSON schema and typed deserializers with no runtime reflection. `ToolRegistry` holds the catalog and `IToolExecutor` runs them. -Access control is layered: `ToolAccessPolicy` (which tools are available), `ToolPathPolicy` (filesystem boundaries), `ShellCommandPolicy` (allowed shell commands). Feature gates control progressive exposure. +Since 0.27.1 every tool call and every shell call goes through one `ToolAuthorizer`: the same authority and consent loop, so there's no second gate to drift out of sync. The older policy classes (`ToolAccessPolicy` for which tools exist, `ToolPathPolicy` for filesystem boundaries, `ShellCommandPolicy` for shell commands) still do the individual checks inside it. Feature gates hide the tools of a disabled subsystem (memory, search, skills, scheduling, subagents) from every audience. The deployment posture caps the audience of every call, including a sub-agent's, so a Team posture gives `netclaw chat` Team tools. See [the security model](/security/security-model/). + +A session starts with a small core of tools and loads the rest on demand, so the prompt stays small. That's progressive disclosure. `search_tools` ranks its results by how many query words match, so a built-in tool like `list_reminders` turns up ahead of a shell command. ### MCP integration -External tool servers connect via the [Model Context Protocol](https://modelcontextprotocol.io/). `McpClientManager` handles server lifecycle including OAuth 2.1 flows. Sessions see only server summaries until a tool is explicitly requested via `search_tools` (progressive disclosure). See [MCP Servers](/configuration/mcp-servers/) for configuration. +External tool servers connect through the [Model Context Protocol](https://modelcontextprotocol.io/). `McpClientManager` handles the server lifecycle, including OAuth 2.1 flows and, since 0.27.0, pre-registered confidential clients with a stored client secret. Servers connect in parallel at startup. Sessions see server summaries until a tool is requested through `search_tools`. A server's prompts can be invoked as skills, and image, audio and resource results from MCP tools are delivered as files. See [MCP servers](/configuration/mcp-servers/). ## Supporting systems -The daemon keeps a [SQLite](https://www.sqlite.org/docs.html)-backed memory store in `~/.netclaw/netclaw.db`. Facts persist across sessions; a background `MemoryCurationEngine` handles pruning and deduplication. +The daemon keeps memory in `~/.netclaw/netclaw.db`: documents, records, a full-text index and, since 0.27.0, vectors. Two small ONNX models downloaded to `~/.netclaw/models/` embed memories and gate recall, entirely in-process. A background `MemoryCurationEngine` extracts, deduplicates and merges proposals. See the [memory model](/architecture/memory-model/). -**Skills** are markdown files in `~/.netclaw/skills/` that extend what the agent knows how to do. Built-ins ship as embedded resources; extras sync from a CDN feed at startup. +**Skills** are markdown files that extend what the agent knows how to do. The built-in system skills ship inside the daemon binary and are restored on every start. Everything else comes from your own skills directory, operator-configured [skill feeds](/skills/skill-feeds/) and external sources, or from the agent itself through the `skill_manage` tool. -For delegated work — research, code analysis, summarization — a session spawns a **subagent** as a child actor. Definitions live in `~/.netclaw/agents/*.md` with YAML frontmatter (built-ins: research-assistant, code-analyst, summarizer), and each subagent run gets up to 30 tool iterations. See [Custom Subagents](/guides/custom-subagents/). +For delegated work, a session spawns a **subagent** as a child actor. Definitions live in `~/.netclaw/agents/*.md` with YAML frontmatter (built-ins: research-assistant, code-analyst, summarizer), and each run goes on until it gives a final reply, is cancelled or goes quiet for too long, with no fixed cap on tool rounds. `spawn_agent` returns a run ID at once. The run is recorded in the session journal, and its result arrives in a later turn as a notice that names a `result.md` file. See [custom subagents](/skills/custom-subagents/). -Scheduled tasks go through `ReminderManagerActor`. See [Reminders](/configuration/reminders/). +Search runs through Brave, DuckDuckGo or SearXNG backends. Images and attachments are normalized before they reach a model. Scheduled tasks go through `ReminderManagerActor`, covered in [Reminders](/configuration/reminders/). ## Configuration Layered, with later sources winning: -1. `~/.netclaw/config/netclaw.json` — base config -2. `~/.netclaw/config/secrets.json` — encrypted credentials (ASP.NET Core Data Protection) -3. `NETCLAW_*` environment variables — highest priority +1. `~/.netclaw/config/netclaw.json`: base config +2. `~/.netclaw/config/secrets.json`: encrypted credentials (ASP.NET Core Data Protection) +3. `NETCLAW_*` environment variables: highest priority + +`ConfigWatcherService` watches `netclaw.json` with a 500 ms debounce. A valid change triggers a coordinated restart: close ingress, drain active sessions, persist state, restart in-process, then warm the active sessions back up. An invalid change is refused and the running daemon keeps its current configuration. Secrets and environment changes aren't watched, so those need a manual restart, and an OAuth token refresh writes `secrets.json` without causing one. -A `ConfigWatcherService` monitors `netclaw.json` with a 500ms debounce. Valid changes trigger a coordinated restart: close ingress, drain active sessions, persist state, restart, then warm active sessions back up. Active CLI sessions reconnect automatically after the restart completes. +The drain stops each session at a safe point, such as the end of a tool batch, and tells the person in the chat. A turn stopped after its first tool batch resumes after the restart when nothing else is waiting. A running sub-agent parks and resumes too, which can add up to about 18 seconds to a graceful restart. A graceful stop of the whole daemon is still bounded to 30 seconds. ## Deployment -The daemon runs on anything .NET 10 supports, from a Raspberry Pi to a cloud VM. Ships as a self-contained tarball: `netclaw-{version}-{os}-{arch}.tar.gz`. +netclaw runs on Linux (x64 and arm64), Windows x64 with native PowerShell, and macOS on Apple Silicon. The installer downloads two archives per platform, `netclaw` and `netclawd`, as `.tar.gz` (`.zip` on Windows). | Method | Guide | |--------|-------| -| **systemd** (recommended for Linux) | [systemd Service](/deployment/systemd/) | -| **Docker** | [Docker Deployment](/deployment/docker/) | +| **systemd** (recommended for Linux) | [systemd service](/deployment/systemd/) | +| **Docker** | [Docker deployment](/deployment/docker/) | | **Direct** | `netclaw daemon start` (detached process) | -Docker runs only the daemon; the CLI stays on the host. For remote access beyond loopback, see [Exposure Modes](/deployment/exposure-modes/). +The Docker image ships both binaries, with `netclaw` on the PATH, so `docker exec netclaw status` works. For remote access beyond loopback, see [exposure modes](/deployment/exposure-modes/). ## Design principles -None of these are aspirational — they're constraints the codebase enforces. +None of these are aspirational. They're constraints the codebase enforces. -1. **Gall's Law first** — ship a simple working system before building a generalized framework. -2. **Actor transport boundary** — channel adapters consume broadcasts via pub/sub and never drive model internals directly. -3. **Serialization boundary** — Netclaw controls its own persistence schema. Never persist framework-external chat types. -4. **Security boundary** — default deny, explicit allow, fail closed. See [Security Model](/security/security-model/). -5. **Everything is just input** — every source (Slack, webhook, CLI, timer) produces a message routed to a session actor. No special-casing by channel. +1. **Gall's Law first.** Ship a simple working system before building a generalized framework. +2. **Actor transport boundary.** Channel adapters consume broadcasts through pub/sub and never drive model internals directly. +3. **Serialization boundary.** netclaw controls its own persistence schema and never persists framework-external chat types. +4. **Security boundary.** Default deny, explicit allow, fail closed. See the [security model](/security/security-model/). +5. **Everything is just input.** Every source (Slack, webhook, CLI, timer) produces a message routed to a session actor. No special-casing by channel. ## Limitations -No clustering or multi-node distribution. One daemon, one machine, one actor system. - -SQLite is the only persistence backend. Good for single-node; won't scale to multi-instance. - -Security is in Phase 1: all connections are loopback-only with `LocalOperator` trust (full access). Bearer token auth and device pairing for remote access are planned for a future release. - -Binary updates require a daemon restart. No hot code reload. - -## What to read next - -Setting up for the first time? Go to [Models](/configuration/models/) to configure your LLM provider. +- No clustering. One daemon, one machine, one actor system. +- SQLite is the only persistence backend. That's right for a single node and doesn't scale to multiple instances. +- Remote access works through exposure modes and paired devices, not through an identity provider. There's no per-user login. +- Binary updates need a daemon restart. There's no hot code reload. +- Local semantic memory loads about 129 MB of models and makes the daemon roughly twice the size (about 480 MB idle against about 235 MB) it'd be without them. Turn it off if the host can't spare the RAM. -For production deployment, see [systemd Service](/deployment/systemd/) or [Docker Deployment](/deployment/docker/). +## Related pages -For the security model, see [Security Model](/security/security-model/) — covers ACLs, trust context, and audience scoping. +- [Sessions and input model](/architecture/sessions/) - lifecycle, timeouts, compaction and storage +- [Memory model](/architecture/memory-model/) - how the daemon remembers across sessions +- [Context structure](/architecture/context-structure/) - what the model sees each turn +- [Security model](/security/security-model/) - audiences, approvals and the tool authorizer +- [Design philosophy](/architecture/design-philosophy/) - why it's built this way ## Resources -- [Akka.NET documentation](https://getakka.net/) — the actor model framework behind netclaw's concurrency -- [Microsoft.Extensions.AI](https://learn.microsoft.com/en-us/dotnet/ai/ai-extensions) — the `IChatClient` abstraction netclaw uses to talk to LLM providers -- [Model Context Protocol specification](https://modelcontextprotocol.io/) — standard for external tool server integration -- [SignalR documentation](https://learn.microsoft.com/en-us/aspnet/core/signalr/introduction) — real-time communication between CLI and daemon -- [SQLite documentation](https://www.sqlite.org/docs.html) — the persistence and memory backend +- [Akka.NET documentation](https://getakka.net/) - the actor framework behind netclaw's concurrency +- [Microsoft.Extensions.AI](https://learn.microsoft.com/en-us/dotnet/ai/ai-extensions) - the `IChatClient` abstraction for LLM providers +- [Model Context Protocol specification](https://modelcontextprotocol.io/) - the standard for external tool servers +- [SignalR documentation](https://learn.microsoft.com/en-us/aspnet/core/signalr/introduction) - real-time communication between the CLI and the daemon +- [SQLite documentation](https://www.sqlite.org/docs.html) - the persistence and memory backend diff --git a/src/content/docs/architecture/security-model.md b/src/content/docs/architecture/security-model.md index ffede98..0063dd9 100644 --- a/src/content/docs/architecture/security-model.md +++ b/src/content/docs/architecture/security-model.md @@ -1,135 +1,124 @@ --- -title: "Security Architecture" -description: "ACL evaluation, audience scoping, and trust context." +title: "Security architecture" +description: "Why netclaw's security model is shaped the way it is: default-deny, audiences as the trust primitive, one authorizer with four outcomes, and a human in the loop." --- -Netclaw gives an LLM shell access, file access, and the ability to talk to external services. Without strong guardrails, that's a loaded weapon pointed at your infrastructure. The security architecture exists to make the agent useful without making it dangerous. +Netclaw gives an LLM shell access, file access and the ability to talk to external services. Without guardrails, that's a loaded weapon pointed at your infrastructure. The security architecture exists to make the agent useful without making it dangerous, and it's enforced from daemon startup with no extra setup. -All of this is enforced automatically from daemon startup — no additional setup required for the defaults to protect you. +The core bet is that explicit grants are safer than implicit restrictions. Rather than enumerate everything dangerous and block it, netclaw starts from zero permissions and asks you to opt in to each capability. We build up from nothing instead of poking holes in a wall. -The core bet: **explicit grants are safer than implicit restrictions.** Rather than trying to enumerate everything dangerous and blocking it, netclaw starts from zero permissions and requires you to opt in to each capability. This inverts the typical "sandbox" approach — instead of poking holes in a wall, you're building up from nothing. +This page is the design rationale. For what netclaw enforces and the exact lists, see the [Security model](/security/security-model/). For how prompts work, see [How tool approvals work](/security/approvals/). -## Design Decisions +## Design decisions ### Default-deny over default-allow -Most tools start permissive and add restrictions. Netclaw does the opposite. A freshly initialized daemon with no config file binds to loopback, disables shell access, and limits tools to basic file operations in a temporary session directory that's wiped on session end. You build up from there. +Most tools start permissive and add restrictions. Netclaw does the opposite. A fresh daemon with no config file binds loopback, disables shell, and gives channel traffic only basic file tools inside a session directory. Even the local TUI gets only those Public tools until a config sets a deployment posture. You build up from there. -Misconfiguration fails safe. Forget to add a tool grant? The tool is invisible. Typo in a channel ID? That channel gets nothing. Config file corrupted? Daemon won't start. +Misconfiguration fails safe. Forget to add a tool grant and the tool is invisible. Mistype a channel ID and that channel gets nothing. Write a malformed tool list and the daemon refuses to start. ### Audience as the trust primitive -Rather than per-user ACLs (which require identity infrastructure most self-hosted deployments don't have), netclaw classifies trust by **channel type**. A message from the TUI (terminal interface, launched by `netclaw chat`) is from the operator sitting at the machine — high trust. A message from Slack could be anyone in the workspace — lower trust. An unknown channel gets the lowest trust. - -Three audiences, ordered by trust: +Per-user ACLs need identity infrastructure that most self-hosted deployments don't have. Netclaw classifies trust by where a message came from instead. A message from the TUI is from the operator at the machine. A message from Slack could be anyone in the workspace. An unknown source gets the least trust. ``` Personal > Team > Public ``` -![Security posture selection during netclaw init](/screenshots/output/init-02-security-posture.png) - -Posture selection during `netclaw init` — the choice that establishes the baseline trust tier for your deployment. - -This maps to how people actually deploy: you trust yourself on your own machine, partially trust your coworkers in Slack, and don't trust unknown sources at all. During setup, you assign each channel to an audience tier individually: +This maps to how people deploy: you trust yourself on your own machine, partly trust coworkers in a channel, and don't trust unknown sources. Each channel or DM is assigned an audience individually, so one platform can span tiers, with DMs as Personal and a shared ops channel as Team. See [Audiences](/security/security-model/#trust-audiences) for how audiences are assigned. -![Per-channel audience configuration during netclaw init](/screenshots/output/init-04-channel-audiences.png) - -The same platform can span multiple audience tiers — Slack DMs as Personal, a shared ops channel as Team. The exact per-audience permission tables are in the [Security Model reference](/security/security-model/#trust-audiences). +The deployment posture is a ceiling on top of that. The audience that every tool decision uses, for a turn and for any sub-agent or routed skill it starts, is the narrower of the posture's audience and the message's source audience. A Team posture therefore runs the TUI with Team tools, which is the point of choosing it: that choice sets the baseline for what the agent can do wherever the message came from. ### Four layers, not one -No tool executes without passing four independent checks. Each layer can only deny — none can override a denial from another. +Every tool call goes through one authorizer. It applies a single ordered list of rules, and the first rule with an answer wins. Conceptually the rules fall into four layers, and a call has to get through all of them. + +```mermaid +flowchart LR + C["Tool call"] --> A["Admission
does this audience
have the tool?"] + A --> P["Prohibition
hard deny and
protected paths"] + P --> F["Filesystem authority
trusted roots"] + F --> K["Consent
mode, grants,
prompt"] + A -.-> D["Denied"] + P -.-> D + F -.-> D + K --> O["Allowed, requires approval,
requires correction or denied"] +``` -| Layer | Class | What it does | -|-------|-------|-------------| -| **Operation hard deny** | `ShellCommandPolicy` | Blocks unconditionally dangerous commands — `rm -rf /`, `sudo`, fork bombs, daemon self-kill. Structural matching on tokenized verb chains. Not approvable. | -| **Resource hard deny** | `ToolPathPolicy` | Blocks access to protected paths — `secrets.json`, key material, control-plane files. Symlink-aware. Maintains separate deny surfaces for reads, writes, and shell path references. | -| **Tool access grant** | `ToolAccessPolicy` | Audience-scoped allowlist. If a tool isn't granted to the current audience, the model never sees it. Binary — available or invisible. | -| **Approval gate** | `IToolApprovalService` | Human-in-the-loop confirmation for operations that pass the first three checks but are operationally risky. Approvals can be scoped to once, session, or persistent. | +The details those layers cover include the rule that Personal file reads reach as far as writes do, a private temp folder for each session, checks on redirects in compound shell commands, and glob checks that follow symlinked folders. The first three layers can only deny, and nothing the operator answers at a prompt reaches back through them. Consent is the only layer that asks a person, and it has two outcomes beyond allow and deny: it can prompt, or it can hand the agent advice so the agent re-authors the call. Putting prohibition before consent is the point. A saved "Always" grant can never cover a hard-denied command or a protected path, because those rules run first. -The execution order in code doesn't match this conceptual ordering — `ToolAccessPolicy` runs first (fail fast on unavailable tools before evaluating shell commands), then `ShellCommandPolicy`, then `ToolPathPolicy`, then the approval gate. The deny-only semantics are the same regardless of order. See the [reference page](/security/security-model/#four-layer-invocation-stack) for exact deny lists and configuration syntax. +Having one authorizer rather than four independent checks matters for a second reason. Built-in tools, MCP tools, shell, subagents and unattended runs all take the same path, so a rule can't be bypassed by finding the tool that skipped a check. The exact order is in [Layers in order](/security/security-model/#layers-in-order). ### Policy before dispatch -Inbound message ACL checks run in the gateway boundary, before messages reach the session layer. Tool execution policy (shell denies, path checks, approval gates) runs in the session's tool executor, synchronously before any tool fires. The [three-boundary design](/architecture/overview/#three-boundaries) (gateway → session → subscriber) enforces this structurally — the gateway is the only ingress point for messages, and the tool executor is the only execution path for tools. +Inbound message ACL checks run at the gateway, before a message reaches the session layer. Tool policy runs in the session's tool executor, before any tool fires. The [three-boundary design](/architecture/overview/#three-boundaries) (gateway, session, subscriber) makes that structural: the gateway is the only way in for messages, and the tool executor is the only way to run a tool. ### Human-in-the-loop as a layer, not a crutch -Approval gates sit at Layer 4 — after hard denies and access grants have already filtered. The human only sees requests that are structurally permitted but operationally risky. You're not approving every `ls` — you're approving `git push` to a remote. +Approval is the last layer, after hard denies and access grants have already filtered. The human only sees requests that are permitted but risky. You're not approving every `ls`. Reviewed read-only commands never prompt, and an answer of "Always" turns a one-off decision into a saved grant on a phrase, so `git push` is approved once rather than every time. -![Approval gate prompt in Slack — the human sees verb, tool, and directory before choosing](/assets/approval-prompt.png) +Prompts are how you build a policy, not the policy itself. When they keep coming, change a mode or a grant. [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/) says how. -The approval system extracts verb-chain patterns from shell commands. A verb-chain is the leading command tokens (`git push`, `docker compose up`) without paths or flags — so approving `git push` covers `git push origin main` and `git push --force-with-lease`. Compound commands (`cmd1 && cmd2`) are split and each segment is checked independently. +## Signals on every message -Channels that don't support interactive prompts (scheduled reminders, webhooks, headless automation) auto-deny tools that require approval — unless those tools have been persistently pre-approved or appear on the safe-list for non-interactive execution. +Every inbound message carries four signals: -## Trust Context Model +| Signal | Source | Describes | +|--------|--------|-----------| +| Principal classification | Channel adapter | Who is speaking, such as an operator, a trusted internal user, an untrusted external user or verified automation | +| Transport authenticity | Connection type | Whether the sender can be verified: local process, verified, unverified | +| Payload taint | Content origin | Where the data came from: trusted, community, public | +| Audience | Channel type or configuration | What permission set applies | -Every inbound message carries a trust context assembled from: +The audience resolves the permission set. The other signals describe the turn and travel with it. -| Signal | Source | Purpose | -|--------|--------|---------| -| **Principal classification** | Channel adapter | Who is speaking? (operator, team member, external, automation) | -| **Transport authenticity** | Connection type | Can we verify the sender? (local process, verified, unverified) | -| **Payload taint** | Content origin | Where did this data come from? (trusted, community, public) | -| **Audience** | Channel type | What trust level does this channel get? | +## MCP security: two-gate model -The audience resolves the final permission set. The other signals flow through to audit logs for post-hoc review. +External tool servers connecting over the [Model Context Protocol](https://modelcontextprotocol.io/) are gated twice. -## MCP Security: Two-Gate Model +1. **Server gate.** The server must be reachable for the audience: `AllowedMcpServers` for Team and Public, everything for Personal. +2. **Tool gate.** If per-tool grants exist for the server (`McpServerToolGrants`), only the listed tools are visible. With no entry for the server, all its tools are. -External tool servers connecting via the [Model Context Protocol](https://modelcontextprotocol.io/) (MCP) are doubly gated: +Approval modes add a third dimension: each server and tool can be `Auto`, `Approval` or `Deny` per audience. `netclaw mcp add` writes locked-down defaults for Team and Public and an open default for Personal. See [MCP tool permissions](/guides/mcp-tool-permissions/). -1. **Server allowlist** — the MCP server must be explicitly permitted for the audience -2. **Tool allowlist** — if per-tool grants are configured for that server, only listed tools are available +Netclaw doesn't load MCP tools into the model's context until the model searches for them with `search_tools`. Granted but unused tools don't consume context or influence behavior. -When no per-tool grants are configured for a server, all tools on that server are accessible (subject to the server-level gate). Configure `McpServerToolGrants` to restrict individual tools — see [`netclaw mcp permissions`](/cli/mcp-tools/) for the operational details. +## Self-configuration boundaries -Netclaw doesn't load MCP tools into the model's context until explicitly searched via `search_tools`. Granted-but-unused tools don't consume context window or influence behavior — the model doesn't know they exist until it asks. +The agent can change its own personality, instructions, reminders and project registry. It can't change security policy. Writes to the config directory are blocked by the path policy for every tool and every command form, so ACL rules, exposure mode, tool grants and audience mappings are read-only from the agent's side. Reminder tools are scoped to the caller's audience, so a Team session can't manage a Personal reminder. -## Self-Configuration Boundaries +## Threat model scope -The agent can modify its own personality, instructions, scheduled tasks, and project registry — but security policy is off-limits. ACL rules, exposure mode, tool grants, and audience mappings are read-only from the agent's perspective. - -This is enforced at two levels: the `config_write` tool category (one of seven grant categories in the [ACL policy](/security/security-model/#per-audience-permissions)) requires explicit grants, and the resource hard-deny layer blocks direct file access to config paths regardless of tool grants. - -## Threat Model Scope - -What the security model protects against: +What the model protects against: - The LLM deciding to run destructive commands -- Prompt injection via tool output or file contents attempting privilege escalation +- Prompt injection through channel messages attempting privilege escalation - Overly broad tool access from misconfiguration - Credential leakage through command output - Unauthorized network exposure -What it explicitly does **not** protect against: +What it doesn't protect against: -- A malicious operator with direct machine access (they can edit config files) +- A malicious operator with direct machine access +- Anything else the agent can read as the OS user. Agent tools are denied the common credential folders (`~/.ssh`, `~/.aws`, `~/.kube` and similar), but other files your user can read are open to it, and a command you approve still runs with your credentials +- A granted command doing more than its name suggests. Approvals aren't a sandbox - Side-channel attacks against the LLM provider's API -- Denial of service against the daemon process itself -- Vulnerabilities in third-party MCP servers (netclaw gates access, not behavior) - -## Limitations +- Denial of service against the daemon +- Vulnerabilities in third-party MCP servers. Netclaw gates access, not behavior -- Prompt injection detection is regex-based, not semantic — novel phrasings can evade it -- Approval gates require an interactive channel — headless sessions (scheduled jobs, reminders, webhooks with no connected user) auto-deny gated tools unless persistently pre-approved -- No per-user identity within a channel — all Slack users in an allowed channel get the same audience -- Secret redaction catches known patterns only — custom formats need custom path deny rules -- No formal sandboxing (seccomp, namespaces) for shell execution — defense is policy-based, not kernel-based +The second and third items are why the [Security model](/security/security-model/#what-netclaw-doesnt-protect-against) tells you to run netclaw as a separate OS user or in a container when the machine holds other credentials. A kernel sandbox for shell commands is planned and tracked in [netclaw-dev/netclaw#2343](https://github.com/netclaw-dev/netclaw/issues/2343). -## Related +## Related pages -- [Security Model reference](/security/security-model/) — operational details: exact deny lists, configuration syntax, per-audience permissions -- [Hardening](/security/hardening/) — production lockdown beyond the defaults -- [Architecture Overview](/architecture/overview/) — the three-boundary design that enforces policy-before-dispatch -- [`netclaw init`](/cli/init/) — where you first choose your security posture -- [`netclaw doctor`](/cli/doctor/) — verify what's actually being enforced at runtime +- [Security model](/security/security-model/) - audiences, the order of checks, protected paths and the exact lists +- [How tool approvals work](/security/approvals/) - modes, outcomes, grants and prompts +- [Hardening](/security/hardening/) - production lockdown beyond the defaults +- [Architecture overview](/architecture/overview/) - the three-boundary design behind policy-before-dispatch +- [`netclaw doctor`](/cli/doctor/) - verify what's enforced at runtime -## External Resources +## Resources -- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) — the attack taxonomy netclaw's security model is designed against -- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) — federal guidance on AI system risk management -- [Principle of Least Privilege (POLP)](https://csrc.nist.gov/glossary/term/least_privilege) — the access control philosophy behind default-deny -- [Defense in Depth (NIST)](https://csrc.nist.gov/glossary/term/defense_in_depth) — the layered-security model behind the four-layer stack +- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) - the attack taxonomy the model is designed against +- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) - federal guidance on AI risk management +- [Principle of least privilege (NIST)](https://csrc.nist.gov/glossary/term/least_privilege) - the philosophy behind default-deny +- [Defense in depth (NIST)](https://csrc.nist.gov/glossary/term/defense_in_depth) - the layered model behind the authorizer's rule order diff --git a/src/content/docs/architecture/sessions.md b/src/content/docs/architecture/sessions.md index de0eefc..9b51d29 100644 --- a/src/content/docs/architecture/sessions.md +++ b/src/content/docs/architecture/sessions.md @@ -1,130 +1,192 @@ --- -title: "Sessions & Input Model" -description: "How netclaw maps conversations to persistent actors, handles multiple clients, and keeps long-running sessions bounded." +title: "Sessions and input model" +description: "How netclaw maps conversations to persistent actors, what ends or stalls a turn, and how long-running sessions stay bounded." --- -Every conversation in netclaw is a session. One Slack thread, one Discord thread, one CLI chat — each maps to a single persistent [actor](https://en.wikipedia.org/wiki/Actor_model) that owns the conversation's state, processes one turn at a time, and survives daemon restarts. +Every conversation in netclaw is a session. One Slack thread, one Discord thread, one Mattermost thread, one CLI chat: each maps to a single persistent [actor](https://en.wikipedia.org/wiki/Actor_model) that owns the conversation's state, processes one turn at a time, and survives daemon restarts. The session doesn't know or care where its messages come from, and that's the foundation everything else builds on. -The session doesn't know or care where its messages come from. That's the foundation everything else builds on. +A session has a few simple obligations. It accepts input, runs a turn, keeps its history from outgrowing the model's context window, and shuts itself down when nobody's talking. The rest of this page is those four things in detail, plus the files a session leaves on disk. ## Session identity -A session's identity is derived from its channel and thread. In Slack, the key is `{channelId}/{threadTs}`. Discord uses a similar channel-plus-thread mapping. The CLI generates its own session ID when you start a chat. +A session's identity comes from its channel and thread. In Slack the key is `{channelId}/{threadTs}`. Discord uses a channel-and-thread mapping, Mattermost binds a channel and a root post, and the CLI generates its own id when you start a chat. -This is a deliberate 1:1 mapping. One thread, one session, one actor. No shared state between conversations, no cross-thread contamination, no global context that leaks between users. If a session actor crashes, the rest keep running — that's what the actor model buys you. +The mapping is 1:1 on purpose. One thread, one session, one actor. There's no shared state between conversations and no global context leaking between users. If a session actor crashes, the rest keep running. -The `SessionManager` routes messages to the right session actor using child-per-entity routing (each session is a child actor created on demand, keyed by session ID). No lookup table, no registry. The actor hierarchy *is* the routing table. +The `SessionManager` routes messages with child-per-entity routing: each session is a child actor created on demand and keyed by session id. There's no lookup table. The actor hierarchy is the routing table. ## Everything is just input -Slack messages, webhook POSTs, scheduled reminders, CLI chat input — they all enter through different channel adapters but produce the same kind of command: a `SendUserMessage` routed to a session actor. The actor processes the turn identically regardless of source. +Slack messages, Discord messages, Mattermost posts, webhook POSTs, scheduled reminders and CLI chat input all enter through different adapters and produce the same command: a `SendUserMessage` routed to a session actor. The turn runs the same way regardless of source. ``` -┌──────────┐ -│ Slack │──┐ -├──────────┤ │ ┌──────────┐ ┌─────────────────┐ -│ Discord │──┼───▸│ Gateway │───▸│ Session Actor │ -├──────────┤ │ └──────────┘ └─────────────────┘ -│ Webhook │──┤ ▲ -├──────────┤ │ │ policy checks -│ CLI/TUI │──┘ │ before dispatch -└──────────┘ +┌───────────────┐ +│ Slack │──┐ +├───────────────┤ │ +│ Discord │──┤ +├───────────────┤ │ ┌──────────┐ ┌─────────────────┐ +│ Mattermost │──┼───▸│ Gateway │───▸│ Session actor │ +├───────────────┤ │ └──────────┘ └─────────────────┘ +│ Webhook │──┤ ▲ +├───────────────┤ │ │ policy checks +│ CLI / TUI │──┘ │ before dispatch +└───────────────┘ ``` -Channel adapters run inside the daemon as hosted services — in-process actors that maintain persistent connections to platform APIs. Slack uses [Socket Mode](https://api.slack.com/apis/socket-mode); Discord uses a WebSocket connection via [Discord.Net](https://discordnet.dev/). - -Adding a new channel means writing a new adapter. No changes to the session layer, the compaction logic, the tool system, or the security model. +Channel adapters run inside the daemon as hosted services: in-process actors holding persistent connections to platform APIs. Adding a channel means writing a new adapter. The session layer, compaction, the tool system and the security model stay untouched. ## Lifecycle -Sessions move through five explicit phases. Transitions are validated at runtime — illegal transitions throw an exception, not a silent fallback. - -``` -Recovering ──▸ Ready ◂──▸ Processing - │ ▴ ▴ │ - │ │ └─────────┘ - ▾ │ - Compacting - │ - ▾ - Passivating ──▸ Ready (on new message) +A session moves through five phases. The actor validates every transition, and an illegal one throws instead of falling back silently. + +```mermaid +stateDiagram-v2 + [*] --> Recovering + Recovering --> Ready: journal replayed + Ready --> Processing: message arrives + Processing --> Ready: turn done + Ready --> Compacting: threshold crossed + Processing --> Compacting: threshold crossed + Compacting --> Ready + Compacting --> Processing: buffered message + Ready --> Passivating: idle + Processing --> Passivating: idle + Passivating --> Ready: new message + Passivating --> [*]: stopped ``` -Ready can also transition directly to Compacting (if a threshold is crossed outside of a turn), and Processing can transition to Compacting mid-turn. +| Phase | What's happening | +|-------|------------------| +| **Recovering** | Journal replay and snapshot recovery. The actor rebuilds state before it accepts a message. | +| **Ready** | Accepting messages, with the idle timer ticking. | +| **Processing** | An LLM call or tool execution is in flight. Incoming messages are buffered, and [who joins the turn](#who-joins-a-turn) depends on who sent them. One turn at a time, always. | +| **Compacting** | The context window is getting full and the session is shrinking its history. Messages are buffered, as in Processing. | +| **Passivating** | The session is winding down: a final memory distillation (5-second grace period), a snapshot, then stop. A new message during passivation aborts it and returns to Ready. | + +## What ends or stalls a turn + +A turn ends when the model finishes, a timeout or watchdog fires, the daemon stops or drains, the [loop guard](#loop-guard) stops it, or a person's message ends an automation turn. A reply cut off at the output limit fails the turn: nothing cut off is posted or stored, and the error category is `ModelBehavior`. A model call that streams only thinking stops after three times `Session.NoProgressTimeoutSeconds`, which is 60 minutes by default. There's no ceiling on tool rounds. The timeouts exist because a hung provider shouldn't hold a session in Processing forever. All of them live under `Session` in `netclaw.json`, in seconds, and the [configuration reference](/configuration/reference/) lists every one. + +| Setting | Default | What it bounds | +|---------|---------|----------------| +| `Session.PrefillTimeoutSeconds` | 1800 | Wait for the first token. Covers queueing and prompt prefill, which can legitimately take minutes on self-hosted backends. | +| `Session.FirstTokenTimeoutSeconds` | 600 | Silence between streaming tokens once the first has arrived. | +| `Session.NoProgressTimeoutSeconds` | 1200 | Time without real output. A backend that sends keepalives but no tokens still gets cut off. | +| `Session.ToolExecutionTimeoutSeconds` | 90 | One tool call. It must produce its first result in this time, and each streamed item resets the clock. | +| `Session.MaxToolIterationsPerTurn` | 60 | Legacy. It still binds, but nothing enforces it, so a turn that keeps doing new work can run past 60 rounds. | +| `Session.SidecarLlmTimeoutSeconds` | 90 | Background calls: title generation, memory distillation, compaction notes. | +| `Session.TurnLlmTimeoutSeconds` | 180 | Fallback. Setting it away from 180 without a `FirstTokenTimeoutSeconds` makes it the silence limit between tokens. | +| `Session.IdleTimeout` | `01:00:00` | Time before an idle session passivates. `00:00:00` disables passivation. | + +A watchdog timeout fails the turn with a timeout error and the session returns to Ready, so the next message works. A slow 27B model behind a busy llama.cpp server is the usual case for raising the first two. An approval prompt isn't on this list because approvals don't time out. + +### Loop guard -**Recovering** — journal replay and snapshot recovery. The actor rebuilds its state from persisted events before accepting any new messages. +A model that repeats itself gets corrected before it burns the turn: -**Ready** — accepting user messages. Idle timeout is ticking. If no one talks to the session for long enough and no subscribers are connected, it passivates. +- **Repeated calls.** After two equal rounds, netclaw refuses the next identical call and returns a correction. The first four refused repeats get a nudge, the fifth a stronger one, and at the sixth the turn ends quietly with the model's text. The person sees no error. +- **Failed calls.** The guard also counts failed calls across different tools since the last batch that held a success. The model gets a nudge naming the repeated cause at a weighted score of 4 and a stronger one at 6. At 8, the next response that asks for tools ends the turn quietly. One success in a batch resets the count. Sub-agent runs use the same guard. -**Processing** — an LLM call or tool execution is in flight. Incoming messages get buffered and processed after the current turn completes. One turn at a time, always. +Only a new turn or new person input resets the counts. -**Compacting** — the context window is getting full. The session runs a [tiered compaction](#compaction) to shrink the history. Messages are buffered during compaction, same as during processing. +### Who joins a turn -**Passivating** — the session is winding down. It extracts any remaining observations to durable memory (with a 5-second grace period), takes a persistence snapshot, and stops the actor. A new message during passivation aborts the shutdown and returns to Ready — passivation is always interruptible. +Each input runs under its own sender's authority, and a turn keeps the authority it started with. + +- A new message from the same person joins the active turn at the next tool-batch boundary. +- A message from another person waits for its own turn. +- An automated input (a reminder, a job result, a webhook or a child notice) waits for its own turn, and it no longer closes an open approval prompt. +- When the operator or a trusted person writes during an automation turn, that turn ends after its current tool batch and the person's turn starts next. Its first model call carries a note that the automated turn stopped unfinished. An interrupted reminder records success and doesn't run its remaining steps. A message from an untrusted external person never ends a turn and waits. ## Multi-subscriber delivery -Multiple clients can attach to the same session simultaneously. This is how you can start a conversation in Slack, then connect to the same session from the CLI to watch tool calls and token usage in real time — each subscriber gets its own filtered view. +Several clients can attach to one session at once. Start a conversation in Slack, then attach from the CLI to watch tool calls and token usage live. Each subscriber gets its own filtered view. -Subscribers join with an `OutputFilter` bitmask that controls what they see: +Subscribers join with an `OutputFilter` bitmask: | Filter | What it includes | |--------|-----------------| | **Text** | Final assembled assistant replies | -| **TextStreaming** | Incremental token-by-token deltas | -| **Thinking** | Reasoning/thinking tokens | +| **TextStreaming** | Incremental token deltas | +| **Thinking** | Reasoning tokens | | **ToolCalls** | Tool call requests and results | -| **Usage** | Token counts and context window consumption | -| **Files** | File attachments produced by the LLM or tools | - -Lifecycle messages — turn completed, session title changes, errors, compaction notifications — are always delivered regardless of the filter. +| **Usage** | Token counts and context window use | +| **Files** | Attachments produced by the model or tools | +| **ProcessingState** | Busy and idle state, for typing indicators | -Adding or removing a subscriber doesn't affect other active subscribers. If a subscriber disconnects (the Slack adapter restarts, a CLI session closes), the actor detects it via a death watch and removes it from the subscriber list. No impact on the conversation. +Lifecycle messages (turn completed, title changes, errors, compaction notices) always arrive regardless of the filter. When a subscriber disconnects, the actor notices through a death watch and drops it. The conversation doesn't notice. ## Compaction -Sessions can run for hours or days. Without compaction, the [context window](https://platform.openai.com/docs/guides/context-window) fills up and the model starts losing earlier context. Netclaw uses a tiered compaction sequence that trades off cost against context preservation. +Sessions can run for days. Without compaction the [context window](https://platform.openai.com/docs/models) fills and the model loses its earlier context. Compaction starts when input tokens pass a fraction of the model's context window: `Session.Tuning.CompactionThreshold`, 0.75 by default. + +It runs in order: + +1. **Clear tool results.** Old tool output is replaced with placeholders, keeping the 3 most recent tool results in full (`KeepRecentToolResults`). No LLM call. +2. **Extractive reduction.** The oldest messages are dropped, keeping the most recent 6 (`KeepRecentMessages`). If what's left is still more than half the context window, the keep count is halved, down to a minimum of 2. +3. **Compaction notes.** One best-effort LLM call, on the Compaction model and failing over to the Fallback model when one is configured, reads the discarded messages and writes notes, which are inserted as a user-role message ahead of the kept ones. If the call fails or returns nothing, the extractive result stands. + +Tool calls stay paired with their results. After compaction the session saves a snapshot, resets its token counter, queues the notes as a [memory](/architecture/memory-model/#compaction-and-memory) checkpoint, and returns to Ready, or to Processing if messages are waiting. The static context layers are rendered again on the next call, as [context structure](/architecture/context-structure/) explains. + +## Storage on disk + +New sessions get a versioned storage envelope under `~/.netclaw/sessions/-/`. Directories appear as they're needed: + +| Path | What it's for | +|------|---------------| +| `workspace/` | The session's working directory (`session_dir` in its context) | +| `attachment-staging/` | Untrusted uploads, before they're copied into the workspace | +| `artifacts/` | Outputs the user or parent session should keep | +| `tmp/` | Windows only. On Linux and macOS the temp folder is `/tmp/netclaw-//p`, outside the envelope and cleaned up by the OS, not when the session ends. netclaw sets `TMPDIR`, `TMP` and `TEMP` for child processes to point at it. Keep anything that must last in `artifacts/`. | +| `worktrees/` | Git worktrees. The agent adds one with `git worktree add`, then calls `set_working_directory`. | +| `logs/session.log` | The log for the run. Tool-call arguments are redacted, and a long string such as a file body shows as a length and hash | +| `subagents//` | The same layout for each subagent run, with its own `artifacts/` and `logs/`. It also holds `task.md` (the agent, requester and brief), `result.md` (written when the run ends) and, for a reply cut off at the output limit, `artifacts/truncated-output.md` | -Compaction triggers when the input token count crosses a threshold — a configurable percentage of the model's context window, set via `CompactionThreshold` in `netclaw.json`. +Sessions that existed before 0.27.0 keep their old paths. netclaw doesn't move or rename their files. -**Phase 1: Tool result clearing.** The cheapest step — no LLM call required. Old tool results get replaced with placeholders while keeping the N most recent tool interactions in full detail. This preserves the reasoning chain ("I called this tool because...") while dropping the bulky output. Often sufficient on its own. +The database stores each session's folder path. If you move or restore the data directory somewhere else, netclaw re-roots the stored path onto the current `sessions/` directory and logs one warning naming the old and new path. -**Phase 2: Memory flush.** An LLM call extracts key facts, decisions, and action items from the conversation and persists them to durable memory storage. This happens *before* the lossy summarization step so that important context survives even if the summary misses it. +:::caution +A pre-0.27 binary can't resume a session that has a versioned storage binding. If you downgrade, sessions created under 0.27 or later won't resume. See [Upgrade netclaw](/guides/upgrading/) before moving backward, and [Data directory layout](/reference/data-directory/) for the full tree. +::: -**Phase 3: Structured summarization.** Another LLM call produces a structured summary of the conversation history covering the primary request, key decisions, pending tasks, and what to do next. The summary replaces the compacted messages. If a prior summary exists, it gets merged iteratively rather than replaced. +## Recovery and restart -Tool call/result pairs stay atomic during compaction. A tool call is never orphaned from its result. The compaction boundary always aligns to a user-message boundary to keep the conversation coherent. +The session journal lives in SQLite through [Akka.Persistence](https://getakka.net/articles/persistence/event-sourcing.html). It records more than turns: accepted input and who sent it, tool batches and their calls, tool approval requests and decisions, sub-agent runs and parked consent requests, compaction, title changes, and background job reaping. A journal or snapshot write that waits on SQLite times out at the connection's `Default Timeout` plus 10 seconds, and a failed save during a turn posts an error that says some actions may already have run. Snapshots are taken every 20 events and after every compaction. -After compaction, the session takes a persistence snapshot, resets its token counter, and returns to Ready (or Processing, if there are buffered messages waiting). +Recovery is automatic. When a session starts, from a restart, a crash or a lazy activation, it replays its journal from the latest snapshot and goes to Ready. Accepted input is journaled before it's acknowledged, so a crash doesn't lose a message you've been told arrived. -## Recovery and warm restart +Restarts come in two kinds. When you edit `netclaw.json`, the daemon drains active sessions, saves a manifest of them, restarts in-process, and warms those sessions back up before normal traffic resumes. A drain stops a tool loop at the next safe point and tells the person ("stopping" for `netclaw daemon stop`, "restarting" otherwise). A turn that stopped after its first tool batch resumes by itself after the restart, as long as no text reached the person, nothing else is waiting and every call has a result. Otherwise the person gets the stop notice. A graceful stop of the whole daemon is bounded to 30 seconds. Sessions that were idle stay cold and recover when someone talks to them. -Sessions persist their state to [SQLite](https://www.sqlite.org/docs.html) via [Akka.Persistence](https://getakka.net/articles/persistence/event-sourcing.html). Events (`TurnRecorded`, `SessionTitleSet`, `SessionCompacted`) are journaled. Snapshots capture the full session state periodically and after every compaction. +A running sub-agent parks and resumes under the original requester's authority. One that can't park in time ends as `owner_restarted`, and its tools don't run again. A graceful restart with an active child can take about 18 seconds longer. -Recovery is automatic. When a session actor starts — whether from a daemon restart, a crash, or a lazy activation — it replays its journal from the latest snapshot forward and transitions to Ready. +## Passivation, idle behavior and background work -For config-triggered restarts (you edited `netclaw.json`, the daemon reloads), the daemon goes further. Before shutting down, it persists a manifest of all active sessions. On startup, those sessions warm proactively — they recover from persistence before normal traffic resumes. The first message after restart includes a transient continuity notice so the model knows it's resuming, not starting fresh. +Passivation keeps the daemon's footprint bounded. After `Session.IdleTimeout` with nothing happening, the session distills memory one last time, takes a snapshot and stops. A new message during the 5-second grace period cancels it. -Sessions that were idle at restart time stay cold. They recover lazily when someone talks to them again. +Three things change the picture: -## Passivation +- **Running background jobs defer it.** A session with a live background job stays resident. Once the job is reaped or finished, the next idle timeout passivates it. +- **Subscribers don't.** A client attached to an idle session doesn't keep it alive. The channel binding stays subscribed until the session commits to stop, then drains its output and shuts down with it. +- **Pending approvals don't.** Approval state is journaled, so an idle session waiting on a person passivates, and the answer rehydrates it and resumes the parked tool batch. +- **Running sub-agents do.** A session with a live or unreported child run stays resident. A child parked on a consent request doesn't hold it, because the request is in the run record and a click wakes the session. -Passivation (idle shutdown) keeps the daemon's memory footprint bounded. Without it, every conversation you've ever had stays resident in memory as a live actor. +[Background jobs](/reference/tools/) (`shell_execute` with background mode) belong to the session that started them, and passivation kills any that are still running. They don't survive a daemon restart either. The next time that session wakes, it sees the job listed as reaped, with its output log still readable. Work that has to outlive the conversation belongs in a [reminder](/configuration/reminders/). -A new message during the passivation window cancels the shutdown and the session resumes normally. The session also defers passivation if subscribers are still connected or if tool interactions are pending — it won't shut down while someone is actively watching or waiting for a decision. +A session can also delegate to a [subagent](/skills/custom-subagents/): a child actor that runs one task in the background and never talks to the user. `spawn_agent` returns a run ID at once, the result arrives in a later turn as a notice that names `result.md`, and `check_agent_run` reads status or cancels. The child can't call `spawn_agent`, `check_agent_run` or `attach_file`, and can't write memory directly. -## Keep reading +## Related pages -- [Architecture Overview](/architecture/overview/) — the three-boundary design and actor internals -- [Context Structure](/architecture/context-structure/) — the layered context each turn assembles, and how it's ordered for caching -- [Design Philosophy](/architecture/design-philosophy/) — why sessions are transport-agnostic -- [Security Architecture](/architecture/security-model/) — how audience dispositions scope what a session can do -- [`netclaw sessions`](/cli/sessions/) — CLI commands for listing and resuming sessions +- [Architecture overview](/architecture/overview/) - the daemon, the actor system and the boundaries between them +- [Context structure](/architecture/context-structure/) - the layers each turn assembles, and how they're ordered for caching +- [Memory model](/architecture/memory-model/) - how a session's transcript becomes memories +- [`netclaw sessions`](/cli/sessions/) - list and resume sessions from the CLI +- [Configuration reference](/configuration/reference/) - every `Session` setting ## Resources -- [Actor model](https://en.wikipedia.org/wiki/Actor_model) — the concurrency model behind session actors -- [Akka.NET Persistence](https://getakka.net/articles/persistence/event-sourcing.html) — journal and snapshot recovery -- [SQLite](https://www.sqlite.org/docs.html) — the persistence backend -- [Slack Socket Mode](https://api.slack.com/apis/socket-mode) — how the Slack adapter connects -- [Discord.Net](https://discordnet.dev/) — the .NET library behind the Discord adapter +- [Actor model](https://en.wikipedia.org/wiki/Actor_model) - the concurrency model behind session actors +- [Akka.NET persistence](https://getakka.net/articles/persistence/event-sourcing.html) - journal and snapshot recovery +- [SQLite](https://www.sqlite.org/docs.html) - the persistence backend +- [Slack Socket Mode](https://api.slack.com/apis/socket-mode) - how the Slack adapter connects +- [Discord.Net](https://discordnet.dev/) - the .NET library behind the Discord adapter diff --git a/src/content/docs/comparisons/hermes-agent.md b/src/content/docs/comparisons/hermes-agent.md index 0eb76aa..14d39b9 100644 --- a/src/content/docs/comparisons/hermes-agent.md +++ b/src/content/docs/comparisons/hermes-agent.md @@ -1,19 +1,19 @@ --- -title: "Netclaw vs. Hermes Agent" +title: "netclaw vs. Hermes Agent" description: "Where we agree with Hermes Agent, where we disagree, and why it matters." --- Nous Research logo -[Hermes Agent](https://hermes-agent.nousresearch.com/) arrived at many of the same instincts as netclaw — small model support, automatic memory synthesis, self-hosted by default. We respect that. These are important beliefs and Nous Research got them right. +[Hermes Agent](https://hermes-agent.nousresearch.com/) arrived at many of the same instincts as netclaw: small model support, automatic memory synthesis, self-hosted by default. We respect that. These are important beliefs and Nous Research got them right. But we draw different conclusions about how much autonomy an agent should have over its own capabilities, how much background work it should do without being asked, and what the right relationship is between the agent and the human operating it. ## What we agree on -**Small models work.** Both projects prove you don't need frontier models for production agent workloads. Hermes runs great on small models — that's exactly the same philosophy we've got. +**Small models work.** Both projects prove you don't need frontier models for production agent workloads. Hermes runs great on small models, and that's exactly the philosophy we've got. -**Memory matters.** Automatic memory formation and cross-session recall are essential, not optional. Both projects invested heavily here. +**Memory matters.** Automatic memory formation and cross-session recall are essential, not optional. Both projects invested heavily here. netclaw's recall is local semantic search: memories are embedded on your own CPU with an ONNX model, and a second small model gates weak matches out so most turns inject nothing. No cloud embedding service is involved. **Self-hosted is the right default.** Your data stays on your infrastructure. Both projects are designed for local deployment. @@ -23,7 +23,7 @@ On these fundamentals, they're right. Hermes Agent's tagline is "The Agent That Grows With You." The core idea: the agent creates skills autonomously, evaluates their performance, archives underperformers, consolidates overlapping ones, and improves over time with minimal human involvement. The agent manages its own capability surface. -Netclaw inverts this. The human decides what the agent knows how to do. Skills come from [curated feeds](/skills/skill-feeds/) managed by your organization, not from the agent's autonomous judgment. The agent can't acquire new capabilities without a human or a managed feed providing them. +netclaw inverts this. The human decides what the agent knows how to do. Skills come from your own skills directory and from [curated feeds](/skills/skill-feeds/) managed by your organization, not from the agent's autonomous judgment. The agent can write or edit a skill with its `skill_manage` tool, but nothing in netclaw scores, archives, consolidates, or rewrites skills in the background, and a human or a managed feed is still the source for everything else it learns. This is a trust boundary, not a capability gap. @@ -36,39 +36,39 @@ Hermes Agent shipped a 2,200-line "Curator" system (April 2026) that: - Spawns auxiliary LLM reviews to propose skill consolidations - Requires users to manually specify which skills the system *shouldn't* auto-archive -That last point inverts the trust relationship. The agent acts by default; the human intervenes to prevent. If you don't notice a skill got archived, it's gone. +That last point inverts the trust relationship. The agent acts by default, and the human intervenes to prevent. If you don't notice a skill got archived, it's gone. -Netclaw's approach is opt-in. Skills are version-controlled, synced from feeds, and managed by humans. The agent uses what it's given. No invisible skill creation, no autonomous archival, no "oops the agent deleted something I needed." +netclaw's approach is opt-in. Skills are version-controlled, synced from feeds, and managed by humans. The agent can write a skill with `skill_manage`, and it lands in your skills directory like any other. There's no background skill creation, no autonomous archival, no "oops the agent deleted something I needed." ## The token cost of autonomy -Every background process the Curator runs — usage tracking, lifecycle transitions, auxiliary LLM reviews for consolidation proposals — is an inference call. On self-hosted hardware, that's GPU time you're already paying for. On a paid inference provider like [OpenRouter](https://openrouter.ai/) or [Nous Portal](https://portal.nousresearch.com/), it's real money on your bill. +Every background process the Curator runs (usage tracking, lifecycle transitions, auxiliary LLM reviews for consolidation proposals) is an inference call. On self-hosted hardware, that's GPU time you're already paying for. On a paid inference provider like [OpenRouter](https://openrouter.ai/) or [Nous Portal](https://portal.nousresearch.com/), it's real money on your bill. -Netclaw's memory sidecar runs one LLM call per session idle timeout to distill observations. The Curator runs continuous background evaluations across all skills. The difference compounds. +netclaw's memory observer makes one LLM call when a session goes idle (or every five turns in a long one) to distill observations, and a curation call happens mainly when a new memory looks like a near-duplicate of one already stored, or when the rules can't decide. Embedding and relevance scoring run locally and use no provider tokens at all. The Curator runs continuous background evaluations across all skills. The difference compounds. -Worth noting: [Nous Research sells inference](https://portal.nousresearch.com/models) as a core part of their business — [20 models available via API](https://pricepertoken.com/pricing-page/provider/nousresearch). An agent architecture that burns more tokens through autonomous background processes aligns with that revenue model. That's not a flaw in their design — it's incentive alignment. +Worth noting: [Nous Research sells inference](https://portal.nousresearch.com/models) as a core part of their business, with [20 models available via API](https://pricepertoken.com/pricing-page/provider/nousresearch). An agent architecture that burns more tokens through autonomous background processes aligns with that revenue model. That's not a flaw in their design. It's incentive alignment. ## Release velocity Hermes Agent publishes impressive numbers per release: 864 commits, 588 merged PRs, 128K insertions in v0.13.0 alone. They present this as a feature. -We see it as a risk signal. Hundreds of PRs per release means massive churn, harder review, and more surface for regressions. The [biggest threat to any project is its own complexity](/architecture/design-philosophy/#simplicity-is-a-feature). Netclaw ships what's necessary and nothing more. +We see it as a risk signal. Hundreds of PRs per release means massive churn, harder review, and more surface for regressions. The [biggest threat to any project is its own complexity](/architecture/design-philosophy/#simplicity-is-a-feature). netclaw ships what's necessary and nothing more. ## Security posture -Hermes Agent has no audience disposition system. There's no way to scope what the agent can do based on who's talking to it or which channel the message came from. No equivalent to netclaw's [four-layer security model](/architecture/security-model/#four-layers-not-one) or [approval gates](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch). +Hermes Agent has no audience system. There's no way to scope what the agent can do based on who's talking to it or which channel the message came from. No equivalent to netclaw's [four-layer security model](/architecture/security-model/#four-layers-not-one) or [approval gates](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch). -For personal use on your own machine, this is probably fine. For team or organizational deployment — where multiple people with different trust levels interact with the same agent — it's a gap. Netclaw's [audience dispositions](/architecture/design-philosophy/#lightweight-access-control-without-an-identity-provider) exist precisely for this use case. +For personal use on your own machine, this is probably fine. For team or organizational deployment, where multiple people with different levels of access interact with the same agent, it's a gap. netclaw's [audiences](/architecture/design-philosophy/#lightweight-access-control-without-an-identity-provider) exist precisely for this use case. ## MCP integration -Hermes Agent doesn't make [MCP](https://modelcontextprotocol.io/) a first-class citizen. Netclaw does — with [progressive tool disclosure](/architecture/design-philosophy/#mcp-first-tooling), server-level and tool-level grants, and a [two-gate security model](/architecture/security-model/#mcp-security-two-gate-model) for external tool servers. +Hermes Agent doesn't make [MCP](https://modelcontextprotocol.io/) central to its design. netclaw does: [progressive tool disclosure](/architecture/design-philosophy/#mcp-first-tooling), server-level and tool-level grants, OAuth for MCP servers (including pre-registered confidential clients), server prompts you can invoke as skills, and a [two-gate security model](/architecture/security-model/#mcp-security-two-gate-model) for external tool servers. MCP is how you give non-technical users agent capabilities without shell access. It's where the ecosystem is heading. Building around it now means less retrofitting later. ## What Hermes Agent does better -The autonomous learning loop is genuinely innovative. It reduces operator toil — the agent gets better at its job without you managing skill inventories. +The autonomous learning loop is genuinely innovative. It reduces operator toil: the agent gets better at its job without you managing skill inventories. Hermes also has strong research backing from Nous Research, and their model alignment work is excellent. Small model performance is comparable to netclaw's. @@ -80,16 +80,16 @@ If you want an agent that actively improves itself with minimal human interventi **Choose netclaw** for human control over agent capabilities, [audience-scoped security](/architecture/security-model/#audience-as-the-trust-primitive), MCP-first tooling, and a smaller codebase where you can see exactly what the agent is doing and why. -## Keep reading +## Related pages -- [Design Philosophy](/architecture/design-philosophy/) — why simplicity and human control matter -- [Security Architecture](/architecture/security-model/) — audience dispositions and approval gates -- [Skills Overview](/skills/overview/) — how netclaw's curated skill system works -- [Memory Model](/architecture/memory-model/) — netclaw's approach to automatic memory +- [Design philosophy](/architecture/design-philosophy/) - why simplicity and human control matter +- [Security architecture](/architecture/security-model/) - audience dispositions and approval gates +- [Skills overview](/skills/overview/) - how netclaw's curated skill system works +- [Memory model](/architecture/memory-model/) - netclaw's approach to automatic memory -## Sources +## Resources -- [Hermes Agent](https://hermes-agent.nousresearch.com/) — Nous Research's autonomous agent -- [Nous Portal — Model API](https://portal.nousresearch.com/models) — Nous Research's inference API -- [Nous Research API Pricing](https://pricepertoken.com/pricing-page/provider/nousresearch) — 20 models, $0.02–$1.00 per million input tokens -- [Nous Research on OpenRouter](https://openrouter.ai/nousresearch) — API access via OpenRouter +- [Hermes Agent](https://hermes-agent.nousresearch.com/) - Nous Research's autonomous agent +- [Nous Portal model API](https://portal.nousresearch.com/models) - Nous Research's inference API +- [Nous Research API pricing](https://pricepertoken.com/pricing-page/provider/nousresearch) - 20 models, $0.02 to $1.00 per million input tokens +- [Nous Research on OpenRouter](https://openrouter.ai/nousresearch) - API access via OpenRouter diff --git a/src/content/docs/comparisons/openclaw.md b/src/content/docs/comparisons/openclaw.md index c309060..b1f289b 100644 --- a/src/content/docs/comparisons/openclaw.md +++ b/src/content/docs/comparisons/openclaw.md @@ -1,51 +1,51 @@ --- -title: "Netclaw vs. OpenClaw" -description: "Why we built netclaw when OpenClaw already exists — and what we do differently." +title: "netclaw vs. OpenClaw" +description: "Why we built netclaw when OpenClaw already exists, and what we do differently." --- OpenClaw logo [OpenClaw](https://openclaw.ai/) pioneered this category: an always-on AI agent that lives in your communication channels, acts autonomously in response to messages and timers, and runs without anyone sitting at a computer. 360K+ GitHub stars, dozens of platform integrations, a massive community. That vision inspired everything we've built. -We built netclaw because OpenClaw's execution philosophy — ship everything, integrate everything, build the biggest ecosystem possible — creates security and complexity debt. The vision is right. The philosophy is wrong. +We built netclaw because OpenClaw's execution philosophy of shipping everything, integrating everything, and building the biggest ecosystem possible creates security and complexity debt. The vision is right. The philosophy is wrong. ## Breadth vs. simplicity -OpenClaw competes on scale — stars, contributor count, lines of code, integration breadth. Their pitch is the ecosystem: 50+ platform integrations, 700+ community skills in [ClawHub](https://hub.openclaw.ai/), 1,900+ contributors, 430K+ lines of code. +OpenClaw competes on scale: stars, contributor count, lines of code, integration breadth. Their pitch is the ecosystem: 50+ platform integrations, 700+ community skills in [ClawHub](https://hub.openclaw.ai/), 1,900+ contributors, 430K+ lines of code. -Netclaw competes on simplicity. Our codebase is deliberately small. Our configuration footprint is minimal. When we're tempted to add a feature, the first question is: can this live in an [MCP server](/architecture/design-philosophy/#mcp-first-tooling) someone else maintains instead? +netclaw competes on simplicity. Our codebase is deliberately small. Our configuration footprint is minimal. When we're tempted to add a feature, the first question is: can this live in an [MCP server](/architecture/design-philosophy/#mcp-first-tooling) someone else maintains instead? Every feature you add is a feature you have to maintain, test, secure, and explain. Most agent projects treat feature count as proof of progress. We think it's a liability. The [philosophy of no](/architecture/design-philosophy/#the-philosophy-of-no) covers why. ## The security record -OpenClaw starts permissive and adds restrictions after security incidents. In March 2026, [nine CVEs were disclosed in four days](https://openclawai.io/blog/openclaw-cve-flood-nine-vulnerabilities-four-days-march-2026) — including [CVE-2026-32922](https://www.armosec.io/blog/cve-2026-32922-openclaw-privilege-escalation-cloud-security/), a CVSS 9.9 privilege escalation. As of April 2026, [138+ CVEs have been disclosed](https://www.betterclaw.io/blog/openclaw-security-2026) and tracked by the community ([jgamblin/OpenClawCVEs](https://github.com/jgamblin/OpenClawCVEs/)). A February 2026 internet-wide scan found [135,000+ publicly accessible OpenClaw instances](https://conscia.com/blog/the-openclaw-security-crisis/), 63% running without gateway authentication. +OpenClaw starts permissive and adds restrictions after security incidents. In March 2026, [nine CVEs were disclosed in four days](https://openclawai.io/blog/openclaw-cve-flood-nine-vulnerabilities-four-days-march-2026), including [CVE-2026-32922](https://www.armosec.io/blog/cve-2026-32922-openclaw-privilege-escalation-cloud-security/), a CVSS 9.9 privilege escalation. As of April 2026, [138+ CVEs have been disclosed](https://www.betterclaw.io/blog/openclaw-security-2026) and tracked by the community ([jgamblin/OpenClawCVEs](https://github.com/jgamblin/OpenClawCVEs/)). A February 2026 internet-wide scan found [135,000+ publicly accessible OpenClaw instances](https://conscia.com/blog/the-openclaw-security-crisis/), 63% running without gateway authentication. This isn't bad luck. Permissive-first architecture creates more attack surface and more trust boundaries to misconfigure. -Netclaw works the other way around. During [`netclaw init`](/cli/init/), you choose an [audience disposition](/architecture/security-model/#audience-as-the-trust-primitive) that sets the baseline for what the agent can do. The [four-layer security model](/architecture/security-model/#four-layers-not-one) — `ShellCommandPolicy`, `ToolPathPolicy`, `ToolAccessPolicy`, `IToolApprovalService` — stacks independent checks so no single misconfiguration gives the agent the keys. Default-deny, fail-closed, [approval gates](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch) for anything destructive. +We work the other way around. During [`netclaw init`](/cli/init/), you choose an [audience](/architecture/security-model/#audience-as-the-trust-primitive) that sets the baseline for what the agent can do. The [four-layer security model](/architecture/security-model/#four-layers-not-one) (hard denies, path restrictions, audience-scoped grants, and approval gates) stacks independent checks so no single misconfiguration gives the agent the keys. Since 0.27.1 one authorizer handles every tool and shell call, which closed several ways around approval. Default-deny, fail-closed, [approval gates](/architecture/security-model/#human-in-the-loop-as-a-layer-not-a-crutch) for anything destructive. ## The skills marketplace problem -OpenClaw's [ClawHub](https://hub.openclaw.ai/) is an open marketplace where anyone can publish agent skills. In February 2026, [Koi Security audited 2,857 ClawHub skills](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html) and found 341 that were actively malicious — a **12% malware rate**. 335 of those traced to a single coordinated campaign called [ClawHavoc](https://www.antiy.net/p/clawhavoc-analysis-of-large-scale-poisoning-campaign-targeting-the-openclaw-skill-market-for-ai-agents/). A follow-up scan found [824+ malicious skills out of 10,700+](https://www.esecurityplanet.com/threats/hundreds-of-malicious-skills-found-in-openclaws-clawhub/) total. +OpenClaw's [ClawHub](https://hub.openclaw.ai/) is an open marketplace where anyone can publish agent skills. In February 2026, [Koi Security audited 2,857 ClawHub skills](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html) and found 341 that were actively malicious, a **12% malware rate**. 335 of those traced to a single coordinated campaign called [ClawHavoc](https://www.antiy.net/p/clawhavoc-analysis-of-large-scale-poisoning-campaign-targeting-the-openclaw-skill-market-for-ai-agents/). A follow-up scan found [824+ malicious skills out of 10,700+](https://www.esecurityplanet.com/threats/hundreds-of-malicious-skills-found-in-openclaws-clawhub/) total. [Snyk's broader ToxicSkills study](https://snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub/) across agent skill registries found that 36% of skills contain security flaws, with active malicious payloads capable of going [from SKILL.md to shell access in three lines of markdown](https://snyk.io/articles/skill-md-shell-access/). -An open, unaudited skill marketplace is an inherently dangerous idea when skills can execute shell commands and call external APIs. The attack surface isn't a bug — it's the architecture. +An open, unaudited skill marketplace is an inherently dangerous idea when skills can execute shell commands and call external APIs. The attack surface isn't a bug, it's the architecture. -Netclaw doesn't have a public skills marketplace. Skills come from [curated feeds](/skills/skill-feeds/) managed by your organization via [SkillServer](/skills/skill-server/) — a self-hosted registry where you control what your agents can learn. Skills are synced, version-controlled, and SHA-256 verified on download. New or changed skills go through a content scan for prompt injection before they're accepted. You can also connect to trusted third-party feeds, but the decision to trust a feed is yours, not the agent's. +netclaw doesn't have a public skills marketplace. Skills come from your own skills directory, from [curated feeds](/skills/skill-feeds/) managed by your organization via [SkillServer](/skills/skill-server/) (a self-hosted registry where you control what your agents can learn), and from external sources you configure. The agent can also write and edit skills itself with the `skill_manage` tool, so a skill can appear because you or the agent put it there, but nothing syncs in from an open registry unless you point netclaw at one. Feed skills are synced, version-controlled, and SHA-256 verified on download. Skill content isn't scanned for prompt injection yet. The scanner is disabled in 0.27.1, so the feeds you subscribe to are your trust boundary. You can connect to trusted third-party feeds, but adding one is your decision, not the agent's. ## Model requirements OpenClaw's documentation recommends using "the strongest latest-generation model available," which in practice means frontier models at $10+ per million input tokens and requires sending your data to a cloud inference provider. -Netclaw runs production workloads on [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B), tested down to Qwen 3.5 9B. [Progressive tool disclosure](/architecture/design-philosophy/#mcp-first-tooling) reduces prompt complexity so smaller models can navigate tool graphs without misfiring. A $3,000 GPU with 64GB VRAM runs local inference indefinitely — bounded capital expenditure, data never leaves your infrastructure. +netclaw runs production workloads on [Qwen 3.6 27B](https://huggingface.co/Qwen/Qwen3.6-27B), tested down to Qwen 3.5 9B. [Progressive tool disclosure](/architecture/design-philosophy/#mcp-first-tooling) reduces prompt complexity so smaller models can navigate tool graphs without misfiring. A $3,000 GPU with 64GB VRAM runs local inference indefinitely: bounded capital expenditure, and your data never leaves your infrastructure. That includes memory. netclaw's [semantic recall](/architecture/memory-model/) runs on two small ONNX models on your own CPU, with no cloud embedding service in the loop. ## What OpenClaw does better OpenClaw genuinely does some things better: -- **Platform coverage.** 12+ messaging platforms vs. netclaw's two (Slack and Discord). If you need Teams, Signal, Telegram, or WhatsApp today, OpenClaw has them. +- **Platform coverage.** 12+ messaging platforms vs. netclaw's three (Slack, Discord, and Mattermost). If you need Teams, Signal, Telegram, or WhatsApp today, OpenClaw has them. - **Community size.** 1,900+ contributors, extensive third-party tutorials, and a massive ecosystem of community-built integrations. - **Maturity.** OpenClaw has been around longer and has more production deployments. The rough edges have been found and reported (sometimes painfully). @@ -55,27 +55,28 @@ If you need maximum platform coverage and a large community right now, use OpenC **Choose OpenClaw if** you need broad platform support across many messaging services, prefer a large community with extensive third-party resources, and are comfortable managing the security surface yourself. -**Choose netclaw** for a smaller codebase you can audit, security that defaults to deny instead of permissive-plus-patches, local inference that keeps your data on your hardware, and control over your agent's capabilities. - -## Keep reading - -- [Design Philosophy](/architecture/design-philosophy/) — why simplicity is a feature -- [Security Architecture](/architecture/security-model/) — the four-layer model in detail -- [Skills Overview](/skills/overview/) — how netclaw's curated skill system works -- [Self-Hosted Providers](/configuration/self-hosted-providers/) — running local inference - -## Sources - -- [Nine CVEs in Four Days — OpenClaw Blog (March 2026)](https://openclawai.io/blog/openclaw-cve-flood-nine-vulnerabilities-four-days-march-2026) -- [OpenClaw Security 2026: 138+ CVEs — BetterClaw](https://www.betterclaw.io/blog/openclaw-security-2026) -- [CVE-2026-32922: CVSS 9.9 Privilege Escalation — ARMO](https://www.armosec.io/blog/cve-2026-32922-openclaw-privilege-escalation-cloud-security/) -- [jgamblin/OpenClawCVEs — Community CVE Tracker](https://github.com/jgamblin/OpenClawCVEs/) -- [The OpenClaw Security Crisis — Conscia](https://conscia.com/blog/the-openclaw-security-crisis/) -- [341 Malicious ClawHub Skills — The Hacker News (Feb 2026)](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html) -- [OpenClaw Ships Verified Skill Screening — Reuters (March 2026)](https://www.tradingview.com/news/reuters.com,2026-03-26:newsml_ACN105904:0-openclawd-ships-verified-skill-screening-after-security-researchers-find-12-of-openclaw-marketplace-skills-are-malware/) -- [ToxicSkills and ClawHavoc — Agensi](https://www.agensi.io/learn/toxicskills-clawhavoc-agent-skills-security-crisis-2026) -- [Snyk ToxicSkills Study — 36% of Skills Contain Flaws](https://snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub/) -- [From SKILL.md to Shell Access — Snyk](https://snyk.io/articles/skill-md-shell-access/) -- [ClawHavoc Analysis — Antiy Labs](https://www.antiy.net/p/clawhavoc-analysis-of-large-scale-poisoning-campaign-targeting-the-openclaw-skill-market-for-ai-agents/) -- [Hundreds of Malicious Skills — eSecurity Planet](https://www.esecurityplanet.com/threats/hundreds-of-malicious-skills-found-in-openclaws-clawhub/) -- [ClawHub Incident — Termdock](https://www.termdock.com/en/blog/clawhub-malicious-skills-incident) +**Choose netclaw** for a smaller codebase you can audit, security that defaults to deny instead of permissive-plus-patches, local inference and local memory that keep your data on your hardware, and a daemon that runs natively on Linux, macOS, and Windows. + +## Related pages + +- [Design philosophy](/architecture/design-philosophy/) - why simplicity is a feature +- [Security architecture](/architecture/security-model/) - the four-layer model in detail +- [Memory model](/architecture/memory-model/) - how recall works without a cloud service +- [Skills overview](/skills/overview/) - how netclaw's curated skill system works +- [Self-hosted providers](/configuration/self-hosted-providers/) - running local inference + +## Resources + +- [Nine CVEs in Four Days (OpenClaw Blog, March 2026)](https://openclawai.io/blog/openclaw-cve-flood-nine-vulnerabilities-four-days-march-2026) +- [OpenClaw Security 2026: 138+ CVEs (BetterClaw)](https://www.betterclaw.io/blog/openclaw-security-2026) +- [CVE-2026-32922: CVSS 9.9 Privilege Escalation (ARMO)](https://www.armosec.io/blog/cve-2026-32922-openclaw-privilege-escalation-cloud-security/) +- [jgamblin/OpenClawCVEs: Community CVE Tracker](https://github.com/jgamblin/OpenClawCVEs/) +- [The OpenClaw Security Crisis (Conscia)](https://conscia.com/blog/the-openclaw-security-crisis/) +- [341 Malicious ClawHub Skills (The Hacker News, Feb 2026)](https://thehackernews.com/2026/02/researchers-find-341-malicious-clawhub.html) +- [OpenClaw Ships Verified Skill Screening (Reuters, March 2026)](https://www.tradingview.com/news/reuters.com,2026-03-26:newsml_ACN105904:0-openclawd-ships-verified-skill-screening-after-security-researchers-find-12-of-openclaw-marketplace-skills-are-malware/) +- [ToxicSkills and ClawHavoc (Agensi)](https://www.agensi.io/learn/toxicskills-clawhavoc-agent-skills-security-crisis-2026) +- [Snyk ToxicSkills Study: 36% of Skills Contain Flaws](https://snyk.io/blog/toxicskills-malicious-ai-agent-skills-clawhub/) +- [From SKILL.md to Shell Access (Snyk)](https://snyk.io/articles/skill-md-shell-access/) +- [ClawHavoc Analysis (Antiy Labs)](https://www.antiy.net/p/clawhavoc-analysis-of-large-scale-poisoning-campaign-targeting-the-openclaw-skill-market-for-ai-agents/) +- [Hundreds of Malicious Skills (eSecurityPlanet)](https://www.esecurityplanet.com/threats/hundreds-of-malicious-skills-found-in-openclaws-clawhub/) +- [ClawHub Incident (Termdock)](https://www.termdock.com/en/blog/clawhub-malicious-skills-incident) diff --git a/src/content/docs/security/approvals.md b/src/content/docs/security/approvals.md new file mode 100644 index 0000000..e6105af --- /dev/null +++ b/src/content/docs/security/approvals.md @@ -0,0 +1,195 @@ +--- +title: "How tool approvals work" +description: "How netclaw decides whether a tool call runs, prompts you, gets corrected or is denied, what each answer at the prompt saves, and how to find out why a call prompted." +--- + +Every tool call the agent makes goes through one authorizer before it runs. The authorizer reads the call, the [audience](/security/security-model/#trust-audiences) it came from and your configuration, then returns one of four outcomes. Most calls are allowed without you seeing anything. A few reach you as a prompt in Slack, Discord, Mattermost, the TUI or the web client, and the call waits until you answer. + +Three things decide whether a call prompts: a **mode** you set per audience and per tool, the **grants** you've already saved, and a built-in catalog of reviewed read-only commands that never prompt. Hard-deny rules and protected paths sit above all three. No mode, grant or answer can override them. + +This page explains the model. To stop a prompt, see [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/). For the commands that manage saved grants, see [`netclaw approvals`](/cli/approvals/). + +## The four outcomes + +| Outcome | What happens | +|---------|--------------| +| Allowed | The tool runs. | +| Requires approval | The channel shows a prompt and the call waits for an answer. | +| Requires correction | The agent gets advice and sends a different call. You see no prompt and nothing runs. | +| Denied | The tool doesn't run, and no answer changes that. | + +A correction is the one most people haven't met. It's what you get when the call is fixable, such as a command over 900 characters that can't be shown in full on every channel, a write that belongs in the managed temporary directory, or an unquoted word the parser can't prove safe. A correction is sent only when the agent can follow it. A word whose value only exists at run time, such as `git $(echo status) origin` or `[ -n "$FOO" ]`, gets a one-time prompt instead. The agent rewrites the call and the new call goes through the authorizer from the top. + +## Modes and where you set them + +Each audience has an `ApprovalPolicy` that gives every tool one of three modes: + +| Mode | Behavior | +|------|----------| +| `Auto` | No prompt. Hard deny, path checks and corrections still apply. | +| `Approval` | Netclaw prompts unless a saved grant, a this-chat answer or the reviewed-safe catalog covers the call. | +| `Deny` | The call is always blocked and nothing is offered. | + +The policy lives at `Tools.AudienceProfiles..ApprovalPolicy` in `~/.netclaw/config/netclaw.json`: + +| Key | What it sets | +|-----|--------------| +| `DefaultMode` | The mode for any tool without a more specific entry. Defaults to `Auto`. | +| `ToolOverrides` | A mode per tool, keyed by tool name (`shell_execute`) or `server/tool` (or `server__tool`) for an MCP tool. | +| `McpServerDefaults` | A mode for every tool of one MCP server, including tools discovered later. | + +Keys match without regard to letter case, so `"File_Write": "Deny"` works. If two keys differ only by case, the stricter mode wins. A key that contains a colon (such as `file_write:control-plane`) or has no value is ignored, and the tool keeps the audience default. The daemon logs a warning at startup. `netclaw doctor` reports a colon key as an error and a key with no value as a warning. Before 0.28.0 a colon key set the whole tool to `Auto` instead ([GHSA-85pr-m799-g776](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-85pr-m799-g776)), so on an older install remove every `ToolOverrides` key that contains a colon. + +Without any policy, Personal prompts for `shell_execute` and every other tool is `Auto`. Writes under `~/.netclaw/config/` aren't a prompt case at all: the path policy denies them before modes are read, and no override or grant changes that. Team and Public can't use shell at all. `netclaw mcp add` seeds each new MCP server with `Auto` for Personal, `Approval` for Team and `Deny` for Public. + +### How netclaw picks the mode for a call + +```mermaid +flowchart TD + A["Tool call"] --> B{{"Key in ToolOverrides?
tool name or server/tool"}} + B -- yes --> M["Use that mode"] + B -- no --> C{{"MCP tool with a
McpServerDefaults entry?"}} + C -- yes --> M + C -- no --> D{{"Personal audience and
shell_execute?"}} + D -- yes --> AP["Approval"] + D -- no --> E["DefaultMode"] +``` + +The third step is why `DefaultMode: "Auto"` doesn't silence shell on Personal. The fail-closed check runs before `DefaultMode` is read, so only a `shell_execute` entry (in any letter case) turns the prompt off. + +## Why did this call prompt? + +Follow the call from the top. The first box that answers "no" or "yes" with an outcome ends the walk. + +```mermaid +%%{init: {"flowchart": {"rankSpacing": 28, "nodeSpacing": 24}}}%% +flowchart TD + A["Tool call"] --> B{{"Profile allows
the tool?"}} + B -- no --> X1["Denied"] + B -- yes --> C{{"Hard deny, protected
path or untrusted root?"}} + C -- yes --> X1 + C -- no --> D{{"Mode for this tool"}} + D -- Deny --> X1 + D -- Auto --> OK["Allowed"] + D -- Approval --> E{{"Saved grant, this-chat
answer or reviewed-safe?"}} + E -- yes --> OK + E -- no --> F{{"Agent can fix
the call itself?"}} + F -- yes --> COR["Requires correction"] + F -- no --> G{{"Anyone can answer
in this run?"}} + G -- no --> X2["Denied:
approval_required_
unattended"] + G -- yes --> P["Prompt and wait"] +``` + +This is the simplified order. The exact rule list lives in the product's [tool authorization architecture](https://github.com/netclaw-dev/netclaw/blob/dev/docs/architecture/tool-authorization.md). For a shell call, "covered" is judged per command: in `git pull && make build`, each command needs its own coverage, and the prompt names only the ones that remain. + +To see which box a specific call stopped at, read the [policy trace](#see-why-a-call-prompted) in the daemon log. + +## What each answer does + +The prompt offers up to six options. Every client shows the same labels: Slack, Discord and Mattermost render them as buttons or lettered replies, and the TUI and web client show an inline list. In the TUI, pick with Up and Down and confirm with Enter. Esc is the same as Deny, and `Ctrl+O` shows the full command. + +| Option | Saved? | Effect | +|--------|--------|--------| +| Once | No | The exact blocked call runs once. | +| This chat | In memory only | The phrase is allowed for the rest of this session. It's gone when the session ends. | +| Always here | `tool-approvals.json` | A grant for the folder the command works on, and everything under it. Shell only. [Which folder that is](#what-always-here-covers) depends on the command. | +| This repository | `tool-approvals.json` | A grant for one Git repository and its registered worktrees. Shell only, and only when every command resolves to one registered repository. | +| Always anywhere | `tool-approvals.json` | A global grant for the phrase. For an MCP tool the label reads **Always allow this tool**. | +| Deny | No | This call doesn't run. Netclaw doesn't remember the refusal. The agent is told you declined, to try no other route to the same goal, and to ask you how to continue. | + +Here's the prompt for a `push` to a Git repository in the TUI, where all six options are on offer. + +![Netclaw chat TUI with an approval prompt for shell_execute running git push origin main, and a six-item list from Once to Deny](/screenshots/output/approval-prompt-tui.png) + +The status bar swaps to the pick-and-confirm keys while the prompt waits, and `Ctrl+O` expands a command that doesn't fit. + +Mattermost without a callback URL shows the same six choices as a lettered text reply, which is what this capture is. + +![Netclaw approval prompt in a Mattermost thread with the Tool, Action and Pattern lines and six lettered choices, A (Once) to F (Deny)](/assets/approval-prompt.png) + +Slack and Discord render the same choices as buttons, and so does Mattermost once a callback URL is set. You get six only inside a Git repository. A command run elsewhere, or by a reminder, offers five because **This repository** drops out. + +After an approved call, netclaw adds one line to the tool result the model reads: `[approval: once]`, `[approval: this chat only]`, `[approval: always in this folder]`, `[approval: always in this repo]` or `[approval: always anywhere]`. + +A saved grant belongs to one audience and one tool. A grant for `shell_execute` under Personal never covers a Team call and never covers another tool. For shell, the grant is a phrase of command words: a phrase of two or more words (`git push`) covers those words plus any later arguments, and a one-word phrase (`gh`) covers only the program, so `gh` doesn't cover `gh pr view`. A shell grant also records the Bash variable assignments the command depends on. A changed value doesn't match the old grant and prompts again. A **Once** answer covers the exact command text it was given for, including after a restart. + +### What Always here covers + +The saved folder comes from the paths the command names, not from where you were standing: + +- **A path whose last name looks like a file** (it has an extension or starts with a dot) saves the parent folder. Approving `touch /srv/app/new.txt` covers `/srv/app`, so `touch /srv/app/other.log` and `touch /srv/app/sub/x.txt` run without a prompt. +- **Any other path** saves the path itself. Approving `touch /srv/app/notes` covers `/srv/app/notes` and what's under it, so `touch /srv/app/other` prompts again. The same goes for a directory argument such as `chmod 600 /srv/app/sub` or a path that ends in `/`. +- **No path in the command** (`curl --version`) saves the working directory. + +A grant never covers a sibling folder or the parent. Netclaw decides from the spelling of the name and never looks at the disk, so it can't tell that `notes` is a file. If an Always here grant doesn't stick, check which of these your command matched with `netclaw approvals list`. + +### Why you sometimes see fewer options + +| You see | Because | +|---------|---------| +| Only Once and Deny | The parser can't prove a reusable phrase for an uncovered command, or a `..` in a shell path leaves a symlink. A command like `rm "$f"` where the loop values only exist at run time, a word built at run time such as `git $(echo status) origin` and `[ -n "$FOO" ]` are the usual causes. No grant covers these. | +| No Always here | The tool isn't shell, the directory is too shallow (`/` or `/etc`), or it's a session-owned directory. A TUI chat starts in a session directory, so a command with no path there offers no Always here. | +| No This repository | The call's commands don't all resolve to one registered Git repository, or the checkout uses `--separate-git-dir`. | +| Only Once and Deny, again | The agent retried a write under `/tmp` after a correction told it to use the managed temporary directory. | + +Only the person who started the request can answer a prompt, unless the request came from verified automation. A prompt never times out. While the session is live, a new message in the thread queues behind the prompt and doesn't dismiss it, so the way to unstick a waiting call is to answer it. If the daemon restarts or the session goes idle first, a new message abandons the waiting call. The daemon re-issues the prompt after a restart and a chat channel may still show it. The TUI doesn't redisplay it, so answer from the channel or send a new message. A sub-agent that needs consent parks. The parent gets one notice and pauses, and you answer on the prompt's buttons because a text reply like "A" resolves nothing. The park is saved in the run record, so a click after a restart still resumes the child. A prompt that nothing can answer any more (the child was cancelled or ended, or the channel can't show it) is redrawn without buttons and with the reason. On Mattermost, a sub-agent's prompt needs a `CallbackUrl`. Without one, netclaw refuses the request. + +## What never prompts + +Some calls skip the prompt even in `Approval` mode. + +- **Reviewed-safe catalog.** Diagnostic commands like `ls`, `cat`, `grep`, `rg`, `jq`, `ps`, `git status`, `git log`, `git diff` and `gh pr view`. The catalog ships with the daemon and the agent can't extend it. It never includes `git push`, `rm`, `sed`, `find`, `env`, `curl` or `sudo`. It matches command words, not flags: `git branch` is listed, so `git branch -D old` also runs without a prompt. +- **Output commands.** `echo`, `printf`, `:`, `true` and `false`. +- **Commands that run no program.** A bare assignment (`FOO=bar`) or a lone redirect (`> notes.txt`) needs no approval of its own. A redirect on another command, as in `echo hi > notes.txt`, gets the same decision as a `file_write` to that path, so on Personal it runs, and a path in a protected location is denied. That includes a redirect on a group or subshell. A redirect target that netclaw can't work out is denied (`shell_redirect_unproved`). +- **Quoted heredocs.** `cat <<'EOF'` with fixed text runs. The text is data. A program that reads it from stdin, as in `python3 - <<'EOF'`, still prompts for the program, and you can grant it. An unquoted delimiter that expands variables prompts. +- **Test builtins** with operands the parser can prove. +- **Anything a saved grant covers.** + +The catalog covers reads of any path the audience profile can read. On Personal that's every path except the protected ones. From 0.28.0 that includes the [credential locations](/security/security-model/#credential-locations), so `cat ~/.ssh/id_ed25519` and `cat ~/.aws/credentials` are denied with `shell_references_protected_path`. A project `.env` file isn't on that list and `cat .env` runs without a prompt, with matching secret names masked in the output. Before 0.28.0 the credential folders were readable too. See [what netclaw doesn't protect](/security/security-model/#what-netclaw-doesnt-protect-against). + +## What never runs, whatever you answer + +Hard deny and protected paths apply in every mode, including `Auto`. The deny list covers daemon self-kill, privilege escalation (`sudo`, `su`, `doas`), system destruction (`rm -rf /`, `mkfs`) and fork bombs. Protected paths cover `secrets.json`, webhook route files, key material, the database and daemon lifecycle files, writes to the config directory and, from 0.28.0, credential folders such as `~/.ssh` and `~/.aws`. Add your own patterns with `Tools.HardDenyPatterns`. See the [security model](/security/security-model/#layers-in-order) for the full list. + +## When nobody is there to answer + +Headless chat (`netclaw chat -p`), reminders and webhooks can't show a prompt. They follow the same rules as a chat of the same audience, with one difference: a call that would prompt is denied with `approval_required_unattended`. Saved grants and the reviewed-safe catalog still allow calls, and a tool set to `Auto` runs. "Once" and "This chat" answers don't carry over to a scheduled run. See [Run reminders, webhooks and scripts unattended](/guides/unattended-runs/). + +## See why a call prompted + +Netclaw has no separate audit store. The daemon log at `~/.netclaw/logs/daemon-.log` records each decision. The log rolls over at 10 MiB into `daemon-.1.log`, `.2.log` and so on, so search them all. Run `grep -E 'Tool authorization|Shell policy trace' ~/.netclaw/logs/daemon-*.log`. This excerpt is a headless call to `echo hi && mkdir /tmp/x` with no grants (IDs trimmed): + +```text +[WRN] Tool authorization evaluated: shell_execute outcome=Denied reason=approval_required_unattended authorizationAttemptId=auth-be49… sessionId=signalr/bb84… callId=call_0 +[INF] Shell policy trace: stage=StoredGrantMatch outcome=Uncovered reason=NoGrant candidate_id=1 executable=mkdir coverage=Uncovered scope_relation=None grant_timestamp=(null) … +[INF] Shell policy trace: stage=ReviewedSafePolicy outcome=Covered reason=ApprovalExemptSideEffect candidate_id=0 executable=echo coverage=ReviewedSafePolicy scope_relation=None grant_timestamp=(null) … +[INF] Shell policy trace: stage=Completion outcome=RequiresApproval reason=UncoveredCandidates candidate_id=(null) executable=(null) coverage= scope_relation=None … +``` + +Read it from the last row back. `Completion/RequiresApproval/UncoveredCandidates` means at least one command had no coverage. Find that candidate in the `StoredGrantMatch` rows: here `mkdir` (candidate 1) has `NoGrant`, while `echo` was covered by the catalog. In a chat the first line reads `outcome=RequiresApproval` and you get the prompt. In an unattended run it's the `Denied` line above. + +Other `reason=` values on a `Denied` line include `hard_deny_self_destructive`, `tool_not_allowed_for_audience_profile`, `shell_references_protected_path`, `shell_path_protected`, `shell_path_outside_trusted_roots` and `shell_redirect_file_tool_denied`. A `StoredGrantMatch` row with `TokenMismatch`, `ShellMismatch`, `AssignmentMismatch`, `OutsideDirectory` or `Symlink` explains a near miss, such as a grant for `git push` that doesn't match because the working directory moved. All rows for one call share an `authorizationAttemptId`. The trace holds enum facts and the executable name, never the full command, arguments or secrets. + +`netclaw doctor` also flags risky or redundant approval setup: Personal `shell_execute` on `Auto` with the host shell enabled, approval mode with shell turned off, and grants that another grant already covers (`netclaw doctor --fix` removes those). + +## Limitations + +- There's no single switch that turns approvals off. Shell is fail-closed on Personal, so a solo user needs an explicit `shell_execute` override. See [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/). +- Approvals decide which commands run. They don't limit what a command can read. A grant for `bash`, `python3` or `node` lets the agent run any code as your user. +- A prompt blocks the session until someone answers. A bot waiting on a prompt looks frozen. +- A grant that's too narrow won't re-match. Grants key on command words and on the directory, repository or "anywhere" scope you picked. +- Hard-deny rules and corrections can't be answered away, and there's no way to approve a denied call. + +## Related pages + +- [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/) - the "I want X, do this" guide +- [`netclaw approvals`](/cli/approvals/) - list, revoke and add saved grants +- [Security model](/security/security-model/) - audiences, layers, protected paths and what netclaw doesn't protect +- [MCP tool permissions](/guides/mcp-tool-permissions/) - per-server and per-tool modes +- [Run reminders, webhooks and scripts unattended](/guides/unattended-runs/) - tool access without a person present + +## Resources + +- [Tool authorization architecture](https://github.com/netclaw-dev/netclaw/blob/dev/docs/architecture/tool-authorization.md) - the rule order in the product repo +- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) - excessive agency and the other risks approvals are meant to contain +- [Principle of least privilege (NIST)](https://csrc.nist.gov/glossary/term/least_privilege) - the idea behind per-tool modes diff --git a/src/content/docs/security/hardening.md b/src/content/docs/security/hardening.md index 2a565e5..f305108 100644 --- a/src/content/docs/security/hardening.md +++ b/src/content/docs/security/hardening.md @@ -1,20 +1,27 @@ --- title: "Hardening" -description: "Lock down MCP permissions and network exposure." +description: "Lock down a netclaw install for a team, the internet or unattended use: isolation, file permissions, shell, MCP, exposure, channels and webhooks." --- -The [security model](/security/security-model/) covers what netclaw enforces by default: default-deny, [audience](/security/security-model/#trust-audiences) scoping, a four-layer invocation stack. This page covers what *you* should do on top of that, especially if you're running it for a team, exposing it to the internet, or leaving it unattended. +The [security model](/security/security-model/) covers what netclaw enforces by default. This page covers what you should do on top of that, especially if you run it for a team, expose it to the internet or leave it unattended. The sections are independent, so work through the ones that apply. -Sections are independent. Work through the ones that apply to your deployment. +All configuration lives in `~/.netclaw/config/netclaw.json` unless noted. Most keys fall back to built-in defaults when you omit them. The exception is `Tools` lists such as `AllowedTools`, file roots, `GlobalReadRoots` and `WebFetch.HttpAllowList`. Since 0.27.1 a configured list is the complete list and replaces the default, so write out every entry you want. A malformed list, such as a scalar where a list belongs, stops the daemon at startup with an error naming the key. `HardDenyPatterns` is the exception to the exception: it adds to the built-in deny list. -All configuration lives in `~/.netclaw/config/netclaw.json` unless noted otherwise. Values merge with built-in defaults, so you only need to specify what you're changing. +## Isolate netclaw from your other credentials -## File Permissions +Netclaw protects its own credential store and config. From 0.28.0 it also denies the agent's tools a fixed list of credential folders (`~/.ssh`, `~/.aws`, `~/.gnupg`, `~/.kube`, `~/.azure`, `~/.config/gh`, `~/.config/gcloud`, `~/.docker/config.json`, `~/.netrc`, `~/.git-credentials`). See [credential locations](/security/security-model/#credential-locations). It doesn't protect anything else your OS user can read, such as a project `.env` file, and approvals don't change that. Before 0.28.0 the agent can read those credential folders too if netclaw runs as you. Giving netclaw its own OS user or container is still the most useful hardening step. -Netclaw stores credentials and webhook secrets in plaintext JSON files. Lock them down: +- **A separate OS user.** Create a user for netclaw and give it only the files it needs. This is the lightest option for a native or systemd install. See [systemd deployment](/deployment/systemd/). +- **A container.** The image runs the daemon as the non-root `netclaw` user, and the agent sees only what you mount. See [Docker deployment](/deployment/docker/). + +Netclaw has no built-in kernel sandbox for shell commands yet. The work is tracked in [netclaw-dev/netclaw#2343](https://github.com/netclaw-dev/netclaw/issues/2343). The full boundary statement is in [what netclaw doesn't protect against](/security/security-model/#what-netclaw-doesnt-protect-against). + +## File permissions + +Credential values in `secrets.json` are encrypted, but the files are only as private as their permissions. Webhook route files hold HMAC secrets in plain JSON. Netclaw creates that directory as `755` and the route files as `644`, and `netclaw doctor` doesn't flag it, so run the `chmod` yourself: ```bash -# Encrypted secrets — only the daemon user should read this +# Encrypted secrets: only the daemon user should read this chmod 600 ~/.netclaw/config/secrets.json # Webhook route files contain HMAC secrets @@ -24,34 +31,39 @@ chmod 700 ~/.netclaw/config/webhooks/ chmod 700 ~/.netclaw/keys/ ``` -[`netclaw doctor`](/cli/doctor/) checks `secrets.json` permissions and flags unencrypted values. Run it after any manual config edits. +[`netclaw doctor`](/cli/doctor/) checks `secrets.json` permissions and flags unencrypted values. Run it after any manual config edit. -## Shell Access +## Shell access -Shell access is the biggest risk surface you can control. Two working modes in `~/.netclaw/config/netclaw.json`: +Shell access is the biggest risk you control. Two modes work: | Mode | Behavior | |------|----------| -| `Off` | Shell completely disabled. No `shell_execute` tool available. | -| `HostAllowed` | Shell runs directly on the host. Approval gates are your only guardrail. | +| `Off` | Shell is disabled. Every `shell_execute` call is denied with `shell_disabled`. | +| `HostAllowed` | Shell runs on the host. Approvals are your guardrail. | -A third value, `SandboxOnly`, is accepted but has no backend yet — setting it **denies shell entirely**, and `shell_execute` fails with `shell_requires_sandbox_backend`. If you see that error, this is why; use `Off` for the same effect. +A third value, `SandboxOnly`, is accepted but has no backend yet. It denies shell entirely and `shell_execute` fails with `shell_requires_sandbox_backend`. Use `Off` for that effect. -Set the mode explicitly: +To turn shell off, set both keys, the way `netclaw init` writes them: ```json { "Security": { "ShellExecutionMode": "Off" + }, + "Tools": { + "ShellMode": "Off" } } ``` -**For Team and Public [postures](/security/security-model/#deployment-postures), shell is off by default.** Only Personal posture enables it, and only with approval gates on `shell_execute`. If you're running Personal posture but don't need shell, turn it off. +`Tools.ShellMode` takes precedence. Setting only the `Security` key can leave shell on if `Tools.ShellMode` says `HostAllowed`. -### Custom Hard-Deny Patterns +Team and Public postures have shell off by default, and shell is Personal-only whatever the posture. If you run Personal posture but don't need shell, turn it off. -The built-in hard-deny list blocks `sudo`, `rm -rf ~/`, `kill`, fork bombs, and commands that would stop the daemon. Add your own patterns for anything dangerous in your environment: +### Custom hard-deny patterns + +The built-in hard-deny list blocks privilege escalation (`sudo`), `rm -rf ~/`, `mkfs`, fork bombs and commands that would stop the daemon or kill its process (`pkill netclawd`). Other `kill` commands prompt like any command. Add your own patterns for anything dangerous in your environment: ```json { @@ -65,27 +77,43 @@ The built-in hard-deny list blocks `sudo`, `rm -rf ~/`, `kill`, fork bombs, and } ``` -Custom patterns augment the defaults, they don't replace them. Netclaw tokenizes compound commands (`&&`, `||`, `;`, `|`) and checks each segment independently, so `echo hello && docker rm foo` still triggers the deny. +Patterns augment the defaults and never replace them. Netclaw splits compound commands (`&&`, `||`, `;`, `|`) and checks each part, so `echo hello && docker rm foo` still matches. You can keep patterns in an optional `~/.netclaw/config/hard-deny-overrides.json` instead, which also adds to the defaults. + +## Review saved grants -## MCP Tool Permissions +Every "Always" answer at an approval prompt becomes a saved grant. Review them occasionally: -New MCP servers start with zero tool grants for all audiences, safe by default. The risk comes from granting too much. +```bash +netclaw approvals list +``` -Audit your grants per audience: +Look hardest at "anywhere" grants for programs that run arbitrary code, such as `bash`, `python3` or `node`. A grant for one of those lets the agent run any code as your user, which defeats the point of approvals. Revoke it with `netclaw approvals revoke`. `netclaw doctor` reports grants that another grant already covers, and `netclaw doctor --fix` removes them. See [`netclaw approvals`](/cli/approvals/). + +## MCP tool permissions + +`netclaw mcp add` seeds each audience differently: + +| Audience | Server access | Tool grants | Approval default | +|----------|---------------|-------------|------------------| +| Personal | All servers | All tools | `Auto` | +| Team | Must be in `AllowedMcpServers` | None until granted | `Approval` | +| Public | Must be in `AllowedMcpServers` | None until granted | `Deny` | + +Personal gets every tool on a new server at `Auto`, so review that audience first. Audit grants per audience: ```bash netclaw mcp permissions ``` -![MCP tool grants for Personal audience showing all tools enabled](/screenshots/output/mcp-tools-personal.png) +![MCP Permissions tool grid for demo-utilities on the personal audience, with the server enabled, default Auto, and ten tools each granted at Auto](/screenshots/output/mcp-tools-personal.png) -Personal audience with all tools granted. Compare against the locked-down Team and Public defaults: +Personal with every tool granted. The footer's `[E] Disable` turns the server off for this audience. Compare it with the locked-down Team and Public defaults: -![Team audience with server disabled](/screenshots/output/mcp-tools-team.png) +![MCP Permissions tool grid for demo-utilities on the team audience, with the server not enabled, default Approval and all ten tools unchecked and greyed out](/screenshots/output/mcp-tools-team.png) -Team audience: server disabled, no tools granted. +Team with the server disabled and nothing granted. Its footer reads `[E] Enable all`. Dim rows are tools that audience hasn't been granted, and the `*unsaved*` marker is there on open (see [`netclaw mcp`](/cli/mcp-tools/#tool-grid)). -Grant the minimum tools each audience actually needs. Don't enable everything for Team because it's faster to configure. For destructive MCP tools (`delete`, `drop`, `write`), set approval mode to `Approval` or `Deny`. Per-tool overrides let you keep the server default on `Auto` while gating the dangerous ones: +Grant each audience the minimum it needs. For destructive MCP tools (`delete`, `drop`, `write`), set `Approval` or `Deny`. Per-tool overrides keep the server on `Auto` while gating the dangerous ones: ```json { @@ -93,9 +121,7 @@ Grant the minimum tools each audience actually needs. Don't enable everything fo "AudienceProfiles": { "Personal": { "ApprovalPolicy": { - "DefaultMode": "Auto", "ToolOverrides": { - "shell_execute": "Approval", "notion/notion-delete-page": "Approval" } } @@ -105,110 +131,85 @@ Grant the minimum tools each audience actually needs. Don't enable everything fo } ``` -Tool override keys use the format `{serverName}/{toolName}`. +MCP override keys use `{serverName}/{toolName}`. Keys are matched without regard to letter case. A key that contains a colon is ignored and `netclaw doctor` flags it. See [`netclaw mcp`](/cli/mcp-tools/) for the permissions TUI and [MCP tool permissions](/guides/mcp-tool-permissions/) for the full recipe. -Review `~/.netclaw/config/tool-approvals.json` periodically and prune stale "approve always" decisions. +## Network exposure -See [`netclaw mcp`](/cli/mcp-tools/) for full details on the permissions TUI and CLI. - -## Network Exposure - -The daemon binds to `127.0.0.1:5199` by default. Keep it that way unless you have a reason not to. +The daemon binds `127.0.0.1:5199` by default. Keep it that way unless you have a reason not to. | Mode | Scope | Risk | |------|-------|------| -| `local` | Loopback only | Minimal — only local processes can connect | -| `reverse-proxy` | Whatever your proxy exposes | Medium to high — depends on final hop and trusted-proxy config | -| `tailscale-serve` | Your tailnet | Low — [Tailscale identity](https://tailscale.com/kb/1312/serve) gates access | -| `tailscale-funnel` | Public internet | High — anyone on the internet can reach it via [Tailscale Funnel](https://tailscale.com/kb/1223/funnel) | -| `cloudflare-tunnel` | Public internet | High — requires a [Cloudflare Access](https://developers.cloudflare.com/cloudflare-one/policies/access/) policy | +| `local` | Loopback only | Minimal. Only local processes connect | +| `reverse-proxy` | Whatever your proxy exposes | Medium to high. Depends on the final hop and trusted-proxy config | +| `tailscale-serve` | Your tailnet | Low. [Tailscale identity](https://tailscale.com/kb/1312/serve) gates access | +| `tailscale-funnel` | Public internet | High. Anyone can reach it through [Tailscale Funnel](https://tailscale.com/kb/1223/funnel) | +| `cloudflare-tunnel` | Public internet | High. Needs a [Cloudflare Access](https://developers.cloudflare.com/cloudflare-one/policies/access/) policy | -![Exposure mode selection during netclaw init](/screenshots/output/init-09-exposure.png) +![The Exposure Mode screen with five options, Local checked and marked recommended, and a warning triangle on Tailscale Funnel and Cloudflare Tunnel](/screenshots/output/config-exposure.png) -The exposure selection during `netclaw init`. Internet-facing modes force explicit confirmation. +Local is the default and the only mode that never leaves the machine. -If you need remote access, prefer `tailscale-serve`. It limits access to your tailnet. Funnel exposes you to the public internet — a completely different threat model. +Set the mode in `netclaw config` under **Security & Access**, or in `netclaw.json`. [Exposure modes](/deployment/exposure-modes/) covers each one. -If you need a conventional reverse proxy, bind netclaw to a non-loopback internal IP and explicitly set `Daemon.TrustedProxies`. `reverse-proxy` mode rejects loopback final-hop proxying on purpose. +For remote access, prefer `tailscale-serve`. Funnel is a different threat model. If you need a conventional reverse proxy, bind netclaw to a non-loopback internal IP and set `Daemon.TrustedProxies`. `reverse-proxy` mode rejects loopback final-hop proxying on purpose, and daemon-host CLI access then goes through paired-device auth rather than "loopback means trusted." -For Docker deployments, bind to loopback explicitly: +For Docker, bind the published port to loopback: ```bash docker run -p 127.0.0.1:5199:5199 ... ``` -Omitting `127.0.0.1` binds to all interfaces. Any machine on your network can reach the daemon. - -Changing exposure mode requires a daemon restart. It's excluded from hot-reload on purpose. - -```bash -netclaw daemon stop && netclaw daemon start -``` +Omitting `127.0.0.1` binds all interfaces, and any machine on your network can reach the daemon. -In `reverse-proxy` mode, daemon-host CLI access is no longer "loopback means trusted." Local control-plane requests may still work, but they do so through explicit paired-device auth when the exposure mode requires it. +In `local` mode, a request from loopback is trusted as the operator only when its `Host` header is `localhost` or a loopback address such as `127.0.0.1` or `[::1]`, and any `Origin` header matches that host. Anything else gets a 401 and a warning in the daemon log. A page in your browser can't reach the daemon through DNS rebinding or a cross-site request. It also means a script that calls the daemon by another host name, or a web app on `http://localhost:3000`, is refused. Call from the app's server process, or pair a device with `netclaw daemon pair` and send its bearer token. Paired devices and the tunnel and reverse-proxy modes aren't affected. -## Channel Access Restrictions +The daemon restarts itself when you change the exposure mode in `netclaw.json`. If you set it through an environment variable, restart by hand: `docker restart `, `systemctl --user restart netclaw`, or `netclaw daemon stop && netclaw daemon start`. -Every chat channel gates who can reach netclaw through the same four controls: +## Channel access restrictions -- **Channel allowlist** -- which channels netclaw answers in. No allowlist and no default channel denies all channel traffic. -- **User allowlist** -- which users get a response. Empty accepts everyone in the allowed channels. -- **Direct messages** -- off by default. Enable them only alongside a user allowlist. -- **Per-channel audiences** -- override the [trust audience](/security/security-model/#trust-audiences) for a single channel, or for DMs. +Every chat channel gates access with the same four controls: -Defaults are restrictive everywhere: `MentionOnly: true`, `AllowDirectMessages: false`, and empty allowlists. The config keys are identical across channels -- only the ID format and where you copy IDs from differ. Each channel page documents its own setup: +- **Channel allowlist.** Which channels netclaw answers in. No allowlist and no default channel denies all channel traffic. +- **User allowlist.** Which users get a response. Empty accepts everyone in the allowed channels. +- **Direct messages.** Off by default. Enable them only alongside a user allowlist. +- **Per-channel audiences.** Override the [audience](/security/security-model/#trust-audiences) for one channel or for DMs. - +Defaults are restrictive: `MentionOnly: true`, `AllowDirectMessages: false` and empty allowlists. The keys are identical across channels. Only the ID format and where you copy IDs from differ: | Channel | Access control reference | |---------|--------------------------| | Slack | [Slack access control](/channels/slack/#access-control) | | Discord | [Discord access control](/channels/discord/#access-control) | +| Mattermost | [Mattermost access control](/channels/mattermost/#access-control) | -Webhooks are the exception -- they have no channel or user allowlist. Inbound webhooks authenticate with HMAC signatures instead. See [`netclaw webhooks`](/cli/webhooks/). - -[`netclaw doctor`](/cli/doctor/) runs a `Slack ACL` check that flags Slack enabled with no channel allowlist and no default channel, or DMs enabled without a user allowlist. +Webhooks have no channel or user allowlist. They authenticate with HMAC signatures instead. See [`netclaw webhooks`](/cli/webhooks/). [`netclaw doctor`](/cli/doctor/) runs a `Slack ACL` check that flags Slack enabled with no channel allowlist and no default channel, or DMs enabled without a user allowlist. -## Approval Gates +## Approval prompts on Team and Public -The last layer before a tool actually runs. Configure per audience: +A prompt waits until someone answers. No timer denies it, and on a chat channel the thread looks frozen while it waits. For Team and Public that's the person in the channel, who may not be your operator. A sub-agent's prompt has buttons only, and on Mattermost it needs a `CallbackUrl` or netclaw refuses the request. If nobody is watching, give each tool a mode that doesn't wait: `Deny` for anything you don't want exposed, `Auto` for what you do. ```json { "Tools": { "AudienceProfiles": { - "Personal": { + "Team": { "ApprovalPolicy": { - "DefaultMode": "Auto", "ToolOverrides": { - "shell_execute": "Approval" + "web_fetch": "Deny", + "file_write": "Auto" } } - }, - "Team": { - "ApprovalPolicy": { - "DefaultMode": "Approval" - } - }, - "Public": { - "ApprovalPolicy": { - "DefaultMode": "Deny" - } } } } } ``` -- Headless sessions (reminders, webhooks, `netclaw chat -p "prompt"`) auto-deny all gated tools. There's no human to ask. -- No response within 5 minutes means deny. -- The "Always here" and "Always anywhere" options persist to `~/.netclaw/config/tool-approvals.json` — manage them with [`netclaw approvals`](/cli/approvals/). - -If nobody is watching the approval prompts in production, set `DefaultMode: "Deny"` for Team and Public. Auto-deny is safer than a 5-minute timeout nobody sees. +Unattended runs (reminders, webhooks, `netclaw chat -p`) can't show a prompt and deny any call that would, with `approval_required_unattended`. Saved grants and `Auto` tools still run. For the model, see [How tool approvals work](/security/approvals/). For recipes, see [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/). -## Webhook Security +## Webhook security -Each webhook route has its own HMAC secret, audience, body size limit, and rate limit: +Each webhook route has its own HMAC secret, audience, body size limit and rate limit. These fields sit inside a route file, which also needs a `prompt`. To create a whole route from the CLI, use `netclaw webhooks set`: ```json { @@ -225,60 +226,72 @@ Each webhook route has its own HMAC secret, audience, body size limit, and rate } ``` -Set the audience to the minimum the webhook actually needs. Most should run as `Public` (fewest tools, session-scoped filesystem, wiped on end). Only use `Team` or `Personal` if the webhook prompt genuinely requires those tools. +Set the audience to the minimum the webhook needs. Most should run as `Public`, which has the fewest tools and a session-scoped filesystem. Use `Team` or `Personal` only if the webhook prompt requires those tools. -`RateLimitPerMinute` applies per webhook route, not globally across all routes. Lower it from the default 30 if the source won't fire that often — GitHub sends roughly one webhook per event, so 10/min is plenty. Keep `MaxBodyBytes` at 1 MB or lower unless you know the payloads are larger. +`RateLimitPerMinute` applies per route, not globally. Lower it from the default 30 if the source fires less often. GitHub sends roughly one webhook per event, so 10 a minute is plenty. Keep `MaxBodyBytes` at 1 MB or lower unless you know the payloads are larger. -Route files contain secrets in plaintext. Keep `~/.netclaw/config/webhooks/` at mode `700` (see [File Permissions](#file-permissions)). +Route files hold the secret in plaintext. Keep `~/.netclaw/config/webhooks/` at mode `700`, as in [File permissions](#file-permissions). See [`netclaw webhooks`](/cli/webhooks/) for route setup. -See [`netclaw webhooks`](/cli/webhooks/) for route setup and HMAC verification details. +## Doctor checks -## Doctor Checks - -After making changes, validate everything: +After changes, validate everything: ```bash netclaw doctor ``` -![netclaw doctor running 16 diagnostic checks](/screenshots/output/doctor.png) +![netclaw doctor output: 24 checks, all PASS, ending with doctor exit code: 0](/screenshots/output/doctor.png) -Security-relevant checks include: +The Security Policy line near the top is the one to read first. -| Check | What it catches | -|-------|----------------| -| **Security Policy** | Missing `DeploymentPosture` | -| **Secrets JSON** | Wrong file permissions, unencrypted values | -| **Slack ACL** | No channel allowlist, DMs enabled without user allowlist | -| **Tool Audience Profiles** | Overly permissive tool grants | -| **exposure-mode** | Internet-reachable exposure without valid auth policy | -| **Inbound Webhook Routes** | Schema errors in route files | +The checks that matter for security: -Wire it into your deployment pipeline: +| Check | What it catches | +|-------|-----------------| +| Security Policy | A missing `Security` section or `DeploymentPosture` (Public posture, shell off), or an implied Personal posture | +| Secrets JSON | Wrong file permissions, unencrypted values | +| Slack ACL | No channel allowlist, DMs enabled without a user allowlist | +| Tool Audience Profiles | Overly permissive grants, copied default lists, Personal `shell_execute` on `Auto` with the host shell on, a profile that can write wider than it can read, and a `ToolOverrides` or `McpServerDefaults` key with a colon (an error you fix by hand) | +| Tool approval grants | Grants that another grant already covers | +| OAuth Token Expiry | A retired `OAuthTokenExpiry` key in `netclaw.json`. `--fix` moves it to `secrets.json` | +| exposure-mode | Internet-reachable exposure without a valid auth policy | +| Inbound Webhook Routes | Schema errors in route files | + +Wire it into a pipeline: ```bash netclaw doctor --format json | jq -e '.exitCode == 0' ``` +## Security advisories + +These published advisories are fixed in 0.28.0 or later. On an older install, upgrade and then check the follow-up. + +| Advisory | What to do | +|----------|------------| +| [GHSA-85pr-m799-g776](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-85pr-m799-g776) | A `ToolOverrides` or `McpServerDefaults` key with a colon set the whole tool to `Auto`. Remove every such key. Netclaw now ignores them and `netclaw doctor` reports them | +| [GHSA-mh95-h6p6-c9j3](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-mh95-h6p6-c9j3) | Through 0.27.1, the TUI commands (`init`, `provider`, `model`, `reminder`, `mcp tools`, `approvals`, `config`) wrote every keystroke to `netclaw--trace.log` in the temp directory. Delete those files and rotate any secret you typed on a shared machine | +| [GHSA-fmjr-hw4g-337j](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-fmjr-hw4g-337j) | Through 0.27.1, a paired remote device could read provider OAuth tokens in the tunnel exposure modes. Tokens now need a local-control proof | +| [GHSA-4638-4hj7-4w4m](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-4638-4hj7-4w4m) | Before 0.28.2, a configuration that couldn't be read made the CLI fall back to the default daemon address. It now reports the problem and sends nothing | + ## Limitations -- Prompt injection detection is regex-based. It catches known patterns (role resets, exfiltration attempts, invisible Unicode) but novel phrasings, Base64 encoding, synonym substitution, and non-English attacks can evade it. Treat it as a tripwire, not a firewall. -- Tool grants are per-audience, not per-channel. `ChannelAudiences` overrides which audience a channel maps to, but you can't give one Slack channel different tools than another channel with the same audience. -- Secret redaction catches `sk-*`, Slack tokens (`xox[baprs]-*`), `ghp_*`, AWS access keys (`AKIA*`), JWTs, PEM blocks, and common JSON key names. Custom secret formats won't be redacted. Use hard-deny path rules to block file access instead. -- Approval gates only work on interactive channels. Headless mode, reminders, and webhooks auto-deny tools that would otherwise prompt for approval. Design automation workflows with that in mind. +- Prompt injection detection is regex-based. It catches known patterns, but novel phrasings, Base64 encoding, synonym substitution and non-English attacks evade it. Treat it as a tripwire. +- Tool grants are per audience, not per channel. `ChannelAudiences` chooses the audience for a channel, but two Slack channels with the same audience get the same tools. +- Secret redaction catches known token formats and secret-named variables only, and skips `file_read` and MCP results. For custom formats, use hard-deny patterns to block the command instead. +- Approvals aren't a sandbox. See [Isolate netclaw from your other credentials](#isolate-netclaw-from-your-other-credentials). -## Related Pages +## Related pages -- [Security Model](/security/security-model/) -- default-deny architecture, audiences, trust layers -- [`netclaw init`](/cli/init/) -- posture and exposure selection wizard -- [`netclaw mcp`](/cli/mcp-tools/) -- MCP tool permissions TUI -- [`netclaw doctor`](/cli/doctor/) -- configuration diagnostics -- [`netclaw webhooks`](/cli/webhooks/) -- webhook route management and HMAC verification -- [`netclaw secrets`](/cli/secrets/) -- encrypted credential storage +- [Security model](/security/security-model/) - audiences, the order of checks, protected paths +- [How tool approvals work](/security/approvals/) - modes, grants and prompts +- [`netclaw doctor`](/cli/doctor/) - configuration diagnostics +- [`netclaw webhooks`](/cli/webhooks/) - route management and HMAC verification +- [Secrets management](/security/secrets/) - encrypted credential storage -## Further Reading +## Resources -- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) -- prompt injection, insecure output handling, and other LLM attack vectors -- [Tailscale ACLs](https://tailscale.com/kb/1018/acls/) -- network-level access control for `tailscale-serve` deployments -- [Cloudflare Access policies](https://developers.cloudflare.com/cloudflare-one/policies/access/) -- IdP-based access control for `cloudflare-tunnel` deployments -- [GitHub webhook security](https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks) -- best practices for HMAC verification and secret rotation +- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) - prompt injection, insecure output handling and other LLM attack vectors +- [Tailscale ACLs](https://tailscale.com/kb/1018/acls/) - network access control for `tailscale-serve` deployments +- [Cloudflare Access policies](https://developers.cloudflare.com/cloudflare-one/policies/access/) - IdP-based access control for `cloudflare-tunnel` +- [GitHub webhook security](https://docs.github.com/en/webhooks/using-webhooks/best-practices-for-using-webhooks) - HMAC verification and secret rotation diff --git a/src/content/docs/security/secrets.md b/src/content/docs/security/secrets.md index 5c54356..46af9a4 100644 --- a/src/content/docs/security/secrets.md +++ b/src/content/docs/security/secrets.md @@ -1,23 +1,23 @@ --- -title: "Secrets Management" -description: "Encrypted credential storage with netclaw secrets." +title: "Secrets management" +description: "How netclaw encrypts credentials at rest, keeps the agent away from them, and scrubs them from tool output." --- -Netclaw encrypts credentials at rest using [ASP.NET Data Protection](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/introduction). No plaintext secret ever touches disk. Key material lives in `~/.netclaw/keys/`, separate from `secrets.json` — copying the secrets file alone is insufficient to decrypt its values. +Netclaw encrypts the credentials in `secrets.json` at rest using [ASP.NET Data Protection](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/introduction). Key material lives in `~/.netclaw/keys/`, separate from `secrets.json`, so copying the secrets file alone isn't enough to decrypt it. The one exception is webhook route files, which hold each route's HMAC verification secret in plain JSON and are protected by path denial rather than encryption. :::note -The goal isn't bulletproof encryption — anyone with shell access to the machine can decrypt these values. The goal is preventing the *agent* from accidentally reading raw credentials during tool use. Encryption at rest means `file_read` on `secrets.json` returns ciphertext, not your API keys. +The goal isn't bulletproof encryption. Anyone with shell access to the machine as the same user can decrypt these values. The goal is to stop the *agent* from reading raw credentials during tool use. Encryption at rest means a copy of `secrets.json`, such as a backup or a leaked volume, is ciphertext and not your API keys. The agent can't read the file at all: `file_read` and shell reads of it are denied. ::: -Three layers protect credentials independently: encryption at rest keeps `secrets.json` opaque, ACL denial blocks the agent from reading specific secret files, and output redaction scrubs known secret patterns before the LLM sees them. If one layer fails, the other two still hold. +Three layers protect credentials. Encryption keeps `secrets.json` opaque, and path denial blocks the agent from reading netclaw's secret files. Output redaction is a third, much weaker layer: it masks known token formats and secret-named variables in tool output and shouldn't be counted on for anything beyond that. -If you're adding credentials for the first time, start with [`netclaw init`](/cli/init/). To add or rotate keys on an existing install, see [`netclaw secrets`](/cli/secrets/). +To add credentials for the first time, use [`netclaw init`](/cli/init/). To add or rotate keys on an existing install, use [`netclaw secrets`](/cli/secrets/). -## Encryption Architecture +## How encryption works -Netclaw wraps the [ASP.NET Data Protection API](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/implementation/key-management) with an application name of `Netclaw` and a purpose string that scopes encryption to `Netclaw.Secrets.v1` so keys can't be cross-used between subsystems. `DataProtectionSecretsProtector` calls `IDataProtector.Protect()` and `Unprotect()` — standard .NET cryptography, not a custom implementation. +Netclaw wraps the [ASP.NET Data Protection API](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/implementation/key-management) with the application name `Netclaw` and a purpose string of `Netclaw.Secrets.v1`, so keys can't be cross-used between subsystems. It's standard .NET cryptography through `IDataProtector.Protect()` and `Unprotect()`, not a custom scheme. -Secrets are encrypted the moment you run `netclaw secrets set` or complete `netclaw init`. Encrypted values are prefixed with `ENC:`: +Secrets are encrypted the moment you run `netclaw secrets set`, complete `netclaw init` or add an MCP server with env vars, headers or an OAuth client secret. Encrypted values carry an `ENC:` prefix: ```json { @@ -32,125 +32,132 @@ Secrets are encrypted the moment you run `netclaw secrets set` or complete `netc } ``` -Re-encrypting an already-encrypted value is safe — the writer checks for the `ENC:` prefix and skips values that already have it. +Re-encrypting an already encrypted value is safe. The writer skips values that already have the prefix. -## Storage Layout +## Storage layout | Path | Contents | Permissions | |------|----------|-------------| -| `~/.netclaw/config/secrets.json` | Encrypted credential values | `chmod 600` (Unix) | -| `~/.netclaw/keys/` | Data Protection key material | Owner-only access (Unix) | +| `~/.netclaw/config/secrets.json` | Encrypted credential values, plus each provider's `OAuthTokenExpiry` date in plain text | `chmod 600` on Unix | +| `~/.netclaw/keys/` | Data Protection key material | Owner-only on Unix | -Both paths are required for decryption. `secrets.json` without `~/.netclaw/keys/` is locked ciphertext, and the keys without the secrets file have nothing to decrypt. Back up both together. +The expiry date is the one plain-text value on purpose. It is refresh timing, not a credential, and the config binder can't read an encrypted date. A token refresh writes it to `secrets.json` and doesn't touch `netclaw.json`, so it no longer restarts the daemon. `netclaw secrets set` rejects an expiry that isn't an ISO 8601 date. If `netclaw.json` still holds an old `OAuthTokenExpiry`, `netclaw doctor --fix` moves it. -On Windows, netclaw relies on user-profile ACLs instead of Unix file modes. Check file properties in the Security tab to verify only your user account has access. +You need both to decrypt. `secrets.json` without `keys/` is locked ciphertext, and the keys without the file have nothing to decrypt. Back them up together. On Windows, netclaw relies on user-profile ACLs instead of file modes. -## Configuration Layering +## Configuration layering -Secrets slot into the standard config priority chain. Highest priority wins: +The highest priority wins: -| Priority | Source | Use Case | -|----------|--------|----------| -| 1 (highest) | `NETCLAW_*` environment variables | CI/CD, containers, temporary overrides | +| Priority | Source | Use | +|----------|--------|-----| +| 1 | `NETCLAW_*` environment variables | CI, containers, temporary overrides | | 2 | `secrets.json` (encrypted) | Persistent credentials | -| 3 (lowest) | `netclaw.json` (plaintext) | Non-sensitive configuration | +| 3 | `netclaw.json` (plaintext) | Non-sensitive configuration | -Environment variable names use the `NETCLAW_` prefix with double underscores for nesting, following the [.NET configuration convention](https://learn.microsoft.com/en-us/dotnet/core/extensions/configuration-providers#environment-variable-configuration-provider): `NETCLAW_Slack__BotToken` maps to `Slack.BotToken`. Case matters on Linux. +Environment variable names use the `NETCLAW_` prefix with double underscores for nesting, following the [.NET convention](https://learn.microsoft.com/en-us/dotnet/core/extensions/configuration-providers#environment-variable-configuration-provider). `NETCLAW_Slack__BotToken` maps to `Slack.BotToken`, and case matters on Linux. -## Agent Isolation +## Agent isolation -The [security model's](/security/security-model/) resource hard-deny layer blocks the agent from accessing its own credential storage. These paths are denied for both read and write operations: +The [security model's](/security/security-model/#protected-paths) protected-path rules keep the agent away from credential storage: -- `~/.netclaw/config/secrets.json` — the encrypted credential file -- `~/.netclaw/keys/` — encryption key material -- `~/.netclaw/config/webhooks/` — webhook route configurations with HMAC secrets +- `~/.netclaw/config/secrets.json` is denied for read and write. +- `~/.netclaw/keys/` is denied for read and write. +- `~/.netclaw/config/webhooks/` is denied for read and write, because route files hold HMAC secrets. +- The SQLite database, process-control files and the tooling shadow directory are also read-denied. +- From 0.28.0, the credential folders of other tools are denied for read and write as well: `~/.ssh`, `~/.aws`, `~/.gnupg`, `~/.kube`, `~/.azure`, `~/.config/gh`, `~/.config/gcloud`, `~/.docker/config.json`, `~/.netrc` and `~/.git-credentials`. `cat ~/.aws/credentials` is denied with `shell_references_protected_path`, and `file_read` returns an access-denied error. `ssh`, `git` and `aws` still use those files when they run. -Symlinks are resolved before path checks, so `ln -s ~/.netclaw/keys/ ./sneaky` won't bypass the policy. If the agent tries to access a denied path, the operation is blocked and the tool call fails. The agent can *use* the credentials (they're injected into provider connections at startup) but can never read, modify, or exfiltrate the raw values. +Netclaw resolves symlinks before checking, so `ln -s ~/.netclaw/keys/ ./sneaky` doesn't bypass the rule. It also denies glob, brace and ANSI-C quoted spellings that could reach a protected path, such as `cat ~/.netclaw/k*/*.xml`. A denied call fails and the agent gets an error. -Note that `netclaw.json` itself is readable by the agent — only files containing secrets are denied. +The agent can use credentials, because they're injected into provider connections at startup. It can't read, modify or exfiltrate the raw values. The rest of `~/.netclaw/config/` is readable but not writable, so the agent can read `netclaw.json` and the other config files. Only the files that hold secrets are read-denied. -## Output Redaction +Path denial protects netclaw's store and that fixed list. It doesn't protect credentials kept anywhere else on the machine, such as a project `.env` file. See [what netclaw doesn't protect against](/security/security-model/#what-netclaw-doesnt-protect-against). -Even with encryption and path denial, a secret can leak through tool output — an API key echoed in a curl response, a token appearing in a log. `SecretOutputRedactor` scrubs tool output before the LLM sees it, replacing matches with `***REDACTED***`. +## Output redaction -Patterns it catches: +A secret can still leak through tool output, such as an API key echoed in a curl response or a token in a log. The output redactor replaces secret-shaped text in tool output with `***REDACTED***` before the model sees it. It's a tripwire. Don't rely on it to keep credentials out of model context. | Pattern | Examples | |---------|----------| -| Provider API keys | `sk-*`, `ghp_*`, `AKIA*` (AWS) | -| Slack tokens | `xoxb-*`, `xoxp-*`, `xoxa-*`, `xoxr-*`, `xoxs-*` | -| Auth headers | `Authorization: Bearer ` | +| Known token formats | `sk-*`, `xox*-*`, `xapp-*`, `ghp_*`, `github_pat_*`, `AKIA*` and `ASIA*` (AWS), Stripe `sk_live_*` and `whsec_*`, Google `AIza*`, `npm_*`, PyPI, Discord bot tokens | +| Slack webhook URLs | `https://hooks.slack.com/services/...` | +| Auth headers | `Authorization` and `Proxy-Authorization` with Basic, Bearer, Token and similar schemes (the scheme stays), `Cookie`, `Set-Cookie`, `X-Api-Key` and other `*-token` or `*-key` header lines | +| URL credentials | The password in `scheme://user:password@host` | +| Option values | `--password hunter2`, `--token x`, `--api-key x`. Not `--password-file`, `--no-password` or `--next-token` | | Connection strings | `Password=...;`, `Pwd=...;` | -| JWT tokens | Three-segment base64 starting with `eyJ` | +| JWTs | Three base64 segments starting with `eyJ` | | PEM private keys | `-----BEGIN * PRIVATE KEY-----` blocks | -| JSON secret fields | Keys matching `api_key`, `token`, `secret`, `password`, `authorization`, `access_token`, `refresh_token`, `client_secret`, `signing_key`, `private_key`, `connection_string`, `credential` | -| Environment variables | `API_KEY=value`, `TOKEN=value`, `PASSWORD=value`, etc. | +| JSON secret fields | Field names that **start with** `api_key`, `token`, `secret`, `password`, `passwd`, `authorization`, `access_token`, `refresh_token`, `client_secret`, `signing_key`, `private_key`, `connection_string`, `credential` or `passphrase` | +| Env-style pairs | `API_KEY=value`, `TOKEN=value`, `PASSWORD=value`. Also names with the word as a segment, such as `DB_PASSWORD`, `GITHUB_TOKEN` and `AWS_SECRET_ACCESS_KEY`, with the whole value masked | -This is regex-based pattern matching, not semantic analysis. Custom or unusual secret formats won't be caught. +Settings that merely carry those words stay readable: `GH_TOKEN_FILE=/run/secrets/gh`, `MAX_TOKENS=4096`, `NEXT_PAGE_TOKEN=abc` and references like `${{ secrets.GITHUB_TOKEN }}`. In a test on 0.28.2, `cat .env` in the shell came back with `API_KEY`, `DB_PASSWORD`, `GITHUB_TOKEN` and `AWS_SECRET_ACCESS_KEY` masked, the `DATABASE_URL` line as `postgres://app:***REDACTED***@db.internal:5432/app`, and `SESSION_KEY=abcdef123456` as written. This is regex matching, not semantic analysis, so custom secret formats aren't caught. -## Write-Only Vault +`file_read` and MCP tool results are the exceptions: they come back unmasked. `file_read` returns the file as it is so an edit doesn't write placeholders into it, and MCP servers are integrations you configured. The tool-call arguments in the daemon and session logs are redacted too, and the spill file behind `tool_output_read` always holds the redacted text. -There is no `secrets get`, `secrets list`, or `secrets delete` command. Write secrets in, never read them back. This eliminates exfiltration via CLI: even if an attacker triggers a prompt injection, there's no netclaw command to dump credentials. +## Write-only vault -To remove a secret, delete the key from `~/.netclaw/config/secrets.json` directly. The agent can't touch this file, but you can. To verify secrets are properly stored, run [`netclaw doctor`](/cli/doctor/). +There's no `secrets get`, `secrets list` or `secrets delete`. You write secrets in and never read them back. That removes exfiltration through the CLI: even a successful prompt injection has no netclaw command to dump credentials. To remove a secret, delete the key from `secrets.json` directly. The agent can't touch that file, but you can. To check that secrets are stored properly, run [`netclaw doctor`](/cli/doctor/). -## Doctor Integration +Provider OAuth tokens follow the same rule. The daemon's OAuth status endpoint returns no tokens. During `netclaw provider` sign-in the CLI reads them once from a local-control endpoint, and only with a proof tied to that sign-in. Keep the CLI and daemon on the same version, because a newer CLI with an older daemon shows an upgrade message ([GHSA-fmjr-hw4g-337j](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-fmjr-hw4g-337j)). + +## Doctor integration [`netclaw doctor`](/cli/doctor/) validates the secrets subsystem on every run: -| Check | What It Catches | -|-------|----------------| -| File existence | `secrets.json` missing (warning, not error) | -| JSON validity | Malformed JSON that would prevent config loading | -| File permissions | Group or other read/write bits set on Unix (should be `600`) | -| Encryption status | Plaintext values that should be encrypted | +| Check | What it catches | +|-------|-----------------| +| File existence | `secrets.json` missing (a warning, not an error) | +| JSON validity | Malformed JSON that would block config loading | +| File permissions | Group or other bits set on Unix (should be `600`) | +| Encryption status | Plaintext values that should be encrypted (the `OAuthTokenExpiry` date doesn't count) | -If doctor flags unencrypted values, you'll need the original plaintext to re-encrypt: run `netclaw secrets set ` for each one. +If doctor flags an unencrypted value, you need the original plaintext to re-encrypt it. Run `netclaw secrets set ` for each one. The value is a command-line argument, so it lands in your shell history. -## Threat Model +## Threat model | Threat | Mitigation | -|--------|-----------| -| Secrets at rest on disk | AES encryption via Data Protection API; `chmod 600` file permissions | -| Agent reads its own credentials | Resource hard-deny on `secrets.json`, `keys/`, and `webhooks/` | -| Secret leaks into tool stdout | Regex-based output redaction before LLM ingestion | -| Secret exposed via CLI command | Write-only vault with no read/list/dump commands | -| Encryption keys copied to another machine | Keys and secrets both required — `secrets.json` alone is useless | -| Permission drift after manual edits | `netclaw doctor` flags incorrect file modes | +|--------|------------| +| Secrets at rest on disk | Data Protection encryption and `chmod 600` | +| Agent reads its own credentials | Read and write denial on `secrets.json`, `keys/` and `webhooks/` | +| Secret leaks into tool output | Partial: regex redaction of known token formats and secret-named variables. Not a control to rely on | +| Secret exposed through the CLI | Write-only vault with no read, list or dump command | +| Keys copied to another machine | Both `keys/` and `secrets.json` are required | +| Permission drift after manual edits | `netclaw doctor` flags incorrect modes | +| Keystrokes logged to disk by the TUI | Fixed in 0.28.0. On older installs delete `netclaw-*-trace.log` from the temp folder ([GHSA-mh95-h6p6-c9j3](https://github.com/netclaw-dev/netclaw/security/advisories/GHSA-mh95-h6p6-c9j3)) | ## Recovery -Back up `~/.netclaw/keys/` alongside `secrets.json`. Without the keys directory, encrypted values are unrecoverable. - -If `~/.netclaw/keys/` is deleted or corrupted, re-run `netclaw secrets set` for each credential. There's no other recovery path. +Back up `~/.netclaw/keys/` with `secrets.json`. Without the keys directory, encrypted values are unrecoverable. If `keys/` is deleted or corrupted, re-run `netclaw secrets set` for each credential. There's no other recovery path. -If file permissions have drifted, reset them and verify: +If permissions have drifted, reset them and verify: ```bash chmod 600 ~/.netclaw/config/secrets.json netclaw doctor ``` -If a secret isn't taking effect, check for environment variable overrides. `NETCLAW_Slack__BotToken` silently wins over the `secrets.json` value. `netclaw doctor` helps diagnose layering issues. +If a secret isn't taking effect, check for an environment variable override first. `NETCLAW_Slack__BotToken` silently beats the `secrets.json` value. The daemon also reads secrets at startup and only restarts itself when `netclaw.json` changes, so restart it after `secrets set`. An OAuth token refresh writes `secrets.json` at run time and doesn't trigger a restart. ## Limitations -- Secrets are portable only if you copy both `~/.netclaw/keys/` and `~/.netclaw/config/secrets.json` together. The keys directory contains the Data Protection key ring needed to decrypt values. -- No built-in key rotation. Re-running `secrets set` overwrites the previous value, but there's no automated rotation schedule. -- Windows relies on user-profile ACLs rather than explicit `chmod`, so permission auditing is less straightforward. -- Single-machine vault, not a team secrets manager. No integration with external secret stores (Vault, AWS Secrets Manager, etc.). +- Secrets are portable only if you copy `keys/` and `secrets.json` together. +- There's no built-in key rotation. Running `secrets set` again overwrites a value, but nothing rotates on a schedule. +- Windows relies on user-profile ACLs instead of `chmod`, so permission auditing is less direct. +- It's a single-machine vault, not a team secrets manager. There's no integration with Vault, AWS Secrets Manager or similar. +- Webhook route files aren't encrypted. Keep `~/.netclaw/config/webhooks/` at mode `700`. -## Related Pages +## Related pages -- [`netclaw secrets`](/cli/secrets/) — CLI command reference for setting secrets -- [`netclaw init`](/cli/init/) — stores initial credentials during first-run setup -- [`netclaw doctor`](/cli/doctor/) — validates secrets encryption and file permissions -- [Security Model](/security/security-model/) — default-deny architecture and the invocation stack -- [Hardening](/security/hardening/) — file permission lockdown and operational best practices +- [`netclaw secrets`](/cli/secrets/) - the command reference for setting secrets +- [`netclaw init`](/cli/init/) - stores initial credentials during setup +- [`netclaw doctor`](/cli/doctor/) - validates secrets encryption and permissions +- [Security model](/security/security-model/) - audiences, the order of checks and protected paths +- [Connect an OAuth MCP server](/guides/oauth-mcp-servers/) - where OAuth tokens for MCP servers are stored and refreshed +- [Hardening](/security/hardening/) - file permission lockdown and isolation -## Further Reading +## Resources -- [ASP.NET Data Protection overview](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/introduction) — the encryption framework netclaw builds on -- [ASP.NET Data Protection key management](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/implementation/key-management) — how encryption keys are stored and rotated -- [.NET environment variable configuration](https://learn.microsoft.com/en-us/dotnet/core/extensions/configuration-providers#environment-variable-configuration-provider) — the double-underscore nesting convention -- [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html) — general best practices for credential storage +- [ASP.NET Data Protection overview](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/introduction) - the encryption framework netclaw builds on +- [ASP.NET Data Protection key management](https://learn.microsoft.com/en-us/aspnet/core/security/data-protection/implementation/key-management) - how keys are stored and rotated +- [.NET environment variable configuration](https://learn.microsoft.com/en-us/dotnet/core/extensions/configuration-providers#environment-variable-configuration-provider) - the double-underscore nesting convention +- [OWASP Secrets Management Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Secrets_Management_Cheat_Sheet.html) - general credential storage practices diff --git a/src/content/docs/security/security-model.md b/src/content/docs/security/security-model.md index da462b4..8492eb9 100644 --- a/src/content/docs/security/security-model.md +++ b/src/content/docs/security/security-model.md @@ -1,86 +1,107 @@ --- -title: "Security Model" -description: "Default-deny ACL, audience scoping, and trust levels." +title: "Security model" +description: "What netclaw enforces: audiences, postures, the order of checks, protected paths and shell policy, and where the boundary ends and a sandbox begins." --- -Netclaw is **default-deny**. Every tool call, file access, and shell command starts blocked and must be explicitly permitted. Any policy failure — invalid config, engine exception, unrecognized channel — results in deny. +Netclaw is default-deny. Every tool call, file access and shell command starts blocked and must be permitted by your configuration. Any policy failure, such as an exception in the policy engine or a channel it can't identify, ends in a denial. This page is the operator's reference for what netclaw enforces. For the reasoning behind the design, see [Security architecture](/architecture/security-model/). For how prompts work, see [How tool approvals work](/security/approvals/). -*Audiences* control what's allowed. A *four-layer invocation stack* enforces it. You pick a deployment posture during [`netclaw init`](/cli/init/), and it governs every permission decision the daemon makes. +:::caution +Netclaw protects its own control plane and gates the tools the agent calls. It isn't a sandbox. Beyond that and a fixed list of credential folders, the agent can read anything your OS user can read. See [what netclaw doesn't protect against](#what-netclaw-doesnt-protect-against) before you run it on a machine that holds other credentials. +::: -## Trust Audiences +## Trust audiences -Netclaw classifies each inbound message into one of three audiences based on the channel it arrived on: +Every inbound message resolves to one of three audiences. Personal is the most trusted and Public the least. The [deployment posture](#deployment-postures) is a ceiling on top of that: the audience a turn runs with, and the audience of any sub-agent it starts, is the narrower of the posture's audience and the source audience. The table below lists the source audience. -| Channel | Audience | -|---------|----------| -| TUI (`netclaw chat`), SignalR (real-time web transport), headless (no UI, daemon-only), console, manual | **Personal** | -| Slack, Discord, reminders, timers | **Team** | -| Unknown channel type | **Public** | +| Source | Audience | +|--------|----------| +| TUI (`netclaw chat`), web client (SignalR), headless (`netclaw chat -p`), console, manual | Personal | +| Slack, Discord, Mattermost | Per channel or DM from `ChannelAudiences`. Otherwise Team for an allowlisted channel or user, Public for anything else | +| Webhook | The route's `Audience`, default Public | +| Reminder | The audience of the session that created it, or a lower one the reminder sets | +| Unknown channel type | Public | -Personal is the most permissive, Public the most restrictive. Unknown channels resolve to Public. If netclaw can't identify who's talking, it assumes the worst. +For chat channels, the order is: the channel's own entry in `ChannelAudiences`, then the `dm` entry for direct messages, then the allowlist rule. A DM from a user who isn't on `AllowedUserIds` resolves to Public, not Team. [`netclaw init`](/cli/init/) and [`netclaw config`](/cli/config/) write the `ChannelAudiences` entries: a regular channel defaults to Team (Public on a Public posture), and a DM defaults to Personal when one user is allowlisted, and otherwise follows the posture (a Personal posture means Personal DMs). See [Hardening](/security/hardening/#channel-access-restrictions) for the access-control keys. -## Deployment Postures +If netclaw can't resolve an audience for a tool call, it denies the call with `audience_unresolved`. It never substitutes Public. -During [`netclaw init`](/cli/init/), you pick a deployment posture that sets the baseline trust level: +### Reminder tools and audiences -![Security posture selection during netclaw init](/screenshots/output/init-02-security-posture.png) +The reminder tools only see reminders at or below the caller's audience. A reminder outside that scope reads as `not found`, and `set_reminder` can't overwrite it. -The posture selection screen during `netclaw init`. Personal mode enables shell access with approval gates; Team and Public disable shell entirely. +| Caller (effective audience) | Reminders it can list, run, cancel, read history for and update | +|--------|------------------------------------------------------------------| +| Personal chat | Personal, Team, Public | +| Team chat | Team, Public | +| Public chat | Public | +| CLI and HTTP API (operator) | All | -| Posture | Shell Access | Default Behavior | -|---------|-------------|-----------------| -| **Personal** | Enabled (with [approval gates](#approval-gates)) | All tools available, all MCP servers, unrestricted filesystem | -| **Team** | Off | Allowlisted tools only, no MCP servers, session-scoped filesystem (temp directory per session, wiped on end) | -| **Public** | Off | Minimal tool set, no MCP servers, session-scoped filesystem (temp directory per session, wiped on end) | +A reminder created before 0.28.0 under a narrower posture (for example a Team posture with Slack DMs mapped to Personal) keeps its Personal audience. `run_reminder` refuses it and a narrower chat doesn't list it, so recreate it from a chat. [Reminders](/configuration/reminders/#audience-scoping) covers which tool groups each audience gets and how to test a reminder at its own audience. -With no config file at all, netclaw defaults to Public posture with shell disabled — the most restrictive option. +## Deployment postures -Posture and audience are related but distinct. Posture is a deployment-wide setting you choose once. Audience is resolved per-message based on channel. A Team-posture deployment still classifies TUI sessions as Personal audience. +You pick a posture during [`netclaw init`](/cli/init/). It sets the shell default for the deployment and caps the tools every turn can use. -## Per-Audience Permissions +![Netclaw Setup, Step 3 of 5, Security Posture, offering Personal, Team and Public with a one-line summary of each](/screenshots/output/init-02-security-posture.png) -Tools, filesystem access, and attachment policies differ by audience: +The posture screen. Personal is the only choice that turns shell on. -| Audience | Tools | MCP Servers | Memory | Filesystem | Attachment Types | -|----------|-------|-------------|--------|------------|-----------------| -| **Personal** | All | All | Full | Unrestricted | Image, PDF, Document, Archive, Media, Other | -| **Team** | `file_read`, `file_list`, `file_write`, `file_edit`, `attach_file`, web search/fetch, skills, reminders, `set_working_directory` | None | Full | Session-scoped only | Image, PDF, Document, Archive, Media | -| **Public** | `file_read`, `file_list`, `attach_file` | None | Disabled | Session-scoped only | Images only | +| Posture | Shell default | +|---------|---------------| +| Personal | `HostAllowed`, with approval prompts | +| Team | `Off` | +| Public | `Off` | -Public is read-only — it reads, lists, and attaches files, but can't write, search the web, or manage skills. Team adds writes and edits, web search and fetch, skill management, and reminders. The two grants cascade, so everything Public can do, Team can too. Filesystem scope is the hard boundary underneath: Public and Team both confine every read and write to a session-scoped temp directory that's wiped on session end, so even a re-granted write tool can't widen what the agent reaches. Personal gets the full tool surface and unrestricted filesystem access. The split is a per-audience default in `netclaw.json` — adjust each audience's `AllowedTools` list if it doesn't suit your deployment. +With no config file, or no `DeploymentPosture`, netclaw resolves to Public with shell off. That applies to `netclaw chat` too: a config with no posture gives the TUI Public tools, and a `shell_execute` call is denied. -Memory is hard-disabled for Public — not just defaulted off. Public sessions get no automatic recall before a turn, write no new memories after one, and never see the memory tools, so they can't pull in cross-session context or leave anything behind. Team and Personal get the full memory subsystem. See [Memory Model](/architecture/memory-model/#audience-scoping) for how stored memories carry audience and boundary context. +Posture is a deployment-wide choice you make once, and audience is resolved per message from the channel. The two combine by taking the narrower. A Team posture runs `netclaw chat` with Team tools (no `shell_execute`, no webhook tools), and a Public posture runs it with Public tools. A Personal posture changes nothing, so a Slack channel mapped to Team stays Team. The admission gate, the shell check, the approval mode, the filesystem roots, tool search and every sub-agent read the narrowed audience. The tool lists themselves come from the [audience profiles](#per-audience-permissions). -MCP server permissions are managed separately per audience through [`netclaw mcp permissions`](/cli/mcp-tools/). New MCP servers start with zero tool grants for all audiences — you must explicitly enable them. +Three keys control shell, in this order of precedence: `Tools.ShellMode`, then `Security.ShellExecutionMode`, then the posture default. `netclaw init` writes both of the first two, so to change shell on an installed system set both. Values are `Off` and `HostAllowed`. `SandboxOnly` is accepted but there's no sandbox backend, so it denies shell entirely with `shell_requires_sandbox_backend`. -## Four-Layer Invocation Stack +## Per-audience permissions -Tool calls pass through four layers in sequence. If any layer rejects, execution stops. +| | Personal | Team | Public | +|-|----------|------|--------| +| Tools | All, including `shell_execute` | `file_read`, `file_list`, `file_search`, `tool_output_read`, `file_write`, `file_edit`, `attach_file`, `web_search`, `web_fetch`, `skill_manage`, reminder tools, `set_working_directory` | `file_read`, `file_list`, `file_search`, `tool_output_read`, `attach_file` | +| MCP servers | All | None until listed in `AllowedMcpServers` | None until listed | +| Filesystem | Everything except protected paths. File reads reach at least as far as writes do | Session directory, plus read-only skills, identity and workspaces roots | Its session directory only | +| Memory | Full | Full | Disabled, not just defaulted off | +| Attachments | Image, PDF, Document, Archive, Media, Other | Image, PDF, Document, Archive, Media | Image only | -``` -┌─────────────────────────────────┐ -│ 1. Operation Hard Deny │ ← blocks dangerous commands -├─────────────────────────────────┤ -│ 2. Resource Hard Deny │ ← blocks protected file paths -├─────────────────────────────────┤ -│ 3. Tool Access Grant │ ← audience-based allowlist -├─────────────────────────────────┤ -│ 4. Approval Gate │ ← human confirms execution -└─────────────────────────────────┘ -``` +File reads on Personal follow the write profile, so the agent can read what its shell writes, for example under `/tmp`. With `ReadFiles.Mode = Roots` and a wider `WriteFiles`, Personal reads (including unattended reminders and webhooks) get wider with no prompt. `netclaw doctor` warns when any audience can write wider than it can read. Protected paths stay denied, and Team and Public are unchanged. + +Shell is Personal-only. A Team or Public call to `shell_execute` is denied, whatever the posture says. The webhook tools (`set_webhook`, `list_webhooks`, `delete_webhook`) are also Personal-only. + +Adjust a list in `Tools.AudienceProfiles..AllowedTools`. Since 0.27.1 a configured list replaces the default, so write out every tool you want, including `tool_output_read`. `netclaw doctor` warns when a list copies an old default, and `netclaw doctor --fix` removes it so the audience follows the current default. See [Memory model](/architecture/memory-model/#audience-scoping) for how stored memories carry audience context. + +## Layers in order + +One authorizer applies these rules to every tool call, first match wins. The same list covers built-in tools, MCP tools, `shell_execute`, subagents and unattended runs. + +| # | Question | If not satisfied | Set by | +|---|----------|------------------|--------| +| 1 | Does the audience profile allow this tool, and for MCP this server and tool? | `tool_not_allowed_for_audience_profile` | `AllowedTools`, `AllowedMcpServers`, `McpServerToolGrants` | +| 2 | For shell: is the audience Personal and the shell mode on? | `shell_requires_personal_context`, `shell_requires_sandbox_backend` | Posture, `Tools.ShellMode` | +| 3 | Is the command on the hard-deny list? | `hard_deny_` | Built in, plus `Tools.HardDenyPatterns` | +| 4 | Does the call name a protected path (control plane or credential location), or reach one with a glob, brace or symlink? | `shell_references_protected_path`, `shell_path_protected`, `path_access_denied` | Built in | +| 5 | Is every path inside the audience's trusted roots? | `shell_path_outside_trusted_roots` | `ReadFiles`, `WriteFiles` in the profile | +| 6 | Is the tool's approval mode `Deny`? | Denied | `ApprovalPolicy` | +| 7 | Can the agent fix the call itself? | Requires correction | Built in | +| 8 | Is the mode `Auto`? | Allowed | `ApprovalPolicy` | +| 9 | Does a saved grant, a this-chat answer or the reviewed-safe catalog cover it? | Prompt, or `approval_required_unattended` when nobody can answer | Grants, [`netclaw approvals`](/cli/approvals/) | -### Layer 1: Operation Hard Deny +Steps 1 through 5 can't be answered away. Steps 6 through 9 are the approval system. Steps 3 and 4 run before any grant lookup, so a saved grant never reaches a hard-denied command or a protected path. Non-shell tools run the same list without steps 2, 3 and 5, and with file-path checks in step 4. -Shell commands hit a hard-deny list first. Blocked regardless of audience or approval status: +### Hard deny | Category | Blocked | |----------|---------| -| Self-destructive | `netclaw daemon stop`, `netclaw daemon kill`, `systemctl stop netclaw`, `systemctl kill netclaw`, `kill`, `killall`, `pkill` | -| Privilege escalation | `sudo`, `su`, `doas` | +| Self-destructive | `netclaw daemon stop`, `systemctl stop netclaw`, `systemctl kill netclaw`, and `kill`, `killall`, `pkill` or `Stop-Process` when the command names netclaw (`pkill netclawd`). Other kills prompt like any command | +| Privilege escalation | `sudo`, `su`, `doas`, and a PowerShell `-Verb RunAs` start | | System destruction | `rm -rf /`, `rm -rf ~/`, `mkfs*` | -| Fork bombs | `:(){ :\|:& };:`, `:(){:\|:&};:` | +| Fork bombs | `:(){ :|:& };:` and the no-space form `:(){:|:&};:`. Other fork bombs aren't detected | -Add your own patterns in `~/.netclaw/config/netclaw.json`: +Add your own with `Tools.HardDenyPatterns` in `netclaw.json` or an optional `~/.netclaw/config/hard-deny-overrides.json`. Both add to the defaults and never replace them. ```json { @@ -90,211 +111,149 @@ Add your own patterns in `~/.netclaw/config/netclaw.json`: } ``` -### Layer 2: Resource Hard Deny +Compound commands (`&&`, `||`, `;`, `|`) are split and each command is checked, so `echo hi && docker rm foo` still matches. -File access is checked against protected paths. Read and write have separate deny lists: +### Protected paths -| Operation | Denied paths | -|-----------|-------------| -| **Read** | `~/.netclaw/config/secrets.json`, `~/.netclaw/keys/`, `~/.netclaw/config/webhooks/` | -| **Write** | `~/.netclaw/config/secrets.json`, `~/.netclaw/keys/`, SQLite DB, PID file, lock file, restart manifest | +Read and write have separate lists. Netclaw resolves symlinks first, and it denies globs, brace words and ANSI-C quoted spellings that could reach a protected path. A glob that crosses a symlinked folder is checked against where the link points too, so a protected path behind a link is refused. -`~/.netclaw/config/netclaw.json` is intentionally **not** denied — the agent can read (but not write) the main config file. - -Netclaw resolves symlinks before checking, so `ln -s ~/.netclaw/keys/ ./sneaky` won't bypass the policy. - -### Layer 3: Tool Access Grant - -The audience profile determines which tools exist at all. Public and Team audiences only see allowlisted tools (see the [per-audience table](#per-audience-permissions)). Ungranted tools are invisible to the model — they don't appear in the tool list at all. - -MCP tool grants are configured separately per server and per audience through [`netclaw mcp permissions`](/cli/mcp-tools/). - -### Layer 4: Approval Gates {#approval-gates} - -Tools that pass layers 1-3 hit the approval gate, which prompts the operator for confirmation: - -![Approval gate prompt in a Slack thread](/assets/approval-prompt.png) - -| Option | Behavior | -|--------|----------| -| Once | Just this invocation | -| This chat | The rest of the current session | -| Always here | The command's verb, scoped to the current directory — persisted to `~/.netclaw/config/tool-approvals.json` | -| Always anywhere | The command's verb, everywhere — a global grant persisted to the same file. The broadest option; use it sparingly. | -| Deny | Blocks this invocation | +| Operation | Denied | +|-----------|--------| +| Read | `secrets.json`, the `keys/` directory, webhook route files in `config/webhooks/`, the SQLite database and its sidecars, process-control files (PID, lock, restart manifest), the tooling shadow directory, [credential locations](#credential-locations) | +| Write | Everything under `~/.netclaw/config/`, `keys/`, the database and sidecars, process-control files, the tooling shadow directory, skill-feed sync state, [credential locations](#credential-locations) | -Grants are keyed by the command's verb (e.g. `git`, `npm`), not the full command string; a compound command records one entry per verb. The full prompt shows all five options, but netclaw shows fewer when some don't apply — a command it can't cleanly parse (shell control flow, or unbalanced quotes) drops to just **Once** and **Deny**. Manage saved approvals with [`netclaw approvals`](/cli/approvals/). +The agent can read `netclaw.json` and the other config files, but not write them. No approval mode or grant changes that for `file_write` or `file_edit`. `ToolOverrides` keys that contain a colon (such as the old `file_write:control-plane`) are ignored, and `netclaw doctor` reports them. A shell program that only reads its operands (`cat`, `head`, `tail`, `wc`, `grep`, `jq`, `diff`) can read a config file by exact path. Any other shell command that names a config file meets write protection. -Approval timeouts work differently depending on the channel: +### Credential locations -- **Interactive channels** (TUI, Slack, Discord, SignalR) — no response within 5 minutes triggers auto-deny. The LLM gets an error message but has no idea an approval gate rejected it. -- **Non-interactive channels** (headless, reminders, webhooks) — approval gates are structurally unsupported, so all gated tools are auto-denied immediately. +From 0.28.0, the well-known folders and files where other tools keep credentials are denied to the agent's tools too, for read and for write: -Compound commands (`cmd1 && cmd2 | cmd3`) are split into segments, each checked independently. Unapproved patterns are batched into a single prompt. +| Location | Denied | +|----------|--------| +| `~/.ssh`, `~/.aws`, `~/.gnupg`, `~/.kube`, `~/.azure` | The whole folder | +| `~/.config/gh`, `~/.config/gcloud` | The whole folder, and the same folders under `XDG_CONFIG_HOME` when it points elsewhere | +| `~/.docker/config.json` | That file only. Other files in `~/.docker` stay readable | +| `~/.netrc`, `~/.git-credentials` | The files | -| Channel | Approval Support | -|---------|-----------------| -| TUI, Slack, Discord, SignalR | Yes | -| Headless, reminders, webhooks | No — auto-deny | +On Windows the list adds `~/_netrc`, `%APPDATA%\gcloud`, `%APPDATA%\GitHub CLI`, `%APPDATA%\gnupg` and `%PROGRAMDATA%\ssh`. -## Secret Redaction +These are the results on 0.28.2, with a credentials file planted in each folder: -The redactor strips secrets from command stdout before the model sees them. It catches: +| Call | Result | +|------|--------| +| `file_read` of `~/.ssh/id_rsa` | `Error: Access denied: '/home/netclaw/.ssh/id_rsa' is protected ... and cannot be read by agent tools, even with approval.` | +| `file_write` or `file_edit` in `~/.ssh` or `~/.aws` | The same denial, ending "cannot be modified by agent tools" | +| `file_list` of `~/.aws` | The same denial. A listing of `~` leaves these entries out | +| `file_search` rooted at `~` | Skips the folders, so a match inside them never shows up | +| `shell_execute` of `cat ~/.aws/credentials`, `ls ~/.aws`, `echo x > ~/.ssh/config` | `Tool access denied: shell_references_protected_path`, including for the read-only catalog commands that never prompt | +| `cat ~/.docker/contexts.json` | Runs: only `config.json` is on the list | -- API key prefixes: `sk-*`, Slack API tokens (`xox[baprs]-...`), `ghp_*`, `AKIA*` -- `Authorization: Bearer` headers -- JSON fields matching sensitive names: `api_key`, `api-key`, `apikey`, `token`, `secret`, `password`, `authorization`, `access_token`, `refresh_token`, `client_secret`, `signing_key`, `private_key`, `connection_string`, `credential` -- Connection strings with `Password=` or `Pwd=` -- JWT tokens and PEM private key blocks +No approval, `ToolOverrides` entry or audience profile changes a denial. The programs that need these files still work: `ssh`, `git` and `aws` read their own credentials as the process netclaw starts, and no path in the command line names them. The check is on the paths a command names, so it protects against the agent reading a credential folder directly. It doesn't turn netclaw into a sandbox. See [what netclaw doesn't protect against](#what-netclaw-doesnt-protect-against). -For encryption at rest, see [`netclaw secrets`](/cli/secrets/). +### Private temp folder -## Prompt Injection Detection +On Linux and macOS each run gets its own temp folder at `/tmp/netclaw-//p` (`/private/tmp` on macOS), created with mode `0700`, and `TMPDIR`, `TMP` and `TEMP` point at it. Netclaw refuses a shell launch, `file_write` or `file_edit` below it when the owner, the mode or a link is wrong, and it doesn't fall back to another folder. A Team or Public session can reach only its own key folder, even if a configured root covers `/tmp`. -Netclaw scans inbound content — tool output, file contents, MCP server responses — for injection patterns. Detection returns a risk level per match; callers (the invocation stack layers) decide whether to reject or warn. +### Shell policy -| Category | Examples | Risk Level | -|----------|---------|------------| -| Prompt injection | "ignore previous instructions" | High | -| Role resets | `YouAreNow`-style patterns | Medium | -| Data exfiltration | "exfiltrate secrets via curl" | High | -| Privilege escalation | Access control list (ACL) modification, admin grants | High | -| Destructive operations | `rm -rf`, DROP TABLE | High | -| Invisible unicode | Zero-width chars, BiDi control characters | Medium | -| Private Use Area chars | PUA codepoints | Low | +Netclaw parses each shell command and judges every command in a pipeline or list separately. The prompt names only the commands that remain uncovered. A command it can't parse into a reusable phrase can still run once if you approve it, but can't be saved as a grant. Headless and unattended runs follow the same rules, with prompts turned into denials. See [How tool approvals work](/security/approvals/) for grants, the reviewed-safe catalog and what each answer saves. -High-risk matches are rejected outright. Medium-risk matches generate warnings. Low-risk matches are allowed through. +## Approval gates -## Content Validation +Approvals decide whether a permitted tool runs without asking. On Personal, `shell_execute` prompts unless you opt it out with an explicit override. Everything else is `Auto`. Writes under the config directory aren't a prompt case. The path policy denies them. MCP servers get a mode at `netclaw mcp add`. The mechanics, the modes and the saved-grant store are on [How tool approvals work](/security/approvals/), and the "stop asking" recipes are in [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/). -Before accepting a file upload, netclaw validates: +## MCP defaults -- MIME type must be on the audience's allowlist -- Maximum file size: 25 MiB -- File headers are checked against declared MIME types ([magic byte validation](https://en.wikipedia.org/wiki/List_of_file_signatures)) — renaming `malware.exe` to `photo.png` won't work +`netclaw mcp add` seeds each audience: -## Network Exposure +| Audience | Servers | Tool grants | Approval mode | +|----------|---------|-------------|---------------| +| Personal | All | All tools | `Auto` | +| Team | Must be in `AllowedMcpServers` | None until granted | `Approval` | +| Public | Must be in `AllowedMcpServers` | None until granted | `Deny` | -Exposure mode controls network reachability. It's separate from audience and posture: +A server absent from `McpServerToolGrants` exposes all its tools to an audience that can reach the server. Personal is therefore open by default, so review it first. See [MCP tool permissions](/guides/mcp-tool-permissions/). -| Mode | Scope | Requires | -|------|-------|----------| -| `local` | Loopback only | Nothing (default) | -| `reverse-proxy` | Whatever your proxy exposes | Trusted proxy config + non-loopback final hop | -| `tailscale-serve` | Your tailnet | [`tailscaled`](https://tailscale.com/kb/) | -| `tailscale-funnel` | Public internet | [`tailscaled`](https://tailscale.com/kb/1223/funnel/) | -| `cloudflare-tunnel` | Public internet | [`cloudflared`](https://developers.cloudflare.com/cloudflare-one/connections/connect-networks/) | +## Secret redaction -Internet-reachable modes force explicit confirmation during [`netclaw init`](/cli/init/) and trigger high-risk diagnostic warnings. Changing exposure mode requires a daemon restart — not hot-reloaded. +The redactor replaces secret-shaped text in tool output with `***REDACTED***` before the model sees it. It's a tripwire for accidental leaks, not a control. Don't rely on it to keep credentials out of model context. -`reverse-proxy` has an extra trust-boundary rule: loopback auto-auth is not inherited through the proxy path. A reverse proxy can front netclaw, but the final hop into the daemon must be a non-loopback internal address and the proxy source must be explicitly trusted. +It masks: -A Personal-posture deployment exposed via Tailscale Funnel is reachable from the internet but still applies Personal audience rules to TUI sessions and Team rules to Slack. Exposure and audience are orthogonal. +- **Known token formats** wherever they appear, whatever the name around them: OpenAI and Anthropic `sk-` keys, Slack `xox*-` and `xapp-` tokens, GitHub `ghp_`, `gho_`, `ghu_`, `ghs_`, `ghr_` and `github_pat_` tokens, AWS `AKIA` and `ASIA` key IDs, Stripe live keys and `whsec_` signing secrets, Google `AIza` keys, npm and PyPI tokens, Discord bot tokens, JWTs, `Authorization: Bearer` headers and Slack webhook URLs. A token followed by `-` or `.` is still masked +- **PEM private key blocks**, so `cat` of an OpenSSH or PEM key comes back as `***REDACTED***` +- **JSON fields** whose name starts with `api_key`, `token`, `secret`, `password`, `passwd`, `authorization`, `client_secret`, `private_key`, `connection_string`, `credential` or `passphrase` +- **`NAME=value` pairs** where the name starts with one of those words (`API_KEY`, `PASSWORD`, `TOKEN_X`) or has one as a whole underscore-separated segment (`DB_PASSWORD`, `GITHUB_TOKEN`, `AWS_SECRET_ACCESS_KEY`). The whole value is masked +- **Connection strings** with `Password=` or `Pwd=` +- **URL credentials**: the password in any `scheme://user:password@host` URL, whatever the name around it +- **Option values** after a secret option, such as `--password hunter2`, `--token x` or `--api-key x`. Options like `--password-file`, `--no-password` and `--next-token` are left alone +- **Header lines** that carry secrets: `Authorization` and `Proxy-Authorization` (the scheme stays), `Cookie`, `Set-Cookie`, and names such as `X-Api-Key` or anything ending in `-token` -## Fail-Closed Guarantees +Prefixed names skip the settings that carry those words without being secrets: names ending in a descriptor such as `_FILE`, `_PATH`, `_URL` or `_LENGTH` (`GH_TOKEN_FILE`), `max_tokens`, pagination cursors like `NEXT_PAGE_TOKEN`, and values that are references (`${{ secrets.GITHUB_TOKEN }}`, `$VAR`). -When in doubt, netclaw denies: +In a test on 0.28.2, `cat .env` in the shell came back with `API_KEY`, `DB_PASSWORD`, `GITHUB_TOKEN`, `AWS_SECRET_ACCESS_KEY` and `SLACK_BOT_TOKEN` masked, and the `DATABASE_URL` line as `DATABASE_URL=postgres://app:***REDACTED***@db.internal:5432/app`. It left `SESSION_KEY=abcdef123456` and `MYTOKEN=abc` as written. A secret in a custom format passes the same way. -- Invalid ACL schema in config → daemon refuses to start -- Policy engine throws an exception → deny -- Unknown channel type → Public audience (most restrictive) -- No config file → Public posture, shell off -- Approval gate timeout → auto-deny -- MCP server with no tool grants → all tools blocked +Two tools aren't masked at all. `file_read` returns file text as it is, so a read-edit-write cycle doesn't write `***REDACTED***` into your file. MCP tool results pass through too, because MCP servers are integrations you configured. The same `.env` read with `file_read` came back unmasked. Tool-call arguments in the logs are redacted too, and the full output behind `tool_output_read` is always the redacted text. -There is no permissive mode — access must be explicitly granted. +For encryption at rest, see [Secrets management](/security/secrets/). -## Customizing Approval Gates (Personal) +## Prompt injection detection -On the Personal audience, most tools auto-approve out of the box. Two things always prompt, regardless of config: +Netclaw scans inbound Slack, Discord and Mattermost messages, and thread history it pulls in, with regex patterns. A high-risk match blocks the message. Medium and low matches are logged and allowed through. Patterns cover instruction overrides, secret exfiltration, ACL or privilege manipulation, `rm -rf /` and disk destruction (high), and role assumption, `DROP TABLE`, mass deletion and invisible Unicode (medium or low). Skill content isn't scanned, so install only skills you trust. -- `shell_execute` — every shell command -- `file_write` / `file_edit` targeting netclaw's own config directory (`~/.netclaw/config`) — the *control plane* +## Content validation -These are fail-closed on Personal: the gate ignores your default mode and forces a prompt unless you explicitly opt the tool into `Auto`. Everything else on Personal is auto-approved by default. - -You tune this per-audience under `Tools.AudienceProfiles..ApprovalPolicy` in `~/.netclaw/config/netclaw.json`. Each tool resolves to one of three modes: - -| Mode | Behavior | -|------|----------| -| `Auto` | Runs immediately, no prompt | -| `Approval` | Prompts the operator before each run (the [Layer 4](#approval-gates) gate) | -| `Deny` | Always blocked — same as not granting the tool | - -### Stop shell from prompting - -Set an explicit override for `shell_execute`: - -```json -{ - "Tools": { - "AudienceProfiles": { - "Personal": { - "ApprovalPolicy": { - "ToolOverrides": { - "shell_execute": "Auto" - } - } - } - } - } -} -``` - -Then restart the daemon: - -```bash -netclaw daemon stop && netclaw daemon start -``` - -:::caution -Two non-obvious traps, because shell is fail-closed: - -- **`DefaultMode: "Auto"` won't do it.** `DefaultMode` is already the default, and the fail-closed check runs *before* it — so shell keeps prompting. You need the explicit `shell_execute` entry in `ToolOverrides`. -- **Deleting the entry won't do it either.** Removing `shell_execute` drops it back to the fail-closed default, which is a prompt. To silence shell you have to set it to `Auto`, not remove it. -::: +Before accepting an upload, netclaw checks that the MIME type is on the audience's allowlist, the file is at most 25 MiB, and the file header matches the declared type, so renaming `malware.exe` to `photo.png` doesn't work. -Netclaw's own config is different: no approval mode or grant lets a file tool (`file_write`, `file_edit`) write to the config directory. As of 0.28.0, override keys that contain a colon (such as the old `"file_write:control-plane"`) are ignored, and `netclaw doctor` reports them. +## Network exposure -### Precedence +Exposure mode controls who can reach the daemon and is separate from audience and posture. The default `local` binds loopback only and trusts a loopback request as the operator only when its `Host` is `localhost` or a loopback address and any `Origin` matches. Other requests get a 401, which blocks DNS rebinding and cross-site requests from a browser. See [Hardening](/security/hardening/#network-exposure). A Personal posture exposed through Tailscale Funnel is internet-reachable but still applies Personal rules to the TUI and Team rules to Slack. See [Exposure modes](/deployment/exposure-modes/). -A tool's mode is resolved in this order, first match wins: +## Fail-closed behavior -1. `ToolOverrides` entry for the tool (`shell_execute`, or an MCP `server/tool` key). Keys match tool names without regard to case. -2. `McpServerDefaults` entry for an MCP tool's server -3. Built-in fail-closed default — shell on Personal -4. `DefaultMode` +- A malformed `Tools` list, such as a scalar where a list belongs, stops the daemon at startup with an error naming the key. +- A policy rule that throws denies the call. +- An unresolved audience denies the call (`audience_unresolved`). +- No config file, or no `DeploymentPosture`, means Public posture with shell off, for the TUI too. +- A `netclaw.json` the CLI can't read leaves the local daemon address unknown. The CLI says so and sends no local control traffic instead of guessing the default address. +- Invalid model or provider configuration stops the daemon at startup, and a running daemon refuses to reload an invalid file. +- A prompt nobody can answer in an unattended run is denied (`approval_required_unattended`). +- A malformed or newer-version grant store denies every call that needs a grant (`approval_store_unavailable`). -### What this doesn't change +There's no permissive mode. Access is granted explicitly or not at all. -Relaxing an approval gate only removes the confirmation prompt. The other layers still apply: +## What netclaw doesn't protect against -- **Layer 1 hard-deny** still blocks `rm -rf /`, `sudo`, fork bombs, and the rest of the [operation hard-deny list](#layer-1-operation-hard-deny) — no override reaches it. -- **Layer 3 tool grants** still decide which tools exist for each audience. -- **Team and Public are untouched** — this is the Personal profile only. +Item 2 applies from 0.28.0. Earlier releases have no credential-location rule. -The tradeoff is real: with `shell_execute` on `Auto`, a [prompt-injected](#prompt-injection-detection) agent can run any non-hard-denied command without a human in the loop. The hard-deny list is your remaining backstop. +1. **Netclaw protects its own control plane.** That's the credential store (encryption keys, `secrets.json` and webhook route secrets), the netclaw database, the process-control files and writes to the netclaw configuration. The agent can't read or change them, in any command form it tries. +2. **It also denies a fixed list of credential locations.** `~/.ssh`, `~/.aws`, `~/.gnupg`, `~/.kube`, `~/.azure`, `~/.config/gh`, `~/.config/gcloud`, `~/.docker/config.json`, `~/.netrc` and `~/.git-credentials` are off limits to the agent's file tools and to shell commands that name them. See [credential locations](#credential-locations). The list is short on purpose: no list of folders covers every secret on a machine. +3. **Everything else, the agent reads as your user.** A `.env` file in a project, a token in a shell history file or a key in a folder that isn't on the list is readable, and the read-only catalog commands such as `cat` run without a prompt on any path the audience can read. Output redaction masks some of what comes back, not all of it. +4. **Approvals aren't a sandbox.** Approvals control which commands the agent runs. They don't limit what a command can read. If you give an "Always anywhere" grant to `bash`, `python3` or `node`, the agent can run any code as the same user as netclaw, and that code can open files the command line never names. Path denial can't see inside a program. +5. **Run netclaw in a sandbox when this matters.** Run it as a separate OS user and give that user only the files netclaw needs, or run it in a container. Netclaw has no built-in kernel sandbox for shell commands yet. The work is tracked in [netclaw-dev/netclaw#2343](https://github.com/netclaw-dev/netclaw/issues/2343). -## Limitations +The container route is the one the [Docker deployment](/deployment/docker/) page covers. The image runs the daemon as the non-root `netclaw` user, so the agent sees only what you mount into the container. -- Prompt injection detection uses regex pattern matching, not semantic analysis — novel phrasings can evade it -- Approval gates require an interactive channel — headless, reminder, and webhook sessions auto-deny all gated tools -- Secret redaction catches known patterns only — custom secret formats need custom hard-deny path rules -- Content validation checks magic bytes but doesn't deep-scan file contents for embedded threats -- Per-channel audience overrides exist ([`ChannelAudiences` config](/security/hardening/)) but require manual channel ID mapping — there's no UI for it yet +## Other limitations - +- Prompt injection detection is regex-based. Novel phrasings evade it. Treat it as a tripwire. +- Secret redaction catches known token formats and secret-named variables only. `file_read` and MCP results aren't masked. A custom secret format needs a path or hard-deny rule. +- Content validation checks magic bytes and doesn't deep-scan file contents. +- Releases through 0.27.1 had gaps that are now closed, such as TUI keystrokes written to a temp file and colon keys in `ToolOverrides`. See [Security advisories](/security/hardening/#security-advisories). +- Tool grants are per audience, not per channel. Two Slack channels with the same audience get the same tools. +- There's no per-user identity inside a channel. Everyone in an allowed channel gets that channel's audience. -## Related Pages +## Related pages -- [`netclaw init`](/cli/init/) — posture selection and network exposure configuration -- [`netclaw secrets`](/cli/secrets/) — encrypted credential storage and output redaction -- [`netclaw mcp`](/cli/mcp-tools/) — per-audience MCP tool permissions -- [`netclaw webhooks`](/cli/webhooks/) — HMAC verification and audience assignment for inbound webhooks -- [Hardening](/security/hardening/) — additional lockdown recommendations for production deployments +- [How tool approvals work](/security/approvals/) - modes, outcomes, grants and prompts +- [Reduce or turn off approval prompts](/guides/reduce-approval-prompts/) - stop prompts safely +- [Hardening](/security/hardening/) - lock down a team or internet-facing install +- [Secrets management](/security/secrets/) - encryption at rest and the write-only vault +- [Security architecture](/architecture/security-model/) - why the model is shaped this way -## Further Reading +## Resources -- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) — common attack vectors for LLM applications -- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) — federal guidance on AI system security -- [Model Context Protocol specification](https://modelcontextprotocol.io/specification) — the protocol netclaw uses for tool server integration +- [OWASP LLM Top 10](https://genai.owasp.org/llm-top-10/) - the risk taxonomy netclaw's model is designed against +- [NIST AI Risk Management Framework](https://www.nist.gov/itl/ai-risk-management-framework) - federal guidance on AI system risk +- [Model Context Protocol specification](https://modelcontextprotocol.io/specification) - the protocol behind MCP tool servers +- [Shell sandbox tracking issue](https://github.com/netclaw-dev/netclaw/issues/2343) - the planned kernel sandbox for shell commands diff --git a/src/content/docs/skills/bundled-resources.md b/src/content/docs/skills/bundled-resources.md index 8d171e8..c1da31c 100644 --- a/src/content/docs/skills/bundled-resources.md +++ b/src/content/docs/skills/bundled-resources.md @@ -1,20 +1,20 @@ --- -title: "Bundling Resources with Skills" -description: "How skills ship supporting files — references, scripts, assets — as deterministic archives, and how netclaw downloads and extracts them." +title: "Bundling resources with skills" +description: "How skills ship references, scripts and assets as deterministic archives, and how netclaw downloads, extracts and reads them." --- -A skill is more than its `SKILL.md`. Many carry supporting files: reference docs the agent can read, scripts it can run, assets it needs. A skill server bundles those into a single deterministic archive so netclaw can pull the whole skill in one verified download instead of fetching files one at a time. +A skill is more than its `SKILL.md`. Many carry reference docs the agent can read, scripts it can run and assets it needs. A skill server bundles those into one deterministic archive, so netclaw pulls the whole skill in a single verified download instead of fetching files one at a time. ## Two artifact shapes -A skill server projects every published skill into one of two shapes, based on whether it has supporting files: +A skill server projects every published skill into one of two shapes, depending on whether it has supporting files: | Shape | When | What's served | |-------|------|---------------| -| `skill-md` | `SKILL.md` only | the single markdown file | -| `archive` | `SKILL.md` + supporting files | a `.zip` at `/api/v1/skills/{name}/{version}/archive.zip` | +| `skill-md` | `SKILL.md` only | The single markdown file | +| `archive` | `SKILL.md` plus supporting files | A `.zip` at `/api/v1/skills/{name}/{version}/archive.zip` | -An archive holds `SKILL.md` at its root and the supporting files under their relative paths — the same layout the skill author used: +An archive holds `SKILL.md` at its root and the supporting files under their relative paths, the same layout the author used: ``` my-skill/ @@ -27,44 +27,59 @@ my-skill/ └── template.json ``` -Archives are **deterministic**: entries are ordered, timestamps are fixed, and Unix permission bits are preserved (masked to standard bits), so an executable script stays executable after extraction and the same skill version always produces a byte-identical archive with the same digest. +Archives are deterministic. Entries are ordered, timestamps are fixed and Unix permission bits are preserved (masked to standard bits), so an executable script stays executable and the same skill version always produces a byte-identical archive with the same digest. ## What the author does -Nothing special. Lay the skill out as a directory — `SKILL.md` plus optional `references/`, `scripts/`, and `assets/` — and publish it. The server decides the shape: a skill with supporting files becomes an archive automatically. See [Skill Server](/skills/skill-server/) for publishing. +Nothing special. Lay the skill out as a directory and publish it. The server decides the shape, so a skill with supporting files becomes an archive automatically. See [Skill server](/skills/skill-server/) and the [`skillserver` CLI](/skills/skillserver-cli/) for publishing. -Skills published before archives existed aren't left behind. The server **backfills** them on startup — it scans versioned skills that have resources but no archive, builds the deterministic archive from the stored files, and records it. The backfill is additive and idempotent: the original `SKILL.md` bytes, digest, and per-file routes stay exactly as they were, so nothing you already depend on changes, and a version that already has an archive is skipped. +Skills published before archives existed aren't left behind. On startup the server backfills them: it finds versioned skills with resources but no archive, builds the archive from the stored files and records it. The original `SKILL.md` bytes, digest and per-file routes stay exactly as they were, and a version that already has an archive is skipped. ## How netclaw consumes an archive -When a synced feed offers a skill as an archive, netclaw takes the bundle instead of individual files: +The server's RFC index lists each skill with its type, URL and digest. When an entry has type `archive`, netclaw takes the bundle instead of individual files: -1. **Download** the `.zip` from the archive URL. -2. **Verify** its SHA-256 digest against the manifest before touching the contents. A mismatch is rejected and logged — nothing gets extracted. -3. **Extract** with hardening: zip-slip path traversal, symlink entries, and duplicate names are all refused, and each file is content-scanned for prompt-injection the same way a single `SKILL.md` is. -4. **Write** the tree under `~/.netclaw/skills/.server-feeds///`, preserving the `references/` and `scripts/` subpaths and the Unix modes. +1. **Download** the `.zip` from the entry's URL. +2. **Verify** its SHA-256 digest against the index entry before touching the contents. A mismatch is rejected and logged, and nothing gets extracted. +3. **Extract** with hardening. Zip-slip path traversal, symlink entries and duplicate names are refused. +4. **Write** the tree under `~/.netclaw/skills/.server-feeds///`, keeping the subpaths and Unix modes. -The agent reads bundled files through the `skill_read_resource` tool, by logical name — it doesn't need to know the on-disk path. And because executable bits survive the round trip, a synced `scripts/check.sh` can be run through the shell tool. +Digest verification covers the archive as a whole, and netclaw extracts only after the whole bundle verifies. A partial or corrupted download never lands on disk. + +## How the agent reads and runs bundled files + +`skill_load` lists a skill's resource files. The agent reads one with `skill_read_resource`, passing the skill name and a path relative to the skill folder: + +```text +skill_read_resource → path: /home/netclaw/.netclaw/skills/hello-greeter/references/tone.md +Warm, brief, no emoji. +``` + +The first line of the result is the file's absolute path, so the agent can run a bundled script by its real path. The tool reads text files only. It refuses `SKILL.md` (use `skill_load`) and any path that crosses a symlink below the skill root. + +The skill folders aren't write-protected, and shell commands can name them. A synced `scripts/check.sh` keeps its executable bit, so the agent can run it through the shell tool. Which sessions can do that depends on the [audience](/security/security-model/): of the default audiences, only Personal includes `shell_execute`, and a Team or Public deployment posture takes it away from the local TUI too. + +The agent can also write into a feed skill folder, but it doesn't last. The next sync compares every installed file against the recorded hashes and reinstalls the published version. :::caution -Bundled resources are code and content from wherever the feed points. They ride the same trust boundary as any synced skill: digest-verified on the way in, content-scanned, and extracted under hardening — but a script you sync from a feed is a script you're trusting. Only subscribe to [skill feeds](/skills/skill-feeds/) you control or trust, and remember that a `Personal`-audience session is the only one that can run a synced script through the shell. +Bundled resources are content and code from wherever the feed points. netclaw verifies the digest, but it doesn't scan the content (skill scanning is off), and a script you sync from a feed is a script you're trusting. Subscribe only to [skill feeds](/skills/skill-feeds/) you control or trust. ::: ## Limitations - Archives are `.zip` only. -- Bundling is the server's call, driven by whether a skill has supporting files — there's no separate "make this an archive" switch. -- Per-file resource routes remain for compatibility, but archives are how netclaw pulls a resourceful skill; a client that only understands `skill-md` still gets the markdown and misses the extras. -- Digest verification covers the archive as a whole. Netclaw extracts only after the whole bundle verifies, so a partial or corrupted download never lands on disk. +- Bundling is the server's call. There's no switch to turn a skill into an archive. +- Per-file resource routes remain for compatibility. A client that only understands `skill-md` gets the markdown and misses the extras. +- Subagents can't bundle resources. They publish as a single file. ## Related pages -- [Native Manifest & Sync](/skills/native-manifest/) — how the archive URL and digest reach netclaw -- [Skill Feeds](/skills/skill-feeds/) — subscribing to the feeds that carry these skills -- [Skill Server](/skills/skill-server/) — publishing skills with resources -- [Images, Audio, and Other File Types](/guides/multimodal-files/) — how the agent reads the files it pulls +- [Skill feeds](/skills/skill-feeds/): subscribe to the feeds that carry these skills +- [Native manifest](/skills/native-manifest/): the feed that carries subagents +- [Skill server](/skills/skill-server/): publish skills with resources +- [Images, audio and other file types](/guides/multimodal-files/): how the agent reads other files it pulls ## Resources -- [AgentSkills.io](https://agentskills.io) — the open skill format these bundles follow -- [Zip Slip](https://security.snyk.io/research/zip-slip-vulnerability) — the path-traversal class netclaw hardens extraction against +- [AgentSkills.io](https://agentskills.io): the open skill format these bundles follow +- [Zip Slip](https://security.snyk.io/research/zip-slip-vulnerability): the path-traversal class netclaw hardens extraction against diff --git a/src/content/docs/skills/custom-subagents.md b/src/content/docs/skills/custom-subagents.md new file mode 100644 index 0000000..0af253e --- /dev/null +++ b/src/content/docs/skills/custom-subagents.md @@ -0,0 +1,193 @@ +--- +title: "Custom subagents" +description: "Define your own subagents as markdown files, spawn them for delegated work, and know their limits, timeouts and approvals." +--- + +A subagent is a child agent that a session spawns to handle bounded, delegated work such as research, code analysis or summarization, without filling the main conversation's context. netclaw ships three built-ins, and you add your own by dropping a markdown file in `~/.netclaw/agents/`. A [skill server](/skills/skill-server/) can also install subagents for you, so a whole team shares the same definitions. See [Distribute across your org](#distribute-across-your-org). + +## Before you begin + +- An initialized netclaw install ([`netclaw init`](/cli/init/)) +- A session with a Team or Personal audience. A `Public` session can't spawn subagents. + +## The agent file + +Each subagent is one markdown file in `~/.netclaw/agents/`: YAML frontmatter on top, the system prompt as the body. + +```markdown +--- +name: release-notes-writer +description: Draft release notes from a git log and merged PRs +modelRole: Main +--- + +You write release notes. Given a version range, read the git log and the merged +PRs, group changes into Features, Fixes and Docs, and write a concise changelog +entry in the project's existing style. Lead with user-visible impact. +``` + +The body is loaded verbatim as the subagent's system prompt, with no templating. The file name is for humans. The `name` field is the identity. + +| Field | Required | Default | Description | +|-------|----------|---------|-------------| +| `name` | Yes | - | Unique identifier, case-insensitive. This is what you pass to `spawn_agent`. | +| `description` | Yes | - | One line. The parent agent reads it to decide when to use this subagent. | +| `modelRole` | No | `Compaction` | `Main` (your primary model) or `Compaction` (the cheaper, faster one). Every role fails over to the Fallback model when one is configured. | +| `prefillTimeoutSeconds` | No | `SubAgents.PrefillTimeoutSeconds` (1800) | Wait for the first token, 5 to 3600. | +| `emitStructuredFindings` | No | `false` | When `true`, a successful run becomes a memory candidate the parent session can review. | +| `visibility` | No | `user-facing` | `user-facing` agents can be spawned and are listed. `internal` ones are hidden. | +| `tools` | No | `[]` | Advisory only. It never grants or restricts a tool. See [Tools and approvals](#tools-and-approvals). | + +A file with a missing `name` or `description`, an empty body, a bad `modelRole` or `visibility`, or an out-of-range `prefillTimeoutSeconds` is skipped. The daemon logs why, and `spawn_agent` names the file and the reason. Other agents keep loading. A file that still sets `timeoutSeconds` loads, but the daemon logs a warning on each reload and ignores the key, so delete it. A duplicate `name` is skipped too, and the first definition wins. + +## Built-in subagents + +`netclaw init` seeds three as ordinary editable files in `~/.netclaw/agents/`: + +| Name | What it does | +|------|-------------| +| `research-assistant` | Deep web research with search and citation | +| `code-analyst` | Analyze code and review files | +| `summarizer` | Summarize documents and content | + +Edit them, delete them, or copy one as a template. They aren't special. A container that never ran `init` starts with an empty `agents/` folder. + +## Reload + +Add, edit or remove a file in `~/.netclaw/agents/` and netclaw picks it up on the next turn. No daemon restart. If you edit a working file into an invalid one, netclaw drops it from the catalog rather than serving the last good copy, so one typo can't take down the others. + +## Spawn a subagent + +The parent agent sees every user-facing subagent and its description on each turn, then calls `spawn_agent`: + +| Parameter | Required | Description | +|-----------|----------|-------------| +| `agent` | Yes | The `name` of a `user-facing` subagent | +| `task` | Yes | A specific, bounded task for this run | +| `context` | No | Background for this run only, placed ahead of the task in the subagent's first message | + +`spawn_agent` returns at once and the subagent runs in the background. The parent sees only this: + +```json +{"run_id":"c574e00eb630433ea912787c5bbfc403","scope_id":"signalr/8c306da35c864a3b95fdc0689611eacf/subagent/release-notes-writer/c574e00eb630433ea912787c5bbfc403","state":"accepted","control_tool":"check_agent_run"} +``` + +The parent is told not to wait or poll. It ends its turn, and the person can keep talking to the session. When the child ends, netclaw writes `result.md` in the run's folder under `~/.netclaw/sessions//subagents//` and the parent gets one notice in a later turn, under the authority of the person who started the child: + +```text +Child run ended. agent=release-notes-writer run=c574e00e... outcome=completed result_file=/home/netclaw/.netclaw/sessions//subagents//result.md +``` + +The parent reads `result.md` with `file_read`. It holds the outcome, the reason, the log path, the artifact directory, the child's output and its working-context changes. The parent treats that text as data from the child and doesn't follow instructions in it. The run folder also holds `task.md`, the agent, requester and brief the run started with. + +```markdown +# Child result + +- agent: release-notes-writer +- run: c574e00eb630433ea912787c5bbfc403 +- outcome: completed +- log: .../subagents/c574e00e.../logs/session.log +- artifacts: .../subagents/c574e00e.../artifacts + +## Output + +**Docs:** Fixed a typo in the README for clearer documentation. +``` + +`outcome` is `completed`, `partial` or `failed`, with a `reason` when it isn't `completed`. Start and finish events also appear in headless output as `[subagent:start]` and `[subagent:done]`. `netclaw chat -p` stays open while a child run is open and prints `[subagent:waiting] N open: ` after the turn. The next turn reads the child's `result.md` and answers, then the command exits 0. `--json` suppresses the waiting line. + +To check on a child or stop it, the parent calls `check_agent_run` with the `RunId` and, to cancel, `Cancel: true`. It can manage only the children it owns. A cancel is a request, not the end of the run, so the parent waits for the notice. A reminder turn can manage the child runs of its own session. `spawn_agent`, like every tool, needs a `_rationale`. + +The subagent inherits the parent's project directory, working directory and audience. It also inherits the parent's [`AGENTS.md` operating rules and project instructions](/architecture/context-structure/#subagent-context), but runs a much thinner context: no persona, memory or conversation history. A skill can run a subagent too, also in the background. See [routing in the overview](/skills/overview/#how-skills-get-used). The bundled `agent-coordination` skill teaches the parent to split substantial work into scoped child tasks, review the evidence and read `result.md` and any partial artifact as data. + +## Limits and timeouts + +| Limit | Value | Notes | +|-------|-------|-------| +| Tool iterations per run | No cap | A final reply ends the run. Cancellation, the watchdogs below and the loop guard can stop it. A sixth refused repeat, or a failed-call score of 8, ends it as `partial` with reason `tool_cycle_stopped`. | +| Silence between tokens | `Session.FirstTokenTimeoutSeconds` (600) | Applies to every subagent after the first token. | +| First token | `prefillTimeoutSeconds`, default `SubAgents.PrefillTimeoutSeconds` (1800) | Covers queue time and cold prefill on a slow local model. | +| No output at all | `SubAgents.NoProgressTimeoutSeconds` (1200, range 60 to 7200) | Text, tool calls and thinking reset it. Keepalives don't, so it also caps the first-token wait. | +| Thinking only | 3 times `SubAgents.NoProgressTimeoutSeconds` (60 minutes by default) | One model call that streams only thinking ends the run as `failed` with `thinking_only_timeout`. | +| Cut-off reply | The model's output limit | A reply that stops at the limit with no tool call ends the run as `failed` with `output_truncated`. | +| Turns | One | A subagent takes a task, runs, and answers. No back and forth with the user. | +| Recursion | None | Subagents can't call `spawn_agent`, `check_agent_run` or `attach_file`. | +| Memory | No durable writes | A subagent can return structured findings for the parent to review. | +| Model | `Compaction` unless `modelRole: Main` | | + +The `spawn_agent` call itself finishes when the run is accepted, so there's nothing to cancel on the parent side. The child belongs to the session and outlives the model call. It stops on `check_agent_run` with `Cancel: true`, on session shutdown, on a restart that can't park it, or on its own watchdogs. + +Set the global timeouts in `netclaw.json`. `Enabled: false` turns the whole subsystem off: + +```json +{ + "SubAgents": { + "Enabled": true, + "PrefillTimeoutSeconds": 1800, + "NoProgressTimeoutSeconds": 1200 + } +} +``` + +## Tools and approvals + +A subagent gets the tools the parent session's audience policy allows, minus `spawn_agent`, `check_agent_run` and `attach_file`. The `tools:` field in the file is advisory metadata for compatibility with other agent systems and never narrows what a subagent can use. To change what a subagent can do, change the audience policy. See the [security model](/security/security-model/). + +If a subagent hits a tool that needs approval, it parks. The rest of the calls in its batch finish first, and the session saves the park and its consent requests in the run record. The parent gets one notice and pauses: while a child request is open, reminders for the session are held and `spawn_agent` is refused. The human who started the turn answers on the prompt's buttons. Time spent waiting for a human doesn't count against any timeout. + +After the last answer, the child runs only the parked calls, once each, and the approval gate checks each call again against the current policy. A resumed child has to load any deferred tools again. + +- With no human around, such as a reminder or webhook run, the gated tool is denied with `approval_required_unattended`. The tool doesn't run, and the subagent sees the denial as a tool result. +- A park survives a config reload, a compaction and a daemon restart, and one click resumes it. A park over 4 MiB ends the run as `park_too_large`, and a damaged one as `park_damaged`. +- A prompt that nothing can answer any more is redrawn without buttons and with the reason. On Mattermost, a child's prompt needs a `CallbackUrl` or netclaw refuses the request. + +Stored "always" grants decide what runs unattended. See [How tool approvals work](/security/approvals/). + +## Distribute across your org + +Subagents travel through the same skill server as skills. Publish a definition once, and every netclaw instance subscribed to the feed installs it under `~/.netclaw/agents/.server-feeds//`: + +- Your own files in `~/.netclaw/agents/` always win. If a feed agent has the same `name`, netclaw skips the feed copy and logs a warning. +- Between feeds, the one listed first in `SkillFeeds.Feeds` wins. +- Don't edit files in `.server-feeds/`. The next published version replaces them. To change a shared agent, publish a new version, or copy it to the top-level folder under the same name. + +[Distributing subagents](/skills/distributing-subagents/) covers the publish and sync steps. + +## Troubleshooting + +### spawn_agent says the agent isn't available + +- **Symptom:** The tool returns `Unknown agent` or `This tool is not available`. +- **Cause:** The `name` doesn't match a loaded file, the agent is `internal`, the session is `Public`, or `SubAgents.Enabled` is `false`. +- **Fix:** Check the exact `name` and that the file loaded. The daemon log in `~/.netclaw/logs/` has a warning for every file it skipped. + +### A subagent stalls or gets killed + +- **Symptom:** A run ends `failed` or `partial` after a long silence from the model. +- **Cause:** A silent stream outlasted `Session.FirstTokenTimeoutSeconds` (600), a slow local model needed longer to produce its first token, or a long think ran past `SubAgents.NoProgressTimeoutSeconds`. +- **Fix:** Raise `Session.FirstTokenTimeoutSeconds` for silence between tokens, `prefillTimeoutSeconds` in the agent file (or `SubAgents.PrefillTimeoutSeconds`) for the first-token wait, and `SubAgents.NoProgressTimeoutSeconds` for long thinking. + +### A subagent ends with a failure reason + +- **Symptom:** `result.md` shows `outcome: failed` or `partial` and a `reason`. +- **Cause:** `output_truncated` means the reply hit the model's output limit. `thinking_only_timeout` and `no_substantive_output_timeout` are the watchdogs above. `owner_restarted` means a restart lost the child, effects after its last checkpoint are uncertain, and netclaw doesn't start it again. `park_too_large` and `park_damaged` are the park limits above. +- **Fix:** For `output_truncated`, netclaw saves the full redacted cut-off text as `artifacts/truncated-output.md` in the run folder, marked partial. The parent reads it with `file_read` as data and doesn't follow instructions in it. Ask the child for shorter output or to write long output to a file. + +### A feed subagent never appears + +- **Symptom:** The server lists it, but `spawn_agent` can't find it. +- **Cause:** A local agent shadows it, or the sync reported `sidecar=failed`. +- **Fix:** Run `netclaw skill sync` and look at the line for the feed. See [Skill feeds](/skills/skill-feeds/#troubleshooting). + +## Related pages + +- [Distributing subagents](/skills/distributing-subagents/): publish and sync subagents through a skill server +- [Skills overview](/skills/overview/): skills, precedence and `metadata.subagent` routing +- [Context structure](/architecture/context-structure/): what a subagent sees compared to the parent +- [Security model](/security/security-model/): how audience controls subagent tool access +- [Architecture overview](/architecture/overview/): where subagents sit in the actor model + +## Resources + +- [YAML frontmatter](https://jekyllrb.com/docs/front-matter/): the metadata format these files use +- [Building effective agents](https://www.anthropic.com/engineering/building-effective-agents): background on delegating work to focused agents diff --git a/src/content/docs/skills/distributing-subagents.md b/src/content/docs/skills/distributing-subagents.md new file mode 100644 index 0000000..efbbdee --- /dev/null +++ b/src/content/docs/skills/distributing-subagents.md @@ -0,0 +1,238 @@ +--- +title: "Distribute subagents with a skill server" +description: "Publish a subagent to a skill server once and have every netclaw instance install it, update it and drop it." +--- + +To share a subagent across your team, publish it to a [skill server](/skills/skill-server/) with `skillserver publish-subagent`, and subscribe each netclaw instance to the server as a feed. The daemon installs the definition under `~/.netclaw/agents/.server-feeds//`, replaces it when you publish a new version, and deletes it when you remove it from the server. A subagent a user wrote in `~/.netclaw/agents/` always wins over a feed copy of the same name. + +## Before you begin + +- A running skill server and an API key that can publish. See [Skill server](/skills/skill-server/#create-a-publishing-key). +- The [`skillserver` CLI](/skills/skillserver-cli/#install) on the machine you publish from. +- netclaw 0.25.0 or later, the first stable release that syncs subagents, with the server configured as a feed. See [Skill feeds](/skills/skill-feeds/#add-a-feed). +- A subagent definition. See [Custom subagents](/skills/custom-subagents/#the-agent-file). + +The examples use `https://skills.example.com`. Swap in your own server. + +## 1. Write the definition + +A subagent is one markdown file in the format described in [Custom subagents](/skills/custom-subagents/#the-agent-file). The server publishes it as an `agent-md` artifact, the same file, byte for byte. + +```markdown +--- +name: changelog-reviewer +description: Review a changelog entry for clarity and missing user impact +modelRole: Compaction +--- + +You review changelog entries. Given a draft entry, point out vague wording and +missing user impact. Reply with a short bulleted list. +``` + +Two rules differ from a local agent: + +- The `name` is 1 to 64 lowercase letters, digits and single hyphens. The server rejects `Bad_Agent`. A file you keep locally isn't checked this strictly. +- The version isn't in the file. You pass it when you publish. Frontmatter `metadata.version` is ignored for subagents. + +## 2. Lint it + +`lint` runs on local files and needs no server or key, so it also works as a pull request check: + +```bash +skillserver lint subagent ./subagents/changelog-reviewer.md +skillserver lint subagents ./subagents +``` + +```text +Linting sub-agents in './subagents'... +Found 1 markdown file(s). + +All sub-agents valid (1 passed, 0 warning(s)) +``` + +It fails on a bad `name` or an out-of-range `prefillTimeoutSeconds`. The skill server still checks `timeoutSeconds` if a file sets it, but netclaw ignores that key, so leave it out. The server applies the same checks when you upload. + +## 3. Publish it + +Set the server URL and key, then publish with an explicit version: + +```bash +export SKILLSERVER_URL=https://skills.example.com +export SKILLSERVER_API_KEY=sk-... # a key from `skillserver api-key create` +skillserver publish-subagent ./subagents/changelog-reviewer.md --version 1.0.0 +``` + +```text +Publishing sub-agent changelog-reviewer@1.0.0... +Published sub-agent changelog-reviewer@1.0.0 + sha256:c914b9938fc543aa1f52753cfa62f14abe7c58dfcb50c2959b49929d8a6ea991 + https://skills.example.com/api/v1/subagents/changelog-reviewer/1.0.0/agent.md +``` + +Leave out `--version` and the command stops with `--version is required for sub-agent publication`. + +To publish a whole directory, use `publish-subagents`. It takes one `--version` and applies it to every file in the directory, and it skips versions that already exist, so you can run it on every push: + +```bash +skillserver publish-subagents ./subagents --version 1.0.0 +``` + +```text + changelog-reviewer@1.0.0 Skipped (Already published) + release-notes-writer@1.0.0 Published + +Results: 1 published, 1 skipped, 0 failed +``` + +Because a batch shares one version number, publish agents that change on different schedules one at a time. + +Confirm what the server holds: + +```bash +skillserver list-subagents +``` + +```text +NAME LATEST VERSIONS UPDATED +changelog-reviewer 1.0.0 1 2026-10-07 +release-notes-writer 1.0.0 1 2026-10-07 +``` + +## 4. Sync it to netclaw + +The daemon picks subagents up on its next sync pass: at startup, on the timer, or when you ask. To pull now, run: + +```bash +netclaw skill sync +``` + +```text +Skill sync pass ddf5b142f67241dfae5ec80af287ecdb +corp-skills: ok changed=0 unchanged=2 rejected=0 failed=0 sidecar=complete +Inventory: ok accepted=13 rejected=0 +``` + +`sidecar=complete` means the subagent pass finished. The `changed` count covers skills only, so a new subagent doesn't move it. The proof is on disk: + +```bash +ls ~/.netclaw/agents/.server-feeds/corp-skills/ +``` + +```text +changelog-reviewer.md +``` + +The next turn lists `changelog-reviewer` among the available subagents, and the agent can `spawn_agent` it. The run starts in the background, and the result comes back in a later turn. No restart is needed. + +## 5. Update or remove it + +Publish a higher version to update. The next sync downloads it, verifies its digest and replaces the file: + +```bash +skillserver publish-subagent ./subagents/changelog-reviewer.md --version 1.1.0 +``` + +To remove one, delete it from the server. The next sync deletes the local file, as long as every page and item in that pass synced cleanly: + +```bash +skillserver delete-subagent changelog-reviewer 1.0.0 --yes +``` + +Don't hand-edit files under `.server-feeds/`. netclaw doesn't hash-check subagent files the way it does skills, so your edit stays until the next version replaces it. To change a shared agent for one machine, copy it to `~/.netclaw/agents/` under the same name. That copy wins, and netclaw logs a warning that it skipped the feed version. + +## Pair a skill with a subagent + +A skill can hand its work to a subagent with `metadata.subagent`. Publishing the skill doesn't publish the agent, so publish both: + +```yaml +--- +name: changelog-check +description: Have the changelog reviewer check an entry before it ships. +argument-hint: "" +metadata: + version: 1.0.0 + subagent: changelog-reviewer +--- + +# Changelog check + +Review the changelog entry the user gives you. Flag vague wording and missing user impact. +``` + +```bash +skillserver publish-all ./skills +``` + +After a sync, `/changelog-check ` starts the subagent in the background with this skill's body as extra instructions and your text as the task. The target has to be `user-facing`, and if it's missing the skill reports an unknown target. See [routing in the overview](/skills/overview/#how-skills-get-used). + +## Publish from CI + +A repository with `skills/` and `subagents/` folders can publish on every merge. Both commands skip what's already published, so the job is safe to re-run: + +```yaml +- name: Lint + run: | + skillserver lint ./skills + skillserver lint subagents ./subagents +- name: Publish + env: + SKILLSERVER_URL: https://skills.example.com + SKILLSERVER_API_KEY: ${{ secrets.SKILLSERVER_API_KEY }} + run: | + skillserver publish-all ./skills + skillserver publish-subagents ./subagents +``` + +`publish-subagents` publishes every file in the folder at the `version` in its frontmatter. Give each agent file its own version and bump it in the file you changed: + +```markdown +--- +name: changelog-reviewer +description: Review a changelog entry for clarity and missing user impact +version: 1.1.0 +--- +``` + +Publishing a version that already exists is a no-op, so only the agents you bumped get republished, and only those re-download on each netclaw instance. Netclaw ignores the `version` key when it loads the agent. A file without one fails the publish unless you pass `--version` as a default. Keep `publish-subagent --version ` for a one-off publish by hand. + +## Trust + +A subagent definition is a system prompt, and it runs with the tools and approvals of the session that spawns it. Whoever can publish to the feed can change how your agents behave. + +- netclaw verifies the digest of every file it downloads and rejects a mismatch. It doesn't scan the prompt text, since content scanning is off. +- Every valid API key can publish. There's no publish-only scope or per-feed ACL, so limit who holds keys and keep the server on a private network. +- A synced subagent can't widen its own permissions. It gets the parent's audience tools, minus `spawn_agent`, `check_agent_run` and `attach_file`, and gated tools still ask for approval. See [Tools and approvals](/skills/custom-subagents/#tools-and-approvals). +- Review a definition before you publish it. A prompt that tells the agent to run commands or write files does what it says. + +## Troubleshooting + +### The subagent doesn't appear after sync + +- **Symptom:** The folder under `~/.netclaw/agents/.server-feeds/` is empty or the agent isn't spawnable. +- **Cause:** The sync line shows `sidecar=absent` (no native manifest at the URL), `sidecar=failed`, or `not-run` because the skill index request failed. +- **Fix:** Open `/subagents/v1/index.json` in a browser or with `curl`. If it 404s, the URL isn't a skill server. Check the daemon log for `Rejected native sub-agent` lines. + +### Sync rejects the file + +- **Symptom:** `sidecar=failed`, and the log says the frontmatter name differs from the advertised name, or the file isn't valid UTF-8. +- **Cause:** The `name` in the file doesn't match the name it was published under, or the definition fails the loader's rules. +- **Fix:** Run `skillserver lint subagent` on the file and publish a corrected version. + +### The local copy keeps winning + +- **Symptom:** You published an update, but `spawn_agent` still behaves the old way. +- **Cause:** A file in `~/.netclaw/agents/` has the same `name` and shadows the feed copy. +- **Fix:** Delete or rename the local file. The daemon log says `declares duplicate name ... earlier definition wins`. + +## Related pages + +- [Custom subagents](/skills/custom-subagents/): the file format, limits and approvals +- [Skill feeds](/skills/skill-feeds/): configure the feed and run syncs +- [Native manifest](/skills/native-manifest/): how the daemon discovers subagents on the server +- [`skillserver` CLI](/skills/skillserver-cli/): every publish and management command +- [Skill server](/skills/skill-server/): run the server and manage keys + +## Resources + +- [skill-server subagent specification](https://github.com/netclaw-dev/skill-server/blob/dev/docs/specs/subagents.md): the `agent-md` artifact and publication model +- [Semantic Versioning](https://semver.org): version numbers for published artifacts diff --git a/src/content/docs/skills/external-skills.md b/src/content/docs/skills/external-skills.md index 1e06fe6..666ffa7 100644 --- a/src/content/docs/skills/external-skills.md +++ b/src/content/docs/skills/external-skills.md @@ -1,39 +1,42 @@ --- -title: "External Skills" -description: "Adding custom skill paths and external sources." +title: "External skills" +description: "Read skills from Claude Code, Open Code or any directory on disk without copying them." --- -External skills let netclaw read skill directories owned by other AI tools — Claude Code, Open Code, or any arbitrary directory on disk. Point netclaw at an existing directory and it scans those skills alongside its own. No copying required. +External skills let netclaw read skill directories owned by other AI tools: Claude Code, Open Code, or any directory on disk. Point netclaw at an existing directory and it scans those skills alongside its own. No copying required. -Skills must follow the [SKILL.md format](https://agentskills.io) (frontmatter with `name` and `description` fields, markdown body). See [Skills Overview](/skills/overview/) for details on the format and lifecycle. +External skills use the [SKILL.md format](https://agentskills.io) with `description` in the frontmatter and a markdown body. See [Skills overview](/skills/overview/) for the format and where external skills rank. -## Quick Start +## Add a source -To add Claude Code's skills, use the CLI: +To add Claude Code's skills: ```bash netclaw skill source add claude-code --well-known claude-code ``` -That's it. The daemon picks up the change via its file watcher — no restart needed. +For any other directory, use `--path`: -Alternatively, run `netclaw config` → Skill Sources, then choose **Add a local skill folder** to browse and select a directory manually. +```bash +netclaw skill source add team-skills --path /opt/skills/shared +``` -## Before You Begin +Adding a source changes `netclaw.json`, so the daemon restarts itself in-process and picks up the new directory. You can also run `netclaw config`, open **Skill Sources** and choose **+ Add local folder** to browse for one. -- Netclaw is installed and initialized (or you're comfortable editing `netclaw.json` directly — it lives at `~/.netclaw/config/netclaw.json` by default) -- The external directory you want to add exists on disk (netclaw logs a warning for missing paths but still configures the source) +The directory has to exist. `skill source add --path` refuses a missing one with `[FAIL] directory not found`. -## Well-Known Sources +## Well-known sources -Well-known aliases expand to standard paths: +An alias expands to a standard set of paths: -| Alias | Resolves to | -|-------|-------------| -| `claude-code` | `~/.claude/skills/`, `~/.claude/commands/`, plus marketplace paths discovered dynamically (`~/.claude/plugins/marketplaces/*/skills/`) | +| Alias | Reads | +|-------|-------| +| `claude-code` | `~/.claude/skills/`, `~/.claude/commands/`, and `~/.claude/plugins/marketplaces/*/skills/` for each marketplace folder that exists | | `open-code` | `~/.open-code/skills/` | -The `claude-code` alias scans marketplace plugin directories alphabetically for stable precedence. Marketplace paths are discovered at runtime — if you install a new plugin, it gets picked up on the next scan. Missing one of those paths is fine — the alias resolves whichever ones exist. +The `claude-code` alias reads marketplace folders in alphabetical order, so precedence stays stable. It works out the list of marketplace folders when the daemon starts. Install a new marketplace and you need to restart the daemon (any config change does it) before netclaw sees it. New skill files inside a folder netclaw already watches show up live. + +The alias reads skills only. It doesn't read Claude Code plugin commands, agents, hooks or MCP servers, and it skips `~/.claude/plugins/cache/`. Flat `.md` files in `~/.claude/commands/` count as skills even without frontmatter. netclaw takes the name from the file name and the description from the first line. ## Configuration @@ -53,127 +56,94 @@ External sources live under `ExternalSkills.Sources` in `netclaw.json`: "Name": "team-skills", "Path": "/opt/skills/shared", "Enabled": true, - "AllowSymlinks": true + "AllowSymlinks": false } ] } } ``` -### Source fields - | Field | Type | Default | Description | |-------|------|---------|-------------| -| `Name` | string | — | Unique identifier for this source | -| `Path` | string | null | Absolute path to a skill directory (mutually exclusive with `WellKnown`) | -| `WellKnown` | string | null | Well-known alias (mutually exclusive with `Path`) | +| `Name` | string | - | Unique identifier for the source | +| `Path` | string | none | Absolute path to a skill directory. Set this or `WellKnown`, not both. | +| `WellKnown` | string | none | Well-known alias. Set this or `Path`, not both. | | `Enabled` | bool | `true` | Whether the source is active | -| `AllowSymlinks` | bool | `false` | Follow symlinks within the directory. Any well-known alias defaults to `true` when added via CLI (`--well-known`). | - -Each source must set either `Path` or `WellKnown`, not both. +| `AllowSymlinks` | bool | `false` | Follow symlinks inside the directory. `skill source add --well-known` sets it to `true`. | -## Managing Sources via CLI - -Add and remove sources without editing `netclaw.json`: +## Manage sources from the CLI ```bash -# List configured sources -netclaw skill source list - -# Add a well-known source -netclaw skill source add claude-code --well-known claude-code - -# Add a custom path -netclaw skill source add team-skills --path /opt/skills/shared - -# Disable a source without removing it -netclaw skill source disable team-skills - -# Re-enable +netclaw skill source list # show configured sources +netclaw skill source disable team-skills # keep it in config, stop scanning netclaw skill source enable team-skills - -# Remove entirely -netclaw skill source remove team-skills +netclaw skill source remove team-skills # config entry only, files stay ``` -All `netclaw skill source` commands work without the daemon running. CLI changes are picked up by a running daemon automatically via its file watcher. - -## Adding Sources via the Config Menu - -`netclaw config` → Skill Sources presents two options: **+ Add local folder** and **+ Add skill server**. - -Local folders are selected via a directory picker (Ctrl+N to create a new folder). After selection, you choose whether to allow symlinks in that folder, then confirm the source name. The source gets added and autosaved immediately. +These commands edit `netclaw.json` and don't need the daemon. A running daemon restarts itself when the file changes. Skill servers aren't managed here. See [Skill feeds](/skills/skill-feeds/). -![Skill Sources editor](/screenshots/output/config-skills.png) +![The Skill Sources screen with one local folder, Claude Code at /home/netclaw/.claude/skills showing 1 skill, one remote skill server, Corp Skills, and the Add local folder, Add skill server, Rescan all and Done rows](/screenshots/output/config-skills.png) -The Skill Sources screen — **+ Add local folder** opens a directory picker; **+ Add skill server** connects a remote feed. - -Leave symlinks off unless your setup requires it (shared filesystems, monorepo layouts with linked skill directories). +The Claude Code folder is the well-known source added with `netclaw skill source add`. In the Skill Sources screen, **+ Add local folder** opens a directory picker (Ctrl+N makes a new folder) and **+ Add skill server** connects a remote feed. ## Precedence -When multiple sources define a skill with the same name: - ``` -native skills > server feeds > external sources +system > native > server feeds > external ``` -[Native skills](/skills/overview/) are skills authored directly through netclaw's agent. [Server feeds](/skills/skill-feeds/) are remotely-synced skill repositories. External sources always have lowest priority. - -Within external sources, order in the `Sources` array determines which wins — higher-priority sources go first. Collisions are logged; run `netclaw skill issues` to see them. +External sources always rank last. Within them, the order of the `Sources` array decides, so put the one you want to win first. A shadowed skill shows up as a `DuplicateName` issue in `netclaw skill issues`. -## Runtime Behavior +## Live changes -The daemon watches every resolved external path for changes. When a `*.md` file changes, it rescans after a 500ms debounce — so multiple rapid writes only trigger one rescan. Drop a new skill file into an external directory and it's available on the next agent turn. No restart needed. - -The daemon logs a warning for missing directories at startup but keeps running. +The daemon watches every external path. When a `*.md` file changes, it rescans after a 500 ms pause, so several quick writes cause one rescan. A new skill file in an external directory is available to new sessions without a restart. A path that doesn't exist at startup is logged and skipped. ## Security -All well-known sources have `AllowSymlinks: true` set automatically when added via CLI — their standard paths often include symlinked directories. Custom sources default to `AllowSymlinks: false` — opt in per source if you trust the targets. - -External skills loaded from disk go through: - -- **Frontmatter validation** — `name` and `description` are required; malformed YAML is rejected -- **Symlink/path safety** — blocked unless `AllowSymlinks: true` for that source -- **[Tool access policies](/security/security-model/)** — same restrictions as native skills; the `allowed-tools` frontmatter field is informational only and doesn't grant tool access +netclaw doesn't scan external skills for prompt injection. Skill scanning is off for every tier, so point netclaw only at directories you trust. -External skills are scanned for prompt injection at load time, the same as every other skill - `skill_load` always scans before returning content. They skip only the write-time scan that runs when you create or edit a skill through netclaw, since you author these files yourself in another tool. +- Symlinks are blocked unless the source sets `AllowSymlinks: true`. Well-known aliases get it automatically because their standard paths often include symlinks. A custom path defaults to `false`. +- `allowed-tools` is informational and grants nothing. Tool access follows the [security model](/security/security-model/). +- Frontmatter has to parse and include a `description`. A skill without one is rejected with `MissingDescription`. -## Validating External Skills - -Validate skills before loading them: +## Validate a skill ```bash -# Validate a single skill netclaw skill validate /opt/skills/shared/deploy/SKILL.md - -# See all scanning/validation issues across all sources netclaw skill issues ``` +`validate` applies the native rules: the frontmatter `name` must match the directory name, and symlinks aren't followed. A skill that the daemon loads happily from an external source can still fail `validate` for either reason. `netclaw skill issues` scans the native and external sources and lists everything it rejected or shadowed. + ## Troubleshooting -### Source shows 0 skills after adding +### A source shows 0 skills -The path doesn't contain valid `*.md` files with [SKILL.md frontmatter](https://agentskills.io). Run `netclaw skill validate ` on a file to see what's wrong. Common causes: missing `name` or `description` in frontmatter, or the directory has subdirectories but no top-level markdown files. +- **Symptom:** `netclaw skill list` shows nothing from the source. +- **Cause:** The directory holds no `SKILL.md` directories or `.md` files with a `description`, or the source is disabled. +- **Fix:** Run `netclaw skill source list`, then `netclaw skill issues`. Missing `description` is the usual cause. -### "Symlink blocked" warnings in logs +### Symlink blocked -The source has `AllowSymlinks: false` and the directory contains symlinks. Either restructure the directory or set `AllowSymlinks: true` if you trust where the links point. +- **Symptom:** `netclaw skill issues` or the daemon log reports a blocked symlink. +- **Cause:** The source has `AllowSymlinks: false` and the directory contains links. +- **Fix:** Restructure the directory, or set `AllowSymlinks: true` if you trust where the links point. -### Skills not updating after file changes +### Edits don't show up -Filesystem events are debounced by 500ms. If changes still aren't showing up, check that the daemon is running (`netclaw status`) and the source is enabled (`netclaw skill source list`). +- **Symptom:** You changed a skill, but the agent still sees the old one. +- **Cause:** The daemon isn't running, the source is disabled, or the session started before the change. +- **Fix:** Check `netclaw status` and `netclaw skill source list`, then start a new session. -## What to Read Next +## Related pages -- [Skills Overview](/skills/overview/) — skill format, source types, and the full lifecycle -- [Skill Feeds](/skills/skill-feeds/) — server-synced skill repositories -- [`netclaw skill`](/cli/skill/) — full CLI reference for skill management +- [Skills overview](/skills/overview/): format, tiers and precedence +- [Skill feeds](/skills/skill-feeds/): skills from a skill server +- [`netclaw skill`](/cli/skill/): the full command reference +- [Slash commands](/reference/slash-commands/): run any skill with `/name` ## Resources -- [AgentSkills.io](https://agentskills.io) — the SKILL.md format specification -- [Claude Code documentation](https://docs.anthropic.com/en/docs/claude-code) — Claude Code setup and commands reference -- [FileSystemWatcher docs](https://learn.microsoft.com/en-us/dotnet/api/system.io.filesystemwatcher) — the .NET file watching API netclaw uses +- [AgentSkills.io](https://agentskills.io): the `SKILL.md` format specification +- [Claude Code documentation](https://docs.anthropic.com/en/docs/claude-code): Claude Code setup and its skill and plugin folders +- [FileSystemWatcher](https://learn.microsoft.com/en-us/dotnet/api/system.io.filesystemwatcher): the .NET file watcher netclaw uses diff --git a/src/content/docs/skills/native-manifest.md b/src/content/docs/skills/native-manifest.md index 6794979..6812d91 100644 --- a/src/content/docs/skills/native-manifest.md +++ b/src/content/docs/skills/native-manifest.md @@ -1,28 +1,32 @@ --- -title: "Native Manifest & Sync" -description: "How a skill server exposes skills and subagents to netclaw through the versioned native manifest, alongside the Cloudflare RFC feed." +title: "Native manifest and subagent sync" +description: "How a skill server publishes subagents through its versioned native manifest, and how netclaw reads it." --- -A skill server publishes two discovery feeds. The [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) feed at `/.well-known/agent-skills/index.json` is the vendor-neutral one — any tool that speaks the RFC can read it. The **native manifest** at `/manifest.json` is a richer, versioned feed for skill-server-aware clients like netclaw: it adds version negotiation, version history, subagents, and archive metadata the RFC feed doesn't carry. +A skill server publishes two discovery feeds. The [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) feed at `/.well-known/agent-skills/index.json` is the vendor-neutral one, and any tool that speaks the RFC can read it. The **native manifest** at `/manifest.json` is a versioned feed for skill-server-aware clients. It carries subagents, which the RFC has no place for. -The native manifest is a **sidecar, not a replacement**. It never supersedes the RFC feed — a client that only understands the RFC keeps working, and netclaw reads both. What the native manifest buys netclaw is the extra material: subagent definitions (which never appear in the RFC feed) and bundled-resource archives. +The two feeds sit side by side. netclaw takes skills from the RFC index, archives included, and takes subagents from the native manifest. ## The two feeds | | RFC feed | Native manifest | |---|----------|-----------------| | Entry point | `/.well-known/agent-skills/index.json` | `/manifest.json` | -| Audience | any RFC-compatible tool | skill-server-aware clients | -| Carries | skills | skills **+ subagents + archive metadata** | -| Versioned | no | yes (API version negotiation) | -| netclaw uses it for | skill discovery, digests | subagents, archives, version history | +| Audience | Any RFC-compatible tool | Skill-server-aware clients | +| Carries | Skills, with type (`skill-md` or `archive`), URL and digest | Skills and subagents, with version history | +| netclaw uses it for | Every skill, including archive downloads | Subagents only | ## How the manifest is shaped -The manifest is [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) — clients follow links rather than building URLs by hand. The root at `/manifest.json` lists the available API versions, each with a set of links: +The manifest is [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS): clients follow links instead of building URLs by hand. This is the root from a running 0.4.1 server: + +```bash +curl -s http://localhost:8080/manifest.json +``` ```json { + "$schema": "https://schemas.netclaw.dev/skillserver/manifest/0.1.0", "apiVersion": "v1", "versions": { "v1": { @@ -36,56 +40,66 @@ The manifest is [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) — clients fol } ``` -From there, a client follows `href` values down two collection trees — one for skills, one for subagents — each with its own index, pages, and per-item version history: +From there, two collection trees each have an index, pages and a per-item history: ``` -/skills/v1/index.json # skill collection -/skills/v1/{skill}/index.json # one skill -/skills/v1/{skill}/versions/{version}.json # a specific version -/subagents/v1/index.json # subagent collection -/subagents/v1/{agent}/versions/{version}.json +/subagents/v1/index.json collection index, lists its pages +/subagents/v1/pages/all.json the items +/subagents/v1/{agent}/index.json one subagent and its versions +/subagents/v1/{agent}/versions/{version}.json one version: type, url, digest +/skills/v1/... the same shape for skills ``` -The manifest links point at the REST API under `/api/v1/` for the actual bytes — `/api/v1/skills/{name}/{version}/archive.zip` for a bundled skill, `/api/v1/subagents/{name}/{version}/agent.md` for a subagent. Follow the links; the paths are readable but the server owns pagination boundaries, so don't hard-code them. +A version document tells a client where the bytes are and what digest to expect: -:::note -The manifest declares the routes, but the wire paths that matter are the ones the running server serves. If you're scripting against a server directly, read them off `/manifest.json` rather than a spec — the REST content lives under `/api/v1/`, which is easy to miss from the shorthand in the collection trees. -::: +```json +{ + "kind": "subagent-version", + "name": "changelog-reviewer", + "version": "1.0.0", + "type": "agent-md", + "url": "http://skills.example.com/api/v1/subagents/changelog-reviewer/1.0.0/agent.md", + "digest": "sha256:c914b9938fc543aa1f52753cfa62f14abe7c58dfcb50c2959b49929d8a6ea991" +} +``` + +The bytes live under `/api/v1/`. Follow the links and don't hard-code page names, because the server owns pagination. ## Version negotiation -`apiVersion` names the version the server recommends; `versions` lists every version it offers. A client picks the newest version it *also* supports: +`apiVersion` names the version the server recommends, and `versions` lists every version it offers. A client library can pick the newest one it also supports. netclaw doesn't negotiate. The daemon requests the `v1` subagent index directly and follows the links in what comes back. -1. The client knows its own supported versions (netclaw today: `v1`). -2. It reads the server's `versions`. -3. It uses the most recent version in the intersection. -4. If there's no overlap, it falls back to the oldest the server offers. +## What netclaw does with it -The payoff is forward compatibility in both directions: when a server ships `v2`, an older netclaw keeps using `v1`, and a newer netclaw automatically prefers `v2` — no coordinated upgrade. +On every sync pass, after the skills for a feed are installed, the daemon runs a subagent pass: -## What netclaw does with it +1. Fetch `/subagents/v1/index.json`, then each page it lists. +2. For each subagent, follow its identity document to the latest version, then to the version document. +3. Accept only type `agent-md`. Download the file and verify its digest. +4. Parse it as an agent definition. The frontmatter `name` has to equal the name the server advertised. +5. Write it to `~/.netclaw/agents/.server-feeds//.md` and record the version and digest in `.sync-state.json`. -On each sync, the daemon fetches the RFC index first, then — for feeds served by a skill server — follows the native manifest to pull what the RFC feed can't describe: +The pass reports its outcome as `sidecar=` on the [`netclaw skill sync`](/cli/skill/#sync-feeds-now) line. A server with no native manifest answers 404 and reports `absent`. -- **Subagents.** It walks the subagent collection, downloads each `agent.md`, verifies its digest, checks the frontmatter `name` matches, and writes it under `~/.netclaw/agents/.server-feeds//`. See [Custom Subagents](/guides/custom-subagents/). -- **Archives.** When a skill is published as an archive, netclaw pulls the bundled `.zip` instead of individual files. See [Bundling Resources with Skills](/skills/bundled-resources/). +A subagent that fails any step leaves the previous file in place. When the server stops advertising a subagent, netclaw deletes its local file, but only if every page and item synced cleanly in that pass. -Sync is **fail-soft and prune-after-confirm**: a feed that's unreachable or serves a bad digest doesn't take down the others or wipe your last-good copy, and stale entries are pruned only after a successful sync replaces them. A server that speaks only the RFC feed still works — netclaw just doesn't get subagents or archives from it. +Unlike skills, netclaw doesn't restore a synced subagent file you edited by hand. The file is replaced when the server publishes a new version. Edit your own agents in `~/.netclaw/agents/` instead, where a local definition always beats a feed copy. See [Distributing subagents](/skills/distributing-subagents/). ## Limitations -- The native manifest is skill-server-specific. A generic RFC feed won't expose subagents or archive metadata, so those features need a [skill server](/skills/skill-server/). -- Version negotiation covers the manifest API, not skill or subagent content versions — those are tracked per item in the collection trees. -- There's no transitive dependency resolution. A skill's `metadata.subagent` route is a runtime pointer, not a package dependency the manifest resolves for you. +- The native manifest is specific to skill servers. A generic RFC feed serves skills only. +- Subagents travel as a single `agent-md` file. There's no archive form for them, so a subagent can't bundle resource files. +- There's no dependency resolution. A skill's `metadata.subagent` route is a runtime pointer. Publishing the skill doesn't publish the subagent it names. ## Related pages -- [Skill Feeds](/skills/skill-feeds/) — configuring a feed and the sync lifecycle -- [Bundling Resources with Skills](/skills/bundled-resources/) — the archive artifacts the manifest points to -- [Custom Subagents](/guides/custom-subagents/) — defining the subagents the manifest distributes -- [Skill Server](/skills/skill-server/) — running the server that publishes it +- [Skill feeds](/skills/skill-feeds/): configure a feed and run a sync +- [Distributing subagents](/skills/distributing-subagents/): publish a subagent end to end +- [Bundling resources with skills](/skills/bundled-resources/): the archive form skills use +- [Skill server](/skills/skill-server/): run the server that publishes the manifest ## Resources -- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) — the vendor-neutral feed the manifest sits beside -- [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS) — the follow-the-links hypermedia style the manifest uses +- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc): the vendor-neutral feed the manifest sits beside +- [HATEOAS](https://en.wikipedia.org/wiki/HATEOAS): the follow-the-links style the manifest uses +- [Native manifest spec](https://github.com/netclaw-dev/skill-server/blob/dev/docs/specs/native-manifest.md): the server's own description diff --git a/src/content/docs/skills/overview.md b/src/content/docs/skills/overview.md index 7f41ae0..6e0d91b 100644 --- a/src/content/docs/skills/overview.md +++ b/src/content/docs/skills/overview.md @@ -1,49 +1,75 @@ --- -title: "Skills Overview" -description: "What skills are and how Netclaw uses them." +title: "Skills overview" +description: "What skills and subagents are, where netclaw finds them, which one wins a name clash, and how they reach a session." --- -Skills are markdown files that teach the agent *how to do things*. Rather than hard-coding every behavior into the daemon, you write procedural knowledge as plain text and the agent loads it on demand. +A skill is a markdown file that teaches the agent how to do one thing: your team's commit conventions, a deploy checklist, how to triage a failing pod. A subagent is a separate markdown file that defines a specialist the agent can hand a bounded task to. Both are plain text you can read, diff and keep in git. -Think of them as cheat sheets. A `commit` skill encodes your team's commit conventions; a `review-pr` skill walks through your code review checklist. The agent sees a one-line-per-skill index on every turn (just names and descriptions), then reads the full content only when it needs it. A large skill library costs almost nothing in context. +netclaw puts a one-line-per-skill index into the system prompt when a session starts. The model reads full instructions only when it needs them, so a big library costs a few lines of context. You can also start a skill yourself by typing `/skill-name` at the start of a message. -All skills follow the [AgentSkills.io](https://agentskills.io) format: a `SKILL.md` file with YAML frontmatter. The format is shared across multiple agent platforms, so skills you write for netclaw can work in Claude Code too. +Skills use the [AgentSkills.io](https://agentskills.io) format, a `SKILL.md` with YAML frontmatter, so most skills you write for netclaw also work in other agent tools. -## How skills reach the agent - -The lifecycle has four phases: +## Where skills come from -1. **Sync** - at startup, the daemon pulls netclaw's built-in skills from netclaw's own feed and (optionally) syncs skills from private [skill servers](/skills/skill-server/). These land in read-only directories under `~/.netclaw/skills/`. +```mermaid +flowchart TD + SRV[("Skill server")] -->|"sync"| FD["Feed folders"] + SYS["System skills
(in the binary)"] --> REG + NAT["Native skills"] --> REG + FD --> REG + EXT["External directories"] --> REG + MCP["MCP prompts"] --> REG + REG["Skill registry"] --> IDX["Index in the system prompt"] + IDX --> S["Session"] + S -->|"skill_load or /name"| BODY["Skill body and resources"] + AG["~/.netclaw/agents/"] --> SUB["Subagent registry"] + FD --> SUB + S -->|"spawn_agent"| SUB +``` -2. **Scan** — the daemon walks all skill directories (native, server feeds, external), validates frontmatter, and merges results. Native skills take precedence over server feeds, which take precedence over external sources. Netclaw logs name collisions from lower-precedence sources as issues and skips the duplicates. Run `netclaw skill issues` to see them. +| Tier | Location | Managed by | +|------|----------|-----------| +| system | `~/.netclaw/skills/.system/` | The daemon. Restored from the binary on every start. | +| native | `~/.netclaw/skills/` | You, or the agent through `skill_manage` | +| server feed | `~/.netclaw/skills/.server-feeds//` | A [skill server](/skills/skill-server/) you subscribe to. See [Skill feeds](/skills/skill-feeds/). | +| external | Directories you add | The tool that owns them. See [External skills](/skills/external-skills/). | +| MCP prompt | Not on disk | A connected [MCP server](/configuration/mcp-servers/) | -3. **Index** — the daemon compresses the merged list into a one-line-per-skill summary and injects it into the agent's system prompt. The agent sees skill names and descriptions but not the full instructions. +Subagent definitions live next to skills but load separately. Your own are in `~/.netclaw/agents/`, and a skill server can install more into `~/.netclaw/agents/.server-feeds//`. See [Custom subagents](/skills/custom-subagents/). -4. **Load** — when the agent needs a skill, it calls the `skill_load` built-in tool to read the full `SKILL.md` body and get a listing of resource files (scripts, references, assets). Users can also invoke skills directly via `/skill-name` slash commands. +### System skills -Changes to skill files trigger a rescan automatically. A file watcher monitors native and external source directories with a 500ms debounce, so edits take effect without a restart. Server feed directories are not watched — they update on the configured sync interval. +The daemon ships nine system skills, embedded in the binary: `agent-coordination`, `netclaw-memory`, `netclaw-operations`, `netclaw-projects`, `run-reminder`, `search-citation`, `skill-authoring`, `subagent-authoring` and `web-content-retrieval`. Nothing downloads. Every daemon start replaces the `.system` folder with the copies in the binary, so edits there don't survive a restart. -## Skill file format +There is no setting to turn system skill sync off. 0.27.0 removed `SkillSync.DisableSystemSkillSync`, and `netclaw doctor --fix` deletes it from an old config. -Two layouts work: +### Precedence -**Directory-based** — for skills with supporting resources: +When two sources define the same name, the higher tier wins and the other copy shows up as a `DuplicateName` issue: ``` -commit/ - SKILL.md # Required — instructions + frontmatter - scripts/ # Optional — helper scripts - references/ # Optional — reference material - assets/ # Optional — images, templates +system > native > server feeds > external ``` -**Flat file** — for self-contained skills: +Feeds rank in the order they appear in `SkillFeeds.Feeds`, and external sources in the order of `ExternalSkills.Sources`. A system skill beats a native skill of the same name, so you can't shadow `netclaw-memory` by dropping a file in `~/.netclaw/skills/`. A native skill beats everything you sync or point at. + +### MCP prompts + +Prompts from connected MCP servers appear as skills named `mcp____`. They show up in [`netclaw skill list`](/cli/skill/) with source `mcp`, and the agent loads them through `skill_load` with an `arguments` map. They aren't files, so there is nothing to edit, and they don't become `/` commands. + +## What a skill looks like + +Two layouts work. A directory holds a skill with supporting files, and a flat file holds a self-contained one: ``` -commit.md -``` +commit/ + SKILL.md # required: frontmatter + instructions + references/ # optional: material the agent can read + scripts/ # optional: scripts the agent can run + assets/ # optional: templates, images -The file name (or directory name) must match the `name` field in the frontmatter for native skills. External sources are more relaxed — the frontmatter `name` is treated as canonical. +commit.md # flat-file alternative +``` ```yaml --- @@ -51,101 +77,40 @@ name: commit description: Create well-structured git commits with conventional commit messages. metadata: version: 1.0.0 -license: MIT -allowed-tools: "bash git" +argument-hint: "" --- -# Commit Changes +# Commit changes -Instructions for the agent go here... +Instructions for the agent go here. ``` -### Frontmatter fields - | Field | Required | Description | |-------|----------|-------------| -| `name` | Yes | Lowercase identifier (must match directory/file name for native skills) | -| `description` | Yes | One-line summary (max 1024 chars per the [AgentSkills.io](https://agentskills.io) spec) | -| `metadata.version` | No | [Semver](https://semver.org) version string | -| `license` | No | [SPDX license identifier](https://spdx.org/licenses/) | -| `compatibility` | No | Which agent platforms can use this skill | -| `allowed-tools` | No | Space-delimited list of tools the skill needs (informational — does not grant access) | -| `disable-model-invocation` | No | When `true`, excluded from the LLM index (still invocable via `/name`) | -| `invocable` | No | Default `true`; when `false`, hidden from slash commands | -| `argument-hint` | No | Hint text shown after the slash command in completions | -| `metadata.subagent` | No | Routes execution to a named subagent instead of returning skill content | - -The display name shown in `netclaw skill list` comes from the first `#` heading in the markdown body, falling back to title-casing the skill name. - -## Quick setup - -To add an external skill directory or connect a skill server, run [`netclaw config`](/cli/config/) → Skill Sources. The TUI opens an interactive picker — no manual JSON editing required. - -```bash -netclaw config -``` - -Select **Skill Sources** to add a local folder (e.g., `~/.claude/skills/`) or subscribe to a private [skill server](/skills/skill-server/). Changes take effect on the next scan without a restart. - -![Skill Sources editor](/screenshots/output/config-skills.png) - -Manual configuration (e.g., for scripted or Docker installs) is covered in [External Skills](/skills/external-skills/) and [Skill Feeds](/skills/skill-feeds/). - -## Where skills come from - -Four source types, each with different trust and management: - -| Source | Location | Managed by | Mutable? | -|--------|----------|-----------|----------| -| **system** | `~/.netclaw/skills/.system/` | Daemon (netclaw's feed) | No | -| **native** | `~/.netclaw/skills/` | You | Yes | -| **server feeds** | `~/.netclaw/skills/.server-feeds/` | Private [skill servers](/skills/skill-server/) | No | -| **external** | Configured directories | Source tool (Claude Code, etc.) | At source | +| `description` | Yes | One line, up to 1024 characters. The model sees it in the index, so say when to use the skill. | +| `name` | No | Lowercase identifier. Defaults to the directory or file name. When you set it on a native skill it must match that name. | +| `metadata.version` | No | A version string like `1.0.0`. A top-level `version:` key is ignored, and `netclaw skill list` shows `-`. | +| `invocable` | No | Defaults to `true`. Set `false` to hide the skill from `/` commands. | +| `disable-model-invocation` | No | Set `true` to keep the skill out of the index. You can still run it with `/name`. | +| `argument-hint` | No | Text shown after the command name, like ``. | +| `metadata.subagent` | No | Name of a subagent to run the skill instead of returning its text. | +| `allowed-tools` | No | Informational only. It never grants access. | +| `license`, `compatibility` | No | Free-form strings. `compatibility` is a single string, not a list. | -Drop a `SKILL.md` into `~/.netclaw/skills/` and it appears on the next scan - that's a native skill. You can also ask netclaw's agent to write one for you - it has a `skill_manage` tool for creating and editing skills. Inspect or remove skills with [`netclaw skill`](/cli/skill/), and run `netclaw skill validate` to check a skill's frontmatter before using it. +netclaw ignores unknown keys, so a misspelled one fails silently. The key is `invocable`, not `user-invocable`. `netclaw skill list` shows the `name` key. The `Display:` line in `netclaw skill show` comes from the first `#` heading in the body, or from the title-cased name when there is none. -System skills ship from netclaw's own feed and are read-only. Built-in skills like `netclaw-memory` and `netclaw-operations` land here on first run. For a fully offline setup, set `SkillSync.DisableSystemSkillSync: true` in your `netclaw.json` (see [Configuration](/configuration/models/) for file location). +Files under a skill directory other than `SKILL.md` count as resources. A skill server can ship them as a zip archive, and netclaw extracts it into an ordinary skill folder. See [Bundling resources with skills](/skills/bundled-resources/). -External skills let netclaw read skill directories from other AI tools. Add a local folder via `netclaw config` → Skill Sources or wire up a well-known alias via the CLI. +## How skills get used -Well-known aliases expand to standard paths: +| Path | What happens | +|------|-------------| +| The model loads a skill | It reads the index, calls `skill_load` with the name, and gets the body plus a list of resource files. It reads a resource with `skill_read_resource`, which returns the file's absolute path on the first line so the agent can run a bundled script. | +| You type `/name` | The text after the name becomes your message, and netclaw adds the skill body to that turn. With no text, netclaw tells the model you invoked the skill. See [Slash commands](/reference/slash-commands/). | +| A skill has `metadata.subagent` | Loading it, or typing its `/name`, starts that subagent in the background with the skill body as extra instructions and your text as the task. The call returns the accepted run, and the result arrives in a later turn as a notice that names `result.md`. The subagent runs with the session's effective audience. The target has to exist and be `user-facing`. The model has to pass a `task` when it loads one. | +| The model calls an MCP prompt skill | `skill_load` returns the prompt's messages. Pass `arguments` for prompts that take them. | -| Alias | Scans | -|-------|-------| -| `claude-code` | `~/.claude/skills/`, `~/.claude/commands/`, `~/.claude/plugins/marketplaces/*/skills/` | -| `open-code` | `~/.open-code/skills/` | - -Server feed skills sync from private [skill servers](/skills/skill-server/) that implement the [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc). These are self-hosted registries that distribute skills across your organization. Configure them in `netclaw config` → Skill Sources or add them later via `netclaw skill source add`. See the [netclaw skill-server](https://github.com/netclaw-dev/skill-server) reference implementation for setting up your own. - -## Precedence - -When multiple sources define a skill with the same name, the highest-precedence source wins: - -``` -native > server feeds > external -``` - -A native skill named `commit` shadows any `commit` from a server feed or external source. You can always override shared skills with your own version. - -## Security - -Skills are just text, but text that lands in an agent's instructions can carry prompt injection - hidden text that tries to hijack the agent's instructions. Netclaw runs a content scanner on both write operations (create, edit, sync) and at load time (every `skill_load` call scans before returning content): - -| Verdict | Risk | Action | -|---------|------|--------| -| **Allowed** | None or low risk detected | Skill proceeds normally | -| **Warning** | Medium | Skill loads, but the event is logged | -| **Rejected** | High (prompt injection detected) | Skill is blocked; `netclaw skill issues` surfaces the rejection | - -Additional guardrails: - -- Symlinks are blocked by default in native and system directories. External sources can opt in via `AllowSymlinks: true` in config. -- Sessions with a [public trust audience](/security/security-model/) cannot load skills. `skill_load` also returns an error when `SkillSync.Enabled` is `false`. -- Skills run through the same [tool access policies](/security/security-model/) as everything else. The `allowed-tools` frontmatter field is informational — actual tool grants are controlled by the security policy. - -## Subagent routing - -Instead of returning instructions, a skill can hand off to a subagent. Add `metadata.subagent` to the frontmatter: +Routing example: ```yaml --- @@ -156,40 +121,57 @@ metadata: --- ``` -When `skill_load` hits this skill, it spawns the named subagent with the skill's body as a system prompt overlay. The caller passes a `task` parameter to the spawned subagent. The target subagent must be registered and user-facing — internal-only subagents cannot be routed to. +The model sees the index only in sessions with a Team or Personal audience. Public sessions get no index, and `skill_load` returns an error for them. `SkillSync.Enabled: false` empties the index and makes `skill_load` and `skill_read_resource` refuse for every audience. + +New and edited skill files are picked up without a restart. A file watcher on the native, feed and external directories rescans after a 500 ms pause. New sessions see the updated index. Changing which sources exist, such as adding an external directory, restarts the daemon in-process. + +## Trust and safety + +netclaw does not scan skill content. The scanner is turned off ([netclaw#976](https://github.com/netclaw-dev/netclaw/issues/976)), so a skill from any tier loads as written. Prompt-injection detection still runs on inbound channel messages. Treat a skill as instructions you are giving the agent and install only skills you've read or whose source you trust. + +What does protect you: + +- Feed downloads are checked against the server's SHA-256 digest, and a mismatch is rejected. +- Synced files are hash-checked. If anything edits a feed skill folder or a system skill, the next sync or daemon start puts the published copy back. +- `skill_manage` refuses to edit system, feed and external skills. Only native skills are editable. +- `allowed-tools` grants nothing. Tool access comes from the [security policy](/security/security-model/), and a skill runs under the same approvals as any other turn. +- Symlinks are rejected in native and system directories. An external source can opt in with `AllowSymlinks`. + +The skill folders are writable by the agent, so a script bundled with a feed skill can run through the shell tool. That only works in sessions whose audience allows `shell_execute`. ## Configuration -Run `netclaw config` to manage skill sources interactively. Two keys in `netclaw.json` also control skill loading directly: +Most setup happens in [`netclaw config`](/cli/config/) under **Skill Sources**, where you add local folders and skill servers. -| Key | Default | Description | -|-----|---------|-------------| -| `SkillSync.Enabled` | `true` | Blocks the agent from loading skills via `skill_load` when `false` | -| `SkillSync.DisableSystemSkillSync` | `false` | Skip CDN sync for system skills | +![The Skill Sources screen with one local folder, Claude Code at /home/netclaw/.claude/skills showing 1 skill, one remote skill server, Corp Skills, and the Add local folder, Add skill server, Rescan all and Done rows](/screenshots/output/config-skills.png) -External sources and skill feeds have their own config sections. See [External Skills](/skills/external-skills/) and [Skill Feeds](/skills/skill-feeds/) for details. +The screen lists local folders with their skill counts, and remote servers appear alongside them with how many skills each advertises. -## Tracking skill usage +| Key | Default | Description | +|-----|---------|-------------| +| `SkillSync.Enabled` | `true` | When `false`, the skill index is empty and the skill tools refuse every call. | +| `SkillFeeds` | none | Skill servers to sync from. See [Skill feeds](/skills/skill-feeds/). | +| `ExternalSkills.Sources` | none | Extra skill directories. See [External skills](/skills/external-skills/). | -Run [`netclaw stats skills`](/cli/stats/) to see which skills the agent loads most often. Good for spotting which skills pull their weight and which to cut. +To see what the agent has loaded, run `netclaw skill list` (the daemon has to be running). To see which skills it uses, run [`netclaw stats skills`](/cli/stats/). ## Limitations -- No built-in skill marketplace or discovery beyond what your configured sources provide. -- Skill content scanning uses regex-based heuristics. It catches common prompt injection patterns but is not a comprehensive security boundary. -- The `allowed-tools` frontmatter field is informational. It does not grant tool access — that is controlled entirely by the [security policy](/security/security-model/). -- Skills are text-only. They cannot register tools, modify daemon behavior, or execute code directly. -- No versioning or rollback for native skills. If you overwrite a skill, the old version is gone. +- Skills are text. They can't register tools or change daemon behavior. +- There's no version history or rollback for native skills. Overwrite one and the old copy is gone, so keep yours in git. +- `skill list` shows skills from a skill server with source `native`, because they live under `~/.netclaw/skills/`. The offline commands `show`, `search`, `issues` and `remove` don't see them. See [`netclaw skill`](/cli/skill/). +- There's no public marketplace. Skills come from the sources you configure. -## What to read next +## Related pages -- [`netclaw skill`](/cli/skill/) — CLI reference for managing skills (list, show, validate, search, source management) -- [External Skills](/skills/external-skills/) — configuring external skill directories -- [Skill Feeds](/skills/skill-feeds/) — subscribing to server-synced skill repositories -- [Skill Server](/skills/skill-server/) — running a private skill registry for your org +- [Skill feeds](/skills/skill-feeds/): sync skills and subagents from a skill server +- [External skills](/skills/external-skills/): read skills from Claude Code or any directory +- [Custom subagents](/skills/custom-subagents/): define subagents and spawn them +- [`netclaw skill`](/cli/skill/): list, inspect and validate skills +- [Slash commands](/reference/slash-commands/): `/name` and the other commands you can type ## Resources -- [AgentSkills.io](https://agentskills.io) — the SKILL.md format specification -- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) — the protocol behind skill feeds and server-based distribution -- [netclaw skill-server](https://github.com/netclaw-dev/skill-server) — reference implementation of a self-hosted skill registry +- [AgentSkills.io](https://agentskills.io): the `SKILL.md` format specification +- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc): the protocol behind skill feeds +- [netclaw skill-server](https://github.com/netclaw-dev/skill-server): the self-hosted skill registry diff --git a/src/content/docs/skills/skill-feeds.md b/src/content/docs/skills/skill-feeds.md index 6509f9d..7dcdf52 100644 --- a/src/content/docs/skills/skill-feeds.md +++ b/src/content/docs/skills/skill-feeds.md @@ -1,52 +1,48 @@ --- -title: "Skill Feeds" -description: "Subscribing to private skill server feeds." +title: "Skill feeds" +description: "Sync skills and subagents from a private skill server, add a feed, and pull updates on demand." --- -Skill feeds connect your netclaw daemon to private skill servers. The daemon discovers available skills via the [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) protocol, downloads them, verifies integrity via SHA-256 digests, and makes them available to the agent — all automatically. +A skill feed connects the daemon to a [skill server](/skills/skill-server/). The daemon reads the server's skill index, downloads what changed, verifies each download against a SHA-256 digest, and installs skills and subagents where the agent can use them. Use a feed to give every netclaw instance in your organization the same skills without copying files around. -Skills follow the [AgentSkills.io](https://agentskills.io) open standard format. Both standards are vendor-neutral and supported across multiple agent platforms, so skills published to a feed work anywhere that speaks the protocol. +Feeds are the remote counterpart of [external skills](/skills/external-skills/), which point at directories on disk. -## How It Works +## How a sync pass works ```mermaid flowchart TD - A[Daemon starts] --> B[Sync all feeds in parallel] - B --> C{For each feed} - C --> D[Fetch /.well-known/agent-skills/index.json] - D --> E[Diff against local sync state] - E --> F{New or changed skills?} - F -->|No| G[Skip — already up to date] - F -->|Yes| H[Download skill files] - H --> I[Verify SHA-256 digest] - I -->|Mismatch| J[Reject — log warning] - I -->|OK| K[Content scan for prompt injection] - K -->|Rejected| J - K -->|Allowed| L[Write to ~/.netclaw/skills/.server-feeds/] - L --> M[Update sync state] - G --> N[Rescan all skill directories] - M --> N - J --> N - N --> O[Update agent skill index] - O --> P[Sleep SyncIntervalMinutes] - P --> B + A["Pass starts: daemon start,
timer or netclaw skill sync"] --> B["Each enabled feed, in order:
fetch the RFC skill index"] + B --> D{"New, changed or
edited locally?"} + D -->|"No"| M + D -->|"Yes"| F["Download SKILL.md
or the archive .zip"] + F --> I{"SHA-256 matches?"} + I -->|"No"| J["Reject and log"] + I -->|"Yes"| K["Install under
skills/.server-feeds/FEED/"] + J --> L + K --> L["Delete skills the
server stopped listing"] + L --> M["Fetch the subagent index and install
agent.md files under agents/.server-feeds/FEED/"] + M --> O["Rescan skills and refresh the index"] ``` -Each feed syncs independently. A failing server never blocks other feeds or daemon startup. On failure, the daemon falls back to on-disk skills from the last successful sync. +Feeds sync one after another. A feed that fails is skipped and the daemon keeps using the files from its last good sync, but a slow feed delays the ones after it by up to its `TimeoutSeconds`. -## Add a Feed via netclaw config +Skills come from the server's [RFC index](https://github.com/cloudflare/agent-skills-discovery-rfc). Each entry is either a single `skill-md` file or an `archive`, a zip that holds the skill's `references/`, `scripts/` and `assets/` as well. netclaw verifies the zip's digest first, then extracts it with hardening against path traversal, symlink entries and duplicate names. See [Bundling resources with skills](/skills/bundled-resources/). -The `netclaw config` → Skill Sources screen lets you add remote skill servers. Select "+ Add skill server," enter the base URL, and the daemon probes for `/.well-known/agent-skills/index.json`, reports the skill count (or shows the error), and suggests a name based on the hostname. If the server requires authentication, you'll be prompted for a bearer token. Add as many feeds as you need. +Subagents come from the server's [native manifest](/skills/native-manifest/), which the RFC index doesn't carry. A server with no native manifest still works for skills, and the pass reports `sidecar=absent`. -![Skill Sources editor](/screenshots/output/config-skills.png) +## Add a feed -The Skill Sources screen — choose **+ Add skill server** to add a remote feed by base URL. +Feeds aren't managed by `netclaw skill source`. That command only handles local directories. Add a server through `netclaw config` or by editing `netclaw.json`. -Each remote server in the list shows how many skills it **advertises** — the count from its `index.json`, rendered as `N advertised` on the server's row. That's what the server publishes, not necessarily what netclaw loads: the daemon can end up with fewer after content-scanning, hashing, and version filtering. A gap between "advertised" and what shows up in `netclaw skill list` is normal, not an error. +### In the config menu -## Manual Configuration +Run `netclaw config`, open **Skill Sources**, choose **+ Add skill server** and enter the base URL. The screen probes the server, shows how many skills it advertises, and suggests a name from the hostname. If the server needs a bearer token, it asks for one. -Remote feeds can only be added through `netclaw config` → Skill Sources, or by editing `~/.netclaw/config/netclaw.json` directly: +![The Skill Sources screen with one local folder, Claude Code at /home/netclaw/.claude/skills showing 1 skill, one remote skill server, Corp Skills, and the Add local folder, Add skill server, Rescan all and Done rows](/screenshots/output/config-skills.png) + +A skill server you add appears under **Remote skill servers**, like Corp Skills here. Once its feed has been read, the count next to it is what it publishes, which can be higher than what `netclaw skill list` shows once sync finishes. + +### In netclaw.json ```json { @@ -55,7 +51,7 @@ Remote feeds can only be added through `netclaw config` → Skill Sources, or by "Feeds": [ { "Name": "corp-skills", - "Url": "https://skills.corp.com", + "Url": "https://skills.example.com", "Enabled": true, "TimeoutSeconds": 30 } @@ -64,145 +60,136 @@ Remote feeds can only be added through `netclaw config` → Skill Sources, or by } ``` -## Feed Config Fields +The daemon restarts itself when `netclaw.json` changes and runs a sync pass as it starts. Run `netclaw doctor` after hand edits to catch schema errors. | Field | Type | Default | Description | |-------|------|---------|-------------| -| `Name` | string | — | Filesystem-safe identifier for this feed | -| `Url` | string | — | Base URL; daemon appends `/.well-known/agent-skills/index.json` | -| `Enabled` | bool | `true` | Toggle without removing the entry | -| `TimeoutSeconds` | int | `30` | HTTP timeout for this feed | -| `ApiKey` | string | `null` | Optional bearer token for authenticated feeds; supports the `ENC:` prefix for encrypted storage | - -Top-level `SkillFeeds` settings: +| `Name` | string | - | Feed identifier and folder name. Lowercase letters, digits and hyphens only (`^[a-z0-9-]+$`). | +| `Url` | string | - | Server base URL. The daemon adds `/.well-known/agent-skills/index.json`. | +| `Enabled` | bool | `true` | Turn a feed off without deleting it. | +| `TimeoutSeconds` | int | `30` | Per-request HTTP timeout, 1 to 300. | +| `ApiKey` | string | none | Bearer token. Supports the `ENC:` prefix for encrypted storage. | -| Field | Default | Description | -|-------|---------|-------------| -| `SyncIntervalMinutes` | `60` | Periodic sync interval. Set to `0` for startup-only sync. | +`SyncIntervalMinutes` (default `60`, range 0 to 1440) sets the timer. `0` means sync at startup only. -## Sync Behavior +A skill server serves reads without a key, so you only need `ApiKey` when something in front of it, like a proxy, requires one. A publishing key is a different thing and never belongs in netclaw's config. See [Skill server](/skills/skill-server/#manage-api-keys). -- **Startup sync** runs immediately with no jitter. Network failures are logged but never block the daemon from starting. -- **Periodic sync** fires every `SyncIntervalMinutes`. The first periodic tick adds 0-5 minutes of random jitter to stagger multiple instances hitting the same server. -- **Startup-only mode**: set `SyncIntervalMinutes: 0` to disable periodic sync entirely. Skills only update when the daemon restarts. -- **Isolation**: each feed syncs in its own background task. One slow or broken server has zero impact on the others. -- **Fallback**: on any failure, the daemon keeps serving the last successfully synced skills from disk. +## When syncs run -## Writing Skills for a Feed - -Any skill server publishes skills that follow the [AgentSkills.io](https://agentskills.io) format. Here's what you need to know. +| Trigger | Timing | +|---------|--------| +| Daemon start, including the restart after a config change | Immediately. A failure never blocks startup. | +| Timer | Every `SyncIntervalMinutes`. The first tick adds 0 to 5 minutes of random jitter. | +| `netclaw skill sync` | Right now. If a pass is already running, it waits for that one. | -### Directory Layout +To pull a skill you just published, run `netclaw skill sync`. It needs the running daemon. -``` -my-skill/ - SKILL.md # Required - references/ # Optional — reference material - scripts/ # Optional — helper scripts - assets/ # Optional — images, templates +```bash +netclaw skill sync ``` -### SKILL.md Format - -```yaml ---- -name: deploy-staging -description: Deploy the current branch to the staging environment. -version: 1.2.0 -category: devops -allowed-tools: "bash docker" ---- +```text +Waiting for the daemon's skill sync pass. Press Ctrl+C to stop this wait. +Skill sync pass a304191e2ce04fc28782ad22f8385c2d +corp-skills: ok changed=0 unchanged=2 rejected=0 failed=0 sidecar=complete +Inventory: ok accepted=12 rejected=0 +``` -# Deploy to Staging +Each feed gets one line. `changed` counts skills installed, updated or deleted, and `sidecar` reports the subagent pass: -Prerequisites: -- Docker CLI authenticated to the registry -- VPN connected to staging network +| `sidecar` | Meaning | +|-----------|---------| +| `complete` | Subagents synced, or the server has none. | +| `absent` | The server has no native manifest, so there's nothing to sync. | +| `failed` | A subagent request or file failed. Existing subagent files stay. | +| `not-run` | The skill index request failed, so subagents weren't attempted. | -## Steps +The command exits 1 if any feed reports `failed` or `rejected` above zero, or the inventory refresh fails. Ctrl+C stops the wait, not the pass. -1. Build the container image -2. Push to the staging registry -3. Trigger the deployment webhook +```text +corp-skills: failed changed=0 unchanged=0 rejected=0 failed=1 sidecar=not-run + Error: The RFC index request failed. ``` -### Frontmatter Fields +## What lands where -| Field | Required | Description | -|-------|----------|-------------| -| `name` | Yes | 1-64 chars, lowercase alphanumeric + hyphens (`^[a-z0-9]+(-[a-z0-9]+)*$`) | -| `description` | Yes | One-line summary visible to agents during discovery | -| `version` | Yes (for feeds) | [Semver](https://semver.org) string — bumping triggers a new publish | -| `category` | No | Aids search and filtering | -| `license` | No | [SPDX identifier](https://spdx.org/licenses/) | -| `compatibility` | No | Which agent platforms can use this | -| `allowed-tools` | No | Space-delimited tool list (informational only, does not grant access) | -| `invocable` | No | Default `true`; `false` hides from slash commands | -| `disable-model-invocation` | No | `true` excludes from LLM index (still invocable via `/name`) | -| `argument-hint` | No | Hint shown after the slash command name | -| `metadata.subagent` | No | Routes to a named subagent instead of returning content | +| What | Location | +|------|----------| +| Skills | `~/.netclaw/skills/.server-feeds///` | +| Subagents | `~/.netclaw/agents/.server-feeds//.md` | +| Sync state | `.sync-state.json` in each of the two feed folders | -The `version` field is required for feed publishing. The server uses it to detect changes — bump the version to trigger downstream syncs. +`netclaw skill list` labels feed skills `native`, because they live under `~/.netclaw/skills/`. To tell them apart, look at the path or the `corp-skills` folder. -## Security +### Updates, edits and removals -Every skill downloaded from a feed goes through the content scanner before it reaches the agent: +- **Updates.** A skill installs again when the server's version or digest differs from the sync state. Bump `metadata.version` in `SKILL.md` and publish. netclaw shows the version from that frontmatter key, not the number you passed to the publish command. +- **Local edits.** netclaw records a hash of every installed file. If anything edits, adds or deletes a file in a feed skill folder, the next pass reinstalls the published version. Subagent files aren't hash-checked, so an edited subagent stays edited until its version changes. +- **Removals.** When the server stops listing a skill, the next pass deletes it. Pruning only runs when the server returns a non-empty index, so an outage or an empty response never wipes your skills. Subagents are pruned only after every page and item synced cleanly. +- **First sync after an upgrade.** Skills installed before hash tracking get downloaded once more to record hashes. -| Verdict | Action | -|---------|--------| -| **Allowed** | Skill proceeds normally | -| **Warning** | Skill loads, event is logged | -| **Rejected** | Skill blocked; surfaces in `netclaw skill issues` | +### Name clashes -Resource files (`references/`, `scripts/`) are also scanned. SHA-256 digest verification runs before scanning — a tampered file is rejected before the content scanner even sees it. +Native and system skills beat feed skills, and feed skills beat external ones. Between feeds, the one listed first in `SkillFeeds.Feeds` wins. For subagents, any definition in `~/.netclaw/agents/*.md` beats a feed copy of the same name, and netclaw logs a warning. To override a feed skill, create a native skill with the same name. -Server feed skills are stored as read-only in `~/.netclaw/skills/.server-feeds/{feed-name}/`. They cannot be modified locally. +## Authoring for a feed -### Precedence +A feed serves skills in the [AgentSkills.io](https://agentskills.io) format. Three feed-specific rules: -``` -native skills > server feeds > external sources -``` +- Put `version` under `metadata:`. The skill server's CLI accepts a top-level `version:` key too, but netclaw ignores it and shows `-`. Under `metadata:` both read it. +- Names are 1 to 64 characters of lowercase letters, digits and single hyphens. +- A skill that has a `references/`, `scripts/` or `assets/` folder publishes as an archive automatically. + +The full format is in [Skills overview](/skills/overview/). Publishing is covered by the [`skillserver` CLI](/skills/skillserver-cli/) page. -A native skill with the same name always wins. To override a feed skill, create a native skill with the same `name` — your local version takes priority. +## Security -### Disabling All Skills +netclaw checks the SHA-256 digest of every skill, archive and subagent a feed serves and rejects a file that doesn't match. It does not scan skill content. The scanner is off ([netclaw#976](https://github.com/netclaw-dev/netclaw/issues/976)), so a feed skill loads exactly as the server published it. Subscribe only to servers you run or trust. See [Trust and safety](/skills/overview/#trust-and-safety). -Two related config keys control broader skill behavior: +Subagent files are prompts. A subagent from a feed runs with the same tools and approvals as one you wrote, so read what a feed publishes. -| Key | Effect | -|-----|--------| -| `SkillSync.Enabled: false` | Blocks the agent from loading *any* skills via `skill_load` | -| `SkillSync.DisableSystemSkillSync: false` | Set `true` to disable the built-in CDN feed; the default `false` keeps it enabled (separate from private feeds) | +`netclaw skill issues` doesn't show feed rejections. A rejected file shows up in the daemon log and in the `rejected` count of `netclaw skill sync`. ## Troubleshooting -### Feed shows 0 skills after adding +### A feed shows 0 skills -Open `netclaw config` → Skill Sources to confirm the feed is enabled, or check `~/.netclaw/config/netclaw.json` under the `SkillFeeds.Feeds` array. Then check the daemon logs — the most common cause is the server not serving `/.well-known/agent-skills/index.json` at the expected path. Verify with: +- **Symptom:** The server row says `0 advertised`, or no skills appear after a sync. +- **Cause:** The URL is wrong, or the server isn't serving its index. +- **Fix:** Check the index, and check that the feed is enabled in `netclaw config` or `SkillFeeds.Feeds`: ```bash -curl -s https://skills.corp.com/.well-known/agent-skills/index.json | head +curl -s https://skills.example.com/.well-known/agent-skills/index.json | head ``` -### Skills not updating +### A published skill doesn't show up + +- **Symptom:** You published a skill, but `netclaw skill list` doesn't show it. +- **Cause:** The timer hasn't fired yet, or the version didn't change. +- **Fix:** Run `netclaw skill sync`. If the line says `unchanged`, publish a new `metadata.version`. -Periodic sync fires every `SyncIntervalMinutes` (default 60). If you need immediate updates, restart the daemon — startup sync runs without delay. Also check that the skill's `version` was bumped on the server side. +### The sync line says `failed` -### "Content rejected" in skill issues +- **Symptom:** `netclaw skill sync` exits 1 with `failed=1`. +- **Cause:** The server was unreachable or slow, or one skill failed to download or verify. +- **Fix:** Read the `Error:` line and the daemon log at `~/.netclaw/logs/`. Existing skills stay in place. Raise `TimeoutSeconds` for a slow server. -The content scanner flagged prompt injection patterns in the skill body. Run `netclaw skill issues` for details. Work with the skill author to revise the content, or file an issue on the skill server if you believe it's a false positive. +### A subagent doesn't appear +- **Symptom:** The server lists a subagent, but `spawn_agent` can't find it. +- **Cause:** A local agent with the same name wins, the feed shows `sidecar=absent` or `failed`, or the file failed validation. +- **Fix:** Check `~/.netclaw/agents/.server-feeds//` for the file. A name clash and a validation failure both log a warning in the daemon log. -## Related Pages +## Related pages -- [Skills Overview](/skills/overview/) — format, lifecycle, and precedence rules -- [External Skills](/skills/external-skills/) — local skill directories from other tools -- [Skill Server](/skills/skill-server/) — running your own server that publishes feeds -- [`netclaw skill`](/cli/skill/) — CLI reference for skill management +- [Skills overview](/skills/overview/): tiers, precedence and trust +- [Skill server](/skills/skill-server/): run the server a feed points at +- [Native manifest](/skills/native-manifest/): how subagents reach netclaw +- [Distributing subagents](/skills/distributing-subagents/): publish a subagent to a feed +- [`netclaw skill`](/cli/skill/): `skill sync` and the other subcommands -## External Resources +## Resources -- [AgentSkills.io](https://agentskills.io) — the SKILL.md format specification -- [Cloudflare Agent Skills Discovery RFC v0.2.0](https://github.com/cloudflare/agent-skills-discovery-rfc) — the discovery protocol behind skill feeds -- [netclaw-dev/skill-server](https://github.com/netclaw-dev/skill-server) — reference implementation of a private skill server +- [AgentSkills.io](https://agentskills.io): the `SKILL.md` format specification +- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc): the discovery protocol behind skill feeds +- [netclaw-dev/skill-server](https://github.com/netclaw-dev/skill-server): the reference skill server diff --git a/src/content/docs/skills/skill-server.md b/src/content/docs/skills/skill-server.md index e0292b3..ac62614 100644 --- a/src/content/docs/skills/skill-server.md +++ b/src/content/docs/skills/skill-server.md @@ -1,42 +1,40 @@ --- -title: "Skill Server" -description: "Running and connecting a self-hosted skill server for skills and sub-agents." +title: "Skill server" +description: "Run a self-hosted skill server for skills and subagents, secure it with API keys, and point netclaw at it." --- -SkillServer is a self-hosted skill registry - a private NuGet feed or npm registry, but for [SKILL.md](https://agentskills.io) files and netclaw sub-agents. Host it behind your firewall, publish proprietary skills and sub-agents, and every netclaw instance on the network syncs from it automatically. +SkillServer is a self-hosted skill registry: a private NuGet feed or npm registry, but for [SKILL.md](https://agentskills.io) files and netclaw subagents. Host it behind your firewall, publish your proprietary skills and subagents, and every netclaw instance on the network syncs from it. Source code and releases: [github.com/netclaw-dev/skill-server](https://github.com/netclaw-dev/skill-server) -It implements two open standards, plus a native feed for netclaw-aware clients: +It implements two open standards and adds a native feed for netclaw: -- [Cloudflare Agent Skills Discovery RFC v0.2.0](https://github.com/cloudflare/agent-skills-discovery-rfc) - discovery via `/.well-known/agent-skills/index.json` -- [AgentSkills.io](https://agentskills.io) - the SKILL.md format for skill definitions -- [Native Manifest](/skills/native-manifest/) - a versioned `/manifest.json` feed that also carries sub-agents and packaged archives +- [Cloudflare Agent Skills Discovery RFC v0.2.0](https://github.com/cloudflare/agent-skills-discovery-rfc): discovery through `/.well-known/agent-skills/index.json` +- [AgentSkills.io](https://agentskills.io): the `SKILL.md` format +- [Native manifest](/skills/native-manifest/): a versioned `/manifest.json` feed that also carries subagents -Any agent that speaks the RFC can consume skills from your server, and netclaw additionally reads the native manifest for sub-agents and archive artifacts. +Any agent that speaks the RFC can read skills from your server. netclaw takes skills and archives from the RFC index and subagents from the native manifest. ## What it does -- Stores versioned skills and sub-agents in content-addressable blob storage (SHA-256) -- Serves an RFC discovery index and a native manifest that agents poll on an interval -- Packages skills with bundled resources as deterministic [archives](/skills/bundled-resources/) -- Ships a web gallery for browsing skills and sub-agents +- Stores versioned skills and subagents in content-addressable blob storage (SHA-256) +- Serves the RFC index and the native manifest that clients poll +- Packages skills with supporting files as deterministic [archives](/skills/bundled-resources/) +- Ships a web gallery for browsing skills and subagents - Protects writes with API keys once a key exists. Reads are open, so agents fetch without credentials -- Runs as a single container with no external dependencies, just SQLite and the filesystem +- Runs as a single container with SQLite and the filesystem, nothing else ## Deploy with Docker -Pull the image: +Pull the image. It's published for `linux/amd64` and `linux/arm64`: ```bash docker pull ghcr.io/netclaw-dev/skillserver:latest ``` -Available for `linux/amd64` and `linux/arm64`. - ### Set a bootstrap key -Generate and save a strong random secret in your password manager. Inject it through your deployment secret manager before the first startup. For a local setup, use a private Bash terminal: +Generate a strong random secret and keep it in your password manager. In production, inject it through your secret manager before the first start. For a local setup, read it into a variable in a private Bash terminal: ```bash read -r -s -p "Bootstrap API key: " SKILLSERVER_APIKEY @@ -44,13 +42,13 @@ printf '\n' export SKILLSERVER_APIKEY ``` -The Compose example below maps `SKILLSERVER_APIKEY` to the server's `SKILLSERVER__APIKEY` variable. Keep the value out of committed YAML and `.env` files. +The Compose file below maps `SKILLSERVER_APIKEY` to the server's `SKILLSERVER__APIKEY`. Keep the value out of committed YAML and `.env` files. :::caution -A database with no API keys leaves publishing, deletion, and key management unauthenticated. Seed a key before exposing the server. Reads and discovery remain open even with keys configured; private content also needs a private network or proxy access policy. +A database with no API keys leaves publishing, deletion and key management open to anyone who can reach the server. Seed a key before you expose it. Reads and discovery stay open even with keys, so private content also needs a private network or a proxy access policy. ::: -### Docker Compose +### Run it with Docker Compose ```yaml name: skillserver @@ -86,47 +84,55 @@ volumes: skill-data: ``` -Start it: +Start it, clear the variable from your shell, and check the health endpoint: ```bash docker compose up -d unset SKILLSERVER_APIKEY +curl http://localhost:8080/health ``` -Verify it's running: +The server hashes and stores the bootstrap key when the database has none. Next, [create a dedicated publishing key](#create-a-publishing-key) for your CLI or CI. -```bash -curl http://localhost:8080/health -``` +### Before you go to production -The bootstrap key is hashed and stored when the database has no keys. Next, [create a dedicated publishing key](#create-a-publishing-key) for your CLI or CI. +- Put a reverse proxy (Caddy, nginx, Traefik) in front for TLS. +- Set `SKILLSERVER__BASEURL` to your public URL, like `https://skills.internal.example.com`, so discovery responses carry correct absolute URLs. +- Mount `/data` on persistent storage. If the volume is lost, you re-publish everything. -## Configuration +All state lives in `SKILLSERVER__DATAPATH`, so a backup of that volume is a backup of the server. -All configuration is via environment variables: +## Publish and connect -| Variable | Default | Description | -|----------|---------|-------------| -| `SKILLSERVER__DATAPATH` | `./data` | SQLite database + blob storage directory | -| `SKILLSERVER__BASEURL` | `http://localhost:8080` | Base URL for absolute URLs in discovery responses | -| `SKILLSERVER__APIKEY` | *(none)* | Bootstrap API key, seeded on first run if no keys exist in DB | -| `ASPNETCORE_URLS` | `http://+:8080` | Listen address and port | +The [`skillserver` CLI](/skills/skillserver-cli/) publishes and manages content from your terminal or CI. The essentials: -All state lives in `SKILLSERVER__DATAPATH`. Back up that volume and you have everything. +```bash +# Publish one skill, or every skill directory under a parent +skillserver publish ./my-skill +skillserver publish-all ./skills + +# Publish subagents. The version comes from --version, not frontmatter. +skillserver publish-subagent ./release-notes-writer.md --version 1.0.0 +skillserver publish-subagents ./subagents --version 1.0.0 +``` + +Publishing a version that already exists is skipped, so `publish-all` and `publish-subagents` are safe to run on every CI push. A skill with a `references/`, `scripts/` or `assets/` folder becomes an [archive](/skills/bundled-resources/) that keeps relative paths and executable bits. A bare `SKILL.md` publishes as a lightweight `skill-md` artifact. The [Distributing subagents](/skills/distributing-subagents/) guide covers a full subagent publish. -### Production +To connect a netclaw instance, add the server as a feed in `netclaw config` under **Skill Sources**, or in `netclaw.json`: -- Put a reverse proxy (Caddy, nginx, Traefik) in front for TLS -- Set `SKILLSERVER__BASEURL` to your public URL (e.g., `https://skills.internal.example.com`) so discovery responses have correct absolute URLs -- Mount `/data` to persistent storage. If the volume is lost, you'll re-publish everything +```json +{ "SkillFeeds": { "Feeds": [ { "Name": "my-server", "Url": "https://skills.internal.example.com", "Enabled": true } ] } } +``` + +The daemon syncs at startup and on a timer. To pull something you just published, run `netclaw skill sync`. Skills land in `~/.netclaw/skills/.server-feeds//` and subagents in `~/.netclaw/agents/.server-feeds//`. netclaw restores a changed skill file at the next sync, but it leaves an edited subagent file alone. [Skill feeds](/skills/skill-feeds/) covers intervals, authentication and the sync output. -## API key management +## Manage API keys -Publishing, deletion, and all key-management commands, including `api-key list`, require an existing valid key once authentication is enabled. Reads and discovery remain open. Every valid key has the same permissions; there is no publisher-only scope. +Publishing, deleting and every key command, including `api-key list`, need a valid key once authentication is on. Reads and discovery stay open. Every valid key has the same permissions, because there's no publish-only scope. ### Create a publishing key -Install the [`skillserver` CLI](/skills/skillserver-cli/#install) on your workstation or CI runner. It is a separate tool from the server container. In a private Bash terminal, authenticate with the bootstrap key or another existing valid key: +Install the [`skillserver` CLI](/skills/skillserver-cli/#install) on your workstation or CI runner. It's a separate tool from the server container. In a private Bash terminal, authenticate with the bootstrap key (or any valid key) and create one for CI: ```bash export SKILLSERVER_URL=https://skills.example.com @@ -138,13 +144,13 @@ skillserver api-key create --label ci-publish unset SKILLSERVER_API_KEY ``` -Use your server's HTTPS URL, or `http://localhost:8080` for local testing. `list` confirms authentication. Creation prints the new `sk-...` key once, along with its ID and label. Save the actual key immediately in your secret manager; it cannot be recovered from the server. +Use your server's HTTPS URL, or `http://localhost:8080` for local testing. `list` proves your key works. `create` prints the new `sk-...` key once, with its ID and label. Copy it into your secret manager right away. The server keeps only a hash, so it can't show the key again. :::caution -The label is a name you choose. The credential is the key the server generates and registers. Saving an arbitrary string as a CI secret will not authenticate. Run key creation in a private terminal, never in shared CI logs. +The label is a name you pick, and the key is what authenticates. A made-up string saved as a CI secret won't work. Create keys in a private terminal and never in shared CI logs. ::: -For GitHub Actions, save the returned value as a **`SKILLSERVER_API_KEY` secret**, then pass it to the CLI: +In GitHub Actions, save the key as a `SKILLSERVER_API_KEY` secret and hand it to the CLI: ```yaml - name: Publish skills @@ -154,142 +160,122 @@ For GitHub Actions, save the returned value as a **`SKILLSERVER_API_KEY` secret* run: skillserver publish-all ./skills ``` -Create the secret under repository **Settings → Secrets and variables → Actions**, or under the deployment environment if the job uses one. See [GitHub secret setup](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions). - -Install the CLI and configure network access before this step. Use a separate CI key so you can revoke or rotate it independently. - -### List, revoke, and rotate +Create the secret under repository **Settings → Secrets and variables → Actions**, or under the environment if the job uses one. GitHub's [secrets guide](https://docs.github.com/en/actions/security-for-github-actions/security-guides/using-secrets-in-github-actions) has the details. Give CI its own key so you can revoke or rotate it without touching anyone else's. -With the CLI authenticated using an existing valid key: +### List, revoke and rotate keys ```bash skillserver api-key list skillserver api-key delete 2 ``` -Listing shows IDs, labels, and dates, never raw keys or hashes. Replace `2` with the ID to revoke. The server refuses to delete the last remaining key. Creation also accepts `--expires-at ` for an expiring key. +`list` shows IDs, labels and dates, never raw keys or hashes. Replace `2` with the ID to revoke. The server won't delete the last remaining key. `create` also takes `--expires-at ` for a key that expires. -To rotate, create a replacement, update the client or CI secret, and run `skillserver api-key list` authenticated with the replacement. Then revoke the old ID. No server restart or database reset is needed. If a newly created key is lost, create another using an existing valid key and revoke the lost key. +To rotate, create a replacement and update the client or CI secret. Then run `skillserver api-key list` with the new key to confirm it works, and delete the old ID. No restart or database reset needed. If you lose a freshly created key, create another with any valid key and revoke the lost one. -### Server bootstrap vs. client authentication +## Reference -| Variable | Consumer | Purpose | -|----------|----------|---------| -| `SKILLSERVER__APIKEY` | Server | Seed the initial bootstrap key when the database has no keys | -| `SKILLSERVER_APIKEY` | This Compose example | Supply the value mapped to `SKILLSERVER__APIKEY` | -| `SKILLSERVER_API_KEY` | CLI and CI | Authenticate using a key already registered with the server | +### Configuration -Once any key exists in the database, `SKILLSERVER__APIKEY` is ignored on later startups. Changing it does not rotate existing credentials. +All configuration is through environment variables: -Generated keys use `sk-` plus 256 bits of randomness encoded as base64url. The server persists only SHA-256 hashes; creation is the only response containing the raw key. - -## Publishing skills and sub-agents - -The [`skillserver` CLI](/skills/skillserver-cli/) handles publishing and management from your terminal or CI, and that page has the full command and flag reference. The essentials: - -```bash -# Publish one skill, or every skill directory under a parent -skillserver publish ./my-skill -skillserver publish-all ./skills - -# Publish sub-agents (version is required - it's not read from frontmatter) -skillserver publish-subagent ./release-notes-writer.md --version 1.0.0 -skillserver publish-subagents ./subagents -``` +| Variable | Default | Description | +|----------|---------|-------------| +| `SKILLSERVER__DATAPATH` | `./data` | SQLite database and blob storage directory | +| `SKILLSERVER__BASEURL` | `http://localhost:8080` | Base URL for the absolute URLs in discovery responses | +| `SKILLSERVER__APIKEY` | none | Bootstrap API key, seeded on first run if the database has no keys | +| `ASPNETCORE_URLS` | `http://+:8080` | Listen address and port | -Publishing a version that already exists is skipped, so `publish-all` and `publish-subagents` are idempotent and safe to run on every CI push. A skill with a `references/`, `scripts/`, or `assets/` folder is packaged as an [archive](/skills/bundled-resources/) that preserves relative paths and executable bits. A bare `SKILL.md` publishes as a lightweight skill-md artifact. +### Which key variable is which - +| Variable | Used by | Purpose | +|----------|---------|---------| +| `SKILLSERVER__APIKEY` | The server | Seeds the initial key when the database has none | +| `SKILLSERVER_APIKEY` | The Compose example above | Carries the value into `SKILLSERVER__APIKEY` | +| `SKILLSERVER_API_KEY` | The CLI and CI | Authenticates with a key the server already knows | -## Feeds and the native manifest +Once any key exists, the server ignores `SKILLSERVER__APIKEY` on later starts. Changing it doesn't rotate anything. Generated keys are `sk-` plus 256 bits of randomness encoded as base64url, and the server stores only SHA-256 hashes. -The server exposes two discovery feeds: +### Feeds | Feed | Path | Contents | |------|------|----------| -| RFC index | `/.well-known/agent-skills/index.json` | Skills only, per the Cloudflare RFC | -| Native manifest | `/manifest.json` | Skills, sub-agents, and archives, with API version negotiation | - -The netclaw daemon reads the native manifest, so it picks up sub-agents and archive resources the RFC feed can't express. See [Native Manifest](/skills/native-manifest/) for the manifest tree and version negotiation, and [Skill Feeds](/skills/skill-feeds/) for how the daemon syncs from it. - -## Connecting netclaw instances +| RFC index | `/.well-known/agent-skills/index.json` | Skills with type, URL and digest. Archives are described here too. | +| Native manifest | `/manifest.json` | Skills and subagents, with version history | -Add a skill server as a feed source through `netclaw config` → Skill Sources (select **+ Add skill server** and enter the base URL), or by adding it to the `SkillFeeds.Feeds` array in `~/.netclaw/config/netclaw.json`: +See [Native manifest](/skills/native-manifest/) for the tree. -```json -{ "SkillFeeds": { "Feeds": [ { "Name": "my-server", "Url": "http://skills.internal.example.com", "Enabled": true } ] } } -``` - -The daemon syncs on a periodic interval. Skills land in `~/.netclaw/skills/.server-feeds/` and sub-agents in `~/.netclaw/agents/.server-feeds/`, both read-only. See [Skill Feeds](/skills/skill-feeds/) for sync intervals, authentication, and selective sync options. +### API endpoints -## API reference +Reads are open. Rows marked `auth` need an `Authorization: Bearer ` header. -Reads are open. Endpoints marked `auth` require an `Authorization: Bearer ` header. - -### Discovery and feeds +#### Discovery and feeds | Method | Path | Description | |--------|------|-------------| | GET | `/health` | Health check | -| GET | `/api/v1/info` | Server info and capabilities | -| GET | `/.well-known/agent-skills/index.json` | RFC-compliant skill discovery index | -| GET | `/manifest.json` | Native manifest root (version negotiation) | -| GET | `/skills/v1/index.json` | Native skill collection (paginated) | -| GET | `/skills/v1/{name}/versions/{version}.json` | Native skill version entry | -| GET | `/subagents/v1/index.json` | Native sub-agent collection (paginated) | -| GET | `/subagents/v1/{name}/versions/{version}.json` | Native sub-agent version entry | - -### Skills +| GET | `/api/v1/info` | Server version | +| GET | `/.well-known/agent-skills/index.json` | RFC discovery index | +| GET | `/manifest.json` | Native manifest root | +| GET | `/skills/v1/index.json` | Native skill collection index | +| GET | `/skills/v1/pages/{page}.json` | One page of skills | +| GET | `/skills/v1/{name}/index.json` | One skill and its versions | +| GET | `/skills/v1/{name}/versions/{version}.json` | One skill version | +| GET | `/subagents/v1/index.json` | Native subagent collection index | +| GET | `/subagents/v1/pages/{page}.json` | One page of subagents | +| GET | `/subagents/v1/{name}/index.json` | One subagent and its versions | +| GET | `/subagents/v1/{name}/versions/{version}.json` | One subagent version | + +#### Skills | Method | Path | Description | |--------|------|-------------| -| GET | `/api/v1/skills` | List all (`?q=`, `?skip=`, `?take=` supported) | +| GET | `/api/v1/skills` | List all (`?q=`, `?skip=`, `?take=`) | | GET | `/api/v1/skills/{name}` | All versions of a skill | | GET | `/api/v1/skills/{name}/latest` | Latest version metadata | -| GET | `/api/v1/skills/{name}/{version}` | Specific version metadata | -| GET | `/api/v1/skills/{name}/{version}/SKILL.md` | Download SKILL.md | -| GET | `/api/v1/skills/{name}/{version}/archive.zip` | Download the packaged archive | +| GET | `/api/v1/skills/{name}/{version}` | One version's metadata | +| GET | `/api/v1/skills/{name}/{version}/SKILL.md` | Download `SKILL.md` | +| GET | `/api/v1/skills/{name}/{version}/archive.zip` | Download the archive | | GET | `/api/v1/skills/{name}/{version}/resources` | List bundled resources (path, digest, mode) | -| GET | `/api/v1/skills/{name}/{version}/{*path}` | Download a single resource file | +| GET | `/api/v1/skills/{name}/{version}/{*path}` | Download one resource file | | POST | `/api/v1/skills/check-updates` | Batch update check | -| POST | `/api/v1/skills` | `auth` Upload a new version (multipart/form-data) | +| POST | `/api/v1/skills` | `auth` Upload a version (multipart/form-data) | | DELETE | `/api/v1/skills/{name}/{version}` | `auth` Delete a version | -### Sub-agents +#### Subagents | Method | Path | Description | |--------|------|-------------| -| GET | `/api/v1/subagents` | List all sub-agents | -| GET | `/api/v1/subagents/{name}` | All versions of a sub-agent | -| GET | `/api/v1/subagents/{name}/{version}` | Specific version metadata | -| GET | `/api/v1/subagents/{name}/{version}/agent.md` | Download the sub-agent definition | -| POST | `/api/v1/subagents` | `auth` Upload a new version | +| GET | `/api/v1/subagents` | List all | +| GET | `/api/v1/subagents/{name}` | All versions of a subagent | +| GET | `/api/v1/subagents/{name}/{version}` | One version's metadata | +| GET | `/api/v1/subagents/{name}/{version}/agent.md` | Download the definition | +| POST | `/api/v1/subagents` | `auth` Upload a version | | DELETE | `/api/v1/subagents/{name}/{version}` | `auth` Delete a version | -### Blobs and API keys +#### Blobs and API keys | Method | Path | Description | |--------|------|-------------| | GET | `/api/v1/blobs/sha256/{digest}` | Download a blob by digest | -| POST | `/api/v1/api-keys` | `auth` Create a key (returns raw key once) | -| GET | `/api/v1/api-keys` | `auth` List key metadata (no raw keys or hashes) | +| POST | `/api/v1/api-keys` | `auth` Create a key (returns the raw key once) | +| GET | `/api/v1/api-keys` | `auth` List key metadata | | DELETE | `/api/v1/api-keys/{id}` | `auth` Revoke a key | ### Web gallery -The server also serves a browser UI: `/`, `/skills`, and `/subagents` return the gallery SPA rather than JSON. +The server also serves a browser UI. `/`, `/skills` and `/subagents` return the gallery SPA instead of JSON. ## Related pages -- [skillserver CLI](/skills/skillserver-cli/) - full publishing and management command reference -- [Native Manifest](/skills/native-manifest/) - the sync feed netclaw reads for skills and sub-agents -- [Bundling Resources with Skills](/skills/bundled-resources/) - archive artifacts and packaged resources -- [Skill Feeds](/skills/skill-feeds/) - configuring netclaw to consume from skill servers -- [Skills Overview](/skills/overview/) - format, lifecycle, source types -- [Security Model](/security/security-model/) - content scanning and trust model +- [`skillserver` CLI](/skills/skillserver-cli/): the publishing and management commands +- [Skill feeds](/skills/skill-feeds/): point netclaw at a server +- [Native manifest](/skills/native-manifest/): how subagents reach netclaw +- [Distributing subagents](/skills/distributing-subagents/): publish a subagent end to end +- [Skills overview](/skills/overview/): tiers, precedence and trust -## External resources +## Resources -- [AgentSkills.io](https://agentskills.io) - the SKILL.md format spec -- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc) - the discovery protocol -- [netclaw-dev/skill-server on GitHub](https://github.com/netclaw-dev/skill-server) - source, issues, releases +- [AgentSkills.io](https://agentskills.io): the `SKILL.md` format spec +- [Cloudflare Agent Skills Discovery RFC](https://github.com/cloudflare/agent-skills-discovery-rfc): the discovery protocol +- [netclaw-dev/skill-server](https://github.com/netclaw-dev/skill-server): source, issues and releases diff --git a/src/content/docs/skills/skillserver-cli.md b/src/content/docs/skills/skillserver-cli.md index 536eb62..196b15e 100644 --- a/src/content/docs/skills/skillserver-cli.md +++ b/src/content/docs/skills/skillserver-cli.md @@ -1,9 +1,9 @@ --- title: "skillserver CLI" -description: "Publish, validate, and manage skills and sub-agents on a skill server from the command line." +description: "Publish, validate, and manage skills and subagents on a skill server from the command line." --- -`skillserver` is the command-line tool for publishing and managing skills and sub-agents on a [skill server](/skills/skill-server/). It's what you run by hand during authoring and what your [CI pipeline](/skills/skill-server/) runs on merge. It speaks to the server's REST API, so anything the [gallery](/skills/skill-server/) shows, `skillserver` can do from a terminal. +`skillserver` is the command-line tool for publishing and managing skills and subagents on a [skill server](/skills/skill-server/). It's what you run by hand during authoring and what your [CI pipeline](/skills/skill-server/#create-a-publishing-key) runs on merge. It speaks to the server's REST API, so anything the [gallery](/skills/skill-server/) shows, `skillserver` can do from a terminal. ## Install @@ -11,11 +11,13 @@ description: "Publish, validate, and manage skills and sub-agents on a skill ser # .NET global tool dotnet tool install --global Netclaw.SkillServer.Cli -# Or a self-contained binary (Linux/macOS), no .NET runtime needed +# Or a self-contained binary (Linux x64 and arm64, macOS arm64), no .NET runtime needed curl -fsSL https://raw.githubusercontent.com/netclaw-dev/skill-server/dev/scripts/install-skillserver.sh | bash ``` -For CI, pin the version. The install script takes one as an argument (`bash -s -- 0.4.0`), or commit a `dotnet-tools.json` manifest and run `dotnet tool restore`. +On Windows, run [`install-skillserver.ps1`](https://github.com/netclaw-dev/skill-server/blob/dev/scripts/install-skillserver.ps1), which installs the `win-x64` build. Releases don't include an Intel macOS binary, so use the .NET tool there. + +For CI, pin the version. The install script takes one as an argument (`bash -s -- 0.4.1`), or commit a `dotnet-tools.json` manifest and run `dotnet tool restore`. ## Configure @@ -39,7 +41,7 @@ skillserver config show # print current config (no secrets) Read-only commands (`list`, `list-subagents`, `versions`, `verify`, `download-subagent`) need only the URL. Everything that writes needs a key. All `api-key` commands also require authentication, including `api-key list`. -Follow [API key management](/skills/skill-server/#api-key-management) to bootstrap the server, create a CI key, or rotate a key. `SKILLSERVER_API_KEY` must hold an actual key registered with that server. It is separate from the server bootstrap variable `SKILLSERVER__APIKEY`. +Follow [API key management](/skills/skill-server/#manage-api-keys) to bootstrap the server, create a CI key, or rotate a key. `SKILLSERVER_API_KEY` must hold an actual key registered with that server. It is separate from the server bootstrap variable `SKILLSERVER__APIKEY`. ## Commands @@ -50,10 +52,10 @@ skillserver publish ./my-skill # one skill; version from SKILL.m skillserver publish ./my-skill --version 2.0.0 # override the version skillserver publish-all ./skills # every skill dir under ./skills skillserver publish-subagent ./agent.md --version 1.0.0 -skillserver publish-subagents ./subagents # every .md under ./subagents +skillserver publish-subagents ./subagents --version 1.0.0 # every .md under ./subagents ``` -`publish` points at a single skill directory (the one holding `SKILL.md`). `publish-all` points at the *parent* directory and publishes each subdirectory. Both skip versions already on the server unless you pass `--force`. Sub-agents take their version from `--version`, which is required. The markdown frontmatter's `metadata.version` is ignored. +`publish` points at a single skill directory (the one holding `SKILL.md`). `publish-all` points at the *parent* directory and publishes each subdirectory. Both skip versions already on the server unless you pass `--force`. Subagents take their version from `--version`, which is required. The frontmatter's `metadata.version` is ignored, and a batch publish gives every file the same version. ```text $ skillserver publish-all ./skills @@ -68,7 +70,7 @@ Results: 2 published, 0 skipped, 0 failed | Flag | Meaning | |------|---------| -| `--version ` | Override the version (required for sub-agents) | +| `--version ` | Override the version (required for subagents) | | `--force`, `-f` | Delete the existing version, then re-publish | | `--dry-run` | Show what would publish without uploading | @@ -101,14 +103,14 @@ All files verified ```bash skillserver lint ./skills # every skill dir under ./skills -skillserver lint subagent ./agent.md # one sub-agent file -skillserver lint subagents ./subagents # every sub-agent .md in a dir +skillserver lint subagent ./agent.md # one subagent file +skillserver lint subagents ./subagents # every subagent .md in a dir ``` -It checks that frontmatter parses, required fields are present, names are lowercase-kebab, and versions are semver. +It checks that frontmatter parses, required fields are present, names are lowercase kebab-case, and versions are semver. The CLI reads each key by line, so a `version:` nested under `metadata:` satisfies the version check too. :::caution -`lint` only exits non-zero on hard errors: a missing `SKILL.md`, a missing required field, or unparseable frontmatter. Softer problems are warnings that still exit 0, including a `name` that doesn't match its directory and a version that isn't valid semver. A name/directory mismatch is the number one publishing mistake, and `lint` won't fail CI over it, so read the warning output instead of trusting the exit code alone. +`lint` exits non-zero on hard errors: a missing `SKILL.md`, a missing `name` or `version`, an invalid name format, or unparseable frontmatter. Softer problems are warnings that still exit 0, including a missing `description`, a `name` that doesn't match its directory, and a version that isn't valid semver. A name and directory mismatch is the number one publishing mistake, and `lint` won't fail CI over it, so read the warning output instead of trusting the exit code alone. ::: ### Manage @@ -121,7 +123,7 @@ skillserver api-key list # ids + labels, never secrets skillserver api-key delete 2 ``` -`--yes`/`-y` skips confirmation for skill and sub-agent deletion. API key deletion has no confirmation prompt. `api-key create` shows the `sk-…` secret a single time, so store it immediately. Every valid key can publish, delete, and manage other keys; choosing a CI label does not restrict its permissions. +`--yes`/`-y` skips confirmation for skill and subagent deletion. API key deletion has no confirmation prompt. `api-key create` shows the `sk-…` secret a single time, so store it immediately. Every valid key can publish, delete and manage other keys. Choosing a CI label does not restrict its permissions. ## Global flags @@ -151,13 +153,14 @@ There's no separate code for usage versus runtime errors. Both are `1`, so scrip ## Related pages -- [Skill Server](/skills/skill-server/) - run the server this CLI talks to. -- [Bundling Resources with Skills](/skills/bundled-resources/) - what `publish` packages when a skill has `references/`, `scripts/`, or `assets/`. -- [Native Manifest](/skills/native-manifest/) - the sync feed a netclaw daemon reads from the server. -- [Skill Feeds](/skills/skill-feeds/) - point a netclaw daemon at the published feed. +- [Skill server](/skills/skill-server/): run the server this CLI talks to +- [Bundling resources with skills](/skills/bundled-resources/): what `publish` packages when a skill has `references/`, `scripts/` or `assets/` +- [Native manifest](/skills/native-manifest/): the feed that carries subagents to netclaw +- [Distribute subagents](/skills/distributing-subagents/): publish a subagent end to end +- [Skill feeds](/skills/skill-feeds/): point a netclaw daemon at the published feed ## Resources -- [skillserver CLI README](https://github.com/netclaw-dev/skill-server/blob/dev/src/Netclaw.SkillServer.Cli/README.md) - upstream reference. -- [AgentSkills.io](https://agentskills.io) - the `SKILL.md` format `lint` validates against. -- [.NET local tools](https://learn.microsoft.com/en-us/dotnet/core/tools/local-tools-how-to-use) - pinning the CLI with `dotnet-tools.json`. +- [skillserver CLI README](https://github.com/netclaw-dev/skill-server/blob/dev/src/Netclaw.SkillServer.Cli/README.md): upstream reference +- [AgentSkills.io](https://agentskills.io): the `SKILL.md` format `lint` validates against +- [.NET local tools](https://learn.microsoft.com/en-us/dotnet/core/tools/local-tools-how-to-use): pinning the CLI with `dotnet-tools.json`