Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 6 additions & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,7 @@ pixi run agent # Robot -> fleet bridge (mote-agent.service)
pixi run fleet-server # Off-board: fleet API + operator dashboard
pixi run fleetctl # Operator CLI: token / operator / robots / dispatch / audit / watch
pixi run fleet-broker-ws # Off-board: MQTT broker with WebSockets (needs docker)
# fleetctl broker sync # regenerate the broker's credentials from the registry

# Dev environment only (installs ros-jazzy-desktop)
pixi run rviz # RViz2 with mote config
Expand Down Expand Up @@ -97,7 +98,11 @@ Milestone M1 of `docs/design/fleet.md`, built in the **`mote_fleet`** package (b

## Fleet: the operator view + dispatch API (M3)

Milestone M3 of `docs/design/fleet.md`, and the end of v0. The **HTTP** wire is specified as its own versioned contract in **`docs/fleet/fleet-api.md`** (M1's MQTT one is `control-plane.md`); the operator flow is `docs/fleet/README.md` §6–9 and the measurements are `m3-verification.md`. **The two directions of the loop take different paths on purpose.** *Reads* ride MQTT: the browser subscribes to `mote/v1/+/{presence,health,pose,task/status}` over WebSockets, and because all of those are retained it has the whole fleet's state within a second of loading — no polling, no service in the middle. *Writes* ride HTTP: `POST /v1/robots/<id>/dispatch` authorizes an operator token (`fleetctl operator new --name <you>`; the name is what the audit row records), writes the audit row, then publishes to the same `task/command` topic. **The topic tree did not change — only who publishes to it**, and `fleetctl dispatch` moved to the API too, so there is one write path rather than one per client. The command grammar is still parsed only by the robot's task layer: a parser in the server would be a second grammar to keep in step. **The browser cannot publish**: `server/ui/mqtt.mjs` is a hand-rolled subscribe-only MQTT 3.1.1 client that implements no PUBLISH packet, so the split is enforced by omission (M7 makes it structural with a subscribe-only broker credential). The UI is static ES modules — no bundler, no npm, no vendored library — served by the same stdlib `http.server`; `map.mjs` holds the Q5 world→pixel transform (`px = (wx-origin_x)/res`, `py = height - (wy-origin_y)/res`) and a pan/zoom/follow canvas, and only draws robots on the *same* site+floor as the selected one because a pose from another floor is a different map frame. **Basemaps come from site bundles on the fleet box** (`--maps-dir`, default `$MOTE_FLEET_HOME/sites`, the layout `sites.py` writes, seeded by rsync until **M4** makes the registry canonical behind the same two routes). **M1's websockets blocker is settled**: `pixi run fleet-broker-ws` runs `eclipse-mosquitto` under docker with the repo's own `mosquitto.conf`, because conda-forge's build has none; `broker.sh` strips the WS stanza when the local binary cannot honour it and says so. Two things that run in the same file (`test_ui.py` → `ui_test.mjs`) are the MQTT codec and the transform, tested under node against the very files the browser loads; `browser_check.mjs` drives a real headless Chrome over CDP against a running stack and is an operator's tool, not a CI test.
Milestone M3 of `docs/design/fleet.md`, and the end of v0. The **HTTP** wire is specified as its own versioned contract in **`docs/fleet/fleet-api.md`** (M1's MQTT one is `control-plane.md`); the operator flow is `docs/fleet/README.md` §6–9 and the measurements are `m3-verification.md`. **The two directions of the loop take different paths on purpose.** *Reads* ride MQTT: the browser subscribes to `mote/v1/+/{presence,health,pose,task/status}` over WebSockets, and because all of those are retained it has the whole fleet's state within a second of loading — no polling, no service in the middle. *Writes* ride HTTP: `POST /v1/robots/<id>/dispatch` authorizes an operator token (`fleetctl operator new --name <you>`; the name is what the audit row records), writes the audit row, then publishes to the same `task/command` topic. **The topic tree did not change — only who publishes to it**, and `fleetctl dispatch` moved to the API too, so there is one write path rather than one per client. The command grammar is still parsed only by the robot's task layer: a parser in the server would be a second grammar to keep in step. **The browser cannot publish**: `server/ui/mqtt.mjs` is a hand-rolled subscribe-only MQTT 3.1.1 client that implements no PUBLISH packet, so the split is enforced by omission — and since **M7** also by the broker's ACL, which gives that credential no write rule at all. The UI is static ES modules — no bundler, no npm, no vendored library — served by the same stdlib `http.server`; `map.mjs` holds the Q5 world→pixel transform (`px = (wx-origin_x)/res`, `py = height - (wy-origin_y)/res`) and a pan/zoom/follow canvas, and only draws robots on the *same* site+floor as the selected one because a pose from another floor is a different map frame. **Basemaps come from site bundles on the fleet box** (`--maps-dir`, default `$MOTE_FLEET_HOME/sites`, the layout `sites.py` writes, seeded by rsync until **M4** makes the registry canonical behind the same two routes). **M1's websockets blocker is settled**: `pixi run fleet-broker-ws` runs `eclipse-mosquitto` under docker with the repo's own `mosquitto.conf`, because conda-forge's build has none; `broker.sh` strips the WS stanza when the local binary cannot honour it and says so. Two things that run in the same file (`test_ui.py` → `ui_test.mjs`) are the MQTT codec and the transform, tested under node against the very files the browser loads; `browser_check.mjs` drives a real headless Chrome over CDP against a running stack and is an operator's tool, not a CI test.

## Fleet: security hardening (M7)

Milestone M7 of `docs/design/fleet.md`, cross-cutting. The authorization rules are specified as the fleet's **third** versioned contract in **`docs/fleet/security.md`** (M1's MQTT wire is `control-plane.md`, M3's HTTP wire is `fleet-api.md`); the measurements are `m7-verification.md`. **Through M3 the fleet had one credential on one path** — an operator token on dispatch — and everything else leaned on the tailnet. The argument for changing that is not that WireGuard is weak but that **a single boundary makes every robot as trusted as the fleet server**: a robot is physically reachable and carries the biggest dependency tree in the system, so one compromised robot would own the fleet. **The broker is no longer anonymous.** `mote_fleet/server/credentials.py` generates mosquitto's `password_file` and `acl_file` *from the registry* into `$MOTE_FLEET_HOME/broker/`, and the broker re-reads them on SIGHUP (`broker.sh reload`, which knows whether it started a local process or a container) — so who may connect is derived, never hand-maintained, and a revoked operator disappears because the query no longer returns them. **Three principals, disjoint by construction**: a robot's username is its `robot_id` and it may publish only under its own prefix and read only its own `task/command`; an operator's is `op_<slug>_<hex>` and may subscribe fleet-wide and **publish nothing**; the server's is `fleet_server` and is the only principal that may write `task/command`. A `robot_id` is a lowercase DNS label so it can never contain the underscore the other two carry — no runtime collision check, and `test_credentials.py` asserts the property rather than the paragraph. Robot credentials are issued **in the enrollment transaction** and returned once (only the hash is stored), which makes **rotation just `pixi run enroll` again**; `$MOTE_HOME/fleet.yaml` and the registry are both `0600`. **Every `/v1` route now needs an operator token**, checked by one gate in front of routing (so a new route is authenticated by default, and an unauthenticated caller gets 401 rather than a 404 that would disclose which routes exist); `/healthz` and the static UI stay open, the latter because the page must load before it can ask for a token. `/v1/config` hands the browser *that operator's* subscribe-only broker credential, which is why it needs a token. The mosquitto `$7$` hash (PBKDF2-HMAC-SHA512, 101 iterations) is generated in stdlib rather than by shelling out to `mosquitto_passwd`, because the fleet server's whole dependency list is "python" — and a test verifies a real `mosquitto_passwd` hash so the two cannot drift. **Gotcha, and every test in `test_broker_acl.py` depends on it: mosquitto denies silently.** A forbidden publish is accepted and dropped; a forbidden subscribe is *granted* at SUBACK and never delivers — so denial must be asserted on delivery, and a suite checking return codes would pass with no ACL loaded. The network half is `mote_bringup/tailscale/policy.hujson` (committed source of truth, pasted into the admin console), whose own `tests` block asserts robots cannot reach each other; Tailscale refuses to save a policy that fails it. **Breaking change for a deployed fleet**: a robot enrolled before M7 has no credential and will be refused — re-run `enroll`. Deliberately *not* done, each for a reason in the doc: mTLS, expiring operator sessions, package signing (nothing to sign until M5 ships a package; pinning is what M7 could deliver — `pixi install --locked` at provisioning), and Foxglove tokens (M2 does not exist yet).

## Sites (maps & zones)

Expand Down
24 changes: 23 additions & 1 deletion docs/design/fleet.md
Original file line number Diff line number Diff line change
Expand Up @@ -732,7 +732,11 @@ per-channel auth on top; do **not** build a PKI or a custom auth server for v1.
tailnets or broker vhosts + topic-prefix ACLs per customer — so one customer's
operator can never see another's robots. Designed-for, not built in v1.
- **Proportionality.** Small trusted fleet, not a public product: mTLS-everywhere
and a custom PKI wait until the fleet runs *outside* the trusted overlay. Note
and a custom PKI wait until the fleet runs *outside* the trusted overlay. The
argument that made M7 worth building anyway is that a single boundary makes
**every robot as trusted as the fleet server** — a robot is physically
reachable and runs the largest dependency tree in the system, so one boundary
means one compromised robot owns the fleet. Note
Zenoh's TLS is *hop-by-hop, not end-to-end*
([Zenoh security analysis](https://census-labs.com/news/2025/03/17/zenoh-protocol-security-analysis/))
— a reason the encryption baseline lives at the WireGuard layer (end-to-end
Expand Down Expand Up @@ -973,6 +977,24 @@ M7 (security hardening) : cross-cutting, folds into each; can start after M0
signed if available) packages. *Accept:* a device off the tailnet reaches nothing;
a robot can't read another robot's command topic. *Depends on:* M0; folds into
each milestone as it lands.
**Built** as [`docs/fleet/security.md`](../fleet/security.md) — the fleet's
third contract, alongside M1's MQTT one and M3's HTTP one — with the policy in
`mote_fleet/server/credentials.py` and
[`mote_bringup/tailscale/policy.hujson`](../../mote_bringup/tailscale/policy.hujson),
and the measurements in [`m7-verification.md`](../fleet/m7-verification.md).
The broker is no longer anonymous: credentials are **generated from the
registry** and the broker reloads them on SIGHUP, so a robot may publish only
under its own `robot_id`, an operator may only subscribe, and only the fleet
API may write `task/command`. Every `/v1` route now needs an operator token,
behind one gate rather than per-handler. Three things the milestone did *not*
do, each for a stated reason: **mTLS** (username/password inside WireGuard is
proportionate, and the seam is unchanged), **expiring operator sessions**
(Q7's OIDC/GitHub answer replaces the minting, not the checking), and
**package signing** — there is nothing to sign until the channel ships a robot
package, so that stays M5's, and pinning is what M7 could actually deliver
(`pixi install --locked` at provisioning, every dependency sha256-pinned).
**Foxglove tokens wait for M2**, which does not exist yet; what M7 contributes
is the tailnet rule already scoping port 8765 to operators.

---

Expand Down
Loading