Versioned releases: publish to the prefix.dev mote channel, replace rsync deploys

.github/workflows/release.yml

@@ -0,0 +1,65 @@
+name: release
+
+# Builds the release for both architectures and proves a robot environment
+# resolves against it. Deliberately does NOT publish: the upload to
+# prefix.dev/mote is a human action (`pixi run release-publish`), because a
+# published version cannot be taken back out of someone's lockfile.
+#
+# See docs/releasing.md.
+
+on:
+  push:
+    tags: ["v*"]
+  # Cutting a release should never be the first time the release builds, so it
+  # can also be run by hand against a branch.
+  workflow_dispatch:
+
+jobs:
+  release:
+    strategy:
+      fail-fast: false
+      matrix:
+        include:
+          - runner: ubuntu-latest # linux-64
+          - runner: ubuntu-24.04-arm # linux-aarch64, same arch as the Pi
+    runs-on: ${{ matrix.runner }}
+    steps:
+      - uses: actions/checkout@v4
+        with:
+          # kinematic_icp and sllidar_ros2 are released too -- a robot
+          # installing from the channel has no source tree to build them from.
+          submodules: recursive
+
+      - uses: prefix-dev/setup-pixi@v0.9.0
+        with:
+          cache: true
+
+      # Cheap and ROS-free, so it fails in seconds rather than after a build:
+      # catches a package missing from the release set, an unpinned backend, or
+      # a stale committed deploy manifest.
+      - name: Test the release tooling
+        run: pixi run test-release
+
+      - name: Check version consistency
+        run: pixi run release-version check
+
+      # The package build does not run tests (BUILD_TESTING=OFF), so they run
+      # here, before anything is built.
+      - name: Test
+        run: pixi run test
+
+      - name: Build the release
+        run: pixi run release-build
+
+      # The step with teeth: resolves the robot deploy manifest against the
+      # freshly built channel, proving the release installs as a set.
+      - name: Verify a robot environment resolves
+        run: pixi run release-verify
+
+      - uses: actions/upload-artifact@v4
+        with:
+          name: mote-packages-${{ matrix.runner }}
+          path: |
+            dist/channel/**/*.conda
+            release/evidence/
+          if-no-files-found: error

.gitignore

@@ -13,6 +13,15 @@ __pycache__/
 
 scratchpad*
 
+# setuptools install-record dropped in each ament_python package by a package
+# build (`pixi run release-build`). Full of absolute build-prefix paths.
+files.txt
+
+# Built release artifacts and the local dry-run channel (`pixi run release`).
+# The packages are never committed; what a release *does* commit is
+# release/evidence/ and the regenerated release/deploy/pixi.toml.
+dist/
+
 # Sim benchmark outputs (timestamped metrics/reports)
 benchmark_results/
 # Bag-replay scoring outputs (timestamped metrics/reports/maps)

CLAUDE.md

@@ -23,7 +23,15 @@ pixi run arm            # SO-101 arm driver: joint states + safe jog control
 pixi run arm-jog        # Interactive per-joint jog CLI (needs `pixi run arm`)
 pixi run arm-check      # Standalone arm bus enumeration + health (read-only)
 pixi run arm-pose       # Teach/replay named arm poses; derive soft limits
-pixi run sync           # rsync project to Pi at SSH host 'mote'
+pixi run sync           # rsync project to Pi at SSH host 'mote' (dev loop only)
+# Releases: versioned packages on the prefix.dev `mote` channel (docs/releasing.md)
+pixi run release-version  # show / check / set X.Y.Z (one version, all packages)
+pixi run release        # full dry run: test + build all + verify. Uploads nothing
+pixi run release-build  # build the release set -> dist/channel (local channel)
+pixi run release-verify # resolve a robot env against dist/channel
+pixi run release-manifest # regenerate the committed release/deploy/pixi.toml
+pixi run release-publish  # THE outward-facing step: upload to prefix.dev/mote
+pixi run test-release   # tests for the release tooling (own minimal env, no ROS)
 pixi run setup          # One-time Pi setup: udev + wifi-powersave + systemd (needs sudo)
 pixi run udev           # Install udev rules + dialout group (needs sudo)
 pixi run wifi-powersave # Disable WiFi power save via NetworkManager (needs sudo)
@@ -99,6 +107,10 @@ Milestone M1 of `docs/design/fleet.md`, built in the **`mote_fleet`** package (b
 
 Milestone M3 of `docs/design/fleet.md`, and the end of v0. The **HTTP** wire is specified as its own versioned contract in **`docs/fleet/fleet-api.md`** (M1's MQTT one is `control-plane.md`); the operator flow is `docs/fleet/README.md` §6–9 and the measurements are `m3-verification.md`. **The two directions of the loop take different paths on purpose.** *Reads* ride MQTT: the browser subscribes to `mote/v1/+/{presence,health,pose,task/status}` over WebSockets, and because all of those are retained it has the whole fleet's state within a second of loading — no polling, no service in the middle. *Writes* ride HTTP: `POST /v1/robots/<id>/dispatch` authorizes an operator token (`fleetctl operator new --name <you>`; the name is what the audit row records), writes the audit row, then publishes to the same `task/command` topic. **The topic tree did not change — only who publishes to it**, and `fleetctl dispatch` moved to the API too, so there is one write path rather than one per client. The command grammar is still parsed only by the robot's task layer: a parser in the server would be a second grammar to keep in step. **The browser cannot publish**: `server/ui/mqtt.mjs` is a hand-rolled subscribe-only MQTT 3.1.1 client that implements no PUBLISH packet, so the split is enforced by omission (M7 makes it structural with a subscribe-only broker credential). The UI is static ES modules — no bundler, no npm, no vendored library — served by the same stdlib `http.server`; `map.mjs` holds the Q5 world→pixel transform (`px = (wx-origin_x)/res`, `py = height - (wy-origin_y)/res`) and a pan/zoom/follow canvas, and only draws robots on the *same* site+floor as the selected one because a pose from another floor is a different map frame. **Basemaps come from site bundles on the fleet box** (`--maps-dir`, default `$MOTE_FLEET_HOME/sites`, the layout `sites.py` writes, seeded by rsync until **M4** makes the registry canonical behind the same two routes). **M1's websockets blocker is settled**: `pixi run fleet-broker-ws` runs `eclipse-mosquitto` under docker with the repo's own `mosquitto.conf`, because conda-forge's build has none; `broker.sh` strips the WS stanza when the local binary cannot honour it and says so. Two things that run in the same file (`test_ui.py` → `ui_test.mjs`) are the MQTT codec and the transform, tested under node against the very files the browser loads; `browser_check.mjs` drives a real headless Chrome over CDP against a running stack and is an operator's tool, not a CI test.
 
+## Releases and deployment
+
+**`pixi run sync` is the dev loop; released packages are the deployment mechanism** — full detail in **`docs/releasing.md`**, which is the file to read before touching anything under `release/`. The first-party packages are built by **`pixi-build-ros` straight from each `package.xml`** and published to the prefix.dev **`mote`** channel, so a robot runs an environment *resolved from a channel* with no checkout and no colcon. **One semver version for all first-party packages**, source of truth `[workspace] version` in `pixi.toml`, propagated by `release/version.py` into every `package.xml` (what the artifact carries) and `setup.py`; the per-package `pixi.toml` build manifests deliberately carry **no** `version`, so there is exactly one place to bump. Ten packages ship: the eight first-party ones plus **`sllidar_ros2` and `kinematic_icp`** — a robot installing from the channel has no source tree to build the submodules from, and they are built from **out-of-tree manifests** in `release/third_party/` because a `pixi.toml` inside a submodule would be an untracked file in someone else's repo. `mote_simulation` is **excluded by policy** (workstation-only; it builds fine). **`release-verify` is the load-bearing step**: it renders the robot deploy manifest against the freshly built local channel and *resolves* it, which is what catches a package that built but is uninstallable (it found `mote_fleet` depending on a nonexistent `ros-jazzy-python3-paho-mqtt`, the backend's fallback for a rosdep key RoboStack does not map — fixed with `extra-package-mappings`, as `scservo_linux` → `scservo-linux` is for `mote_hardware`). Two backend traps, both worked around and recorded: builds are always **`--clean`** because the incremental hash covers `package.xml` and sources but *not* `[package.build.config]`, so a mapping edit silently reuses a cached recipe; and the backend is pinned to **`pixi-build-ros 0.5.0.*`** because pixi 0.70.2 provides build API v4 while the 0.6.x backends need v5. **Publishing is never automatic** — CI builds and verifies both arches on a tag but does not upload; `pixi run release-publish` is a human action behind a typed confirmation. **A robot updates by version into slots** (`release/deploy/mote-update`: `stage` / `cutover` / `rollback` / `prune`) under `~/mote-deploy/`, with `current` flipped in one atomic rename and units pointed at the *symlink* so the next cutover redirects them without reinstalling; a cutover is stop-then-start (the Pi has neither the CPU for two stacks nor a second set of serial ports), health-gated, and rolls back by itself if the units do not come back. **`$MOTE_HOME` is outside the deploy root entirely**, so identity/maps/calibration cannot be touched by an update — asserted by `release/test/test_mote_update.py`, which runs a full stage→cutover→rollback→prune cycle against a stub pixi and checks `~/.mote` is byte-identical afterwards. `mote_bringup` therefore ships `udev/`, `systemd/`, `networkmanager/` and `tailscale/` into `share/` (its `setup.py`), because a checkout-free robot still has to run `pixi run setup`; `systemd/install.sh` honours a `MOTE_REPO` override and resolves `@DDS_CONFIG@` for either layout.
+
 ## Sites (maps & zones)
 
 Everything that is only meaningful relative to one mapped place — the Nav2 map pair, the slam_toolbox posegraph, and named zones — lives together as a **site bundle** under `~/.mote/sites/<site>/floors/<floor>/`, managed by `mote_bringup/sites.py` (CLI: `pixi run site`, docs in the module docstring). A floor is one SLAM session (one map frame); a site groups floors sharing a location. `~/.mote/active.yaml` selects the active site/floor per robot; launch files resolve the map (`nav2_launch.py`, `robot_launch.py`) and zones (`tasks_launch.py`) from it at launch time (zones fall back to the committed default). `MOTE_HOME` overrides `~/.mote` for tests/experiments. Map artifacts are immutable **revisions** under `floors/<floor>/maps/<rev>/`, published by atomically flipping the `floors/<floor>/map` symlink once the revision is complete — a half-written save or interrupted transfer is never visible, and `site use-map <rev>` rolls back. `save-map` stores the posegraph alongside the map so mapping can be *continued* in the same frame later (extend, don't remap — remapping breaks zone coordinates). Mapping runs also record the `mapping` rosbag stream by default (`mapping_launch.py record:=true`; the sim passes false), and `save-map` stamps the session's bag into the revision's `meta.yaml` for provenance (`site info` shows it). Zones are taught by driving there and running `pixi run save-zone <name>`, not by editing YAML; a zone is a named pose (a fetch waypoint or a `goto <zone>` target) that may optionally carry an area **footprint** — a taught `--radius` circle, or a `polygon` outline that follows the actual room walls — so it reads as a room and answers "am I in it"; one concept, one `zones.yaml`, `site info` shows the zone/footprint counts. Maps are saved as PNG (map_server reads it natively; browsers can render it directly). `save-map` automatically runs an FFT structure-extraction **cleaning pass** (`mote_bringup/map_cleanup`, `sites._promote_cleaned`): it keeps the untouched map_saver output as `map_raw.png` and promotes the decluttered image to the served `map.png` (plus a `diagnostics.png`), so navigation always consumes the cleaned map while the raw is retained for provenance/audit. The `map.yaml` frame is identical for both, so zones/localization are unaffected; a cleaning failure falls back to serving the raw. The posegraph belongs to the raw map — mapping continuation extends from raw, never the cleaned image.
@@ -245,7 +257,7 @@ With multiple identical USB-serial adapters, pin by serial number — see commen
 
 ## Environment
 
-pixi activates `install/setup.sh` and sets `RMW_IMPLEMENTATION=rmw_cyclonedds_cpp` automatically. Dependencies come from the `mote` prefix.dev channel, robostack-jazzy, and conda-forge. The default environment is what runs on the robot (a Raspberry Pi, deployed with `pixi run sync`); the `dev` feature adds `ros-jazzy-desktop` and the `rviz` task.
+pixi activates `install/setup.sh` and sets `RMW_IMPLEMENTATION=rmw_cyclonedds_cpp` automatically. Dependencies come from the `mote` prefix.dev channel, robostack-jazzy, and conda-forge. The default environment is what runs on the robot (a Raspberry Pi); the `dev` feature adds `ros-jazzy-desktop` and the `rviz` task. A robot reached by `pixi run sync` runs this workspace from source — that is the dev loop. A *released* robot runs the generated deploy manifest instead (`release/deploy/pixi.toml`, see Releases above), which carries the same dependencies plus the released `ros-jazzy-mote-*` packages and a curated subset of the tasks.
 
 **DDS scoping.** The `sim` environment additionally sets `ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST`, so sims and benchmarks are invisible to the LAN and to each other's machines; `bench.py` claims a free `ROS_DOMAIN_ID` + `GZ_PARTITION` per invocation so two runs on one machine stay separate. Discovery visibility is one-way: a `LOCALHOST` participant still finds same-host default-range ones, but not vice-versa — hence `pixi run rviz-sim` (RViz joined to the sim's host-local graph) alongside `pixi run rviz` (default range, for the robot). The robot itself keeps LAN discovery and is tuned instead by `config/cyclonedds.xml` (systemd only).
 

README.md

@@ -225,25 +225,37 @@ The simulated lidar uses RPLIDAR C1 datasheet values from
 
 ### Deploying to the Pi
 
-The above section assumes you are developing entirely on the Pi which is
-definitely feasible at this stage. I do however want to support a more
-"professional" workflow and so we need a way to develop on laptops and run on
-the Pi. The exact mechanism that we will use is TBD.
+The above section assumes you are developing entirely on the Pi, which is
+definitely feasible at this stage. Developing on a laptop and running on the Pi
+takes one of two paths, depending on whether you are iterating or shipping.
 
-For now, I currently develop on a workstation and push to the Pi with rsync. The
-`sync` task targets an SSH host named `mote` — change the host in the
-[`pixi.toml`](pixi.toml) `[tasks]` `sync` entry to match your Pi, then:
+**Iterating — rsync.** The `sync` task targets an SSH host named `mote` — change
+the host in the [`pixi.toml`](pixi.toml) `[tasks]` `sync` entry to match your
+Pi, then:
 
 ```bash
 pixi run sync             # one-shot push
 pixi run sync-watch       # keep pushing on every save (needs the dev env)
 ```
 
-For pushing a finished build to one or more robots, the direction is to publish
-the first-party packages to the `prefix.dev/mote` channel (built with
-[`pixi-build-ros`](https://pixi.prefix.dev/latest/build/ros/)) so a robot just
-needs `pixi install` — no source checkout or compile on the bot. That work is in
-progress.
+Because the build uses `colcon --symlink-install`, edits to launch files,
+`robot.yaml` and Python go live with no rebuild; only C++ needs `pixi run build`.
+
+**Shipping — versioned packages.** The first-party packages are published to the
+`prefix.dev/mote` channel (built with
+[`pixi-build-ros`](https://pixi.prefix.dev/latest/build/ros/)), so a robot runs
+an environment resolved from that channel — no source checkout and no compile on
+the bot. Updates are by version, and reversible:
+
+```bash
+mote-update stage 0.2.0     # download and install; the robot keeps running
+mote-update cutover 0.2.0   # stop, flip, restart, health-gate
+mote-update rollback        # back to the previous version
+```
+
+`~/.mote` — identity, maps, calibration — lives outside the deploy and survives
+every update. See [`docs/releasing.md`](docs/releasing.md) for the version
+scheme, how a release is cut, and the full robot update procedure.
 
 ## SO-101 Follower Arm