Skip to content

Versioned releases: publish to the prefix.dev mote channel, replace rsync deploys - #66

Open
MJohnson459 wants to merge 1 commit into
mainfrom
release/versioned-packages
Open

Versioned releases: publish to the prefix.dev mote channel, replace rsync deploys#66
MJohnson459 wants to merge 1 commit into
mainfrom
release/versioned-packages

Conversation

@MJohnson459

Copy link
Copy Markdown
Contributor

Versioned releases: the first-party packages are now published as versioned conda packages to the prefix.dev mote channel, replacing rsync as the deployment mechanism. pixi run sync stays the dev loop. Branch release/versioned-packages, commit 6fa5217. Nothing has been pushed and nothing has been published.

WHAT CHANGED

Build. Every released package gets a small pixi.toml next to its package.xml pinning pixi-build-ros 0.5.0.* and the distro. They deliberately carry no versionpackage.xml is the single source of truth, and release/version.py (show/check/set) propagates one repo-wide semver from [workspace] version into every package.xml and setup.py. Set to 0.1.0.

Ten packages ship: the eight first-party ones, plus sllidar_ros2 and kinematic_icp. The submodules are not optional — a robot installing from the channel has no source tree to build them from, so without them it has no lidar driver and no odometry. They build from out-of-tree manifests under release/third_party/, because a pixi.toml inside a submodule would be an untracked file in someone else's repo. mote_simulation is excluded by policy (workstation-only, already excluded from sync); it builds fine if that changes.

Release chain. pixi run release = test -> regenerate deploy manifest -> build all ten into dist/channel (a local indexed channel) -> release-verify, which renders the robot deploy manifest against that channel and resolves it. Publishing is a separate human step (release-publish, typed confirmation); CI (.github/workflows/release.yml) builds and verifies both architectures on a v* tag but never uploads.

Robot updates. release/deploy/mote-update (plain bash, no Mote env needed) keeps versions in slots under ~/mote-deploy/ with an atomically flipped current symlink: stage / cutover / rollback / prune. Cutover is stop-then-start — the Pi has neither the CPU for two ROS stacks nor a second set of serial ports — health-gated, and rolls itself back if the units do not return. The deploy manifest's dependencies are generated from the workspace pixi.toml (a robot resolving a different set than the release was tested against is not running the release); its tasks are hand-written and curated, with names matching the workspace so runbooks and systemd units work on either.

Supporting fixes this exposed. mote_bringup now ships udev/, systemd/, networkmanager/ and tailscale/ into share/ — without that, pixi run setup was a checkout-only command and a released robot could never install its udev rules or systemd units. systemd/install.sh honours a MOTE_REPO override and resolves a new @DDS_CONFIG@ placeholder for either layout, with units pointed at the current symlink so a cutover redirects them without reinstalling.

WHY release-verify EXISTS

It caught ros-jazzy-mote-fleet declaring a dependency on ros-jazzy-python3-paho-mqtt — a package that does not exist, invented by the backend's fallback naming for a rosdep key RoboStack has no entry for. All ten packages built cleanly; the release was uninstallable. Fixed with extra-package-mappings, as scservo_linux -> scservo-linux already was for mote_hardware.

Two backend traps found and worked around, both documented in docs/releasing.md: release builds are always --clean, because the incremental hash covers package.xml and sources but not [package.build.config], so editing a dependency mapping silently reuses the cached recipe; and the backend is pinned to 0.5.0.* because pixi 0.70.2 provides build API v4 while the 0.6.x backends need v5.

HOW IT WAS VERIFIED

  • All 410 existing tests pass (pixi run test), unaffected by the package.xml and setup.py changes.
  • 32 new tests (pixi run test-release, own ROS-free env): no first-party package missing from the release set, backends pinned, no manifest duplicating the version, committed deploy manifest not stale, every task the systemd units invoke present in a deploy.
  • A full stage -> cutover -> rollback -> prune cycle runs against a stub pixi and asserts ~/.mote comes out byte-identical — the runtime-state guarantee, tested rather than asserted in prose.
  • pixi run release completes end to end on linux-64 (exit 0). All ten packages resolve from the freshly built channel; evidence committed at release/evidence/dry-run-linux-64.json.
  • Package contents spot-checked: mote_hardware carries the plugin and all five tools with scservo-linux as a runtime dep; mote_bringup carries the udev/systemd/networkmanager/tailscale assets.
  • pixi run lint clean.

NOT DONE, DELIBERATELY

Nothing published, no linux-aarch64 build produced (CI does it; that workflow has not run), and no on-robot rehearsal — the update flow is verified only against a stub pixi, not real systemd and hardware. Those are task 262, which needs Michael and a Pi. Task 263 proposes condafying py-trees and feetech-servo-sdk, the last two dependencies a released robot still resolves from PyPI.

Per the repo's git rule the branch is committed locally only — not pushed, and no PR opened.

Replaces rsync as the *deployment* mechanism with versioned conda packages on
the prefix.dev `mote` channel. `pixi run sync` stays the development loop; a
robot now runs an environment resolved from a channel, with no source checkout
and no colcon, and moves between versions with rollback.

Ten packages are released, built by pixi-build-ros straight from each
package.xml: the eight first-party ones at a single repo-wide semver, plus
sllidar_ros2 and kinematic_icp. The submodules are not optional -- a robot
installing from the channel has no source tree to build them from, and without
them it has no lidar driver and no odometry. They build from out-of-tree
manifests under release/third_party, because a pixi.toml inside a submodule
would be an untracked file in someone else's repo. mote_simulation is excluded
by policy: workstation-only, and already excluded from sync.

One version for all first-party packages, source of truth [workspace] version,
propagated by release/version.py into every package.xml and setup.py. The
per-package build manifests deliberately carry no version, so there is exactly
one place to bump.

`pixi run release` is the whole dry run and uploads nothing: tests, then a
build into a local indexed channel, then release-verify, which renders the
robot deploy manifest against that channel and resolves it. That last step is
the one with teeth -- it caught mote_fleet depending on a nonexistent
ros-jazzy-python3-paho-mqtt, the backend's fallback naming for a rosdep key
RoboStack has no entry for. Every package had built fine. Publishing stays a
separate, human, typed-confirmation step; CI builds and verifies both
architectures on a tag but does not upload.

Robots update by version into slots under ~/mote-deploy with an atomically
flipped `current` symlink (release/deploy/mote-update: stage/cutover/rollback/
prune). Cutover is stop-then-start -- the Pi has neither the CPU for two ROS
stacks nor a second set of serial ports -- health-gated, and self-rolls-back if
the units do not return. $MOTE_HOME is outside the deploy root entirely, so an
update cannot touch identity, maps or calibration.

Supporting fixes this exposed: mote_bringup now ships udev/, systemd/,
networkmanager/ and tailscale/ into share/, because a checkout-free robot still
has to run `pixi run setup`; systemd/install.sh honours a MOTE_REPO override
and resolves the DDS config path for either layout, with the units pointed at
the `current` symlink so a cutover redirects them without reinstalling.

Verified: all 410 existing tests pass; 32 new release-tooling tests, including
a full stage/cutover/rollback/prune cycle against a stub pixi that asserts
~/.mote comes out byte-identical; `pixi run release` completes end to end on
linux-64 with the dry-run evidence committed under release/evidence/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Me6xVFErTy21hnARnMLt9M
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant