[xyyeh-flexiv]

Train and roll out ACT with chunk-relative SE(3) pose actions

Policies trained on absolute poses learn the frame they were recorded in: the SLAM map for a UMI gripper, the robot base for teleop. This adds UMI-style relative pose actions as a training option, so one policy can be trained on both teleop and UMI data and rolled out on the robot.

What changes

• Relative pose (policies/lerobot_plugins/relative_pose.py):
  • Each action pose in a chunk is trained as T_cur^-1 * T_action, where T_cur is the newest observed pose. Rollout converts each chunk back with T_cur * T_rel.
  • LeRobot processor steps wrap the normalizers. The pre step makes action poses relative and puts the identity in place of the state pose. The post step converts back. They are saved with the checkpoint, so rollout needs no setting.
  • Poses are found by axis name, in both this repo's layout and the UMI export's.
  • Action statistics are recomputed in the relative frame.
• Normalization as UMI does it:
  • The rotation inputs (tcp_motion_<ms>ms, tcp_rotation_wrt_start) normalize to themselves. Before, mean/std blew a fast turn up to about -110.
  • The gripper width maps its recorded range onto [-1, 1].
• Training:
  • POST /training takes relative_pose, for act, diffusion and multi_task_dit only, on datasets with named pose axes.
  • The training form has a relative_pose checkbox. It is hidden for fine-tunes and warns when temporal ensembling is on.
  • Each checkpoint gets a state_names.json sidecar.
• Rollout:
  • Feeds exactly the trained state columns, in trained order. Old checkpoints fall back to their dataset's info.json or the state width.
  • rollout/pose_history.py computes UMI's pose inputs (recent motion, rotation since start) from the measured TCP poses each tick, in both runners. gripper.force is fed the gripper's measured force.
  • The anchor is held while a cached chunk plays out.
  • Refuses ACT with temporal ensembling, relative-pose step skills, and absolute-pose checkpoints trained on a UMI export (those would drive the arm to SLAM map coordinates).
  • Warms up the policy (warmup_inferences, default 3) before the first tick, so compiling no longer drops the first chunk's waypoints.
• README: covers capping a rollout with FLEXIV_TRAINER_ROLLOUT__MAX_STEPS, the warm-up, and how settings map to environment variables.
• open_items/todo.md: lists what is done, what is open, and what to check before rolling out.

Tests

• tests/test_relative_pose.py: the conversion round trip, the held anchor, layout checks, which statistics change, rotations passing through and the width mapping onto [-1, 1].
• tests/test_pose_history.py: UMI pose inputs computed at rollout.
• tests/test_rollout_observations.py, tests/test_rollout_service.py: state fed by name, and the rollout refusals.
• tests/test_hub_integration.py: the state names sidecar.
• UI contract tests follow the asset version bump.

Notes

• Relative pose rollout will disable temporal ensembling