Undergraduate student working on robot learning in simulation: teleoperation data collection, MimicGen-style data generation on NVIDIA Isaac Lab, and imitation-learning policies (Diffusion Policy, ACT, π0.5) evaluated in closed loop.
本科在读，方向是仿真中的机器人学习：遥操作采集、Isaac Lab / MimicGen 数据生成、模仿学习策略的闭环评测。
Every link goes to the upstream repository, so each item can be checked there.
NVIDIA Isaac Lab · isaac-sim/IsaacLab
| PR | What it fixes | Status |
|---|---|---|
| #7381 | On multi-GPU machines the CloudXR runtime could end up on a different GPU than the one rendering the XR frames, so the headset showed noise. Keeps the runtime on the renderer's GPU. | Merged |
| #7433 | DataGenConfig.max_num_failures was assigned in 18 Mimic environment configs but never read, so a low-success-rate generation run retried without bound. Makes it a real bound, with regression tests. |
Under review |
| #7434 | The cube-stacking success check accepted a cube that was still falling, and the data generator wrote such episodes into the dataset as successful demos. Adds an at-rest velocity gate. | Under review |
| #7380 | The CloudXR EULA prompt could not be accepted without a terminal, so headless and CI teleop runs failed at startup. Adds a non-interactive opt-in. | Under review |
AgiBot Genie Sim · AgibotTech/genie_sim
| PR | What it does | Status |
|---|---|---|
| #182 | Multi-episode teleop recording without restarting the simulator: one launch, many episodes, instead of paying the full Isaac Sim startup cost per episode. | Merged |
| #186 | The ROS server node was polled with a zero-timeout spin_once from the physics callback and could silently miss messages, after which no /sim/* topic worked and recording could not start. Spins it from a background executor instead. |
Merged |
| #179 | Keeps the VR UDP listener alive when a packet is not valid UTF-8. | Merged |
| #180 | Imports pynput lazily so teleop starts on headless containers. |
Merged |
| #190 | Resets scene objects to their initial poses at the end of each episode. | Under review |
Also issue #181, proposing multi-episode capture and domain randomization in the teleop path; the maintainers adopted the suggestion.
Hugging Face LeRobot · huggingface/lerobot
Issue #4259: LeRobot's Diffusion Policy had no EMA of the weights, although the reference implementation enables it in every config. It led to PR #4323, which added opt-in EMA and was merged.
Ongoing study on the Isaac Lab Franka cube-stacking task. Free-space segments of MimicGen-generated trajectories receive a smooth pose perturbation (a sin²-family envelope that is zero in value and slope at both ends), while contact segments stay untouched. The goal is to map where data generation and the downstream policy break down as the perturbation grows.
- Generation side. Two sweeps, 20 joint-noise levels and 15 arc amplitudes, each with 5 seeds × 500 attempts (87,500 attempts in total), giving success-rate-vs-perturbation curves for the generator itself.
- Policy side. π0.5 fine-tuned on clean, perturbed and mixed datasets (300+ demos and roughly 90k frames each), each model evaluated with 300 closed-loop rollouts in simulation.
- Main finding so far is methodological. With one training run per dataset the comparison is not trustworthy on this task: re-training on the same data with a different seed moved the success rate by 3–4 pp, and an unconverged run moved it by 22.5 pp between two of its own checkpoints. An apparent 11 pp gap between datasets shrank to −1.8 pp, within noise, once the learning-rate schedule was fixed and seeds were averaged.
Write-ups so far: why arc-generation success falls with amplitude · evidence behind the two Isaac Lab Mimic fixes. A standalone repository with the full results is in preparation.
lerobot-dp-toolkit: small scripts for running LeRobot's Diffusion Policy at the 10 Hz operating point the original paper was tuned for (dataset FPS downsampling, action hold at deployment, checkpoint averaging, camera-resolution alignment). Argued from source code, with what it does not claim stated up front.