Party OS

The ROBO PARTY LAB system stack for humanoid robots:
from motion control to interaction, whole-body manipulation and agentic humanoids.

English · 简体中文

🗺️ Roadmap · 🧩 Projects · 📦 Datasets · 📈 Star History

---

Party OS connects data collection and generation, motion retargeting, and supervised and unsupervised motion learning into a motion control layer. On top of it, the stack grows through interaction and perception (human–scene and human–object interaction), whole-body manipulation (VLA / world-action models), and finally agentic humanoids: a complete path from low-level tools to high-level interactive capabilities.

🗺️ Roadmap

_{The figure is generated by scripts/build_roadmap.py; edit the text there and rerun to update both languages.}

🧩 Projects

From low-level capabilities to high-level interaction, following the roadmap:

Layer	Module	Description	Repository
📚 Know-how	🧠 Knowledge base	ROBO PARTY technical know-how and documentation	Humanoid Motion Control Know-how (Chinese)
🏃 Motion Control Infra	🏭 Data generation generate	Motion data generation	🚧 Coming soon
	🔁 Retargeting hhtools	Fast retargeting of human or robot motion to any humanoid	Roboparty/human-humanoid-tools
	🎯 Supervised learning MimicLite	Model-free motion tracking (PPO / PPO-ROA): loading, tracking, training, evaluation and deployment; real-robot teleoperation	Roboparty/MimicLite
	⚡ Supervised learning PartyMimic	Model-based motion tracking: one general tracker in 7–30 minutes on a single RTX 4090, lowest held-out error on G1 and RP1 · vs MimicLite →	Roboparty/PartyMimic
	🛸 Unsupervised learning UFO	Unsupervised RL framework with FB / TeCH training, robot-aware motion import and real-world teleoperation	Roboparty/UFO
👀 Interaction & Perception Infra	📦 Interaction BFM I-BFM	Humanoid interaction control with FB representations and LOGO: carrying, pushing and kicking boxes	Roboparty/I-BFM-Code
	⛰️ Perceptive BFM P-BFM	Terrain-adaptive humanoid control: motion tracking, goal reaching, reward inference and teleoperation	Roboparty/P-BFM
🤲 Whole-Body Manipulation Infra	🌐 JEPA world model INTACT-JEPA	Efficient JEPA framework built on action-intent encoding	Roboparty/INTACT-JEPA zju3dv/INTACT-JEPA (upstream)

📊 Choosing a motion tracker? See MimicLite vs PartyMimic.

📦 Datasets

Curated public datasets for each layer of the stack. Party OS does not redistribute any data: download from the upstream source, follow each dataset's license, and cite it when you publish.

🏃 Motion control: human motion, interaction and terrain motion for tracking policies (14)

Retargeting modes follow hhtools: mimic flat-ground motion, intermimic human–object interaction, meshmimic terrain, R2R robot-to-robot.

Dataset	Content	hhtools mode	Paper	Download	Used in Party OS
AMASS	Unified optical mocap (SMPL-X)	mimic	📄 arXiv	🌐 site
LAFAN1	Mocap BVH: locomotion, dance, fall and get-up	mimic	📄 arXiv	💻 GitHub	PartyMimic, MimicLite training
BONES-SEED (SOMA)	Large-scale mocap library	mimic	📄 arXiv	🤗 HF	PartyMimic, MimicLite training
100STYLE	Stylized locomotion (Xsens BVH, 60 fps)	mimic	📄 ACM	🌐 site	MimicLite training (A3)
Motion-X	Whole-body motion extracted from video	mimic	📄 NeurIPS	💻 GitHub
PHUMA	Physically grounded humanoid motion	mimic	📄 arXiv	💻 GitHub
GVHMR	Monocular video → human motion (tool)	mimic	📄 arXiv	💻 GitHub
OMOMO	Human–object interaction	intermimic	📄 arXiv	🤗 HF
OmniContact	Optical-mocap human–object interaction (90 Hz)	intermimic	📄 arXiv	🤗 HF
holosoma	Terrain and parkour motion	meshmimic	📄 arXiv	💻 GitHub
PARC MS	Terrain and parkour motion	meshmimic	📄 arXiv	🤗 HF
Parkour Mocap	Parkour dataset: optical-mocap vaults, slopes, balance beam and flips with obstacle terrain (120 Hz, 143 clips)	meshmimic	—	🤗 HF
MotionDecode	Unitree G1 robot trajectories (120 Hz)	R2R	🌐 site	🤗 HF	Held-out evaluation (sim2real leaderboard)
any4hdmi collection	Pre-converted MimicLite training sets	—	—	🤗 HF	MimicLite training

🤲 Manipulation: robot teleoperation, humanoid, egocentric human and simulation data (19)

Category	Dataset	Content	Paper	Download
🦾 Cross-embodiment robot	Open X-Embodiment	1M+ real-robot trajectories across 22 embodiments	📄 arXiv	🌐 site
	DROID	In-the-wild Franka manipulation, 76k trajectories	📄 arXiv	🌐 site
	BridgeData V2	WidowX manipulation across many environments	📄 arXiv	🌐 site
	AgiBot World	Large-scale real-world dual-arm manipulation	📄 arXiv	🤗 HF
	RoboMIND	Multi-embodiment teleoperation, including a humanoid	📄 arXiv	🌐 site
	Galaxea Open-World	Mobile dual-arm manipulation in real homes and workplaces	📄 arXiv	🤗 HF
🤖 Humanoid	Humanoid Everyday	Open-world humanoid manipulation	📄 arXiv	🌐 site
	GR00T X-Embodiment Sim	Simulated humanoid and arm trajectories used by GR00T N1	📄 arXiv	🤗 HF
	Unitree open datasets	G1 teleoperation with dexterous hands (LeRobot format)	—	🤗 HF
👐 Egocentric human & hand–object	Ego4D	3,000+ hours of egocentric video	📄 arXiv	🌐 site
	EgoDex	Large-scale egocentric dexterous manipulation with 3D hand tracking	📄 arXiv	💻 GitHub
	HOT3D	Egocentric multi-view 3D hand and object tracking	📄 arXiv	🌐 site
	ARCTIC	Bimanual dexterous manipulation of articulated objects	📄 arXiv	🌐 site
	OakInk2	Bimanual hand–object manipulation for complex tasks	📄 arXiv	🌐 site
	DexYCB	Hand grasping of YCB objects	📄 arXiv	🌐 site
	GigaHands	Large annotated dataset of bimanual hand activities	📄 arXiv	🌐 site
🧪 Simulation benchmark	LIBERO	Lifelong-learning manipulation benchmark	📄 arXiv	🌐 site
	RoboCasa	Large-scale simulation of everyday household tasks	📄 arXiv	🌐 site
	BEHAVIOR-1K	1,000 everyday activities in realistic simulation	📄 arXiv	🌐 site

Many of these are available in the LeRobot format, which is a convenient common format for training.

🧠 Agent: task planning, embodied QA, navigation and 3D scene understanding (14)

Grouped by the Agentic Humanoid items on the roadmap.

Roadmap item	Dataset	Content	Paper	Download
📋 Task planning & multi-task decomposition	ALFRED	Household instruction following in simulation	📄 arXiv	🌐 site
	EgoPlan-Bench	Egocentric task planning for multimodal LLMs	📄 arXiv	💻 GitHub
	RoboVQA	Long-horizon multimodal reasoning for robots	📄 arXiv	🌐 site
	ShareRobot (RoboBrain)	Planning, affordance and trajectory annotations	📄 arXiv	🌐 site
	EmbodiedBench	Benchmark for vision-driven embodied MLLM agents	📄 arXiv	🌐 site
❓ Embodied QA (EQA)	EQA	Embodied question answering in 3D houses	📄 arXiv	🌐 site
	OpenEQA	Open-vocabulary EQA from episodic memory and active exploration	CVPR 2024	🌐 site
🧭 Scene semantics & navigation (VLN)	R2R	Vision-and-language navigation in real buildings	📄 arXiv	🌐 site
	RxR	Multilingual VLN with dense grounding	📄 arXiv	💻 GitHub
	VLN-CE	VLN in continuous environments	📄 arXiv	🌐 site
	HM3D	1,000 large-scale 3D environments for embodied AI	📄 arXiv	🌐 site
🏠 3D scene understanding	ScanNet	Annotated 3D reconstructions of indoor scenes	📄 arXiv	🌐 site
	ScanQA	3D question answering	📄 arXiv	💻 GitHub
	SQA3D	Situated question answering in 3D scenes	📄 arXiv	🌐 site

📈 Star History

_{Stars of Party OS plus every project listed above (INTACT-JEPA counts both zju3dv/INTACT-JEPA and its Roboparty fork). Star counts are recorded daily by GitHub Actions since 2026-11-11; GitHub does not expose when each star was given, so the hatched part before that is interpolated from each repository's creation date.}

📜 License

This repository is licensed under the GNU General Public License v3.0 (GPL-3.0). Each sub-project keeps its own license; check the corresponding repository before use. Datasets listed above are subject to their own licenses.