[pi0.5] Robot reaches object but fails to grasp, keeps pushing object with small actions

[Caesar107]

Hi,

We are currently using pi0.5 and fine-tuning it on our self-developed robot platform. Our action space is based on end-effector pose (xyz + Euler angles).

We are encountering a consistent issue during pick tasks, and would really appreciate any guidance from the team or the community.

---

Problem Description

During inference:

• The robot successfully moves towards the object
• The trajectory looks smooth and reasonable
• The end-effector reaches the target object correctly

However:

• The robot does not perform a grasp
• Instead, it keeps outputting very small action values
• As a result, the robot continuously pushes the object (e.g., a bottle) instead of grasping it

This behavior is consistent across multiple runs.

---

Additional Observations

• Our open-loop results look good
• The model seems to understand how to approach and align with the object
• The issue appears specifically at the grasp execution stage

---

Setup Details

• Model: pi0.5
• Action space: end-effector pose (xyz + Euler angles)
• Task: pick / grasp object (e.g., bottle)
• Robot: custom platform

---

Questions

We would really appreciate any insights on:

• Have others encountered similar behavior (approach works but grasp never happens)?

• Are there recommended practices for grasp execution when using pose-based action space?

• Is this kind of behavior related to:

  • action scaling
  • gripper control representation
  • or training data distribution?

---

Any suggestions would be greatly appreciated 🙏

We are happy to provide more details if needed.

Thanks a lot!

[BoxiangWang1215]

I recently encountered the same problem. In my Mujoco scene, when I tried to pick up a bcube, the robotic arm would quickly approach in the same direction and then, just like you described, it wouldn't grab it but would continue to fall down. Have you fine-tuning on your data or you use pi05 DROID?

[Caesar107]

@BoxiangWang1215 Yes, we fine-tuned it on our own robot with around 200 teleoperation episodes collected on our platform.

The reaching behavior looks good, but at the grasp stage it keeps producing very small actions and pushes the object instead of grasping it. We are still checking whether this comes from the gripper representation, action scaling, or insufficient grasp data coverage.

[continuity3]

i ues the same state and action to train the pi0.5 for our robots too. For pick the cube, we find that robot arm can go around the cube but can not pick it, for the loss is around 0.025. We used torch for full fine tuning