Not getting the same result as offline foundationpose

[ArghyaChatterjee]

Hello,

I am trying to see if Isaac ROS Pose Estimation can be used safely for estimating pose of a door lever but seems like it's breaking at some point.

There are some differences between the original offline FoundationPose and the Isaac ROS FoundationPose which I have been trying to investigate.

The original offline FoundationPose does:

• Initial registration with iterative refinement (iteration=5)
• Tracking with iterative refinement (track_one(... iteration=2))

In contrast, the Isaac ROS FoundationPose registration graph also supports iterative refinement (refine_iterations) but that is only for pose registration (estimation) not for tracking.

• It seems to do a single render --> refine --> transform pass per frame
• Found no exposed tracking refine_iterations parameter in the runtime tracking graph
• Tracking output is recursively fed back as the next tracking input

Looks like for thin/small objects like door lever, small residual tracking error --> reused next frame --> error accumulates --> pose drifts away over time.

Initial Analysis:

• Registration/Initial Pose Estimation output looks good
• Tracking output gradually drifts farther away
• Increasing registration refine iterations in Isaac ROS did not improve tracking drift
• Continuous reinitialization because of tracking drift (Set a threshold limit to reinitialize if the tracking drift is too much).

Isaac ROS Pose Estimation Results:
https://www.youtube.com/watch?v=V2CSoLekMIY
https://www.youtube.com/watch?v=5Dz9PhT-LBM

FoundationPose Offline Pose Estimation Results:
https://www.youtube.com/watch?v=SyeEkQUFTsQ
https://www.youtube.com/watch?v=B6aruRUEm5E

The same data has been fed to both of them but the result is different.

[ArghyaChatterjee]

I think I have framed it wrongly above. I have tried the offline version of foundationpose with refine_iterations = 1 for pose registration and refine_iterations = 1 for pose tracking to see if it fails like isaac ros version did but original fp is still giving the same output (better than isaac ros version) as above. There is something wrong with the isaac ros version implementation of the foundationpose that I am not able to figure out for this case specific. Do you need more info like the data that I am feeding ? I can share the data.

BTW, I am getting good results for other objects. This is hapenning with this object only.

[ArghyaChatterjee]

Here is the data that I am using. Let me know if you find any issue to download the data.