Skip to content

[Bug] Re-enable IsaacContrib-DrLegs-Walk in test_contrib_environments after fixing Kamino NaN divergence #7625

Description

@StafaH

Summary

IsaacContrib-DrLegs-Walk (and its deprecated alias Isaac-DrLegs-Walk-v0) is skipped in source/isaaclab_tasks/test/contrib/test_contrib_environments.py as of #7623 because the Kamino P-ADMM solver intermittently diverges to NaN under random actions. This issue tracks fixing the underlying instability and re-enabling the test.

What was observed

  • CI failure on an unrelated PR ([Tests] Consolidate installation CI environments #7610): test-contrib-environments run 34042243157, test_contrib_environments[IsaacContrib-DrLegs-Walk], AssertionError: Invalid data on the policy observation.
  • The first observation row is NaN across projected gravity, base angular velocity and joint positions: the robot state itself is non-finite, not a single observation term.
  • Reproduced locally by mirroring env_test_utils._run_environments (2 envs, 20 random-action steps, fresh env per seed): in 1 of 12 seeds the whole state of env 0 (root position, root quaternion, joint positions and velocities) turns NaN at step 6. The heading-driven velocity command inherits the NaN through heading_w.
  • IsaacContrib-DrLegs-HoldPose (same robot, same Kamino preset, same random actions) stayed finite in 9 of 9 local trials and passes in CI. The Walk task adds the foot contact sensor, the velocity command and the gait rewards.
  • The root_height and bad_orientation termination terms cannot catch a NaN state because every comparison against NaN is false, so the episode never resets.

Environment

  • Solver preset: DrLegsPhysicsCfg.default in source/isaaclab_tasks/isaaclab_tasks/contrib/dr_legs/hold_pose_env_cfg.py (KaminoPADMMSolverCfg, use_fk_solver=True, max_contacts_per_world=32, sparse_jacobian=True, rho_0=0.02, use_cuda_graph=True), dt=1/150, decimation=3.
  • Runtime log during the failing run: [KAMINO] Capping rigid_contact_max to 64 (32/world * 2 worlds) and Falling back to the 'euler' integrator: 'moreau' requires use_collision_detector=True.

To close this issue

  1. Identify why the Kamino solve diverges for DR Legs Walk under random joint targets (solver bug, contact capacity, or preset tuning) and fix it in Newton/Kamino or in the DR Legs solver preset.
  2. Consider a NaN-robust guard so a diverged environment terminates and resets instead of propagating (the termination terms are blind to NaN today).
  3. Remove "DrLegs-Walk" from _SKIPPED_TASK_SUBSTRINGS in test_contrib_environments.py and confirm the test passes repeatedly (the failure rate was roughly 1 in 12 seeds, so a single green run is not sufficient).

Related: #7623 (skip), #6542 (DR Legs success metric), #6667 (DR Legs PhysX support).

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Labels

bugSomething isn't workingisaac-labRelated to Isaac Lab team

Type

No type

Projects

  • Status
    In progress

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions