Skip to content

Fix reversed argument order in H1JoystickGaitTracking action-rate penalty - #352

Open
rootkiller6788 wants to merge 1 commit into
google-deepmind:mainfrom
rootkiller6788:fix-h1-action-rate-arg-order
Open

rootkiller6788 wants to merge 1 commit into
google-deepmind:mainfrom
rootkiller6788:fix-h1-action-rate-arg-order

Conversation

@rootkiller6788

Copy link
Copy Markdown

Bug

In mujoco_playground/_src/locomotion/h1/joystick_gait_tracking.py, the action_rate reward for the registered H1JoystickGaitTracking environment was called with its three arguments in the wrong order:

"action_rate": self._cost_action_rate(
    info["last_act"], info["last_last_act"], action
),

The method signature is _cost_action_rate(self, act, last_act, last_last_act) and computes:

def _cost_action_rate(self, act, last_act, last_last_act):
    c1 = jp.sum(jp.square(act - last_act))
    c2 = jp.sum(jp.square(act - 2 * last_act + last_last_act))
    return c1 + c2

So with the current call, act=info["last_act"] and last_act=info["last_last_act"]: the first-derivative penalty c1 measures the change between the two previous actions and completely ignores the action just taken, and the second-derivative term c2 is computed about the wrong point.

Every other locomotion environment with the same 3-argument helper passes the arguments in the intended order, e.g. g1/joystick.py, go1/joystick.py, t1/joystick.py, op3/joystick.py, berkeley_humanoid/joystick.py:

"action_rate": self._cost_action_rate(
    action, info["last_act"], info["last_last_act"]
),

Impact

H1JoystickGaitTracking has action_rate=-0.01 in its reward config, so this penalty is active during training. Because the arguments are shifted, the penalty signal is wrong at every step: the current action is excluded from the first-derivative term, so the policy receives a mis-scoped action-rate cost.

Fix

Reorder the arguments to (action, info["last_act"], info["last_last_act"]), matching the method signature and every sibling environment.

Verification

  • Confirmed the method signature (mujoco_playground/_src/locomotion/h1/joystick_gait_tracking.py:456-462) and that all ~7 sibling envs with the same 3-arg helper call it with action first.
  • Confirmed H1JoystickGaitTracking is a registered environment (_envs/_cfgs in mujoco_playground/_src/locomotion/__init__.py) and action_rate=-0.01 is non-zero in its default reward config.
  • Confirmed last_act/last_last_act are initialized in reset() and updated each step(), so the bug manifests on every transition.
  • Confirmed the bug is still present in current upstream main (fetched via GitHub API).
  • py_compile passes on the edited file.

…alty

H1JoystickGaitTracking called _cost_action_rate with (info["last_act"], info["last_last_act"], action), but the signature is (act, last_act, last_last_act). The current action therefore never enters the first-derivative term c1=(act-last_act)^2, and the second-derivative term is computed about the wrong point. Every other locomotion env (g1, go1, t1, op3, berkeley_humanoid, ...) passes (action, info["last_act"], info["last_last_act"]).
@rootkiller6788
rootkiller6788 marked this pull request as ready for review September 3, 2026 05:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant