Skip to content

Prevent torch.inference_mode tensor tainting across Environment and Simulator - #12

Open
M-Colley wants to merge 1 commit into
google-deepmind:mainfrom
M-Colley:fix/environment-inference-mode-reset
Open

M-Colley wants to merge 1 commit into
google-deepmind:mainfrom
M-Colley:fix/environment-inference-mode-reset

Conversation

@M-Colley

Copy link
Copy Markdown

Summary

Prevent torch.inference_mode() contexts (used by RL runners such as RSL-RL during rollout collection and episode_length_buf updates) from tainting internal environment, device-layer, task-layer, and TorchWrapper simulator buffers:

  • Wrap zero-copy view creation in TorchTensor.__init__ with with torch.inference_mode(False): so lazily cached sim.model / sim.data views (e.g. sim.data.ctrl or sim.data.time) are never permanently marked is_inference=True on first access inside torch.inference_mode().
  • Guard Environment, DeviceLayer, TaskLayer, Simulator, Clock, and flat_scene.build_* entrypoints (plus DeviceLayer._output_buffers allocation) with torch.inference_mode(False) so internal state tensors remain mutable across transitions out of torch.inference_mode().
  • Avoid mutating self._last_timestep.observation and self._last_timestep.step_type in place inside Environment._reset and Environment.step so previously returned TimeStep tensors are not overwritten by subsequent steps or resets.

Testing

  • Added parameterized EnvironmentInferenceModeTest (Impl.WARP & Impl.CPP, dict & single-tensor observations, constructed inside and outside torch.inference_mode()) covering rollouts with auto-reset and sim.data.time writes inside torch.inference_mode() followed by step(), partial reset_with_options(), and full reset() outside torch.inference_mode(), as well as TimeStep tensor immutability.
  • Ran pyink --check, isort --check, pyrefly check, and pytest reaf/simulation/torch/_src/core/ reaf/simulation/torch/_src/builders/flat_scene/build_test.py (210 passed).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant