Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file added docs/images/xr_enable_second_viewport.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/images/xr_second_viewport_robot_camera.jpg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,53 @@ Step 2: Start Arena Teleop
Arena teleop session with XR running. Stereoscopic view (left) and OpenXR settings in the XR tab (right).


Step 2b: Monitor Recording with a Second Viewport (Optional)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

For higher-quality datasets, we recommend a two-person workflow when collecting demonstrations in
Step 4: one person teleoperates from the headset, while a second person watches the host monitor
to confirm each trajectory stays inside the robot's head-camera field of view. Anything that
drifts outside the recording FOV is absent from the saved HDF5 and absent from the policy's view
at training time, so catching it live saves a re-record.

The Arena application's default viewport shows the *teleoperator's* stereoscopic perspective —
what the headset wearer sees, not what ``record_demos.py`` will store. To watch both side-by-side,
open a second viewport bound to the robot's head camera:

#. In the running Arena application, open the **Window** menu and toggle on **Viewport 2**.

.. figure:: ../../../images/xr_enable_second_viewport.jpg
:width: 100%
:alt: Isaac Lab Window menu with the Viewport 2 toggle highlighted.
:align: center

Enable a second viewport from the **Window** menu.

#. In the new Viewport 2, click the camera selector in the viewport toolbar and choose the
robot's head-mounted camera (``RobotHeadCam``, under
``/World/envs/env_0/Robot/head_link``). This is the camera that ``record_demos.py`` writes
to the HDF5 file in Step 4, so any motion that leaves this frame will be absent from the
dataset.

.. figure:: ../../../images/xr_second_viewport_robot_camera.jpg
:width: 100%
:alt: Two viewports side-by-side: stereoscopic XR view (left) and the robot head-camera view (right).
:align: center

Dual-viewport layout: the stereoscopic XR view (left) is the teleoperator's perspective,
and the head-camera view (right) is what the dataset captures. The observer keeps every
grasp and placement inside the right viewport and gives the teleoperator live feedback
("move a touch to your right — your hand is at the edge of frame").

.. note::

``RobotHeadCam`` is only spawned when ``--enable_cameras`` is set. The ``record_demos.py``
command in Step 4 enables it by default, so the camera shows up in the camera selector once
you are recording. The smoke-test ``teleop.py`` command above omits ``--enable_cameras`` for
performance; pass it there too if you want to validate the dual-viewport layout before
entering VR.


Step 3: Connect from the headset device
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Expand Down
Loading