Skip to content

Expose Meta IOBT body tracking as get_body_meta_* API - #16

Open
zerenluo123 wants to merge 1 commit into
XR-Robotics:mainfrom
zerenluo123:main
Open

Expose Meta IOBT body tracking as get_body_meta_* API#16
zerenluo123 wants to merge 1 commit into
XR-Robotics:mainfrom
zerenluo123:main

Conversation

@zerenluo123

Copy link
Copy Markdown

Expose Meta IOBT body tracking as get_body_meta_*

Companion to XR-Robotics/XRoboToolkit-Unity-Client-Quest#6,
which adds body tracking to the Quest client. That PR publishes a new BodyMeta key; this one
parses it and exposes it to Python.

183 insertions, 0 deletions — the PICO Body path is untouched.

API

import xrobotoolkit_sdk as xrt

xrt.init()
if xrt.is_body_meta_available():              # check first, see below
    pose = xrt.get_body_meta_joints_pose()    # (70, 7) — x y z qx qy qz qw
    info = xrt.get_body_meta_info()           # jointSet / fidelity / calib / confidence / count
Function Returns
is_body_meta_available() bool
get_body_meta_joints_pose() (N, 7) position + quaternion per joint
get_body_meta_joint_ids() (N,) joint ids
get_body_meta_position_valid() (N,) per joint, False when occluded
get_body_meta_orientation_valid() (N,) fails independently of position
get_body_meta_tracking_space() (7,) tracking-space origin in the world frame
get_body_meta_timestamp_ns() int shared with the other sources in the frame
get_body_meta_info() dict of the metadata fields

Naming follows the existing get_body_* group with _meta_ inserted, so the two are adjacent in
autocomplete and neither shadows the other.

Why separate storage instead of reusing the Body arrays

The existing arrays are std::array<..., 24> and the parser truncates:

int jointCount = std::min(static_cast<int>(joints.size()), 24);

Meta IOBT sends 70 (or 84 for FullBody), so reusing them would silently drop 46 joints. The two
skeletons are also not interchangeable — PICO's comes from IMU trackers with velocity and
acceleration per joint, Meta's is camera-inferred and carries neither, since
OVRPlugin.BodyJointLocation holds only LocationFlags and Pose.

BodyMeta storage therefore uses std::vector sized from the incoming count rather than a fixed
array, so a future switch to the 84-joint FullBody set needs no change here.

Two behaviours worth knowing

isActive is checked before the joints are stored. When the headset leaves the head, tracking
stops but the client's JSON still holds the previous frame — it reuses its objects rather than
clearing them. The parser treats that as unavailable, because a stale frame read as live is worse
than no frame: nothing downstream can tell the difference.

calib is passed through, not acted on. Joints stream complete from the first frame; calib
reaching Valid only means the runtime stopped adjusting the skeleton's scale. Gating on it would
stall the stream 30–60 s on every re-don for no gain.

Testing

Verified end-to-end against a Quest 3 running the companion client build:

info: {'available': True, 'count': 70, 'jointSet': 'UpperBody', 'fidelity': 'High',
       'calib': 'Valid', 'confidence': 1.0, 'timeStampNs': 1785486843349566208}
pose shape (70, 7)         ids 0..69 contiguous
valid: pos 70/70  orient 70/70
quaternion norms: min 1.0  max 1.0
timestamps advancing: True         max joint delta over 0.5 s: 0.0676 m
PICO "Body" path: unaffected

Built against Python 3.12; import xrobotoolkit_sdk and all 8 functions resolve.


Happy to rename anything or restructure if you'd prefer a different shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant