Meta Hand Tracking Unextrapolated Poses OpenXR feature

Access hand joint poses that the runtime has not extrapolated, along with the time at which they were captured.

Unity OpenXR provides support for the XR_META_hand_tracking_unextrapolated_poses extension. By default, the runtime extrapolates hand joint poses forward in time so that rendered hands stay aligned with the user's real hands despite hand-tracking latency. This extension surfaces the poses as they were captured, without that extrapolation, together with the capture time.

This feature makes unextrapolated poses available in addition to the extrapolated poses. The hands you retrieve without specifying an extrapolation mode are unaffected by this feature.

Enable this feature when your application needs the poses as the runtime captured them, together with the time of capture, rather than the runtime's estimate of where the hands are now. For rendering hands, continue to use the extrapolated poses.

Prerequisites

To use the Meta Hand Tracking Unextrapolated Poses feature, your project must meet the following requirements:

Enable the feature

To enable the Meta Hand Tracking Unextrapolated Poses feature:

  1. Go to Project Settings > XR Plug-in Management > OpenXR.
  2. Enable Meta Hand Tracking Unextrapolated Poses in the feature list.

If the current runtime doesn't support the extension, this package disables the feature and hand tracking continues to operate normally with extrapolated poses only.

Check that the feature is active

Check the feature's enabled property once the hand subsystem is running. By then, enabled is false if the feature is off in the project settings, or if the runtime doesn't support the extension:

using UnityEngine.XR.Hands.OpenXR;
using UnityEngine.XR.OpenXR;

var feature = OpenXRSettings.Instance.GetFeature<MetaHandTrackingUnextrapolatedPoses>();
if (feature != null && feature.enabled)
{
    // Unextrapolated poses are available.
}

Access unextrapolated poses

An unextrapolated hand is an ordinary XRHand: subscribe to updates, read the root pose, iterate the joints, and check tracking state exactly as described in Access hand data. The only difference is how you retrieve it: pass XRHandExtrapolation.Unextrapolated to XRHandSubsystem.TryGetHand instead of reading leftHand or rightHand:

void OnUpdatedHands(XRHandSubsystem subsystem,
    XRHandSubsystem.UpdateSuccessFlags updateSuccessFlags,
    XRHandSubsystem.UpdateType updateType)
{
    if (updateType == XRHandSubsystem.UpdateType.Dynamic)
    {
        if (subsystem.TryGetHand(Handedness.Left, XRHandExtrapolation.Unextrapolated, out XRHand hand))
        {
            // Use hand as you would subsystem.leftHand
        }
    }
}

When to read unextrapolated poses

Unextrapolated poses are refreshed once per frame, during the Dynamic update, which is why the example checks updateType. A BeforeRender callback sees the same values a second time.

When the call fails:

TryGetHand returns false when the current provider can't surface unextrapolated poses. Treat that as the result of an individual call, not as a capability check. Determine support from the feature's enabled property, as shown in Check that the feature is active. XRHandSubsystem.GetHand takes the same extrapolation argument but throws an InvalidOperationException instead of returning false.

Interpret the tracking data

updateSuccessFlags, trackingAcquired, trackingLost, and XRHandTrackingEvents report tracking for the extrapolated hands only. For an unextrapolated hand, check its isTracked property and its joints' trackingState instead.

XRHand.extrapolationMode tells you which set of poses a hand holds.

Get the capture time

Call XRHand.TryGetTimestamp to get the time at which the poses were captured:

if (hand.TryGetTimestamp(out long timestamp))
    Debug.Log($"Poses captured at {timestamp}");

The timestamp is the raw XrTime value reported by the OpenXR runtime, on that runtime's clock. It isn't a Unity time value, so only compare it against other XrTime from the same clock. The capture time is available only on hands retrieved with XRHandExtrapolation.Unextrapolated, so TryGetTimestamp returns false for extrapolated hands.

When a hand isn't tracked, it keeps the capture time of its last successfully tracked poses.

Important considerations

Be aware of the following when using unextrapolated poses:

  • These poses lag behind the user's real hands by the hand-tracking latency of the device. Keep driving hand visuals from the extrapolated hands, leftHand and rightHand, which the runtime extrapolates to compensate for that latency.
  • An existing joint processor leaves these poses untouched. Refer to Process unextrapolated joints to process them as well.
  • Other hand tracking features, such as Hand Joints Motion Range and Hand Tracking Data Source, apply only to extrapolated hands.
Warning

The runtime reports incorrect joint velocities for unextrapolated poses. Read XRHandJoint.TryGetLinearVelocity and XRHandJoint.TryGetAngularVelocity from the extrapolated hands instead, or derive the velocities you need from successive unextrapolated poses and their capture times.

Additional resources