Access hand data from Unity components in the scene
Access hand-tracking data from Unity components in the scene.
The XRHand
There are also standard components to drive a Skinned Mesh Renderer component and its skeleton from these events which is outlined in the Hand Visuals section.
Access hand data from the XR Hand Subsystem
Access hand tracking data from the XRHand
The XRHand
The best way to access the data is through the updated
You can also access XRHand objects directly from the XRHand
Get the XRHandSubsystem instance
Get the hand subsystem from the active XR loader using SubsystemManager.GetSubsystems
:
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.Hands;
// ...
var handSubsystems = new List<XRHandSubsystem>();
SubsystemManager.GetSubsystems(handSubsystems);
You can get the subsystem after the XR system has finished initialization. By default, initialization occurs before any MonoBehaviour Start methods are called. However, if your project initializes XR manually, you must wait for your code to finish loading the subsystem.
Subscribe to hand update events
To subscribe to the hand update event, assign an Action delegate function to the XRHandXRHandSubsystem
calls your delegate function.
Use the updateUpdate
event, or "BeforeRender" which occurs just before rendering begins.
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.XR.Hands;
public class HandsExample : MonoBehaviour
{
XRHandSubsystem m_HandSubsystem;
void Start()
{
var handSubsystems = new List<XRHandSubsystem>();
SubsystemManager.GetSubsystems(handSubsystems);
for (var i = 0; i < handSubsystems.Count; ++i)
{
var handSubsystem = handSubsystems[i];
if (handSubsystem.running)
{
m_HandSubsystem = handSubsystem;
break;
}
}
if (m_HandSubsystem != null)
m_HandSubsystem.updatedHands += OnUpdatedHands;
}
void OnUpdatedHands(XRHandSubsystem subsystem,
XRHandSubsystem.UpdateSuccessFlags updateSuccessFlags,
XRHandSubsystem.UpdateType updateType)
{
switch (updateType)
{
case XRHandSubsystem.UpdateType.Dynamic:
// Update game logic that uses hand data
break;
case XRHandSubsystem.UpdateType.BeforeRender:
// Update visual objects that use hand data
break;
}
}
}
For a complete code example including how to handle late initialization, refer to the HandVisualizer
component in the Hand
Get joint data
Get the data for individual joints with the XRHand.
The joints of the hand are indexed by the XRHand
for(var i = XRHandJointID.BeginMarker.ToIndex();
i < XRHandJointID.EndMarker.ToIndex();
i++)
{
var trackingData = hand.GetJoint(XRHandJointIDUtility.FromIndex(i));
if (trackingData.TryGetPose(out Pose pose))
{
// displayTransform is some GameObject's Transform component
displayTransform.localPosition = pose.position;
displayTransform.localRotation = pose.rotation;
}
}
Note that some or all of the data for a joint might not be successfully tracked in a given update. The TryGet functions of the XRHand
In addition, the hand data plug-in providing the hand data might not support every joint in the XRHand
Note
The XRHandSubsystem stores the data associated with each joint in an internal native array and updates the elements in place when new hand data becomes available. If you make a copy of an XRHand object, the copy still points to the original native array. To take a snapshot of the joint data, you must copy the individual XRHandJoint objects at the desired point in time.
Check data validity
Hand data can be unreliable for a variety of reasons. A hand or part of a hand might be occluded or out of sensor range. The provider plug-in supplying the data might not support every tracked point or might not calculate certain aspects of the data, such as velocity. Before you use the hand data, you should make sure that it is valid.
The XR Hands API provides several APIs that you can use to check data validity.
API | Purpose |
---|---|
XRHand |
Indicates which joints are supported by the current hand data provider. You can use this property before the XRHand |
XRHand |
A callback function invoked when the system starts tracking a hand. |
XRHand |
A callback function invoked when tracking of a hand is lost. |
XRHand |
Flags describing which types of data are available in the most recent update. These flags apply to the XRHand |
XRHand. |
Indicates whether the user's corresponding hand is currently being tracked by the system. |
XRHand |
Indicates which types of data in the joint are valid. A specific type of data might be invalid because the system could not determine the value in the current update or because the hand data provider does not support that type of data. |
XRHand |
The XRHand |
Get supported joints array
The XRHand
The following example uses the jointsInLayout
array to instantiate a prefab for each supported joint, which are stored in a dictionary keyed by the joint ID so that they can be updated when new hand data is available:
Dictionary<XRHandJointID, Transform> CreateHandDisplay(
GameObject jointPrefab,
Transform sceneParent,
XRHandSubsystem handSubsystem)
{
var displayObjects = new Dictionary<XRHandJointID, Transform>();
for(var i = XRHandJointID.BeginMarker.ToIndex();
i < XRHandJointID.EndMarker.ToIndex();
i++)
{
if (handSubsystem.jointsInLayout[i])
{
XRHandJointID jointID = XRHandJointIDUtility.FromIndex(i);
var go = Instantiate(jointPrefab, sceneParent);
go.name = jointID.ToString();
displayObjects.Add(jointID, go.transform);
}
}
return displayObjects;
}
With this dictionary, you could update the transforms of these game objects when a hand update is available:
void UpdateJointTransforms(XRHand hand,
Dictionary<XRHandJointID, Transform> displayObjects)
{
foreach (var joint in displayObjects)
{
var trackingData = hand.GetJoint(joint.Key);
var displayTransform = joint.Value;
if (trackingData.TryGetPose(out Pose pose))
{
displayTransform.localPosition = pose.position;
displayTransform.localRotation = pose.rotation;
}
}
}