Face tracking

This page supplements the AR Foundation Face tracking manual. The following sections only contain information about APIs where Google's Android XR runtime exhibits platform-specific behavior.

Tip

When developing an AR app, refer to both the AR Foundation documentation as well as the required packages for each platform you support.

Important

You must ensure to configure the appropriate Permissions to use face tracking features on Android XR.

Optional feature support

Android XR implements the following optional features of AR Foundation's XRFaceSubsystem:

Feature	Descriptor Property	Supported
Face pose	supportsFacePose
Face mesh vertices and indices	supportsFaceMeshVerticesAndIndices
Face mesh UVs	supportsFaceMeshUVs
Face mesh normals	supportsFaceMeshNormals
Eye tracking	supportsEyeTracking	Yes

Note

Refer to AR Foundation Face tracking platform support for more information on the optional features of the Face subsystem.

Face data

This platform exposes face data for the active user (the person wearing the headset). Currently, only gaze data is surfaced.

To know which face is the inward avatar-eyes gaze object, cast an XRFaceSubsystem to an AndroidOpenXRFaceSubsystem and read its inwardID property.

Permissions

AR Foundation's face tracking feature requires an Android system permission on the Android XR runtime. Your user must grant your app the android.permission.EYE_TRACKING permission before it can track face data.

To avoid permission-related errors at runtime, set up your scene with the AR Face Manager component disabled, then enable it only after the required permission is granted.

The following example code demonstrates how to handle required permissions and enable the AR Face Manager component:

const string k_Permission = "android.permission.EYE_TRACKING";

[SerializeField]
ARFaceManager m_ARFaceManager;

#if UNITY_ANDROID
void Start()
{
    if (!Permission.HasUserAuthorizedPermission(k_Permission))
    {
        var callbacks = new PermissionCallbacks();
        callbacks.PermissionDenied += OnPermissionDenied;
        callbacks.PermissionGranted += OnPermissionGranted;

        Permission.RequestUserPermission(k_Permission, callbacks);
    }
    else
    {
        // enable the AR Face Manager component if permission is already granted
        m_ARFaceManager.enabled = true;
    }
}

void OnPermissionDenied(string permission)
{
    // handle denied permission
}

void OnPermissionGranted(string permission)
{
    // enable the AR Face Manager component after permission is granted
    m_ARFaceManager.enabled = true;
}
#endif // UNITY_ANDROID

For a code sample that involves multiple permissions in a single request, refer to the Permissions page.