Face tracking

This page is a supplement to the AR Foundation Face tracking manual. The following sections only contain information about APIs where ARCore exhibits unique platform-specific behavior.

Tip

When developing an AR app, refer to both the AR Foundation documentation as well as the required packages for each platform you support.

Optional feature support

ARCore implements the following optional features of AR Foundation's XRFaceSubsystem:

Feature	Descriptor Property	Supported
Face pose	supportsFacePose	Yes
Face mesh vertices and indices	supportsFaceMeshVerticesAndIndices	Yes
Face mesh UVs	supportsFaceMeshUVs	Yes
Face mesh normals	supportsFaceMeshNormals	Yes
Eye tracking	supportsEyeTracking

Note

Refer to AR Foundation Face tracking platform support for more information on the optional features of the face subsystem.

Session configuration

Face tracking on ARCore requires the use of the user-facing or "selfie" camera. It is the responsibility of your session's XRSessionSubsystem.configurationChooser to choose the camera facing direction. You can override the configuration chooser to meet your app's needs. For more information on the ConfigurationChooser, refer to the What’s new in Unity’s AR Foundation | Unite Now 2020 video (YouTube). You can access a sample that shows how to use the ConfigurationChooser to choose between the user-facing and world-facing camera on the AR Foundation samples GitHub repository.

Face regions

The ARCore face subsystem provides face tracking methods that allow access to "regions". Regions are specific to ARCore. ARCore provides access to the following regions that define features on a face:

• Nose tip
• Forehead left
• Forehead right

Each region has a Pose associated with it. To access face regions, obtain an instance of the ARCoreFaceSubsystem using the following script:

XRFaceSubsystem faceSubsystem = ...
#if UNITY_ANDROID
var arcoreFaceSubsystem = faceSubsystem as ARCoreFaceSubsystem;
if (arcoreFaceSubsystem != null)
{
    var regionData = new NativeArray<ARCoreFaceRegionData>(0, Allocator.Temp);
    arcoreFaceSubsystem.GetRegionPoses(faceId, Allocator.Temp, ref regionData);
    using (regionData)
    {
        foreach (var data in regionData)
        {
            Debug.LogFormat("Region {0} is at {1}", data.region, data.pose);
        }
    }
}
#endif