Camera
This page is a supplement to the AR Foundation Camera manual. The following sections only contain information about APIs where ARKit exhibits unique platform-specific behavior.
Tip
When developing an AR app, refer to both the AR Foundation documentation as well as the required packages for each platform you support.
Optional feature support
ARKit implements the following optional features of AR Foundation's XRCameraSubsystem:
Feature | Descriptor Property | Supported |
---|---|---|
Brightness | supportsAverageBrightness | |
Color temperature | supportsAverageColorTemperature | Yes |
Color correction | supportsColorCorrection | |
Display matrix | supportsDisplayMatrix | Yes |
Projection matrix | supportsProjectionMatrix | Yes |
Timestamp | supportsTimestamp | Yes |
Camera configuration | supportsCameraConfigurations | Yes |
Camera image | supportsCameraImage | Yes |
Average intensity in lumens | supportsAverageIntensityInLumens | Yes |
Focus modes | supportsFocusModes | Yes |
Face tracking ambient intensity light estimation | supportsFaceTrackingAmbientIntensityLightEstimation | Yes |
Face tracking HDR light estimation | supportsFaceTrackingHDRLightEstimation | Yes |
World tracking ambient intensity light estimation | supportsWorldTrackingAmbientIntensityLightEstimation | Yes |
World tracking HDR light estimation | supportsWorldTrackingHDRLightEstimation | |
Camera grain | supportsCameraGrain | iOS 13+ |
Image stabilization | supportsImageStabilization | |
Exif data | supportsExifData | iOS 16+ |
Note
Refer to AR Foundation Camera platform support for more information on the optional features of the camera subsystem.
Light estimation
ARKit light estimation can only be enabled
or disabled
. The availability of either Ambient Intensity
or Environmental HDR
data is governed by the active tracking mode. See the following table for more details.
Tracking configuration | Ambient intensity (lumens) | Color temperature | Main light direction | Main light intensity (lumens) | Ambient spherical harmonics |
---|---|---|---|---|---|
World Tracking | Yes | Yes | No | No | No |
Face Tracking | Yes | Yes | Yes | Yes | Yes |
Camera configuration
XRCameraConfiguration contains an IntPtr
field nativeConfigurationHandle
which is a platform-specific handle. For ARKit, this handle is a pointer to the native ARVideoFormat Objective-C object.
Advanced camera hardware configuration
On supported devices with iOS 16 or newer, you can manually configure advanced camera hardware properties such as exposure. This is useful in situations where you want more control over the camera.
Note
In addition to iOS 16 or newer, advanced camera hardware configuration also requires a device with an ultra-wide camera. Most iOS devices starting with iPhone 11 have an ultra-wide camera. For more device-specific information you can check Apple's Tech Specs.
To configure the camera's advanced hardware properties, you must first lock the camera for configuration using ARKitCameraSubsystem.TryGetLockedCamera. If this method returns true
, you can configure the device's advanced camera hardware properties using the ARKitLockedCamera instance passed as its out argument.
The following code samples demonstrate how to configure advanced camera hardware properties:
Check Support
The following example method checks whether the ARKitCameraSubsystem is available and whether it supports the advanced camera configuration feature. This method is used by the other code examples on this page.
bool AdvancedConfigurationSupported(out ARKitCameraSubsystem subsystem)
{
// This is inefficient. You should re-use a saved reference instead.
var cameraManager = FindAnyObjectByType<ARCameraManager>();
// check if arkit subsystem is available
subsystem = cameraManager.subsystem as ARKitCameraSubsystem;
if (subsystem == null)
{
Debug.LogError("Advanced camera configuration requires ARKit.");
return false;
}
// check whether the device supports advanced camera configuration
if (!subsystem.advancedCameraConfigurationSupported)
{
Debug.LogError("Advanced camera configuration is not supported on this device.");
return false;
}
return true;
}
Exposure
The following example method tries to lock the camera and, if successful, sets the exposure.
void UpdateCameraExposure()
{
if (!AdvancedConfigurationSupported(out ARKitCameraSubsystem subsystem))
return;
// try to get a locked camera
if (!subsystem.TryGetLockedCamera(out var lockedCamera))
{
Debug.LogError("Unable to lock the camera for advanced camera configuration.");
return;
}
// using statement will automatically dispose the locked camera
using (lockedCamera)
{
// set the exposure
const double duration = 0.1f;
const float iso = 500f;
lockedCamera.exposure = new ARKitExposure(duration, iso);
}
}
White Balance
The following example method tries to lock the camera and, if successful, sets the white balance.
void UpdateCameraWhiteBalance()
{
if (!AdvancedConfigurationSupported(out ARKitCameraSubsystem subsystem))
return;
// try to get a locked camera
if (!subsystem.TryGetLockedCamera(out var lockedCamera))
{
Debug.LogError("Unable to lock the camera for advanced camera configuration.");
return;
}
// using statement will automatically dispose the locked camera
using (lockedCamera)
{
// set the white balance
const float blueGain = 2.0f;
const float greenGain = 1.0f;
const float redGain = 1.5f;
lockedCamera.whiteBalance = new ARKitWhiteBalanceGains(blueGain, greenGain, redGain);
}
}
Focus
The following example method tries to lock the camera and, if successful, sets the focus.
void UpdateCameraFocus()
{
if (!AdvancedConfigurationSupported(out ARKitCameraSubsystem subsystem))
return;
// try to get a locked camera
if (!subsystem.TryGetLockedCamera(out var lockedCamera))
{
Debug.LogError("Unable to lock the camera for advanced camera configuration.");
return;
}
// using statement will automatically dispose the locked camera
using (lockedCamera)
{
// set the focus
const float lensPosition = 2.0f;
lockedCamera.focus = new ARKitFocus(lensPosition);
}
}
High resolution CPU image
You can asynchronously capture a high resolution XRCpuImage (or simply, CPU Image) using ARKitCameraSubsystem.TryAcquireHighResolutionCpuImage on iOS 16 and newer.
The example below demonstrates a coroutine to set up and handle the asynchronous request:
IEnumerator CaptureHighResolutionCpuImage()
{
// This is inefficient. You should re-use a saved reference instead.
var cameraManager = FindAnyObjectByType<ARCameraManager>();
if (cameraManager.subsystem is not ARKitCameraSubsystem subsystem)
{
Debug.LogError("High resolution CPU image capture requires ARKit.");
yield break;
}
// Yield return on the promise returned by the ARKitCameraSubsystem
var promise = subsystem.TryAcquireHighResolutionCpuImage();
yield return promise;
// If the promise was not successful, check your Console logs for more
// information about the error.
if (!promise.result.wasSuccessful)
yield break;
// If the promise was successful, handle the result.
DoSomethingWithHighResolutionCpuImage(promise.result.highResolutionCpuImage);
promise.result.highResolutionCpuImage.Dispose();
}
Whenever you successfully acquire a high resolution CPU image, you should Dispose it as soon as possible, as CPU images require native memory resources. If you retain too many high-resolution images, ARKit can be prevented from rendering new frames.
For a complete usage example, see the AR Foundation Samples repository.
Select a resolution
The exact resolution of the high resolution CPU image you receive depends on your camera manager's currentConfiguration. For the highest resolution capture, choose a configuration with a non-binned video format such as 4K resolution (3840x2160).
For more information on binned vs non-binned video formats, see Apple's Discover ARKit 6 video, which explains the ARKit camera architecture in greater detail.
EXIF data
You are able to access camera frame's EXIF data on devices running iOS 16 or newer.
For more information, refer to the EXIF specification.
Apple and ARKit are trademarks of Apple Inc., registered in the U.S. and other countries and regions.