Add images to a reference image library at runtime

On supported platforms, use this workflow to create a mutable reference image library and add images at runtime.

If this is your first exposure to reference image libraries, first refer to Introduction to image tracking.

Create a runtime reference image library

When AR Foundation uses a reference image library at runtime, it converts the library from type XRReferenceImageLibrary, the Editor-friendly representation, to RuntimeReferenceImageLibrary, the runtime representation. You need a RuntimeReferenceImageLibrary to use mutable library APIs, and you can create one from an XRReferenceImageLibrary using CreateRuntimeLibrary:

void CreateRuntimeLibrary(ARTrackedImageManager m, XRReferenceImageLibrary lib)
{
    var runtimeLibrary = m.CreateRuntimeLibrary(lib);
}
Note

The ordering of the XRReferenceImages in the RuntimeReferenceImageLibrary is undefined, so it might not match the order in which the images appeared in the source XRReferenceImageLibrary.

Alternatively, to create an empty library, call CreateRuntimeLibrary without arguments:

void CreateEmptyRuntimeLibrary(ARTrackedImageManager m)
{
    var runtimeLibrary = m.CreateRuntimeLibrary();
}

Add reference images at runtime

If your target platform supports mutable libraries, any RuntimeReferenceImageLibrary is also a mutable reference library. To add images to the library, simply typecast to MutableReferenceImageLibrary as shown in the following code sample:

void AddImage(
    ARTrackedImageManager m, RuntimeReferenceImageLibrary lib, Texture2D tex)
{
    if (lib is MutableRuntimeReferenceImageLibrary mutableLibrary)
    {
        mutableLibrary.ScheduleAddImageWithValidationJob(
            tex,
            "my new image",
            0.5f /* 50 cm */);
    }
}

ScheduleAddImageWithValidationJob returns a JobHandle that you can use to determine when the job is complete, as adding an image can be computationally resource-intensive and might require a few frames. You can safely discard the job handle if you don't need to monitor the completion of the job.

Multiple add image jobs can be processed concurrently, regardless of whether image tracking is already running in your app when you schedule the job.

Note

You can't remove reference images from a reference image library at runtime. To stop tracking a particular reference image, instead use the AR Tracked Image Manager to set a different library at runtime.

Add reference images from NativeArrays

AR Foundation also supports adding reference images from a NativeArray instead of Texture2D. ScheduleAddImageJobImpl accepts a NativeSlice or pointer. Note that you are responsible for freeing the memory in your NativeArray when the job completes. You can do this by scheduling a dependent job that frees the memory:

[SerializeField]
ARTrackedImageManager m_Manager;

struct DeallocateJob : IJob
{
    [DeallocateOnJobCompletion]
    public NativeArray<byte> data;

    public void Execute() { }
}

void AddImage(NativeArray<byte> grayscaleImageBytes,
              int widthInPixels, int heightInPixels, float widthInMeters)
{
    if (m_Manager.referenceLibrary is MutableRuntimeReferenceImageLibrary lib)
    {
        var aspectRatio = (float)widthInPixels / (float)heightInPixels;
        var sizeInMeters = new Vector2(widthInMeters, widthInMeters * aspectRatio);
        var referenceImage = new XRReferenceImage(
            SerializableGuid.empty, // Guid is assigned after image is added
            SerializableGuid.empty, // We don't have a Texture2D
            sizeInMeters, "My Image", null);

        var jobState = lib.ScheduleAddImageWithValidationJob(
            grayscaleImageBytes,
            new Vector2Int(widthInPixels, heightInPixels),
            TextureFormat.R8,
            referenceImage);

        // Schedule a job that deallocates the image bytes after the image
        // is added to the reference image library.
        new DeallocateJob { data = grayscaleImageBytes }
            .Schedule(jobState.jobHandle);
    }
    else
    {
        // Cannot add the image, so dispose its memory.
        grayscaleImageBytes.Dispose();
    }
}

Texture import settings

Image detection algorithms rely on accurate knowledge of a reference image's aspect ratio. However, Unity's default texture import setting adjusts texture resolutions to the nearest powers of 2 (PoT). This can have a negative effect on tracking capabilities when tracking non-power of 2 (NPoT) images.

When you import an NPoT image with default settings and add it to a MutableRuntimeReferenceImageLibrary, you might see squashing, stretching, or scaling artifacts.

Configure texture import settings

To ensure images maintain their aspect ratio when imported:

  1. Select the images from your Assets folder to view the Import Settings in the Inspector.
  2. Under Advanced settings, change the Non-Power of 2 texture import setting to None.
Note

For textures with power-of 2 aspect ratios, Non-Power of 2 setting will not be selectable as no adjustments are necessary.