Access tensor data directly

To avoid having to do a slow readback of a tensor from a device when you want to access a tensor, or when you need to pass a tensor between multiple models, you can read from and write to the tensor underlying native data directly instead.

Refer to Tensor fundamentals in Sentis for more information about how Sentis stores tensor data.

Check where the data for a tensor is stored

Use the dataOnBackend.backendType property of a tensor to check where the tensor data is stored. The property is either CPU, GPUPixel or GPUCompute.

For example:

using UnityEngine;
using Unity.Sentis;

public class CheckTensorLocation : MonoBehaviour
{

    public Texture2D inputTexture;
    
    void Start()
    {
        // Create input data as a tensor
        Tensor inputTensor = TextureConverter.ToTensor(inputTexture);

        // Check if the tensor is stored in CPU or GPU memory, and write to the Console window.
        Debug.Log(inputTensor.dataOnBackend.backendType);
    }
}

If you want to force a tensor to the other device, use the following:

For example:

// Create a tensor
TensorFloat inputTensor = TensorFloat.AllocZeros(new TensorShape(1, 3, 2, 2));

// Force the tensor into GPU memory
ComputeTensorData computeTensorData = ComputeTensorData.Pin(inputTensor);

Note:

  • If the tensor data is already on the device you force it to, the method is a passthrough.
  • If not the previous data will be disposed and new memory will be allocated on the target backend.

Access a tensor in GPU memory

To access a tensor in GPU-compute memory, first get the tensor data as a ComputeTensorData by using ComputeTensorData.Pin.

You can then use the buffer property of the ComputeTensorData object to access the tensor data in the compute buffer directly. Refer to ComputeBuffer in the Unity API reference for more information about how to access a compute buffer.

Refer to the Read output asynchronously example in the sample scripts for an example.

Access a tensor in CPU memory

To access a tensor in CPU memory, first get the tensor data as a BurstTensorData object by using BurstTensorData.Pin.

You can then use the object in a Burst function like IJobParallelFor to read from and write to the tensor data. You can also use the read and write fence (fence and reuse respectively) properties of the object to handle Burst job depedencies.

Refer to the following:

You can then use methods in the NativeTensorArray class to read from and write to the tensor data as a native array.

Additional resources