Namespace Unity.Barracuda

Classes

Asset Importer Editor of NNModel (the serialized file generated by ONNXModelImporter)

ONNXModelImporter

Asset Importer for Open Neural Network Exchange (ONNX) files. For more information about ONNX file format see: https://github.com/onnx/onnx

RecurrentState

Object that represent memory (recurrent state) between the executions of a given model.

Suspends the coroutine execution until worker has completed execution on a device and contents of the specified tensor are downloaded to the main CPU memory. WaitForCompletion is not necessary and should NOT be used, unless tensor contents are accessed on CPU! WaitForCompletion can only be used with a yield statement in coroutines.

WorkerExtensions

WorkerFactory

Factory to create worker that executes specified model on a particular device (GPU, CPU, etc) using particular backend. See IWorker for usage of the worker itself.

Structs

ComputeDebugUtils.KernelAssertInfo

ComputeFunc

ComputeFunc.TensorDecl

Layer.DataSet

Model.Input

Model.Memory

ONNXTensor

TensorShape

TensorShape are immutable representation of a Tensor dimensions and rank. At the moment a TensorShape is always of rank 4 and channels last ie NHWC. However an axis can be of size 1. For example a tensor without spatial information will be N,1,1,C

VariableTensor

WorkerFactory.WorkerConfiguration

Worker configuration compareAgainstType if different than the worker type, the model will be run on both backend and result of every layer will be compared, checking for divergence. Great for debugging, but very slow because of the sync needed. verbose will log scheduling of layers execution to the console (default == false). compareLogLevel define how difference will be reported (default == Warning). compareEpsilon the maximum tolerance before a difference is reported (default == 0.0001f).

Interfaces

BLASPlugin

IDependableTensorData

Interface for device dependent representation of Tensor data that provides a read fence for scheduling data consumer job.

IOps

Interfaces for backend implementers see ModelBuilder.cs for detail on layers.

ITensorAllocator

Interfaces for tensor allocator

ITensorData

Interface for device dependent representation of Tensor data.

IVars

Interfaces for variables

IWorker

The main interface to execute neural networks (a.k.a models). IWorker abstracts implementation details associated with various hardware devices (CPU, GPU and NPU in the future) that can execute neural networks and provides clean and simple interface to: 1) specify inputs, 2) schedule the work and 3) retrieve outputs. Internally IWorker translates description of the neural network provided by Model instance into the set of operations that are sent to hardware device for execution in a non-blocking (asynchronous) manner.

The following is a simple example of image classification using pretrained neural network:

    using UnityEngine;
    using Unity.Barracuda;

    public class ImageRecognitionSample : MonoBehaviour
    {
        // small ready to use image classification neural network in ONNX format can be obtained from https://github.com/onnx/models/tree/master/vision/classification/mobilenet
        public NNModel onnxAsset;
        public Texture2D imageToRecognise;

        private IWorker worker;
        void Start()
        {
            worker = onnxAsset.CreateWorker();
        }

        void Update()
        {
            // convert texture into Tensor of shape [1, imageToRecognise.height, imageToRecognise.width, 3]
            using (var input = new Tensor(imageToRecognise, channels:3))
            {
                // execute neural network with specific input and get results back
                var output = worker.Execute(input).PeekOutput();

                // the following line will access values of the output tensor causing the main thread to block until neural network execution is done
                var indexWithHighestProbability = output.ArgMax()[0];

                UnityEngine.Debug.Log($"Image was recognised as class number: {indexWithHighestProbability}");
            }
        }

        void OnDisable()
        {
            worker.Dispose();
        }
    }

The following example demonstrates the use of coroutine to continue smooth app execution while neural network executes in the background:

    using UnityEngine;
    using Unity.Barracuda;
    using System.Collections;
    public class CoroutineImageRecognitionSample : MonoBehaviour
    {
        // small ready to use image classification neural network in ONNX format can be obtained from https://github.com/onnx/models/tree/master/vision/classification/mobilenet
        public NNModel onnxAsset;
        public Texture2D imageToRecognise;

        private IWorker worker;
        void Start()
        {
            worker = onnxAsset.CreateWorker();
            StartCoroutine(ImageRecognitionCoroutine());
        }

        IEnumerator ImageRecognitionCoroutine()
        {
            while (true)
            {
                // convert texture into Tensor of shape [1, imageToRecognise.height, imageToRecognise.width, 3]
                using (var input = new Tensor(imageToRecognise, channels:3))
                {
                    // execute neural network with specific input and get results back
                    var output = worker.Execute(input).PeekOutput();

                    // allow main thread to run until neural network execution has finished
                    yield return new WaitForCompletion(output);

                    var indexWithHighestProbability = output.ArgMax()[0];
                    UnityEngine.Debug.Log($"Image was recognised as class number: {indexWithHighestProbability}");
                }

                // wait until a new image is provided
                var previousImage = imageToRecognise;
                while (imageToRecognise == previousImage)
                   yield return null;
            }
        }

        void OnDisable()
        {
            worker.Dispose();
        }
    }

Use WorkerFactory.CreateWorker or Model.CreateWorker to create new worker instance.

Namespace Unity.Barracuda

Classes

Structs

Interfaces

Enums