Suspends the coroutine execution until worker has completed execution on a device and contents of the specified tensor are downloaded to the main CPU memory. WaitForCompletion is not necessary and should NOT be used, unless tensor contents are accessed on CPU! WaitForCompletion can only be used with a yield statement in coroutines.

WorkerExtensions

IWorker interface extensions

WorkerFactory

Factory to create worker that executes specified model on a particular device (GPU, CPU, etc) using particular backend. See IWorker for usage of the worker itself.

Structs

Layer.DataSet

Layer param data structure

MemoryPeakSummary

High level model execution peak memory usage information

Model.Input

Input data structure

Model.Memory

Memory data structure. Used by recurrent models to store information about recurrent inputs/outputs

TensorIterator

Helper structure to iterate over tensor shape

TensorShape

TensorShape are immutable representation of a Tensor dimensions and rank. Depending on which constructor is used, the TensorShape will either be rank 8 and channels last (ie NHWC) or actual rank with unnamed tensor dimensions when using the constructor that takes int[]. With legacy use (explicit named constructors) of TensorShape an axis can be of size 1. For example, a tensor without spatial information will be N,1,1,C. With the use of TensorShape via the int[] constructor, then axes can have values of 0.

WorkerFactory.WorkerConfiguration

Worker configuration compareAgainstType if different than the worker type, the model will be run on both backend and result of every layer will be compared, checking for divergence. Great for debugging, but very slow because of the sync needed. verbose will log scheduling of layers execution to the console (default == false). compareLogLevel define how difference will be reported (default == Warning). compareEpsilon the maximum tolerance before a difference is reported (default == 0.0001f).

Interfaces

BLASPlugin

BLAS plugin interface, allows to supply platform specific implementation of matrix multiplication

IAllocatorStatistics

IDependableMemoryResource

Job system dependency fences for the memory resource

IDependableTensorData

Interface for device dependent representation of Tensor data that provides fences for scheduling data job.

IModelExecutionsReporter

Interfaces for model execution reporter

IOps

Interfaces for backend implementers see ModelBuilder.cs for detail on layers.

IOpsStatistics

ITensorAllocator

Interfaces for tensor allocator

ITensorData

Interface for device dependent representation of Tensor data.

ITensorDataStatistics

ITensorStatistics

IUniqueResource

IVars

Interfaces for variables

IVarsStatistics

IWorker

The main interface to execute neural networks (a.k.a models). IWorker abstracts implementation details associated with various hardware devices (CPU, GPU and NPU in the future) that can execute neural networks and provides clean and simple interface to: 1) specify inputs, 2) schedule the work and 3) retrieve outputs. Internally IWorker translates description of the neural network provided by Model instance into the set of operations that are sent to hardware device for execution in a non-blocking (asynchronous) manner.

The following is a simple example of image classification using pretrained neural network:

    using UnityEngine;
    using Unity.Barracuda;

    public class ImageRecognitionSample : MonoBehaviour
    {
        // small ready to use image classification neural network in ONNX format can be obtained from https://github.com/onnx/models/tree/master/vision/classification/mobilenet
        public NNModel onnxAsset;
        public Texture2D imageToRecognise;

        private IWorker worker;
        void Start()
        {
            worker = onnxAsset.CreateWorker();
        }

        void Update()
        {
            // convert texture into Tensor of shape [1, imageToRecognise.height, imageToRecognise.width, 3]
            using (var input = new Tensor(imageToRecognise, channels:3))
            {
                // execute neural network with specific input and get results back
                var output = worker.Execute(input).PeekOutput();

                // the following line will access values of the output tensor causing the main thread to block until neural network execution is done
                var indexWithHighestProbability = output.ArgMax()[0];

                UnityEngine.Debug.Log($"Image was recognised as class number: {indexWithHighestProbability}");
            }
        }

        void OnDisable()
        {
            worker.Dispose();
        }
    }

The following example demonstrates the use of coroutine to continue smooth app execution while neural network executes in the background:

    using UnityEngine;
    using Unity.Barracuda;
    using System.Collections;
    public class CoroutineImageRecognitionSample : MonoBehaviour
    {
        // small ready to use image classification neural network in ONNX format can be obtained from https://github.com/onnx/models/tree/master/vision/classification/mobilenet
        public NNModel onnxAsset;
        public Texture2D imageToRecognise;

        private IWorker worker;
        void Start()
        {
            worker = onnxAsset.CreateWorker();
            StartCoroutine(ImageRecognitionCoroutine());
        }

        IEnumerator ImageRecognitionCoroutine()
        {
            while (true)
            {
                // convert texture into Tensor of shape [1, imageToRecognise.height, imageToRecognise.width, 3]
                using (var input = new Tensor(imageToRecognise, channels:3))
                {
                    // execute neural network with specific input and get results back
                    var output = worker.Execute(input).PeekOutput();

                    // allow main thread to run until neural network execution has finished
                    yield return new WaitForCompletion(output);

                    var indexWithHighestProbability = output.ArgMax()[0];
                    UnityEngine.Debug.Log($"Image was recognised as class number: {indexWithHighestProbability}");
                }

                // wait until a new image is provided
                var previousImage = imageToRecognise;
                while (imageToRecognise == previousImage)
                   yield return null;
            }
        }

        void OnDisable()
        {
            worker.Dispose();
        }
    }

Use WorkerFactory.CreateWorker or Model.CreateWorker to create new worker instance.

Namespace Unity.Barracuda

Classes

Structs

Interfaces

Enums