Interface IWorker
The main interface to execute neural networks (a.k.a models).
IWorker abstracts implementation details associated with various hardware devices (CPU, GPU and NPU in the future)
that can execute neural networks and provides clean and simple interface to:
1) specify inputs, 2) schedule the work and 3) retrieve outputs.
Internally IWorker translates description of the neural network provided by Model instance
into the set of operations that are sent to hardware device for execution in a non-blocking (asynchronous) manner.
The following is a simple example of image classification using pretrained neural network:
using UnityEngine;
using Unity.Barracuda;
public class ImageRecognitionSample : MonoBehaviour
{
// small ready to use image classification neural network in ONNX format can be obtained from https://github.com/onnx/models/tree/master/vision/classification/mobilenet
public NNModel onnxAsset;
public Texture2D imageToRecognise;
private IWorker worker;
void Start()
{
worker = onnxAsset.CreateWorker();
}
void Update()
{
// convert texture into Tensor of shape [1, imageToRecognise.height, imageToRecognise.width, 3]
using (var input = new Tensor(imageToRecognise, channels:3))
{
// execute neural network with specific input and get results back
var output = worker.Execute(input).PeekOutput();
// the following line will access values of the output tensor causing the main thread to block until neural network execution is done
var indexWithHighestProbability = output.ArgMax()[0];
UnityEngine.Debug.Log($"Image was recognised as class number: {indexWithHighestProbability}");
}
}
void OnDisable()
{
worker.Dispose();
}
}
The following example demonstrates the use of coroutine to continue smooth app execution while neural network executes in the background:
using UnityEngine;
using Unity.Barracuda;
using System.Collections;
public class CoroutineImageRecognitionSample : MonoBehaviour
{
// small ready to use image classification neural network in ONNX format can be obtained from https://github.com/onnx/models/tree/master/vision/classification/mobilenet
public NNModel onnxAsset;
public Texture2D imageToRecognise;
private IWorker worker;
void Start()
{
worker = onnxAsset.CreateWorker();
StartCoroutine(ImageRecognitionCoroutine());
}
IEnumerator ImageRecognitionCoroutine()
{
while (true)
{
// convert texture into Tensor of shape [1, imageToRecognise.height, imageToRecognise.width, 3]
using (var input = new Tensor(imageToRecognise, channels:3))
{
// execute neural network with specific input and get results back
var output = worker.Execute(input).PeekOutput();
// allow main thread to run until neural network execution has finished
yield return new WaitForCompletion(output);
var indexWithHighestProbability = output.ArgMax()[0];
UnityEngine.Debug.Log($"Image was recognised as class number: {indexWithHighestProbability}");
}
// wait until a new image is provided
var previousImage = imageToRecognise;
while (imageToRecognise == previousImage)
yield return null;
}
}
void OnDisable()
{
worker.Dispose();
}
}
Use WorkerFactory.CreateWorker or Model.CreateWorker to create new worker instance.
Syntax
public interface IWorker : IDisposable
Properties
scheduleProgress
Reports the fraction (from 0.0 to 1.0) of the model that was scheduled for the execution since the last call to StartManualSchedule.
This property will return 0.0 immediately after calling StartManualSchedule and will return 1.0 once the complete model was scheduled.
This property will monotonuosly increase with the every iteration of IEnumerator that was obtained by calling StartManualSchedule.
Declaration
float scheduleProgress { get; }
Property Value
Methods
Execute()
Non-blocking API that schedules network execution in one go.
Declaration
Returns
Execute(IDictionary<String, Tensor>)
Non-blocking API that takes mutliple input tensors and schedules network execution in one go.
Declaration
IWorker Execute(IDictionary<string, Tensor> inputs)
Parameters
Returns
Execute(Tensor)
Non-blocking API that takes single input tensor and schedules network execution in one go.
Useful when network have only one input as input name is not needed.
Declaration
IWorker Execute(Tensor input)
Parameters
| Type |
Name |
Description |
| Tensor |
input |
|
Returns
FlushSchedule(Boolean)
Non-blocking API that starts immediate execution on the part of the network that was scheduled so far.
Optional blocking flag can force this function to block until execution is complete.
Declaration
void FlushSchedule(bool blocking = false)
Parameters
| Type |
Name |
Description |
| Boolean |
blocking |
|
PeekConstants(String)
Returns references to constants tensors for a layer. This reference might be valid only until the next Execute() or Dispose() method is called on the worker.
IMPORTANT: if you want tensor to outlive the worker, use CopyOutput() method or follow with TakeOwnership() call on the tensor, also worker Execute()
or PrepareForInput() should have been called at least once for the tensors to exist.
Declaration
Tensor[] PeekConstants(string layerName)
Parameters
| Type |
Name |
Description |
| String |
layerName |
|
Returns
PeekOutput()
Non-blocking API that returns a reference to the main output tensor. This reference will be valid only until the next Execute() or Dispose() method is called on the worker.
Useful when network has only one output.
IMPORTANT: if you want tensor to outlive the worker, use CopyOutput() method or follow with TakeOwnership() call on the tensor.
Declaration
Returns
PeekOutput(String)
Non-blocking API that returns a reference to output tensor by specified name. This reference will be valid only until the next Execute() or Dispose() method is called on the worker.
IMPORTANT: if you want tensor to outlive the worker, use CopyOutput() method or follow with TakeOwnership() call on the tensor.
Declaration
Tensor PeekOutput(string name)
Parameters
| Type |
Name |
Description |
| String |
name |
|
Returns
Optional API to prepare network execution for inputs of particular shapes.
Useful to initialize execution device ahead of the first call to Execute.
Declaration
void PrepareForInput(IDictionary<string, TensorShape> inputShapes)
Parameters
Assign tensor x to the named input of the network. String name specifies the name of the input.
Declaration
void SetInput(string name, Tensor x)
Parameters
Specify single tensor x as the only input for the network.
Useful when network has only one input and caller does not need to specify input's name.
Declaration
Parameters
| Type |
Name |
Description |
| Tensor |
x |
|
StartManualSchedule()
Non-blocking API that allows manual scheduling of the model one layer at the time.
Call MoveNext on the IEnumerator obtained from calling this function to schedule next layer of the model.
Declaration
IEnumerator StartManualSchedule()
Returns
StartManualSchedule(IDictionary<String, Tensor>)
Non-blocking API that takes mutliple input tensors and schedules network execution one layer at the time.
Call MoveNext on the IEnumerator obtained from calling this function to schedule next layer of the model.
Declaration
IEnumerator StartManualSchedule(IDictionary<string, Tensor> inputs)
Parameters
Returns
StartManualSchedule(Tensor)
Non-blocking API that takes single input tensor and schedules network execution one layer at the time.
Call MoveNext on the IEnumerator obtained from calling this function to schedule next layer of the model.
Declaration
IEnumerator StartManualSchedule(Tensor input)
Parameters
| Type |
Name |
Description |
| Tensor |
input |
|
Returns
Summary()
Returns a string summary after execution.
Declaration
Returns
Extension Methods