Workflow example

The following example includes a simple script that takes an image of a handwritten digit and predicts the likelihood of the image representing a digit.

Use this example to Understand the sentis workflow.

Use the example

To use the example, follow these steps:

Attach the following script to a GameObject in your scene:

using UnityEngine;
using Unity.Sentis;

public class ClassifyHandwrittenDigit : MonoBehaviour
{
    public Texture2D inputTexture;
    public ModelAsset modelAsset;

    Model runtimeModel;
    Worker worker;
    public float[] results;

    void Start()
    {
        Model sourceModel = ModelLoader.Load(modelAsset);

        // Create a functional graph that runs the input model and then applies softmax to the output.
        FunctionalGraph graph = new FunctionalGraph();
        FunctionalTensor[] inputs = graph.AddInputs(sourceModel);
        FunctionalTensor[] outputs = Functional.Forward(sourceModel, inputs);
        FunctionalTensor softmax = Functional.Softmax(outputs[0]);

        // Create a model with softmax by compiling the functional graph.
        runtimeModel = graph.Compile(softmax);

        // Create input data as a tensor
        using Tensor inputTensor = TextureConverter.ToTensor(inputTexture, width: 28, height: 28, channels: 1);

        // Create an engine
        worker = new Worker(runtimeModel, BackendType.GPUCompute);

        // Run the model with the input data
        worker.Schedule(inputTensor);

        // Get the result
        Tensor<float> outputTensor = worker.PeekOutput() as Tensor<float>;

        // outputTensor is still pending
        // Either read back the results asynchronously or do a blocking download call
        var results = outputTensor.DownloadToArray();
    }

    void OnDisable()
    {
        // Tell the GPU we're finished with the memory the engine used
        worker.Dispose();
    }
}

Download a handwriting recognition ONNX model file from the ONNX Model Zoo (GitHub). For example, the MNIST Handwritten Digit Recognition model mnist-8.onnx.
Drag the downloaded model file into the Assets folder of the Project window.
Open the Inspector window of the GameObject and drag the model asset into the Model Asset field.
Download the digit.png image below and drag it into the Assets folder of the Project window.
Open the Inspector window of the imported image.
In Import Settings, expand the Advanced section to reveal more settings.
Set Non-Power of 2 to None and click Apply.
Open the Inspector window of the GameObject and drag the digit asset into the Input Texture field.
Click Play to run the project.

In the Inspector window of the GameObject, each item of the Results array shows how highly the model predicts the image to be a digit. For example, item 0 of the array represents how highly the model predicts the image being a handwritten zero.

Workflow example

Use the example

Additional resources