Workflow example

The following example includes a simple script that takes an image of a handwritten digit and predicts the likelihood of the image representing a digit.

Use this example to Understand the sentis workflow

Use the example

To use the example, follow these steps:

Attach the following script to a GameObject in your scene:

using UnityEngine;
using Unity.Sentis;

public class ClassifyHandwrittenDigit : MonoBehaviour
{
    public Texture2D inputTexture;
    public ModelAsset modelAsset;
    Model runtimeModel;
    IWorker worker;
    public float[] results;

    void Start()
    {
        var sourceModel = ModelLoader.Load(modelAsset);

        // Create the runtime model
        runtimeModel = Functional.Compile(
            inputs =>
            {
                var output = FunctionalTensor.FromModel(sourceModel, inputs)[0];
                // rescale output of source model
                return new[] { Functional.Softmax(output) };
            },
            InputDef.FromModel(sourceModel)
        );

        // Create input data as a tensor
        using Tensor inputTensor = TextureConverter.ToTensor(inputTexture, width: 28, height: 28, channels: 1);

        // Create an engine
        worker = WorkerFactory.CreateWorker(BackendType.GPUCompute, runtimeModel);

        // Run the model with the input data
        worker.Execute(inputTensor);

        // Get the result
        using TensorFloat outputTensor = worker.PeekOutput() as TensorFloat;

        // Move the tensor data to the CPU before reading it
        outputTensor.CompleteOperationsAndDownload();

        results = outputTensor.ToReadOnlyArray();
    }

    void OnDisable()
    {
        // Tell the GPU we're finished with the memory the engine used
        worker.Dispose();
    }
}

Download a handwriting recognition ONNX model file from the ONNX Model Zoo (GitHub). For example, the MNIST Handwritten Digit Recognition model mnist-8.onnx.
Drag the downloaded model file into the Assets folder of the Project window.
Open the Inspector window of the GameObject and drag the model asset into the Model Asset field.
Download the digit.png image below and drag it into the Assets folder of the Project window.
Open the Inspector window of the imported image.
In Import Settings, expand the Advanced section to reveal more settings.
Set Non-Power of 2 to None and click Apply.
Open the Inspector window of the GameObject and drag the digit asset into the Input Texture field.
Click Play to run the project.

In the Inspector window of the GameObject, each item of the Results array shows how highly the model predicts the image to be a digit. For example, item 0 of the array represents how highly the model predicts the image being a handwritten zero.

Workflow example

Use the example

Additional resources