Transcoders

A transcoder node gets GPU data as input, enqueues command to process it into a command buffer, and outputs the GPU data, transformed by a compute shader. It's a base class you can use to implement your own transformations.

You pass the command buffer to the transcoder node using the inTranscode input. Once you've added your commands to the command buffer, you output it to outTranscode to allow further nodes access to the command buffer.

Use an intermediate base classes

SensorSDK provides helper base classes for transcoders manipulating specific data types on the GPU:

• PhotosensorTranscoder: a transcoder taking photosensor data as input.
• PhotosensorToTextureV2: a transcoder taking photosensor data as input and outputting render textures.
• RenderTextureTranscoder: a transcoder having render textures as input and output.

Use existing SensorSDK transcoders

SensorSDK provides the following transcoders:

• ACESToneMapping
• AzimuthCorrection
• BlueFilter
• CorrectPointCloudMotion
• DemosaickingBoxFilter
• GreenFilter
• IntensityBufferToTexture
• LuminanceFilter
• LutMapping
• MosaickingFilter
• ParallelStereo
• PhotosensorToInstanceSegmentation v2
• PhotosensorToIntensity v2
• PhotosensorToIntensityBuffer v2
• PhotosensorToPointCloud v2
• PhotosensorToRangeNode v2
• PhotosensorToSemanticSegmentationNode v2
• PointCloudToDepth
• RedFilter
• SegmentationAsBuffer
• ShotNoise
• WhiteBalance

Create your own transcoder

To create your own transcoder, determine first the types of input and output you will use, and whether there is an existing intermediate base class that covers that case. If so, your transcoder should inherit from that class.

For example, to create a transcoder that modify an image coming from a photosensor array, use render textures for both the input and output types, and inherit your class from RenderTextureTranscoder.

Then, find a transcoder provided by SensorSDK that uses the same base class, and use it as template to create your own. To follow up on the previous example, use the RedFilter class as template.

Here is the implementation of a transcoder inspired from the RedFilter class.

using UnityEngine;
using UnityEngine.Rendering;
using UnityEngine.Rendering.HighDefinition;
using Mechatronics.SystemGraph;
using Mechatronics.SensorSDK;

[NodeCategory("My transcoders", "My filter", NodeTick.Asynchronous)]
public class MyFilter : RenderTextureTranscoder
{
    public override void Enable(Scheduler.ClockState clockState)
    {
        // Register a callback to Transcode() for change events on the inTranscode port.
        EnableInterfaceOnChangeEvent();

        // Load a compute shader with the filename MyTranscoderShader from the Resources
        // directory. Load the kernel named MyFilterKernel inside that shader.
        _computeShader = new ComputeShaderInstance("MyTranscoderShader", "MyFilterKernel");
    }

    public override void Disable()
    {
        // Unregister callback and release all GPU resources.
        DisableInterfaceOnChangeEvent();
        Release();
    }

    protected override void Transcode(CustomPassContext ctx)
    {
        // Ensure we have all it takes to run the shader kernel.
        RenderTexture inBuffer = inRenderTexture.Read;
        if (_computeShader == null || _computeShader.kernelIdx == -1 || inBuffer == null)
            return;

        // Ensure the output buffer is large enough to contain the transcoded data.
        // In this example, we choose to output a float texture.
        PrepareOutputBuffer(ctx.hdCamera.actualWidth, ctx.hdCamera.actualHeight,
            RenderTextureFormat.RFloat);

        // Enqueue all kernel parameters in the command buffer, dispatch the call to the GPU.
        CommandBuffer cmd = ctx.cmd;
        cmd.SetComputeTextureParam(_computeShader, ShaderIDs.inBuffer, inBuffer);
        cmd.SetComputeTextureParam(_computeShader, ShaderIDs.outBuffer, _outputBuffer);
        cmd.SetComputeIntParam(_computeShader, ShaderIDs.frameWidth, _outputBuffer.width);
        cmd.SetComputeIntParam(_computeShader, ShaderIDs.frameHeight, _outputBuffer.height);
        cmd.DispatchCompute(_computeShader,
            ComputeShaderInstance.DivRoundUp(_outputBuffer.width,
                                             ShaderIDs.kThreadGroupWidth),
            ComputeShaderInstance.DivRoundUp(_outputBuffer.height,
                                             ShaderIDs.kThreadGroupHeight), 1);

        // Write to the outputs, outTranscode last.
        // This will call other transcoders down the chain.
        outRenderTexture.Write = _outputBuffer;
        outTranscode.Write = inTranscode.Read;
    }

    private static class ShaderIDs
    {
        public static readonly int inBuffer = Shader.PropertyToID("inBuffer");
        public static readonly int outBuffer = Shader.PropertyToID("outBuffer");
        public static readonly int frameWidth = Shader.PropertyToID("frameWidth");
        public static readonly int frameHeight = Shader.PropertyToID("frameHeight");

        public const int kThreadGroupWidth = 32;
        public const int kThreadGroupHeight = 32;
    }
}