Supported ONNX operators

The table shows which Open Neural Network Exchange (ONNX) operators Sentis supports, and which data types Sentis supports for each back end type.

When you import a model, each ONNX operator in the model graph becomes a Sentis layer. A Sentis layer has the same name as the ONNX operator, unless the table shows the operator maps to a different layer. Refer to How Sentis optimizes a model for more information.

If you use a GPU Worker and the model has a layer that isn't supported, or has a layer that uses an input tensor data type that isn't supported, Sentis falls back to using BackendType.CPU for the layer, which uses Burst. If BackendType.CPU isn't supported, Sentis falls back to using a slower CPU implementation of the layer that doesn't use Burst.

Refer to the API reference for more information on each Sentis layer.

Supported operators

Ops utility methods

You can use an Ops object to operate on tensors on a backend outside of a model. There are methods for most of the ONNX layers, and methods for operating between tensors and floats.

Method name	First argument	Second argument
Add	Tensor	float
Add	float	Tensor
Sub	Tensor	float
Sub	float	Tensor
Mul	Tensor	float
Mul	float	Tensor
Div	Tensor	float

You can also use the following methods which are similar to numpy methods:

Split, which returns a slice of a tensor along a certain axis
Set, which sets a slice of a tensor along a certain axis

ONNX operators

Name	Supported data types with `BackendType.CPU`	Supported data types with `BackendType.GPUCompute`	Supported data types with `BackendType.GPUPixel`	Notes
Abs	float, int	float, int	float, int
Acos	float	float	float
Acosh	float	float	float
Add	float, int	float, int	float, int
And	int	int	int
ArgMax	float, int	float, int	float, int
ArgMin	float, int	float, int	float, int
Asin	float	float	float
Asinh	float	float	float
Atan	float	float	float
Atanh	float	float	float
AveragePool	float	float (1D and 2D only)	float (1D and 2D only)	The `ceil_mode` and `count_include_pad` parameters aren't supported.
BatchNormalization	float	float	float	The `momentum`, `spatial` and `training_mode` parameters aren't supported.
Bernoulli	float	float	float
Cast	float, int	float, int	float, int
CastLike	float, int	float, int	float, int
Ceil	float	float	float
Celu	float	float	float
Clip	float	float	float
Compress	float, int	Not supported	Not supported
Concat	float, int	float, int	float, int
Constant	-	-	-	The `sparse_value` parameter isn't supported.
ConstantOfShape	float, int	float, int	float, int
Conv	float	float (1D, 2D and 3D)	float (1D, 2D and 3D)
ConvTranspose	float	float (1D, 2D and 3D)	float (1D, 2D and 3D)	The `dilations`, `group` and `output_shape` parameters aren't supported.
Cos	float	float	float
Cosh	float	float	float
CumSum	float, int	float, int	float, int
DepthToSpace	float	float	float
Div	float, int	float, int	float, int
Dropout	-	-	-	The operator maps to the Sentis layer `Identity`
Einsum	float	float (1 or 2 inputs only)	Not supported
Elu	float	float	float
Equal	float, int	float, int	float, int
Erf	float	float	float
Exp	float	float	float
Expand	float, int	float, int	float, int
Flatten	float, int	float, int	float, int
Floor	float	float	float
Gather	float, int	float, int	float, int
GatherElements	float, int	float, int	float, int
GatherND	float, int	float, int	float, int
Gemm	float	float	float
GlobalAveragePool	float	float	float
GlobalMaxPool	float	float	float
Greater	float, int	float, int	float, int
GreaterOrEqual	float, int	float, int	float, int
Hardmax	float	float	float
HardSigmoid	float	float	float
HardSwish	float	float	float
Identity	float, int	float, int	float, int
InstanceNormalization	float	float	float
IsInf	float	float	float (Infs not supported)
IsNaN	float	float	float (NaNs not supported)
LayerNormalization	float	float	float
LeakyRelu	float	float	float
Less	float, int	float, int	float, int
LessOrEqual	float, int	float, int	float, int
Log	float	float	float
LogSoftmax	float	float	float
LRN	float	Not supported	Not supported
LSTM	float	float	Not supported
MatMul	float	float	float
Max	float, int	float, int	float, int
MaxPool	float	float (1D and 2D only)	float (1D and 2D only)	The `ceil_mode`, `dilations` and `storage_order` parameters aren't supported.
Mean	float	float	float
Min	float, int	float, int	float, int
Mod	float, int	float, int	float, int
Mul	float, int	float, int	float, int
Multinomial	float	Not supported	Not supported
Neg	float, int	float, int	float, int
NonMaxSuppression	float	Not supported	Not supported
NonZero	float, int	Not supported	Not supported
Not	int	int	int
OneHot	float, int	float, int	float, int
Or	int	int	int
Pad	float	float	float
Pow	float, int	float, int	float, int
PRelu	float	float	float
RandomNormal	float	float	float
RandomNormalLike	float	float	float
RandomUniform	float	float	float
RandomUniformLike	float	float	float
Range	float, int	float, int	float, int
Reciprocal	float	float	float
ReduceL1	float, int	float, int	float, int
ReduceL2	float	float	float
ReduceLogSum	float	float	float
ReduceLogSumExp	float	float	float
ReduceMax	float, int	float, int	float, int
ReduceMean	float	float	float
ReduceMin	float, int	float, int	float, int
ReduceProd	float, int	float, int	float, int
ReduceSum	float, int	float, int	float, int
ReduceSumSquare	float, int	float, int	float, int
Relu	float	float	float
Reshape	float, int	float, int	float, int
Resize	float	float (1D, 2D and 3D)	float (1D, 2D and 3D)	The `cubic_coeff_a`, `exclude_outside`, `extrapolation_value` and `roi` parameters aren't supported.
RoiAlign	float	float	float	The `coordinate_transformation_mode` parameter isn't supported.
Round	float	float	float
Scatter (deprecated)	float, int	float, int	float, int	The operator maps to the Sentis layer `ScatterElements`.
ScatterElements	float, int	float, int (no ScatterReductionMode)	float, int
ScatterND	float, int	float, int	float, int
Selu	float	float	float
Shape	-	-	-
Shrink	float	float	float
Sigmoid	float	float	float
Sign	float, int	float, int	float, int
Sin	float	float	float
Sinh	float	float	float
Size	-	-	-
Slice	float, int	float, int	float, int
Softmax	float	float	float
Softplus	float	float	float
Softsign	float	float	float
SpaceToDepth	float	float	float
Split	float, int	float, int	float, int
Sqrt	float	float	float
Squeeze	float, int	float, int	float, int
Sub	float, int	float, int	float, int
Sum	float, int	float, int	float, int
Tan	float	float	float
Tanh	float	float	float
ThresholdedRelu	float	float	float
Tile	float, int	float, int	float, int
TopK	float	float	Not supported
Transpose	float, int	float, int	float, int
Trilu	float, int	float, int	float, int
Unsqueeze	float, int	float, int	float, int
Upsample (deprecated)	float	float (1D, 2D and 3D)	float (1D, 2D and 3D)	The operator maps to the Sentis layer `Resize`.
Where	float, int	float, int	float, int
Xor	int	int	int

Sentis-only layers

Sentis might create the following layers when it optimizes the model.

Name	Supported data types with `BackendType.CPU`	Supported data types with `BackendType.GPUCompute`	Supported data types with `BackendType.GPUPixel`
Dense	float	float	float
Gelu	float	float	float
MatMul2D	float	float	float
Relu6	float	float	float
ScalarMad	float	float	float
Square	float	float	float
Swish	float	float	float
ScaleBias	float	float	float

Unsupported operators

The following ONNX operators aren't supported in the current version of Sentis.

BitShift
ConcatFromSequence
ConvInteger
DequantizeLinear
Det
DynamicQuantizeLinear
EyeLike
If
GRU
Loop
LpPool
MatMulInteger
MaxUnpool
MeanVarianceNormalization
NegativeLogLikelihoodLoss
Optional
OptionalGetElement
OptionalHasElement
QLinearConv
QLinearMatMul
QuantizeLinear
ReverseSequence
RNN
Scan
SequenceAt
SequenceConstruct
SequenceEmpty
SequenceErase
SequenceInsert
SequenceLength
SoftmaxCrossEntropyLoss
SplitToSequence
StringNormalizer
TfIdfVectorizer
Unique