Supported ONNX operators
The table shows which Open Neural Network Exchange (ONNX) operators Sentis supports, and which data types Sentis supports for each back end type.
When you import a model, each ONNX operator in the model graph becomes a Sentis layer. A Sentis layer has the same name as the ONNX operator, unless the table shows the operator maps to a different layer. Refer to How Sentis optimizes a model for more information.
If you use a GPU Worker and the model has a layer that isn't supported, or has a layer that uses an input tensor data type that isn't supported, Sentis falls back to using BackendType.CPU
for the layer, which uses Burst. If BackendType.CPU
isn't supported, Sentis falls back to using a slower CPU implementation of the layer that doesn't use Burst.
Refer to the API reference for more information on each Sentis layer.
Supported operators
Ops utility methods
You can use an Ops
object to operate on tensors on a backend outside of a model. There are methods for most of the ONNX layers, and methods for operating between tensors and floats.
Method name | First argument | Second argument |
---|---|---|
Add | Tensor | float |
Add | float | Tensor |
Sub | Tensor | float |
Sub | float | Tensor |
Mul | Tensor | float |
Mul | float | Tensor |
Div | Tensor | float |
You can also use the following methods which are similar to numpy methods:
Split
, which returns a slice of a tensor along a certain axisSet
, which sets a slice of a tensor along a certain axis
ONNX operators
Name | Supported data types with BackendType.CPU |
Supported data types with BackendType.GPUCompute |
Supported data types with BackendType.GPUPixel |
Notes |
---|---|---|---|---|
Abs | float, int | float, int | float, int | |
Acos | float | float | float | |
Acosh | float | float | float | |
Add | float, int | float, int | float, int | |
And | int | int | int | |
ArgMax | float, int | float, int | float, int | |
ArgMin | float, int | float, int | float, int | |
Asin | float | float | float | |
Asinh | float | float | float | |
Atan | float | float | float | |
Atanh | float | float | float | |
AveragePool | float | float (1D and 2D only) | float (1D and 2D only) | The ceil_mode and count_include_pad parameters aren't supported. |
BatchNormalization | float | float | float | The momentum , spatial and training_mode parameters aren't supported. |
Bernoulli | float | float | float | |
Cast | float, int | float, int | float, int | |
CastLike | float, int | float, int | float, int | |
Ceil | float | float | float | |
Celu | float | float | float | |
Clip | float | float | float | |
Compress | float, int | Not supported | Not supported | |
Concat | float, int | float, int | float, int | |
Constant | - | - | - | The sparse_value parameter isn't supported. |
ConstantOfShape | float, int | float, int | float, int | |
Conv | float | float (1D, 2D and 3D) | float (1D, 2D and 3D) | |
ConvTranspose | float | float (1D, 2D and 3D) | float (1D, 2D and 3D) | The dilations , group and output_shape parameters aren't supported. |
Cos | float | float | float | |
Cosh | float | float | float | |
CumSum | float, int | float, int | float, int | |
DepthToSpace | float | float | float | |
Div | float, int | float, int | float, int | |
Dropout | - | - | - | The operator maps to the Sentis layer Identity |
Einsum | float | float (1 or 2 inputs only) | Not supported | |
Elu | float | float | float | |
Equal | float, int | float, int | float, int | |
Erf | float | float | float | |
Exp | float | float | float | |
Expand | float, int | float, int | float, int | |
Flatten | float, int | float, int | float, int | |
Floor | float | float | float | |
Gather | float, int | float, int | float, int | |
GatherElements | float, int | float, int | float, int | |
GatherND | float, int | float, int | float, int | |
Gemm | float | float | float | |
GlobalAveragePool | float | float | float | |
GlobalMaxPool | float | float | float | |
Greater | float, int | float, int | float, int | |
GreaterOrEqual | float, int | float, int | float, int | |
Hardmax | float | float | float | |
HardSigmoid | float | float | float | |
HardSwish | float | float | float | |
Identity | float, int | float, int | float, int | |
InstanceNormalization | float | float | float | |
IsInf | float | float | float (Infs not supported) | |
IsNaN | float | float | float (NaNs not supported) | |
LayerNormalization | float | float | float | |
LeakyRelu | float | float | float | |
Less | float, int | float, int | float, int | |
LessOrEqual | float, int | float, int | float, int | |
Log | float | float | float | |
LogSoftmax | float | float | float | |
LRN | float | Not supported | Not supported | |
LSTM | float | float | Not supported | |
MatMul | float | float | float | |
Max | float, int | float, int | float, int | |
MaxPool | float | float (1D and 2D only) | float (1D and 2D only) | The ceil_mode , dilations and storage_order parameters aren't supported. |
Mean | float | float | float | |
Min | float, int | float, int | float, int | |
Mod | float, int | float, int | float, int | |
Mul | float, int | float, int | float, int | |
Multinomial | float | Not supported | Not supported | |
Neg | float, int | float, int | float, int | |
NonMaxSuppression | float | Not supported | Not supported | |
NonZero | float, int | Not supported | Not supported | |
Not | int | int | int | |
OneHot | float, int | float, int | float, int | |
Or | int | int | int | |
Pad | float | float | float | |
Pow | float, int | float, int | float, int | |
PRelu | float | float | float | |
RandomNormal | float | float | float | |
RandomNormalLike | float | float | float | |
RandomUniform | float | float | float | |
RandomUniformLike | float | float | float | |
Range | float, int | float, int | float, int | |
Reciprocal | float | float | float | |
ReduceL1 | float, int | float, int | float, int | |
ReduceL2 | float | float | float | |
ReduceLogSum | float | float | float | |
ReduceLogSumExp | float | float | float | |
ReduceMax | float, int | float, int | float, int | |
ReduceMean | float | float | float | |
ReduceMin | float, int | float, int | float, int | |
ReduceProd | float, int | float, int | float, int | |
ReduceSum | float, int | float, int | float, int | |
ReduceSumSquare | float, int | float, int | float, int | |
Relu | float | float | float | |
Reshape | float, int | float, int | float, int | |
Resize | float | float (1D, 2D and 3D) | float (1D, 2D and 3D) | The cubic_coeff_a , exclude_outside , extrapolation_value and roi parameters aren't supported. |
RoiAlign | float | float | float | The coordinate_transformation_mode parameter isn't supported. |
Round | float | float | float | |
Scatter (deprecated) | float, int | float, int | float, int | The operator maps to the Sentis layer ScatterElements . |
ScatterElements | float, int | float, int (no ScatterReductionMode) | float, int | |
ScatterND | float, int | float, int | float, int | |
Selu | float | float | float | |
Shape | - | - | - | |
Shrink | float | float | float | |
Sigmoid | float | float | float | |
Sign | float, int | float, int | float, int | |
Sin | float | float | float | |
Sinh | float | float | float | |
Size | - | - | - | |
Slice | float, int | float, int | float, int | |
Softmax | float | float | float | |
Softplus | float | float | float | |
Softsign | float | float | float | |
SpaceToDepth | float | float | float | |
Split | float, int | float, int | float, int | |
Sqrt | float | float | float | |
Squeeze | float, int | float, int | float, int | |
Sub | float, int | float, int | float, int | |
Sum | float, int | float, int | float, int | |
Tan | float | float | float | |
Tanh | float | float | float | |
ThresholdedRelu | float | float | float | |
Tile | float, int | float, int | float, int | |
TopK | float | float | Not supported | |
Transpose | float, int | float, int | float, int | |
Trilu | float, int | float, int | float, int | |
Unsqueeze | float, int | float, int | float, int | |
Upsample (deprecated) | float | float (1D, 2D and 3D) | float (1D, 2D and 3D) | The operator maps to the Sentis layer Resize . |
Where | float, int | float, int | float, int | |
Xor | int | int | int |
Sentis-only layers
Sentis might create the following layers when it optimizes the model.
Name | Supported data types with BackendType.CPU |
Supported data types with BackendType.GPUCompute |
Supported data types with BackendType.GPUPixel |
---|---|---|---|
Dense | float | float | float |
Gelu | float | float | float |
MatMul2D | float | float | float |
Relu6 | float | float | float |
ScalarMad | float | float | float |
Square | float | float | float |
Swish | float | float | float |
ScaleBias | float | float | float |
Unsupported operators
The following ONNX operators aren't supported in the current version of Sentis.
- BitShift
- ConcatFromSequence
- ConvInteger
- DequantizeLinear
- Det
- DynamicQuantizeLinear
- EyeLike
- If
- GRU
- Loop
- LpPool
- MatMulInteger
- MaxUnpool
- MeanVarianceNormalization
- NegativeLogLikelihoodLoss
- Optional
- OptionalGetElement
- OptionalHasElement
- QLinearConv
- QLinearMatMul
- QuantizeLinear
- ReverseSequence
- RNN
- Scan
- SequenceAt
- SequenceConstruct
- SequenceEmpty
- SequenceErase
- SequenceInsert
- SequenceLength
- SoftmaxCrossEntropyLoss
- SplitToSequence
- StringNormalizer
- TfIdfVectorizer
- Unique