Tracking why operators are not covered
January 30, 2018 ยท View on GitHub
ONNX backend test script reports the coverage on the operators and attributes. But we have various of reasons for the missing test coverage on operators. This doc keeps tracking why operators are not covered by the testcases.
- ๐ The ONNX operator can map to a Caffe2 operator.
- ๐ The solution is not perfect/finished, for example, the operator can map to a combination of Caffe2 operators.
- ๐ Hard to find a solution with existing Caffe2 operators.
| Operator | Test Coverage | PyTorch | Caffe2 |
|---|---|---|---|
| Abs | Yes | OK | ๐OK |
| Add | Yes | OK | ๐OK |
| And | Support int tensor, but no bool tensor | ๐OK | |
| ArgMax | ๐No op | ||
| ArgMin | ๐No op | ||
| AveragePool | Yes | OK | ๐OK |
| BatchNormalization | Yes | OK | ๐OK |
| Cast | ๐No op | ||
| Ceil | ๐No op | ||
| Clip | Yes | OK | ๐OK |
| Concat | Yes | OK | ๐OK |
| Constant | Yes | OK | ๐Special handling |
| Conv | Yes | OK | ๐OK |
| ConvTranspose | ๐OK | ||
| DepthToSpace | ๐Should be BatchToSpace, no tests | ||
| Div | Yes | OK | ๐OK |
| Dropout | Yes | OK | ๐OK |
| Elu | Yes | OK | ๐OK |
| Equal | Yes | OK | ๐OK |
| Exp | Yes | OK | ๐OK |
| Flatten | Yes | OK | ๐OK |
| Floor | ๐No op | ||
| GRU | ๐Under development | ||
| Gather | Yes | OK | ๐C2 only support axis=0 or 1 |
| Gemm | Yes | OK | ๐C2 use FC or MatMul + Add |
| GlobalAveragePool | Yes | No direct mapping | ๐OK |
| GlobalLpPool | ๐No op | ||
| GlobalMaxPool | ๐OK | ||
| Greater | ๐Only support int tensor | ||
| HardSigmoid | ๐No op | ||
| Hardmax | ๐No op | ||
| InstanceNormalization | ๐OK | ||
| LRN | Yes | OK | ๐OK |
| LSTM | ๐Under development | ||
| LeakyRelu | Yes | OK | ๐OK |
| Less | ๐Only support int tensor | ||
| Log | Yes | OK | ๐OK |
| LogSoftmax | OK | ๐No op, translated in onnx-caffe2 | |
| LpNormalization | ๐Should be LpNorm, no tests | ||
| LpPool | ๐Should be LpPool, no tests | ||
| MatMul | Yes | OK | ๐OK |
| Max | Yes | OK | ๐OK |
| MaxPool | Yes | OK | ๐OK |
| MaxRoiPool | ๐No op | ||
| Mean | ๐No op | ||
| Min | Yes | OK | ๐OK |
| Mul | Yes | OK | ๐OK |
| Neg | Yes | OK | ๐OK |
| Not | ๐OK | ||
| Or | ๐OK | ||
| PRelu | Yes | OK | ๐OK |
| Pad | Yes | OK | ๐OK |
| Pow | OK | ๐Under development, C2 only accepts exponent as argument, not an input | |
| RNN | ๐Under development | ||
| RandomNormal | ๐No op | ||
| RandomNormalLike | ๐No op | ||
| RandomUniform | ๐No op | ||
| RandomUniformLike | ๐No op | ||
| Reciprocal | ๐Use Pow to implement | ||
| ReduceL1 | ๐No op | ||
| ReduceL2 | ๐No op | ||
| ReduceLogSum | ๐No op | ||
| ReduceLogSumExp | ๐No op | ||
| ReduceMax | ๐No op | ||
| ReduceMean | ๐No op | ||
| ReduceMin | ๐No op | ||
| ReduceProd | ๐No op | ||
| ReduceSum | ๐No op | ||
| ReduceSumSquare | ๐No op | ||
| Relu | Yes | OK | ๐OK |
| Reshape | Yes | OK | ๐OK |
| Selu | Yes | OK | ๐OK |
| Sigmoid | Yes | OK | ๐OK |
| Slice | Yes | OK | ๐ScatterAssign + Cast, very hacky implementaion, Slice in C2 only supports one dimension |
| Softmax | Yes | OK | ๐Axis and dim has different semantics |
| Softplus | Yes | OK | ๐OK |
| Softsign | ๐OK, no tests | ||
| SpaceToDepth | ๐Should be SpaceToBatch, no tests | ||
| Split | Yes | OK | ๐OK |
| Sqrt | ๐Use Pow to implement | ||
| Squeeze | ๐OK, no tests | ||
| Sub | OK | ๐OK | |
| Sum | Yes | OK | ๐OK |
| Tanh | Yes | OK | ๐OK |
| Tile | ๐OK, no tests | ||
| Transpose | Yes | OK | ๐OK |
| Xor | ๐OK | ||
| experimental ATen | ๐OK | ||
| experimental Affine | ๐No op | ||
| experimental ConstantFill | ๐OK | ||
| experimental Crop | ๐No op | ||
| experimental FC | ๐OK | ||
| experimental GRUUnit | ๐OK, no tests | ||
| experimental GivenTensorFill | ๐OK | ||
| experimental Identity | ๐OK | ||
| experimental ImageScaler | ๐No op | ||
| experimental MeanVarianceNormalization | ๐No op | ||
| experimental ParametricSoftplus | ๐No op | ||
| experimental Scale | ๐OK | ||
| experimental ScaledTanh | ๐No op | ||
| experimental ThresholdedRelu | ๐No op | ||
| experimental Upsample | ๐No bilinear |