Tracking why operators are not covered

January 30, 2018 ยท View on GitHub

ONNX backend test script reports the coverage on the operators and attributes. But we have various of reasons for the missing test coverage on operators. This doc keeps tracking why operators are not covered by the testcases.

  • ๐Ÿ’š The ONNX operator can map to a Caffe2 operator.
  • ๐Ÿ’› The solution is not perfect/finished, for example, the operator can map to a combination of Caffe2 operators.
  • ๐Ÿ’” Hard to find a solution with existing Caffe2 operators.
OperatorTest CoveragePyTorchCaffe2
AbsYesOK๐Ÿ’šOK
AddYesOK๐Ÿ’šOK
AndSupport int tensor, but no bool tensor๐Ÿ’šOK
ArgMax๐Ÿ’”No op
ArgMin๐Ÿ’”No op
AveragePoolYesOK๐Ÿ’šOK
BatchNormalizationYesOK๐Ÿ’šOK
Cast๐Ÿ’”No op
Ceil๐Ÿ’”No op
ClipYesOK๐Ÿ’šOK
ConcatYesOK๐Ÿ’šOK
ConstantYesOK๐Ÿ’›Special handling
ConvYesOK๐Ÿ’šOK
ConvTranspose๐Ÿ’šOK
DepthToSpace๐Ÿ’›Should be BatchToSpace, no tests
DivYesOK๐Ÿ’šOK
DropoutYesOK๐Ÿ’šOK
EluYesOK๐Ÿ’šOK
EqualYesOK๐Ÿ’šOK
ExpYesOK๐Ÿ’šOK
FlattenYesOK๐Ÿ’šOK
Floor๐Ÿ’”No op
GRU๐Ÿ’›Under development
GatherYesOK๐Ÿ’›C2 only support axis=0 or 1
GemmYesOK๐Ÿ’›C2 use FC or MatMul + Add
GlobalAveragePoolYesNo direct mapping๐Ÿ’šOK
GlobalLpPool๐Ÿ’”No op
GlobalMaxPool๐Ÿ’šOK
Greater๐Ÿ’”Only support int tensor
HardSigmoid๐Ÿ’”No op
Hardmax๐Ÿ’”No op
InstanceNormalization๐Ÿ’šOK
LRNYesOK๐Ÿ’šOK
LSTM๐Ÿ’›Under development
LeakyReluYesOK๐Ÿ’šOK
Less๐Ÿ’”Only support int tensor
LogYesOK๐Ÿ’šOK
LogSoftmaxOK๐Ÿ’›No op, translated in onnx-caffe2
LpNormalization๐Ÿ’šShould be LpNorm, no tests
LpPool๐Ÿ’šShould be LpPool, no tests
MatMulYesOK๐Ÿ’šOK
MaxYesOK๐Ÿ’šOK
MaxPoolYesOK๐Ÿ’šOK
MaxRoiPool๐Ÿ’”No op
Mean๐Ÿ’”No op
MinYesOK๐Ÿ’šOK
MulYesOK๐Ÿ’šOK
NegYesOK๐Ÿ’šOK
Not๐Ÿ’šOK
Or๐Ÿ’šOK
PReluYesOK๐Ÿ’šOK
PadYesOK๐Ÿ’šOK
PowOK๐Ÿ’›Under development, C2 only accepts exponent as argument, not an input
RNN๐Ÿ’›Under development
RandomNormal๐Ÿ’”No op
RandomNormalLike๐Ÿ’”No op
RandomUniform๐Ÿ’”No op
RandomUniformLike๐Ÿ’”No op
Reciprocal๐Ÿ’›Use Pow to implement
ReduceL1๐Ÿ’”No op
ReduceL2๐Ÿ’”No op
ReduceLogSum๐Ÿ’”No op
ReduceLogSumExp๐Ÿ’”No op
ReduceMax๐Ÿ’”No op
ReduceMean๐Ÿ’”No op
ReduceMin๐Ÿ’”No op
ReduceProd๐Ÿ’”No op
ReduceSum๐Ÿ’”No op
ReduceSumSquare๐Ÿ’”No op
ReluYesOK๐Ÿ’šOK
ReshapeYesOK๐Ÿ’šOK
SeluYesOK๐Ÿ’šOK
SigmoidYesOK๐Ÿ’šOK
SliceYesOK๐Ÿ’”ScatterAssign + Cast, very hacky implementaion, Slice in C2 only supports one dimension
SoftmaxYesOK๐Ÿ’”Axis and dim has different semantics
SoftplusYesOK๐Ÿ’šOK
Softsign๐Ÿ’šOK, no tests
SpaceToDepth๐Ÿ’›Should be SpaceToBatch, no tests
SplitYesOK๐Ÿ’šOK
Sqrt๐Ÿ’›Use Pow to implement
Squeeze๐Ÿ’šOK, no tests
SubOK๐Ÿ’šOK
SumYesOK๐Ÿ’šOK
TanhYesOK๐Ÿ’šOK
Tile๐Ÿ’šOK, no tests
TransposeYesOK๐Ÿ’šOK
Xor๐Ÿ’šOK
experimental ATen๐Ÿ’šOK
experimental Affine๐Ÿ’”No op
experimental ConstantFill๐Ÿ’šOK
experimental Crop๐Ÿ’”No op
experimental FC๐Ÿ’šOK
experimental GRUUnit๐Ÿ’šOK, no tests
experimental GivenTensorFill๐Ÿ’šOK
experimental Identity๐Ÿ’šOK
experimental ImageScaler๐Ÿ’”No op
experimental MeanVarianceNormalization๐Ÿ’”No op
experimental ParametricSoftplus๐Ÿ’”No op
experimental Scale๐Ÿ’šOK
experimental ScaledTanh๐Ÿ’”No op
experimental ThresholdedRelu๐Ÿ’”No op
experimental Upsample๐Ÿ’”No bilinear