benchmark

December 27, 2021 ยท View on GitHub

environment

Hardware environment:

  • GPU: V100 * 8
  • CPU: Intel Xeon

Software environment:

  • Ptyhon 3.7
  • PyTorch 1.3
  • CUDA 10.0

Image Classification on ImageNet

MethodModel NameAccuracyParas(M)
EfficientNetB076.83
B482.8
B8 67285.788
B8 83285.888
DARTS-73.30-
AmeobaNet-A-83.90-
ProxylessNAS-75.10-
StacNAS-76.78-

Image Classification on Cifar-10

MethodModel Name#Paras(M)Accuracy
CARSCARS-A1.40295.92
CARS-B1.69796.58
CARS-C1.91396.74
CARS-D2.22597.05
CARS-E2.40897.25
CARS-F3.76797.30
CARS-G4.37797.38
CARS-H4.50697.43
DARTS-3.3097.24
NSGANet-3.3097.25
SNASAggressive2.3096.90
Mild2.9097.02
AmeobaNet-A-3.1096.88
ProxylessNAS-5.7097.92
StacNAS-3.9097.98

Detection on CULane

MethodModel NameFLOPs(G)ParamsF1 Score
AutoLaneCULane-S2.094.5771.5
CULane-M8.546.674.6
CULane-L2.087.3275.2
SCNN-328.4-71.6
SAD-162.2-71.8
PointLane-25.1-70.2

Super-Resolution on Set5

MethodModel NameModel Size/MFlops/GPSNRSSIM
ESR-EAESRN-V-11.3240.61637.790.9566
ESRN-V-21.3140.2137.840.9569
ESRN-V-31.3141.67637.790.9570
ESRN-V-41.3540.1737.830.9567
SR_EAM2Mx2-A3.20196.2738.060.9588
M2Mx2-B0.6135.0337.730.9562
M2Mx2-C0.2413.4937.560.9556
SRCNN--52.736.660.9524
CARN-M--91.237.530.9583
FALSR-B-0.3274.7037.610.9585

Super-Resolution on Set14

MethodModel NameModel Size/MFlops/GPSNRSSIM
ESR-EAESRN-V-11.3240.61633.370.8887
ESRN-V-21.3140.2133.370.8911
ESRN-V-31.3141.67633.350.8878
ESRN-V-41.3540.1733.350.8902
SR_EAM2Mx2-A3.20196.2733.650.8943
M2Mx2-B0.6135.0333.320.8870
M2Mx2-C0.2413.4933.130.8829
SRCNN--52.732.420.9063
CARN-M--91.233.260.9141
FALSR-B-0.3274.7033.290.9143

Super-Resolution on B100

MethodModel NameModel Size/MFlops/GPSNRSSIM
ESR-EAESRN-V-11.3240.61632.090.8802
ESRN-V-21.3140.2132.080.8810
ESRN-V-31.3141.67632.050.8789
ESRN-V-41.3540.1732.060.8810
SR_EAM2Mx2-A3.20196.2732.200.8842
M2Mx2-B0.6135.0332.000.8989
M2Mx2-C0.2413.4931.890.8783
SRCNN--52.731.260.8879
CARN-M--91.231.920.8960
FALSR-B-0.3274.7031.970.8967

Super-Resolution on Urban100

MethodModel NameModel Size/MFlops/GPSNRSSIM
ESR-EAESRN-V-11.3240.61631.650.8814
ESRN-V-21.3140.2131.690.8829
ESRN-V-31.3141.67631.470.8803
ESRN-V-41.3540.1731.580.8814
SR_EAM2Mx2-A3.20196.2732.200.8948
M2Mx2-B0.6135.0331.370.8796
M2Mx2-C0.2413.4930.920.8717
SRCNN--52.729.500.8946
CARN-M--91.231.230.9144
FALSR-B-0.3274.7031.280.9191

Segmentation on VOC2012

MethodModel NameModel Size/MFlops/GParams/KmIOU
Adelaide_EA-10.60.57843822.140.7602
MV2 + LW RefineNet--0.9241630.7313

Click-Through Rate Prediction on Avazu

MethodModel NameModel Size/MAccuracy
auto_group-1110.790
auto_fis-5000.788
FM-1110.7793
DeepFM-1110.7836