Diffusion Time Estimation
March 19, 2024 ยท View on GitHub
Official implementation of On Diffusion Modeling for Anomaly Detection from The Twelfth International Conference on Learning Representations (ICLR 2024). Includes the code for diffusion-based models for tabular data and image datasets with Diffusion Time Estimation (DTE) and multiple other baselines.
Citation
@inproceedings{
livernoche2024on,
title={On Diffusion Modeling for Anomaly Detection},
author={Victor Livernoche and Vineet Jain and Yashar Hezaveh and Siamak Ravanbakhsh},
booktitle={The Twelfth International Conference on Learning Representations},
year={2024},
url={https://openreview.net/forum?id=lR3rk7ysXz}
}
Setup Instructions
1. Install the required packages
You will need to install ADBench and torchvision for this project. Python needs to be version 3.8+. ADBench has already most of the dependencies needed for the project.
To install all packages, run the following command:
pip install -r requirements.txt
After this is done, as of right now there is a dependency issue with ADBench and PyOD, so you need to install manually PyoD with:
pip install --upgrade PyOD
Note: this is not necessary to run only the diffusion models. You would only needs:
- numpy
- torch
- scikit-learn
- scipy
2. Download the ADBench Datasets
You will have to download all the necessary datasets to reproduce the results. This is not necessary if you do not want to train on ADBench datasets. The diffusion methods and other implemented baselines do not rely on it. You can do this by running the following command:
from adbench.myutils import Utils
utils = Utils() # utility function
utils.download_datasets(repo='github')
Run the ADBench experiments
To reproduce the results from the paper uses run.py with the argument for the supervision setting to use.
For unsupervised, use:
python run.py --setting unsup --seed 0
For semi-supervised, use:
python run.py --setting semi --seed 0
We ran the seeds 0,1,2,3,4 for the experiments and report the average and standard deviations over thoses seeds.
Run the image embeddings experiments
For VisA, you first need to follow the data preprocessing from https://github.com/amazon-science/spot-diff/ with a split type of 1cls. Place the folder "/VisA_pytorch" in the main directory. To create the embeddings, you can select the model wanted to create the embedding with either resnet34 or vicreg and run:
python vision/data/visa_preprocess.py --model resnet34
For CIFAR10 and MNIST, to get the ResNet-34 pre-trained classification embeddings, run python vision/data/preprocess.py.
For CIFAR10 with vicreg pre-trained embeddings, you first need to download the weights available here, found at https://github.com/augustwester/vicreg, weights provided by August Wester (2023) (c). Then, run python vision/data/preprocess.py --model vicreg and python vision/data/preprocess.py --model resnet34 keep the file "checkpoint.pt" in the main directory.
You can now run the experiment using python run_embeddings.py
Run the image experiments
As in the previous section, place the folder "/VisA_pytorch" in the main directory after having followed the preprocessing at https://github.com/amazon-science/spot-diff/. Then simply run python run_images.py
Using the models on new datasets
To run using a new dataset, here is an example on thyroid of ADBench. Note the importance of Standard Scaling for the diffusion-based models; we found that it is crucial since the added noise assumes that each feature is centered at 0 and, as we use gaussian noise, having standard scaling helps the noise to cover the whole space for anomaly detection.
import numpy as np
import sklearn.metrics as skm
from sklearn.preprocessing import StandardScaler
from diffusion.DTE import DTECategorical
import os
import pkg_resources
DATA_PATH = pkg_resources.resource_filename('adbench', 'datasets/')
dataset = "38_thyroid"
data = np.load(os.path.join(DATA_PATH, 'Classical', dataset + '.npz'), allow_pickle=True)
X = data['X']
y = data['y']
test_size = 0.5 # number of normal samples to go in the test set
# split the data to have the normal data as training set (semi-supervised setting)
indices = np.arange(len(X))
normal_indices = indices[y == 0]
anomaly_indices = indices[y == 1]
train_size = round((1-test_size) * normal_indices.size)
train_indices, test_indices = normal_indices[:train_size], normal_indices[train_size:]
test_indices = np.append(test_indices, anomaly_indices)
X_train = X[train_indices]
y_train = y[train_indices]
X_test = X[test_indices]
y_test = y[test_indices]
# Standard scaling
scaler = StandardScaler().fit(X_train)
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
model = DTECategorical()
model.fit(X_train)
scores = model.predict_score(X_test)
auc_roc = skm.roc_auc_score(y_test, scores)
print(auc_roc)
Settings and Results
For the semi-supervised setting, we use 0.5 of the normal data in the training set, and the rest is in the test set with anomalies. For the unsupervised setting, we sample the whole dataset with replacement for the training data, while the test data is the whole dataset. This bootstrapping method allows us to test the variance over the training dataset for each method.
The total amount of compute required to reproduce our experiments with five seeds, including all of the baselines and the proposed DTE model amounts to 473 GPU-hours for the unsupervised setting and 225 GPU-hours for the semi-supervised setting on an RTX8000 GPU with 48 gigabytes of memory. There are a few models that are running on CPU only, which account for a few hours of runtime. Here are figures of the results on ADBench of all the methods. The first figure is for the semi-supervised setting, the second is for the unsupervised setting.
Figure 1: Mean AUC ROC with standard deviation over 5 seeds for the semi-supervised setting.
Figure 2: Mean AUC ROC with standard deviation over 5 seeds for the unsupervised setting.
Table 1: AUC ROC with standard deviation over 5 seeds for the semi-supervised setting.
| Dataset | CBLOF | COPOD | ECOD | FeatureBagging | HBOS | IForest | KNN | LODA | LOF | MCD | OCSVM | PCA | DAGMM | DeepSVDD | DROCC | GOAD | ICL | PlanarFlow | VAE | GANomaly | SLAD | DIF | DDPM | DTE-NP | DTE-IG | DTE-C |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aloi | 53.69(0.15) | 49.51(0.0) | 51.73(0.0) | 49.07(0.53) | 52.23(0.0) | 50.74(0.68) | 51.04(0.0) | 49.24(2.75) | 48.76(0.0) | 48.54(0.35) | 54.29(0.0) | 54.04(0.0) | 50.84(2.97) | 50.89(2.05) | 50.0(0.0) | 48.01(0.92) | 47.5(0.98) | 48.52(2.34) | 54.04(0.0) | 53.94(1.65) | 50.76(0.29) | 51.1(0.0) | 49.91(0.35) | 51.19(0.46) | 50.87(1.17) | 50.44(0.19) |
| amazon | 58.17(0.12) | 56.78(0.0) | 53.79(0.0) | 57.94(0.04) | 56.32(0.0) | 56.4(0.95) | 60.58(0.0) | 52.23(2.84) | 57.88(0.0) | 60.36(0.09) | 56.48(0.0) | 54.9(0.0) | 50.47(2.01) | 51.2(4.42) | 50.0(0.0) | 56.07(0.9) | 54.21(0.22) | 49.94(2.12) | 54.9(0.0) | 53.42(0.94) | 52.01(0.06) | 51.43(0.34) | 55.09(0.1) | 60.82(0.0) | 51.93(3.5) | 56.71(2.15) |
| annthyroid | 90.14(1.08) | 76.77(0.0) | 78.45(0.0) | 88.95(1.61) | 66.02(0.0) | 90.28(1.52) | 92.81(0.0) | 77.35(7.51) | 88.63(0.0) | 90.19(0.02) | 88.45(0.0) | 85.19(0.0) | 72.23(15.13) | 55.01(3.62) | 88.9(2.31) | 81.01(5.16) | 81.11(1.07) | 93.19(2.14) | 85.44(0.0) | 67.52(5.54) | 93.34(0.36) | 88.36(0.0) | 88.82(1.34) | 92.9(0.25) | 87.6(4.46) | 97.52(0.15) |
| backdoor | 69.65(5.23) | 50.0(0.0) | 50.0(0.0) | 94.83(0.62) | 70.81(0.89) | 74.89(2.67) | 93.75(0.53) | 47.62(22.63) | 95.33(0.25) | 85.13(8.87) | 62.52(0.64) | 64.57(0.65) | 54.36(19.91) | 91.14(2.62) | 94.25(0.73) | 52.9(14.48) | 93.62(0.69) | 76.03(11.56) | 64.67(0.74) | 87.17(1.43) | 50.0(0.0) | 83.73(0.76) | 80.93(0.56) | 93.31(1.7) | 94.02(1.46) | 91.65(1.73) |
| breastw | 99.11(0.23) | 99.46(0.09) | 99.14(0.22) | 59.07(15.44) | 99.23(0.17) | 99.5(0.08) | 99.05(0.26) | 98.13(0.35) | 88.91(6.8) | 98.66(0.65) | 99.39(0.16) | 99.21(0.17) | 89.53(10.2) | 96.96(0.92) | 47.32(31.95) | 98.86(0.33) | 98.28(0.45) | 97.93(0.82) | 99.22(0.18) | 94.75(2.73) | 99.53(0.13) | 56.34(11.9) | 98.7(0.43) | 99.28(0.12) | 78.65(11.1) | 92.78(1.75) |
| campaign | 77.05(0.31) | 78.15(0.0) | 76.86(0.0) | 69.1(3.85) | 77.06(0.0) | 73.64(1.48) | 78.48(0.0) | 58.88(4.48) | 70.55(0.0) | 78.51(0.81) | 77.67(0.0) | 77.07(0.0) | 61.47(2.73) | 62.21(12.88) | 50.0(0.0) | 47.89(12.32) | 80.92(0.79) | 69.75(3.75) | 77.07(0.0) | 69.21(3.28) | 76.75(0.16) | 57.87(0.0) | 74.51(0.42) | 78.79(0.25) | 74.81(1.69) | 77.95(1.11) |
| cardio | 93.49(1.47) | 93.16(0.0) | 94.95(0.0) | 92.12(0.56) | 80.7(0.0) | 93.32(1.43) | 92.0(0.0) | 91.34(2.93) | 92.21(0.0) | 82.82(0.85) | 95.61(0.0) | 96.54(0.01) | 77.92(8.85) | 65.43(4.37) | 62.14(23.76) | 96.01(0.27) | 80.01(2.12) | 88.9(0.93) | 96.55(0.0) | 86.06(4.31) | 83.05(1.1) | 68.25(0.0) | 86.94(1.96) | 91.8(0.59) | 73.79(14.09) | 87.26(1.0) |
| cardiotocography | 67.61(2.21) | 66.35(0.0) | 79.3(0.0) | 63.64(2.4) | 61.24(0.0) | 74.24(2.88) | 62.11(0.0) | 72.79(7.6) | 64.49(0.0) | 57.11(1.41) | 75.22(0.0) | 78.89(0.0) | 67.11(9.03) | 47.75(8.59) | 45.98(16.75) | 76.06(1.4) | 54.2(1.8) | 69.88(5.65) | 78.9(0.0) | 62.77(8.62) | 47.32(0.31) | 41.79(0.0) | 54.54(2.96) | 63.76(1.88) | 52.44(3.67) | 60.13(2.59) |
| celeba | 79.28(1.3) | 75.72(0.59) | 76.33(0.63) | 46.89(1.8) | 76.68(0.66) | 71.23(2.27) | 73.14(0.72) | 62.46(13.17) | 43.73(0.75) | 84.37(2.34) | 79.79(0.77) | 80.53(0.7) | 63.81(4.28) | 56.17(22.46) | 68.89(1.11) | 43.8(10.49) | 72.21(0.82) | 71.64(7.91) | 80.32(0.48) | 52.25(11.79) | 67.43(1.64) | 66.69(3.55) | 78.56(1.99) | 70.4(0.37) | 74.51(2.39) | 82.18(2.38) |
| census | 70.84(0.28) | 50.0(0.0) | 50.0(0.0) | 55.92(1.0) | 62.5(0.47) | 62.55(2.38) | 72.26(0.29) | 51.12(11.19) | 58.46(1.06) | 74.14(1.94) | 70.02(0.21) | 70.51(0.21) | 52.24(1.19) | 54.16(4.33) | 55.36(3.62) | 35.24(4.19) | 70.56(0.35) | 59.33(2.89) | 70.52(0.22) | 68.07(3.66) | 57.91(10.84) | 61.45(2.04) | 70.15(0.23) | 72.1(0.4) | 61.79(4.94) | 69.62(0.91) |
| cover | 94.04(0.28) | 88.2(0.27) | 91.86(0.21) | 99.16(0.63) | 71.11(0.82) | 86.31(2.07) | 97.54(0.15) | 94.93(3.07) | 99.18(0.1) | 70.02(0.66) | 96.17(0.11) | 94.41(0.14) | 75.94(14.06) | 49.12(14.74) | 95.79(0.69) | 13.83(13.34) | 89.34(4.02) | 47.52(8.02) | 94.35(0.15) | 76.35(19.11) | 73.97(13.8) | 57.69(5.24) | 98.35(0.66) | 97.73(0.55) | 95.83(1.55) | 97.76(1.28) |
| donors | 93.47(0.23) | 81.5(0.21) | 88.74(0.38) | 95.21(1.72) | 81.19(0.61) | 89.44(2.22) | 99.49(0.06) | 63.52(27.27) | 96.97(0.24) | 81.93(10.78) | 92.09(0.22) | 88.12(0.57) | 62.15(16.19) | 72.95(17.81) | 74.18(22.09) | 33.57(16.0) | 99.9(0.05) | 91.64(3.52) | 88.6(0.25) | 75.34(11.69) | 88.51(5.51) | 90.04(1.79) | 82.5(1.83) | 99.26(0.29) | 99.25(0.6) | 98.15(0.37) |
| fault | 59.0(1.29) | 49.14(0.0) | 50.37(0.0) | 48.32(0.95) | 53.06(0.0) | 55.86(2.03) | 58.73(0.0) | 50.27(1.88) | 47.42(0.0) | 59.44(3.79) | 57.21(0.0) | 55.87(0.0) | 52.85(7.19) | 54.31(1.62) | 55.73(5.33) | 58.89(0.61) | 60.63(0.5) | 57.51(5.15) | 55.87(0.0) | 59.5(5.14) | 63.93(0.19) | 62.31(0.0) | 61.09(1.16) | 58.64(0.65) | 59.42(1.46) | 59.46(1.45) |
| fraud | 94.91(1.08) | 94.3(1.41) | 94.89(1.27) | 94.83(1.56) | 95.02(0.67) | 94.73(1.23) | 95.43(1.04) | 89.05(8.12) | 94.35(1.41) | 91.1(1.79) | 95.61(0.69) | 95.38(0.68) | 85.33(6.76) | 83.13(6.6) | 50.0(0.0) | 69.75(21.3) | 92.78(1.46) | 90.72(2.26) | 95.47(0.75) | 93.25(2.85) | 94.58(1.01) | 82.58(2.27) | 93.65(0.92) | 95.64(1.05) | 90.79(3.24) | 93.52(1.53) |
| glass | 89.35(1.48) | 76.0(1.94) | 71.14(3.54) | 88.49(1.71) | 82.59(3.23) | 81.09(2.56) | 92.04(1.12) | 67.34(5.39) | 88.82(1.98) | 79.71(1.37) | 69.73(5.89) | 73.44(2.22) | 65.33(15.53) | 83.67(16.2) | 64.89(23.64) | 59.03(12.64) | 99.44(0.58) | 85.33(6.2) | 72.55(1.66) | 79.77(8.72) | 86.04(5.69) | 96.45(1.39) | 66.67(13.63) | 89.64(3.54) | 98.53(0.86) | 92.42(2.3) |
| hepatitis | 86.26(2.35) | 80.9(1.22) | 73.84(1.99) | 67.76(6.5) | 84.84(0.78) | 82.69(2.75) | 96.46(1.46) | 68.98(3.97) | 66.92(7.01) | 80.64(4.23) | 90.58(1.79) | 84.48(2.29) | 70.22(6.66) | 99.57(0.24) | 51.8(17.93) | 84.5(3.25) | 99.94(0.13) | 95.8(1.67) | 84.84(2.27) | 87.39(7.55) | 99.93(0.15) | 96.07(2.32) | 97.74(1.18) | 93.22(3.9) | 99.93(0.15) | 98.78(0.88) |
| http | 99.93(0.01) | 99.19(0.09) | 97.95(0.12) | 92.1(0.57) | 98.58(1.04) | 99.35(0.29) | 100.0(0.0) | 47.72(45.49) | 99.98(0.03) | 99.95(0.01) | 100.0(0.0) | 99.95(0.01) | 91.78(17.91) | 61.31(51.49) | 50.0(0.0) | 99.68(0.13) | 98.24(3.45) | 99.38(0.08) | 99.94(0.01) | 50.14(34.85) | 99.91(0.08) | 99.36(0.07) | 100.0(0.0) | 99.98(0.03) | 80.72(43.08) | 99.45(0.1) |
| imdb | 49.94(0.01) | 51.05(0.0) | 46.88(0.0) | 49.53(0.1) | 49.94(0.0) | 49.53(0.78) | 50.08(0.0) | 47.23(2.24) | 49.57(0.0) | 51.24(0.18) | 48.72(0.0) | 47.97(0.0) | 48.6(0.38) | 49.97(5.71) | 51.35(2.05) | 48.46(0.65) | 52.34(0.49) | 49.23(2.75) | 47.97(0.0) | 51.58(0.71) | 51.26(0.1) | 51.4(0.49) | 47.91(0.1) | 50.43(0.0) | 50.97(3.41) | 48.05(2.22) |
| internetads | 65.16(0.08) | 65.94(0.0) | 66.01(0.0) | 71.38(2.32) | 49.18(0.0) | 47.87(2.11) | 68.08(0.0) | 58.73(3.84) | 71.72(0.0) | 47.73(0.01) | 65.63(0.0) | 65.12(0.0) | 49.47(5.05) | 72.96(3.24) | 53.4(7.55) | 65.65(0.21) | 72.2(0.55) | 70.87(0.84) | 65.12(0.0) | 69.86(0.23) | 75.94(0.11) | 49.33(0.34) | 65.76(0.06) | 69.96(2.22) | 71.52(3.89) | 77.57(1.54) |
| ionosphere | 96.78(1.5) | 78.32(2.13) | 71.77(1.43) | 94.47(2.1) | 70.68(2.85) | 91.21(1.37) | 97.44(0.98) | 85.56(3.59) | 94.29(2.2) | 95.4(0.64) | 96.32(0.93) | 89.11(1.31) | 73.95(5.98) | 97.2(1.26) | 61.14(28.54) | 91.54(3.05) | 98.98(0.32) | 96.86(1.21) | 89.76(1.2) | 93.89(2.03) | 98.21(0.6) | 93.59(2.06) | 94.6(0.85) | 97.77(1.39) | 95.15(3.6) | 95.42(0.58) |
| landsat | 57.21(0.26) | 49.29(0.0) | 42.01(0.0) | 66.38(0.17) | 73.21(0.0) | 58.8(2.21) | 68.25(0.0) | 44.65(3.3) | 66.58(0.0) | 56.78(6.08) | 47.98(0.0) | 43.9(0.0) | 56.27(3.46) | 59.44(1.41) | 53.86(2.57) | 40.52(2.32) | 65.13(0.44) | 50.85(2.13) | 54.22(9.45) | 55.34(10.03) | 65.03(0.18) | 56.56(0.0) | 51.37(1.0) | 68.2(1.75) | 44.72(5.52) | 52.79(1.63) |
| letter | 33.24(0.66) | 36.53(0.0) | 45.37(0.0) | 44.84(1.0) | 35.91(0.0) | 32.04(1.64) | 35.43(0.0) | 30.2(0.94) | 44.83(0.0) | 31.47(4.16) | 32.17(0.0) | 30.3(0.0) | 38.97(8.38) | 36.4(3.05) | 55.26(11.04) | 31.08(0.55) | 42.68(1.17) | 38.73(3.53) | 30.23(0.0) | 34.02(0.74) | 36.8(0.41) | 74.07(0.0) | 38.05(1.12) | 34.38(0.98) | 39.86(2.29) | 36.72(0.95) |
| lymphography | 99.83(0.02) | 99.53(0.2) | 99.52(0.15) | 96.61(2.84) | 99.69(0.17) | 99.45(0.32) | 99.93(0.08) | 67.04(13.87) | 98.21(0.75) | 98.88(0.55) | 100.0(0.0) | 99.86(0.05) | 94.94(3.85) | 99.73(0.3) | 32.42(37.75) | 99.89(0.08) | 100.0(0.0) | 99.58(0.51) | 99.88(0.09) | 99.09(0.86) | 100.0(0.01) | 99.84(0.22) | 99.94(0.09) | 99.93(0.09) | 99.98(0.05) | 98.99(0.4) |
| magic.gamma | 75.81(0.0) | 68.0(0.0) | 63.58(0.0) | 84.19(0.72) | 74.53(0.0) | 77.09(1.29) | 83.27(0.0) | 70.53(1.36) | 83.4(0.0) | 73.67(0.12) | 74.25(0.0) | 70.64(0.0) | 59.23(4.32) | 62.97(1.07) | 78.83(0.66) | 69.46(2.39) | 75.56(0.42) | 74.12(2.75) | 70.64(0.0) | 59.18(1.6) | 72.0(0.01) | 63.86(0.0) | 85.97(1.08) | 83.57(0.76) | 86.46(1.12) | 87.5(0.9) |
| mammography | 84.74(0.02) | 90.59(0.0) | 90.67(0.0) | 86.31(0.35) | 85.01(0.0) | 88.02(0.3) | 87.58(0.0) | 89.62(0.87) | 85.52(0.0) | 72.87(0.64) | 88.63(0.0) | 89.93(0.0) | 76.03(14.65) | 71.5(7.4) | 81.82(1.94) | 69.94(8.59) | 71.87(9.11) | 78.93(5.71) | 89.58(0.17) | 85.54(7.43) | 74.51(0.67) | 73.87(0.2) | 81.01(2.04) | 87.62(0.09) | 84.64(3.47) | 86.42(1.72) |
| mnist | 91.1(0.23) | 50.0(0.0) | 50.0(0.0) | 92.55(0.4) | 62.34(0.0) | 86.6(1.99) | 93.85(0.0) | 64.74(7.74) | 92.93(0.0) | 88.3(1.03) | 90.56(0.0) | 90.21(0.0) | 72.19(7.16) | 66.37(11.03) | 83.13(1.64) | 90.07(0.35) | 90.11(1.13) | 81.9(2.67) | 90.21(0.0) | 77.9(6.44) | 89.73(0.44) | 50.21(0.0) | 87.27(3.22) | 94.02(0.42) | 80.78(5.91) | 87.43(2.48) |
| musk | 100.0(0.0) | 99.71(0.0) | 99.87(0.0) | 100.0(0.0) | 100.0(0.0) | 90.58(6.2) | 100.0(0.0) | 99.67(0.35) | 100.0(0.0) | 93.91(2.55) | 100.0(0.0) | 100.0(0.0) | 95.01(4.27) | 99.99(0.01) | 32.99(33.18) | 100.0(0.0) | 99.37(0.63) | 76.65(18.72) | 100.0(0.0) | 100.0(0.0) | 100.0(0.0) | 97.76(0.0) | 100.0(0.0) | 100.0(0.0) | 94.22(12.93) | 100.0(0.0) |
| optdigits | 83.52(1.69) | 50.0(0.0) | 50.0(0.0) | 96.27(0.49) | 89.92(0.0) | 81.07(3.28) | 93.72(0.0) | 32.77(8.62) | 96.65(0.0) | 64.86(0.92) | 63.38(0.0) | 58.17(0.0) | 40.04(20.45) | 39.45(18.65) | 85.25(2.9) | 67.46(4.94) | 97.18(0.8) | 34.12(8.29) | 58.17(0.0) | 74.3(12.58) | 95.28(0.17) | 48.64(0.0) | 90.76(2.12) | 94.28(1.67) | 79.81(9.35) | 82.38(2.68) |
| pageblocks | 91.23(0.11) | 80.85(0.0) | 87.95(0.0) | 91.11(0.3) | 65.62(0.0) | 82.64(0.89) | 89.65(0.0) | 83.62(2.6) | 91.3(0.0) | 87.07(0.02) | 88.58(0.0) | 86.12(0.0) | 82.8(10.44) | 78.39(1.53) | 92.33(1.18) | 88.05(1.19) | 88.39(0.77) | 84.85(1.19) | 86.16(0.0) | 72.83(9.94) | 87.86(0.01) | 87.39(0.0) | 86.93(0.42) | 89.32(0.26) | 85.66(1.77) | 89.89(0.55) |
| pendigits | 96.67(0.07) | 90.74(0.0) | 92.95(0.0) | 99.5(0.11) | 93.55(0.0) | 97.22(0.48) | 99.87(0.0) | 92.13(1.02) | 99.05(0.0) | 83.69(0.09) | 96.36(0.0) | 94.37(0.0) | 56.49(21.83) | 46.29(11.35) | 75.91(13.38) | 89.97(2.03) | 96.71(0.83) | 83.45(6.13) | 94.5(0.0) | 67.93(21.6) | 94.61(0.41) | 89.71(0.0) | 98.11(0.23) | 99.61(0.17) | 96.96(1.63) | 97.79(0.68) |
| pima | 72.94(1.07) | 66.59(1.32) | 60.56(1.57) | 71.93(2.22) | 74.76(1.6) | 74.26(1.63) | 76.94(1.87) | 62.68(7.55) | 70.53(2.19) | 73.64(1.37) | 71.53(1.79) | 72.28(1.99) | 54.54(6.0) | 57.99(2.85) | 47.53(16.52) | 62.31(13.7) | 79.68(3.06) | 72.17(2.71) | 73.18(1.93) | 60.45(3.01) | 60.62(4.21) | 55.19(6.92) | 70.27(2.28) | 81.5(2.57) | 68.59(3.93) | 69.88(1.95) |
| satellite | 73.2(0.92) | 68.34(0.0) | 62.22(0.0) | 80.11(0.11) | 85.5(0.0) | 77.46(1.48) | 82.24(0.0) | 69.73(1.15) | 80.3(0.0) | 72.76(3.68) | 73.91(0.0) | 66.63(0.0) | 72.79(2.01) | 76.19(2.66) | 73.38(4.44) | 68.76(0.92) | 85.15(0.48) | 72.3(1.73) | 74.14(0.25) | 79.87(0.56) | 87.49(0.1) | 66.91(0.0) | 77.7(0.53) | 82.11(0.67) | 76.52(2.68) | 78.61(0.7) |
| satimage-2 | 99.42(0.01) | 97.92(0.0) | 97.09(0.0) | 99.47(0.03) | 97.95(0.0) | 99.12(0.21) | 99.71(0.0) | 98.67(0.52) | 99.38(0.0) | 99.92(0.0) | 99.61(0.0) | 98.17(0.0) | 91.82(4.43) | 92.94(2.84) | 99.22(0.42) | 98.99(0.06) | 99.48(0.22) | 96.67(0.57) | 98.98(0.04) | 97.57(1.19) | 99.77(0.0) | 86.9(0.0) | 99.62(0.16) | 99.67(0.0) | 95.34(1.84) | 99.34(0.07) |
| shuttle | 99.72(0.02) | 99.47(0.0) | 99.33(0.0) | 86.89(8.22) | 98.64(0.0) | 99.65(0.07) | 99.91(0.0) | 71.68(33.88) | 99.98(0.0) | 98.98(0.0) | 99.62(0.0) | 99.36(0.0) | 84.58(18.68) | 99.79(0.07) | 50.0(0.0) | 70.44(16.28) | 99.92(0.04) | 86.5(5.68) | 99.35(0.0) | 97.53(0.74) | 99.9(0.01) | 99.76(0.0) | 99.91(0.01) | 99.93(0.02) | 99.86(0.12) | 99.75(0.0) |
| skin | 91.82(0.21) | 47.21(0.18) | 49.14(0.18) | 78.39(0.91) | 76.92(0.35) | 89.42(0.58) | 99.49(0.08) | 75.51(5.43) | 86.34(1.77) | 88.37(0.26) | 90.25(0.24) | 59.73(0.31) | 67.91(30.02) | 59.95(4.47) | 89.48(1.07) | 64.95(2.25) | 6.58(0.63) | 91.27(7.09) | 66.05(0.19) | 48.48(2.87) | 91.05(2.61) | 87.55(0.76) | 88.74(4.4) | 98.86(0.46) | 98.71(1.13) | 91.77(0.22) |
| smtp | 87.28(5.65) | 91.15(1.58) | 88.26(2.46) | 84.81(3.66) | 82.75(5.31) | 90.36(2.13) | 92.43(2.7) | 73.03(6.65) | 93.42(2.48) | 94.87(0.84) | 84.65(4.49) | 81.81(7.32) | 87.08(5.31) | 85.24(6.6) | 57.13(15.93) | 78.78(13.12) | 74.36(7.07) | 84.22(6.88) | 81.93(5.67) | 54.55(5.98) | 92.15(1.87) | 95.51(1.36) | 95.43(1.26) | 92.98(2.91) | 81.64(9.87) | 95.27(1.28) |
| spambase | 81.52(0.55) | 72.09(0.0) | 68.83(0.0) | 69.64(2.09) | 77.88(0.0) | 85.18(1.69) | 83.36(0.0) | 72.39(6.9) | 73.23(0.0) | 80.69(3.02) | 81.7(0.0) | 81.4(0.0) | 69.41(4.42) | 70.24(5.02) | 75.37(4.45) | 81.78(0.36) | 83.53(0.45) | 82.26(3.35) | 81.4(0.0) | 82.57(1.34) | 84.86(0.2) | 41.31(0.0) | 64.54(0.8) | 83.74(0.69) | 77.5(3.37) | 83.01(0.41) |
| speech | 35.88(0.15) | 37.03(0.0) | 35.96(0.0) | 37.48(0.37) | 36.66(0.0) | 37.7(1.72) | 36.36(0.0) | 38.02(2.67) | 37.53(0.0) | 38.81(0.36) | 36.57(0.0) | 36.38(0.0) | 50.66(3.9) | 48.88(2.88) | 48.96(2.21) | 36.63(1.14) | 48.86(2.72) | 48.56(4.54) | 36.38(0.0) | 38.7(3.43) | 41.37(0.94) | 52.29(3.42) | 36.96(0.86) | 41.37(0.0) | 39.57(1.54) | 38.17(0.57) |
| stamps | 93.41(1.66) | 93.14(0.4) | 87.62(0.91) | 94.21(2.26) | 91.8(1.0) | 93.47(1.42) | 95.89(1.44) | 91.93(3.55) | 93.74(2.36) | 84.93(2.01) | 93.72(1.74) | 92.7(1.68) | 80.11(11.56) | 71.09(3.68) | 50.15(22.93) | 81.46(15.36) | 96.68(1.09) | 87.3(10.69) | 93.28(1.28) | 66.2(7.84) | 81.97(8.29) | 84.92(4.03) | 91.84(4.15) | 97.87(0.37) | 93.38(3.31) | 91.6(2.04) |
| thyroid | 98.54(0.06) | 93.81(0.0) | 97.55(0.0) | 93.15(1.52) | 98.65(0.0) | 98.96(0.2) | 98.68(0.0) | 96.06(1.65) | 92.72(0.0) | 98.49(0.01) | 98.56(0.0) | 98.55(0.0) | 91.08(6.68) | 88.77(3.69) | 94.96(1.55) | 95.15(0.85) | 95.4(0.98) | 98.42(0.51) | 98.55(0.0) | 94.78(3.22) | 95.27(0.24) | 96.29(0.0) | 97.95(0.22) | 98.63(0.04) | 89.43(11.73) | 98.74(0.14) |
| vertebral | 54.43(1.97) | 26.34(2.49) | 41.95(4.78) | 64.13(2.15) | 40.09(3.97) | 45.64(3.83) | 57.67(3.58) | 31.66(5.28) | 64.3(1.31) | 47.1(1.82) | 50.47(2.23) | 42.08(3.49) | 50.6(11.93) | 44.79(1.71) | 43.84(24.28) | 46.73(8.74) | 79.19(5.06) | 49.78(7.82) | 42.63(2.75) | 50.67(10.71) | 44.96(9.25) | 57.21(3.04) | 70.67(6.28) | 54.3(15.47) | 74.63(7.58) | 66.41(1.9) |
| vowels | 78.72(4.01) | 52.82(0.0) | 61.47(0.0) | 85.32(1.16) | 53.31(0.0) | 61.83(0.62) | 82.21(0.0) | 55.52(8.1) | 86.3(0.0) | 27.66(0.28) | 75.91(0.0) | 52.29(0.0) | 42.55(11.56) | 55.73(4.43) | 54.74(21.32) | 68.49(3.71) | 85.1(2.13) | 54.59(10.55) | 52.12(0.02) | 63.11(9.82) | 85.02(0.02) | 88.48(0.0) | 86.38(1.94) | 81.42(1.44) | 85.68(3.16) | 86.93(2.25) |
| waveform | 72.93(0.9) | 72.36(0.0) | 59.44(0.0) | 76.95(1.07) | 69.28(0.0) | 72.29(1.52) | 75.21(0.0) | 60.96(5.08) | 76.0(0.0) | 58.39(0.03) | 70.44(0.0) | 64.68(0.0) | 51.89(7.39) | 59.94(2.58) | 67.7(5.0) | 64.99(3.1) | 68.68(3.66) | 64.8(2.42) | 64.84(0.0) | 75.95(7.58) | 48.92(0.08) | 50.63(0.0) | 62.17(2.64) | 74.48(0.0) | 73.68(2.68) | 65.21(1.05) |
| wbc | 98.31(1.31) | 99.4(0.25) | 99.39(0.25) | 58.05(18.08) | 99.0(0.41) | 99.41(0.37) | 99.12(0.19) | 97.86(1.59) | 80.52(4.47) | 98.88(1.02) | 99.63(0.19) | 99.35(0.13) | 86.76(15.33) | 91.44(3.64) | 44.24(28.21) | 99.14(0.2) | 99.66(0.36) | 95.97(0.81) | 99.25(0.27) | 95.91(3.0) | 99.78(0.18) | 97.46(1.82) | 99.2(0.39) | 99.54(0.28) | 90.98(11.64) | 80.53(7.15) |
| wdbc | 98.73(0.43) | 99.18(0.21) | 96.72(0.71) | 99.63(0.18) | 98.55(0.28) | 98.73(0.64) | 99.05(0.27) | 96.98(2.04) | 99.62(0.21) | 97.03(0.47) | 99.34(0.23) | 99.14(0.22) | 73.78(25.27) | 99.31(0.4) | 40.08(33.97) | 98.96(0.25) | 99.78(0.27) | 98.86(0.76) | 99.14(0.36) | 96.22(3.1) | 99.49(0.25) | 68.03(7.24) | 99.3(0.17) | 99.52(0.38) | 99.6(0.31) | 98.48(0.77) |
| wilt | 42.9(1.14) | 32.09(0.0) | 37.48(0.0) | 73.35(10.5) | 39.1(0.0) | 47.97(3.12) | 63.66(0.0) | 41.1(7.0) | 68.81(0.0) | 81.72(0.01) | 34.81(0.0) | 26.07(0.0) | 41.81(7.29) | 34.41(1.7) | 49.49(12.62) | 51.38(5.33) | 76.42(3.46) | 74.62(4.18) | 35.41(0.0) | 44.01(5.48) | 61.76(0.11) | 55.02(0.0) | 71.66(0.81) | 62.91(5.59) | 93.75(3.23) | 85.1(1.13) |
| wine | 97.75(0.59) | 86.37(4.37) | 73.86(5.42) | 97.93(0.87) | 95.63(2.63) | 93.92(1.79) | 99.19(0.22) | 90.94(4.75) | 98.36(0.38) | 97.28(1.8) | 97.82(0.45) | 93.79(1.58) | 66.17(39.61) | 92.16(4.34) | 43.8(32.33) | 94.11(1.86) | 99.87(0.29) | 95.38(2.4) | 94.25(1.47) | 74.32(28.64) | 100.0(0.0) | 91.09(2.25) | 99.61(0.09) | 99.44(0.97) | 99.95(0.11) | 99.97(0.04) |
| wpbc | 59.57(1.98) | 52.33(2.93) | 49.5(2.47) | 56.79(1.86) | 60.91(2.67) | 56.33(2.72) | 63.67(2.34) | 51.32(3.74) | 57.36(2.01) | 63.36(0.93) | 53.37(2.44) | 52.54(2.3) | 46.99(2.56) | 82.67(5.47) | 43.78(4.34) | 51.39(5.74) | 96.61(1.17) | 57.46(2.93) | 54.42(2.76) | 60.11(5.48) | 95.53(2.21) | 82.51(3.26) | 66.49(3.17) | 83.16(13.46) | 70.71(9.06) | 68.88(2.7) |
| yeast | 50.4(0.08) | 38.88(0.0) | 44.64(0.0) | 46.4(1.33) | 42.88(0.0) | 41.8(0.75) | 44.74(0.0) | 46.51(5.8) | 45.79(0.0) | 43.05(0.1) | 44.84(0.0) | 43.24(0.0) | 51.03(3.92) | 47.62(5.96) | 48.42(5.46) | 52.53(3.88) | 48.98(2.39) | 45.07(3.33) | 42.39(0.01) | 47.64(6.7) | 48.69(0.11) | 38.44(0.0) | 49.13(2.85) | 44.58(0.33) | 48.58(2.89) | 47.08(1.1) |
| yelp | 63.8(0.07) | 60.21(0.0) | 57.39(0.0) | 67.06(0.12) | 59.95(0.0) | 61.07(0.53) | 68.07(0.0) | 56.26(3.78) | 67.2(0.0) | 66.15(0.04) | 62.08(0.0) | 59.16(0.0) | 49.87(1.25) | 49.9(3.3) | 50.67(1.15) | 61.07(0.98) | 55.79(0.5) | 53.6(2.03) | 59.14(0.04) | 56.11(1.6) | 54.88(0.19) | 48.36(0.23) | 59.3(0.09) | 68.66(0.0) | 57.32(4.3) | 59.94(5.32) |
| MNIST-C | 81.11(0.09) | 50.0(0.0) | 50.0(0.0) | 87.33(0.15) | 70.43(0.0) | 76.75(1.45) | 84.11(0.0) | 69.39(5.31) | 87.21(0.0) | 75.26(1.24) | 79.55(0.0) | 78.35(0.0) | 63.71(6.37) | 64.7(4.6) | 57.2(11.1) | 79.34(0.41) | 85.62(0.52) | 71.21(1.5) | 78.35(0.01) | 80.08(1.16) | 83.51(0.11) | 54.05(2.27) | 80.12(0.14) | 84.74(0.0) | 79.92(3.93) | 86.1(0.84) |
| FashionMNIST | 89.1(0.15) | 50.0(0.0) | 50.0(0.0) | 91.67(0.11) | 75.42(0.0) | 84.15(1.07) | 89.87(0.0) | 79.28(4.1) | 91.6(0.0) | 84.37(1.11) | 88.16(0.0) | 87.6(0.0) | 70.8(4.89) | 75.45(2.22) | 51.58(14.44) | 88.03(0.24) | 90.56(0.27) | 82.19(0.96) | 87.6(0.0) | 89.43(0.52) | 89.94(0.04) | 65.53(1.77) | 88.52(0.07) | 90.14(0.0) | 84.32(2.44) | 90.21(0.55) |
| CIFAR10 | 67.87(0.21) | 54.98(0.0) | 56.88(0.0) | 70.31(0.21) | 57.89(0.0) | 64.04(0.93) | 67.53(0.0) | 61.62(4.35) | 70.3(0.0) | 65.15(0.58) | 67.79(0.0) | 67.42(0.0) | 53.97(2.96) | 56.12(2.2) | 49.63(3.46) | 67.53(0.72) | 63.6(0.83) | 62.75(1.09) | 67.42(0.0) | 67.19(0.84) | 66.6(0.1) | 52.06(1.59) | 67.91(0.13) | 67.82(0.0) | 62.4(3.33) | 68.53(1.59) |
| SVHN | 60.97(0.17) | 50.0(0.0) | 50.0(0.0) | 63.93(0.14) | 54.65(0.0) | 58.99(0.88) | 61.69(0.0) | 54.5(4.02) | 63.82(0.0) | 58.87(0.65) | 61.25(0.0) | 60.79(0.0) | 53.36(2.12) | 53.92(3.05) | 49.99(2.47) | 60.75(0.49) | 61.7(0.5) | 58.87(0.87) | 60.79(0.0) | 61.25(0.77) | 60.87(0.07) | 52.98(0.67) | 61.37(0.08) | 62.13(0.0) | 59.19(2.5) | 62.91(1.1) |
| MVTec-AD | 79.96(2.02) | 50.0(0.0) | 50.0(0.0) | 80.45(2.2) | 75.97(1.84) | 77.38(1.95) | 81.51(1.82) | 72.31(3.37) | 80.36(2.14) | 86.75(2.12) | 77.44(2.03) | 76.37(1.91) | 64.69(5.63) | 89.55(2.19) | 60.52(11.42) | 77.06(2.1) | 94.75(0.84) | 72.8(2.39) | 76.21(1.81) | 81.1(1.8) | 93.24(1.09) | 81.99(2.78) | 78.0(1.99) | 89.66(1.51) | 85.88(3.63) | 89.39(2.0) |
| 20news | 57.1(1.23) | 52.92(0.43) | 54.11(0.24) | 60.25(0.7) | 53.57(0.35) | 54.91(1.17) | 57.36(0.66) | 53.45(4.0) | 60.19(0.62) | 62.85(1.63) | 56.25(0.62) | 54.39(0.41) | 51.48(4.08) | 55.64(4.17) | 52.77(4.09) | 54.92(1.04) | 61.3(1.05) | 51.59(3.0) | 54.59(0.74) | 59.68(1.87) | 59.47(0.8) | 53.18(2.56) | 54.88(0.52) | 59.97(0.74) | 58.28(6.15) | 64.3(3.17) |
| agnews | 62.82(0.08) | 55.05(0.0) | 55.11(0.0) | 74.64(0.09) | 55.69(0.0) | 58.43(1.09) | 67.05(0.0) | 56.95(3.54) | 74.58(0.0) | 67.98(0.25) | 60.64(0.0) | 56.9(0.0) | 51.03(3.6) | 49.83(5.63) | 49.65(0.76) | 59.87(0.86) | 62.55(0.47) | 50.15(1.19) | 56.9(0.0) | 58.63(1.14) | 58.43(0.07) | 52.02(0.89) | 57.82(0.12) | 67.98(0.0) | 56.55(4.15) | 68.16(3.22) |
Table 2: AUC ROC with standard deviation over 5 seeds for the unsupervised setting.
| Dataset | CBLOF | COPOD | ECOD | FeatureBagging | HBOS | IForest | KNN | LODA | LOF | MCD | OCSVM | PCA | DAGMM | DeepSVDD | DROCC | GOAD | ICL | PlanarFlow | VAE | GANomaly | SLAD | DIF | DDPM | DTE-NP | DTE-IG | DTE-C |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| aloi | 55.58(0.2) | 51.53(0.01) | 53.06(0.01) | 79.15(0.56) | 53.11(0.21) | 54.22(0.39) | 61.32(0.04) | 49.52(1.02) | 76.66(0.35) | 52.04(0.17) | 54.86(0.01) | 54.9(0.08) | 51.69(2.13) | 51.42(2.92) | 50.0(0.0) | 49.69(0.64) | 54.84(0.62) | 52.01(1.37) | 54.85(0.0) | 54.75(1.09) | 54.24(0.68) | 49.77(1.14) | 53.22(0.25) | 64.5(0.09) | 54.1(0.99) | 52.48(0.31) |
| amazon | 57.92(0.24) | 57.05(0.06) | 54.1(0.05) | 57.18(0.49) | 56.3(0.08) | 55.76(0.65) | 60.27(0.04) | 52.63(3.04) | 57.09(0.56) | 59.73(0.42) | 56.47(0.1) | 54.95(0.1) | 50.12(1.97) | 46.38(2.16) | 50.0(0.0) | 55.97(2.07) | 52.84(0.7) | 49.49(0.97) | 54.98(0.0) | 55.12(1.1) | 51.8(0.34) | 50.91(1.39) | 55.13(0.09) | 60.3(0.4) | 53.45(1.61) | 55.64(2.54) |
| annthyroid | 67.57(0.98) | 77.67(0.17) | 78.91(0.11) | 78.77(2.68) | 60.84(2.38) | 81.63(1.18) | 76.05(0.14) | 45.33(12.81) | 70.95(1.05) | 91.8(0.35) | 68.17(0.21) | 67.56(0.39) | 54.81(7.26) | 73.9(1.57) | 63.07(2.69) | 45.25(10.03) | 59.94(3.53) | 96.58(1.49) | 67.44(0.01) | 61.91(1.12) | 57.26(6.06) | 49.5(0.85) | 81.37(1.37) | 78.11(0.22) | 92.32(2.08) | 96.36(0.52) |
| backdoor | 89.71(0.72) | 50.0(0.0) | 50.0(0.0) | 79.03(3.04) | 74.04(0.77) | 72.46(3.35) | 82.64(0.5) | 51.5(16.4) | 76.42(2.72) | 84.78(9.77) | 88.86(0.77) | 88.75(0.73) | 75.24(8.88) | 73.49(2.66) | 50.0(0.0) | 58.68(10.74) | 93.62(0.75) | 78.66(7.09) | 88.79(0.76) | 90.75(3.52) | 50.0(0.0) | 50.21(0.9) | 89.18(0.62) | 80.56(0.48) | 75.34(11.82) | 87.52(1.0) |
| breastw | 96.08(0.93) | 99.44(0.16) | 99.04(0.25) | 40.83(2.6) | 98.44(0.3) | 98.32(0.46) | 98.02(0.53) | 96.97(3.27) | 44.61(3.25) | 98.52(0.45) | 93.49(2.12) | 94.63(1.2) | 81.1(6.33) | 62.54(12.59) | 84.73(6.24) | 84.54(9.29) | 80.73(3.93) | 96.48(1.43) | 92.8(4.07) | 94.27(3.9) | 81.74(1.91) | 51.14(2.06) | 76.64(4.22) | 97.62(0.47) | 90.45(2.0) | 89.07(2.24) |
| campaign | 73.78(0.3) | 78.28(0.04) | 76.94(0.05) | 59.37(4.19) | 76.81(0.27) | 70.37(1.81) | 74.95(0.14) | 49.27(8.77) | 61.44(0.34) | 77.47(0.94) | 73.65(0.07) | 73.4(0.12) | 58.07(2.76) | 50.79(8.34) | 50.0(0.0) | 44.28(5.98) | 76.61(0.5) | 56.61(2.97) | 73.42(0.0) | 65.19(5.23) | 70.41(0.71) | 49.87(0.72) | 72.38(0.77) | 74.59(0.16) | 65.99(6.09) | 78.91(0.61) |
| cardio | 83.16(1.77) | 92.08(0.3) | 93.49(0.12) | 57.89(2.77) | 83.94(1.23) | 92.21(1.22) | 83.02(1.85) | 85.59(7.1) | 55.12(2.42) | 81.48(1.65) | 93.42(0.35) | 94.9(0.18) | 62.47(10.89) | 49.78(16.06) | 65.54(4.78) | 90.77(3.72) | 46.07(4.06) | 79.59(4.61) | 95.0(0.04) | 77.01(18.8) | 49.5(4.13) | 49.52(2.41) | 72.33(5.93) | 77.67(2.16) | 63.11(10.52) | 72.13(3.16) |
| cardiotocography | 56.09(2.47) | 66.42(3.42) | 78.4(0.22) | 53.79(1.73) | 59.5(1.09) | 68.09(2.61) | 50.3(0.49) | 70.8(13.23) | 52.67(2.09) | 49.99(0.49) | 69.13(0.43) | 74.66(0.68) | 54.6(6.58) | 48.8(5.25) | 44.9(4.67) | 62.42(12.71) | 37.2(2.07) | 64.25(6.21) | 75.25(0.08) | 66.57(10.45) | 38.27(3.61) | 50.42(0.97) | 57.86(4.27) | 49.28(0.7) | 50.62(6.64) | 51.03(2.7) |
| celeba | 75.34(1.76) | 75.7(0.62) | 76.31(0.65) | 51.39(2.7) | 75.41(0.65) | 70.72(1.27) | 73.58(0.38) | 59.97(11.91) | 43.21(1.22) | 80.25(3.67) | 78.11(0.7) | 79.23(0.61) | 62.66(3.98) | 49.12(17.53) | 72.57(0.97) | 43.2(12.41) | 68.35(2.27) | 70.32(11.86) | 78.97(0.39) | 43.67(19.16) | 60.49(1.66) | 49.93(0.99) | 79.58(1.88) | 69.87(0.51) | 69.96(4.42) | 81.22(1.54) |
| census | 66.4(0.18) | 50.0(0.0) | 50.0(0.0) | 53.75(0.31) | 61.09(0.34) | 60.73(2.15) | 67.08(0.26) | 45.44(13.11) | 56.2(0.61) | 73.12(2.05) | 65.46(0.18) | 66.15(0.16) | 49.07(0.64) | 52.7(5.12) | 44.31(2.74) | 48.8(5.06) | 66.76(0.69) | 60.36(1.36) | 66.07(0.13) | 64.99(5.58) | 62.5(6.99) | 50.13(1.0) | 65.87(0.14) | 67.2(0.27) | 62.87(3.68) | 64.64(1.14) |
| cover | 92.24(0.17) | 88.2(0.32) | 91.85(0.2) | 57.14(2.15) | 70.68(1.16) | 87.32(2.69) | 86.57(2.41) | 92.18(3.85) | 56.75(1.89) | 69.58(0.9) | 95.2(0.16) | 93.39(0.25) | 74.15(14.15) | 58.04(22.39) | 74.69(22.59) | 12.37(7.08) | 68.07(5.04) | 41.68(7.76) | 93.25(0.23) | 70.13(19.1) | 72.34(1.89) | 50.41(1.34) | 80.76(1.65) | 83.81(2.71) | 63.52(7.37) | 69.68(4.69) |
| donors | 80.77(0.71) | 81.53(0.27) | 88.83(0.32) | 69.06(1.74) | 74.31(0.63) | 77.08(1.5) | 82.9(0.42) | 56.62(38.02) | 62.89(1.35) | 76.48(7.46) | 77.02(0.67) | 82.5(0.94) | 55.76(15.63) | 51.11(24.73) | 74.66(13.89) | 22.45(11.53) | 73.92(4.79) | 89.89(1.85) | 82.01(0.45) | 63.87(15.51) | 62.72(1.88) | 49.76(0.41) | 80.63(1.88) | 83.23(0.45) | 79.55(10.75) | 78.49(7.9) |
| fault | 66.5(2.51) | 45.49(0.16) | 46.81(0.19) | 59.1(1.16) | 50.62(6.66) | 54.39(1.01) | 71.52(0.62) | 47.78(2.34) | 57.88(1.35) | 50.5(1.2) | 53.69(0.3) | 47.97(0.55) | 49.54(5.92) | 52.22(2.87) | 66.84(2.7) | 54.63(4.8) | 66.05(1.1) | 46.87(3.43) | 48.46(0.08) | 48.92(6.42) | 70.48(0.88) | 50.69(0.61) | 56.22(0.95) | 72.55(0.3) | 57.74(5.42) | 58.95(3.36) |
| fraud | 95.39(0.81) | 94.29(1.42) | 94.89(1.27) | 61.59(7.15) | 94.51(1.22) | 94.95(1.15) | 95.5(1.13) | 85.55(5.41) | 54.77(7.3) | 91.13(1.91) | 95.37(0.72) | 95.23(0.72) | 85.71(5.32) | 76.91(6.34) | 50.0(0.0) | 72.4(22.06) | 93.09(1.05) | 89.48(2.1) | 95.24(0.76) | 84.93(5.45) | 93.79(1.88) | 50.22(3.43) | 92.37(1.37) | 95.6(1.18) | 94.18(1.16) | 93.78(1.29) |
| glass | 85.5(4.18) | 75.95(1.51) | 71.04(3.41) | 65.86(12.47) | 82.02(1.77) | 78.95(3.72) | 87.03(1.1) | 62.43(12.75) | 61.76(12.42) | 79.47(1.22) | 66.07(6.69) | 71.51(2.1) | 62.99(16.27) | 51.69(18.75) | 74.29(16.19) | 54.46(20.97) | 72.85(5.65) | 76.62(8.46) | 69.95(1.81) | 80.81(13.83) | 74.87(2.69) | 50.05(5.2) | 56.04(9.11) | 88.13(1.24) | 68.05(23.1) | 86.39(4.15) |
| hepatitis | 63.47(14.61) | 80.74(0.93) | 73.7(1.73) | 46.94(11.14) | 76.77(1.57) | 68.29(1.42) | 66.94(7.9) | 55.74(15.73) | 46.77(8.48) | 72.05(2.95) | 70.38(1.52) | 74.78(2.64) | 60.0(9.71) | 36.08(22.07) | 58.18(4.23) | 63.65(22.94) | 61.64(3.75) | 65.42(7.96) | 74.84(1.89) | 65.51(6.12) | 43.06(10.04) | 49.28(3.43) | 46.12(8.03) | 63.1(2.79) | 45.14(13.88) | 57.69(9.3) |
| http | 99.61(0.03) | 99.07(0.26) | 97.96(0.1) | 28.82(1.45) | 99.11(0.16) | 99.95(0.08) | 5.05(1.98) | 5.95(9.35) | 33.75(1.07) | 99.95(0.01) | 99.36(0.05) | 99.66(0.03) | 83.81(35.3) | 24.89(42.35) | 50.0(0.0) | 99.56(0.04) | 92.11(7.15) | 99.38(0.05) | 99.62(0.02) | 77.91(18.04) | 99.43(0.1) | 49.66(2.23) | 99.76(0.16) | 5.06(1.99) | 97.33(4.05) | 99.51(0.21) |
| imdb | 49.56(0.21) | 51.2(0.04) | 47.05(0.03) | 49.89(0.54) | 49.86(0.08) | 48.93(0.66) | 49.44(0.14) | 46.6(2.52) | 50.02(0.55) | 50.36(0.21) | 48.39(0.13) | 47.82(0.06) | 48.68(0.29) | 52.56(4.11) | 49.95(0.12) | 48.58(1.58) | 52.07(0.36) | 49.29(0.76) | 47.83(0.0) | 48.96(1.32) | 50.54(0.4) | 49.88(0.83) | 47.79(0.11) | 49.49(0.3) | 48.61(4.27) | 48.44(2.75) |
| internetads | 61.55(0.12) | 67.61(0.07) | 67.66(0.06) | 49.42(4.74) | 69.56(0.04) | 68.61(1.81) | 61.62(0.08) | 54.06(5.45) | 58.73(1.53) | 65.96(4.33) | 61.52(0.07) | 60.91(0.22) | 51.48(2.63) | 58.3(4.5) | 50.0(1.4) | 61.41(0.35) | 59.21(1.26) | 60.78(0.48) | 61.47(0.0) | 68.75(1.0) | 60.08(2.14) | 49.38(1.28) | 61.37(0.03) | 63.42(0.33) | 63.53(3.89) | 65.58(2.8) |
| ionosphere | 89.19(1.14) | 78.27(1.27) | 71.65(1.21) | 87.64(1.63) | 54.44(3.14) | 83.3(1.31) | 92.18(1.24) | 78.84(4.28) | 86.38(2.12) | 95.14(0.75) | 83.79(1.44) | 77.68(1.24) | 64.1(4.77) | 51.37(9.79) | 76.6(7.87) | 82.92(2.69) | 62.9(2.96) | 88.37(1.49) | 78.55(1.59) | 88.07(2.14) | 88.4(1.06) | 49.46(2.76) | 75.75(2.41) | 92.4(1.63) | 69.72(11.39) | 91.14(1.82) |
| landsat | 54.77(4.28) | 42.17(0.12) | 36.83(0.13) | 54.03(1.15) | 57.5(0.58) | 47.4(2.32) | 61.44(0.32) | 38.23(3.78) | 54.86(1.33) | 60.7(0.07) | 42.33(0.26) | 36.55(0.29) | 53.25(1.1) | 63.13(2.62) | 62.56(0.47) | 50.59(9.05) | 64.86(1.86) | 46.44(0.89) | 54.85(1.48) | 54.4(4.8) | 67.27(0.61) | 50.23(0.98) | 49.62(0.59) | 60.2(0.39) | 47.31(8.29) | 54.42(1.2) |
| letter | 76.32(1.8) | 56.02(0.07) | 57.26(0.13) | 88.58(0.51) | 58.85(0.59) | 61.56(1.17) | 81.21(0.3) | 53.71(4.29) | 87.81(0.48) | 80.43(0.42) | 59.81(0.28) | 52.35(0.29) | 50.31(5.01) | 51.74(2.09) | 78.03(2.72) | 59.77(1.85) | 73.65(1.25) | 68.92(3.77) | 52.45(0.12) | 68.15(9.18) | 86.52(0.55) | 50.45(1.04) | 84.73(1.44) | 85.0(0.5) | 67.63(4.86) | 78.13(1.49) |
| lymphography | 99.35(0.74) | 99.6(0.09) | 99.49(0.14) | 52.34(18.31) | 99.49(0.18) | 99.85(0.1) | 99.51(0.28) | 90.04(11.06) | 63.63(18.01) | 98.9(0.24) | 99.56(0.27) | 99.67(0.25) | 84.0(4.77) | 68.07(30.75) | 87.75(8.04) | 99.53(0.28) | 88.43(4.62) | 94.03(5.01) | 99.72(0.21) | 86.43(20.22) | 96.11(1.79) | 51.15(6.83) | 95.77(4.02) | 98.89(0.72) | 85.23(7.81) | 83.41(11.37) |
| magic.gamma | 72.53(0.1) | 68.1(0.04) | 63.8(0.05) | 69.96(0.64) | 70.93(0.55) | 72.1(0.79) | 79.5(0.15) | 65.5(1.41) | 67.84(0.38) | 69.87(0.1) | 67.26(0.18) | 66.73(0.11) | 58.44(3.35) | 60.37(1.2) | 72.79(0.59) | 44.24(4.6) | 67.55(1.02) | 74.23(4.69) | 66.86(0.0) | 57.73(3.14) | 63.78(0.4) | 50.2(0.48) | 76.26(1.48) | 80.07(0.15) | 78.21(2.19) | 76.45(1.39) |
| mammography | 79.5(1.84) | 90.54(0.03) | 90.62(0.03) | 72.61(0.51) | 83.77(0.9) | 85.96(1.73) | 85.17(0.24) | 86.69(2.45) | 70.15(1.87) | 69.02(1.35) | 87.11(0.13) | 88.84(0.29) | 71.91(17.95) | 45.13(6.77) | 77.92(1.81) | 41.41(13.31) | 65.76(9.12) | 78.22(2.96) | 88.67(0.07) | 87.12(4.62) | 73.23(2.4) | 50.17(2.0) | 74.94(2.14) | 84.86(0.22) | 79.94(5.76) | 81.02(2.74) |
| mnist | 84.26(1.11) | 50.0(0.0) | 50.0(0.0) | 66.44(1.42) | 57.36(0.38) | 81.06(2.17) | 86.66(0.59) | 56.4(11.07) | 65.77(1.94) | 85.62(1.32) | 84.91(0.22) | 84.76(0.43) | 63.14(6.24) | 60.48(14.11) | 61.51(1.13) | 69.76(8.76) | 69.06(1.61) | 64.49(15.63) | 85.0(0.04) | 72.05(3.57) | 50.0(0.0) | 49.51(1.34) | 81.6(1.2) | 85.27(0.77) | 75.56(4.12) | 81.89(1.32) |
| musk | 100.0(0.0) | 94.81(0.48) | 95.28(0.29) | 57.54(10.67) | 100.0(0.0) | 99.77(0.38) | 96.42(3.15) | 99.3(0.77) | 58.07(8.89) | 99.96(0.05) | 100.0(0.0) | 100.0(0.0) | 91.22(7.47) | 53.84(15.38) | 57.49(5.9) | 100.0(0.0) | 79.0(2.58) | 74.83(25.71) | 100.0(0.0) | 100.0(0.0) | 83.66(4.44) | 50.03(3.54) | 99.95(0.03) | 88.17(5.61) | 78.48(12.07) | 96.47(1.74) |
| optdigits | 78.48(0.95) | 50.0(0.0) | 50.0(0.0) | 53.93(5.59) | 86.81(0.25) | 69.63(4.81) | 39.52(2.17) | 49.29(9.35) | 53.82(5.25) | 41.27(5.26) | 50.66(1.46) | 51.78(0.53) | 40.78(19.75) | 51.94(25.08) | 56.52(3.54) | 65.66(7.61) | 53.29(6.15) | 49.15(13.23) | 51.47(0.2) | 44.79(14.06) | 56.08(3.06) | 50.18(1.59) | 40.15(3.71) | 38.61(2.33) | 51.34(4.5) | 50.77(6.25) |
| pageblocks | 89.3(2.14) | 87.48(0.21) | 91.37(0.1) | 75.83(1.79) | 77.89(2.07) | 89.72(0.35) | 91.94(0.32) | 71.24(10.47) | 70.33(1.55) | 92.3(0.22) | 91.42(0.21) | 90.67(0.47) | 75.29(4.49) | 59.24(11.13) | 91.38(1.48) | 60.93(7.81) | 76.75(1.0) | 90.8(2.02) | 90.57(0.0) | 80.13(2.79) | 74.57(4.01) | 50.26(1.09) | 82.0(0.91) | 90.64(0.63) | 85.02(8.84) | 92.39(0.84) |
| pendigits | 86.38(7.22) | 90.6(0.43) | 92.69(0.15) | 51.82(5.22) | 92.47(0.38) | 94.69(0.78) | 82.81(3.01) | 89.51(0.85) | 53.42(4.61) | 83.41(0.94) | 92.86(0.48) | 93.58(0.12) | 54.8(21.37) | 38.26(12.39) | 52.01(4.93) | 59.23(15.6) | 65.02(4.5) | 77.99(9.67) | 93.66(0.03) | 83.75(10.03) | 68.63(2.86) | 49.65(2.66) | 70.03(5.43) | 78.64(3.13) | 62.39(10.39) | 71.33(6.62) |
| pima | 65.49(1.65) | 66.2(1.21) | 60.38(1.54) | 57.31(3.88) | 70.38(1.21) | 67.37(1.88) | 72.33(1.74) | 59.52(7.09) | 56.34(3.89) | 68.64(2.09) | 63.11(1.82) | 65.11(2.53) | 52.16(3.2) | 51.01(3.01) | 54.18(11.49) | 60.57(18.25) | 52.4(2.56) | 61.51(5.6) | 67.03(1.72) | 55.05(6.29) | 47.11(3.76) | 51.24(2.19) | 53.74(5.29) | 70.71(2.47) | 59.92(3.39) | 62.37(3.09) |
| satellite | 74.19(3.23) | 63.34(0.04) | 58.3(0.05) | 54.54(1.26) | 76.18(0.55) | 69.5(1.07) | 72.09(0.14) | 61.39(2.92) | 54.98(1.29) | 80.39(0.07) | 66.24(0.21) | 60.14(0.07) | 67.46(2.21) | 56.21(5.85) | 60.8(1.23) | 70.21(3.55) | 62.65(1.77) | 67.05(1.01) | 75.55(0.75) | 68.13(6.86) | 74.56(1.53) | 49.84(0.7) | 71.53(0.52) | 70.15(0.19) | 58.17(6.67) | 71.05(1.82) |
| satimage-2 | 99.88(0.01) | 97.46(0.04) | 96.52(0.05) | 52.55(7.36) | 97.6(0.12) | 99.28(0.16) | 99.24(0.61) | 98.14(0.49) | 53.91(7.29) | 99.53(0.06) | 99.67(0.01) | 97.72(0.03) | 91.13(4.48) | 55.12(12.77) | 57.86(11.12) | 99.56(0.06) | 89.76(2.38) | 97.03(0.86) | 98.51(0.12) | 98.36(1.01) | 96.65(1.42) | 50.77(2.25) | 99.55(0.05) | 97.95(1.48) | 85.81(5.97) | 94.56(1.27) |
| shuttle | 62.06(3.96) | 99.5(0.1) | 99.3(0.01) | 49.29(3.4) | 98.63(0.23) | 99.71(0.03) | 73.16(0.66) | 38.87(13.16) | 52.55(1.09) | 99.01(0.01) | 99.18(0.0) | 98.99(0.03) | 89.77(5.59) | 57.58(13.24) | 50.0(0.0) | 20.78(28.35) | 64.17(14.05) | 85.16(6.0) | 98.98(0.0) | 98.11(0.69) | 90.72(18.12) | 50.08(0.35) | 97.5(2.28) | 69.83(0.34) | 66.86(26.52) | 97.62(1.2) |
| skin | 67.51(3.8) | 47.12(0.19) | 48.97(0.1) | 53.38(0.29) | 58.84(0.37) | 66.97(0.65) | 71.97(0.13) | 44.16(1.92) | 54.97(0.17) | 89.15(0.15) | 54.73(0.45) | 44.69(0.58) | 55.44(17.81) | 54.82(7.49) | 70.84(1.68) | 57.86(4.33) | 26.45(9.05) | 77.33(3.97) | 52.27(0.25) | 46.54(3.91) | 75.01(0.51) | 50.15(0.38) | 46.09(2.11) | 71.81(0.13) | 74.05(3.66) | 74.05(2.35) |
| smtp | 86.29(4.76) | 91.17(1.63) | 88.22(2.47) | 79.36(5.06) | 80.91(4.76) | 90.5(2.2) | 93.26(1.74) | 81.88(3.56) | 89.92(5.31) | 94.84(0.93) | 84.49(4.46) | 85.62(4.92) | 86.77(5.56) | 89.53(5.1) | 50.0(0.0) | 91.53(1.82) | 65.61(12.08) | 78.37(11.9) | 81.98(5.52) | 59.01(16.05) | 92.13(1.68) | 54.1(6.53) | 95.56(1.3) | 92.97(1.89) | 76.92(14.33) | 95.07(1.36) |
| spambase | 54.13(0.68) | 68.79(0.08) | 65.57(0.12) | 42.41(0.84) | 66.41(1.18) | 63.72(2.0) | 56.56(0.27) | 47.98(10.23) | 45.29(0.42) | 44.55(3.6) | 53.43(0.18) | 54.77(0.69) | 48.84(4.87) | 58.44(7.73) | 49.04(1.5) | 49.57(9.44) | 45.88(1.79) | 52.83(6.32) | 54.99(0.03) | 55.18(2.65) | 49.26(0.44) | 49.51(0.82) | 51.01(1.51) | 54.47(0.38) | 50.93(4.84) | 51.52(1.9) |
| speech | 47.11(0.24) | 48.89(0.29) | 46.95(0.04) | 50.85(0.71) | 47.32(0.19) | 47.62(0.7) | 48.0(0.09) | 46.57(1.85) | 51.15(0.65) | 49.37(1.87) | 46.59(0.1) | 46.89(0.04) | 52.22(4.41) | 51.24(3.1) | 48.27(1.74) | 45.81(1.99) | 51.16(2.23) | 49.56(3.55) | 46.94(0.05) | 50.66(2.29) | 50.85(2.15) | 48.09(2.89) | 46.57(1.17) | 48.73(1.02) | 48.84(4.05) | 49.46(1.46) |
| stamps | 66.04(2.92) | 92.92(0.83) | 87.72(1.03) | 50.16(7.61) | 90.4(1.0) | 90.72(1.09) | 86.97(2.04) | 83.1(9.15) | 51.18(9.24) | 83.83(3.64) | 88.24(1.44) | 90.94(1.59) | 71.89(9.56) | 46.51(7.98) | 76.0(7.44) | 77.44(13.78) | 50.51(4.05) | 83.81(4.86) | 90.63(1.26) | 77.4(12.76) | 51.2(3.79) | 50.7(4.35) | 55.55(12.83) | 82.02(4.05) | 69.17(18.33) | 75.25(5.15) |
| thyroid | 90.93(1.07) | 93.91(0.26) | 97.71(0.11) | 70.68(1.69) | 94.84(1.01) | 97.91(0.55) | 96.52(0.21) | 81.89(12.49) | 65.69(3.08) | 98.57(0.04) | 95.83(0.12) | 95.48(0.31) | 71.85(12.11) | 50.45(2.0) | 88.89(4.95) | 57.42(27.16) | 69.25(5.72) | 99.18(0.26) | 95.56(0.0) | 81.9(17.35) | 80.02(3.3) | 50.85(2.85) | 87.06(1.5) | 96.43(0.3) | 82.75(12.16) | 99.02(0.12) |
| vertebral | 46.34(1.41) | 26.28(3.94) | 41.67(3.03) | 47.32(5.6) | 31.73(3.46) | 36.18(3.44) | 37.9(2.03) | 29.44(5.11) | 48.68(3.4) | 38.9(2.16) | 42.61(0.9) | 37.82(2.71) | 46.96(9.72) | 39.42(2.69) | 42.45(15.87) | 46.81(12.68) | 44.9(1.08) | 40.86(14.66) | 38.4(2.54) | 37.66(8.85) | 39.75(3.09) | 49.28(2.03) | 56.34(7.41) | 40.04(2.58) | 45.07(14.43) | 45.76(8.79) |
| vowels | 88.38(0.5) | 49.6(0.44) | 59.3(0.38) | 93.29(1.73) | 67.91(0.98) | 76.26(3.14) | 95.09(0.23) | 70.52(4.82) | 93.18(1.07) | 73.21(8.48) | 77.87(0.98) | 60.35(0.6) | 46.44(13.27) | 51.37(2.96) | 73.81(15.9) | 79.08(5.62) | 78.42(2.32) | 88.82(3.83) | 62.08(0.09) | 55.43(18.53) | 93.19(0.78) | 49.29(2.87) | 90.34(2.76) | 96.42(0.42) | 70.49(8.5) | 91.42(5.8) |
| waveform | 70.14(0.99) | 73.89(0.4) | 60.31(0.21) | 71.52(2.12) | 69.38(0.64) | 70.69(5.94) | 74.97(0.82) | 59.4(6.16) | 69.26(1.81) | 57.16(0.49) | 66.88(0.3) | 63.53(0.26) | 52.3(7.42) | 60.86(7.12) | 67.4(1.94) | 59.15(5.89) | 66.05(2.92) | 63.97(5.61) | 63.85(0.05) | 62.01(16.16) | 43.65(0.95) | 48.7(3.49) | 61.72(3.41) | 72.91(1.34) | 52.29(6.3) | 60.2(3.55) |
| wbc | 97.71(1.1) | 99.4(0.1) | 99.38(0.12) | 38.83(10.4) | 98.7(0.16) | 99.58(0.15) | 98.21(0.47) | 99.17(0.13) | 60.66(8.68) | 98.77(1.03) | 98.72(0.54) | 99.28(0.25) | 82.09(14.37) | 50.34(17.76) | 82.08(10.35) | 94.86(4.74) | 85.34(4.73) | 93.36(3.36) | 99.15(0.18) | 95.76(3.23) | 96.08(4.57) | 48.01(3.85) | 94.8(1.16) | 97.94(0.76) | 89.38(7.64) | 87.06(3.83) |
| wdbc | 98.98(0.42) | 99.29(0.14) | 97.05(0.65) | 86.65(8.37) | 98.94(0.24) | 98.83(0.39) | 98.04(0.64) | 98.04(0.77) | 84.87(9.38) | 96.9(1.12) | 98.42(0.37) | 98.8(0.31) | 71.51(25.04) | 60.2(20.82) | 34.73(16.78) | 98.28(1.25) | 73.78(7.19) | 98.54(0.81) | 97.79(0.97) | 96.23(2.96) | 78.99(12.13) | 48.52(3.7) | 96.54(0.93) | 97.49(1.11) | 56.64(26.56) | 83.51(6.61) |
| wilt | 39.59(1.76) | 34.49(0.27) | 39.4(0.12) | 66.59(7.27) | 34.81(2.83) | 45.13(3.34) | 51.05(0.65) | 31.31(9.9) | 67.81(1.65) | 85.88(0.06) | 31.66(0.3) | 23.94(0.42) | 43.18(5.91) | 46.5(1.58) | 39.99(3.03) | 55.46(8.59) | 64.85(3.41) | 79.36(2.41) | 33.08(0.0) | 41.32(1.11) | 68.06(1.02) | 50.91(1.72) | 65.9(3.92) | 55.15(0.61) | 83.38(7.17) | 85.12(1.73) |
| wine | 45.27(33.4) | 86.46(4.66) | 73.77(4.79) | 32.34(5.54) | 90.72(3.31) | 78.61(6.71) | 47.04(3.12) | 82.17(3.91) | 32.97(14.63) | 97.54(1.59) | 67.14(3.79) | 81.91(2.83) | 51.27(29.94) | 50.72(23.41) | 62.05(21.99) | 73.37(23.81) | 45.47(4.33) | 38.99(21.89) | 81.22(3.63) | 68.17(16.01) | 37.42(8.26) | 48.45(3.14) | 37.39(8.37) | 42.45(6.29) | 31.01(10.98) | 55.71(11.39) |
| wpbc | 48.68(3.14) | 51.87(3.09) | 48.91(2.35) | 43.61(3.13) | 54.81(2.85) | 51.55(1.88) | 51.22(2.52) | 50.07(3.44) | 44.66(3.87) | 53.41(4.03) | 48.51(2.34) | 48.62(2.27) | 44.94(2.82) | 49.34(2.52) | 48.26(3.3) | 46.64(5.66) | 48.76(5.04) | 48.34(3.69) | 46.77(1.77) | 48.87(4.0) | 48.33(1.22) | 48.82(2.44) | 49.31(3.97) | 50.22(3.27) | 48.91(3.81) | 46.79(6.57) |
| yeast | 46.06(1.23) | 38.03(0.15) | 44.31(0.16) | 46.45(1.53) | 40.17(0.97) | 39.42(1.16) | 39.62(0.95) | 46.13(4.68) | 45.3(2.12) | 40.64(1.1) | 41.95(0.43) | 41.75(0.4) | 50.33(4.55) | 52.04(3.8) | 39.57(4.74) | 50.31(3.51) | 46.59(2.15) | 44.2(5.92) | 40.07(0.01) | 47.64(5.98) | 49.55(1.21) | 50.01(2.19) | 46.31(0.72) | 39.98(1.1) | 44.61(4.27) | 42.03(2.62) |
| yelp | 63.53(0.34) | 60.52(0.15) | 57.78(0.04) | 66.1(0.42) | 59.97(0.11) | 60.15(0.34) | 67.01(0.17) | 58.1(2.89) | 66.11(0.48) | 65.49(0.64) | 62.08(0.04) | 59.19(0.1) | 49.83(1.27) | 52.43(8.88) | 50.35(0.54) | 59.02(3.78) | 54.52(0.49) | 52.67(1.98) | 59.29(0.08) | 60.51(0.91) | 54.37(0.27) | 49.1(1.34) | 59.38(0.12) | 67.08(0.37) | 51.37(3.24) | 60.16(3.22) |
| MNIST-C | 75.71(1.0) | 50.0(0.0) | 50.0(0.0) | 70.19(1.05) | 68.92(0.22) | 73.34(1.89) | 78.64(0.13) | 59.06(9.66) | 69.94(0.95) | 73.9(1.62) | 75.11(0.09) | 74.05(0.18) | 58.13(6.73) | 55.24(9.15) | 59.37(2.13) | 75.16(1.73) | 66.95(1.05) | 70.48(1.97) | 74.05(0.01) | 72.56(3.12) | 74.59(0.67) | 49.73(1.14) | 75.1(0.14) | 78.83(0.24) | 70.3(3.97) | 74.56(2.21) |
| FashionMNIST | 87.06(0.33) | 50.0(0.0) | 50.0(0.0) | 74.78(0.94) | 74.82(0.2) | 83.08(0.94) | 87.51(0.14) | 67.23(7.64) | 73.81(0.86) | 83.95(1.2) | 86.03(0.05) | 85.32(0.17) | 66.37(5.59) | 64.66(7.04) | 56.39(3.06) | 85.99(0.52) | 75.8(1.2) | 81.88(1.33) | 85.32(0.01) | 86.65(0.97) | 85.83(2.38) | 50.31(1.74) | 86.1(0.08) | 87.31(0.25) | 76.67(6.18) | 84.05(1.28) |
| CIFAR10 | 66.31(0.3) | 54.78(0.07) | 56.66(0.06) | 68.71(0.63) | 57.23(0.17) | 62.9(1.09) | 65.85(0.09) | 59.09(5.51) | 68.62(0.59) | 63.9(0.75) | 66.25(0.08) | 65.93(0.18) | 53.0(2.94) | 55.45(4.1) | 50.32(2.39) | 65.85(1.04) | 55.68(1.4) | 62.13(1.33) | 65.92(0.01) | 66.19(1.19) | 64.81(0.56) | 50.32(1.77) | 66.34(0.14) | 66.0(0.22) | 59.52(2.81) | 62.87(1.57) |
| SVHN | 60.13(0.25) | 50.0(0.0) | 50.0(0.0) | 62.9(0.4) | 54.22(0.12) | 58.03(0.7) | 60.36(0.08) | 53.4(4.29) | 62.8(0.41) | 58.31(0.84) | 60.38(0.08) | 59.92(0.17) | 52.75(2.11) | 52.05(3.23) | 52.12(2.09) | 59.7(1.44) | 57.06(1.05) | 58.04(1.09) | 59.91(0.01) | 60.17(1.06) | 59.57(0.4) | 49.66(1.16) | 60.45(0.13) | 60.69(0.2) | 56.73(2.89) | 59.96(1.24) |
| MVTec-AD | 75.41(2.26) | 50.0(0.0) | 50.0(0.0) | 74.54(2.66) | 73.18(1.9) | 74.67(1.85) | 76.28(1.56) | 64.42(5.16) | 74.17(2.73) | 61.79(4.89) | 73.52(1.86) | 72.44(1.94) | 59.57(4.79) | 60.28(6.07) | 54.38(4.52) | 72.96(2.61) | 68.3(3.37) | 63.73(6.77) | 72.46(1.61) | 73.85(2.83) | 69.88(3.91) | 49.94(2.48) | 73.21(1.92) | 76.11(2.26) | 65.48(5.76) | 72.95(3.11) |
| 20news | 56.38(1.44) | 53.26(0.59) | 54.42(0.42) | 60.97(1.63) | 53.69(0.62) | 55.0(1.81) | 56.65(1.23) | 53.85(4.56) | 60.98(1.65) | 58.29(2.78) | 55.92(0.97) | 54.48(0.69) | 51.78(3.92) | 51.49(5.34) | 49.58(2.77) | 55.28(1.84) | 54.73(2.58) | 51.27(2.61) | 54.59(0.69) | 55.74(1.98) | 55.31(1.21) | 50.99(3.11) | 54.74(0.56) | 56.98(1.59) | 52.72(5.59) | 57.87(3.45) |
| agnews | 61.91(0.31) | 55.1(0.04) | 55.24(0.04) | 71.5(0.62) | 55.4(0.08) | 58.43(1.32) | 64.65(0.1) | 56.81(3.57) | 71.36(0.64) | 66.5(0.5) | 60.09(0.11) | 56.61(0.08) | 50.8(3.45) | 49.36(4.46) | 50.01(0.55) | 59.16(3.06) | 59.08(0.61) | 49.65(1.27) | 56.6(0.0) | 64.53(1.65) | 57.44(0.25) | 50.17(1.17) | 57.06(0.1) | 65.22(0.31) | 54.45(5.22) | 62.66(3.75) |
Note: DTE-NP is for the non-parametric DTE DTENonParametric, DTE-IG is for the inverse gamma DTE DTEInverseGamma, and DTE-C is for the categorical DTE DTECategorical.