Combining Statistical Depth and Fermat Distance for Uncertainty Quantification

This Repo contains all the files related for computing Lens Depth and for conducting experiments in the related paper.

Dependencies

The code is implemented based mainly on python library Pytorch (and torchvision). All needed libraries can be found in requirements.txt. The code is supposed to be run in Linux but can be easily adapted for other systems. We strongly recommend to create virtual environment for a proper running (such as conda virtual env). This can be easily done in linux terminal as follow:

conda create -n yourenvname python=x.x anaconda

Then, to activate this virtual env:

conda activate yourenvname

To install a package in this virtual env:

conda install -n yourenvname [package]

To quit this env:

conda deactivate

Data

In this work, we use only publicly available datasets MNIST, FashionMNIST, SVHN, CIFAR10, CIFAR100 and TinyImageNet. All these datasets can be downloaded using standard Deep Learning library Pytorch, except for TinyImageNet we use library tinyimagenet to download. Besides, for toydataset and proof-of-concept, we simulate dataset using python library scikit-learn.

Running

To make it easier for practitioners, we intentionally make 4 independent directories:

1. background_toy_dataset: all the codes are in notebooks, so that we can demonstrate how the method works step by step. Besides, this aims to make sure that the method work correctly in 2D space with visual results.
2. MNIST_FashionMNIST: all the codes for conducting experiments related to MNIST vs FashionMNIST.
3. CIFAR10_SVHN_TinyImageNet_CIFAR100: same for CFAR10 vs SVHN/CIFAR100/TinyImageNet.
4. CIFAR100: same for CIFAR100 vs TinyImageNet.

The codes in each of these directories can be runned independently. However, we strongly recommend to start by background_toy_dataset for a good understanding of method as well as to know how to correctly use our method.

Notice:

In this work, for benchmark purpose, we do use ResNet and WideResNet with Spectral Normalization, proposed in "J. Mukhoti, A. Kirsch, J. van Amersfoort, P. H. Torr, and Y. Gal. Deep deterministic uncertainty: A new simple baseline". If you use these architectures, please refer to their work for citation.