Energy-based Out-of-distribution Detection (Energy OOD)

This repository is the official implementation of Energy-based Out-of-distribution Detection by Weitang Liu, Xiaoyun Wang, John Owens and Yixuan Li. This method is an effective and easy OOD detector with and without fine-tuning. Our code is implemented with courtesy of Outlier-Exposure. If you have any code related questions, such as this issue and this issue, we highly recommened to check the couterpart in Outlier-Exposure.

Pretrained Models and Datasets

Pretrained models are provided in folder

./CIFAR/snapshots/

Please download the datasets in folder

./data/

Testing and Fine-tuning

run energy score testing for cifar10 WRN

test
bash run.sh energy 0

run energy score testing for cifar100 WRN

test
bash run.sh energy 1

run energy score training and testing for cifar10 WRN

train
bash run.sh energy_ft 0

run energy score training and testing for cifar100 WRN

train
bash run.sh energy_ft 1

Results

Our model achieves the following average performance on 6 OOD datasets:

1. MSP vs energy score with and without fine-tuned on CIFAR-10

| Model name | FPR95 | | ------------------ |---------------- | | Softmax score | 51.04% | | Energy score (ours) | 33.01% | | Softmax score with fine-tune | 8.53% | | Energy score with fine-tune (ours) | 3.32% |

2. CIFAR-10 (in-distribution) vs SVHN (out-of-distribution) Score Distributions

3. Performance among different baselines for WideResNet

CIFAR-10: | Model name | FPR95 | | ------------------ |---------------- | | Softmax score | 51.04% | | Energy score (ours) | 33.01% | | ODIN | 35.71% | | Mahalanobis | 37.08% | | Outlier Exposure| 8.53% | | Energy score with fine-tune (ours) | 3.32% |

CIFAR-100: | Model name | FPR95 | | ------------------ |---------------- | | Softmax score | 80.41% | | Energy score (ours) | 73.60% | | ODIN | 74.64% | | Mahalanobis | 54.64% | | Outlier Exposure| 58.10% | | Energy score with fine-tune (ours) | 47.55% |

Outlier Datasets

These experiments make use of numerous outlier datasets. Links for less common datasets are as follows, 80 Million Tiny Images Textures, Places365, LSUN-C, LSUN-R, iSUN and SVHN.

Citation

 @article{liu2020energy,
      title={Energy-based Out-of-distribution Detection},
      author={Liu, Weitang and Wang, Xiaoyun and Owens, John and Li, Yixuan},
      journal={Advances in Neural Information Processing Systems},
      year={2020}
 }