README.md

August 20, 2026 · View on GitHub

SafetyCage is a Python package for detecting misclassified predictions from machine learning models in classification tasks. It provides a unified interface for multiple statistical detection methods, enabling users to quantify prediction reliability and flag potentially incorrect outputs across different models and datasets easily.

Available on PyPI: https://pypi.org/project/safetycage/.

Full documentation is available at https://safetycage.readthedocs.io/.

Background

The idea behind safetycage is that we can find statistics on each predicted sample and compare that statistic to some statistic threshold to predict whether the sample prediction was incorrectly classified.

Description

Machine learning models can produce incorrect predictions with high confidence. SafetyCage addresses this by providing post-hoc misclassification detection methods that operate on model outputs or internal representations.

The package includes several methods:

MSP (Maximum Softmax Probability)
DOCTOR (Error probability estimation)
Mahalanobis (Distance-based statistical testing)
SPARDACUS (Projection + density estimation approach)

Each method outputs a statistic or p-value that reflects how likely a prediction is to be incorrect.

Alternatively, you can implement your own method by initializing a base class from the safetycage abstract base class, which defines how methods should be implemented.

Requirements

safetycage requires Python 3.13 or later.

Core dependencies (installed automatically): joblib, matplotlib, numpy.

Installation

pip install safetycage
# or: uv add safetycage

Some methods need extra dependencies, installed via pip install safetycage[extra] (or uv add "safetycage[extra]"):

Extra	Uses	Adds
`red`	`RED`	`torch`, `gpytorch`
`spardacus`	`SPARDACUS`	`statsmodels`, `scipy`, `scikit-learn`, `tqdm`
`mahalanobis`	`Mahalanobis`	`statsmodels`, `scipy`
`torch`	`TorchModelModule`	`torch`