VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

June 7, 2026 ยท View on GitHub

CVPR 2026 arXiv License

This is the official PyTorch implementation of:

VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer

CVPR 2026

Highlights

  • Language-Free: VisualAD removes the text encoder entirely and learns anomaly/normal prototypes purely in the visual feature space.
  • Two Learnable Tokens: An anomaly token and a normal token are inserted into a frozen ViT, interacting with patch tokens through multi-layer self-attention to encode normality and abnormality.
  • SCA & SAF Modules: Spatial-Aware Cross-Attention (SCA) injects fine-grained spatial evidence into the tokens; Self-Alignment Function (SAF) recalibrates patch features before anomaly scoring.
  • 13 Benchmarks: State-of-the-art performance across 6 industrial and 7 medical zero-shot anomaly detection benchmarks.
  • Backbone Agnostic: Adapts seamlessly to CLIP (ViT-L/14@336px) and DINOv2 (ViT-L/14).

Main Results

Image-Level ZSAD Performance (AUROC / F1-max / AP)

DatasetWinCLIPAPRIL-GANAnomalyCLIPAdaCLIPVisualAD (CLIP)VisualAD (DINOv2)
MVTec-AD90.4 / 92.7 / 95.686.1 / 90.4 / 93.691.6 / 92.7 / 96.292.0 / 92.7 / 96.492.2 / 93.2 / 96.790.1 / 92.4 / 94.8
VisA75.6 / 78.2 / 78.877.4 / 78.6 / 80.981.0 / 80.3 / 84.479.7 / 79.6 / 83.284.7 / 82.5 / 87.683.1 / 81.4 / 86.8
BTAD68.2 / 67.8 / 70.973.7 / 68.7 / 69.988.7 / 86.0 / 90.690.0 / 87.2 / 91.594.9 / 93.9 / 97.088.2 / 84.7 / 89.7
KSDD293.5 / 86.4 / 94.290.4 / 82.9 / 92.091.9 / 84.5 / 93.494.9 / 90.3 / 96.298.0 / 93.9 / 98.397.7 / 93.1 / 98.1
DAGM91.8 / 75.8 / 79.594.4 / 80.3 / 83.998.0 / 90.6 / 92.498.3 / 91.5 / 94.299.5 / 95.0 / 97.893.2 / 83.9 / 86.1
DTD-Synthetic95.1 / 94.1 / 97.785.5 / 89.1 / 94.093.7 / 94.3 / 97.492.1 / 92.4 / 96.397.5 / 96.6 / 99.191.0 / 94.4 / 97.4

Pixel-Level ZSAD Performance (AUROC / F1-max / AP / PRO)

DatasetWinCLIPAPRIL-GANAnomalyCLIPAdaCLIPVisualAD (CLIP)VisualAD (DINOv2)
MVTec-AD82.3 / 24.8 / 18.2 / 62.087.5 / 42.3 / 39.1 / 43.791.0 / 38.9 / 34.4 / 81.788.5 / 43.9 / 41.0 / 47.690.8 / 43.9 / 41.2 / 87.591.3 / 47.4 / 45.4 / 88.6
VisA73.2 / 9.0 / 5.4 / 51.193.8 / 32.6 / 26.2 / 86.595.4 / 27.6 / 20.7 / 86.495.1 / 33.8 / 29.2 / 71.395.8 / 34.6 / 28.4 / 91.095.3 / 35.2 / 29.9 / 88.2
BTAD72.7 / 18.5 / 12.9 / 27.391.3 / 40.1 / 37.7 / 21.093.0 / 47.1 / 41.5 / 71.087.7 / 42.3 / 36.6 / 17.191.1 / 49.8 / 43.1 / 80.493.4 / 42.6 / 38.7 / 76.7
DTD-Synthetic79.5 / 16.1 / 9.8 / 51.594.9 / 60.4 / 61.0 / 33.897.5 / 55.8 / 52.5 / 87.995.1 / 58.4 / 56.1 / 34.398.1 / 64.3 / 65.5 / 94.896.7 / 65.8 / 67.7 / 92.4

All backbones use ViT-L/14@336px (CLIP) or ViT-L/14 (DINOv2). Full results including medical benchmarks (OCT17, BrainMRI, Brain_AD, HIS, CVC-ClinicDB, Endo, Kvasir) are available in our paper.

Getting Started

1. Environment

pip install -r requirements.txt

Main dependencies: PyTorch >= 2.0, torchvision, timm, scikit-learn, scipy, tqdm.

2. Datasets and Data Preparation

VisualAD is evaluated on 13 zero-shot anomaly detection benchmarks across industrial inspection and medical imaging. The dataset packages used by this project are available below:

DatasetDomainDescriptionDownload
MVTec-ADIndustrialObject and texture anomaly detection benchmark with image-level labels and pixel-level masks.Google Drive
VisAIndustrialVisual anomaly benchmark for industrial object inspection across multiple product categories.Google Drive
BTADIndustrialBTech industrial anomaly detection benchmark with three product categories.Google Drive
KSDD2IndustrialSurface defect detection benchmark for production-line visual inspection.Google Drive
DAGMIndustrialSynthetic textured-surface defect benchmark with annotated defects.Google Drive
DTD-SyntheticIndustrialSynthetic anomaly localization benchmark built from texture images.Google Drive
OCT17MedicalRetinal OCT scans for retinal disease anomaly/classification evaluation.Google Drive
BrainMRIMedicalBrain MRI images for brain tumor or lesion anomaly classification.Google Drive
Brain_ADMedicalBrain MRI benchmark for brain tumor or lesion anomaly classification.Google Drive
HISMedicalHistopathology images for abnormal tissue analysis.Google Drive
CVC-ClinicDBMedicalColonoscopy polyp segmentation benchmark with pixel-level annotations.Google Drive
EndoMedicalEndoscopic anomaly/lesion benchmark with segmentation annotations.Google Drive
KvasirMedicalGastrointestinal endoscopy benchmark for polyp/anomaly segmentation.Google Drive

We adopt the same dataset structure and JSON format as AnomalyCLIP. After downloading a dataset, place it under your local dataset directory and make sure meta.json exists in the dataset root. We also provide scripts to generate the required JSON metadata files:

python generate_dataset_json/mvtec.py
python generate_dataset_json/visa.py

3. Pre-trained Weights

We provide pre-trained checkpoints with the CLIP (ViT-L/14@336px) backbone:

Training SetCheckpointEvaluation Set
VisAweight/train_on_visa/CLIP.pthMVTec-AD and other cross-dataset benchmarks
MVTec-ADweight/train_on_mvtec/CLIP.pthVisA

4. Quick Start

Run the full cross-dataset training and evaluation pipeline (MVTec-AD <-> VisA) with a single command:

bash scripts/CLIP.sh

Please modify the dataset paths in scripts/CLIP.sh before running.

Citation

If you find this work useful, please consider citing:

@InProceedings{Hou_2026_CVPR,
  author    = {Hou, Yanning and Li, Peiyuan and Liu, Zirui and Wang, Yitong and Ruan, Yanran and Qiu, Jianfeng and Xu, Ke},
  title     = {VisualAD: Language-Free Zero-Shot Anomaly Detection via Vision Transformer},
  booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  month     = {June},
  year      = {2026},
  pages     = {21346-21356}
}

Acknowledgements

This project builds upon CLIP and DINOv2. We thank the authors of AnomalyCLIP and AdaCLIP for their open-source implementations.

Contact

If you have any questions, feel free to reach out: