FB-CLIP

March 20, 2026 · View on GitHub

[CVPR 26] [FB-CLIP: Fine-Grained Zero-Shot Anomaly Detection with Foreground-Background Disentanglement]

Introduction

Fine-grained anomaly detection is crucial in industrial and medical applications, but labeled anomalies are often scarce, making zero-shot detection challenging. While vision-language models like CLIP offer promising solutions, they struggle with foreground-background feature entanglement and coarse textual semantics. We propose FB-CLIP, a framework that enhances anomaly localization via multi-strategy textual representations and foreground-background separation. In the textual modality, it combines End-of-Text features, global-pooled representations, and attention-weighted token features for richer semantic cues. In the visual modality, multi-view soft separation along identity, semantic, and spatial dimensions, together with background suppression, reduces interference and improves discriminability. Semantic Consistency Regularization (SCR) aligns image features with normal and abnormal textual prototypes, suppressing uncertain matches and enlarging semantic gaps. Experiments show that FB-CLIP effectively distinguishes anomalies from complex backgrounds, achieving accurate fine-grained anomaly detection and localization under zero-shot settings. All experiments are conducted in PyTorch-2.0.0 with a single NVIDIA RTX 3090 24GB.

How to Run

conda create -n FBCLIP python=3.8.20
conda activate FBCLIP
pip install torch==2.0.0 torchvision==0.15.1 --index-url https://download.pytorch.org/whl/cu118
pip install -r requirements.txt

Prepare your dataset

Following AnomalyCLIP ,download the dataset below:

Generate the dataset JSON

Take MVTec AD for example (With multiple anomaly categories)

Structure of MVTec Folder:

mvtec/
│
├── meta.json
│
├── bottle/
│   ├── ground_truth/
│   │   ├── broken_large/
│   │   │   └── 000_mask.png
|   |   |   └── ...
│   │   └── ...
│   └── test/
│       ├── broken_large/
│       │   └── 000.png
|       |   └── ...
│       └── ...
│   
└── ...

cd generate_dataset_json
python mvtec.py

Run FB-CLIP

“Our pretrained weights are available at: https://drive.google.com/drive/folders/1YnNLLfdoQFNlvFB8ez_3XZ9Ox5ZrGDCa?usp=drive_link”

  • Quick start (use the pre-trained weights)
bash test.sh
  • Train your own weights
bash train_on_mvtec.sh
bash train_on_visa.sh

To Test Real-IAD

Due to the large size of the Real-IAD dataset, we recommend testing on a per-category basis.

bash test_realAD_by_class.sh
  • Our code is largely based on AnomalyCLIP and AF-CLIP. Thanks for these authors for their valuable work, hope our work can also contribute to related research.

Email: huming708@gmail.com