MoECLIP: Patch-Specialized Experts for Zero-shot Anomaly Detection

This is the official PyTorch implementation of:

MoECLIP: Patch-Specialized Experts for Zero-shot Anomaly Detection

CVPR 2026

Abstract

  The CLIP model's outstanding generalization has driven recent success in Zero-Shot Anomaly Detection (ZSAD) for detecting anomalies in unseen categories. The core challenge in ZSAD is to specialize the model for anomaly detection tasks while preserving CLIP's powerful generalization capability. Existing approaches attempting to solve this challenge share the fundamental limitation of a patch-agnostic design that processes all patches monolithically without regard for their unique characteristics. To address this limitation, we propose \textbf{MoECLIP}, a Mixture-of-Experts (MoE) architecture for the ZSAD task, which achieves patch-level adaptation by dynamically routing each image patch to a specialized Low-Rank Adaptation (LoRA) expert based on its unique characteristics. Furthermore, to prevent functional redundancy among the LoRA experts, we introduce (1) Frozen Orthogonal Feature Separation (FOFS), which orthogonally separates the input feature space to force experts to focus on distinct information, and (2) a simplex equiangular tight frame (ETF) loss to regulate the expert outputs to form maximally equiangular representations. Comprehensive experimental results across 14 benchmark datasets spanning industrial and medical domains demonstrate that MoECLIP outperforms existing state-of-the-art methods.

The framework of MoECLIP

  MoE is integrated into multiple layers of the CLIP Vision Encoder, enabling dynamic expert routing for each image patch to learn patch-specific representations for ZSAD. Within each MoE, FOFS enforces expert specialization by orthogonally separating the feature space and ETF loss further enhances expert diversity by maximizing the equiangular separation of expert outputs. PAA then aggregates the refined patch features across multiple scales to capture anomalies of different sizes.MoE is integrated into multiple layers of the CLIP Vision Encoder, enabling dynamic expert routing for each image patch to learn patch-specific representations for ZSAD. Within each MoE, FOFS enforces expert specialization by orthogonally separating the feature space and ETF loss further enhances expert diversity by maximizing the equiangular separation of expert outputs. PAA then aggregates the refined patch features across multiple scales to capture anomalies of different sizes.

Quick Start

1. Installation

cd MoECLIP
conda create -n moeclip python=3.10.18 -y  
conda activate moeclip  
pip install -r requirements.txt  

2. Dataset & Path

Download the dataset:

• Industrial Domain: MVTec, VisA, BTAD, RSDD, DTD-Synthetic

• Medical Domain: BrainMRI, HeadCT, LiverCT, RetinaOCT, ColonDB, ClinicDB, CVC-300, Endo, Kvasir

Please update the "BASE_PATH" in ./dataset/constants to match your data directory To run the code, download the OpenCLIP ViT-L-14-336px weights and place them in the ./model/ directory

3. Training & Evaluation

# training
python train.py
# evaluation
python test.py
# (Optional) bash script for training and evaluating all the datasets
bash scripts.sh

Comparison with State-of-the-art methods

  Comparison with state-of-the-art methods across industrial and medical domains under the ZSAD setting. The symbol † indicates results obtained from models re-trained under our setting. The best performance is in bold and the second-best is underlined.

Visualization of Grad-CAM and Patch Selection Map

  Visualization of Grad-CAM and Patch Selection Map for each expert at layer 18. The Ground Truth image is shown on the far left. The top row (Grad-CAM) highlights each expert's focus region. The bottom row (Patch Selection) illustrates the patches where the corresponding expert was the router's Top-1 choice (shown in green). The value in each subplot title represents the expert's average renormalized routing weight based on the Top-1 setting for its Top-1 assigned patches.

Visualization of Anomaly Map

• Bottle(MVTec)

  

• Capsule(VisA)

  

• Brain MRI

  

• ClinicDB

  

BibTex Citation

If you find this work helpful, please consider citing the following BibTeX entry.

@InProceedings{Park_2026_CVPR,
    author    = {Park, Jun Yeong and Seo, JunYoung and Kang, Minji and Park, Yu Rang},
    title     = {MoECLIP: Patch-Specialized Experts for Zero-shot Anomaly Detection},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
    month     = {June},
    year      = {2026},
    pages     = {35534-35544}
}