GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning

Introduction

Zero-shot 3D Anomaly Detection (ZS3DAD) is an emerging task that aims to detect anomalies in a target dataset without any target training data, which is particularly important in scenarios constrained by sample scarcity and data privacy concerns. While current methods leverage CLIP by projecting 3D point clouds into 2D images, they suffer from lost geometric details during projection and incomplete visual understanding due to their reliance on a single 2D representation. To address these limitations, we propose the Geometry-Aware Prompt and Synergistic View Representation Learning (GS-CLIP) framework, which enables the model to identify geometric anomalies through a two-stage learning process.In the first stage, we dynamically generate text prompts embedded with 3D geometric priors. These prompts contain global shape context and local defect information identified by our Geometric Defect Distillation Module (GDDM). In the second stage, we introduce a Synergistic View Representation Learning architecture that processes rendered and depth images in parallel. A Synergistic Refinement Module (SRM) subsequently fuses the features of both streams, capitalizing on their complementary strengths. Comprehensive experimental results on four large-scale public datasets show that GS-CLIP achieves state-of-the-art performance in both object-level and point-level metrics, validating the effectiveness of our proposed method.

Overview

Prepare Dataset

Download the original dataset at Mvtec3D-AD, Eyecandies, Real3D-AD, Anomaly-ShapeNet

The rendering and depth images of Anomaly-ShapeNet are avalible at here.

The rendering images of MVTecAD-3D, Eyecandies, and Real3D-AD are avalible at this,

You can also genarate rendering and depth images through ./data_preprocess and this.

Generate the dataset JSON

Generate dataset json for training:

bash generate_dataset_json/generate_training_datasets_class_specific.sh

Generate dataset json for testing:

bash generate_dataset_json/generate_training_datasets_whole.sh

Download Pretrained Weight

Download the CLIP weights pretrained by OpenAI [ViT-L-14-336.pt].

Download the PointNet++ initial weights at here .

Put them to ./pretrained_weights/

Create Environments

conda create -n gsclip python=3.9
conda activate gsclip
pip install -r requirements.txt

Train and Test

The two-stage training and test are included in train2.sh:

Change the data path and run this script:

bash train2.sh

• We thank for the code repository: PointAD, AnomalyCLIP

Citation

If you find our work useful, please cite us. Thank you.

@article{deng2026gs,
  title={GS-CLIP: Zero-shot 3D Anomaly Detection by Geometry-Aware Prompt and Synergistic View Representation Learning},
  author={Deng, Zehao and Liu, An and Wang, Yan},
  journal={arXiv preprint arXiv:2602.19206},
  year={2026}
}