Open-Vocabulary DETR with Conditional Matching

arXiv | Project Page | Code

This repository contains the implementation of the following paper:

Open-Vocabulary DETR with Conditional Matching
Yuhang Zang, Wei Li, Kaiyang Zhou, Chen Huang, Chen Change Loy
European Conference on Computer Vision (ECCV), 2022

Installation

We use the same environment as Deformable DETR. You are also required to install the following packages:

• CLIP
• cocoapi
• lvis-api

We test our models under python=3.8, pytorch=1.11.0, cuda=10.1, 8 Nvidia V100 32GB GPUs.

Data

Please refer to dataset_prepare.md.

Running the Model

Please refer to run_scripts.md.

Model Zoo

• Open-vocabulary COCO (AP50 metric)

Base	Novel	All	Model
61.0	29.4	52.7	Google Drive

Citation

If you find our work useful for your research, please consider citing the paper:

@InProceedings{zang2022open,
 author = {Zang, Yuhang and Li, Wei and Zhou, Kaiyang and Huang, Chen and Loy, Chen Change},
 title = {Open-Vocabulary DETR with Conditional Matching},
 journal = {European Conference on Computer Vision},
 year = {2022}
}

License

This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Acknowledgement

We would like to thanks Deformable DETR, CLIP and ViLD for their open-source projects.

Contact

Please contact Yuhang Zang if you have any questions.