Visual Parser: Representing Part-whole Hierarchies with Transformers

This repository contains the official implementation to reproduce object detection results of ViP. It is based on mmdetection.

Results and Models

Cascade Mask R-CNN

Backbone	Pretrain	Lr Schd	box mAP	mask mAP	#params	FLOPs	config	log	model
ViP-Ti	ImageNet-1K	1x	45.3	39.8	69.2M	678G	config	Google Drive	Google Drive
ViP-S	ImageNet-1K	1x	48.0	42.0	87.1M	725G	config	Google Drive	Google Drive
ViP-M	ImageNet-1K	1x	49.9	43.5	107.0M	785G	-	-	Coming Soon

RetinaNet

Backbone	Pretrain	Lr Schd	box mAP	#params	FLOPs	config	log	model
ViP-Ti	ImageNet-1k	1x	39.9	21.4M	181G	config	Google Drive	Google Drive
ViP-S	ImageNet-1k	1x	42.7	39.9M	227G	config	Google Drive	Google Drive
ViP-S	ImageNet-1k	3x	43.9	39.9M	227G	config	Google Drive	Google Drive
ViP-M	ImageNet-1k	1x	44.3	59.8M	287G	-	-	Coming Soon

Notes:

• Pre-trained models can be downloaded from Visual Parser.

Usage

Installation

Please refer to get_started.md for installation and dataset preparation.

Inference

# single-gpu testing
python tools/test.py <CONFIG_FILE> <DET_CHECKPOINT_FILE> --eval bbox segm

# multi-gpu testing
tools/dist_test.sh <CONFIG_FILE> <DET_CHECKPOINT_FILE> <GPU_NUM> --eval bbox segm

Training

To train a detector with pre-trained models, run:

# single-gpu training
python tools/train.py <CONFIG_FILE>

# multi-gpu training
tools/dist_train.sh <CONFIG_FILE> <GPU_NUM>

Citing ViP

@article{sun2021visual,
  title={Visual Parser: Representing Part-whole Hierarchies with Transformers},
  author={Sun, Shuyang and Yue, Xiaoyu, Bai, Song and Torr, Philip},
  journal={arXiv preprint arXiv:2107.05790},
  year={2021}
}