SOTR: Segmenting Objects with Transformers [ICCV 2021]

October 13, 2021 · View on GitHub

By Ruohao Guo, Dantong Niu, Liao Qu, Zhenbo Li

Introduction

This is the official implementation of SOTR.

Models

COCO Instance Segmentation Baselines with SOTR

Name	mask AP	AP₅₀	AP₇₅	AP_S	AP_M	AP_L	download
SOTR_R50	39.6	60.7	42.6	10.3	58.7	72.1	model
SOTR_R101	40.2	61.2	43.4	10.2	59.0	73.1	model
SOTR_R101_DCN	42.0	63.3	45.5	11.4	60.7	74.5	model

Note: The area of AP_S, AP_M and AP_L are calculated by segmentation mask without using bbox information.

Installation & Quick start

First install Detectron2 following the official guide: INSTALL.md.
Then build SOTR with:

https://github.com/easton-cau/SOTR
cd SOTR
python setup.py build develop

Then follow datasets/README.md to set up the datasets (e.g., MS-COCO).

Evaluating

Download the trained models for COCO.

Run the following command

python tools/train_net.py \
    --config-file configs/SOTR/R101.yaml \
    --eval-only \
    --num-gpus 4 \
    MODEL.WEIGHTS work_dir/SOTR_R101/SOTR_R101.pth

Training

Run the following command

python tools/train_net.py \
    --config-file configs/SOTR/R101.yaml \
    --num-gpus 4 \

Acknowledgement

Thanks Detectron2 and AdelaiDet contribution to the community!

The work is supported by National Key R&D Program of China (2020YFD0900204) and Key-Area Research and Development Program of Guangdong Province China (2020B0202010009).

FAQ

If you want to improve the usability or any piece of advice, please feel free to contant directly (ruohguo@foxmail.com).

Citation

Please consider citing our paper in your publications if the project helps your research. BibTeX reference is as follow.

@inproceedings{guo2021sotr,
  title={SOTR: Segmenting Objects with Transformers},
  author={Guo, Ruohao and Niu, Dantong and Qu, Liao and Li, Zhenbo},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  pages={7157--7166},
  year={2021}
}