BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation;
Hao Chen, Kunyang Sun, Zhi Tian, Chunhua Shen, Yongming Huang, and Youliang Yan;
In: Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2020.

[Paper] [BibTeX]

This project contains training BlendMask for instance segmentation and panoptic segmentation on COCO and configs for segmenting persons on PIC.

Quick Start

Demo

wget -O blendmask_r101_dcni3_5x.pth https://cloudstor.aarnet.edu.au/plus/s/vbnKnQtaGlw8TKv/download
python demo/demo.py \
    --config-file configs/BlendMask/R_101_dcni3_5x.yaml \
    --input datasets/coco/val2017/000000005992.jpg \
    --confidence-threshold 0.35 \
    --opts MODEL.WEIGHTS blendmask_r101_dcni3_5x.pth

Training and evaluation

To train a model with "train_net.py", first setup the corresponding datasets following datasets/README.md,

Then follow these steps to generate blendmask format annotations for instance segmentation.

then run:

OMP_NUM_THREADS=1 python tools/train_net.py \
    --config-file configs/BlendMask/R_50_1x.yaml \
    --num-gpus 4 \
    OUTPUT_DIR training_dir/blendmask_R_50_1x

To evaluate the model after training, run:

OMP_NUM_THREADS=1 python tools/train_net.py \
    --config-file configs/BlendMask/R_50_1x.yaml \
    --eval-only \
    --num-gpus 4 \
    OUTPUT_DIR training_dir/blendmask_R_50_1x \
    MODEL.WEIGHTS training_dir/blendmask_R_50_1x/model_final.pth

Models

COCO Instance Segmentation Baselines

Model	Name	inf. time	box AP	mask AP	download
Mask R-CNN	R_50_1x	13 FPS	38.6	35.2
BlendMask	R_50_1x	14 FPS	39.9	35.8	model
Mask R-CNN	R_50_3x	13 FPS	41.0	37.2
BlendMask	R_50_3x	14 FPS	42.7	37.8	model
Mask R-CNN	R_101_3x	10 FPS	42.9	38.6
BlendMask	R_101_3x	11 FPS	44.8	39.5	model
BlendMask	R_101_dcni3_5x	10 FPS	46.8	41.1	model

BlendMask Real-time Models

Model	Name	inf. time	box AP	mask AP	download
Mask R-CNN	550_R_50_3x	16 FPS	39.1	35.3
BlendMask	550_R_50_3x	28 FPS	38.7	34.5	model
BlendMask	RT_R_50_4x_syncbn_shtw	31 FPS	39.3	35.1	model
BlendMask	RT_R_50_4x_bn-head_syncbn_shtw	31 FPS	39.3	35.1	model
BlendMask	DLA_34_4x	32 FPS	40.8	36.3	model

COCO Panoptic Segmentation Baselines with BlendMask

Model	Name	PQ	PQTh	PQSt	download
Panoptic FPN	R_50_3x	41.5	48.3	31.2
BlendMask	R_50_3x	42.5	49.5	32.0	model
Panoptic FPN	R_101_3x	43.0	49.7	32.9
BlendMask	R_101_3x	44.3	51.6	33.2	model
BlendMask	R_101_dcni3_5x	46.0	52.9	35.5	model

Citing BlendMask

If you use BlendMask in your research or wish to refer to the baseline results, please use the following BibTeX entries.

@inproceedings{chen2020blendmask,
  title     =  {{BlendMask}: Top-Down Meets Bottom-Up for Instance Segmentation},
  author    =  {Chen, Hao and Sun, Kunyang and Tian, Zhi and Shen, Chunhua and Huang, Yongming and Yan, Youliang},
  booktitle =  {Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR)},
  year      =  {2020}
}