pytorch-deep-image-matting

This repository includes the non-official pytorch implementation of deep image matting.

Performance

model	SAD ↓	MSE ↓	Grad ↓	Conn ↓	link
stage0-paper	59.6	0.019	40.5	59.3
stage1-paper	54.6	0.017	36.7	55.3
stage0-our	56.01	0.0173	33.71	57.57
stage1-our	54.42	0.0175	35.01	54.85	download
stage1-our-skip	52.99	0.0171	31.56	53.24	download

• Lower metrics show better performance.
• Training batch=1, images=43100, epochs=12 for stage1-our-skip model, it takes about 1 days.
• Test maxSize=1600.
• GPU memory >= 10GB

Updates

• 2020.09.09: run the demo.py with the latest model (stage1-skip-sad-52.9.pth), and update the visualization results.
• 2020.09.09: adopt VGG-16 backbone with skip connection, and we get better performance with less training costs (12 epochs), Get Stage1-Skip-SAD=52.9
• 2019.10.29: conduct stage0 experiment using current code. Get Stage0-SAD=56.0.
• 2019.09.09: conv6 kernel size from 1x1 to 3x3. Get Stage1-SAD=54.4. The performance of stage1 is as good as paper. While using model released before this day, please change the kernel_size=1 and padding=0 of conv6 in file core/net.py.
• 2019.08.24: Fix cv2.dilate and cv2.erode iterations is set default = 1 and set triamp dilate and erode as the test 1k tirmap (k_size:2-5, iterations:5-15). Get Stage1-SAD=57.1.
• 2019.07.05: Training with refine stage, fixed encoder-decoder. Get Stage2-SAD=57.7.
• 2019.06.23: Training with alpha loss and composite loss. Get Stage1-SAD=58.7.
• 2019.06.17: Training trimap generated by erode as well as dialte to balance the 0 and 1 value. Get Stage0-SAD=62.0.
• 2019.04.22: Input image is normalized by mean=[0.485, 0.456, 0.406] and std=[0.229, 0.224, 0.225] and fix crop error. Get Stage0-SAD=69.1.
• 2018.12.14: Initial. Get Stage0-SAD=72.9.

Installation

• Python 2.7.12 or 3.6.5
• Pytorch 0.4.0 or 1.0.0
• OpenCV 3.4.3

Demo

Download our model to the ./model and run the following command. Then the predict alpha mattes will locate in the folder ./result/example/pred.

python core/demo.py

Training

Adobe-Deep-Image-Matting-Dataset

Please concat author for available.

MSCOCO-2017-Train-Dataset

Download

PASCAL-VOC-2012

Download

Composite-Dataset

Run the following command and the composite training and test dataset will locate in Combined_Dataset/Training_set/comp and Combined_Dataset/Test_set/comp, Combined_Dataset is the extracted folder of Adobe-Deep-Image-Matting-Dataset

python tools/composite.py

Pretrained-Model

Run the following command and the pretrained model will locate in ./model/vgg_state_dict.pth

python tools/chg_model.py

Start Training

Run the following command and start the training

bash train.sh

Test

Run the following command and start the test of Adobe-1k-Composite-Dataset

bash deploy.sh

Evaluation

Please eval with official Matlab Code. and get the SAD, MSE, Grad Conn.

Visualization

Running model is Stage1-Skip-SAD=52.9, please click to view whole images.

Image	Trimap	Pred-Alpha	GT-Alpha

Disclaimer

As covered by the ADOBE IMAGE DATASET LICENSE AGREEMENT, the pre-trained models included in this repository can only be used and distributed for non-commercial purposes.