SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection (ECCV 2022)

Authors: Minhyeok Lee, Chaewon Park, Suhwan Cho, Sangyoun Lee

This repository provides code for paper "SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection" accepted by the ECCV 2022 conference.

Our paper can be found [arXiv]

Prepared Datasets

Download the train and test dataset from Google Drive and set the data_root path in config.py.

Requirements

For the superpixel algorithm we use fast_slic. You can install it like this:

pip install fast_slic

Training Model

1. First, clone this repository.

git clone https://github.com/Hydragon516/SPSN

2. Edit config.py. The data root path option and GPU index should be modified.
3. Train the model. The model is trained with DDP (DistributedDataParallel) on all GPUs listed in config.py.

python3 train.py

Evaluation

When training is complete, the prediction results for the test set are saved in the ./log folder. Two popular evaluation toolboxes are available. (Matlab version: https://github.com/DengPingFan/CODToolbox Python version: https://github.com/lartpang/PySODMetrics)

Result

./log directory structure

.
├── root
└── log/
    └── 2022-xx-xx xx:xx:xx/
        ├── model/
        │   └── best_model.pth
        ├── result/
        │   ├── gt/        # ground truth images
        │   │   ├── NJU2K
        │   │   ├── NLPR
        │   │   ├── DES
        │   │   ├── SIP
        │   │   └── STERE
        │   ├── pred/      # predicted mask images (only mask)
        │   │   ├── NJU2K
        │   │   ├── NLPR
        │   │   ├── DES
        │   │   ├── SIP
        │   │   └── STERE
        │   └── total/     # includes RGB, depth, GT, superpixel sampling maps, prediction mask, and more
        │       ├── NJU2K
        │       ├── NLPR
        │       ├── DES
        │       ├── SIP
        │       └── STERE
        └── train/
            └── config.py

An example of the resulting image is shown below.

• A : RGB image
• B : Depth map
• C : Pred map
• D : GT
• E : Pred superpixel map from RGB
• F : GT superpixel map from RGB
• G : Pred superpixel map from depth
• H : GT superpixel map from depth

The prediction mask results of our proposed model can be found here.

Citation

@article{lee2022spsn,
  title={SPSN: Superpixel Prototype Sampling Network for RGB-D Salient Object Detection},
  author={Lee, Minhyeok and Park, Chaewon and Cho, Suhwan and Lee, Sangyoun},
  journal={arXiv preprint arXiv:2207.07898},
  year={2022}
}