PaddleStereo: A Unified Framework for Stereo Matching

May 13, 2023 · View on GitHub

English | 简体中文

A lightweight, easy-to-extend, easy-to-learn, high-performance, and for-fair-comparison toolkit based on PaddlePaddle for Stereo Matching. It is a part of the Paddledepth project.

Implemented Algorithms

As initial version, we support the following algoirthms. We are working on more algorithms. Of course, you are welcome to add your algorithms here.

  1. PCWNet (ECCV2022)[1]
  2. PSMNet (CVPR2018)[2]
  3. RAFT-Stereo (3DV2021)[3]

Please click the hyperlink of each algorithm for more detailed explanation.

Installation

You can either git clone this whole repo by:

  • Install PaddlePaddle
    • Version requirements: PaddlePaddle>=2.3.2, Python>=3.8
    • Multi-GPU Version need suitable cuda,cudnn,nccl version. see the official website for more detailed explanation.
git clone https://github.com/PaddlePaddle/PaddleDepth
cd PaddleDepth/Paddlestereo
conda env create -f environment.yml
conda activate paddle_env

Dataset

see guidance in dataset_prepare for dataset preparation.

Usage

Train

  1. Pre-training
$ ./Scripts/start_train_sceneflow_stereo_net_multi.sh
  1. Fine-tuning (KITTI 2012)
$ ./Scripts/start_train_kitti2012_stereo_net_multi.sh

Test

  1. KITTI2012
$ ./Scripts/start_test_kitti2012_stereo_net.sh

Customization

The file structure of PaddleStereo is shown below:

PaddleStereo
    │- data_prepare
    │- model_document
    │- README.md 
    │- environment.yml
    │- Datasets
    │- Example
    │- Scripts
    └─ Source
       │- Algorithm
       │- Core
       │- FileHandler
       │- ImgHandler
       │- SysBasic
       │- Template
       │- Tools
       └─ UserModelImplementation
          │- Dataloaders
          └─ Models

It is easy to design your own method following the 3 steps:

  1. Check and write your own model's to Models
  2. Check and write your own dataloder's to Dataloders
  3. Write your own start file (.sh) to Scripts

Results

We present results of our implementations on 2 popular benchmarks: KITTI2015 and KITTI2012. We did not perform careful parameter tuning and simply used the default config files. You can easily reproduce our results using provided shell scripts!

KITTI2015

Contribution

The toolkit is under active development and contributions are welcome! Feel free to submit issues or emails to ask questions or contribute your code. If you would like to implement new features, please submit a issue or emails to discuss with us first.

Acknowledgement

PaddleDepth is an open source project that is contributed by researchers and engineers from various colleges and companies. We appreciate all the contributors who implement their methods or add new features, as well as users who give valuable feedbacks. We wish that the toolbox and benchmark could serve the growing research community by providing a flexible toolkit to reimplement existing methods and develop their new algorithms.

References

[1] Shen, Zhelun, et al. "PCW-Net: Pyramid Combination and Warping Cost Volume for Stereo Matching." European Conference on Computer Vision. 2022.

[2] Chang, Jia-Ren, and Yong-Sheng Chen. "Pyramid stereo matching network." Proceedings of the IEEE conference on computer vision and pattern recognition. 2018.

[3] Lipson, Lahav, Zachary Teed, and Jia Deng. “RAFT-Stereo: Multilevel Recurrent Field Transforms for Stereo Matching.” arXiv, September 15, 2021.

Contact