README.md

Installation

• Python 3.10
• CUDA >= 11.6

pip install -r requirements.txt

Data Preparation

DTU Dataset

We use DTU training data processed by PatchmatchNet for training, each dataset is organized as follows:

dtu
├── scan1 (scene_name1)
├── scan2 (scene_name2) 
│     ├── cams (camera parameters)
│     │   ├── 00000000_cam.txt   
│     │   ├── 00000001_cam.txt   
│     │   └── ...                
│     ├── depth_gt (ground truth depth maps)
│     │   ├── 00000000.pfm   
│     │   ├── 00000001.pfm   
│     │   └── ...                
│     ├── images (images at 7 light indexes) 
│     │   ├── 0 (light index 0)
│     │   │   ├── 00000000.jpg       
│     │   │   ├── 00000001.jpg
│     │   │   └── ...
│     │   ├── 1 (light index 1)
│     │   └── ...                
│     ├── masks (depth map masks) 
│     │   ├── 00000000.png       
│     │   ├── 00000001.png       
│     │   └── ...                
│     └── pair.txt
└── ...

Camera file cam.txt stores the camera parameters, which includes extrinsic, intrinsic, minimum depth and maximum depth:

extrinsic
E00 E01 E02 E03
E10 E11 E12 E13
E20 E21 E22 E23
E30 E31 E32 E33

intrinsic
K00 K01 K02
K10 K11 K12
K20 K21 K22

DEPTH_MIN DEPTH_MAX

Please pay attention to the last line of data: Depth_min Depth_max, which is not Depth_min Depth_interval!

We use DTU testing data processed by PatchmatchNet for testing. The resolution of the testing images are 1600x1200, we scale the images to 1200x900 by setting --image_max_dim 1200 in eval.sh for final testing.

BlendedMVS Dataset

BlendedMVS(low-res) for finetuning. The structure is just like:

blendedmvs                          
├── 5a3ca9cb270f0e3f14d0eddb
│     ├── blended_images
│     │    ├── 00000000.jpg
│     │    ├── 00000000_masked.jpg 
│     │    ├── 00000001.jpg
│     │    ├── 00000001_masked.jpg  
│     │    └── ...  
│     ├── cams
│     │    ├── 00000000_cam.txt   
│     │    ├── 00000001_cam.txt   
│     │    └── ...  
│     └── rendered_depth_maps
│          ├── 00000000.pfm   
│          ├── 00000001.pfm   
│          └── ...       
├── 5a3cb4e4270f0e3f14d12f43   
└── ...

Tanks&Temples Dataset

Download the intermediate and advanced subsets of Tanks&Temples Dataset pre-processed by MVSNet. Each dataset is organized as follows:

tanksandtemples
├── advanced
│   ├── ...
│   └── Temple
│       ├── cams_1
│       ├── images
│       ├── pair.txt
│       └── Temple.log
└── intermediate
    ├── ...
    └── Train
        ├── cams_1
        ├── images
        ├── pair.txt
        └── Train.log

Submit the results to the Tanks & Temples benchmark website to receive the F-score.