Extracting Databases

RealEstate-10K

Currently, the code to extract RealEstate-10K dataset is not complete. This will be completed soon. In the meantime, please down the dataset following instructions at the official webpage and follow the instructions for custom databases. For camera extrinsincs, our data-loader uses the same convention (world-to-camera in (x, -y, -z) format) as the original dataset for camera extrinsics. So you only need to copy the camera parameters to appropriate csv files (CameraIntrinsics.csv (9 columns per row) and CameraExtrinsics.csv (16 columns per row)) and the video frames to rgb folder.

NeRF-LLFF

1. Download the nerf_llff_data.zip file from original release in google drive. Place the downloaded file at Data/databases/NeRF_LLFF/data/all/nerf_llff_data.zip.
2. Run the data extractor file:

cd src/database_utils/nerf_llff/data_organizers/
python DataExtractor01.py

3. Train/test configs are already provided in the github repository. In case you want to create them again:

python TrainTestCreator01_UniformSparseSampling.py
python VideoPoseCreator01_Spiral.py

4. Return to root directory

cd ../../../../

DTU

1. Download the dataset provided by pixelNeRF here.
2. Unzip the downloaded file and place the unzipped data in data/databases/DTU/data/all/unzipped_data/PixelNeRF/
   The path of scan1 should be data/databases/DTU/data/all/unzipped_data/PixelNeRF/rs_dtu_4/DTU/scan1
3. Extract the data:

cd src/database_utils/dtu/data_organizers/
python DataExtractor01_PixelNeRF.py

4. Download the object masks data provided by RegNeRF here.
5. Place the downloaded files in data/databases/DTU/data/all/unzipped_data/RegNeRF
   The path of scan8 should be data/databases/DTU/data/all/unzipped_data/RegNeRF/idrmasks/scan8
6. Extract the object masks data:

python DataExtractor02_RegNeRF.py

7. Train/test configs are already provided in the github repository. In case you want to create them again:

python TrainTestCreator01_PixelNeRF.py
python TrainTestCreator02_PixelNeRF.py

To use different train/test sets, modify the above files accordingly and regenerate the train/test configs. 8. Return to root directory

cd ../../../../

Custom Databases

We use the Open CV convention: (x, -y, -z) world-to-camera format to store the camera poses. The camera intrinsics and extrinsics are stored in the csv format after flattening them, i.e., if a scene contains 50 frames, intrinsics and extrinsics are stores as csv files with 50 rows each and 9 & 16 columns respectively. The directory tree in the following shows an example. Please refer to one of the data-loaders for more details. Organize your custom dataset in accordance with the data-loader or write the data-loader file to load the data directly from your custom database format.

Example directory tree:

<DATABASE_NAME>
 |--data
    |--all
    |  |--database_data
    |     |--scene0001
    |     |  |--rgb
    |     |  |  |--0000.png
    |     |  |  |--0001.png
    |     |  |  |-- ...
    |     |  |  |--0049.png
    |     |  |--CameraExtrinsics.csv
    |     |  |--CameraIntrinsics.csv
    |     |--scene0002
    |     | ...
    |--train_test_sets

Our code also requires a config file specify the train/validation/test images. Please look into train-test-creators and replicate a similar file for your custom dataset.