DATA.md

Format of the prepare data is as follows:

other_data/
└── category/
    ├── 4_pointcloud     # point clouds sampled from CAD annotation
    ├── 6_images         # images that contains this images
    ├── 8_proj_matrix    # project matrix, that project 3d points to 2D image coordinate
    └── 5_partial_points/        # partial point clouds cropped from laser scan and rgb-d scan
        └── cadid/
                ├── highres_partial_points_0.ply    # point clouds cropped from laser scan
                └── lowres_partial_points_0.ply    	# point clouds cropped from rgb-d scan
 
occ_data/
└── category/
        ├── cadid.npz   # contains samples and occupancy data 
        └── cadid.npy   # contains scale information (default is 1, except for ShapeNet dataset)

The category and sub-category mapping is as followed:

Put occ_data other_data under /LASA/submodules/DisCo/data, put open_clip_pytorch_model.bin under /LASA/submodules/DisCo/data. Currently, the synthetic dataset such as ShapeNet, ABO, and 3D-FUTURE only provide preprocessed data for download. Then, run the following command to unzip all the data in occ_data and other_data by following commands (PS. make sure unrar and 7zip is installed):

cd LASA/process_scripts
python unzip_all_data.py --unzip_occ --unzip_other

Run the following commands to generate augmented partial point cloud for synthetic dataset and LASA dataset

cd LASA/process_scripts
python augment_arkit_partial_points.py
python augment_synthetic_partial_points.py

Finally, run the following command to generate train/val splits, please check ./dataset/taxonomy for the sub-cateory definition, and run all the categories that you want to use.

cd LASA/process_scripts
python generate_split_for_arkit.py
python generate_split_for_synthetic_data.py

cd LASA/arkitscene_process_script
CUDA_VISIBLE_DEVICES='0,1,2,3' torchrun --master_port 15000 --nproc_per_node=4 \
SR_images.py --image_dir <path_to_arkit_dataset/images>

Then, run the following command to crop and pad the images, meanwhile recompute the projection matrix, so that the points can be projected to the crop images directly using this matrix:

cd LASA/arkitscene_process_script
python crop_arkit_images.py --image_dir <path_to_arkit_dataset/images> --lasa_dir <path_to_LASA_dataset> \
--arkit_dir <path_to_arkit_dataset/3dod> --consider_alignment

The mesh annotation is originally aligned with the Laser scan, and might be slightly misaligned with the RGB-D scan. Therefore, we set --consider_alignment flag, which is used to further align the annotation with the RGB-D scan. (In most of the cases, RGB-D mesh and laser point clouds are well aligned.)