MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps

Created by Yating Xu from National University of Singapore.

Introduction

This repository contains the PyTorch implementation for NeurIPS 2024 work MVSDet: Multi-View Indoor 3D Object Detection via Efficient Plane Sweeps. Training and testing are conducted on two A5000 (48GB) GPUs.

Installation

• Install mmdetection3d
• Install packages related to Gaussian Splatting: we install pixelsplat.
• Install torch-scatter: pip install torch-scatter==2.1.2 -f https://data.pyg.org/whl/torch-2.1.0%2Bcu118.html

Dataset

ScanNet

We follow this instruction to prepare ScanNet data.

ARKitScenes

We follow CN-RMA to prepare ARKitScenes data.

Train

CUDA_VISIBLE_DEVICES=0,1  bash tools/dist_train.sh projects/NeRF-Det/configs/mvsdet_res50_2x_low_res.py 2 --log_dir ModelName

Test

CUDA_VISIBLE_DEVICES=0,1  bash tools/dist_test.sh projects/NeRF-Det/configs/mvsdet_res50_2x_low_res.py path/to/checkpoint.pth 2

Acknowledgement

We thank mmdetection3d, PixelSplat and MVSNet for sharing their source code.