PointGPT for KITTI and nuScenes Datasets

• ArXiv
• Original Repository

PointGPT: Auto-regressively Generative Pre-training from Point Clouds

• Unofficial implementation of PointGPT, a novel auto-regressive generative pre-training framework for point clouds.
• The code is based on PyTorch and is designed to be easy to use and extend for autonomous driving applications.

1. Requirements

PyTorch >= 1.7.0; python >= 3.7; CUDA >= 9.0; GCC >= 4.9; torchvision;

pip install -r requirements.txt

# Chamfer Distance & emd
cd ./extensions/chamfer_dist
python setup.py install --build_ext --inplace
cd ./extensions/emd
python setup.py install --build_ext --inplace
# PointNet++
pip install "git+https://github.com/erikwijmans/Pointnet2_PyTorch.git#egg=pointnet2_ops&subdirectory=pointnet2_ops_lib"
# GPU kNN
pip install --upgrade https://github.com/unlimblue/KNN_CUDA/releases/download/0.2/KNN_CUDA-0.2-py3-none-any.whl

2. Datasets

See guidance for preparation of KITTI and nuScenes datasets.

3. Pre-training

PYTHON_PATH="/home/bit/anaconda3/envs/pytorch/bin/python"
SCRIPT_PATH="main.py"
CONFIG="cfgs/PointGPT-S/pretrain_kitti_tiny.yaml"
EXP_NAME="kitti_tiny"
echo "Run Exp: ${EXP_NAME}"
${PYTHON_PATH} ${SCRIPT_PATH} "--config" ${CONFIG} "--exp_name" ${EXP_NAME} >> debug.log

Explanation of variables:

• PYTHON_PATH: The path to the Python executable in your environment. Activate your environment and use which python to find the correct path.
• SCRIPT_PATH: The path to the main training script.
• CONFIG: The path to the YAML configuration file that contains the training settings. For a 12 GB GPU, use pretrain_kitti_tiny.yaml to train a PointGPT-S model.
• EXP_NAME: The name of the experiment, which will be used for logging and saving checkpoints. For the above configuration, log files and checkpoints will be saved in ./experiments/pretrain_kitti_tiny/PointGPT-S/kitti_tiny/.

4. Reference

@article{chen2024pointgpt,
  title={Pointgpt: Auto-regressively generative pre-training from point clouds},
  author={Chen, Guangyan and Wang, Meiling and Yang, Yi and Yu, Kai and Yuan, Li and Yue, Yufeng},
  journal={Advances in Neural Information Processing Systems},
  volume={36},
  year={2024}
}