3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis

CVPR 2024

Project Page | Arxiv

Zhicheng Lu¹*, Xiang Guo¹*, Le Hui¹, Tianrui Chen^1,2, Min Yang², Xiao Tang², Feng Zhu², Yuchao Dai¹

¹School of Electronics and Information, Northwestern Polytechnical University   ²Samsung R&D Institute

* Equal Contribution

Installation

conda env create --file environment.yml
conda activate gags
pip install einops open3d 

Install torchsparse according to the following link:

https://github.com/PJLab-ADG/OpenPCSeg/blob/master/docs/INSTALL.md

Usage

Data Preparation

Taking the D-NeRF synthetic dataset as an example, you can download the data from the following link: D-NeRF Dataset. You can download our pretrained model for D-NeRF Dataset Google Drive, and modify the source_path in <output/extpname/config.txt> and <output/extpname/cfd_args> to your DNeRF dataset directory.

Training

python train.py -s <DATA_DIR>  --eval \
     --port 4810 --expname 'bouncingballs' --voxelsize 0.005 

Rendering

After optimization, the numerical result can be evaluated via:

python render.py -m ./output/<OUT_DIR> --render_type 'metrics'

You can fix the viewpoint and obtain the changes of the scene over time:

python render.py -m ./output/<OUT_DIR> --render_type 'time' --frame_pose 5

Or you can fix time and obtain the changes of the scene over viewpoint:

python render.py -m ./output/<OUT_DIR> --render_type 'pose' 

Meanwhile, you can change time and viewpoint at the same time:

python render.py -m ./output/<OUT_DIR> --render_type 'time_pose' 

Evaluation

python metrics.py -m ./output/<OUT_DIR> 

Citation

@inproceedings{lu2024gagaussian,
  title={3D Geometry-aware Deformable Gaussian Splatting for Dynamic View Synthesis},
  author={Lu, Zhicheng and Guo, Xiang and Hui, Le and Chen, Tianrui and Yang, Ming and Tang, Xiao and Zhu, Feng and Dai, Yuchao},
  booktitle={Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2024}
}