README.md

April 7, 2025 · View on GitHub

IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera

Jian Huang1,2    Chengrui Dong1,2    Xuanhua Chen2,3    Peidong Liu2*

* denotes corresponding author.

1Zhejiang University    2Westlake University    3Northeastern University   

If you like our project, please give us a star ⭐ on GitHub.

arXiv

This repository is the official implementation of the paper "IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera".

📢 News

2025.04.04 🔥Our paper has been selected as CVPR 2025 Highlight!

2025.03.03 🔥Our paper has been accepted by CVPR 2025!

2024.10.11 Our paper is available on arXiv.

📋 Overview

Pipeline

IncEventGS processes incoming event stream by dividing it into chunks and representing the camera trajectory as a continuous model. It randomly samples two close consecutive timestamps to integrate the corresponding event streams. Two brightness images are rendered from 3D Gaussian distributions at the corresponding poses, and we minimize the log difference between the rendered images and the accumulated event images. During initialization, a pre-trained depth estimation model estimates depth from the rendered images to bootstrap the system.

📋 Qualitative evaluation of novel view image synthesis on synthetic dataset.

nvs_synthetic

📋 Qualitative evaluation of novel view image synthesis on real dataset.

nvs_real

📋 Representative trajectory comparison

traj

QuickStart

1. Installation

In the path where your want to store code, enter the following terminal command:

conda create -n iegs python=3.10
conda activate iegs
conda install -c "nvidia/label/cuda-11.8.0" cuda-toolkit # install CUDA toolkit
# Install the pytorch first (Please check the cuda version)
conda install pytorch torchvision torchaudio pytorch-cuda=11.8 -c pytorch -c nvidia

pip install -r requirements.txt

pip install ninja git+https://github.com/NVlabs/tiny-cuda-nn/#subdirectory=bindings/torch

pip install h5py hdf5plugin scikit-learn jaxtyping kornia

pip install -U  "huggingface-hub<0.26.0"
pip install pytorch_msssim
pip install tyro
pip install diffusers==0.27.1
pip install accelerate
pip install -U "transformers<=4.48.1"

2. Download Weights and Datasets

You can download pretrained weight of Marigold and datasets (Replica, pre-processed TUMVIE) by clicking this link.

# unzip files
tar -xvf xxx.tar

3. Running demo

Before that, need to modify the depthEst_pretrained_model_path and data:datadir

python main.py --config configs/SimuEvent/replica_room0.yaml

4. Metrics Calculation

  • NVS Performance
cd metrics/event_nerf
sh auto_single.sh [PATH_TO_IMG_EVAL] est gt

  • Pose Accuracy

    • install evo, pip install evo
    • run:
    cd [PATH_TO_EXPERIMENT]/global_BA
evo_ape kitti [EST_PATH_NAME] [GT_PATH_NAME] --align -as

5. Dataset Format

Our Replica(event) dataset format is described as follows. It is recommended that you organize your custom datasets in this format if you want to run our method on your own datasets.

<dataset_name>              # e.g. room0
├── poses_ts.txt
├── traj.txt
├── images
    ├── frame000100.jpg   
    ├── frame000150.jpg
    ├── frame000200.jpg
    ...
├── event_threshold_0.1
    ├── gray_events_data.npy
  • traj.txt format
r11 r12 r13 r14 r21 r22 r23 r24 r31 r32 r33 r34 r41 r42 r43 r44

r is transformation matrix (4*4).

Citation

If you find this repository useful, please consider citing our paper:

@inproceedings{huang2025inceventgs,
    author = {Huang, Jian and Dong, Chengrui and Chen, Xuanhua and Liu, Peidong},
    title = {IncEventGS: Pose-Free Gaussian Splatting from a Single Event Camera},
    booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition},
    year = {2025}
}

Acknowledgments

This project is developed with several fantastic repos: gsplat, Co-SLAM, E-NeRF, EventNeRF, BAD-Gaussians, Marigold, Diffusers and PyPose. We appreciate the effort of the contributors to these amazing repositories.