Gaga: Group Any Gaussians via 3D-aware Memory Bank

📖 TMLR 2026 📖

Weijie Lyu, Xueting Li, Abhijit Kundu, Yi-Hsuan Tsai, Ming-Hsuan Yang
University of California, Merced - NVIDIA Reaserch - Google DeepMind - Atmanity Inc.

Gaga groups any Gaussians in an open-world 3D scene and renders multi-view consistent class-agnostic segmentation.

Usage

Please refer to USAGE.md for installation and usage.

Results

🗺️ Open-world 3D Segmentation

MipNeRF 360

https://github.com/weijielyu/Gaga/assets/47323245/62a7ff01-30da-4c5e-ab79-c60c091935c1

Replica

https://github.com/weijielyu/Gaga/assets/47323245/d0d5eece-c838-4be9-a3b9-71d051e97270

ScanNet

https://github.com/weijielyu/Gaga/assets/47323245/f1099e6b-40e1-46af-9c14-f480765065bf

🖌️ Scene Editing

✨ Change the color of cushion on to maroon 🟥
✨ Remove

https://github.com/weijielyu/Gaga/assets/47323245/803f049f-8930-445c-bc1a-b8bb12df0fbf

Citation

If you find our work useful for your project, please consider citing our paper.

@misc{lyu2024gaga,
      title={Gaga: Group Any Gaussians via 3D-aware Memory Bank}, 
      author={Weijie Lyu and Xueting Li and Abhijit Kundu and Yi-Hsuan Tsai and Ming-Hsuan Yang},
      year={2024},
      eprint={2404.07977},
      archivePrefix={arXiv},
      primaryClass={cs.CV}
}