README.md

View on GitHub

MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction

Arxiv Home Page Code

Jiacheng Dong2*, Huan Li1*, Sicheng Zhou2*, Wenhao Hu2, Weili Xu2, Yan Wang1†

1Institute for AI Industry Research, Tsinghua University 2Zhejiang University

*Equal contribution. †Corresponding author.

TL;DR: Training-free selective memory updates for long-horizon recurrent streaming 3D reconstruction.

https://github.com/user-attachments/assets/56b08162-c8b6-4251-9a01-038cd5f746b4

Getting Started

Installation

  1. Clone MeMix.
git clone https://github.com/dongjiacheng06/MeMix
cd MeMix
  1. Create the environment.
conda create -n memix python=3.11 cmake=3.14.0
conda activate memix
pip install torch==2.9.1 torchvision==0.24.1 --index-url https://download.pytorch.org/whl/cu128
pip install -r requirements.txt
# issues with pytorch dataloader, see https://github.com/pytorch/pytorch/issues/99625
conda install 'llvm-openmp<16'
  1. Compile the RoPE CUDA kernels if you want the fast implementation:
cd src/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../

Download Checkpoints

CUT3R provides a checkpoint trained on 4-64 views: cut3r_512_dpt_4_64.pth.

Download it with:

cd src
gdown --fuzzy https://drive.google.com/file/d/1Asz-ZB3FfpzZYwunhQvNPZEUA8XUNAYD/view?usp=drive_link
cd ..

Inference Demo

demo.py runs on a user-provided video file or image folder. Choose one public variant from:

  • cut
  • ttt
  • ttsa
  • cut_memix
  • ttt_memix
  • ttsa_memix

Set --seq_path to your own local input and choose the variant with --model_variant. If --views is not passed, the demo uses all decoded frames; if it is passed, the demo uniformly samples that many views from the decoded sequence after applying --frame_interval.

MODEL_VARIANT=cut_memix
CUDA_VISIBLE_DEVICES=0 python demo.py \
    --model_path src/cut3r_512_dpt_4_64.pth \
    --seq_path /path/to/your_video.mp4 \
    --output_dir tmp/demo_run \
    --port 8080 \
    --model_variant ${MODEL_VARIANT} \
    --frame_interval 4 \
    --views 200 \
    --reset_interval 100 \
    --downsample_factor 20 \
    --vis_threshold 2.0

Demo outputs are written to --output_dir.

Evaluation

Please refer to the eval.md for more details.

Acknowledgements

Our code is built on top of the following open-source projects:

We thank the authors for releasing their code.

Citation

If you find our work useful, please cite:

@article{dong2026memix,
    title = {MeMix: Writing Less, Remembering More for Streaming 3D Reconstruction},
    author = {Dong, Jiacheng and Li, Huan and Zhou, Sicheng and Hu, Wenhao and Xu, Weili and Wang, Yan},
    journal = {arXiv preprint arXiv:2603.15330},
    year = {2026},
}