README.md
July 17, 2026 · View on GitHub
ReCal3R: Reliability-Calibrated Learning Rates for Streaming 3D Reconstruction
Xinze Li, Yiyuan Wang, Pengxu Chen, Weifeng Su, Weisi Lin, Wentao Cheng
🔭 Overview
ReCal3R estimates state token reliability and a candidate learning rate from the recurrent forward pass, and uses reliability calibration to obtain the final state learning rate. This prevents unreliable state tokens from receiving aggressive updates and leads to cleaner reconstructions than CUT3R over long image streams.
Getting Started
Installation
- Clone ReCal3R.
git clone https://github.com/Powertony102/ReCal3R.git
cd ReCal3R
- Create the environment.
conda create -n recal3r python=3.11 cmake=3.14.0
conda activate recal3r
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia # use the correct version of cuda for your system
pip install -r requirements.txt
# issues with pytorch dataloader, see https://github.com/pytorch/pytorch/issues/99625
conda install 'llvm-openmp<16'
# for evaluation
pip install evo
pip install open3d
- Compile the cuda kernels for RoPE (as in CroCo v2).
cd src/croco/models/curope/
python setup.py build_ext --inplace
cd ../../../../
Download Checkpoints
CUT3R provide checkpoints trained on 4-64 views: cut3r_512_dpt_4_64.pth.
To download the weights, run the following commands:
cd src
gdown --fuzzy https://drive.google.com/file/d/1Asz-ZB3FfpzZYwunhQvNPZEUA8XUNAYD/view?usp=drive_link
cd ..
Evaluation
Please refer to the eval.md for more details.