RePoseD: Efficient Relative Pose Estimation With Known Depth Information

ICCV 2025 Oral

Yaqing Ding · Viktor Kocur · Václav Vávra · Zuzana Berger Haladová · Jian Yang · Torsten Sattler · Zuzana Kukelova

https://github.com/user-attachments/assets/aea6ac6d-9f4c-42d1-8c49-fb0462b40d41

Demos

We provide demos for the estimation of relative pose of two images and following dense two-view reconstruction using MoGev2 + SuperPoint + LightGlue: or MoGev2 + RoMa:

We also provide a script to create a nice visualization of the resulting pointcloud:

https://github.com/user-attachments/assets/7cae9b8e-2008-4b1b-97cc-9e07f0ee03d0

You can also try a script to perform dynamic scene reconstruction. Note that this script does not currently include foreground/background segmentation so it works only for videos with static background with sufficient features for matching. Below is an example.

https://github.com/user-attachments/assets/7825f13a-d1e3-4572-a16c-a7827cfd04ea

To run the last two scripts you need to install MoGe, LightGlue and Open3D. You will also need to follow the instructions in the next section.

Use in your own project

To use RePoseD in your own project you must first install PoseLib with our PR. Currently, you need to compile the master branch of PoseLib yourself. Using pip install poselib will not work until a new version with wheels is released.

pip install https://github.com/PoseLib/PoseLib

If this is not sufficient you may need to first install some extra packages and/or clone the repo manually:

git clone https://github.com/PoseLib/PoseLib
cd PoseLib
pip install pybind11_stubs
apt-get install libeigen3-dev
python setup.py install

Once installed you can use the new methods added to poselib for relative pose estimation.

import poselib

# extract keypoints and their corresponding depths
# make sure you remove any nans or infs

# set your thresholds
ransac_dict = {'max_epipolar_error': 2.0, 'max_reproj_error': 16.0}
# set this to true if you also want to estimate shit (calib case only)
ransac_dict['monodepth_estimate_shift'] = False

# use this loss for better estimation in final optimization
bundle_dict = {'loss_type': 'TRUNCATED_CAUCHY'}

# if you know intrinsics you can use this
camera1 = {'model': 'SIMPLE_PINHOLE', 'width': -1, 'height': -1, 'params': [f1, px1, py1]}
camera2 = {'model': 'SIMPLE_PINHOLE', 'width': -1, 'height': -1, 'params': [f2, px2, py2]}
geometry, info = poselib.estimate_monodepth_relative_pose(points1, points2, depths1, depths2, camera1, camera2, ransac_dict, bundle_dict)
pose = geometry.pose

# for uknown and shared focals you can use (pp is the principal point - usually image center)
image_pair, info = poselib.estimate_monodepth_shared_focal_relative_pose(points1 - pp1, points2 - pp2, depths1, depths2, ransac_dict, bundle_dict)
f = image_pair.camera1.focal()
geometry = image_pair.geometry
pose = geometry.pose

# for uknown and different focals you can use
image_pair, info = poselib.estimate_monodepth_varying_focal_relative_pose(points1 - pp1, points2 - pp2, depths1, depths2, ransac_dict, bundle_dict)
f1 = image_pair.camera1.focal()
f2 = image_pair.camera2.focal()
geometry = image_pair.geometry
pose = geometry.pose

# to transform the pointcloud from the first image into the coordinates of the second image you can use:
xyz1_in_camera2_frame = (1/geometry.scale) * ((pose.R @ xyz1.T).T + pose.t)

Citation

If you find our work useful please consider citing:

@inproceedings{ding2025reposed,
  title={RePoseD: Efficient Relative Pose Estimation With Known Depth Information},
  author={Ding, Yaqing and Kocur, Viktor and V{\'a}vra, V{\'a}clav and Haladov{\'a}, Zuzana Berger and Yang, Jian and Sattler, Torsten and Kukelova, Zuzana},
  booktitle={Proceedings of the IEEE/CVF International Conference on Computer Vision},
  year={2025}
}

Extended Results

After finishing the camera-ready version for ICCV we have implemented an improved LO for PoseLib which optimizes the Sampson and reprojection errors jointly. This results in significant improvement in terms of accuracy often surpassing MADPose results while being significantly faster.

Benchmark results are presented EXTENDED_RESULTS.md. Below is a preview of results on the PhotoTourism dataset.

Phototourism (Calibrated)

Depth	Method	Scale	Shift	SP+LG			RoMA
				$\epsilon(^\circ)\downarrow$	mAA $\uparrow$	Runtime (ms)	$\epsilon(^\circ)\downarrow$	mAA $\uparrow$	Runtime (ms)
-	5-Point			1.42	76.56	63.79	0.78	86.18	264.61
MoGe	3P-RelDepth			8.12	53.40	55.85	1.69	67.22	221.06
	P3P			1.40	77.37	32.95	0.78	86.42	148.76
	MADPose	✔	✔	1.27	80.28	788.18	0.87	86.85	1753.49
	Ours	✔	✔	1.24	81.34	28.93	0.74	88.58	125.66
	Ours*	✔		1.75	80.29	30.11	1.03	88.02	135.95
UniDepth	3P-RelDepth			4.07	51.60	52.49	1.33	67.56	214.73
	P3P			1.40	77.47	34.30	0.78	86.43	150.95
	MADPose	✔	✔	1.15	82.09	720.34	0.78	87.60	1695.57
	Ours	✔	✔	1.04	83.71	30.88	0.69	89.27	131.52
	Ours*	✔		1.16	84.56	31.19	0.81	90.18	137.26

Depth	Method	Scale	Shift	MASt3R
				$\epsilon(^\circ)\downarrow$	mAA $\uparrow$	Runtime (ms)
-	5-Point			1.14	81.66	137.75

* Denotes the use of P3P + our new optimization strategy.

ICCV (2025) Evaluation

To run the experiments from the paper you can use the provided evaluation code. We are currently working on improvements to the methods. For reproducibility we keep the original code available in the iccv-eval branches.

Setting up eval repo, PoseLib and Madpose

You will need to clone this repo and install our PoseLib and Madpose forks with all variants.

Note that the PoseLib variant used in the evaluation scripts is different from the one mentioned in previous sections which includes only our version.

# create a conda environment
conda create -n mdrp
conda activate mdrp
conda install numpy, scipy, tqdm, hd5py, tectonic, prettytable, matplotlib, seaborn, eigen=3.4

# cloning this repo
git clone -b iccv-eval https://github.com/kocurvik/mdrp

# installing Madpose
git clone --recursive https://github.com/kocurvik/madpose
cd madpose
# this step is optional if you have the libs installed
# if you do not have root access you can install them for user or using conan etc.
# sudo apt-get install libceres-dev libopencv-dev
pip install . -v
cd ..

# installing PoseLib fork with all evaluated variants
# note that you need to have Eigen version 3.4 to run this
git clone -b iccv-eval --recursive https://github.com/kocurvik/PoseLib-mdrp
cd PoseLib-mdrp
pip install . -v
cd ..

Download the eval dataset

You can download the extracted matches along with dephts obtained using various MDE methods used in our paper here.

Running the evaluation scripts

You can run the evaluations using the eval scripts. For each scene you can run:

conda activate mdrp

cd mdrp
export PYTHONPATH=.

# -nw parameter is based on the number of CPUs available
# replace e.g. /path/to/dataset/features/dataset/scene -> /path/to/dataset/splg/pt/florence_cathedral_splg.h5 
python eval.py -t 2.0 -r 16.0 -a -o -nw 16 /path/to/dataset/per/individual/scene.h5
python eval_shared_f.py -t 2.0 -r 16.0 -a -o -nw 16 /path/to/dataset/per/individual/scene.h5
python eval_varying_f.py -t 2.0 -r 16.0 -a -o -nw 16 /path/to/dataset/per/individual/scene.h5

# this runs the graph-based evals - not all results are included in paper
python eval.py -g -t 2.0 -r 16.0 -a -o -nw 16 /path/to/dataset/per/individual/scene.h5
python eval_shared_f.py -g -t 2.0 -r 16.0 -a -o -nw 16 /path/to/dataset/per/individual/scene.h5
python eval_varying_f.py -g -t 2.0 -r 16.0 -a -o -nw 16 /path/to/dataset/per/individual/scene.h5

We used a SLURM-based cluster to run the jobs. You can run experiments using the script to spawn jobs for each scene for all experiments.

# Modify the slurm script before running to have correct paths etc.
# Here you should run it for the desired dataset type e.g. splg/pt/ for SP+LG and Phototourism, roma/eth3d for RoMA on ETH3D etc...
# The last number is used just for logging.
/path/to/mdrp/slurm_scripts/eval_mdrp_spawn_all.sh /path/to/one/of/dataset/mdrp/splg/pt/ 001 

Generating tables and graph

You can generate all the tables from the paper using the following scripts

cd mdrp
export PYTHONPATH=.
# Main paper tables
python tables.py
# Main paper graphs
python vis.py
# SM tables
python tables_sideways.py