FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry

Muxin Liu^1,2,*, Xiaoyang Lyu^1,*, Tianhe Ren¹, Peng Dai¹, Xiaoshan Wu¹, Zhiyue Zhang¹, Jiaqi Zhang¹, Jiehong Lin¹, Shaoshuai Shi^2,✉, Xiaojuan Qi^1,✉

¹The University of Hong Kong    ²Voyager Research, Didi Chuxing
^*Equal Contribution   ^✉Corresponding Author

TODO

• Release paper and project page
• Release Stage-I Base model and code
• Release Stage-II FoundationGeo model and code
• Release training code and config details
• Release full model zoo

⚙️ Installation

git clone https://github.com/mx-liu6/FoundationGeo.git
cd FoundationGeo
conda create -n foundationgeo python=3.10 -y
conda activate foundationgeo
pip install -r requirements.txt

📦 Pretrained Models

Stage	Hugging Face Model	Version	Release Type	Metric Scale	# Params
Stage-I Base Model	FoundationGeo-Base-v1	v1	Submission version	-	313M
	FoundationGeo-Base-v1.1	v1.1	Recommended version	-	313M
Stage-II FoundationGeo	FoundationGeo-v1	v1	Submission version	✅	314M
	FoundationGeo-v1.1	v1.1	Recommended version	✅	314M

🚀 Training

We provide training configurations and launch scripts for both stages:

• Stage-I Base model: configs/train/BaseModel.json, start_scripts/BaseModel.sh
• Stage-II FoundationGeo model: configs/train/FoundationGeo.json, start_scripts/FoundationGeo.sh

Additional Requirements

Training uses accelerate for distributed training and mlflow for optional logging:

pip install accelerate mlflow sympy tqdm

Data Preparation

Training datasets are expected under data/train. Each dataset should contain an index file and per-sample folders:

data/train/somedataset
├── index.txt
├── sample_000001
│   ├── image.jpg
│   ├── depth.png
│   └── meta.json
└── ...

index.txt stores one sample folder per line. meta.json should include normalized camera intrinsics:

{
  "intrinsics": [[fx, 0.0, cx], [0.0, fy, cy], [0.0, 0.0, 1.0]]
}

Depth maps can be read and written with the helpers in foundationgeo/utils/io.py. You can inspect prepared samples with:

python foundationgeo/scripts/vis_data.py data/train/somedataset --ply --depth_vis

Run Training

The provided launch scripts are designed for multi-GPU or multi-node training environments. They read distributed settings from environment variables such as RESOURCE_NUM_GPU, DISTRIBUTED_NODE_COUNT, DISTRIBUTED_NODE_RANK, and DISTRIBUTED_MASTER_HOSTS.

# Train Stage-I Base model
bash start_scripts/BaseModel.sh

# Train Stage-II FoundationGeo model
bash start_scripts/FoundationGeo.sh

For a single-machine launch, you can call accelerate directly and adjust --num_processes, batch size, workspace, and checkpoint path as needed:

accelerate launch --multi_gpu --num_processes 8 \
  foundationgeo/scripts/train.py \
  --config configs/train/FoundationGeo.json \
  --workspace workspace/FoundationGeo \
  --gradient_accumulation_steps 1 \
  --batch_size_forward 8 \
  --checkpoint latest \
  --enable_gradient_checkpointing False \
  --enable_mlflow True

To train the Stage-I Base model directly, replace the script and config with foundationgeo/scripts/train_base.py and configs/train/BaseModel.json.

📏 Evaluation

We provide a unified evaluation pipeline that wraps a baseline model, evaluates it on multiple benchmarks, and writes metrics to a JSON file.

Benchmarks

Download the processed evaluation datasets from Hugging Face Datasets and place them under data/eval:

mkdir -p data/eval
huggingface-cli download Ruicheng/monocular-geometry-evaluation \
  --repo-type dataset \
  --local-dir data/eval \
  --local-dir-use-symlinks False

Then unzip the downloaded benchmark files:

cd data/eval
unzip '*.zip'

Configuration

See configs/eval/all_benchmarks.json for the default benchmark configuration. You can edit this file to change dataset paths, image sizes, or benchmark subsets.

Run Evaluation

The provided scripts evaluate the Stage-I Base model and Stage-II FoundationGeo model from local checkpoints:

# Evaluate Stage-I Base model
CHECKPOINT_PATH=checkpoints/FoundationGeo_BaseModel.pt \
bash start_scripts/Eval/FoundationGeo_Base_Eval.sh

# Evaluate Stage-II FoundationGeo model
CHECKPOINT_PATH=checkpoints/FoundationGeo_v1.pt \
bash start_scripts/Eval/FoundationGeo_Eval.sh

You can also run the evaluation script directly. Extra arguments after --output are passed to the baseline loader:

# Evaluate Stage-II FoundationGeo from a Hugging Face repo or local checkpoint
python foundationgeo/scripts/eval_baseline.py \
  --baseline baselines/foundationgeo.py \
  --config configs/eval/all_benchmarks.json \
  --output eval_output/FoundationGeo.json \
  --pretrained mxliu-hku/FoundationGeo-1.1 \
  --resolution_level 9 \
  --version v1

# Evaluate Stage-I Base model
python foundationgeo/scripts/eval_baseline.py \
  --baseline baselines/foundationgeo.py \
  --config configs/eval/all_benchmarks.json \
  --output eval_output/FoundationGeo_Base.json \
  --pretrained mxliu-hku/FoundationGeo-Base-1.1 \
  --resolution_level 9 \
  --version base

Useful evaluation options include --oracle for GT intrinsics, --dump_pred for prediction dumps, and --dump_gt for ground-truth dumps. To evaluate a customized method, implement the interface in foundationgeo/test/baseline.py; see baselines/foundationgeo.py for an example.

🏗️ Architecture

📊 Main Results

Quantitative results for metric and relative depth estimation. AbsRel and delta1 are in percentage. The best values are highlighted in bold, and the second-best ones are underlined. * indicates the model requires GT intrinsics as input. Gray numbers denote models trained on respective benchmarks or requiring GT intrinsics, and are therefore excluded from ranking.

📚 Citation

If you find our work useful, please consider citing:

@inproceedings{liu2026foundationgeo,
  title={FoundationGeo: Learning Spatial Pixel-Wise Fields for Monocular Metric Geometry},
  author={Liu, Muxin and Lyu, Xiaoyang and Ren, Tianhe and Dai, Peng and Wu, Xiaoshan and Zhang, Zhiyue and Zhang, Jiaqi and Lin, Jiehong and Shi, Shaoshuai and Qi, Xiaojuan},
  booktitle={European Conference on Computer Vision},
  pages={353--371},
  year={2026},
  organization={Springer}
}

🔗 Links

• Paper
• Project Page
• Code
• Hugging Face

📄 License

This project is licensed under the MIT License.

🙏 Acknowledgments

We thank the MoGe series of works and DINOv3.

🌟 Related Work

For efficient monocular geometry estimation in optically challenging scenes, we also recommend our related work, OptiGeo:

@inproceedings{liu2026optigeo,
  title={OptiGeo: Efficient Monocular Geometry for Embodied Perception in Optically Challenging Scenes},
  author={Liu, Muxin and Liu, Tianbo and Xia, Jing and Lyu, Xiaoyang and Wu, Xiaoshan and Wang, Bo and Dai, Peng and Wang, Zhongrui and Shi, Shaoshuai and Qi, Xiaojuan},
  booktitle={Conference on Robot Learning (CoRL)},
  year={2026}
}