README.md

Martian World Model
Controllable Video Synthesis with Physically Accurate 3D Reconstructions

Introduction

Martian World Model introduces a comprehensive solution for synthesizing realistic Martian landscape videos, addressing the critical need for mission rehearsal and robotic simulation in planetary exploration.

Installation

To set up the environment for Martian World Model, follow these steps:

# Create a new conda environment
conda create -n mars python=3.10.13 cmake=3.14.0 -y

# Activate the environment
conda activate mars

# Install PyTorch and CUDA dependencies
conda install pytorch torchvision pytorch-cuda=12.1 -c pytorch -c nvidia

# Install required Python packages
pip install -r requirements.txt

# Install submodules
pip install submodules/simple-knn
pip install submodules/diff-gaussian-rasterization
pip install submodules/fused-ssim

Download Image Matching Module Weights

To download the GIM weights required for the project, use the following link:

Google Drive

After downloading, place the weights in the gim/weights/ directory.

Usage

To use the M3arsSynth for synthesizing Martian landscape videos, follow these steps:

1. Replace the DATA_ROOT_DIR variable in scripts/infer.sh with the absolute path to your project directory.

2. Run the following command:

bash scripts/infer.sh

This will execute the pipeline for co-visible global geometry initialization, pose optimization, and video rendering.