WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation

May 1, 2026 · View on GitHub

Installation

Tested with Python 3.10, PyTorch 2.9, and a single NVIDIA H100 (80 GB) GPU.

git clone https://github.com/cvlab-kaist/WorldCam.git
cd WorldCam

conda create -n worldcam python=3.10 -y
conda activate worldcam

pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
pip install -r requirements.txt

Download Pretrained Weights

pip install huggingface_hub
hf download worldcam/worldcam --local-dir weights/ --include "*.safetensors" --repo-type model

This downloads the fine-tuned WorldCam DiT (~3 GB) into weights/. The model was trained on CS:GO gameplay.

The base Wan2.1-T2V-1.3B weights (text encoder, VAE, base DiT, tokenizer) are pulled automatically the first time you run inference.py.

Quick Start

python inference.py

Inference settings can be adjusted in the configuration block at the top of inference.py.

Download Dataset

We release gameplay recordings from two open-source games for research use:

GameLicenseFolder
XonoticGPL v3data_1/
UnvanquishedCC BY-SA 2.5data_2/
# Download all
hf download worldcam/worldcam-dataset --local-dir data/ --repo-type dataset

# Download only Xonotic (data_1)
hf download worldcam/worldcam-dataset --local-dir data/ --repo-type dataset --include "data_1/*"

Each recording consists of:

  • video_*.mp4 — raw gameplay footage
  • input_*.txt — recorded player actions: keyboard (W, A, S, D, Shift, Space as booleans) and mouse movement (dx, dy as relative pixel deltas)

Note: These are raw recorded actions, not used in the paper. You can use these raw recorded actions for your own research. Camera poses and captions are not included — you can extract them using off-the-shelf models such as ViPE / DA3 for camera poses and Qwen2.5-VL-7B for captions.

Action Overlay Visualization

To visualize the recorded actions overlaid on video:

python overlay_actions.py \
    --video data/data_1/batch_001/video_20250926_193542.mp4 \
    --log data/data_1/batch_001/input_20250926_193542.txt \
    --output overlay_demo.mp4

Citation

@article{nam2026worldcam,
  title={WorldCam: Interactive Autoregressive 3D Gaming Worlds with Camera Pose as a Unifying Geometric Representation},
  author={Nam, Jisu and Hong, Yicong and Huang, Chun-Hao Paul and Liu, Feng and Lee, JoungBin and Kim, Jiyoung and Jin, Siyoon and Lee, Yunsung and Jung, Jaeyoon and Choi, Suhwan and others},
  journal={arXiv preprint arXiv:2603.16871},
  year={2026}
}

Acknowledgements