One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Control

Zhenxing Mi, Yuxin Wang, Dan Xu
The Hong Kong University of Science and Technology (HKUST)
Accepted to ECCV 2026

News

• 2026.06: One4D has been accepted to ECCV 2026.

📝 Abstract

One4D is a unified framework for 4D generation and reconstruction that can seamlessly transition between 4D generation from a single image, 4D reconstruction from a full video, mixed generation and reconstruction from sparse frames, and 4D generation from a text prompt via Unified Masked Conditioning (UMC). With Decoupled LoRA Control (DLC), which employs two modality-specific LoRA adapters to form decoupled computation branches for RGB frames and pointmaps, connected by lightweight, zero-initialized control links that gradually learn mutual pixel-level consistency, One4D produces high-quality RGB frames and accurate pointmaps across both generation and reconstruction tasks.

🧠 Methodology

Unified Framework

Figure 1: The One4D Unified Framework architecture.

• 🎛️ Unified Masked Conditioning (UMC): Enables seamlessly transition between 4D generation from a single image, 4D reconstruction from a full video, and mixed generation and reconstruction from sparse frames using a single unified model.
• 🧩 Decoupled LoRA Control (DLC): Decouples RGB and XYZ computation to minimize interference while maintaining pixel-wise cross-modal control.

Architecture Comparison

Figure 2: Comparison of Decoupled LoRA Control against other architectures.

🎨 Results Showcase

1. Single Image to 4D

Generating a consistent 4D scene from a single input image.

2. Sparse Frames to 4D

Reconstructing the 4D scene given only a few sparse frames.

3. Full Video to 4D

High-fidelity reconstruction from a full video input.

4. Text to 4D

Generating a consistent 4D scene from a pure text prompt.

📖 BibTeX

If you find our work useful for your research, please consider citing us:

@inproceedings{mi2026one4d,
  title={One4D: Unified 4D Generation and Reconstruction via Decoupled LoRA Control},
  author={Mi, Zhenxing and Wang, Yuxin and Xu, Dan},
  booktitle={European Conference on Computer Vision (ECCV)},
  year={2026}
}