WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting

Accepted to NeurIPS 2025

Paper | Project Website

Requirements

source ./scripts/install_deps.sh

Checkpoints

We have uploaded the pre-trained models required for image preprocessing, training, inference, and editing to Google Drive. After downloading, please place them in the corresponding directories (./pose_estimation, ./pretrained_models, ./editings).

Dataset preparation

Training Dataset

Download the FFHQ Dataset as the training dataset; optionally extend it with the LPFF Dataset.

Follow EG3D or LPFF for pose extraction and face alignment, or simply download the ready-to-use official releases of both datasets (FFHQ Dataset and LPFF Dataset).

Testing Dataset

We follow HFGI to preprocess customized images.

cd ./pose_estimation
python extract_pose.py 0 ori_data tmp_data align_data

We provide a few pre-processed images in ./data/test_img; download our pre-trained weights and jump to Inference for a quick start.

Training

Training 3DGAN Inversion Encoder

Run the following command to train the encoder. Config file is configs/train_vanilla.yaml.

CUDA_VISIBLE_DEVICES=0 python scripts/train_vanilla.py

Generating Static Dataset

Leverage synthetic images sampled from the 3D GAN to assist in training SVINet. Besides, use the encoder trained above to reconstruct these images and generate novel views beforehand, accelerating SVINet training.

CUDA_VISIBLE_DEVICES=0 python scripts/gen_synthimg.py

Employ the encoder already trained to reconstruct real images and generate novel views beforehand, accelerating SVINet training.

CUDA_VISIBLE_DEVICES=0 python scripts/gen_novelview.py

Training SVINet

Run the following command to train SVINet. Config file is configs/train_inpainting.yaml.

CUDA_VISIBLE_DEVICES=0 python scripts/train_inpainting.py

Inference

Run the following command to synthesize novel-view images from the input. Config file is configs/infer.yaml.

CUDA_VISIBLE_DEVICES=0 python scripts/infer.py

PTI

Use novel-view images synthesized by WarpGAN to assist PTI training. Config file is configs/pti.yaml.

CUDA_VISIBLE_DEVICES=0 python ./scripts/run_pti.py

Editing

Perform editing with the latent code and fine-tuned generator obtained from PTI. Config file is configs/editing.yaml.

CUDA_VISIBLE_DEVICES=0 python scripts/editing_ptiG.py

Acknowlegement

We thank the authors of EG3D, LPFF, Triplanenet, GOAE, PTI, HFGI, LaMa and Deep3DFaceRecon for sharing their code.

Citation

@misc{huang2025warpganwarpingguided3dgan,
      title={WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting}, 
      author={Kaitao Huang and Yan Yan and Jing-Hao Xue and Hanzi Wang},
      year={2025},
      eprint={2511.08178},
      archivePrefix={arXiv},
      primaryClass={cs.CV},
      url={https://arxiv.org/abs/2511.08178}, 
}