Track, Inpaint, Resplat: Subject-driven 3D and 4D generation with Progressive Texture Infilling (NeurIPS 2025)

Track Stage

We utilize CoTracker to obtain the inpainting masks.

First, we need to prepare the input frames as below:

── DIR_TO_FRAMES
    └── frames
        └── 1.png
        └── 2.png
        └── ...
        └── 180.png
    └── mask
        └── 1.png
        └── 2.png
        └── ...
        └── 180.png

Inside the track folder:

cd track
python generate_tracking_traj.py --frames_dir {DIR_TO_FRAMES}/frames --output_dir <path_to_output> --start_frame 0
python generate_tracking_traj.py --frames_dir {DIR_TO_FRAMES}/frames --output_dir <path_to_output> --start_frame 45

Then, we can visualize the masks rendered with the Track stage:

python visualize_tracking.py --frames_dir {DIR_TO_FRAMES}/frames --traj_dir <path_to_trajectories> --save_dir <path_to_mask_visualization> --start_frame 0
python visualize_tracking.py --frames_dir {DIR_TO_FRAMES}/frames --traj_dir <path_to_trajectories> --save_dir <path_to_mask_visualization> --start_frame 45

📌 ToDo

• Release GitHub repo.
• Release arXiv paper.
• Code for Track stage.
• Code for Inpaint stage.
• Code for Resplat stage.

Citation

If you find our work useful, please consider citing:

@inproceedings{zheng2025trackinpaintresplat,
  title={Track, Inpaint, Resplat: Subject-driven 3D and 4D Generation with Progressive Texture Infilling},
  author={Zheng, Shuhong and Mirzaei, Ashkan and Gilitschenski, Igor},
  booktitle={NeurIPS},
  year={2025}
}