BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization

Project Page | Paper (ArXiv) | Supplemental Material | Code (Github)

This repository is the official pytorch implementation of our paper, BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization.

Yixin Yang¹, Zhongzheng Peng¹, Xiaoyu Du¹, Zhulin Tao², Jinhui Tang¹, Jinshan Pan¹

¹Nanjing University of Science and Technology, ²Communication University of China

🔥 News

• [2023-12-05] Integrated to 🐼 OpenXLab. Try out online demo! .
• [2023-12-05] Add inference code using two reference images, see test_BiSTNet.py.
• [2023-12-02] Colab demo for BiSTNet is available .

🔥 NTIRE2023 Video Colorization Challenge Champion 🏆

We are excited to announce that our method based on BiSTNet won 1st place in the NTIRE2023 Video Colorization Challenge Track 1, Fréchet Inception Distance (FID) Optimization! You can find our code and the pre-trained weights in the following links:

• Colorization results
• Testing code
• Pre-trained models

Framework

Input Videos (left column) and Colorized Videos (right column)

To Do

• Release training code
• Release testing code
• Release pre-trained models

Citation

If this work is helpful for your research, please consider citing the following entry.

@article{bistnet,
  title={BiSTNet: Semantic Image Prior Guided Bidirectional Temporal Feature Fusion for Deep Exemplar-based Video Colorization},
  author={Yang, Yixin and Peng, Zhongzheng and Du, Xiaoyu and Tao, Zhulin and Tang, Jinhui and Pan, Jinshan},
  journal={arXiv preprint arXiv:2212.02268},
  year={2022}
}

Contact

This repo is currently maintained by Yixin Yang (@yyang181) and is for academic research use only.