LRW1000 (CAS-VSR-W1k)

This repo provides the link of the data and the code of DenseNet3D Model in LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild. Our paper can be found here.

Data Preparation

Download LRW1000 Dataset and place LRW1000_Public in the root of this repository. Instead, you can create symbolic links to this project:

ln -s LRW1000_Public Lipreading-DenseNet3D/LRW1000_Public

Training And Testing

You can train or test the model by running:

python main.py options_lip.toml

Model architecture details and data annotation items are configured in options_lip.toml. Please pay attention that you may need modify the code in options_lip.toml and change the parameters to make the scripts work just as expected.

Dependencies

• PyTorch 1.0+
• toml
• tensorboardX
• imageio

Reference

If this repository was useful for your research, please cite our work:

@inproceedings{yang2019lrw,
  title={LRW-1000: A Naturally-Distributed Large-Scale Benchmark for Lip Reading in the Wild},
  author={Yang, Shuang and Zhang, Yuanhang and Feng, Dalu and Yang, Mingmin and Wang, Chenhao and Xiao, Jingyun and Long, Keyu and Shan, Shiguang and Chen, Xilin},
  booktitle={2019 14th IEEE International Conference on Automatic Face \& Gesture Recognition (FG 2019)},
  pages={1--8},
  year={2019},
  organization={IEEE},
  url={https://github.com/Fengdalu/Lipreading-DenseNet3D}
}

Related Projects

Another implmentation Of DenseNet-3D

Learn an Effective Lip Reading Model without Pains (Strong Recommended)

LipNet-PyTorch (The state-of-the-art PyTorch Version)

CAS-VSR-S68: A sentence-level audio-visual speech dataset for speaker-adaptive lip-reading

CAS-VSR-S101: A wild sentence-level audio-visual speech dataset

CAS-VSR-MOV20: A challenging lip-reading dataset with various visual conditions