README.md

Emo-DiT: Emotional Speech Synthesis with a Diffusion Model Approach to Enhance Naturalness and Emotional Expressiveness

This repository provides the official implementation of Emo-DiT, along with synthesized speech samples generated from different datasets for emotion-controllable text-to-speech (TTS).

Description

TTS

Contains audio samples synthesized using the modified TTS model, demonstrating the effects of the proposed improvements.

ESD / Emov-DB

Contain emotional speech samples generated using models trained on the ESD and Emov-DB datasets, respectively. 

Emo-DiT

Includes the complete source code for the Emo-DiT model, covering data preprocessing, model training, and inference.

Data Preparation

Before training, users must organize their training data according to the required format.

Please refer to the following file for the expected data list format: Emo-DiT/Emo_Dit/resources/filelists/train.txt

Users should modify the directory structure and file paths of their datasets to match the format specified in train.txt, ensuring that the model can correctly load and process the data during training.

⚠️ Make sure that the audio paths, text transcriptions, and any related annotations strictly follow the format defined in train.txt; otherwise, the training process may fail.

🚀 Model Training

After preparing the dataset, start model training by running:

python train.py

This script will initialize the training process and optimize the Emo-DiT model.

🔊 Inference (Speech Synthesis)

Once training is complete, speech synthesis can be performed using:

python inference.py

This script loads the trained model and generates emotional speech from the given text input.

📌 Notes

Please ensure that all required dependencies are properly installed before running the code.

During inference, make sure to specify the correct path to the trained model checkpoint.