Table of contents
November 1, 2025 · View on GitHub
In this work, we present Recursive fusion of Joint Cross-Attention across audio and visual modalities for person verification.
References
If you find this work useful in your research, please consider citing our work :pencil: and giving a star :star2: :
@INPROCEEDINGS{10581998,
author={Praveen, R. Gnana and Alam, Jahangir},
booktitle={2024 IEEE 18th International Conference on Automatic Face and Gesture Recognition (FG)},
title={Dynamic Cross Attention for Audio-Visual Person Verification},
year={2024},
}
There are three major blocks in this repository to reproduce the results of our paper. This code uses Mixed Precision Training (torch.cuda.amp). The dependencies and packages required to reproduce the environment of this repository can be found in the environment.yml file.
Creating the environment
Create an environment using the environment.yml file
conda env create -f environment.yml
Models and Text Files
The pre-trained models of audio and visual backbones are obtained here
The fusion models trained using our fusion approach can be found here
The text files can be found here
train_list : Train list
val_trials : Validation trials list
val_list : Validation list
test_trials : VoX1-O trials list
test_list : Vox 1-O list
Table of contents
Preprocessing
Step One: Download the dataset
Return to Table of Content Please download the following.
- The images of Voxceleb1 dataset can be downloaded here
Step Two: Preprocess the visual modality
- The downloaded images are not properly aligned. So the images are aligned using Insightface The preprocessing scripts are provided in preprocessing folder
Training
- sbatch run_train.sh
Inference
- sbatch run_eval.sh
👍 Acknowledgments
Our code is based on AVCleanse