SAMA Overview

PyTorch implementation of "Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment" (arXiv/AAAI), which has been accepted by AAAI-2024.

This code is modified from FAST-VQA.

Usage

IQA

For image quality assessment (IQA), please refer to IQA/demo_train_iqa_baseline.py.

VQA

For video quality assessment (VQA), please refer to VQA/demo_train.py to get the training result, and refer to VQA/demo_finetune.py to get the finetuning result. We also provide the training log for VQA.

The main idea/contribution lies in the data sampling, which can be found in IQA and VQA.

Make sure the configuration has been properly set in

• fast-sama-iqa.yml for IQA training;
• fast-SAMA-train.yml for VQA training on LSVQ;
• and fast-SAMA-tinetune.yml for VQA finetuning.

And please prepare the pretrained models of video-swin for VQA and swin-v2 for IQA.

Testing with pretrained model on videos

We have provided the pretrained weights (trained on LSVQ train set): GoogleDrive / BaiDu (Code:xyns). please check the pretrained weights in ./VQA/pretrained_weights folder and put the weights in the folder.

To test on your own dataset or video files, please construct the dataset information as the examplar in ./VQA/examplar_data_labels, and set the configuration in fast-SAMA-test.yml. Run the file demo_test.py to check the details.

Environment

Different environment may induce possible fluctuation of performance.

Python 3.8.10
PyTorch 1.7.0

The installation can refer to FAST-VQA.

Citation

If you are interested in the work, or find the code helpful, please cite our work

@article{sama2024,
        title={Scaling and Masking: A New Paradigm of Data Sampling for Image and Video Quality Assessment},
        volume={38},
        number={4},
        journal={Proceedings of the AAAI Conference on Artificial Intelligence},
        author={Liu, Yongxu and Quan, Yinghui and Xiao, Guoyao and Li, Aobo and Wu, Jinjian},
        year={2024},
        month={Mar.},
        pages={3792-3801},
        url={https://ojs.aaai.org/index.php/AAAI/article/view/28170},
        DOI={10.1609/aaai.v38i4.28170}
}

Contact

Feel free to contact me via yongxu.liu@xidian.edu.cn if any question or bug.

License

Copyright (c) [2024] [Yongxu Liu]

Permission to use, copy, or modify this software and its documentation for educational and research purposes only and without fee is here granted. This program shall not be used, rewritten, or adapted as the basis of a commercial software or hardware product without first obtaining permission of the authors. The authors make no representations about the suitability of this software for any purpose. It is provided "as is" without express or implied warranty.