Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment
June 12, 2025 Ā· View on GitHub
š„ Authors
**Zhuoxuan Cai**Ā¹'Ā² ā¢ **Jian Zhang**Ā²ā ā¢ **Xinbin Yuan**Ā² ā¢ **Peng-Tao Jiang**Ā²
**Wenxiang Chen**Ā¹ ā¢ **Bowen Tang**Ā² ā¢ **Lujian Yao**Ā² ā¢ **Qiyuan Wang**Ā¹ ā¢ **Jinwei Chen**Ā² ā¢ **Bo Li**Ā²āļø
Ā¹ Fudan University Ā Ā Ā Ā Ā² vivo Mobile Communication Co., Ltd
ā Project Lead Ā Ā Ā Ā āļø Corresponding Author
š Abstract
Recent studies demonstrate that multimodal large language models (MLLMs) can proficiently evaluate visual quality through interpretable assessments. However, existing approaches typically treat quality scoring and reasoning descriptions as separate tasks with disjoint optimization objectives, leading to a trade-off: models adept at quality reasoning descriptions struggle with precise score regression, while score-focused models lack interpretability. This limitation hinders the full potential of MLLMs in visual quality assessment, where accuracy and interpretability should be mutually reinforcing.
To address this, we propose a unified two-stage training framework comprising a cold-start stage and a reinforcement learning-based fine-tuning stage. Specifically, in the first stage, we distill high-quality data from a teacher model through expert-designed prompts, initializing reasoning capabilities via cross-entropy loss supervision. In the second stage, we introduce a novel reward with Group Relative Policy Optimization (GRPO) to jointly optimize scoring accuracy and reasoning consistency.
We designate the models derived from these two stages as Q-Ponder-CI and Q-Ponder. Extensive experiments show that Q-Ponder achieves state-of-the-art (SOTA) performance on quality score regression benchmarks, delivering up to 6.5% higher SRCC on cross-domain datasets. Furthermore, Q-Ponder significantly outperforms description-based SOTA models, including its teacher model Qwen-2.5-VL-72B, particularly in description accuracy and reasonableness, demonstrating the generalization potential over diverse tasks.
š£ News
- š„ [2025.06] Paper and project page are released!
- š [Coming Soon] Model weights and inference code will be released.
- š [Coming Soon] Training code and datasets will be released.
ā Roadmap
| Item | Status | Description |
|---|---|---|
| Paper & Project Page | ā | Official paper and project website |
| Inference Code | š | Model inference scripts and examples |
| Model Weights | š | Pre-trained Q-Ponder-CI and Q-Ponder models |
| Training Code | š | Complete training pipeline implementation |
| Training Datasets | š | Curated datasets for model training |
| Evaluation Benchmarks | š | Comprehensive evaluation suite |
ā Released Ā Ā Ā Ā š In Progress Ā Ā Ā Ā ā³ Planned
āļø Installation
System Requirements
| Component | Requirement |
|---|---|
| Python | 3.10 |
| PyTorch | 2.5.1 |
| CUDA | 12.4 |
Quick Setup
# 1ļøā£ Create conda environment
conda create -n qponder python=3.10
conda activate qponder
# 2ļøā£ Install dependencies (will be provided soon)
pip install -r requirements.txt
š Quick Start
Inference
# Example usage (code will be released soon)
Training
Training code and detailed instructions will be provided upon release.
š Evaluation
Detailed evaluation scripts and benchmarks will be provided to reproduce the results reported in our paper.
š Citation
If you find our work helpful, please consider citing our paper:
@article{cai2025q,
title={Q-Ponder: A Unified Training Pipeline for Reasoning-based Visual Quality Assessment},
author={Cai, Zhuoxuan and Zhang, Jian and Yuan, Xinbin and Jiang, Pengtao and Chen, Wenxiang and Tang, Bowen and Yao, Lujian and Wang, Qiyuan and Chen, Jinwen and Li, Bo},
journal={arXiv preprint arXiv:2506.05384},
year={2025}
}