Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of video content. In this paper, we present a large-scale multi-dimensional VQA dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each video in our dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based alignment in advancing video quality assessment.
Figures & tables
Figure 1: Cases for each category and distribution of statistics of our UltraVQA.
Figure 2: Overview of our method. We curate data and human annotators to annotate video quality scores and quality tags with GPT generated reasons. Then we use 2-stage training to elicit the model’s thinking ability before scoring.
Model
Motion Quality
Motion Amplitude
Aesthetic Quality
Content Quality
Clarity Quality
Acc
MAE
SRCC
PLCC
Acc
MAE
SRCC
PLCC
Acc
MAE
SRCC
PLCC
Acc
MAE
SRCC
PLCC
Acc
MAE
SRCC
PLCC
VLM-APIs
GPT-4.1
48.0%
1.151
0.418
0.386
66.8%
0.595
0.697
0.693
56.9%
0.744
0.611
0.615
52.5%
0.992
0.093
0.096
75.3%
0.514
0.848
0.839
Gemini-2.5Pro
57.0%
0.849
0.608
0.590
49.7%
0.888
0.744
0.723
56.0%
0.812
0.654
0.643
40.3%
1.138
0.322
0.277
65.8%
0.611
0.850
0.824
Open-source VLMs
Qwen2.5-VL
38.0%
1.233
0.532
0.536
44.2%
1.042
0.749
0.733
41.0%
1.187
0.373
0.361
43.3%
1.220
0.093
0.097
54.4%
0.786
0.793
0.799
Table 1: Main Results on UltraVQA. We divided all models into four groups and uniformly used [email protected], MAE, PLCC, and SRCC to evaluate them across five quality dimensions.
Model
LSVQ-1080p
KoNViD-1k
VideoPhy2
MJ-Video
SRCC
PLCC
SRCC
PLCC
Acc (%)
Acc (%)
VLM-APIs
GPT-4.1
0.651
0.677
0.608
0.623
26.1
44.7
Gemini-2.5Pro
0.682
0.691
0.676
0.681
28.8
46.9
Open-source VLMs
Qwen2.5-VL
0.624
0.610
0.658
0.642
20.2
34.6
Table 2: Cross-benchmark evaluation. We report SRCC/PLCC for VQA datasets and Accuracy (%) for VideoPhy2/MJ-Bench-Video. Symbol ‘-’ denotes results not reported in the original literature.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 3: Prompt used for rationale expansion on Motion Quality and Motion Amplitude.
Figure 4: Prompt used for rationale expansion on Aesthetic Quality.
Figure 5: Prompt used for rationale expansion on Content Quality.
Figure 6: Prompt used for rationale expansion on Clarity Quality.
Figure 7: Word cloud display of rationale expansion.
Annotation Phase
Motion
Motion
Aesthetic
Content
Clarity
Quality
Amplitude
Quality
Quality
Quality
( R / α )
( R / α )
( R / α )
( R / α )
( R / α )
Pilot Phase ( n=2k )
85.2% / 0.68
88.4% / 0.71
81.5% / 0.62
83.1% / 0.65
89.0% / 0.74
Final Phase ( n=40k )
92.4% / 0.81
94.6% / 0.85
88.7% / 0.76
90.5% / 0.79
96.2% / 0.88
Appendix
Table 3: Inter-Annotator Agreement (IAA) analysis across five quality dimensions. We report Relaxed Match ( R ) , defined as the percentage of annotator pairs with score difference ≤1.0 , and Krippendorff’s Alpha ( α ) to measure reliability. The significant improvement from the Pilot to the Final phase demonstrates the effectiveness of our refined guidelines and annotator training.
Large Multimodal Models (LMMs) have shown promise for video quality assessment, but most methods still predict an absolute score for each video. Such pointwise supervision often mixes perceptual quality with dataset-specific calibration, including annotation protocols, rating habits, and score distributions. As a result, the learned scoring rule may work well within a benchmark but transfer poorly across unseen domains. We argue that relative comparisons alleviate the absolute-scale calibration bias by focusing purely on perceptual differences rather than dataset-specific rating habits. Consequently, we propose VersusQ, a pairwise margin reasoning framework driven entirely by direct comparisons. Specifically, VersusQ performs LMM-based comparison between two videos, reasons about their visual and temporal quality differences, and predicts a signed continuous margin that captures both the preferred choice and the degree of difference. Furthermore, to align interpretable comparison rationales with fine-grained numerical differences, we introduce Margin-Coupled GRPO, which jointly optimizes rollout-based relational reasoning and continuous margin regression. Extensive experiments on multiple public VQA benchmarks demonstrate that VersusQ achieves state-of-the-art performance, strong cross-domain generalization, and reliable fine-grained ranking under heterogeneous evaluation scenarios.
Shibei Meng, Binxin Yang, Yuan Liu +4
The Chinese University of Hong Kong · Beijing Normal University · WeChat Vision, Tencent Inc. +2
Recent multimodal large language models (MLLMs) have shown promising performance on video quality assessment (VQA) tasks. However, adapting them to new scenarios remains expensive due to large-scale retraining and costly mean opinion score (MOS) annotations. In this paper, we argue that a pretrained MLLM already provides a useful perceptual prior for VQA, and that the main challenge is to efficiently calibrate this prior to the target MOS space. Based on this insight, we propose DPC-VQA, a decoupling perception and calibration framework for video quality assessment. Specifically, DPC-VQA uses a frozen MLLM to provide a base quality estimate and perceptual prior, and employs a lightweight calibration branch to predict a residual correction for target-scenario adaptation. This design avoids costly end-to-end retraining while maintaining reliable performance with lower training and data costs. Extensive experiments on both user-generated content (UGC) and AI-generated content (AIGC) benchmarks show that DPC-VQA achieves competitive performance against representative baselines, while using less than 2% of the trainable parameters of conventional MLLM-based VQA methods and remaining effective with only 20% of MOS labels. The code will be released upon publication.
Xinyue Li, Shubo Xu, Zhichao Zhang +3
Shanghai Jiao Tong University, Shanghai, China · Baidu Inc., Beijing, China · Xinjiang University, Urumqi, China
Evaluating AI-generated video (AIGV) quality hinges on three crucial dimensions: visual quality, dynamic quality, and text-video alignment. While numerous evaluation datasets and algorithms have been proposed, existing approaches are constrained by two limitations: the absence of systematic definitions for evaluation dimensions, and the isolated treatment of the three dimensions in separate models. Therefore, we introduce Q-Save, a holistic benchmark dataset and unified evaluation model for AIGV quality assessment. The Q-Save dataset contains nearly 10,000 video samples, each annotated with Mean Opinion Scores (MOS) and fine-grained attribution explanations across the three core dimensions. Leveraging this attribution-annotated dataset, we train the proposed Q-Save model, which adopts the SlowFast framework to balance accuracy and efficiency, and employs a three-stage training strategy with Chain-of-Thought (COT) formatted data: Supervised Fine-Tuning (SFT), Grouped Relative Policy Optimization (GRPO), and a final SFT round for stability, to jointly perform quality scoring and attribution generation. Experimental results demonstrate that Q-Save achieves superior performance in AIGV quality prediction while providing interpretable justifications. Code and dataset will be released upon publication.
Xiele Wu, Mingtao Chen, Zicheng Zhang +11
Shanghai Jiao Tong University · Hunyuan · Shanghai Innovation Institute