Analytic Score Optimization for Multi Dimension Video Quality Assessment

Boda Lin; Yongjie Zhu; Wenyu Qin; Meng Wang; Pengfei Wan

cs.CVFeb 18, 2026

Analytic Score Optimization for Multi Dimension Video Quality Assessment

Authors: Boda Lin, Yongjie Zhu, Wenyu Qin, Meng Wang, Pengfei Wan

Organizations: Kling Team, Kuaishou Technology, Beijing, China.

Abstract

Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of video content. In this paper, we present a large-scale multi-dimensional VQA dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each video in our dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based alignment in advancing video quality assessment.

Figures & tables

Figure 1: Cases for each category and distribution of statistics of our UltraVQA.

Figure 2: Overview of our method. We curate data and human annotators to annotate video quality scores and quality tags with GPT generated reasons. Then we use 2-stage training to elicit the model’s thinking ability before scoring.

Model	Motion Quality				Motion Amplitude				Aesthetic Quality				Content Quality				Clarity Quality
Model	Acc	MAE	SRCC	PLCC	Acc	MAE	SRCC	PLCC	Acc	MAE	SRCC	PLCC	Acc	MAE	SRCC	PLCC	Acc	MAE	SRCC	PLCC
VLM-APIs
GPT-4.1	$48.0\%$	$1.151$	$0.418$	$0.386$	$66.8\%$	$0.595$	$0.697$	$0.693$	$56.9\%$	$0.744$	$0.611$	$0.615$	$52.5\%$	$0.992$	$0.093$	$0.096$	$75.3\%$	$0.514$	$0.848$	$0.839$
Gemini-2.5Pro	$57.0\%$	$0.849$	$0.608$	$0.590$	$49.7\%$	$0.888$	$0.744$	$0.723$	$56.0\%$	$0.812$	$0.654$	$0.643$	$40.3\%$	$1.138$	$0.322$	$0.277$	$65.8\%$	$0.611$	$0.850$	$0.824$
Open-source VLMs
Qwen2.5-VL	$38.0\%$	$1.233$	$0.532$	$0.536$	$44.2\%$	$1.042$	$0.749$	$0.733$	$41.0\%$	$1.187$	$0.373$	$0.361$	$43.3\%$	$1.220$	$0.093$	$0.097$	$54.4\%$	$0.786$	$0.793$	$0.799$

Table 1: Main Results on UltraVQA. We divided all models into four groups and uniformly used [email protected], MAE, PLCC, and SRCC to evaluate them across five quality dimensions.

Model	LSVQ-1080p		KoNViD-1k		VideoPhy2	MJ-Video
Model	SRCC	PLCC	SRCC	PLCC	Acc (%)	Acc (%)
VLM-APIs
GPT-4.1	0.651	0.677	0.608	0.623	26.1	44.7
Gemini-2.5Pro	0.682	0.691	0.676	0.681	28.8	46.9
Open-source VLMs
Qwen2.5-VL	0.624	0.610	0.658	0.642	20.2	34.6

Table 2: Cross-benchmark evaluation. We report SRCC/PLCC for VQA datasets and Accuracy (%) for VideoPhy2/MJ-Bench-Video. Symbol ‘-’ denotes results not reported in the original literature.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Prompt used for rationale expansion on Motion Quality and Motion Amplitude.

Figure 4: Prompt used for rationale expansion on Aesthetic Quality.

Figure 5: Prompt used for rationale expansion on Content Quality.

Figure 6: Prompt used for rationale expansion on Clarity Quality.

Figure 7: Word cloud display of rationale expansion.

Annotation Phase	Motion	Motion	Aesthetic	Content	Clarity
Annotation Phase	Quality	Amplitude	Quality	Quality	Quality
	( $\mathcal{R}$ / $\alpha$ )	( $\mathcal{R}$ / $\alpha$ )	( $\mathcal{R}$ / $\alpha$ )	( $\mathcal{R}$ / $\alpha$ )	( $\mathcal{R}$ / $\alpha$ )
Pilot Phase ( $n=2k$ )	85.2% / 0.68	88.4% / 0.71	81.5% / 0.62	83.1% / 0.65	89.0% / 0.74
Final Phase ( $n=40k$ )	92.4% / 0.81	94.6% / 0.85	88.7% / 0.76	90.5% / 0.79	96.2% / 0.88

Appendix

Table 3: Inter-Annotator Agreement (IAA) analysis across five quality dimensions. We report Relaxed Match (

\mathcal{R}

) , defined as the percentage of annotator pairs with score difference

\leq 1.0

, and Krippendorff’s Alpha (

\alpha

) to measure reliability. The significant improvement from the Pilot to the Final phase demonstrates the effectiveness of our refined guidelines and annotator training.

Explore similar work

VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment
May 20, 2026Shibei Meng, Binxin Yang, Yuan Liu +4Pairwise Comparison Learning to Rank
The Chinese University of Hong Kong · Beijing Normal University · WeChat Vision, Tencent Inc. +2
DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment
Apr 14, 2026Xinyue Li, Shubo Xu, Zhichao Zhang +3Video Quality Assessment VLM Calibration
Shanghai Jiao Tong University, Shanghai, China · Baidu Inc., Beijing, China · Xinjiang University, Urumqi, China
Q-Save: Towards Scoring and Attribution for Generated Video Evaluation
Nov 24, 2025Xiele Wu, Mingtao Chen, Zicheng Zhang +11Video Generation Evaluation Video Quality Assessment
Shanghai Jiao Tong University · Hunyuan · Shanghai Innovation Institute

May 20, 2026J/K move · Enter open · S save

VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment

Shibei Meng, Binxin Yang, Yuan Liu, Jiexuan Zhang+3

The Chinese University of Hong Kong · Beijing Normal University · WeChat Vision, Tencent Inc.+2

Pairwise Comparison Learning to Rank Pairwise Ranking Video Quality Assessment VLM Reasoning

Open paper AskPDF arXiv

Large Multimodal Models (LMMs) have shown promise for video quality assessment, but most methods still predict an absolute score for each video. Such pointwise supervision often mixes perceptual quality with dataset-specific calibration, including annotation protocols, rating habits, and score distributions. As a result, the learned scoring rule may work well within a benchmark but transfer poorly across unseen domains. We argue that relative comparisons alleviate the absolute-scale calibration bias by focusing purely on perceptual differences rather than dataset-specific rating habits. Consequently, we propose VersusQ, a pairwise margin reasoning framework driven entirely by direct comparisons. Specifically, VersusQ performs LMM-based comparison between two videos, reasons about their visual and temporal quality differences, and predicts a signed continuous margin that captures both the preferred choice and the degree of difference. Furthermore, to align interpretable comparison rationales with fine-grained numerical differences, we introduce Margin-Coupled GRPO, which jointly optimizes rollout-based relational reasoning and continuous margin regression. Extensive experiments on multiple public VQA benchmarks demonstrate that VersusQ achieves state-of-the-art performance, strong cross-domain generalization, and reliable fine-grained ranking under heterogeneous evaluation scenarios.

Abstract

Explore similar work

VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment

DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment

Q-Save: Towards Scoring and Attribution for Generated Video Evaluation