MDS-VQA: Model-Informed Data Selection for Video Quality Assessment

Jian Zou; Xiaoyu Xu; Zhihua Wang; Yilin Wang; Balu Adsumilli; Kede Ma

cs.CVMar 12, 2026

MDS-VQA: Model-Informed Data Selection for Video Quality Assessment

Authors: Jian Zou, Xiaoyu Xu, Zhihua Wang, Yilin Wang, Balu Adsumilli, Kede Ma

Organizations: City University of Hong Kong · Google Inc.

Abstract

Learning-based video quality assessment (VQA) has advanced rapidly, yet progress is increasingly constrained by a disconnect between model design and dataset curation. Model-centric approaches often iterate on fixed benchmarks, while data-centric efforts collect new human labels without systematically targeting the weaknesses of existing VQA models. Here, we describe MDS-VQA, a model-informed data selection mechanism for curating unlabeled videos that are both difficult for the base VQA model and diverse in content. Difficulty is estimated by a failure predictor trained with a ranking objective, and diversity is measured using deep semantic video features, with a greedy procedure balancing the two under a constrained labeling budget. Experiments across multiple VQA datasets and models demonstrate that MDS-VQA identifies diverse, challenging samples that are particularly informative for active fine-tuning. With only a 5% selected subset per target domain, the fine-tuned model improves mean SRCC from 0.651 to 0.722 and achieves the top gMAD rank, indicating strong adaptation and generalization.

Figures & tables

Figure 1: System diagram of MDS-VQA. We predict failure ( i.e . , difficulty) on unlabeled target videos, combine difficulty and content diversity to select a small subset for human labeling, and actively fine-tune the VQA model on existing labeled data and the selected subsets. Bottom: on CGVDS [ 49 ] , this $5\%$ subset (marked by red crosses) yields strong failure identification and improved fine-tuning performance.

Figure 2: Training and inference of MDS-VQA. During training, we freeze the base quality model $f(\cdot)$ and optimize an auxiliary failure predictor $g(\cdot)$ by minimizing a fidelity loss under a Thurstone model [ 29 ] . During inference, we rank unlabeled videos by combining predicted difficulty scores with a content diversity measure to select a (sub-)optimal subset for human labeling and active fine-tuning.

You are doing a video quality assessment task.

Here is the question: Assess how difficult it is to evaluate this video’s quality. The difficulty rating should be a float between

1

and

5

, rounded to two decimal places, with

1

representing very easy and

5

representing very difficult.

Please only output the final answer with only one score in the <answer> </answer> tags.

Table 1: Structured text prompt used for training

g(\cdot)

Method	CGVDS [ 49 ]	LIVE- Livestream [ 25 ]	YouTube-SFV SDR [ 37 ]	YouTube-SFV HDR2SDR [ 37 ]	AIGVQA-DB [ 34 ]	Mean
Base model [ 44 ]	0.544 / 0.635	0.473 / 0.493	0.665 / 0.710	0.538 / 0.591	0.733 / 0.740	0.591 / 0.634
Random sampling	0.673 / 0.782	0.521 / 0.555	0.642 / 0.787	0.438 / 0.407	0.652 / 0.729	0.585 / 0.652
Core-set selection [ 23 ]	0.415 / 0.599	0.289 / 0.378	0.599 / 0.741	0.516 / 0.555	0.676 / 0.742	0.499 / 0.603
Sampling by RD [ 42 ]	0.411 / 0.412	0.253 / 0.352	0.495 / 0.535	0.372 / 0.592	0.640 / 0.660	0.434 / 0.510
MC dropout [ 21 ]	0.379 / 0.440	0.333 / 0.350	0.411 / 0.452	0.359 / 0.490	0.754 / 0.757	0.447 / 0.498
Greedy sampling [ 2 ]	0.331 / 0.564	0.461 / 0.586	0.472 / 0.532	0.334 / 0.451	0.659 / 0.663	0.451 / 0.559

Table 2: Failure identification performance. We compare MDS-VQA with eight competing methods on simulated unlabeled pools

\mathcal{U}

from CGVDS, LIVE-Livestream, YouTube-SFV SDR, YouTube-SFV HDR2SDR, and AIGVQA-DB. Each entry reports SRCC/PLCC between the base model predictions and MOSs on the samples selected by each method; lower values indicate stronger failure identification. Best results are highlighted in bold.

Method	YouTube- UGC [ 35 ]	CGVDS [ 49 ]	LIVE- Livestream [ 25 ]	YouTube-SFV SDR [ 37 ]	YouTube-SFV HDR2SDR [ 37 ]	AIGVQA-DB [ 34 ]	Mean
Base model [ 44 ]	0.708 / 0.709	0.766 / 0.780	0.561 / 0.587	0.666 / 0.718	0.495 / 0.557	0.711 / 0.748	0.651 / 0.683
Random sampling	0.760 / 0.756	0.807 / 0.804	0.569 / 0.628	0.703 / 0.761	0.518 / 0.588	0.756 / 0.751	0.686 / 0.715
Core-set selection [ 23 ]	0.776 / 0.765	0.832 / 0.842	0.601 / 0.610	0.700 / 0.760	0.517 / 0.593	0.788 / 0.787	0.702 / 0.726
Sampling by RD [ 42 ]	0.795 / 0.784	0.780 / 0.825	0.597 / 0.602	0.729 / 0.778	0.517 / 0.587	0.772 / 0.767	0.698 / 0.724
MC dropout [ 21 ]	0.760 / 0.753	0.819 / 0.831	0.595 / 0.609	0.710 / 0.764	0.508 / 0.585	0.758 / 0.764	0.692 / 0.718
Greedy sampling [ 2 ]	0.781 / 0.773	0.831 / 0.834	0.585 / 0.605	0.711 / 0.774	0.500 / 0.580	0.775 / 0.777	0.697 / 0.724

Table 3: Active fine-tuning performance. Models are fine-tuned on the YouTube-UGC training set and subsets selected by different methods under a

5\%

labeling budget per target domain. Each entry reports SRCC/PLCC on the corresponding test set.

Figure 3: Representative gMAD pairs between VQA models induced by MDS-VQA and core-set selection [ 23 ] . Left: gMAD pairs found by fixing MDS-VQA-induced model predictions and searching for videos that maximally differentiate the core-set-induced model. Right: the reverse setting. Predicted scores and MOSs are shown for each pair, both on a $[1,5]$ scale where higher values indicate better predicted and perceived quality, respectively.

Method	SRCC Rank	gMAD Rank	$\Delta$ Rank
MDS-VQA (Ours)	1	1	0
FreeSel [ 45 ]	2	2	0
NoiseStability [ 13 ]	3	6	-3
ALCS [ 46 ]	4	5	-1
Core-set selection [ 23 ]	5	8	-3
Sampling by RD [ 42 ]	6	3	3

Table 4: Quality prediction performance comparison under SRCC and gMAD competition [ 17 ] . We report each method’s overall SRCC rank (based on the mean results in Table 3 ) and gMAD rank (evaluated on LSVQ-1080p [ 47 ] ).

\Delta

Rank denotes the difference between the two rankings.

	Failure identification		Active fine-tuning
Loss	SRCC	PLCC	SRCC	PLCC
Base model [ 44 ]	0.665	0.710	0.666	0.718
Removing $g(\cdot)$	0.509	0.605	0.705	0.770
Classification loss	0.415	0.482	0.715	0.771
Regression loss	0.396	0.472	0.714	0.780
Ranking loss	0.264	0.361	0.726	0.789

Table 5: Ablation on training losses for the failure predictor

g(\cdot)

. The default setting is highlighted in bold.

	Failure identification		Active fine-tuning
Weight	SRCC	PLCC	SRCC	PLCC
Base model [ 44 ]	0.665	0.710	0.666	0.718
$\lambda=0$	0.317	0.385	0.715	0.781
$\lambda=0.125$	0.295	0.367	0.720	0.784
$\boldsymbol{\lambda}$ $\lambda$ $\mathbf{\;=0.250}$	0.264	0.361	0.726	0.789
$\lambda=0.500$	0.281	0.356	0.725	0.784

Table 6: Ablation on the difficulty-diversity trade-off weight

\lambda

	Failure identification		Active fine-tuning
Diversity feature	SRCC	PLCC	SRCC	PLCC
Base model [ 44 ]	0.665	0.710	0.666	0.718
VisualQuality-R1 [ 44 ]	0.397	0.452	0.720	0.783
CLIP [ 22 ]	0.264	0.361	0.726	0.789
SigLIP 2 [ 33 ]	0.279	0.350	0.730	0.791

Table 7: Ablation on diversity feature representations.

	Failure identification		Active fine-tuning
LoRA rank	SRCC	PLCC	SRCC	PLCC
$r$ = 8	0.297	0.425	0.727	0.788
r = 64	0.264	0.361	0.726	0.789
$r$ = 128	0.273	0.418	0.724	0.789

Table 8: Ablation on the LoRA rank

r

for the failure predictor.

CNN-based UVQ [ 36 ]
	Failure identification		Active fine-tuning
Method	SRCC	PLCC	SRCC	PLCC
Base model [ 36 ]	0.590	0.644	0.577	0.648
Random sampling	0.577	0.632	0.591	0.664
Core-set selection [ 23 ]	0.417	0.389	0.625	0.707
Sampling by RD [ 42 ]	0.451	0.510	0.623	0.716

Table 9: Ablation on base VQA models.

Figure 4: Representative gMAD pairs between VQA models induced by MDS-VQA and ALCS [ 46 ] .

Figure 5: Representative gMAD pairs between VQA models induced by MDS-VQA and FreeSel [ 45 ] .

Figure 6: Representative gMAD pairs between VQA models induced by MDS-VQA and NoiseStability [ 13 ] .

Figure 7: Representative challenging videos from YouTube-SFV SDR [ 37 ] selected by MDS-VQA. (a)-(f) show samples chosen without the diversity term, whereas (g)-(l) show samples chosen with diversity, resulting in a broader coverage of content and distortion patterns.

You are doing a video quality assessment task.

Here is the question: What is your overall rating on the quality of this video? The rating should be a float between

1

and

5

, rounded to two decimal places, with

1

representing very poor quality and

5

representing excellent quality.

First output the thinking process in <think> </think> tags and then output the final answer with only one score in <answer> </answer> tags.

Table 10: Structured text prompt used for training

f(\cdot)

Training Data Selection

Training data selection studies how to choose, filter, weight, or schedule examples to improve learning while reducing data and compute costs.

See all 184 papers

DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment
Apr 14, 2026Xinyue Li, Shubo Xu, Zhichao Zhang +3Video Quality Assessment VLM Calibration
Shanghai Jiao Tong University, Shanghai, China · Baidu Inc., Beijing, China · Xinjiang University, Urumqi, China
Analytic Score Optimization for Multi Dimension Video Quality Assessment
Feb 18, 2026Boda Lin, Yongjie Zhu, Wenyu Qin +2Video Quality Assessment
Kling Team, Kuaishou Technology, Beijing, China.
VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment
May 20, 2026Shibei Meng, Binxin Yang, Yuan Liu +4Pairwise Comparison Learning to Rank
The Chinese University of Hong Kong · Beijing Normal University · WeChat Vision, Tencent Inc. +2

Apr 14, 2026J/K move · Enter open · S save

DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment

Xinyue Li, Shubo Xu, Zhichao Zhang, Zhaolin Cai+2

Shanghai Jiao Tong University, Shanghai, China · Baidu Inc., Beijing, China · Xinjiang University, Urumqi, China

Video Quality Assessment VLM Calibration Video Generation Evaluation

Open paper AskPDF arXiv

Recent multimodal large language models (MLLMs) have shown promising performance on video quality assessment (VQA) tasks. However, adapting them to new scenarios remains expensive due to large-scale retraining and costly mean opinion score (MOS) annotations. In this paper, we argue that a pretrained MLLM already provides a useful perceptual prior for VQA, and that the main challenge is to efficiently calibrate this prior to the target MOS space. Based on this insight, we propose DPC-VQA, a decoupling perception and calibration framework for video quality assessment. Specifically, DPC-VQA uses a frozen MLLM to provide a base quality estimate and perceptual prior, and employs a lightweight calibration branch to predict a residual correction for target-scenario adaptation. This design avoids costly end-to-end retraining while maintaining reliable performance with lower training and data costs. Extensive experiments on both user-generated content (UGC) and AI-generated content (AIGC) benchmarks show that DPC-VQA achieves competitive performance against representative baselines, while using less than 2% of the trainable parameters of conventional MLLM-based VQA methods and remaining effective with only 20% of MOS labels. The code will be released upon publication.

Abstract

Training Data Selection

Explore similar work

DPC-VQA: Decoupling Quality Perception and Residual Calibration for Video Quality Assessment

Analytic Score Optimization for Multi Dimension Video Quality Assessment

VersusQ: Pairwise Margin Reasoning for Generalizable Video Quality Assessment