Learning-based video quality assessment (VQA) has advanced rapidly, yet progress is increasingly constrained by a disconnect between model design and dataset curation. Model-centric approaches often iterate on fixed benchmarks, while data-centric efforts collect new human labels without systematically targeting the weaknesses of existing VQA models. Here, we describe MDS-VQA, a model-informed data selection mechanism for curating unlabeled videos that are both difficult for the base VQA model and diverse in content. Difficulty is estimated by a failure predictor trained with a ranking objective, and diversity is measured using deep semantic video features, with a greedy procedure balancing the two under a constrained labeling budget. Experiments across multiple VQA datasets and models demonstrate that MDS-VQA identifies diverse, challenging samples that are particularly informative for active fine-tuning. With only a 5% selected subset per target domain, the fine-tuned model improves mean SRCC from 0.651 to 0.722 and achieves the top gMAD rank, indicating strong adaptation and generalization.
Figures & tables
Figure 1: System diagram of MDS-VQA. We predict failure ( i.e . , difficulty) on unlabeled target videos, combine difficulty and content diversity to select a small subset for human labeling, and actively fine-tune the VQA model on existing labeled data and the selected subsets. Bottom: on CGVDS [ 49 ] , this 5% subset (marked by red crosses) yields strong failure identification and improved fine-tuning performance.
Figure 2: Training and inference of MDS-VQA. During training, we freeze the base quality model f(⋅) and optimize an auxiliary failure predictor g(⋅) by minimizing a fidelity loss under a Thurstone model [ 29 ] . During inference, we rank unlabeled videos by combining predicted difficulty scores with a content diversity measure to select a (sub-)optimal subset for human labeling and active fine-tuning.
You are doing a video quality assessment task.
Here is the question: Assess how difficult it is to evaluate this video’s quality. The difficulty rating should be a float between 1 and 5 , rounded to two decimal places, with 1 representing very easy and 5 representing very difficult.
Please only output the final answer with only one score in the <answer> </answer> tags.
Table 1: Structured text prompt used for training g(⋅) .
Method
CGVDS [ 49 ]
LIVE- Livestream [ 25 ]
YouTube-SFV SDR [ 37 ]
YouTube-SFV HDR2SDR [ 37 ]
AIGVQA-DB [ 34 ]
Mean
Base model [ 44 ]
0.544 / 0.635
0.473 / 0.493
0.665 / 0.710
0.538 / 0.591
0.733 / 0.740
0.591 / 0.634
Random sampling
0.673 / 0.782
0.521 / 0.555
0.642 / 0.787
0.438 / 0.407
0.652 / 0.729
0.585 / 0.652
Core-set selection [ 23 ]
0.415 / 0.599
0.289 / 0.378
0.599 / 0.741
0.516 / 0.555
0.676 / 0.742
0.499 / 0.603
Sampling by RD [ 42 ]
0.411 / 0.412
0.253 / 0.352
0.495 / 0.535
0.372 / 0.592
0.640 / 0.660
0.434 / 0.510
MC dropout [ 21 ]
0.379 / 0.440
0.333 / 0.350
0.411 / 0.452
0.359 / 0.490
0.754 / 0.757
0.447 / 0.498
Greedy sampling [ 2 ]
0.331 / 0.564
0.461 / 0.586
0.472 / 0.532
0.334 / 0.451
0.659 / 0.663
0.451 / 0.559
Table 2: Failure identification performance. We compare MDS-VQA with eight competing methods on simulated unlabeled pools U from CGVDS, LIVE-Livestream, YouTube-SFV SDR, YouTube-SFV HDR2SDR, and AIGVQA-DB. Each entry reports SRCC/PLCC between the base model predictions and MOSs on the samples selected by each method; lower values indicate stronger failure identification. Best results are highlighted in bold.
Method
YouTube- UGC [ 35 ]
CGVDS [ 49 ]
LIVE- Livestream [ 25 ]
YouTube-SFV SDR [ 37 ]
YouTube-SFV HDR2SDR [ 37 ]
AIGVQA-DB [ 34 ]
Mean
Base model [ 44 ]
0.708 / 0.709
0.766 / 0.780
0.561 / 0.587
0.666 / 0.718
0.495 / 0.557
0.711 / 0.748
0.651 / 0.683
Random sampling
0.760 / 0.756
0.807 / 0.804
0.569 / 0.628
0.703 / 0.761
0.518 / 0.588
0.756 / 0.751
0.686 / 0.715
Core-set selection [ 23 ]
0.776 / 0.765
0.832 / 0.842
0.601 / 0.610
0.700 / 0.760
0.517 / 0.593
0.788 / 0.787
0.702 / 0.726
Sampling by RD [ 42 ]
0.795 / 0.784
0.780 / 0.825
0.597 / 0.602
0.729 / 0.778
0.517 / 0.587
0.772 / 0.767
0.698 / 0.724
MC dropout [ 21 ]
0.760 / 0.753
0.819 / 0.831
0.595 / 0.609
0.710 / 0.764
0.508 / 0.585
0.758 / 0.764
0.692 / 0.718
Greedy sampling [ 2 ]
0.781 / 0.773
0.831 / 0.834
0.585 / 0.605
0.711 / 0.774
0.500 / 0.580
0.775 / 0.777
0.697 / 0.724
Table 3: Active fine-tuning performance. Models are fine-tuned on the YouTube-UGC training set and subsets selected by different methods under a 5% labeling budget per target domain. Each entry reports SRCC/PLCC on the corresponding test set.
Figure 3: Representative gMAD pairs between VQA models induced by MDS-VQA and core-set selection [ 23 ] . Left: gMAD pairs found by fixing MDS-VQA-induced model predictions and searching for videos that maximally differentiate the core-set-induced model. Right: the reverse setting. Predicted scores and MOSs are shown for each pair, both on a [1,5] scale where higher values indicate better predicted and perceived quality, respectively.
Method
SRCC Rank
gMAD Rank
Δ Rank
MDS-VQA (Ours)
1
1
0
FreeSel [ 45 ]
2
2
0
NoiseStability [ 13 ]
3
6
-3
ALCS [ 46 ]
4
5
-1
Core-set selection [ 23 ]
5
8
-3
Sampling by RD [ 42 ]
6
3
3
Table 4: Quality prediction performance comparison under SRCC and gMAD competition [ 17 ] . We report each method’s overall SRCC rank (based on the mean results in Table 3 ) and gMAD rank (evaluated on LSVQ-1080p [ 47 ] ). Δ Rank denotes the difference between the two rankings.
Failure identification
Active fine-tuning
Loss
SRCC
PLCC
SRCC
PLCC
Base model [ 44 ]
0.665
0.710
0.666
0.718
Removing g(⋅)
0.509
0.605
0.705
0.770
Classification loss
0.415
0.482
0.715
0.771
Regression loss
0.396
0.472
0.714
0.780
Ranking loss
0.264
0.361
0.726
0.789
Table 5: Ablation on training losses for the failure predictor g(⋅) . The default setting is highlighted in bold.
Failure identification
Active fine-tuning
Weight
SRCC
PLCC
SRCC
PLCC
Base model [ 44 ]
0.665
0.710
0.666
0.718
λ=0
0.317
0.385
0.715
0.781
λ=0.125
0.295
0.367
0.720
0.784
λλ=0.250
0.264
0.361
0.726
0.789
λ=0.500
0.281
0.356
0.725
0.784
Table 6: Ablation on the difficulty-diversity trade-off weight λ .
Failure identification
Active fine-tuning
Diversity feature
SRCC
PLCC
SRCC
PLCC
Base model [ 44 ]
0.665
0.710
0.666
0.718
VisualQuality-R1 [ 44 ]
0.397
0.452
0.720
0.783
CLIP [ 22 ]
0.264
0.361
0.726
0.789
SigLIP 2 [ 33 ]
0.279
0.350
0.730
0.791
Table 7: Ablation on diversity feature representations.
Failure identification
Active fine-tuning
LoRA rank
SRCC
PLCC
SRCC
PLCC
r = 8
0.297
0.425
0.727
0.788
r = 64
0.264
0.361
0.726
0.789
r = 128
0.273
0.418
0.724
0.789
Table 8: Ablation on the LoRA rank r for the failure predictor.
Failure identification
Active fine-tuning
Method
SRCC
PLCC
SRCC
PLCC
CNN-based UVQ [ 36 ]
Base model [ 36 ]
0.590
0.644
0.577
0.648
Random sampling
0.577
0.632
0.591
0.664
Core-set selection [ 23 ]
0.417
0.389
0.625
0.707
Sampling by RD [ 42 ]
0.451
0.510
0.623
0.716
Table 9: Ablation on base VQA models.
Figure 4: Representative gMAD pairs between VQA models induced by MDS-VQA and ALCS [ 46 ] .
Figure 5: Representative gMAD pairs between VQA models induced by MDS-VQA and FreeSel [ 45 ] .
Figure 6: Representative gMAD pairs between VQA models induced by MDS-VQA and NoiseStability [ 13 ] .
Figure 7: Representative challenging videos from YouTube-SFV SDR [ 37 ] selected by MDS-VQA. (a)-(f) show samples chosen without the diversity term, whereas (g)-(l) show samples chosen with diversity, resulting in a broader coverage of content and distortion patterns.
You are doing a video quality assessment task.
Here is the question: What is your overall rating on the quality of this video? The rating should be a float between 1 and 5 , rounded to two decimal places, with 1 representing very poor quality and 5 representing excellent quality.
First output the thinking process in <think> </think> tags and then output the final answer with only one score in <answer> </answer> tags.
Table 10: Structured text prompt used for training f(⋅) .
Recent multimodal large language models (MLLMs) have shown promising performance on video quality assessment (VQA) tasks. However, adapting them to new scenarios remains expensive due to large-scale retraining and costly mean opinion score (MOS) annotations. In this paper, we argue that a pretrained MLLM already provides a useful perceptual prior for VQA, and that the main challenge is to efficiently calibrate this prior to the target MOS space. Based on this insight, we propose DPC-VQA, a decoupling perception and calibration framework for video quality assessment. Specifically, DPC-VQA uses a frozen MLLM to provide a base quality estimate and perceptual prior, and employs a lightweight calibration branch to predict a residual correction for target-scenario adaptation. This design avoids costly end-to-end retraining while maintaining reliable performance with lower training and data costs. Extensive experiments on both user-generated content (UGC) and AI-generated content (AIGC) benchmarks show that DPC-VQA achieves competitive performance against representative baselines, while using less than 2% of the trainable parameters of conventional MLLM-based VQA methods and remaining effective with only 20% of MOS labels. The code will be released upon publication.
Xinyue Li, Shubo Xu, Zhichao Zhang +3
Shanghai Jiao Tong University, Shanghai, China · Baidu Inc., Beijing, China · Xinjiang University, Urumqi, China
Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of video content. In this paper, we present a large-scale multi-dimensional VQA dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each video in our dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based alignment in advancing video quality assessment.
Large Multimodal Models (LMMs) have shown promise for video quality assessment, but most methods still predict an absolute score for each video. Such pointwise supervision often mixes perceptual quality with dataset-specific calibration, including annotation protocols, rating habits, and score distributions. As a result, the learned scoring rule may work well within a benchmark but transfer poorly across unseen domains. We argue that relative comparisons alleviate the absolute-scale calibration bias by focusing purely on perceptual differences rather than dataset-specific rating habits. Consequently, we propose VersusQ, a pairwise margin reasoning framework driven entirely by direct comparisons. Specifically, VersusQ performs LMM-based comparison between two videos, reasons about their visual and temporal quality differences, and predicts a signed continuous margin that captures both the preferred choice and the degree of difference. Furthermore, to align interpretable comparison rationales with fine-grained numerical differences, we introduce Margin-Coupled GRPO, which jointly optimizes rollout-based relational reasoning and continuous margin regression. Extensive experiments on multiple public VQA benchmarks demonstrate that VersusQ achieves state-of-the-art performance, strong cross-domain generalization, and reliable fine-grained ranking under heterogeneous evaluation scenarios.
Shibei Meng, Binxin Yang, Yuan Liu +4
The Chinese University of Hong Kong · Beijing Normal University · WeChat Vision, Tencent Inc. +2