The relationship between subjective test vote variance and vote mean (or MOS) is well-studied, and the mathematically admissible vote variance region has been previously defined. We propose a reduced admissible variance region called the Unimodal Variance Region (UVR) that better describes real subjective rating behavior of multimedia. Further, subjective vote variance is often modeled as parabolic. We explain that, in practice, the parabolic model often violates the admissible region in the variance vs. MOS plane and we propose alternatives that respect the admissible region. We also present a parametrized random process to model votes that mixes voting processes and produces a realistic range of vote variances within the UVR at any desired MOS. This process was inspired by and comports with voting behavior that is observed in many subjective tests. By modeling vote variance from a subjective experiment, this vote model offers additional interpretable insights into voting behavior observed in a given experiment. We present example results from 16 datasets spanning speech, image, and video subjective quality experiments.
Figures & tables
Domain
p0
p1
vM(y)
1≤y<2
(7−2y)/5
(y−1)/10
(y−1)(4−y)
2≤y<3
(4−y)/5
(2y−3)/15
−y2+17y/3−6
3≤y<4
(9−2y)/15
(y−2)/5
−y2+19y/3−8
4≤y<5
(5−y)/10
(2y−5)/5
(y−2)(5−y)
Table 1: Parameters and variance for the MVU vote PMF.
Figure 1: The Unimodal Variance Region and other variance relationships.
Figure 2: Raw and averaged variance data for ITS2013 (left) and NISQA P501 MOS (middle) votes, the fourth-degree fitting ( 7 ) and other variance relationships as indicated. One outlier variance value above 3 not shown for ITS2013. Mixture parameters are shown in right panel.
Table 2: Datasets and fitting parameters for the parabola in ( 3 ) and the fitting in ( 7 ). † indicates image quality tests; †† indicates video quality tests; all others are speech quality tests. * Indicates tests of the land-mobile radio system in [ 1 ] .
Figure 4: Summary of mixing parameter values for 16 datasets.
Majority voting is one of the few black-box interventions that can improve a fixed stochastic predictor: repeated access can be cheaper than changing a high-capability model. Classical fixed-competence theory makes this intervention look monotone -- more votes help above the majority threshold and hurt below it. We show that this picture is fundamentally incomplete. Under the de Finetti representation for exchangeable repeated correctness, voting is governed by a latent distribution of per-example correctness probabilities. Even simple latent mixtures can generate sharply different voting curves, including nonmonotone behavior and, in an explicit construction, infinitely many trend changes. The full latent law determines the curve, but the curve does not determine the law. The exact object recovered by voting is a signed voting signature: at each binomial variance scale, it records excess latent mass above rather than below the majority threshold. Our main theorem proves that the complete odd-budget curve and this signature are equivalent: the curve increments are signed Hausdorff moments, and the full curve recovers the signature uniquely. This viewpoint explains shape phenomena, branch-symmetric nonidentifiability, realizability, variation, and endpoint rates. It also separates estimation regimes: direct per-example success-probability information targets the full signature, whereas fixed-depth grouped labels reveal only a finite prefix.
Learning-based video quality assessment (VQA) has advanced rapidly, yet progress is increasingly constrained by a disconnect between model design and dataset curation. Model-centric approaches often iterate on fixed benchmarks, while data-centric efforts collect new human labels without systematically targeting the weaknesses of existing VQA models. Here, we describe MDS-VQA, a model-informed data selection mechanism for curating unlabeled videos that are both difficult for the base VQA model and diverse in content. Difficulty is estimated by a failure predictor trained with a ranking objective, and diversity is measured using deep semantic video features, with a greedy procedure balancing the two under a constrained labeling budget. Experiments across multiple VQA datasets and models demonstrate that MDS-VQA identifies diverse, challenging samples that are particularly informative for active fine-tuning. With only a 5% selected subset per target domain, the fine-tuned model improves mean SRCC from 0.651 to 0.722 and achieves the top gMAD rank, indicating strong adaptation and generalization.
Video Quality Assessment (VQA) is evolving beyond single-number mean opinion score toward richer, multi-faceted evaluations of video content. In this paper, we present a large-scale multi-dimensional VQA dataset UltraVQA that encompasses diverse User-Generated Content~(UGC) annotated across five key quality dimensions: Motion Quality, Motion Amplitude, Aesthetic Quality, Content Quality, and Clarity Quality. Each video in our dataset is scored by over 3 human raters on these dimensions, with fine-grained sub-attribute labels, and accompanied by an explanatory rationale generated by GPT based on the collective human judgments. To better leverage these rich annotations and improve discrete quality score assessment, we introduce Analytic Score Optimization (ASO), a theoretically grounded post-training objective derived for multi-dimensional VQA. By reframing quality assessment as a regularized decision-making process, we obtain a closed-form solution that naturally captures the ordinal nature of human ratings, ensuring alignment with human ranking preferences. In experiments, our method outperforms most baselines including closed-source APIs and open-source models, while also reducing mean absolute error (MAE) in quality prediction. Our work highlights the importance of multi-dimensional, interpretable annotations and reinforcement-based alignment in advancing video quality assessment.