The proliferation of user-generated video content has made accurate and robust no-reference video quality assessment (NR-VQA) a highly important challenge. Existing research increasingly relies on architecturally complex multi-branch pipelines or computationally intensive fine-tuning of single-branch backbones. In this paper, we investigate whether a single frozen video foundation model can serve as an effective NR-VQA backbone, and propose HiVE-VQA, the first framework (to our knowledge) to apply a JEPA-based video encoder to this task. HiVE-VQA extracts token grids from multiple depths of a pre-trained V-JEPA 2.1 encoder and stacks them into a hierarchical spatio-temporal volume, which a compact 3D convolutional head fuses across its layer, spatial, and temporal axes, combining low-level distortion cues from shallow layers with the semantic and temporal context captured at greater depths. The encoder remains frozen throughout, so only the head is trained: 5.2M parameters (5.7% of the total), with no auxiliary multi-modal models and an inference cost that is invariant to video resolution. Trained solely on LSVQ and evaluated without any fine-tuning, HiVE-VQA outperforms 17 state-of-the-art (SOTA) methods in SROCC and PLCC on both intra-dataset (LSVQ-test, LSVQ-1080p) and cross-dataset (KoNViD-1k, LIVE-VQC) benchmarks. These results indicate that hierarchical, depth-wise fusion of features from a single frozen video foundation model provides a more effective and efficient basis for NR-VQA than task-specific multi-network fusion. Code and model checkpoints are available at https://github.com/RobHam99/HiVE-VQA.
Figures & tables
Figure 1: The performance of the proposed HiVE-VQA model, and selected benchmark NR-VQA methods. Here, SROCC and PLCC are used to measure each model’s correlation with human-based mean opinion scores, and the results are averaged across the four test datasets. HiVE-VQA is the top performing model compared to 17 classical, deep (single-branch) and deep (multi-branch) models. The results shown here are based on frozen model parameters, without performing any fine-tuning on these datasets.
Figure 2: The framework of the proposed HiVE-VQA method.
Testing Type
Intra-Dataset
Cross-Dataset
Average
Testing Datasets
LSVQ test
LSVQ 1080p
KoNViD-1k
LIVE-VQC
Overall
Type
Methods
Source
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
Classical
BRISQUE [ 27 ]
TIP, 2012
0.569
0.576
0.497
0.531
0.646
0.647
0.524
0.536
0.559
0.573
TLVQM [ 13 ]
TIP, 2019
0.772
0.774
0.589
0.616
0.732
0.724
0.670
0.691
0.691
0.701
VIDEVAL [ 37 ]
TIP, 2021
0.794
0.783
0.545
0.554
0.751
0.741
0.630
0.640
0.680
0.680
Deep (Multi-branch)
PVQ w/o patch [ 50 ]
CVPR, 2021
0.814
0.816
0.686
0.708
0.782
0.781
0.747
0.776
0.757
0.770
Table 1: Evaluation of the proposed HiVE-VQA methods and state-of-the-art NR-VQA metrics on UGC video quality datasets. Red corresponds to best, blue corresponds to second best and teal corresponds to third best (excluding HiVE-VQA-M).
Figure 3: Representation of artifacts . We apply 4 synthetic distortions (blur, compression, darkening, super-resolution) to a source sequence, in order to determine if the encoder is able to see artifacts. We perform principal component analysis (PCA) on output features from the V-JEPA 2.1 encoder; the first three principal components are then mapped to the RGB channels.
Type
Methods
Source
LIVE-VQC
KoNViD-1k
YouTube-UGC
Average
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
Classical
TLVQM [ 13 ]
TIP, 2019
0.799
0.803
0.773
0.768
0.669
0.659
0.732
0.726
VIDEVAL [ 37 ]
TIP, 2021
0.752
0.751
0.783
0.780
0.779
0.773
0.772
0.772
RAPIQUE [ 38 ]
OJSP, 2021
0.755
0.786
0.803
0.817
0.759
0.768
0.774
0.790
Classical + Deep
CNN+TLVQM [ 12 ]
ACMMM, 2020
0.825
0.834
0.816
0.818
N/A
N/A
0.821
0.826
CNN+VIDEVAL [ 37 ]
TIP, 2021
0.785
0.810
0.815
0.817
0.808
0.803
0.806
0.810
Table 2: Fine-tuning results across multiple UGC datasets. Red corresponds to best, blue corresponds to second best and teal corresponds to third best (excluding HiVE-VQA-M). DVLTA-VQA* [ 51 ] was not included in Table 1 , as their code has not yet been released.
Models
Input Size
Depth
Dim
Param
FLOPs
HiVE-VQA-M
16×224×224
12
768
91M
188G
HiVE-VQA
32×384×384
12
768
91M
2406G
Table 3: Details of the HiVE-VQA models.
Methods
Params (M)
540p
720p
1080p
Trainable
Total
GFLOPs
Time
GFLOPs
Time
GFLOPs
Time
VSFA [ 16 ]
0.5
24.1
6440
1.506
11426
2.556
25712
5.291
PVQ [ 50 ]
54.2
54.2
9203
1.792
13842
2.968
36760
6.556
BVQA [ 15 ]
57.8
57.8
17705
3.145
31533
7.813
70714
14.34
FAST-VQA [ 45 ]
27.7
27.7
284
0.056
284
0.056
284
0.056
FasterVQA [ 46 ]
27.7
27.7
70
0.032
70
0.032
70
0.032
Table 4: Parameter counts, FLOPs and running time (average of 10 runs) on GPU (RTX 4090). Parameters are in M, FLOPs in G, and time in s. “Trainable” denotes parameters updated during training; methods that fine-tune end-to-end have trainable equal to total. Counts were computed from publicly released checkpoints where not stated in the original paper.
Encoder Layers
Efficiency
LSVQ test
LSVQ 1080p
KoNViD-1k
LIVE-VQC
Average
L3
L6
L9
L12
GFLOPs
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
✓
✓
✓
2403
0.905
0.905
0.830
0.861
0.897
0.900
0.856
0.878
0.872
0.886
✓
✓
✓
2403
0.903
0.903
0.833
0.862
0.895
0.897
0.853
0.873
0.871
0.884
✓
✓
✓
2403
0.903
0.903
0.830
0.860
0.894
0.896
0.860
0.879
0.872
0.885
✓
✓
✓
2403
0.901
0.901
0.827
0.858
0.891
0.891
0.860
0.876
0.870
0.882
✓
2395
0.853
0.852
0.738
0.770
0.837
0.834
0.780
0.796
0.802
0.813
Table 5: Layer selection analysis. Checkmarks indicate which encoder blocks were included; the final row is the proposed configuration. Best result per column in bold.
Quality Network (Backbone)
LSVQ test
LSVQ 1080p
KoNViD-1k
LIVE-VQC
Average
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
SROCC
PLCC
Ridge Regressor (V-JEPA 2.1 ViT-B)
0.878
0.875
0.779
0.821
0.871
0.873
0.818
0.847
0.837
0.854
HiVE-VQA (ViT-B)
0.850
0.851
0.741
0.772
0.835
0.839
0.763
0.810
0.797
0.818
HiVE-VQA (VideoMAE ViT-B)
0.815
0.813
0.699
0.727
0.793
0.794
0.782
0.806
0.772
0.785
HiVE-VQA (V-JEPA 2.1 ViT-B)
0.906
0.906
0.835
0.867
0.902
0.903
0.864
0.882
0.877
0.890
Table 6: Ablation of the volumetric head and V-JEPA 2.1 encoder. “Ridge Regressor” replaces the volumetric network and token restructuring with a ridge regressor. “ViT-B” replaces the backbone with the image-pretrained vit_base_patch16_224 checkpoint from timm [ 44 ] , keeping the head fixed. We also compare with another unsupervised video backbone - VideoMAE [ 36 ] .
No-reference video quality assessment (NR VQA) has recently seen promising progress with deep learning. However, video data is inherently large, and processing them with deep models incurs high computational cost. This challenge is particularly acute in VQA, where preserving original-resolution cues and dense temporal information is critical for accuracy. Existing efficiency-driven preprocessing strategies, such as fragmenting, reduce computation but alter the input data distribution, limiting effective reuse of pretrained video foundation models (ViFMs). To address these challenges, we propose High-Fidelity Video Quality Assessment (HFVQA), a framework built on fixed-size spatio-temporal (ST) patches that is fully compatible with pretrained ViFMs. HFVQA samples ST patches across multiple scales, including the original resolution, with minimal temporal subsampling to preserve low-level quality cues and semantic context. To limit computation, HFVQA introduces a lightweight auxiliary network trained end-to-end with the ViFM encoder to learn VQA-specific saliency. Distilled directly from quality supervision, this saliency captures task-specific importance patterns, reflecting that video quality perception is dominated by a small subset of spatio-temporal regions. By combining high-fidelity spatio-temporal cues with learned, task-specific saliency, HFVQA achieves SOTA performance on standard NR VQA benchmarks while processing as little as 12% of candidate ST patches, making high-fidelity ViFM-based VQA computationally tractable.
Hakan Emre Gedik, Shashank Gupta, Alan Bovik
The University of Texas at Austin · University of Colorado Boulder
This report presents our solutions to the QoMEX 2026 Grand Challenge on Video Quality Assessment for Asymmetric Encoded Videos, comprising a full-reference (FR) model, CompressedVQA-AEV-FR, and a no-reference (NR) model, CompressedVQA-AEV-NR. The FR approach leverages a Swin-B backbone to extract multi-stage similarity statistics between reference and distorted videos for quality prediction. For the NR setting, our model employs complementary frame-level encoders based on SigLIP2 and Swin-B, followed by temporal mean pooling and cross-fold ensembling to estimate perceptual quality without reference data. Our CompressedVQA-AEV-FR achieves first place in the FR track of QoMEX 2026 Grand Challenge, while CompressedVQA-AEV-NR secures fourth place in the NR track, demonstrating the effectiveness of our proposed models. The code is available at https://github.com/sunwei925/CompressedVQA-AEV.
Wei Sun, Xingwei Liu, Dandan Zhu +3
East China Normal University · Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University
Short-form video poses new challenges to the quality assessment of user-generated content (UGC) due to its complex generation pipeline, rapid content variation, and mixed distortions. To address this challenge, we propose an end-to-end video quality assessment (VQA) framework that employs a dense visual encoder based on CLIP, and incorporates compression priors derived from the frequency domain to generate artifact- and structure-aware weight maps for feature aggregation. By explicitly decomposing artifact, structure, and original visual feature branches and adaptively fusing them over time through a learned gating module, the proposed method achieves accurate and efficient quality prediction. Experimental results show that our method achieves strong performance on short-form video datasets in terms of average rank and linear correlation (SRCC: 0.736, PLCC: 0.787), while maintaining efficient inference runtime. The code and additional results are available at: https://github.com/xinyiW915/FGSVQA.
Xinyi Wang, Angeliki Katsenou, Junxiao Shen +1
School of Computer Science, University of Bristol, Bristol BS1 8UB, UK