Organizations: College of Computing and Data Science, Nanyang Technological University · Department of Computer Science, South China Normal University, China · School of Computer Science, City University of Hong Kong
Existing AI-generated video quality assessment (AIGVQA) methods mainly focus on global perceptual realism and coarse text-video alignment, while overlooking a critical requirement in educational scenarios: concept correctness. In early mathematics education, subtle errors in numerical quantities, geometric relations, or spatial configurations may fundamentally alter the conveyed knowledge despite visually plausible generation. To address this problem, we introduce EduAVQABench, the first benchmark for concept-aware educational AIGV assessment, containing 1,130 videos generated by ten state-of-the-art T2V models together with over 310,650 fine-grained human annotations spanning perceptual quality and semantic alignment. Built upon this benchmark, we further propose EduVQA, a concept-aware AIGVQA framework equipped with a Structured 2D Mixture-of-Experts (S2D-MoE) architecture. By jointly modeling fine-grained concept assessment and overall quality prediction through shared experts and adaptive two-dimensional routing, EduVQA effectively captures subtle concept-level inconsistencies overlooked by conventional global scoring methods. Extensive experiments demonstrate that EduVQA consistently outperforms existing AIGVQA approaches across both perceptual and semantic evaluation tasks while exhibiting strong generalization capability on unseen benchmarks. Code and dataset will be publicly available at: https://github.com/EduVQA/EduVQA.
Figures & tables
Fig. 1: Annotation structure of our constructed EduAVQABench dataset. Each educational video is annotated with spatial and temporal fidelity and word-level semantic consistency, enabling fine-grained assessment of perceptual quality and prompt alignment. The red and blue elliptical regions indicate temporal inconsistencies that negatively impact temporal quality.
Fig. 2: An overview of our dataset, divided into four categories: Numbers , Geometry , Measurement , and Probability .
Metric
Avg. SRCC
Avg. PLCC
#Annotators (SRCC > 0.8)
Perceptual Quality
0.810
0.803
12
Prompt Alignment
0.759
0.762
9
TABLE I: Annotation consistency analysis across evaluators.
Fig. 3: Annotation Analysis. (a)-(e): MOS distributions across five dimensions; (f): Average MOS of each scene category.
Fig. 4: Overview of EduVQA framework. We jointly predict five quality dimensions via a dual-path framework equipped with 2D MoE.
Metric
Setting
Method
SRCC ↑
PLCC ↑
KRCC ↑
RMSE ↓
Perceptual Quality
Zero-shot
ImageReward [ 43 ]
0.409
0.432
0.282
3.369
Q-Align [ 44 ]
0.644
0.669
–
–
Fine-tuned
IPCE [ 45 ]
0.822
0.822
0.631
0.494
IP-IQA [ 46 ]
0.852
0.863
0.666
0.434
DOVER [ 47 ]
0.832
0.840
0.645
0.479
BVQA [ 48 ]
0.848
0.862
–
–
TABLE II: Comparison on Perceptual Quality and Prompt Alignment of EduAVQABench. ↑ : higher is better, ↓ : lower is better.
Method
LGVQ [ 54 ] Dataset
EvalCrafter [ 27 ] Dataset
Spatial
Temporal
Alignment
Perceptual
Alignment
SRCC / PLCC
SRCC / PLCC
SRCC / PLCC
SRCC / PLCC
SRCC / PLCC
BVQA [ 48 ]
0.518 / 0.608
0.235 / 0.550
– / –
0.161 / 0.172
– / –
CLIPVQA [ 49 ]
0.509 / 0.480
0.355 / 0.287
– / –
0.319 / 0.325
– / –
FasterVQA [ 42 ]
0.511 / 0.539
0.380 / 0.427
– / –
0.378 / 0.359
– / –
GSTVQA [ 50 ]
0.507 / 0.555
0.459 / 0.529
– / –
0.276 / 0.281
– / –
TABLE III: Cross-dataset performance comparison. All models are trained on our EduAVQABench dataset.
Train Dataset
Test Dataset
Perceptual Quality
Prompt Alignment
SRCC
PLCC
SRCC
PLCC
EduAVQABench (Ours)
LGVQ [ 54 ]
0.5558
0.6346
0.5745
0.5941
AIGVE-Bench [ 55 ]
LGVQ [ 54 ]
0.2111
0.1986
0.0759
0.0777
TVGE [ 56 ]
LGVQ [ 54 ]
0.1488
0.1688
0.4950
0.5223
EduAVQABench (Ours)
TVGE [ 56 ]
0.3950
0.3866
0.5580
0.5343
AIGVE-Bench [ 55 ]
TVGE [ 56 ]
0.2532
0.2630
0.1772
0.1685
TABLE IV: Cross-dataset training analysis using different training datasets. All models are evaluated on the same target benchmark without fine-tuning.
Fig. 5: Comparison between predicted scores and MOS across multiple quality dimensions and quality levels.
Fig. 6: Qualitative comparison of perceptual quality (top row) and prompt alignment (bottom row). We compare our EduVQA model against state-of-the-art baselines, IP-IQA and T2VQA, in each quality dimensions. In each video pair, the right video exhibits superior perceptual quality or prompt alignment compared to the left. EduVQA consistently aligns with human judgments, while IP-IQA and T2VQA produce rankings contrary to the MOS.
Fig. 7: gMAD competition results between T2VQA and our EduVQA. Top: perceptual quality comparison; Bottom: prompt alignment comparison. zoomed-in view for clarity.
ID
Fusion
MoE
Sub-dim.
Perception
Alignment
Vanilla
Per.
Aln.
ST
WL
PLCC
SRCC
PLCC
SRCC
1
✗
✗
✗
✗
✗
✗
0.845
0.836
0.744
0.737
2
✓
✗
✗
✗
✗
✗
0.863
0.851
0.756
0.750
3
✓
✗
✗
✗
✓
✓
0.869
0.859
0.760
0.751
4
✓
✗
✗
✓
✗
✓
0.860
0.849
0.762
0.754
5
✓
✗
✓
✗
✓
✗
0.873
0.862
0.747
0.741
TABLE V: Ablation results on cross-modality fusion, MoE configurations, and fine-grained sub-dimension supervision. “ST” and “WL” denote spatial–temporal perceptual and word-level alignment branches, respectively.
Num. Experts
Top-K
Perceptual Quality
Prompt Alignment
SRCC
PLCC
SRCC
PLCC
4
2
0.845
0.858
0.751
0.759
8
1
0.856
0.869
0.746
0.753
8
2
0.869
0.879
0.757
0.768
8
4
0.850
0.863
0.737
0.745
8
8
0.853
0.864
0.756
0.761
TABLE VI: Impact of Expert Number and Top-K Selection in MoE.
Video generation models (VGMs) are rapidly entering classrooms, yet existing benchmarks evaluate only perceptual quality, intrinsic faithfulness, generic safety, or video as a reasoning medium, and none assesses whether the outputs are educationally valid. In this work, we present EduVideoBench, the first balanced benchmark in the education domain, grounded in the Knowledge-Skills-Attitude (KSA) framework so that pedagogical adequacy and educational safety are evaluated jointly rather than as ad-hoc quality dimensions. Across five frontier VGMs, our results show substantial room for improvement across knowledge, skills, and attitude before they are classroom-ready. We complement this with a qualitative analysis of expert comments, finding that educational validity is multi-component, where a single misaligned element such as pacing, legibility, or notation can invalidate an otherwise correct video. We hope EduVideoBench will guide the development of VGMs that are pedagogically grounded and safe for the classroom.
Unggi Lee, Hoyoung Ahn, Yoon Choi +16
Korea University Sejong Campus · Cardiff Metropolitan University · Seoul National University +10
Recently, AI-driven video generation has attracted considerable attention. This surge increases the demand for reliable video quality assessment (VQA) metrics to evaluate AI-generated content (AIGC) videos and guide model optimization. Existing studies assess video quality through visual harmony, video-text consistency, and domain-specific alignment, yet lack quantitative metrics for measuring fidelity to physical laws. To address this limitation, we present a novel benchmark that evaluates the quality of AIGC videos based on their compliance with physical principles by quantitatively measuring geometric consistency across frames extracted from generated sequences. This serves as a proxy for estimating the extent to which generated videos conform to real-world physical rules. Specifically, GeoCon-Bench captures global motion through translation estimation, fits homography or fundamental matrix models using background correspondences, and reports complementary metrics, including inlier ratio and geometric error. We also release a dataset containing 20 scenes across six motion categories. Experiments on state-of-the-art AIGC models demonstrate the reliability of GeoCon-Bench as a video quality assessment metric.
Evaluating AI-generated video (AIGV) quality hinges on three crucial dimensions: visual quality, dynamic quality, and text-video alignment. While numerous evaluation datasets and algorithms have been proposed, existing approaches are constrained by two limitations: the absence of systematic definitions for evaluation dimensions, and the isolated treatment of the three dimensions in separate models. Therefore, we introduce Q-Save, a holistic benchmark dataset and unified evaluation model for AIGV quality assessment. The Q-Save dataset contains nearly 10,000 video samples, each annotated with Mean Opinion Scores (MOS) and fine-grained attribution explanations across the three core dimensions. Leveraging this attribution-annotated dataset, we train the proposed Q-Save model, which adopts the SlowFast framework to balance accuracy and efficiency, and employs a three-stage training strategy with Chain-of-Thought (COT) formatted data: Supervised Fine-Tuning (SFT), Grouped Relative Policy Optimization (GRPO), and a final SFT round for stability, to jointly perform quality scoring and attribution generation. Experimental results demonstrate that Q-Save achieves superior performance in AIGV quality prediction while providing interpretable justifications. Code and dataset will be released upon publication.
Xiele Wu, Mingtao Chen, Zicheng Zhang +11
Shanghai Jiao Tong University · Hunyuan · Shanghai Innovation Institute