VQQA: An Agentic Approach for Video Evaluation and Quality Improvement

Yiwen Song; Tomas Pfister; Yale Song

cs.CVMar 12, 2026

VQQA: An Agentic Approach for Video Evaluation and Quality Improvement

Authors: Yiwen Song, Tomas Pfister, Yale Song

Organizations: Google

Abstract

Despite rapid advancements in video generation models, aligning their outputs with complex user intent remains challenging. Existing test-time optimization methods are typically either computationally expensive or require white-box access to model internals. To address this, we present VQQA (Video Quality Question Answering), a unified, multi-agent framework generalizable across diverse input modalities and video generation tasks. By dynamically generating visual questions and using the resulting Vision-Language Model (VLM) critiques as semantic gradients, VQQA replaces traditional, passive evaluation metrics with human-interpretable, actionable feedback. This enables a highly efficient, closed-loop prompt optimization process via a black-box natural language interface. Extensive experiments demonstrate that VQQA effectively isolates and resolves visual artifacts, substantially improving generation quality in just a few refinement steps. Applicable to both text-to-video (T2V) and image-to-video (I2V) tasks, our method achieves absolute improvements of +11.57% on T2V-CompBench and +8.43% on VBench2 over vanilla generation, significantly outperforming state-of-the-art stochastic search and prompt optimization techniques.

Figures & tables

Figure 1: Videos generated by V Q Q A , showing improvements in categories such as numeracy, interaction, dynamic attributes, and spatial relationship.

Figure 2: Qualitative example of V Q Q A iterative refinement process. Utilizing low-scoring QA pairs to isolate visual flaws, V Q Q A constructs an optimized prompt that successfully mitigates the localized artifacts in the next generation.

Figure 3: The V Q Q A framework: Given generation conditions $C$ and a prompt $p_{t}$ , the model $M$ produces a video $v_{t}$ . The multi-agent framework uses a Question Generation (QG) agent to formulate visual queries $Q$ and a Question Answering (QA) agent to evaluate the video and produce a score report. These outputs inform the Prompt Refinement (PR) agent , which uses semantic gradient to update the prompt for the next iteration. Finally, a Global VLM Rater assesses the candidate set of generated videos against the original conditions to select the optimal video $v^{*}$ .

Method		Consist-attr	Dynamic-attr	Spatial	Motion	Action	Interaction	Numeracy	AVG
Vanilla Generation
	CogVideoX-5B ( Yang et al., 2024 )	61.64	2.19	51.72	26.58	53.33	60.69	37.06	41.89
Prompt Optimizer
	VPO ( Cheng et al., 2025 )	76.43	4.52	54.18	26.07	64.09	68.83	45.72	48.55
Best-of-N $\And$
	VQAScore ( Lin et al., 2024 )	80.83	2.48	56.62	26.24	64.31	65.96	44.48	48.70

Table 1: T2V-CompBench evaluation results on CogVideoX-5B . Performance is reported using

N=5

for Best-of-

N

strategy, and 4 optimization rounds for V Q Q A . Vanilla generation results are obtained from the official leaderboard. Best and second-best scores are highlighted in green and yellow. All numbers are percentages.

Method		Creativity	Commonsense	Controllability	Human Fidelity	Physics	Total Score
Vanilla Generation
	CogVideoX-5B ( Yang et al., 2024 )	42.99	54.19	23.81	80.13	38.57	41.98
Prompt Optimizer
	VPO ( Cheng et al., 2025 )	44.44	54.19	24.79	84.45	39.01	43.34
Best-of-N $\And$
	VQAScore ( Lin et al., 2024 )	51.51	57.92	30.96	79.85	42.49	46.95

Table 2: VBench2 evaluation results on CogVideoX-5B . Performance is reported with

N=5

(Best-of-

N

) and 4 optimization rounds ( V Q Q A ). All numbers are percentages.

Method		I2V Background	Background Consistency	I2V Subject	Subject Consistency	Motion Smoothness	Avg Score
Vanilla Generation
	CogVideoX-5B-I2V ( Yang et al., 2024 )	96.74	96.42	97.19	94.34	98.40	96.62
Best-of-N $\And$
	VQAScore ( Lin et al., 2024 )	98.96	97.19	97.61	95.33	98.29	97.48
Best-of-N $\And$ VLM-Rating
	Gemini-3-Pro ( Pichai et al., 2025 )	99.03	97.37	97.77	95.68	98.32	97.63

Table 3: VBench-I2V evaluation results on CogVideoX-5B-I2V . Performance is reported with

N=5

(Best-of-

N

) and 4 optimization rounds ( V Q Q A ). Vanilla Generation results are obtained from the leaderboard. All numbers are percentages.

Method	Precision	Q-Recall	E2E-Recall
VLM Direct Analysis	99.94	–	70.18
V Q Q A	99.28	96.55	82.08

Table 4: Quality of the visual flaw identification process (Judged by Gemini-3-Flash) . Both methods use Gemini-3-Pro as the backbone. V Q Q A Score threshold for E2E-Recall measurement is set to 60. All numbers are percentages.

Panel of Figure 4 — Figure 4: Convergence analysis of V Q Q A on T2V-CompBench. Evaluations are performed for CogVideoX-5B generations using Gemini-3-Pro. (a) Correlations between the maximum global score $S^{*}_{t}$ and the T2V-CompBench metric across optimization steps. The blue shaded region indicates $\pm 1$ standard deviation of $S^{*}_{t}$ across the 1400 evaluated samples. The detailed performance breakdown for each individual category is provided in Appendix C.4 . (b) Sensitivity analysis of average iterations to converge given the patience window $k$ and saturation threshold $\epsilon$ ( Equation 8 ).

Method		Consist-attr	Dynamic-attr	Spatial	Motion	Action	Interaction	Numeracy	AVG
Vanilla Generation
	Veo3.1 ( Google DeepMind, 2025 )	76.45	25.70	59.76	51.18	71.81	62.87	43.77	55.93
Best-of-N $\And$
	VQAScore ( Lin et al., 2024 )	78.21	26.13	63.76	52.03	74.83	66.99	54.59	59.51
	VideoScore2 ( He et al., 2025b )	78.19	24.76	60.99	54.49	73.52	65.98	46.7	57.80
Best-of-N $\And$ VLM-Rating

Table 5: T2V-CompBench evaluation results on Veo3.1 . Performance is reported with

N=5

(Best-of-

N

) and 4 optimization rounds (VQQA). All numbers are percentages.

Figure 5: Ablation Study on Global Selection Mechanism . The number of prompt optimization rounds for V Q Q A is fixed at N = 4.

Method		Consist-attr	Dynamic-attr	Spatial	Motion	Action	Interaction	Numeracy	AVG
V Q Q A GS-in-the-loop
	Gemini-3-Pro ( Pichai et al., 2025 )	81.77	8.40	67.05	27.94	68.15	70.83	50.69	53.55
	Gemini-2.5-Pro ( Comanici et al., 2025 )	83.92	6.32	64.47	28.19	66.33	72.69	46.67	52.66
	AVG	82.85	7.36	65.76	28.07	67.24	71.76	48.68	53.11
V Q Q A Standard
	Gemini-3-Pro ( Pichai et al., 2025 )	84.58	7.92	66.03	27.73	68.50	68.54	50.91	53.46

Table 6: Ablation Study on GlobalScore in the loop . Number of V Q Q A optimization rounds is fixed at N = 4. Best and second-best average scores across models for each strategy are highlighted in green and yellow. All numbers are percentages.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Method	#VLM Calls			T2V-CompBench Total ( $N=5,k=0$ )
Method	Generation	Evaluation	Total	T2V-CompBench Total ( $N=5,k=0$ )
Vanilla Generation	0	0	0	0
Best-of- $N$ (VLM-Rating)	0	$N$	$N$	5
V Q Q A (Ours)	$(4+k)T$	$T+1$	$(5+k)T+1$	$\sim 7.23$

Appendix

Table 7: Comparison of VLM inference costs per prompt. For V Q Q A ,

k

denotes the number of reference images in I2V settings (

k=0

for T2V). When compared against a Best-of-5 (

N=5

) baseline, V Q Q A requires an average of

T=1.245

optimization rounds to converge on T2V-CompBench (capped at a maximum of 4 optimization rounds).

Figure 6: Full convergence analysis of V Q Q A on T2V-CompBench. Evaluations are performed using CogVideoX-5B as the generator and Gemini-3-Pro as the VLM.

Figure 7: Side-by-side comparison of direct prompting, VPO, and V Q Q A using CogVideoX-5B.

Figure 8: Side-by-side comparison of direct prompting, VPO, and V Q Q A using CogVideoX-5B.

Figure 9: Side-by-side comparison of direct prompting, VPO, and V Q Q A using CogVideoX-5B.

Figure 10: Side-by-side comparison of direct prompting, VPO, and V Q Q A using CogVideoX-5B.

Figure 11: V Q Q A Example Trajectory Iteration 0

Figure 12: V Q Q A Example Trajectory Iteration 1

Figure 13: V Q Q A Example Trajectory Iteration 2

Prompt Optimization for T2V Generation

See all 12 papers

VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality
Aug 26, 2026Longteng Jiang, DanDan Zheng, Qianqian Qiao +7Video Generation Evaluation Video Quality Assessment
Ant Group, Beijing, China · Beijing Film Academy, Beijing, China · State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China
Comparison Drives Preference: Reference-Aware Modeling for AI-Generated Video Quality Assessment
Apr 18, 2026Minghao Zou, Gen Liu, Guanghui Yue +5Video Quality Assessment Video Generation Evaluation
Cardiff University Cardiff, UK · Shandong University of Science and Technology Qingdao, China · Shenzhen University Shenzhen, China +2
Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence
Aug 12, 2026Aman Tyagi, Hemanth Boinpally, Jonathan Chen +2Image-to-Video Generation Agentic Video Generation
Google Cloud · Google DeepMind

Aug 26, 2026J/K move · Enter open · S save

VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality

Longteng Jiang, DanDan Zheng, Qianqian Qiao, Heng Huang+6

Ant Group, Beijing, China · Beijing Film Academy, Beijing, China · State Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China

Video Generation Evaluation Video Quality Assessment Vision-Language Model-as-a-Judge

Open paper AskPDF arXiv

We introduce VGA-BenchV2, an extended human-aligned benchmark and optimization framework for jointly evaluating and improving video generation quality and aesthetic value. Built upon VGA-Bench, VGA-BenchV2 preserves the original fine-grained taxonomy with two primary dimensions-Aesthetic and Generation-and 52 sub-dimensions. Guided by this taxonomy, we curate 1,016 diverse prompts and collect over 60,000 videos generated by 12 mainstream video generation models. More importantly, VGA-BenchV2 substantially expands human-labeled supervision by adding 36,000 task-level annotations, including 16,200 for aesthetic quality, 13,200 for aesthetic tagging, and 6,600 for generation quality, corresponding to 13.46x, 11.15x, and 1.55x scale-ups over VGA-Bench, respectively. Leveraging this enlarged annotation corpus, we develop a hybrid evaluator architecture consisting of VAQA-Net for continuous aesthetic scoring and two Qwen-based Large Vision-Language Model evaluators, VTag-Net and VGQA-Net, for aesthetic tagging and generation quality assessment. Extensive experiments demonstrate strong alignment with human judgments across diverse generation models. Beyond evaluation, VGA-BenchV2 further introduces an evaluation-to-optimization pipeline, where the learned aesthetic evaluator serves as a reward model for reinforcement learning-based generator fine-tuning. This closes the loop from benchmark construction and human supervision to automated evaluation and model optimization, enabling video generators to improve not only in realism but also in aesthetic quality and human preference alignment. Resources are available at https://huggingface.co/datasets/BestiVictoryLab/VGA-Bench.

Abstract

Prompt Optimization for T2V Generation

Explore similar work

VGA-BenchV2: An Expanded Unified Benchmark and Multi-Model Framework for Evaluating Video Aesthetics and Generation Quality

Comparison Drives Preference: Reference-Aware Modeling for AI-Generated Video Quality Assessment

Beyond Trial-and-Error: Agentic Optimization for Image-to-Video Adherence