cs.AINov 18, 2026

EVA-RSI: Recursive Self-Improvement of Video Evaluators for Judging Temporal Quality

Authors: Yu Yuan, Ruizhou Ding, Shaoheng Fang, Stanley H. Chan, Zhuo Chen

Organizations: Roblox · Purdue University · University of Texas at Austin

Abstract

Video evaluators play an important role in video generation, yet they miss motion defects, limiting the quality of generated videos. Recursive self-improvement (RSI) has made strong progress in mathematics and coding, where explicit verifiers provide reliable feedback on a single goal. Video evaluation poses two difficulties for RSI: it has no verifier, and it must judge many dimensions at once, such as physics, smoothness and dynamics, where a gain on one can cost another. We introduce EVA-RSI, a multi-agent framework for exploring RSI under these two constraints. We build a fixed proxy exam with construction-defined targets by applying controlled changes to videos, such as temporal reversal and frame dropping. These edits supply supervision without human annotation. A small set of human-labeled real videos anchors the exam to human judgment. Each dimension has an LLM agent that proposes improvements. A leader agent coordinates the proposals, and a Nash bargaining rule selects updates that balance progress across dimensions. Humans design the exam, review candidate changes and revise the curriculum between stages. Starting from Qwen3-VL-8B, EVA-RSI improves motion evaluation, with the largest gain on smoothness, and the gains carry over to defect types never seen during the run. On the dynamics subsets of public benchmarks, our evaluator achieves the highest observed scores among 21 tested models. As an RL reward, it increases generated motion and earns human preference. A data-selection pipeline incorporating it also increases motion.

Figures & tables

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Video-RSI: Recursive Self-Improvement of Video Understanding Agents via Harness Evolution

    Sep 29, 2026Bingjun Luo, Jialin Guo, Siqi LiVideo UnderstandingAgent Harness Optimization

  2. VideoHarness-SRSI: Structured Recursive Self-Improvement for Video Understanding Harness

    Oct 14, 2026Zihao Lin, Haoyu Dong, Haobo Xu +10LLM Agent HarnessesRecursive Self-Improvement