Video models answer from sampled frames and may miss the decisive moment. A common remedy is to commit to an answer, re-examine the video, and confirm or revise it, a loop now mainly learned with reinforcement learning (RL). We find that RL erodes the initial answer, a phenomenon we term confidence erosion, where confidence means the label-verified reliability of the initial answer instead of self-reported certainty: because errors can be repaired later, a correct initial answer earns less reward, and the initial answer stalls even as the final answer's gain over it rises. We propose the principle confident first, reflective second: the initial answer must stay reliable on its own, above a floor set by a reference policy, and reflection may only add to it. Our method, CoFire, implements this principle with one mechanism for each erosion channel: reward incentives, parameter drift, and gradient interference. CoFire reaches 67.0% mean accuracy on Video-MME, MLVU, and LongVideoBench, 1.7 points above the strongest compared 7B method, and improves initial-answer and final accuracy over outcome-reward RL by 4.1 and 3.7 points on average: a stronger initial answer carries through to a stronger final one.
Figures & tables
Figure 1: Reflection gain is not reflective progress. Video-MME policies by initial-answer accuracy and reflection gain Δ ; dashed lines are iso-lines of final accuracy. From the same SFT policy, Outcome RL moves up (more gain, same initial answer), while CoFiRe moves right to a higher final accuracy. Gray dots: configurations from Table 8 .
Figure 2: Answering and re-examining a Video-MME example. From a 2 fps preview and an eight-frame first look showing only two mugshots, Spre answers y0= C. Sgate reports low confidence and chooses Seek ; frames at 258–261 s reveal a third mugshot, and Srev revises to yf= D, a fix . More examples are in Appendix G .
Figure 3: Overview of CoFiRe . Top: G rollouts of one policy answer ( Spre ), choose how to check ( Sgate ), and confirm or revise ( Srev ). Middle: stage-routed advantages penalize the initial answer of an err-then-fix rollout but credit the repair. Bottom: one mechanism per erosion channel: λ rises below the initial-answer accuracy floor (incentive), verified reference answers anchor the decision token (drift), and only the conflicting part of the reflection gradient is removed (interference).
Figure 4: Penalizing revision does not protect the initial answer. Video-MME Seek rate, the share of questions routed to Seek (bars, left) and final accuracy (line, right), varying one penalty from the default β=1.5 , γ=0.2 .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Fix: the gate catches a misread first-look frame (Video-MME 180-3, 66 s video). Spre reports KOR from a scoreboard that lists CHN first; the gate re-reads it, and dense retrieval around the finish confirms the revision to C.
Figure 6: Fix: localizing a few seconds of evidence in a long video (Video-MME 766-1, 19 min). Spre guesses the athlete from the uniform color; the gate requests the bib, and all retrieved frames fall in the treatment segment, where the bib reads 377.
Figure 7: Fix: replacing a language prior with on-screen text (Video-MME 713-3, 42 min). Spre answers from the documentary title; retrieval returns the full headline attributing the threat to Dutch drug gangs.
Figure 8: Failure: a correct initial answer broken by a salient but irrelevant number (Video-MME 568-1, 13 min). Spre is correct but not grounded in an on-screen figure; the query retrieves a 1916 rail-network statistic that matches option A verbatim, and the revision attributes it to the traveler.
Current multimodal reflection mechanisms for long video understanding predominantly rely on closed-loop self-reflection within internal parameters. Lacking objective external evidence, models are frequently trapped in blind confidence and often fail to correct errors. Furthermore, applying reinforcement learning to multi-stage reflection pipelines introduces severe policy coupling, which is exacerbated by a critical scarcity of dedicated training data. To address these limitations, this work proposes Reflect-R1, the first Evidence-Driven self-correction framework for long video understanding. The framework constructs a three-stage pipeline consisting of intuition, verification, and arbitration. By dynamically retrieving objective visual evidence to verify initial intuitions and autonomously executing multiple temporal searches to resolve conflicts, it completely breaks the hallucination loop. To overcome policy coupling, we design a stage-decoupled reinforcement learning algorithm named SD-GRPO that independently computes advantage functions across different reasoning stages. Concurrently, we construct a dataset of 120K samples to bridge the training data gap. Extensive experiments on benchmarks such as VideoMME and LongVideoBench demonstrate that Reflect-R1 achieves state-of-the-art performance. Our method significantly improves the genuine rectification rate and enables authentic self-correction strictly grounded in objective evidence.
Shuimu Chen, Yuteng Chen, Yuanshen Guan +7
Tsinghua University · Nanyang Technological University · University of Science and Technology of China +4
Video reasoning has advanced with large multimodal models (LMMs), yet their inference is often a single pass that returns an answer without verifying whether the reasoning is evidence-aligned. We introduce Reinforce to Learn, Elect to Reason (RLER), a dual paradigm that decouples learning to produce evidence from obtaining a reliable answer. In RLER-Training, we optimize the policy with group-relative reinforcement learning (RL) and 3 novel task-driven rewards: Frame-sensitive reward grounds reasoning on explicit key frames, Think-transparency reward shapes readable and parsable reasoning traces, and Anti-repetition reward boosts information density. These signals teach the model to emit structured, machine-checkable evidence and potentiate reasoning capabilities. In RLER-Inference, we apply a train-free orchestrator that generates a small set of diverse candidates, parses their answers and cited frames, scores them by evidence consistency, confidence, transparency, and non-redundancy, and then performs a robust evidence-weighted election. This closes the loop between producing and using evidence, improving reliability and interpretability without enlarging the model. We comprehensively evaluate RLER against various open-source and RL-based LMMs on 8 representative benchmarks. RLER achieves state of the art across all benchmarks and delivers an average improvement of 6.3% over base models, while using on average 3.1 candidates per question, indicating a favorable balance between compute and quality. The results support a simple thesis: making evidence explicit during learning and electing by evidence during inference is a robust path to trustworthy video reasoning.
Songyuan Yang, Weijiang Yu, Jilin Ma +5
National University of Defense Technology · Sun Yat-sen University
Video diffusion models have made rapid progress in perceptual realism and temporal coherence, but they remain primarily optimized for plausible generation rather than verifiable reasoning. This limitation is especially pronounced in tasks where generated videos must satisfy explicit spatial, temporal, or logical constraints. Inspired by the role of reinforcement learning with verifiable rewards (RLVR) in reasoning-oriented language models, we introduce VideoRLVR, a practical recipe for optimizing video diffusion models with rule-based feedback. VideoRLVR formulates video reasoning as the generation of verifiable visual trajectories and consists of an SDE-GRPO optimization backbone, dense decomposed rewards, and an Early-Step Focus strategy for efficient training. The Early-Step Focus strategy restricts policy optimization to the early denoising phase, reducing training latency by about 40% while preserving performance. We evaluate VideoRLVR on Maze, FlowFree, and Sokoban, three procedurally generated domains with objective success criteria. Across these tasks, VideoRLVR consistently improves over supervised fine-tuning baselines, with dense decomposed rewards proving especially important in low-success-rate settings. Our RL-optimized model also outperforms the evaluated proprietary and open-source video generation models on these verifiable reasoning benchmarks and out-of-domain benchmarks. These results suggest that verifiable RL can move video models beyond perceptual imitation toward more reliable rule-consistent visual reasoning.
Tinghui Zhu, Sheng Zhang, James Y. Huang +5
University of California, Davis · Microsoft Research · University of Southern California +1