cs.CVMar 26, 2026
SaveReinforcing Structured Chain-of-Thought for Video Understanding
Organizations: Stony Brook University · Amazon
Abstract
Multi-modal Large Language Models (MLLMs) show promise in video understanding. However, their reasoning often suffers from thinking drift and weak temporal comprehension, even when enhanced by Reinforcement Learning (RL) techniques like Group Relative Policy Optimization (GRPO). Moreover, existing RL methods usually depend on Supervised Fine-Tuning (SFT), which requires costly Chain-of-Thought (CoT) annotation and multi-stage training, and enforces fixed reasoning paths, limiting MLLMs' ability to generalize and potentially inducing bias. To overcome these limitations, we introduce Summary-Driven Reinforcement Learning (SDRL), a novel single-stage RL framework that obviates the need for SFT by utilizing a Structured CoT format: Summarize -> Think -> Answer. SDRL introduces two self-supervised mechanisms integrated into the GRPO objective: 1) Consistency of Vision Knowledge (CVK) enforces factual grounding by reducing KL divergence among generated summaries; and 2) Dynamic Variety of Reasoning (DVR) promotes exploration by dynamically modulating thinking diversity based on group accuracy. This novel integration effectively balances alignment and exploration, supervising both the final answer and the reasoning process. Our method achieves state-of-the-art performance on seven public VideoQA datasets.
Figures & tables
Figure 1: Performance comparison and training paradigms for video reasoning models. (a) Performance comparison among several benchmarks. (b)-(d) Training paradigm analysis: (b) Pure RL often yields ungrounded and unstable Chain-of-Thought (CoT) outputs. (c) SFT+RL is costly and complex. In contrast, (d) SDRL (Summary-driven RL) utilizes a Structured CoT and self-supervision to achieve stable and grounded video reasoning.
Figure 2: Overview of the SDRL Framework, which introduces Structured Chain-of-Thought. The Policy Model ( ) generates reasoning sequences, each structured as Summary, Think, and Answer. The framework introduces two structured objectives implemented via token-wise weight: (a) Consistency of Vision Knowledge (CVK) and (b) Dynamic Variety of Reasoning (DVR). These structured weights, along with standard rewards (Accuracy, Format), are combined to derive the group advantage for policy optimization.
Figure 3: BLEU and sBERT scores between different predictions.
Figure 4: Accuracy comparison across different tag types under training-free, inference-only settings for selecting an appropriate structural format prior to RL optimization.
| CVK | DVR | Accuracy | ||||
| sBERT | BLEU | KL | Entropy | Dynamic | ||
| Orig. | - | - | - | - | - | 42.37 |
| (a) | ✓ | 43.85 | ||||
| (b) | ✓ | 46.32 | ||||
| (c) | ✓ | ✓ | 48.56 | |||
| (d) | ✓ | ✓ | ✓ | 46.71 | ||
Table 1: Ablation study of CVK and DVR modules on the EventFlowQA. Rows (a–g) use GT supervision, while (h–l) adopt self-supervision. Adding semantic (sBERT, BLEU) and distributional (KL, Entropy, Dynamic) constraints improves accuracy.
| Models | Training | Video Reasoning Benchmark | Video General Benchmark | |||||
| NExT-GQA | MMVU | VideoMMMU | VSIBench | MVBench | TempCompass | VideoMME | ||
| LLaMA-VID [ 19 ] | None | - | - | - | - | 41.9 | 45.6 | - |
| VideoLLaMA2 [ 4 ] | - | 44.8 | - | - | 54.6 | - | 47.9 | |
| LongVA-7B [ 43 ] | - | - | 23.9 | 29.2 | - | 56.9 | 52.6 | |
| VILA-1.5-8B [ 21 ] | - | - | 20.8 | 28.9 | - | 58.8 | - | |
| Video-UTR-7B [ 40 ] | - | - | - | - | 58.8 | 59.7 | 52.6 | |
Table 2: Evaluation of Video MLLMs showing performance (Accuracy %) across various benchmarks, categorized by their training strategy (SFT, RL, SFT+RL). Our model achieves state-of-the-art results by employing the RL-only strategy, demonstrating superior performance across most metrics. ‘Ours † ’ denotes the variant trained using the EventFlow dataset. VideoRFT * indicates evaluation with 16-frame input.
| Model size | Original | GT sup. | Self sup. |
| 3B | 41.46 | 44.47 (+3.01) | 43.86 (+2.40) |
| 7B | 42.37 | 48.56 (+6.19) | 54.28 (+11.91) |
Table 3: Effect of model size under different supervision types (Accuracy %). Smaller model gains more under GT supervision.
| Accuracy (%) | BLEU (%) | sBERT (%) | |
| w/o | 42.37 | 8.84 | 70.33 |
| w | 54.28 | 12.57 | 79.76 |
Table 4: Comparison of BLEU and sBERT scores based on CVK.
Figure 5: Comparison of CoTs and final answers generated by VideoChat-R1 and our proposed SDRL method. SDRL demonstrates superior grounding and logic flow, evidenced by higher BLEU and sBERT scores relative to the ground truth summary.
| Category | Description | Example Template |
| Action-Level Temporal Reasoning | ||
| Action Order Reasoning | Identify the action that happens immediately before or after a given action. | “What action happened immediately before { A } ?” |
| Temporal Causality | Select the action that causes or results from a given action. | “What was the most recent action that led to { A } ?” |
| Action Anticipation | Predict the most likely next action following a given step. | “What action will the person perform immediately after { A } ?” |
| Sequential Prediction | Choose the next likely action(s) in an ongoing sequence. | “Predict the next { k } actions after { A } .” |
| Duration Estimation | Estimate how long an action takes. | “How long does the action { A } take?” |
Table 5: Overview of the temporal reasoning templates used in the EventFlow dataset, grouped into action-level and video-level categories.
Figure 6: Comparison of the average output token length across six video benchmarks. We observe that Video-R1 consistently produces longer reasoning sequences than VideoRFT and SDRL, while SDRL yields the most concise outputs across all datasets.
Figure 7: Data distribution of the EventFlowQA-53K dataset. The dataset consists of two hierarchical levels: Action-level questions and Video-level questions. The outer ring shows the detailed proportions of each temporal reasoning category, including action order, duration, comparison, causality, anticipation, and others.
| Prompt Template | |
| {Question} {Type of Template} Given the video clip and the question, first summarize the sequence of actions observed in the video using <summary></summary>tags. Then, use that summary to guide your reasoning inside <think></think>tags. Finally, state the answer clearly in <answer></answer>tags. | |
| Type of Template | |
| multiple choice | Please provide only the single option letter (e.g., A, B, C, D, etc.). |
| numerical | Please provide the numerical value (e.g., 42 or 3.14). |
| OCR | Please transcribe text from the image/video clearly and provide your text answer. |
| free-form | Please provide your text answer. |
Table 6: Prompt for training and inference.
| Question Prompt (tag, accuracy) | Model Output |
| action (26.45%) | |
| Given a video clip, first list the sequence of actions observed in the video in order in <action></action>tags. Then, answer the question with output the thinking process in <think></think >tags, followed by the final answer in <answer></answer >tags. | <action> 1. put 2 tsp of olive oil add cut bread dices and fry 2. tear fresh lettuce leaves and add them with seasoned kale leaves 3. cook bacon in a pan until crispy 4. add roasted bread crumbs crispy bacon slices of parmesan cheese shaved to seasoned greens and pour dressing blend on top </action> |
| caption (26.95%) | |
| Given a video clip, first describe the sequence of actions observed in the video in the exact order they occur using <caption><caption>tags. Then, answer the question with output the thinking process in <think></think >tags, followed by the final answer in <answer></answer >tags. | <caption> A woman is standing in her kitchen. She picks up a piece of bread and places it on a cutting board. She then cuts the bread into slices. Next, she puts some butter in a frying pan and cooks the foie gras. After that, she cooks the apple slices in the same pan. Finally, she places the apple slices and the foie gras on a plate. </caption > |
| event (16.12%) | |
Table 7: Effect of different prompt tags on model behavior. For each tag, the left column shows the question prompt and the right column shows the model output. Accuracy values reflect the QA performance under each tag. The examples show how prompt structure shapes the granularity, ordering, and coherence of the generated summaries.
Figure 8: Summary quality during training measured by . The curve shows steady improvement without collapse.
Figure 9: An example on Temcompass dataset. Video-R1 and Video-RFT generate lower information density descriptions ( All red part is the video description ). Video-R1’s description is iterative, which may contribute to a longer reasoning process. This extended path may be associated with the observed instability in the inference chain (yellow highlighting) , where intermediate thoughts contradict the final outcome, resulting in the selection of the incorrect answer. Video-RFT’s context is more dispersed, which potentially leads to an initial misrepresentation of the visual facts. SDRL produces a concise, highly focused summary by directly extracting the necessary visual facts (”ball on the right, hoop on the left”). This concentrated context supports a shorter and more direct reasoning path.
Figure 10: An example on Temcompass dataset. Although all three methods describe roughly the same scene ( descriptions in red ), their narrative focus differs significantly. SDRL produces a concise and coherent summary that preserves global context, enabling the model to capture the overall movement pattern of the sheep ( green highlighting ) and correctly identify the key action in the video. In contrast, Video-R1 and Video-RFT provide fragmented or overly broad descriptions that dilute the salient motion cues, causing the reasoning to focus on partial details and miss the essential dynamics of the scene.
Figure 11: An example on MMVU dataset. Video-R1 provides almost no grounding in the actual video content and instead jumps directly into option analysis, making it highly susceptible to thinking drift. Video-RFT includes an initial video description, but its reasoning process gradually shifts away from the visual evidence, leading to potential drift as well. SDRL maintains tight grounding by consistently referencing key visual cues throughout the reasoning process, thereby preventing drift and enabling more reliable decision-making.
Figure 12: An example on MMVU dataset. Both Video-R1 and VideoRFT devote excessive attention to analyzing the answer options, producing overly long reasoning chains that deviate from the actual visual evidence and easily lead to thinking drift. In contrast, SDRL anchors the reasoning in the video content, maintains a concise and focused explanation, and thus avoids unnecessary detours caused by option-driven overanalysis.
Figure 13: A failure case from EventFlow dataset. Without ground-truth supervision, SDRL may produce incomplete or temporally misaligned summaries, particularly when multiple actions in the video exhibit similar visual patterns.
Figure 14: A failure case from MVBench dataset. When actions become highly fine-grained, SDRL often generates coarse, high-level summaries instead of enumerating precise motion steps.