cs.CVAug 27, 2026
SaveVideo-FLAIR: Not Whether to Reason, But How
Organizations: Arizona State University
Abstract
Multimodal queries can require different types of reasoning. Some can be answered via perceptual reasoning, extracting information directly from the visual signal, while others require compositional reasoning that combines observations or deliberative reasoning that evaluates competing hypotheses. However, many existing methods apply a uniform reasoning strategy across queries, leading to unnecessary computation on simple tasks and insufficient reasoning on complex ones. We introduce Video-FLAIR, a training framework that learns to select the appropriate reasoning mode for each query using reinforcement learning. During training, the model generates responses under all three modes for the same prompt, enabling direct comparison. A composite reward compares these responses to favor the most effective one based on correctness, grounding, and cost, while discouraging unsupported or misaligned deliberation. This yields a supervision signal for learning adaptive reasoning without per-query annotations. Video-FLAIR improves accuracy over the Qwen2.5-VL base model by +5.4 on MathVista, +4.8 on Video-Holmes, and +4.8 on Video-MMMU, while reducing average token usage to 95 compared to 417 for always-thinking baselines.
Figures & tables
Figure 1: Video-FLAIR routes each query to one of three reasoning modes based on its complexity. Yellow marks the key evidence driving each mode: screen-text reads ( < direct > ), timestamped event anchors ( < concise > ), and multi-observation cues synthesized into a conclusion ( < deep > ).
Figure 2: Video-FLAIR training pipeline. (Left) Each video-query pair generates mode-structured rollouts: two controlled rollouts per mode ( < direct > , < concise > , < deep > ) and two < Adaptive > . A composite reward is computed for each rollout. GDPO [ 48 ] normalizes each reward component before aggregation. The policy is then updated via DAPO [ 81 ] . (Right) The verifier provides per-rollout feedback aggregated into . It is periodically refreshed via DPO [ 59 ] on highest- versus lowest-reward rollout pairs to remain calibrated with the improving policy.
Figure 3: Penalty scales with length.
Figure 4: Which mode wins? Decision regions over . More expensive modes win only when their accuracy-grounding gain exceeds the cost threshold.
Figure 5: Token-level credit assignment. Yellow spans (grounding tags, factual observations) receive amplified gradients, while purple spans (hedging, filler) are suppressed.
Figure 6: Reasoning mode distribution across representative benchmarks. Video-FLAIR selects < direct > for simple visual retrieval (AI2D), shifts toward < concise > as reasoning chains become necessary (MMMU, Video-Holmes), with < deep > reserved for the small fraction of queries requiring hypothesis elimination. Full distributions across remaining benchmarks are in § B .
| Mode | MMMU | EMMA | V-MMMU | Tok |
| Forced < direct > | 51.4 | 22.1 | 49.7 | 57 |
| Forced < concise > | 53.1 | 25.6 | 53.2 | 247 |
| Forced < deep > | 52.8 | 28.3 | 55.8 | 963 |
| Video-FLAIR (adaptive) | 54.5 | 27.3 | 56.9 | 67 |
Table 5: Reasoning mode forced or left adaptive at inference.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Per-category gains over the base model. Each bar shows the absolute accuracy change vs. the base model across five task categories (averaged over benchmarks within each group). Always-thinking models collapse on image perception (hallucination pressure) and spatial reasoning; auto-thinking recovers perception partially but lacks consistent gains. Video-FLAIR is the only method with positive gains across all five categories in both base models.
Figure 8: Reasoning mode distribution across remaining benchmarks. The < direct > -to- < concise > shift tracks benchmark cognitive demand: retrieval-oriented tasks (HallusionBench, Video-TT) remain < direct > -heavy, while inference-heavy and research-level benchmarks (EMMA, SciVideoBench) shift toward < concise > . < deep > usage stays below 6% throughout.
Figure 9: Social reasoning failure: < deep > under-invoked. Evaluating Darren Cross’s emotional state after shrinking the lamb (top) requires multi-frame deliberation across posture, gaze, and reaction cues, but the policy routes to < direct > and commits to a single-frame conclusion. Grounding tags ( <obj> , <box> , <t> ) appear inside the < direct > block, and signaling routing instability. Locating the Yellowjacket suit across timestamps (bottom) is a sequential scan well matched to < concise > , which correctly chains three grounded appearances.
Figure 10: Compositional-surface query requiring hypothesis testing; correct perceptual lookup. Determining whether the rover moves autonomously (top) requires falsifying the autonomous hypothesis across behavioral cues (controller presence, motion smoothness, gaze tracking). The policy routes to < concise > and chains co-location observations without testing any of those alternatives; < deep > was needed. Counting wheels (bottom) is a one-step lookup with no competing hypotheses; < direct > is cost-optimal.
Figure 11: Correct causal hypothesis testing; failure on temporal counting. The causality query (top) requires falsifying the green-cube hypothesis; < deep > correctly establishes the green cube does not enter the frame until after the initial impact, then confirms gold-cube contact via timestamped bounding boxes. The counting query (bottom) demands a running tally across the full video; the policy routes to < direct > , inspects a single frame, and reports objects instead of the correct , missing those that move in other segments.
Figure 12: Correct deliberative causal reasoning; over-routing a perceptual lookup. The distress query (top) presents three competing explanations (flame fear, physical discomfort, social overwhelm); < deep > tests each with timestamped grounding and rules out the first two before confirming social overwhelm via co-occurring onset and recovery cues. The object-identification query (bottom) is answerable from any single frame; < concise > chains three cross-frame confirmations when < direct > ’s one-step extraction was sufficient, wasting tokens with no accuracy gain.
Figure 13: DIRECT mode prompt. Targets fast, minimal-verification responses ( 60 <think> words). Used for controlled slots 1–2 during training.
Figure 14: CONCISE mode prompt. Targets compact grounded reasoning ( 200 <think> words) with optional spatio-temporal grounding tags. Used for controlled slots 3–4.
Figure 15: DEEP mode prompt. Targets thorough hypothesis-falsification reasoning ( 1000 <think> words) with grounding evidence. Used for controlled slots 5–6.
Figure 16: ADAPTIVE mode inference prompt. Instructs the model to select the cheapest reliable mode via a cost-first decision policy. Used for adaptive slots 7–8 at both training and inference time.
Figure 17: Verifier system prompt. The verifier (a Qwen3-VL model) receives each candidate completion and returns a structured JSON signal covering temporal grounding, spatial grounding, human alignment, preferred mode, reasoning quality, and span-level scores.
More in this topic
VLM Reasoning
Vision-language model reasoning studies how multimodal systems use visual evidence and language to answer questions, plan, and explain.
See all 853 papers