cs.CVNov 17, 2026

VIVID: Beyond Static Multi-Image Reasoning with Localized Visual Differences and Textual Entity Dynamics in Video MLLMs

Authors: Chengqian Ma, Yuanze Zeng, Hongyu Wang, Mingfeng Lin, YiCheng Xiao, Yu Chen, Pengyu Li, WenHao Wang, 

Organizations: Peking University · Shanghai Jiao Tong University · Harbin Institute of Technology · Southeast University · 

Abstract

Video question answering under tight frame budgets requires preserving localized motion and event order, yet current video multimodal large language models (MLLMs) reduce comprehension to static multi-image reasoning by discarding unselected frames and encoding keyframes independently: (1) Spatially, whole-frame pooling dilutes small moving targets into static backgrounds; (2) Temporally, dropping intermediate frames or averaging tokens erases inter-frame motion, confusing direction, speed, and order. We propose VIVID (Visual Inter-frame Variations and Individual Dynamics), which moves beyond static snapshots like human vision: keyframes capture core states while inter-frame Patch Diffs preserve neighboring changes within KfK_f keyframes. Autonomous Delta-Driven Keyframe Planning (ADKP) selects anchors from a 3 fps3\text{ fps} pool and feeds surrounding patch differences into Dual-Road Change Memory (DRCM), forming Dynamic Tokens at zero extra anchor tokens alongside a camera-stabilized Active Motion Region of Interest (ROI) and chronological Entity State Chains. Adding 0.22%0.22\% trainable parameters to a frozen 27B27\text{B} backbone, VIVID-Small (5050 examples, ∼1.5\sim 1.5 GPU hours) beats all eleven baselines in overall accuracy across 8,0838,083 questions and trails only VIVID-Large (3,6003,600 examples), which leads on MotionBench, Video-MME, and TempCompass.

Figures & tables

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList