Long-Video Understanding

Follow this topic Ask about this topic

Latest papers 147

18 papers in the last four weeks, down 5% on the four weeks before. 0.1% of all new papers.

All topics

VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG
Apr 7, 2026Honghao Fu, Miao Xu, Yiwei Wang +3Multimodal RAG Long-Video Understanding
University of Queensland · University of California, Merced · Institute of Automation, CAS +1
Query-Conditioned Evidential Keyframe Sampling for MLLM-Based Long-Form Video Understanding
Apr 1, 2026Yiheng Wang, Lichen Zhu, Yueqian Lin +4Query-Aware Video Frame Selection Long-Video Understanding
Duke University, Durham, North Carolina, USA · Independent Researcher
Diagnosing Long-Video Quantitative Reasoning in Multimodal LLMs via Enumeration and Counting
Mar 31, 2026Fumihiko Tsuchiya, Taiki Miyanishi, Shunsuke Yasuki +5Long-Video QA Multimodal Large Language Models
The University of Tokyo, Japan · Institute of Science Tokyo, Japan · National Institute of Informatics, Japan
Scaling the Long Video Understanding of Multimodal Large Language Models via Visual Memory Mechanism
Mar 31, 2026Tao Chen, Kun Zhang, Qiong Wu +5Long-Video Understanding Memory-Augmented Video Understanding
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · National University of Defense Technology, 230000, P.R. China.
AdaptToken: Entropy-based Adaptive Token Selection for MLLM Long Video Understanding
Mar 30, 2026Haozhe Qi, Kevin Qu, Mahdi Rad +3Long-Video Understanding Multimodal Large Language Models
Microsoft Spatial AI Lab · EPFL · ETH Zurich
Seeing the Scene Matters: Revealing Forgetting in Video Understanding Models with a Scene-Aware Long-Video Benchmark
Mar 28, 2026Seng Nam Chen, Hao Chen, Chenglam Ho +4Long-Video Understanding Vision-Language Models
CUHK (SZ) · University of Cambridge · UESTC +3
GIFT: Global Irreplaceability Frame Targeting for Efficient Video Understanding
Mar 26, 2026Junpeng Ma, Sashuai Zhou, Guanghao Li +9Video Frame Selection Video Understanding
Institute of Science and Technology for Brain-inspired Intelligence, Fudan University · State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · Alibaba Group Holding Limited +2
VideoTIR: Accurate Understanding for Long Videos with Efficient Tool-Integrated Reasoning
Mar 26, 2026Zhe Gao, Shiyu Shen, Taifeng Chai +7Tool Use in VLMs Efficient VLM Inference
Nanjing University, Nanjing, China · Nankai University, Tianjin, China · East China Normal University, Shanghai, China +3
VideoDetective: Clue Hunting via both Extrinsic Query and Intrinsic Relevance for Long Video Understanding
Mar 23, 2026Ruoliu Yang, Chu Wu, Caifeng Shan +2Long-Video Understanding Long-Video QA
Nanjing University · Institute of Automation, Chinese Academy of Sciences
VideoSeek: Long-Horizon Video Agent with Tool-Guided Seeking
Mar 20, 2026Jingyang Lin, Jialian Wu, Jiang Liu +6Long-Video Understanding Video Reasoning
AMD · University of Rochester
LVOmniBench: Pioneering Long Audio-Video Understanding Evaluation for Omnimodal LLMs
Mar 19, 2026Keda Tao, Yuhua Zheng, Jia Xu +13Audio-Visual Understanding Long-Video Understanding
Zhejiang University · Westlake University · Ant Group +4
VideoAtlas: Navigating Long-Form Video in Logarithmic Compute
Mar 18, 2026Mohamed Eltahir, Ali Habibullah, Yazan Alshoibi +3Long-Video Understanding Video Representation Learning
King Abdullah University of Science and Technology (KAUST), Thuwal, Saudi Arabia · Department of Computer Science, King Khalid University (KKU), Abha, Saudi Arabia · Department of Computer Science, Edge Hill University, Ormskirk, England
Shot-Aware Frame Sampling for Video Understanding
Mar 18, 2026Mengyu Zhao, Di Fu, Yongyu Xie +4Video Frame Selection Long-Video Understanding
ByteDance · Rutgers University · Georgia Tech
Symphony: A Cognitively-Inspired Multi-Agent System for Long-Video Understanding
Mar 18, 2026Haiyang Yan, Hongyun Zhou, Peng Xu +2Long-Video Understanding Multi-Agent LLM Systems
Institute of Automation, Chinese Academy of Sciences · School of Future Technology, University of Chinese Academy of Sciences · Kuaishou Technology
Question-guided Visual Compression with Memory Feedback for Long-Term Video Understanding
Mar 16, 2026Sosuke Yamao, Natsuki Miyahara, Yuankai Qi +1Long-Video Understanding Temporal Video Understanding
Fujitsu Research · Macquarie University
MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
Mar 14, 2026Arushi Goel, Sreyan Ghosh, Vatsal Agarwal +16Multimodal Understanding Multimodal Reasoning
NVIDIA, USA · University of Maryland, College Park, USA
VirtueBench: Evaluating Trustworthiness under Uncertainty in Long Video Understanding
Mar 7, 2026Xueqing Yu, Bohan Li, Yan Li +1VLM Evaluation Video-Language Model Evaluation
Peking University · ByteDance
Think, Then Verify: A Hypothesis-Verification Multi-Agent Framework for Long Video Understanding
Mar 5, 2026Zheng Wang, Haoran Chen, Haoxuan Qin +3Long-Video Understanding Long-Video QA
College of Computer Science, Zhejiang University of Technology, Zhejiang, China · Zhejiang Key Laboratory of Visual Information Intelligent Processing, Zhejiang, China · UC Berkeley, CA, USA +1
From Verbatim to Gist: Distilling Pyramidal Multimodal Memory via Semantic Information Bottleneck for Long-Horizon Video Agents
Mar 2, 2026Niu Lian, Yuting Wang, Hanshu Yao +5Multimodal Memory Memory-Augmented Video Understanding
Tsinghua Shenzhen International Graduate School, Tsinghua University · Harbin Institute of Technology, Shenzhen · Peng Cheng Laboratory
Wavelet-based Frame Selection by Detecting Semantic Boundary for Long Video Understanding
Feb 28, 2026Wang Chen, Yuhui Zeng, Yongdong Luo +5Long-Video Understanding Query-Aware Video Frame Selection
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · College of Computer and Data Science, Fuzhou University
LongVideo-R1: Smart Navigation for Low-cost Long Video Understanding
Feb 24, 2026Jihao Qiu, Lingxi Xie, Xinyue Huo +2Long-Video Understanding Long-Video QA
University of Chinese Academy of Sciences · Huawei Consumer Business Group
Think with Grounding: Curriculum Reinforced Reasoning with Video Grounding for Long Video Understanding
Feb 21, 2026Houlun Chen, Xin Wang, Guangyao Li +4Long-Video Understanding Video QA
Tsinghua University, Beijing, China
Learning Compact Video Representations for Efficient Long-form Video Understanding in Large Multimodal Models
Feb 19, 2026Yuxiao Chen, Jue Wang, Zhikang Zhang +8Long-Video Understanding Video Token Compression
Amazon AGI
ReMoRa: Multimodal Large Language Model based on Refined Motion Representation for Long-Video Understanding
Feb 18, 2026Daichi Yashima, Shuhei Kurita, Yusuke Oda +1Long-Video Understanding Multimodal Large Language Models
Keio University · NII LLMC · NII
Agentic Very Long Video Understanding
Jan 26, 2026Aniket Rege, Arka Sadhu, Yuliang Li +5Question Answering Egocentric Video Understanding
Reality Labs Research at Meta · University of Wisconsin-Madison · Work done during internship at Meta
Training-free Uncertainty Guidance for Complex Visual Tasks with MLLMs
Oct 1, 2025Sanghwan Kim, Rui Xiao, Stephan Alaniz +2Multimodal Large Language Models Long-Video Understanding
1Technical University of Munich · 2Munich Center for Machine Learning · 20263Helmholtz Munich +2
NeMo: Needle in a Montage for Video-Language Understanding
Sep 29, 2025Zi-Yuan Hu, Shuo Liang, Duo Zheng +10Temporal Video Grounding Video-Language Model Evaluation
The Chinese University of Hong Kong · Phoenix TV · Stanford University +2

Apr 7, 2026J/K move · Enter open · S save

VideoStir: Understanding Long Videos via Spatio-Temporally Structured and Intent-Aware RAG

Honghao Fu, Miao Xu, Yiwei Wang, Dailing Zhang+2

University of Queensland · University of California, Merced · Institute of Automation, CAS+1

Multimodal RAG Long-Video Understanding Query-Aware Video Frame Selection Video Retrieval

Open paper AskPDF arXiv

Scaling multimodal large language models (MLLMs) to long videos is constrained by limited context windows. While retrieval-augmented generation (RAG) is a promising remedy by organizing query-relevant visual evidence into a compact context, most existing methods (i) flatten videos into independent segments, breaking their inherent spatio-temporal structure, and (ii) depend on explicit semantic matching, which can miss cues that are implicitly relevant to the query's intent. To overcome these limitations, we propose VideoStir, a structured and intent-aware long-video RAG framework. It firstly structures a video as a spatio-temporal graph at clip level, and then performs multi-hop retrieval to aggregate evidence across distant yet contextually related events. Furthermore, it introduces an MLLM-backed intent-relevance scorer that retrieves frames based on their alignment with the query's reasoning intent. To support this capability, we curate IR-600K, a large-scale dataset tailored for learning frame-query intent alignment. Experiments show that VideoStir is competitive with state-of-the-art baselines without relying on auxiliary information, highlighting the promise of shifting long-video RAG from flattened semantic matching to structured, intent-aware reasoning. Codes and checkpoints are available at https://github.com/RomGai/VideoStir.