Video Question Answering

Also known as VQA

Video question answering asks models to identify events, people, actions, and relationships in video, including when things happen and why. Approaches include vision-language models, question-aware frame selection and retrieval, temporal memory, scene graphs, and agentic systems that inspect evidence with tools. Benchmarks test settings such as long-form and streaming video, egocentric activity, causal reasoning, and precise temporal grounding. A central challenge is that high answer accuracy can coexist with weak use of visual evidence, poor temporal grounding, and overconfidence when evidence is incomplete.

Latest papers

185 papers25 papers in the last four weeks, up 47% on the four weeks before. 0.1% of all new papers.

AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents
Sep 18, 2026Seoyeon An, Hyeonseo Jang, Minsu Kim +3Video Question Answering AI Agent Benchmarks
KRAFTON
CapMem: A Benchmark for Caption-Based Episodic Memory in Egocentric Video
Sep 15, 2026Dingli Liang, Yiqiao Xie, Yukai Huang +6Memory-Augmented VLMs Egocentric VQA
University College London · Imperial College London · Durham University +4
Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models
Sep 15, 2026Rwiddhi Chakraborty, Yinong Oliver Wang, Cheng Zhang +6Video-Language Model Evaluation Video Question Answering
Oliver · University of Copenhagen · Carnegie Mellon University +4
Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning
Sep 14, 2026Mantek Singh, Jeshwanth Challagundla, Siddharth Raina +1CoT Distillation VLM Distillation
Liverpool John Moores University, Liverpool, England · Carnegie Mellon University, Pittsburgh, USA · Meta, Sunnyvale, USA +1
Long-to-Short Video Evidence Reasoning for Grounded Question Answering
Sep 14, 2026Kaiyan Chen, Junbin Xiao, Xun YangTemporal Video Grounding Video Question Answering
University of Science and Technology of China
DementiaCare-Bench: A Modality-Validated Video Benchmark
Sep 11, 2026Afrouz Sheikholeslami, Yuankai Qi, Xuyun Zhang +4Alzheimer's Disease Healthcare
School of Computing, Macquarie University, Sydney, NSW, Australia · School of Electrical and Computer Engineering, University of Sydney, Sydney, NSW, Australia · University of California, Merced, CA, USA
BodyCam-VQA: Enhanced Body-Worn Camera Video Captioning via Multimodal Reasoning and Probe Question Generation
Sep 9, 2026Karish Gupta, Matthew Alex, Alex Li +6Video Question Answering Multimodal Reasoning
Worcester Polytechnic Institute, Worcester, MA, USA · Axon, Scottsdale, AZ, USA
Target-Checked Reliability Score Refinement for Video Question Answering
Sep 9, 2026Guoxiang Ren, Rohitash ChandraConfidence Estimation for VLMs Video Question Answering
Transitional Artificial Intelligence Research Group, School of Mathematics and Statistics, UNSW Sydney, NSW 2052, Australia
EgoSIS: From Factorized Visual Ego-Transitions to Motion-Canonical Spatial Evidence for UAV Reasoning
Sep 8, 2026Jingpu Yang, Fengxian Ji, Mingxuan Cui +4Visual Spatial Reasoning Egocentric VQA
Beihang University, Beijing, China · Zhongguancun Academy, Beijing, China · Northeastern University, Shenyang, China +1
Companion-style QA Assistance in Ego-Vision
Sep 6, 2026Hangyu Qin, Junbin Xiao, Shenglang Zhang +1Egocentric VQA Video Question Answering
National University of Singapore · University of Science and Technology of China
CAM: Question Answering on Entity-Centric Videos with Continuous Extraction and Adaptive Querying
Sep 6, 2026Yizhou Tian, Zizhe Chen, Shiyuan Deng +7Video Question Answering Long-Video QA
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Knowin AI · Institute for Math and AI, Wuhan University
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
Sep 2, 2026Jitai Hao, Ke Yang, Di Yan +3Streaming Video Understanding Efficient Multimodal Inference
Harbin Institute of Technology (Shenzhen)
Cite or Decline: A Strict Course-Grounded Chatbot for STEM Lecture Videos
Sep 1, 2026S M Masrur Ahmed, Jaspal SubhlokEvidence Retrieval Video Question Answering
University of Houston
StreamScout: Learning When to Look Deeper for Streaming Video Understanding
Aug 31, 2026Ce Zhang, Jing Bi, Jinxi He +9Adaptive Inference Streaming Video Understanding
Carnegie Mellon University · TikTok · University of Rochester +2
SocialReasonBench: A Video-QA Benchmark for Social Reasoning with Counterfactual Narrative Videos
Aug 31, 2026Zheyu Huang, Zijing Shi, Haozhe Luo +4Video Question Answering Multimodal Reasoning
AAII, University of Technology Sydney · Northeastern University · Fudan University +1
Dynamic Hub-and-Spoke Memory for Streaming Video Understanding
Aug 31, 2026Xinru Jiang, Lin Zhao, Xi Xiao +7Memory-Augmented VLMs Streaming Video Understanding
Northeastern University · University of Alabama at Birmingham · Tulane University +2
Post-Training VLMs for Video Mistake Detection
Aug 28, 2026Federico Spurio, Olga Zatsarynna, Lars Doorenbos +3VLM Post-Training Vision-Language Models
University of Bonn Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence Bonn, Germany · Toyota Motor Europe Belgium
AdaVDR: Adaptive Tool Use and Reflection for Video Deep Research
Aug 26, 2026Xintong Zhang, Xiaomeng Fan, Shilin Yan +7Autonomous Research Agents Tool Use in VLMs
Accio Team, Alibaba Group · Beijing Key Laboratory of Intelligent Information Technology, School of Computer Science & Technology, Beijing Institute of Technology
Where to Look Matters: On-Policy Self-Distillation for Long-Video Understanding
Aug 26, 2026Kaishen Wang, Dongdi Zhao, Yijun Liang +4On-Policy Self-Distillation VLM Distillation
University of Maryland, College Park · Johns Hopkins University
Multi-Agent Self-Improving Reinforcement Learning for Video Reasoning
Aug 25, 2026Mingwen Zhang, Jisheng Dang, Minqiang Yang +3Evidence-Grounded VQA Video Reasoning
School of information Science and Engineering, Lanzhou University, LanZhou, China · Gansu Provincial Key Laboratory of Wearable Computing, School of Information Science and Engineering, Lanzhou University, Lanzhou, China · Brain Health Engineering Laboratory, Institute of Engineering Medicine, Beijing Institute of Technology, Beijing, China +1
Towards Comprehensive Basketball Understanding
Aug 24, 2026Yirong Hu, Jiayuan Rao, Yu Zhang +2Sports Video Analysis Multimodal QA
School of Mathematical Sciences, Peking University, China · School of Artificial Intelligence, Shanghai Jiao Tong University, China
IntentQA: Intent Question Answering in Videos by Cognitive Context Reasoning
Aug 24, 2026Jiapeng Li, Ping Wei, Wenjuan Han +2Video Reasoning Video Question Answering
National Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Xi’an Jiaotong University, Xi’an, China · School of Computer and Information Technology, Beijing Jiaotong University, Beijing, China · National Key Laboratory of General Artificial Intelligence, Beijing Institute for General Artificial Intelligence (BIGAI), Beijing, China
Thinking Beyond Videos: Unifying Video Reasoning and Deep Research for Open-World Video Agents
Aug 24, 2026Wenqi Liu, Shijie Ma, Yunxiao Wang +21Tool-Using Agents Video Question Answering
Shandong University · Institute of Automation, Chinese Academy of Sciences · Beihang University +4
TRACE: Temporal Retrieval with Anchored and Convergent Evidence for Long-Horizon Video Understanding
Aug 23, 2026Pengyiang Liu, Junbo Niu, Xiaoyang Hu +4Evidence-Grounded VQA Video Question Answering
Colab, Beihang University · Shanghai AI Laboratory · Peking University
OVIBench: Benchmarking Online Video Question Answering under Interruption
Aug 23, 2026Naiming Liu, Zhiheng Wu, Shuning Wang +3Video Question Answering VLM Evaluation
HIT · CASIA · ZJU +2
Enhancing Localized Reasoning for Long Video Understanding via Efficient Segment-to-Video Supervision
Aug 21, 2026Beibei Zhang, Chao Xu, Jun Lan +4Long-Video Understanding VLM Reasoning
State Key Laboratory for Novel Software Technology, Nanjing University · Ant Group · School of Computer Science, Shanghai Jiao Tong University
RISE: Roadside Infrastructure Sequence Understanding across 3D Tracking and Structured Vision-Language Reasoning
Aug 17, 2026Yanbo Jiang, Haotian Zheng, Jiahao Wang +10Temporal Reasoning in VLMs 3D MOT
School of Vehicle and Mobility, Tsinghua University · TsingCloud
What You Ask is What You Ground: Bridging Question Intent to Temporal Evidence for Grounded VideoQA
Aug 16, 2026Jinhwan Seo, Kyubeom Han, Jumin Lee +2Evidence-Grounded VQA Video Question Answering
KAIST · Ewha Womans University
Counterfactual Sensitivity Is Not Repairability: Auditing Replay Probes for Video Evidence
Aug 16, 2026Rama AlHamidi, Rasul Khanbayov, Erchin Serpedin +1AI Agent Auditing Counterfactual Evaluation
Department of Computer and Electrical Engineering, Texas A&M University at Qatar, Doha, Qatar · College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar · Department of Computer and Electrical Engineering, Texas A&M University, College Station, TX, USA
Catching Hallucinated Citations in Video-LLM Question Answering: A Self-Verification Pipeline and Verifier Ablation Study
Aug 16, 2026Yogesh KumarEvidence-Grounded VQA Hallucination Detection in VLMs

Sep 18, 2026J/K move · Enter open · S save

AgentVidBench: A Multi-Hop Video Question Answering Benchmark for Evaluating MLLM Agents

Seoyeon An, Hyeonseo Jang, Minsu Kim, Chanho Lee+2

KRAFTON

Video Question Answering AI Agent Benchmarks Temporal Video Understanding Causal Reasoning in Videos

Open paper AskPDF arXiv

Comprehensive video understanding is crucial for advancing artificial intelligence toward the intricate dynamics of the physical world. While recent advances in Multimodal Large Language Models (MLLMs) have demonstrated remarkable capabilities in video understanding, existing benchmarks remain confined to simple scene-level queries or global summaries that require only single-step inference. Real-world video understanding involves more challenging tasks that require multi-hop multimodal reasoning, and there is a critical absence of video benchmarks equipped to rigorously evaluate these agentic capabilities. To bridge this gap, we introduce AgentVidBench, a multi-hop video question answering benchmark focused on evaluating the spatial, temporal, and causal reasoning capabilities of MLLM agents. Beyond standard question-answer pairs, AgentVidBench provides step-by-step solution traces to support trajectory evaluation that assesses whether agents explicitly acquire the evidence needed to justify their answers. Experiments with 12 proprietary and open-source MLLMs show that single-turn performance remains limited on AgentVidBench, while integrating these models into state-of-the-art agentic workflows generally improves performance with respect to both accuracy and trajectory scores. We further present a simple yet effective agentic strategy that serves as a competitive baseline on AgentVidBench, establishing our benchmark as a holistic testbed for future research on agentic video understanding. Code and datasets are available at https://github.com/krafton-ai/agentvidbench and https://huggingface.co/datasets/agentvidbench/agentvidbench.