Video Question Answering

Also known as VQA

Video question answering asks models to identify events, people, actions, and relationships in video, including when things happen and why. Approaches include vision-language models, question-aware frame selection and retrieval, temporal memory, scene graphs, and agentic systems that inspect evidence with tools. Benchmarks test settings such as long-form and streaming video, egocentric activity, causal reasoning, and precise temporal grounding. A central challenge is that high answer accuracy can coexist with weak use of visual evidence, poor temporal grounding, and overconfidence when evidence is incomplete.

Latest papers

185 papers25 papers in the last four weeks, up 47% on the four weeks before. 0.1% of all new papers.

An Attribute-Based Measure of Video Complexity
May 30, 2026Aditya Sarkar, Yi Li, Zihao Wang +8VLM Evaluation Video Question Answering
UMIACS-University of Maryland College Park · University of California San Diego · Meta AI +1
Semantic and Visual Evidence for Efficient Long-Video Reasoning: A Solution for the HD-EPIC VQA Challenge
May 28, 2026Yinsong Xu, Wei Jing, Liuxin Zhang +2Egocentric VQA Video Question Answering
Lenovo, China
Reflective Dialogue between Teacher and Solver Agents for Video Question Answering
May 27, 2026Takuya Murakawa, Toru TamakiVLM Adaptation Video Question Answering
Nagoya Institute of Technology, Nagoya, Japan
EgoAdapt: A Multi-Scene Egocentric Adaptation Method for CVPR 2026 HD-EPIC VQA Challenge
May 23, 2026Zhiwei Chen, Yupeng Hu, Zixu Li +4Test-Time Adaptation VLM Adaptation
Shandong University · Harbin Institute of Technology (Shenzhen)
CaST-Bench: Benchmarking Causal Chain-Grounded Spatio-Temporal Reasoning for Video Question Answering
May 22, 2026Mingfang Zhang, Jingjing Pan, Ashutosh Kumar +7VLM Evaluation Temporal Video Grounding
Woven by Toyota · The University of Tokyo
Cambrian-P: Pose-Grounded Video Understanding
May 21, 2026Jihan Yang, Zifan Zhao, Xichen Pan +5Vision-Language Models Camera Pose Estimation
New York University · UC Berkeley · Meta FAIR
MuKV: Multi-Grained KV Cache Compression for Long Streaming Video Question-Answering
May 21, 2026Junbin Xiao, Jiajun Chen, Tianxiang Sun +2KV Caching Video Question Answering
University of Science and Technology of China · National University of Singapore
FineBench: Benchmarking and Enhancing Vision-Language Models for Fine-grained Human Activity Understanding
May 19, 2026Gueter Josmy Faure, Min-Hung Chen, Jia-Fong Yeh +2VLM Evaluation Temporal Video Grounding
National Taiwan University · Google · Independent Researcher
CRAFT: Critic-Refined Adaptive Key-Frame Targeting for Multimodal Video Question Answering
May 18, 2026Mahesh Bhosale, Abdul Wasi, Vishvesh Trivedi +3Claim Verification Video Question Answering
University at Buffalo · 2New York University
EgoExoMem: Cross-View Memory Reasoning over Synchronized Egocentric and Exocentric Videos
May 18, 2026Ruiping Liu, Junwei Zheng, Yufan Chen +7Multimodal Memory Video Question Answering
Karlsruhe Institute of Technology · ETH Zurich · University of Oxford +2
ViMU: Benchmarking Video Metaphorical Understanding
May 14, 2026Qi Li, Xinchao WangVideo Understanding Video Question Answering
National University of Singapore
ReTool-Video: Recursive Tool-Using Video Agents with Meta-Augmented Tool Grounding
May 13, 2026Xiao Liu, Nayu Liu, Junnan Zhu +6Video Question Answering Tool-Using Vision-Language Agents
Chongqing University · Tianjin University · MAIS, Institute of Automation, Chinese Academy of Sciences +2
VideoSEAL: Mitigating Evidence Misalignment in Agentic Long Video Understanding by Decoupling Answer Authority
May 12, 2026Chenhao Qiu, Yechao Zhang, Xin Luo +2Large Vision-Language Models Video Question Answering
Mango TV · Mango TV, Changsha, China · Nanyang Technological University +1
ChronoSC: Task-Oriented Semantic Communication via Temporal-to-Color Encoding
May 11, 2026Phuc H. Nguyen, Trung T. Nguyen, Quy N. Duong +1Deep Joint Source-Channel Coding Semantic Communication
Smart Green Transformation Center (GREEN-X), VinUniversity, Vietnam. · School of Computer Science and Statistics, Trinity College Dublin, Ireland
Semantic-Aware Adaptive Visual Memory for Streaming Video Understanding
May 8, 2026Hang Wu, Sherin Mary Mathews, Yujun Cai +2Streaming Video Understanding Efficient VLM Inference
University of California, Merced · US Bank · University of Queensland
VTAgent: Agentic Keyframe Anchoring for Evidence-Aware Video TextVQA
May 6, 2026Haibin He, Maoyuan Ye, Jing Zhang +2Video Question Answering Video Frame Selection
School of Computer Science, National Engineering Research Center for Multimedia Software, Institute of Artificial Intelligence, and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, Wuhan, Hubei, China
KARMA-MV: A Benchmark for Causal Question Answering on Music Videos
May 5, 2026Archishman Ghosh, Abhinaba Roy, Dorien HerremansAudio-Visual Understanding Video Question Answering
AMAAI Lab, Singapore University of Technology and Design
VideoNet: A Large-Scale Dataset for Domain-Specific Action Recognition
May 4, 2026Tanush Yadav, Mohammadreza Salehi, Jae Sung Park +6VLM Evaluation VLM Adaptation
University of Washington · Stanford University · Allen Institute for AI
Video Active Perception: Effective Inference-Time Long-Form Video Understanding with Vision-Language Models
May 3, 2026Martin Q. Ma, Willis Guo, Aditya Agrawal +4Active Perception Efficient VLM Inference
Carnegie Mellon University · MIT
SF20K Competition 2025: Summary and findings
May 2, 2026Ridouane Ghermi, Xi Wang, Vicky Kalogeiton +1VLM Evaluation Video Understanding
CurEvo: Curriculum-Guided Self-Evolution for Video Understanding
Apr 29, 2026Guiyi Zeng, Junqing Yu, Yi-Ping Phoebe Chen +3Video Question Answering Curriculum Learning
Huazhong University of Science and Technology Wuhan, China · La Trobe University Melbourne, Australia · Beijing Institute of Computer Technology and Applications Beijing, China
Interactive Episodic Memory with User Feedback
Apr 27, 2026Nikesh Subedi, Loris Bazzani, Ziad Al-HalahEgocentric VQA Episodic Memory
University of Utah · University of Verona
Don't Pause! Every prediction matters in a streaming video
Apr 27, 2026Dibyadip Chatterjee, Zhanzhong Pang, Fadime Sener +2Streaming Video Understanding Efficient VLM Inference
National University of Singapore · Google Inc.
UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
Apr 25, 2026Jason Nguyen, Ameet Rao, Alexander Chang +2Multimodal Large Language Models Video Question Answering
Lincoln North Star High School · The Charter School of Wilmington · Greenwich High School +2
Grounding Video Reasoning in Physical Signals
Apr 23, 2026Alibay Osmanli, Zixu Cheng, Shaogang GongTemporal Video Grounding Video Question Answering
Queen Mary University of London
CCTVBench: Contrastive Consistency Traffic VideoQA Benchmark for Multimodal LLMs
Apr 22, 2026Xingcheng Zhou, Hao Guo, Rui Song +5Visual Question Answering Video Question Answering
Technical University of Munich · University of California, Los Angeles
Progressive Online Video Understanding with Evidence-Aligned Timing and Transparent Decisions
Apr 20, 2026Kecheng Zhang, Zongxin Yang, Mingfei Han +6Streaming Video Understanding Efficient VLM Inference
University of Science and Technology of China · Harvard University · Department of CV, MBZUAI +3
When Text Hijacks Vision: Benchmarking and Mitigating Text Overlay-Induced Hallucination in Vision Language Models
Apr 19, 2026Cui Yakun, Xingqun Qi, TianTian Geng +3VLM Evaluation VLM Hallucination
The Hong Kong University of Science and Technology · Hong Kong, China · University of Birmingham
EgoEsportsQA: An Egocentric Video Benchmark for Perception and Reasoning in Esports
Apr 14, 2026Jianzhe Ma, Zhonghao Cao, Shangkui Chen +3Egocentric VQA Video Question Answering
Renmin University of China · Beijing University of Posts and Telecommunications
ReXSonoVQA: A Video QA Benchmark for Procedure-Centric Ultrasound Understanding
Apr 13, 2026Xucheng Wang, Xiaoman Zhang, Sung Eun Kim +2Medical Visual Question Answering Video Question Answering
Department of Biomedical Informatics, Harvard Medical School, Boston, MA

May 30, 2026J/K move · Enter open · S save

An Attribute-Based Measure of Video Complexity

Aditya Sarkar, Yi Li, Zihao Wang, Hao Yuan+7

UMIACS-University of Maryland College Park · University of California San Diego · Meta AI+1

VLM Evaluation Video Question Answering

Open paper AskPDF arXiv

A new framework for the estimation of the complexity posed by video-question pairs to video-LLMs, Video Attribute-Based Complexity (VideoABC), is proposed. Video complexity is defined as the probability of failure of a video-LLM for a given video-question pair. VideoABC is a non-parametric complexity measure, using a reference video dataset and a pre-defined vocabulary of video attributes informative of complexity, e.g. the scene complexity or the speed of the video event relevant to the question. In a training phase, reference videos are projected into the space of these attributes, which is then quantized. The expected ABC of each quantization cell is then computed. Given a new video and its projection into the attribute space, complexity is estimated by the expected ABC of the associated quantization cell. To enable the use of VideoABC with small reference video datasets, two quantizers are combined: a k-means quantizer that enables accurate complexity estimates for samples in the distribution of the reference dataset and a universal lattice quantizer that guarantees generalization to out-of-distribution samples. A synthetic video generation procedure, inspired by target-distractor manipulations of psychophysics studies, is proposed to populate the cells of the lattice quantizer during training, enabling the computation of their expected ABCs. Experimental results show that VideoABC is effective even with very low-dimensional attribute representations, substantially outperforming approaches like 'video-LLM as judge' with much less complexity. Finally, the explainable nature of the VideoABC score, in terms of well-defined attributes, is shown to provide insights on how the attribute composition of benchmarks affects their complexity.