Temporal Reasoning in VLMs

VLM: Vision-Language Model

Follow this topic Ask about this topic

Latest papers

42 papers13 papers in the last four weeks, against 2 the four weeks before. 0.1% of all new papers.

V-STORM: A Structure-Aware Grounded Chain of Thought Dataset for Faithful and Fine-Grained Spatio-Temporal Video Reasoning
Nov 6, 2026Dian Jiao, Yuqian Yuan, Jiaqi Zhu +9Video Reasoning Evidence-Grounded VQA
Zhejiang University · National University of Singapore
AtariBench: Evaluating Spatiotemporal Reasoning of Vision-Language Models in Atari Gameplay
Nov 5, 2026Mingjia Huo, Yao Fu, Bo Chang +8Temporal Reasoning in VLMs
University of California San Diego · Google DeepMind · Peking University +1
SPOD: Recovering Suppressed Temporal Evidence in Video-LLMs via Single-Pass Orthogonal Decomposition
Nov 3, 2026Yangguang Lin, Quan Fang, Xiaoyan Liu +1Video-Language Models Temporal Reasoning in VLMs
Beijing University of Posts and Telecommunications, Beijing, China
SurgEye: Instruction Tuning-Free Perception-Reasoning Framework for Surgical Scene Understanding
Nov 2, 2026Qing Xu, Xiechen Shen, Xianjian He +9Surgical Video Understanding Medical VLMs
School of Computer Science, University of Nottingham Ningbo China · University of Nottingham · Department of Data Science and Artificial Intelligence, The Hong Kong Polytechnic University
Trace4D: Grounded Spatiotemporal Reasoning via 4D State Sequences
Nov 2, 2026Wooseok Jang, Seungryong Kim, Yan Lu +1Spatiotemporal Reasoning Temporal Reasoning in VLMs
KAIST AI · Microsoft Research Asia
TwinPast: Benchmarking Historical Evidence Use in Streaming Video Understanding with Evidence Contrastive Video Pairs
Oct 30, 2026Haotian Yang, Yong Chen, Cheng Peng +4Evidence-Grounded VQA Streaming Video Understanding
School of Software Technology, Zhejiang University · HeyGen · Zhejiang University
GenDSR: Transferring Representations from Video Generation Models for Dynamic Spatial Reasoning
Oct 30, 2026Ke Yang, Zhenyu Zhang, Jun LiSpatial Reasoning Temporal Reasoning in VLMs
Nanjing University
Leaving, Staying, or Living There? Duration Commonsense Reasoning with Human-LLM Co-annotations
Oct 28, 2026Chenyang Yu, Zhaomin Xiao, Yan Huang +1Commonsense Reasoning Temporal Reasoning in VLMs
University of North Texas · University of Arizona
The Prestige: Benchmarking Cognitive Visual Reasoning using Magic Tricks
Oct 28, 2026Shuo Wen, Beixi Du, Edwin Meriaux +5Visual Reasoning Video Reasoning
McGill University · Mila – Quebec AI Institute · Cornell University +1
RoboReason: Boosting Robotics-Relevant VLM Reasoning Through Scalable Data Generation
Oct 24, 2026Sagnik Anupam, Utkarsh Lal, Aneesh Edara +5VLMs for Robotics Visual Reasoning
University of Pennsylvania
4DMemory: Benchmarking Long-Horizon Spatiotemporal Memory with Harness-Augmented Agents
Oct 23, 2026Yanhao Jia, Yi Zhu, Yiran Zhong +7Long-Horizon Agent Evaluation Temporal Reasoning in VLMs
Nanyang Technological University · Token Foundry, Alibaba · The Chinese University of Hong Kong +2
RVG-Bench: A Benchmark for Reasoning-Driven Audio-Visual Temporal Grounding
Oct 23, 2026Yue Feng, Boshen Xu, Zihao Yue +5Temporal Video Grounding Multimodal Grounding
AIM3 Lab, Renmin University of China · Kling AI
CellTrack-Bench: Microscopic Cell Tracking Benchmark for Vision-Language Models
Oct 22, 2026Ilan Theodoro, Jordão Bragantini, Loïc A. RoyerVLM Evaluation Microscopy Image Analysis
Biohub San Francisco, CA
Ask About the Right Moment: Training-Free Object Identity Reasoning for Video-Language Models
Oct 22, 2026Di Yang, Xiaochang Li, Muyang Liu +2Video Reasoning Video-Language Models
College of William & Mary · SparcAI Inc.
Failure Modes of Vision-Language Models for Human Activity Understanding in Thermal HRI
Oct 21, 2026Han Hu, He ChenHuman Activity Recognition VLMs for Robotics
Department of Information Engineering, The Chinese University of Hong Kong, Hong Kong SAR, China.
Holi-World: Building Holistic World Intelligence from Human-Ego Experience
Oct 19, 2026Hao Li, Xiyuan Zheng, Yuanyuan Gao +7Egocentric Video Understanding 4D Scene Understanding
Ropedia · NTU · HKUST +1
TIM: Temporal Decoupling with Iterative Mutual-Refinement Model for Longitudinal Radiology Report Generation
Oct 19, 2026Yiheng Dong, Yi Lin, Shilong Huang +2Radiology Report Generation Radiology VLMs
Huazhong University of Science and Technology, Wuhan, China · HKUST, Hong Kong, China
Think Visually Across Space and Time: Grounded Reasoning for Video Segmentation
Oct 18, 2026Junwei Wen, Deshui Miao, Guangming Lu +3Temporal Reasoning in VLMs Video Object Segmentation
Harbin Institute of Technology, Shenzhen · University of California at Merced · Pengcheng Laboratory +1
ActionLens: Diagnosing Spatial-Temporal Binding Failures in Vision-Language Models
Sep 28, 2026Gueter Josmy Faure, Min-Hung Chen, Hao Ping Wang +3Video-Language Model Evaluation Temporal Video Understanding
National Taiwan University · NVIDIA
EgoTSR++: Egocentric Spatiotemporal Reasoning for Task Progress Understanding
Sep 23, 2026Xiaoda Yang, Can Wang, Yuxiang Liu +3Robot Task Progress Estimation Temporal Reasoning in VLMs
Zhejiang University, Hangzhou, China · Qingdao University, Qingdao, China · Tianjin University, Tianjin, China +1
From Vision to Language: Investigating Causal Information Flow in Multimodal Decision-Making
Sep 4, 2026Davide Testa, Hugh Mee Wong, Alessandro Lenci +2Temporal Reasoning in VLMs Vision-Language Models
Fondazione Bruno Kessler (FBK) - Trento, Italy · Università di Roma La Sapienza - Rome, Italy · Utrecht University - Utrecht, The Netherlands +1
What's the Catch? Evaluating Temporal Consistency in Vision-Language Models
Aug 24, 2026Marek Hradil, Danae Sánchez VillegasTemporal Reasoning in VLMs VLM Evaluation
University of Copenhagen · The National Center for AI in Society (CAISA)
RISE: Roadside Infrastructure Sequence Understanding across 3D Tracking and Structured Vision-Language Reasoning
Aug 17, 2026Yanbo Jiang, Haotian Zheng, Jiahao Wang +10Temporal Reasoning in VLMs 3D MOT
School of Vehicle and Mobility, Tsinghua University · TsingCloud
DynaPix: Can Vision-Language Models Identify the Exact Future?
Aug 6, 2026Thong Nguyen, Vinh-Hien Do, Quynh Vo +2VLM Evaluation Vision-Language Models
Centre for AI Research, VinUniversity · National University of Singapore
PhysMind: From Video to Executable Worlds for Training-Free Physical Reasoning
Aug 5, 2026Chen Yang, Shenxiang Zeng, Haoyang Zhao +6Counterfactual Reasoning Physical Reasoning in Video
1Tsinghua University · 2The University of Hong Kong
STAR-VLM: Spatiotemporal Grounding Vision-Language Models for Motion and Velocity Estimation via Automotive Radar Supervision
Aug 2, 2026Pou-Chun Kung, Aryaman Rao, Utkrisht Sahai +4VLMs for Autonomous Driving VLM Reasoning
University of Michigan, Ann Arbor, MI 48109
Animation2Code: Evaluating Temporal Visual Reasoning in Video-to-Code Generation
Jun 26, 2026Anya Ji, Abhijith Varma Mudunuri, David M. Chan +1VLM Evaluation Temporal Reasoning in VLMs
University of California, Berkeley
Chronological Blindness: Benchmarking Temporal Reasoning in Vision-Language Models with CHRONOSIGHT
Jun 15, 2026Parthaw Goswami, Jaynto Goswami DeepVLM Evaluation Temporal Reasoning
Department of Computer Science, University of Missouri, Columbia, 65201, MO, USA · SAP, Prague, Czech Republic
4DP-QA: Scalable QA for 4D Perception in Vision Language Models
Jun 10, 2026Seokju Cho, Abhishek Badki, Hang Su +5VLM Evaluation VLM Reasoning
1NVIDIA · 2Yale University · *Work done during internship at NVIDIA. +1
Seeing Time: Benchmarking Chronological Reasoning and Shortcut Biases in Vision-Language Models
Jun 4, 2026Haoyu Zhou, Qing Qing, Caichong Li +6VLM Evaluation Vision-Language Models
College of Computer Science and Technology, Jilin University, Changchun 130012, China · College of Computing and Data Science, Nanyang Technological University, 639798, Singapore · School of Computer Science, Wuhan University, China +1

Nov 6, 2026J/K move · Enter open · S save

V-STORM: A Structure-Aware Grounded Chain of Thought Dataset for Faithful and Fine-Grained Spatio-Temporal Video Reasoning

Dian Jiao, Yuqian Yuan, Jiaqi Zhu, Mingjian Gao+8

Zhejiang University · National University of Singapore

Video Reasoning Evidence-Grounded VQA Temporal Reasoning in VLMs CoT Reasoning

Open paper AskPDF arXiv

While MLLMs can often identify what happens in a video, they still struggle with fine-grained spatio-temporal reasoning, which requires precise grounding when and where the supporting evidence appears and composing such evidence into a coherent reasoning chain. This challenge is exacerbated by the lack of scalable data construction and evaluation pipelines that provide granular, verifiable supervision linking reasoning steps to explicit spatio-temporal evidence.To address this gap, we present V-STORM, a structure-aware framework that transforms videos into Spatio-Temporal Scene Graphs (STSGs) and derives Grounded Chain of Thought (GCoT) from validated evidence paths, making each reasoning step traceable to structured and verifiable spatio-temporal evidence.Under this framework, we construct a 260K-sample training set and a 3K-sample human-verified benchmark, both featuring GCoT annotations that link reasoning to explicit spatio-temporal evidence.Experiments reveal a clear separation between evidence localization and evidence composition: current MLLMs can often identify relevant spatio-temporal cues, yet still fail to integrate them into correct multi-step reasoning, exposing a bottleneck that answer-only evaluation largely obscures. Moreover, fine-tuning with V-STORM substantially improves both answer accuracy and grounding faithfulness, validating its effectiveness as supervision for faithful and evidence-grounded video reasoning. Our dataset, benchmark, and source code will be publicly released.