Multimodal CoT Reasoning

CoT: Chain-of-Thought

Follow this topic Ask about this topic

Momentum

6 papers in the last four weeks, up 100% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 99

All topics

Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding
May 14, 2026Arsha Nagrani, Jasper Uijilings, Shyamal Buch +6Egocentric Video QA Egocentric Video Understanding
Google DeepMind
Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
May 14, 2026Longxiang Zhang, Weilong Dai, Guanghao Zhang +2Efficient Multimodal Inference Multimodal Embedding
Alibaba Group
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
May 12, 2026Houcheng Jiang, Jiajun Fu, Junfeng Fang +4Efficient Multimodal Inference Multimodal Large Language Models
University of Science and Technology of China · Zhongguancun Academy · National University of Singapore +1
OmniThoughtVis: A Scalable Distillation Pipeline for Deployable Multimodal Reasoning Models
May 12, 2026Yuanhao Yue, Chengyu Wang, Yuanjie Lyu +2VLM Distillation Multimodal Reasoning
Alibaba Group, Hangzhou, China
Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
May 11, 2026Xuanchen Li, Yuheng Lu, Chenrui Cui +6Cross-Modal Learning Multimodal QA
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin, China · Tencent, China · Huiyan Technology Company, Ltd., Tianjin, China +1
Do multimodal models imagine electric sheep?
May 10, 2026Santhosh Kumar Ramakrishnan, Carl Vondrick, Raja Giryes +2Visual Spatial Reasoning World Model Learning
Apple
Reflection Anchors for Propagation-Aware Visual Retention in Long-Chain Multimodal Reasoning
May 10, 2026Xuan Gong, Hanbo Huang, Hao Zheng +4Vision-Language Models Vision-Language Grounding
Shanghai Jiao Tong University · Lanzhou University
Enhancing Visual Question Answering with Multimodal LLMs via Chain-of-Question Guided Retrieval-Augmented Generation
May 5, 2026Quanxing Xu, Ling Zhou, Xian Zhong +3Multimodal RAG Visual Question Answering
School of Computer Science and Engineering, Macau University of Science and Technology, Macao SAR 999078, China · Hubei Key Laboratory of Transportation Internet of Things, School of Computer Science and Artificial Intelligence, Wuhan University of Technology, Wuhan, Hubei 430070, China · State Key Laboratory of Maritime Technology and Safety, Wuhan University of Technology, Wuhan 430063, China +3
Chart-FR1: Visual Focus-Driven Fine-Grained Reasoning on Dense Charts
May 3, 2026Hongkun Pan, Yuwei Wu, Wanyi Hong +8Data Visualization Multimodal CoT Reasoning
Zhejiang University · State Key Lab of CAD&CG, Zhejiang University · HiThink Research
CheXthought: A global multimodal dataset of clinical chain-of-thought reasoning and visual attention for chest X-ray interpretation
Apr 29, 2026Sonali Sharma, Jin Long, George Shih +7Chest X-Ray Classification Medical Image Analysis
Center for Artificial Intelligence in Medicine and Imaging, Stanford University, Stanford, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA · Department of Pediatrics, Stanford University School of Medicine, Stanford, California, USA +6
UpstreamQA: A Modular Framework for Explicit Reasoning on Video Question Answering Tasks
Apr 25, 2026Jason Nguyen, Ameet Rao, Alexander Chang +2Multimodal Large Language Models Video QA
Lincoln North Star High School · The Charter School of Wilmington · Greenwich High School +2
ProcessThinker: Enhancing Multi-modal Large Language Models Reasoning via Rollout-based Process Reward
Apr 23, 2026Jingpei Wu, Xiao Han, Weixiang Shen +3Process Reward Models Multimodal QA
LMU Munich · Harvard University · University of Cambridge +2
S1-VL: Scientific Multimodal Reasoning Model with Thinking-with-Images
Apr 23, 2026Qingxiao Li, Lifeng Xu, QingLi Wang +4Tool Use in VLMs Multimodal Reasoning
ScienceOne AI
Thinking Like a Botanist: Challenging Multimodal Language Models with Intent-Driven Chain-of-Inquiry
Apr 22, 2026Syed Nazmus Sakib, Nafiul Haque, Shahrear Bin Amin +4Visual Question Answering Plant Disease Classification
Department of Robotics and Mechatronics Engineering, University of Dhaka, Dhaka, Bangladesh · Department of Computer Science and Engineering, University of Dhaka, Dhaka, Bangladesh · Department of Agronomy, Gazipur Agricultural University, Gazipur, Bangladesh +1
V-tableR1: Process-Supervised Multimodal Table Reasoning with Critic-Guided Policy Optimization
Apr 22, 2026Yubo Jiang, Yitong An, Xin Yang +7Multimodal Reasoning Reinforcement Learning with Verifiable Rewards
School of Astronautics, Beihang University, Beijing 102206, China · Longcat Interaction Team, Meituan, Beijing 100102, China · Tianmushan Laboratory, Beihang University, Hangzhou 311115, China
SurgCoT: Advancing Spatiotemporal Reasoning in Surgical Videos through a Chain-of-Thought Benchmark
Apr 22, 2026Gui Wang, YongSong Zhou, Kaijun Deng +4Spatiotemporal Reasoning Healthcare
School of Computer Science and Software Engineering, Shenzhen University · University of Nottingham · School of AI, Shenzhen University +1
Unveiling Fine-Grained Visual Traces: Evaluating Multimodal Interleaved Reasoning Chains in Multimodal STEM Tasks
Apr 21, 2026Jing Jin, Hao Liu, Yan Bai +9Reasoning Evaluation Multimodal Reasoning
1Tsinghua University · 3Central South University · 2Meituan Inc +3
A-MAR: Agent-based Multimodal Art Retrieval for Fine-Grained Artwork Understanding
Apr 21, 2026Shuai Wang, Hongyi Zhu, Jia-Hong Huang +6Multimodal RAG Multimodal Grounding
University of Amsterdam · Part of this work was done during interning at Amazon. · Also with Amazon AGI. +2
Audio-DeepThinker: Progressive Reasoning-Aware Reinforcement Learning for High-Quality Chain-of-Thought Emergence in Audio Language Models
Apr 20, 2026Xiang He, Chenxing Li, Jinting Wang +5RL for Language Model Reasoning Audio Reasoning
Tencent AI Lab, Beijing, China · The Hong Kong University of Science and Technology (Guangzhou), Guangzhou, China
SpatialImaginer: Towards Adaptive Visual Imagination for Spatial Reasoning
Apr 19, 2026Yian Li, Yang Jiao, Bin Zhu +4Visual Spatial Reasoning Unified Multimodal Models
College of Computer Science and Artificial Intelligence, Fudan University · School of Computing and Information Systems, Singapore Management University · School of Computer Science and Technology, East China Normal University +2
Q-DeepSight: Incentivizing Thinking with Images for Image Quality Assessment and Refinement
Apr 18, 2026Xudong Li, Jiaxi Tan, Ziyin Zhou +6Tool Use in VLMs Multimodal CoT Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China · Beijing Institute of Technology
Chain-of-Thought Degrades Visual Spatial Reasoning Capabilities of Multimodal LLMs
Apr 17, 2026Sai Srinivas Kancheti, Aditya Sanjiv Kanade, Vineeth N. Balasubramanian +1Spatial Reasoning Benchmarks Visual Spatial Reasoning
IIT, Hyderabad · Microsoft Research India
Chain-of-Glimpse: Search-Guided Progressive Object-Grounded Reasoning for Video Understanding
Apr 16, 2026Zhixuan Wu, Quanxing Zha, Teng Wang +6Video Reasoning Multimodal CoT Reasoning
State Key Laboratory of Networking and Switching Technology, Beijing University of Posts and Telecommunications, Beijing, China · Institute of Big Data, College of Computer Science and Artificial Intelligence, Fudan University, China · ARC Lab, Tencent PCG, Shenzhen, China +2
UniRect-CoT: Enhancing Generation in Unified Multimodal Models via Reflective Rectification with Inherent Understanding
Apr 15, 2026Yibo Jiang, Tao Wu, Rui Jiang +4Diffusion Model Guidance Unified Multimodal Models
School of Software Technology, Zhejiang University · College of Computer Science and Technology, Zhejiang University
Internalized Reasoning for Long-Context Visual Document Understanding
Mar 31, 2026Austin VeselkaLong-Context Modeling VLM Reasoning
LightOn
M3CoTBench: Benchmark Chain-of-Thought of MLLMs in Medical Image Understanding
Jan 13, 2026Juntao Jiang, Jiangning Zhang, Yali Bi +7Medical VLMs Multimodal CoT Reasoning
1Zhejiang University · University of Science and Technology of China · 3East China Normal University +2
Mimic Human Cognition, Master Multi-Image Reasoning: A Meta-Action Framework for Enhanced Visual Understanding
Jan 12, 2026Jianghao Yin, Qingbin Li, Kun Sun +10Visual Reasoning Multimodal Reasoning
East China Normal University · ByteDance
Ophiuchus: Incentivizing Tool-augmented "Think with Images" for Joint Medical Segmentation, Understanding and Reasoning
Dec 16, 2025Yankai Jiang, Yujie Zhang, Peng Zhang +5Image Segmentation Tool Use in VLMs
Zhejiang University · Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute +5
See, Think, Learn: A Self-Taught Multimodal Reasoner
Dec 2, 2025Sourabh Sharma, Sonam Gupta, SadbhawnaSelf-Training VLM Reasoning
Malaviya National Institute of Technology Jaipur · IBM Research
VisReason: A Large-Scale Dataset for Visual Chain-of-Thought Reasoning
Nov 21, 2025Lingxiao Li, Yifan Wang, Xinyan Gao +3Visual Spatial Reasoning Visual Reasoning
Boston University, Boston, MA, USA · Stony Brook University, Stony Brook, NY, USA · MMLab, The Chinese University of Hong Kong, Hong Kong SAR, China

May 14, 2026, cs.CVJ/K move · Enter open · S save

Minerva-Ego: Spatiotemporal Hints for Egocentric Video Understanding

Arsha Nagrani, Jasper Uijilings, Shyamal Buch, Tobias Weyand+5

Google DeepMind

Egocentric Video QA Egocentric Video Understanding Multimodal QA Video Reasoning Multimodal CoT Reasoning Visually Grounded Reasoning

Open paper AskPDF arXiv

Video reasoning models are a core component of egocentric and embodied agents. However, standard benchmarks for assessing models provide only evaluation of the output (e.g. the answer to a question), without evaluation of intermediate reasoning steps, and most provide answers only in the text domain. We introduce Minerva-Ego, a benchmark for evaluating complex egocentric visual reasoning. We extend recent high-quality video data sources recorded from egocentric / embodied settings with a set of challenging, multi-step multimodal questions and spatiotemporally-dense human-annotated reasoning traces. Benchmarking experiments show that state-of-the-art models still have a large gap to human performance. To investigate this gap in detail, we annotate each reasoning trace in the dataset with the objects of interest required to solve the question, as spatiotemporal mask annotations. Through extensive evaluations, we identify that prompting frontier models with hints of 'where' and 'when' to look yields substantial improvements in performance. Minerva-Ego can be downloaded at https://github.com/google-deepmind/neptune.

Momentum

Related topics

Latest papers 99