Multimodal Reasoning

Follow this topic Ask about this topic

Multimodal reasoning studies how models perceive information across modalities, connect that evidence to knowledge, and use it to answer questions, plan actions, or explain decisions. Common approaches include instruction tuning and chain-of-thought training, reinforcement learning with verifiable rewards, tool-using agents, and structured methods such as graphs or symbolic solvers. Benchmarks such as MathVista and MMMU test visual problem solving, while video, document, scientific, and medical tasks probe reasoning across time and specialized evidence. A central challenge is ensuring that correct answers actually depend on relevant multimodal evidence rather than language shortcuts; grounding, cross-modal integration, and reliable generalization remain difficult.

Latest papers

512 papers47 papers in the last four weeks, down 10% on the four weeks before. 0.3% of all new papers.

Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning
Oct 29, 2026Guizhen Chen, Weiwen Xu, Wotao Yin +2Multimodal Reasoning Visually Grounded Reasoning
Nanyang Technological University · DAMO Academy, Alibaba Group US · Department of Artificial Intelligence, University of Macau +1
Grounding Multimodal Reasoning with Evidence-Ablated Negatives
Oct 29, 2026Chenxi Liu, Tianyi Xiong, Yanshuo Chen +1Multimodal Reasoning RL for VLMs
Department of Computer Science, University of Maryland, College Park, College Park, MD 20742
Leaving, Staying, or Living There? Duration Commonsense Reasoning with Human-LLM Co-annotations
Oct 28, 2026Chenyang Yu, Zhaomin Xiao, Yan Huang +1Commonsense Reasoning Temporal Reasoning in VLMs
University of North Texas · University of Arizona
Alpha-TCM: A Nine-Layer Framework for TCM-Guided Multimodal Inference and Structured-Prior Auditing
Oct 28, 2026Yucheng Liu, Everett X. WangMultimodal Reasoning Multimodal Fusion
Department of Engineering Science, University of Oxford, Oxford, United Kingdom · Beijing Institute of Nanoenergy and Nanosystems, Chinese Academy of Sciences, Beijing, China
Beyond the Readout: A Latent Numerical Frame Shared Across Modality and Operation
Oct 27, 2026Oriol Juan Sabater, Alex Ferrando de las Morenas, Pau Rodríguez +1Multimodal Model Interpretability Multimodal Reasoning
Universitat Autònoma de Barcelona · Apple
Hierarchical Multimodal Interaction Reasoning for Dyadic Relationship Recognition
Oct 26, 2026Wang Tang, Fethiye Irmak Dogan, Linbo Qing +2Relational Reasoning Multimodal Reasoning
School of Software Engineering, Chengdu University of Information Technology, Chengdu 610103, China · AFAR Lab, Department of Computer Science and Technology, University of Cambridge, CB3 0FT Cambridge, U.K. · College of Electronics and Information Engineering, Sichuan University, Chengdu 610065, China +1
Beyond Post-Hoc Rewards: Privileged Reasoning Exploration for Multimodal Embeddings
Oct 23, 2026Tianyu Yang, Yijun Tian, Lars Benedikt Kaesberg +4Multimodal Embedding Cross-Modal Retrieval
University of Göttingen · University of Notre Dame · State and University Library of Göttingen
PEAR: Adaptive Advantage Reshaping for Perception and Exploration in Large Vision-Language Models
Oct 23, 2026Junxin Wang, Yingfeng Luo, Xiangchen Song +12Reasoning for VLMs Multimodal Reasoning
School of Computer Science and Engineering, Northeastern University, Shenyang, China · University of Michigan, Ann Arbor, US · Kunming University of Science and Technology, Kunming, China +1
MOSAIC: An Omni-Modal Benchmark for Long-Term Memory and Cross-Modal Reasoning in Conversational Agents
Oct 21, 2026Hong Ting Tsang, Boyi Zhang, Haoyu Huang +7Multimodal Memory Long-Term Conversational Memory
The Hong Kong University of Science and Technology · University of Edinburgh
TriGeoBench: A Multimodal Mathematical Geometry Benchmark with Figures and TikZ
Oct 20, 2026Jing Wang, Yitong Yang, Hao Dou +6Mathematical Reasoning Benchmarks Multimodal Reasoning
School of Computing and Artificial Intelligence, Shanghai University of Finance and Economics, Shanghai, China · School of Information Management and Engineering, Shanghai University of Finance and Economics, Shanghai, China
AudiPsy: Evidence-Grounded Multimodal Reasoning over Spoken Counseling Conversations for Depression Assessment
Oct 20, 2026Chuanbo Zhu, Run Luo, Weibo Zhao +6Depression Detection Clinical Speech Processing
The Hong Kong University of Science and Technology · National University of Singapore · University of the Chinese Academy of Sciences +2
Look Before You Reason: Implicit Visual Thinking for Efficient Multimodal Reasoning
Oct 19, 2026Zerui Chen, Changrui Chen, Fei Ni +1Multimodal Reasoning Efficient Multimodal Inference
MVP Lab, Imperial College London
EDMA: Entropy-Driven Multimodal Answering
Oct 19, 2026Emanuele Mezzi, Gertjan Burghouts, Fabio Massacci +1Multimodal QA Multimodal Reasoning
Vrije Universiteit Amsterdam · TNO · University of Trento
OpenVLHarness: Real World Visual Agentic Tool-Use with Multimodal Memory
Oct 19, 2026Yu Zhou, Cheng-Fu Yang, Rui Sun +8Tool Use in VLMs Multimodal Reasoning
University of California, Los Angeles · Apple
CODA: Cohort- and Drift-aware Foundation Model for Multimodal Clinical Reasoning
Oct 19, 2026Changshuo Liu, Jiaqi Zhu, Wenqiao Zhang +2EHR Foundation Models Multimodal Reasoning
School of Computing National University of Singapore · School of Software Technology Zhejiang University
Visual Scaffolding: Toward Native Multimodal Reasoning
Oct 18, 2026Run Shao, Zhaoyang Zhang, Ziyu Li +6Multimodal Reasoning VLM Reasoning
Central South University
MAMath-Bench: Process-Aware and Robust Evaluation of Multimodal Multi-Agent Mathematical Reasoning
Oct 18, 2026Xinping Wang, Jie Ma, Rui Xing +6Mathematical Reasoning Benchmarks Language Model Reasoning Evaluation
School of Cyber Science and Engineering, Xi’an Jiaotong University · MOE KLINNS Lab, Xi’an Jiaotong University · School of Computer Science and Technology, Xi’an Jiaotong University +3
Alliance of Stochastic Routing and Reinforcement Fusion for Multi-Model Reasoning
Oct 16, 2026Selim Furkan Tekin, Ling LiuMultimodal Model Routing Adaptive Model Routing
Georgia Institute of Technology
Enhancing Multimodal RLVR with Pairwise Token Contrast and Adaptive Trajectory Confidence
Oct 15, 2026Yiran Zhang, Hanbo Huang, Jielan Li +3RL for VLMs Multimodal Reasoning
Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory
ReasonUni: Reasoning-Augmented Reconstruction Advances Unified Multimodal Models
Oct 14, 2026Feize Wu, Zhiyuan Yan, Yuhuan Hong +10Unified Multimodal Models Multimodal Generation
Peking University · Meituan · Tsinghua University +1
Designing Textual Interfaces To Enhance Llm Reasoning On Audio-Visual Question Answering
Oct 14, 2026Yi Wang, Tianzi Wang, Tong Lei +2Multimodal Reasoning Automatic Prompt Optimization
School of Informatics, University of Edinburgh, UK · Department of Systems Engineering and Engineering Management, Chinese University of Hong Kong, Hong Kong SAR · Key Laboratory of Modern Acoustics, Institute of Acoustics, Nanjing University, China
Integrating Simulation and Chain-of-thought Reasoning in Multimodal-Language Models For Physical Reasoning
Oct 14, 2026YingQiao Wang, Eric Bigelow, Tomer Ullman +2Multimodal Reasoning CoT Reasoning
Department of Psychology, Harvard University · Sakana AI · Goodfire AI
ReDiS: Reasoning-Diagnosis Self-Evolution for Multimodal Reasoning in Remote Sensing
Oct 13, 2026Yi Liu, Wentao Jiang, Shuo Ni +5Remote Sensing VLMs Multimodal Reasoning
School of Computer Science, Wuhan University · Zhongguancun Academy · National Key Laboratory of Science and Technology on Space-Born Intelligent Information Processing, Beijing Institute of Technology
MedEvo: Medical Reasoning Evolution through Knowledge Navigation
Oct 13, 2026Xiaoliang Wang, Zhuo Chen, Chuanyang Zheng +11Medical Reasoning Multimodal Reasoning
Zhejiang University · Ant Group · Shanghai Artificial Intelligence Laboratory +1
ChemExplain: A Multimodal Benchmark for Explanatory Chemical Reasoning
Oct 12, 2026Aotian Chen, Frazier N. Baker, Reza Averly +5Scientific Reasoning Explainability Evaluation
The Ohio State University · Adelaide University
Neos: Learning to Interleave Search and Visual Reasoning
Oct 12, 2026Ekko He, Jack Hong, Shilin Yan +8Multimodal Search Agents Multimodal Reasoning
Accio Team, Alibaba Group · Fudan University · Shanghai Innovation Institute
Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning
Oct 11, 2026Quanlin Chen, Xiuwei Chen, Kun Xiang +5Multimodal Large Language Models Multimodal Reasoning
Sun Yat-sen University · Shenzhen Loop Area Institute · Beijing Zhongguancun Academy +2
From Stages to States: Unified Iterative Reasoning for MLLMs
Oct 11, 2026Chuang Yu, Xuefei Ma, Yuanhao Feng +9Multimodal Reasoning Multimodal CoT Reasoning
MMLab, CUHK · PolyU · Tencent +2
Apertus 1.5: From Open Text Models to Open Multimodal Reasoners
Oct 11, 2026Project Apertus, :, Yixuan Xu +130MLLM Reasoning Multimodal Large Language Models
ETH Zurich · EPFL · HES-SO Valais-Wallis +5
MemeLogos: Knowledge-Driven Retrieval-Augmented Reasoning for Harmful Meme Detection
Oct 11, 2026Hexiang Gu, Saihui Hou, Changlan Liu +4Retrieval-Augmented Reasoning Harmful Meme Detection
Beijing University of Posts and Telecommunications, Beijing 100876, China · the School of Artificial Intelligence, Beijing Normal University, Beijing 100875, China · the College of Electrical Engineering, Zhejiang University, Zhejiang 310027, China +2

Oct 29, 2026J/K move · Enter open · S save

Diagnosing the Reference Gap: What Visual Scaffolds Can and Cannot Fix in Multimodal Reasoning

Guizhen Chen, Weiwen Xu, Wotao Yin, Anh Tuan Luu+1

Nanyang Technological University · DAMO Academy, Alibaba Group US · Department of Artificial Intelligence, University of Macau+1

Multimodal Reasoning Visually Grounded Reasoning MLLM Evaluation Visual Grounding

Open paper AskPDF arXiv

Multimodal large language models can recognize visual content yet struggle to reason reliably over it across multiple steps. In counting, a model may see all objects but lose track of which ones it has counted. We argue that such failures, commonly attributed to perception, reveal a reference gap: the failure to bind language to the relevant visual entities, regions, or locations. Motivated by the theory of visual routines, we test whether external references such as numbering can narrow this gap. We introduce ReGap-Bench, a paired benchmark of five visual reasoning tasks in which the scene, question, and answer are fixed and only the reference varies. Results show that task-matched references improve accuracy by up to 25%, while mismatched references provide little benefit or even hurt performance. These findings identify the reference gap as a bottleneck distinct from visual perception and show that constructing task-appropriate visual references can serve as a training-free visual skill for improving multimodal reasoning.