Multimodal Reasoning

Follow this topic Ask about this topic

Momentum

39 papers in the last four weeks, up 70% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 314

All topics

GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods
Jun 2, 2026Sujay Belsare, Sudarshan Nikhil, Sushant Kumar +2Multimodal Reasoning Multimodal Model Interpretability
1IIIT Hyderabad, India · University of Virginia, USA
World Models Meet Language Models: On the Complementarity of Concrete and Abstract Reasoning
Jun 2, 2026Yucheng Zhou, Wei Tao, Yiwen Guo +1On-Policy Self-Distillation World Models
University of Macau · LIGHTSPEED · Independent Researcher
CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection
Jun 2, 2026Jinjie Shen, Yaxiong Wang, Yujiao Wu +5Fake News Detection Multimodal Large Language Models
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · Wuhan University, Wuhan, China · Lab for Intelligence and visiON (LION) +2
MUSE: A Unified Agentic Harness for MLLMs
Jun 2, 2026Jianglin Lu, Hailing Wang, Xu Ma +4Multimodal Reasoning Multimodal Agents
Northeastern University
Spectral-Progressive Thought Flow for Lightweight Multimodal Reasoning
Jun 1, 2026Yixian Shen, Zhiheng Yang, Qi Bi +6Efficient Multimodal Inference Visual Spatial Reasoning
Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands · University of Thessaly, Volos, Greece · Department of Computer Science, University College London +3
Do Multimodal Agents Really Benefit from Tool Use? A Systematic Study of Capability Gains
Jun 1, 2026Garvin Guo, Donglei Yu, Yu Chen +6Tool-Use Evaluation LLM Agent Evaluation
University of Chinese Academy of Sciences · Tongyi Lab, Alibaba Group · Amap, Alibaba Group +1
Towards Understanding Modality Interaction in Multimodal Language Models via Partial Information Decomposition
May 31, 2026Wanlong Fang, Tianle Zhang, Wen Tao +1Multimodal Grounding Multimodal Large Language Models
College of Computing and Data Science, Nanyang Technological University, Singapore · Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore · Centre of AI in Medicine (C-AIM), Nanyang Technological University, Singapore
CASTLE2026 Team WDL Technical Report
May 30, 2026Zhengyang Li, Zhenglin Du, Yi Wen +3Multimodal RAG Egocentric Video QA
Key Laboratory of Intelligent Perception and Image Understanding
Through the PRISM: Principle-Aware, Interpretable, and Multi-Scale Evaluation of Visual Designs
May 30, 2026Mona Gandhi, KJ Joseph, Srinivasan Parthasarathy +1VLM Evaluation VLM Interpretability
Ohio State University · Adobe Research
VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning
May 29, 2026Hengbo Xu, Shengjie Jin, Yanbiao Ma +1Efficient Multimodal Inference Visual Attention
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.
FAM-Bench: A Multimodal Benchmark for Condition-Aware Food-as-Medicine Reasoning
May 29, 2026Mingyang Mao, Bhargav Rishi Medisetti, Utkarsh Grover +4Healthcare Clinical Decision Support
Department of Electrical Engineering, University of South Florida · Muma College of Business, University of South Florida · Computer Science, University of Copenhagen
StemBind: When MLLMs Get Lost Between Rules and Instances in Abstract Visual Reasoning
May 29, 2026Xixiang He, Baiqi Wu, Xingming Li +4VLM Evaluation Visual Reasoning
1National University of Defense Technology · 2Zhejiang University · 3Intelligent Game and Decision Lab
MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization
May 28, 2026Anisha Saha, Varsha Suresh, Teodora Kamova +3VLM Robustness Multimodal Reasoning
Max Planck Institute for Informatics, Saarland Informatics Campus · Saarland University · The University of Edinburgh +1
OmniMatBench: A Human-Calibrated Multimodal Reasoning Benchmark Across 19 Materials Science Subfields
May 28, 2026Wanhao Liu, Jiaqing Xie, Qian Tan +10Benchmark Design Scientific QA
University of Science and Technology of China · Shanghai Artificial Intelligence Laboratory · Fudan University +4
DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning
May 28, 2026Junzhe Zhang, Huixuan Zhang, Guirong Wang +5Counterfactual Evaluation Counterfactual Reasoning in Language Models
Wangxuan Institute of Computer Technology, Peking University · Alibaba Group
Agent Explorative Policy Optimization for Multimodal Agentic Reasoning
May 27, 2026Minki Kang, Shizhe Diao, Ryo Hachiuma +4Multimodal Reasoning Agentic Reasoning
1NVIDIA · 2KAIST
VeriTrip: A Verifiable Benchmark for Travel Planning Agents over Unstructured Web Corpora
May 27, 2026Yuting Xu, Jiayi Tian, Jian Liang +4Multimodal Reasoning AI Agent Benchmarks
Institute of Information Engineering, CAS · School of Cyber Security, UCAS · Amap, Alibaba Group +2
DREAM-R: Multimodal Speculative Reasoning with RL-Based Refined Drafting, Precise Verification, and Fully Parallel Execution
May 27, 2026Yunhai Hu, Zining Liu, Xiangyang Yin +5Efficient Multimodal Inference RL for Language Model Reasoning
New York University · University of Pennsylvania · Cerebras Systems
CyberJurors: A Multi-Agent Simulation Task for E-Commerce Disputes Verdict
May 27, 2026Yanhui Sun, Wu Liu, Haifeng Ming +3Multimodal Reasoning LLM Decision-Making
School of Information Science and Technology, University of Science and Technology of China, Hefei, China.
Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
May 27, 2026Ke Xu, Yuhao Wang, Ziyang Cheng +3Audio-Visual Understanding Active Perception
Shanghai Jiao Tong University
Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning
May 27, 2026Yang Zhang, Xiaoshuai Sun, Rui Zhao +5Active Perception Multimodal Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Sino-Russian ResearchCenter for Digital Economy. · School of Informatics, Xiamen University, China. +2
Escape the Language Prior: Mitigating Late-Stage Modality Collapse in Audio Reasoning via Modality-Aware Policy Optimization
May 26, 2026Cihan Xiao, Yiwen Shao, Chenxing Li +5RL for Language Model Reasoning Audio Reasoning
Johns Hopkins University · Tencent Hunyuan
Touch-R1: Reinforcing Touch Reasoning in MLLMs
May 26, 2026Yingxin Lai, Yafei Zhou, Fucai Zhu +2Tactile Sensing Multimodal Large Language Models
1Xiamen University · 5Daimon Robotics · 2Great Bay University +2
LiveK12Bench: Have Large Multimodal Models Truly Conquered High School-level Examinations?
May 26, 2026Xiaohan Wang, Mingze Yin, Yilin Zhao +2Mathematical Reasoning Benchmarks Educational Assessment
Tencent PCG · College of Computer Science and Technology, Zhejiang University
InterSketch: An Interleaved Reasoning Model with Self-correcting Visual Sketch and Stepwise Reward
May 26, 2026Zhiwei Ning, Wenwen Tong, Xiangli Kong +12Multimodal Reasoning VLM Reasoning
Shanghai Jiao Tong University · SenseTime Research · Shandong Normal University
Advancing Creative Physical Intelligence in Large Multimodal Models
May 25, 2026Cheng Qian, Hyeonjeong Ha, Jiayu Liu +10Multimodal Grounding Object Hallucination in VLMs
1UIUC · 2Amazon
MULTISEISMO: A Multimodal Seismic Dataset and Model for Cross-Modal Seismic Understanding
May 25, 2026Sai Munikoti, Ian Stewart, Chengping Chai +4Multimodal Reasoning Seismology
Pacific Northwest National Laboratory (PNNL), Richland, WA 99354 USA · Oak Ridge National Laboratory (ORNL), Oak Ridge, TN 37830 USA · Sandia National Laboratory (SNL), Albuquerque, NM 87123 USA +1
AnE: Pushing the Reasoning Frontier of Multimodal LLMs via Anchor Evolution
May 25, 2026Zehao Wang, Yihan Zeng, Zidong Gong +5RL for Language Model Reasoning Multimodal Large Language Models
Harbin Institute of Technology · Huawei Noah’s Ark Lab · Independent Researcher
Does Seeing More Mean Knowing More? Mono-Anchored Advantage Normalization for Multi-Source Visual Reasoning
May 25, 2026Fanhu Zeng, Zhicong Luo, Zefan Wang +3Visual Reasoning Multimodal Reasoning
Tsinghua University · Northwest Polytechnical University · Beijing Jiaotong University
AOEPT: Breaking the Implicit Modality-Reduction Bottleneck in Modality-Missing Prompt Tuning
May 24, 2026Jian Lang, Rongpei Hong, Ting Zhong +1Missing-Modality Learning Multimodal Reasoning
University of Electronic Science and Technology of China, Chengdu, Sichuan · Intelligent Digital Media Technology Key Laboratory of Sichuan Province, Chengdu, Sichuan

Jun 2, 2026, cs.CVJ/K move · Enter open · S save

GridVQA-X: A Framework for Evaluating Multimodal Explainability Methods

Sujay Belsare, Sudarshan Nikhil, Sushant Kumar, Ponnurangam Kumaraguru+1

1IIIT Hyderabad, India · University of Virginia, USA

Multimodal Reasoning Multimodal Model Interpretability Explainability Evaluation Multimodal Model Evaluation Multimodal Learning Multimodal Understanding

Open paper AskPDF arXiv

With the increasing development of Vision-Language Models, it becomes imperative that their predictions are readily explainable to relevant stakeholders. However, the field of explainability has not kept pace with the multimodal surge. While recent Multimodal Explainable AI (MxAI) methods generate explanations to attribute the interaction between different modalities, current evaluation protocols lack the ground truth required to distinguish between true cross-modal reasoning (e.g., spatial composition) and shallow cross-modal shortcuts (e.g., Bag-of-Words attribute matching). It remains unknown whether MxAI methods faithfully capture synergistic interactions or merely hallucinate reasoning on models acting as simple feature detectors. In this paper, we introduce GridVQA-X, the first diagnostic framework specifically designed to evaluate cross-modal explainability. Unlike natural datasets, GridVQA-X leverages a closed-world synthesis logic to generate unique, mathematically guaranteed explanations. We utilize this controlled environment to train paired ground-truth models on identical architectures:

M_{\text{pure}}

, which learns robust spatial-relational reasoning and

M_{\text{spur}}

, which is structurally forced to rely on cross-modal shortcuts. This behavioral divergence creates a rigorous testbed: a faithful explainer must report distinct reasoning pathways for each model. Our findings reveal that widely used methods fail to distinguish between models relying on genuine spatial-relational reasoning and those exploiting cross-modal shortcuts, highlighting a critical gap in capturing true cross-modal synergy and misrepresenting how multimodal models actually make decisions.

Momentum

Related topics

Latest papers 314