Multimodal Reasoning

Follow this topic Ask about this topic

Multimodal reasoning studies how models perceive information across modalities, connect that evidence to knowledge, and use it to answer questions, plan actions, or explain decisions. Common approaches include instruction tuning and chain-of-thought training, reinforcement learning with verifiable rewards, tool-using agents, and structured methods such as graphs or symbolic solvers. Benchmarks such as MathVista and MMMU test visual problem solving, while video, document, scientific, and medical tasks probe reasoning across time and specialized evidence. A central challenge is ensuring that correct answers actually depend on relevant multimodal evidence rather than language shortcuts; grounding, cross-modal integration, and reliable generalization remain difficult.

Latest papers

512 papers47 papers in the last four weeks, down 10% on the four weeks before. 0.3% of all new papers.

SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning
Mar 5, 2026Zhu Li, Yongjian Chen, Huiyuan Lai +3Sarcasm Detection Multimodal Reasoning
Speech Technology Lab, University of Groningen, The Netherlands · Center for Language and Cognition, University of Groningen, The Netherlands
Mario: Multimodal Graph Reasoning with Large Language Models
Mar 5, 2026Yuanfu Sun, Kang Li, Pengkang Guo +2LLM Reasoning with Graphs Multimodal Reasoning
New York University Shanghai · New York University · Tsinghua University +1
From Narrow to Panoramic Vision: Attention-Guided Cold-Start Reshapes Multimodal Reasoning
Mar 4, 2026Ruilin Luo, Chufan Shi, Yizhen Zhang +10Multimodal Reasoning VLM Reasoning
1Tsinghua University · 3Qwen Team, Alibaba Group · University of South California +3
InEdit-Bench: Benchmarking Intermediate Logical Pathways for Intelligent Image Editing Models
Mar 4, 2026Zhiqiang Sheng, Xumeng Han, Zhiwei Zhang +6Image Editing Evaluation Image Editing
State Key Laboratory of Optical Field Manipulation Science and Technology, Institute of Optics and Electronics, Chinese Academy of Sciences, Chengdu, 610209, China · National Laboratory on Adaptive Optics, Chengdu, 610209, China · University of Chinese Academy of Sciences, Beijing, 101408, China
Social Norm Reasoning in Multimodal Language Models: An Evaluation
Mar 3, 2026Oishik Chowdhury, Anushka Debnath, Bastin Tony Roy SavarimuthuSocial Reasoning Multimodal Large Language Models
School of Computing, University of Otago, New Zealand
Modeling Cross-vision Synergy for Unified Large Vision Model
Mar 3, 2026Shengqiong Wu, Lanhu Wu, Mingyang Bao +5Vision Foundation Models Unified Multimodal Models
National University of Singapore · Nanyang Technological University
UniG2U-Bench: Do Unified Models Advance Multimodal Understanding?
Mar 3, 2026Zimo Wen, Boxiu Li, Wanbo Zhang +11Unified Multimodal Models Multimodal Model Evaluation
Microsoft Research Asia · Shanghai Jiao Tong University · Fudan University +2
iGVLM: Dynamic Instruction-Guided Vision Encoding for Question-Aware Multimodal Understanding
Mar 3, 2026Hanpeng Liu, Yaqian Li, Zidan Wang +5Large Vision-Language Models VLM Reasoning
School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan, China · Li Auto Inc. · Institute of AI for Industries, Chinese Academy of Sciences.
Evaluating Cross-Modal Reasoning Ability and Problem Characteristics with Multimodal Item Response Theory
Mar 3, 2026Shunki Uebayashi, Kento Masui, Kyohei Atarashi +5MLLM Evaluation Multimodal Model Evaluation
Kyoto University · CyberAgent. · The Institute of Statistical Mathematics +1
MMR-Life: Piecing Together Real-life Scenes for Multimodal Multi-image Reasoning
Mar 2, 2026Jiachun Li, Shaoping Huang, Zhuoran Jin +5Multi-Image Reasoning Multimodal Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences · School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences
Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection
Mar 2, 2026Yuchen Zhang, Yaxiong Wang, Kecheng Han +4Multimodal Deception Detection Explainable Deepfake Detection
School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Information Engineering, Hefei University of Technology · CSIRO +2
Graph-of-Mark: Promote Spatial Reasoning in Multimodal Language Models with Graph-Based Visual Prompting
Mar 2, 2026Giacomo Frisoni, Lorenzo Molfetta, Mattia Buzzoni +1Spatial Reasoning in Language Models VLM Reasoning
Department of Computer Science and Engineering, University of Bologna
DIVA-GRPO: Enhancing Multimodal Reasoning through Difficulty-Adaptive Variant Advantage
Mar 1, 2026Haowen Gao, Zhenyu Zhang, Liang Pang +7Group Relative Policy Optimization RL for VLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, CAS, Beijing, China · University of Chinese Academy of Sciences, Beijing, China · Kuaishou Technology, Beijing, China
PaLMR: Towards Faithful Visual Reasoning via Multimodal Process Alignment
Feb 28, 2026Yantao Li, Qiang Hui, Chenyang Yan +8Multimodal Reasoning Faithfulness in VLMs
National Key Laboratory for Novel Software Technology, Nanjing University · Data Science & Artificial Intelligence Research Institute, China Unicom · Unicom Data Intelligence, China Unicom
CURE: A Multimodal Benchmark for Clinical Understanding and Retrieval Evaluation
Feb 28, 2026Yannian Gu, Zhongzhen Huang, Linjie Mu +3Clinical Language Model Evaluation Multimodal Model Evaluation
Shanghai Jiao Tong University, Shanghai, China · SenseTime Research, China · Shanghai Innovation Institute, Shanghai, China
Proof-of-Perception: Certified Tool-Using Multimodal Reasoning with Compositional Conformal Guarantees
Feb 27, 2026Arya Fayyazi, Haleh AkramiMultimodal Reasoning Multimodal Tool Use
University of Southern California, Los Angeles, CA · Nuro, Mountain View, CA
How Well Do Multimodal Models Reason on ECG Signals?
Feb 27, 2026Maxwell A. Xu, Harish Haresamudram, Catherine W. Liu +11Electrocardiogram Interpretation Multimodal Model Evaluation
University of Illinois Urbana Champaign · Google Inc · Rush University +4
VisRef: Visual Refocusing while Thinking Improves Test-Time Scaling in Multi-Modal Large Reasoning Models
Feb 27, 2026Soumya Suvra Ghosal, Youngeun Kim, Zhuowei Li +6Test-Time Scaling for VLMs Multimodal Reasoning
University of Maryland, College Park · Amazon · Physion Labs
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
Feb 27, 2026Yiyang Fang, Wenke Huang, Pei Fu +5Multimodal Large Language Models Multimodal Reasoning
School of Computer Science, Wuhan University. · MiLM Plus, Xiaomi Inc.
Reasoning-Driven Multimodal LLM for Domain Generalization
Feb 27, 2026Zhipeng Xu, Zilong Wang, Xinyang Jiang +3Domain Generalization Multimodal Domain Generalization
Xidian University · Microsoft Research Asia · National Engineering Laboratory for Integrated Aero-Space-Ground-Ocean Big Data Application
U-Mind: A Unified Framework for Real-Time Multimodal Interaction with Audiovisual Generation
Feb 27, 2026Xiang Deng, Feng Gao, Yong Zhang +5Multimodal Agents Unified Multimodal Models
Tsinghua University · Meituan
ThinkOmni: Lifting Textual Reasoning to Omni-modal Scenarios via Guidance Decoding
Feb 26, 2026Yiran Guan, Sifan Tu, Dingkang Liang +6Multimodal Reasoning MLLM Reasoning
Huazhong University of Science and Technology · MiLM Plus, Xiaomi Inc.
PATRA: Pattern-Aware Alignment and Balanced Reasoning for Time Series Question Answering
Feb 26, 2026Junkai Lu, Peng Chen, Xingjian Wu +4Time Series Reasoning Time Series QA
East China Normal University, Shanghai, China · Aalborg University, Aalborg, Denmark
MM-NeuroOnco: A Multimodal Benchmark and Instruction Dataset for MRI-Based Brain Tumor Diagnosis
Feb 26, 2026Feng Guo, Jiaxiang Liu, Yang Li +2Medical Image Benchmarks Brain MRI
Guangdong Institute of Intelligence Science and Technology, Hengqin, China · Center for Brain-Inspired Computing Research (CBICR), Department of Precision Instrument, Tsinghua University, Beijing, China
OmniGAIA: Towards Native Omni-Modal AI Agents
Feb 26, 2026Xiaoxi Li, Wenxiang Jiao, Jiarui Jin +10Multimodal Reasoning AI Agent Benchmarks
1Renmin University of China · 2Xiaohongshu Inc. · 3Peking University +2
MEDSYN: Benchmarking Multi-EviDence SYNthesis in Complex Clinical Cases for Multimodal Large Language Models
Feb 25, 2026Boqi Chen, Xudong Liu, Jiachuan Peng +5Medical Diagnosis Medical VLMs
ETH Zurich · MBZUAI · Amazon +4
EmoOmni: Bridging Emotional Understanding and Expression in Omni-Modal LLMs
Feb 25, 2026Wenjie Tian, Zhixian Zhao, Jingbin Hu +4Multimodal Large Language Models Emotion-Aware Dialogue Generation
Northwestern Polytechnical University, Xi’an, China · University of Surrey, Guildford, United Kingdom
Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization
Feb 25, 2026Jinghan Li, Junfeng Fang, Jinda Lu +5Multimodal Large Language Models Multimodal Reasoning
University of Science and Technology of China · National University of Singapore
See It, Say It, Sorted: An Iterative Training-Free Framework for Visually-Grounded Multimodal Reasoning in LVLMs
Feb 25, 2026Yongchang Zhang, Oliver Ma, Tianyi Liu +2Multimodal Reasoning VLM Reasoning
Southeast University · AIIA, Ministry of Education, China · University of Oxford
Synergizing Understanding and Generation with Interleaved Analyzing-Drafting Thinking
Feb 24, 2026Shengqiong Wu, Bobo Li, Xinkai Wang +6VLM Reasoning Unified Multimodal Models
National University of Singapore · University of Oxford · Nanyang Technological University +1

Mar 5, 2026J/K move · Enter open · S save

SarcasmMiner: A Dual-Track Post-Training Framework for Robust Audio-Visual Sarcasm Reasoning

Zhu Li, Yongjian Chen, Huiyuan Lai, Xiyuan Gao+2

Speech Technology Lab, University of Groningen, The Netherlands · Center for Language and Cognition, University of Groningen, The Netherlands

Sarcasm Detection Multimodal Reasoning Multimodal Robustness RL for Language Model Reasoning Multimodal Reward Modeling

Open paper AskPDF arXiv

Multimodal sarcasm detection requires resolving pragmatic incongruity across textual, acoustic, and visual cues through cross-modal reasoning. To enable robust sarcasm reasoning with foundation models, we propose SarcasmMiner, a reinforcement learning based post-training framework that resists hallucination in multimodal reasoning. We reformulate sarcasm detection as structured reasoning and adopt a dual-track distillation strategy: high-quality teacher trajectories initialize the student model, while the full set of trajectories trains a generative reward model (GenRM) to evaluate reasoning quality. The student is optimized with group relative policy optimization (GRPO) using decoupled rewards for accuracy and reasoning quality. On MUStARD++, SarcasmMiner increases F1 from 59.83% (zero-shot), 68.23% (supervised finetuning) to 70.22%. These findings suggest that reasoning-aware reward modeling enhances both performance and multimodal grounding.