Multimodal Reasoning

Follow this topic Ask about this topic

Multimodal reasoning studies how models perceive information across modalities, connect that evidence to knowledge, and use it to answer questions, plan actions, or explain decisions. Common approaches include instruction tuning and chain-of-thought training, reinforcement learning with verifiable rewards, tool-using agents, and structured methods such as graphs or symbolic solvers. Benchmarks such as MathVista and MMMU test visual problem solving, while video, document, scientific, and medical tasks probe reasoning across time and specialized evidence. A central challenge is ensuring that correct answers actually depend on relevant multimodal evidence rather than language shortcuts; grounding, cross-modal integration, and reliable generalization remain difficult.

Latest papers

512 papers47 papers in the last four weeks, down 10% on the four weeks before. 0.3% of all new papers.

When Models Judge Themselves: Unsupervised Self-Evolution for Multimodal Reasoning
Mar 22, 2026Zhengxian Wu, Kai Shi, Chuanrui Zhang +10Multimodal Reasoning Self-Training for Language Models
OPPO AI Center · Tsinghua University · Nanyang Technological University +1
CoVR-R:Reason-Aware Composed Video Retrieval
Mar 20, 2026Omkar Thawakar, Dmitry Demidov, Vaishnav Potlapalli +5Composed Video Retrieval VLM Reasoning
Mohamed bin Zayed University of AI · University of Chicago · University of Wisconsin-Madison +1
Beyond detection: cooperative multi-agent reasoning for rapid onboard EO crisis response
Mar 20, 2026Alejandro D. Mousist, Pedro Delgado de Robles Martín, Raquel Lladró Climent +1Multi-Agent Reasoning Multi-Agent Orchestration
Thales Alenia Space España, C/ Einstein 7 (PTM), 28760 Tres Cantos, Madrid, Spain
Cognitive Mismatch in Multimodal Large Language Models for Discrete Symbol Understanding
Mar 19, 2026Yinghui Li, Jiayi Kuang, Peng Xing +11Multimodal Large Language Models Multimodal Model Evaluation
Tsinghua University, China · Sun Yat-sen University, China · Independent Researcher +3
Interpretable Traffic Responsibility from Dashcam Video via Legal Multi Agent Reasoning
Mar 18, 2026Jingchun Yang, Jinchang ZhangMultimodal Reasoning Multi-Agent Reasoning
Northeast University · SUNY Binghamton University
Dental-TriageBench: Benchmarking Multimodal Reasoning for Hierarchical Dental Triage
Mar 18, 2026Ziyi He, Yushi Feng, Shuangyu Yang +7Clinical Triage Multimodal Reasoning
School of Computing and Data Science, The University of Hong Kong · Faculty of Dentistry, The University of Hong Kong · The Prince Philip Dental Hospital +2
A Progressive Visual-Logic-Aligned Framework for Ride-Hailing Adjudication
Mar 18, 2026Weiming Wu, Zi-Jian Cheng, Jie Meng +5Legal Judgment Prediction Multimodal Reasoning
Nanjing University · Didichuxing Co. Ltd
Surg $Σ$ : A Spectrum of Large-Scale Multimodal Data and Foundation Models for Surgical Intelligence
Mar 17, 2026Zhitao Zeng, Mengya Xu, Jian Jiang +13Multimodal Foundation Models Surgical Video Understanding
NUS · CUHK · SJTU +1
Follow the Clues, Frame the Truth: Hybrid-evidential Deductive Reasoning in Open-Vocabulary Multimodal Emotion Recognition
Mar 17, 2026Yu Liu, Lei Zhang, Haoxun Li +4Multimodal Emotion Recognition Multimodal Reasoning
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences, Hangzhou, China.
VisBrowse-Bench: Benchmarking Visual-Native Search for Multimodal Browsing Agents
Mar 17, 2026Zhengbo Zhang, Jinbo Su, Zhaowen Zhou +14Multimodal Search Agents AI Agent Benchmarks
CASIA · Ant Digital Technologies, Ant Group · RUC +4
MSRAMIE: Multimodal Structured Reasoning Agent for Multi-instruction Image Editing
Mar 17, 2026Zhaoyuan Qiu, Ken Chen, Xiangwei Wang +3Image Editing Multimodal Agents
University Of Melbourne, Grattan Street, Parkville, Victoria 3010, Australia
OMNIFLOW: A Physics-Grounded Multimodal Agent for Generalized Scientific Reasoning
Mar 16, 2026Hao Wu, Yongheng Zhang, Yuan Gao +7Scientific Reasoning in Language Models Physics-Informed ML
Tsinghua University · Tencent · Shenzhen Loop Area Institute +2
Evaluating Time Awareness and Cross-modal Active Perception of Large Models via 4D Escape Room Task
Mar 16, 2026Yurui Dong, Ziyue Wang, Shuyun Lu +5Multimodal Reasoning Multimodal Model Evaluation
Fudan University, Shanghai, China · Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China · School of Computer & Communication Engineering, University of Science and Technology Beijing, Beijing, China +3
CAMD: Coverage-Aware Multimodal Decoding for Efficient Reasoning of Multimodal Large Language Models
Mar 16, 2026Huijie Guo, Jingyao Wang, Lingyu Si +3Efficient Multimodal Inference Multimodal Large Language Models
Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences · Wangxuan Institute of Computer Technology, Peking University
AgroOmni: A Large-Scale Multi-view Agricultural Dataset for Cross-Scale Multimodal Reasoning
Mar 15, 2026Jiarui Zhang, Junqi Hu, Zurong Mai +10Multimodal Reasoning Multimodal Large Language Models
Sun Yat-sen University · The Chinese University of Hong Kong · HuanTian Wisdom Technology Co., Ltd. +4
MMOU: A Massive Multi-Task Omni Understanding and Reasoning Benchmark for Long and Complex Real-World Videos
Mar 14, 2026Arushi Goel, Sreyan Ghosh, Vatsal Agarwal +16Multimodal Understanding Multimodal Reasoning
NVIDIA, USA · University of Maryland, College Park, USA
QTrack: Query-Driven Reasoning for Multi-modal MOT
Mar 14, 2026Tajamul Ashraf, Tavaheed Tariq, Sonia Yadav +4Referring MOT Multimodal Visual Object Tracking
King Abdullah University of Science and Technology (KAUST), Saudi Arabia · Gaash Research Lab, National Institute of Technology Srinagar, India · The University of Queensland, Australia
MIBench: Evaluating LMMs on Multimodal Interaction
Mar 13, 2026Yu Miao, Zequn Yang, Yake Wei +5Multimodal Large Language Models Multimodal Model Evaluation
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China · Beijing Key Laboratory of Research on Large Models and Intelligent Governance, Beijing, China · Beihang University, Beijing, China +1
GRADE: Benchmarking Discipline-Informed Reasoning in Image Editing
Mar 12, 2026Mingxin Liu, Ziqian Fan, Zhaokai Wang +13Image Editing Evaluation Text-Guided Image Editing
Shanghai Jiao Tong University · South China University of Technology · Fudan University +2
SciMDR: Advancing Scientific Multimodal Document Reasoning
Mar 12, 2026Ziyu Chen, Yilun Zhao, Chengye Wang +3Multimodal Reasoning Multimodal QA
University of Chicago · Yale University · TCS Research
LatentGeo: Learnable Auxiliary Constructions in Latent Space for Multimodal Geometric Reasoning
Mar 12, 2026Haiying Xu, Zihan Wang, Song Dai +3Multimodal Large Language Models Multimodal Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Nankai University · Communication University of China
Chem4DLLM: 4D Multimodal LLMs for Chemical Dynamics Understanding
Mar 12, 2026Xinyu Li, Zhen Zhang, Qi Chen +3Multimodal Large Language Models Multimodal Reasoning
Australian Institute for Machine Learning, Adelaide University, Adelaide, Australia · CSIRO’s Data61, Sydney, Australia
GeoSense: Internalizing Geometric Necessity Perception for Multimodal Reasoning
Mar 11, 2026Ruiheng Liu, Haihong Hao, Mingfei Han +4Visual Spatial Reasoning Multimodal Reasoning
University of Science and Technology of China, Anhui, China · Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, UAE · University of Chinese Academy of Sciences, Beijing, China +1
InternVL-U: Democratizing Unified Multimodal Models for Understanding, Reasoning, Generation and Editing
Mar 10, 2026Changyao Tian, Danni Yang, Guanzhou Chen +26Unified Multimodal Models Multimodal Generation
Shanghai AI Laboratory · CUHK MMLab · Fudan University +6
Logics-Parsing-Omni Technical Report
Mar 10, 2026Xin An, Jingyi Cai, Xiangyang Chen +22Multimodal Understanding Semantic Parsing
Alibaba Group
Reading, Not Thinking: Understanding and Bridging the Modality Gap When Text Becomes Pixels in Multimodal LLMs
Mar 10, 2026Kaiser Sun, Xiaochuang Yuan, Hongjun Liu +4Multimodal Large Language Models Modality Gap in VLMs
Johns Hopkins University · Amazon · New York University +1
M $^3$ -ACE: Rectifying Visual Perception in Multimodal Math Reasoning via Multi-Agentic Context Engineering
Mar 9, 2026Peijin Xie, Zhen Xu, Bingquan Liu +1Multimodal Reasoning Visually Grounded Reasoning
Platform and Content Group Tencent
A Survey of Multimodal Mathematical Reasoning: From Perception, Alignment to Reasoning
Mar 9, 2026Tianyu Yang, Sihong Wu, Yilun Zhao +6Multimodal Reasoning Multimodal Model Evaluation
University of Notre Dame · Yale University · Columbia University +2
MultiHaystack: Benchmarking Multimodal Retrieval and Reasoning over 40K Images, Videos, and Documents
Mar 5, 2026Dannong Xu, Zhongyu Yang, Jun Chen +6Multimodal IR Multimodal Model Evaluation
INSAIT, Sofia University “St. Kliment Ohridski” · Lanzhou University · King Abdullah University of Science and Technology +3
Distributed Partial Information Puzzles: Examining Common Ground Construction Under Epistemic Asymmetry
Mar 5, 2026Yifan Zhu, Mariah Bradford, Kenneth Lai +4Multi-Agent Collaboration Multimodal Reasoning
Brandeis University, 415 South St, Waltham, MA 02453 · Colorado State University, Fort Collins, CO 80523

Mar 22, 2026J/K move · Enter open · S save

When Models Judge Themselves: Unsupervised Self-Evolution for Multimodal Reasoning

Zhengxian Wu, Kai Shi, Chuanrui Zhang, Zirui Liao+9

OPPO AI Center · Tsinghua University · Nanyang Technological University+1

Multimodal Reasoning Self-Training for Language Models RL for Language Model Reasoning

Open paper AskPDF arXiv

Recent progress in multimodal large language models has led to strong performance on reasoning tasks, but these improvements largely rely on high-quality annotated data or teacher-model distillation, both of which are costly and difficult to scale. To address this, we propose an unsupervised self-evolution training framework for multimodal reasoning that achieves stable performance improvements without using human-annotated answers or external reward models. For each input, we sample multiple reasoning trajectories and jointly model their within group structure. We use the Actor's self-consistency signal as a training prior, and introduce a bounded Judge based modulation to continuously reweight trajectories of different quality. We further model the modulated scores as a group level distribution and convert absolute scores into relative advantages within each group, enabling more robust policy updates. Trained with Group Relative Policy Optimization (GRPO) on unlabeled data, our method consistently improves reasoning performance and generalization on five mathematical reasoning benchmarks, offering a scalable path toward self-evolving multimodal models. The code are available at https://github.com/OPPO-Mente-Lab/LLM-Self-Judge.