Multimodal Reasoning
Overview
Multimodal reasoning studies how models perceive information across modalities, connect that evidence to knowledge, and use it to answer questions, plan actions, or explain decisions. Common approaches include instruction tuning and chain-of-thought training, reinforcement learning with verifiable rewards, tool-using agents, and structured methods such as graphs or symbolic solvers. Benchmarks such as MathVista and MMMU test visual problem solving, while video, document, scientific, and medical tasks probe reasoning across time and specialized evidence. A central challenge is ensuring that correct answers actually depend on relevant multimodal evidence rather than language shortcuts; grounding, cross-modal integration, and reliable generalization remain difficult.
Latest papers
531 papers29 papers in the last four weeks, down 44% on the four weeks before. 0.2% of all new papers.
MMVistaReason: Toward Open-Data and Post-Training Recipes for Multimodal Reasoning
ReCast: Contract-Preserving Protection for Fixed-Interface Multimodal Reasoning
OmniReasoning: Pushing the Limits of Audio-Visual Joint Reasoning
Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
From Image Interpretation to Clinical Reasoning: Upstream Physician-Context-Aware Multimodal Learning with Causal Reinforcement Learning
OmniVCBench: Benchmarking Evidence-Grounded Multimodal Reasoning Towards AI Virtual Cells
LEGO-OPD: Factorized Teacher Composition for Multimodal On-Policy Distillation
DSPO: Diversity-aware Subjective Policy Optimization for Robust Emotional Reasoning
ThinkingGuard: Decoding Implicit Hazards via Step-by-Step Risk Attribution in Multimodal Large Language Models
Medical Image Alignment Assessment as a Test of Generalist Visual Reasoning in Frontier Multimodal Models
V-Gym: Enhancing Agentic Visual Reasoning via Skill-Data Co-Evolution
IVT-Guard: All-in-One Reasoning Model for AI-Generated Content Detection
VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings
OPERA: A Unified Omnimodal Progressive Spatio-Temporal Reasoning Agent for Referring Video Segmentation
PhysAlign: A Benchmark for Evidence-Grounded Role Alignment in Multimodal Physics Reasoning
EngIntervene: Benchmarking Multimodal Engineering State Understanding and Design Intervention Reasoning
CT-OPD: Counterfactual Trace On-Policy Distillation for Diffusion Vision-Language Models
MM-OPD: Towards One More Bottleneck Between Perception and Reasoning
OmniSmartHome: A Multimodal Reasoning Benchmark for Smart-Home Agents
RLHarness: Co-evolving Procedural Skills with Reinforcement Learning for Long-horizon Multimodal Reasoning
RAO-Nav: Probing Omni-Language Models for Zero-shot Semantic Audio-Visual Navigation
hear'', see'', reason'', and act'' in the environment. To further elicit the built-in thinking ability of OLMs, we propose a test-time Latent Navigation Reasoning (LNR) module that can be seamlessly integrated into the decoding space. LNR encourages the model to retrieve more target-relevant observations and make effective navigation decisions. Through comprehensive experiments, we show that our framework surpasses existing state-of-the-art baselines on public SAVN benchmarks without using any training data. Moreover, we introduce a new Global Navigation Instruction setting to further evaluate the ability of OLMs to serve as embodied navigation agents. Code: https://github.com/rikeilong/OmniAV\_Nav.