Multimodal Understanding

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 413

All topics

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards
Nov 10, 2025Hunar Batra, Haoqin Tu, Hardy Chen +3Visual Question Answering Scene Graph Generation
University of Oxford · University of California, Santa Cruz
M4FC: a Multimodal, Multilingual, Multicultural, Multitask Real-World Fact-Checking Dataset
Oct 27, 2025Jiahui Geng, Jonathan Tonglet, Iryna GurevychAutomated Fact-Checking Multimodal Understanding
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI), UAE · Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, TU Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany · Department of Electrical Engineering, KU Leuven, Belgium +2
From Charts to Code: A Hierarchical Benchmark for Multimodal Models
Oct 20, 2025Jiahao Tang, Henry Hengyuan Zhao, Lijian Wu +8Data Visualization Code Generation
CSU-JPG, Central South University · National University of Singapore · Nanyang Technological University
Evidence Recomposition and Predictive Context Residualization for Visual Attribution in Multimodal Large Language Models
Sep 26, 2025Jiawei Liang, Jianjie Huang, Ruoyu Chen +4Feature Attribution Multimodal Large Language Models
Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · Zhongguan-cun Academy, Beijing 100094, China · University of Chinese Academy of Sciences, Beijing 100049, China +2
Lavida-O: Elastic Large Masked Diffusion Models for Unified Multimodal Understanding and Generation
Sep 23, 2025Shufan Li, Jiuxiang Gu, Kangning Liu +4Masked Diffusion Models Unified Multimodal Models
Adobe · UCLA
Do Modern Video-LLMs Need to Listen? A Benchmark Audit and Scalable Remedy
Sep 22, 2025Geewook Kim, Minjoon SeoVLM Evaluation Audio-Visual Understanding
NAVER Cloud AI, South Korea · KAIST AI, South Korea
Reconstruction Alignment Improves Unified Multimodal Models
Sep 8, 2025Ji Xie, Trevor Darrell, Luke Zettlemoyer +1Unified Multimodal Models Cross-Modal Alignment
1UC Berkeley · University of Washington · 3Duke University
VMMU: A Vietnamese Multitask Multimodal Understanding and Reasoning Benchmark
Aug 19, 2025Vy Tuong Dang, An Vo, Emilio Villa-Cueva +4VLM Evaluation Multimodal Reasoning
KAIST · MBZUAI
Inverse-LLaVA: Rethinking Multimodal Alignment via Text-to-Vision Mapping
Aug 17, 2025Xuhui Zhan, Tyler DerrCross-Modal Learning Cross-Modal Representation Learning
Data Science Institute, Vanderbilt University, 2201 West End Ave, Nashville, 37235, Tennessee, United States. · Computer Science Department, Vanderbilt University, 2201 West End Ave, Nashville, 37235, Tennessee, United States.
Position: Reasoning After Perception Means Reasoning Without Vision
Jul 21, 2025Hongcheng Gao, Zihao Huang, Jingyi Tang +12Visual Reasoning VLM Reasoning
Tsinghua University · BIT · Peking University +6
Lingshu: A Generalist Foundation Model for Unified Multimodal Medical Understanding and Reasoning
Jun 8, 2025Weiwen Xu, Hou Pong Chan, Long Li +15Medical Report Generation Multimodal Pretraining
LASA Team · DAMO Academy, Alibaba Group
LaViDa: A Large Diffusion Language Model for Multimodal Understanding
May 22, 2025Shufan Li, Konstantinos Kallidromitis, Hritik Bansal +7Vision-Language Models Efficient VLM Inference
1UCLA · 2Panasonic AI Research · 4Salesforce Research +1
HumaniBench: A Human-Centric Framework for Large Multimodal Models Evaluation
May 16, 2025Shaina Raza, Aravind Narayanan, Vahid Reza Khazaie +6Algorithmic Fairness Multimodal Large Language Models
Vector Institute for Artificial Intelligence, Toronto, Canada · University of Central Florida, Orlando, USA
ZeroBench: An Impossible Visual Benchmark for Contemporary Large Multimodal Models
Feb 13, 2025Jonathan Roberts, Mohammad Reza Taesiri, Ansh Sharma +31Visual Reasoning VLM Reasoning
University of Cambridge
MultiViewDx: Evidence-Linked Multi-View Clinical Diagnosis
Oct 19, 2024Junda Wang, Zonghai Yao, Yujan Ting +5Healthcare Medical Image Analysis
United Imaging Intelligence, MA, USA · Manning College of Information and Computer Sciences, University of Massachusetts Amherst, MA, USA · Department of Medicine, University of Massachusetts Medical School, Worcester, MA, USA +1
VLMEvalKit: An Open-Source Toolkit for Evaluating Large Multi-Modality Models
Jul 16, 2024Haodong Duan, Xinyu Fang, Junming Yang +41VLM Evaluation LLM Evaluation
MAD: Modality-Adaptive Decoding for Mitigating Cross-Modal Hallucinations in Multimodal Large Language Models
Date pendingSangyun Chung, Se Yeon Kim, Youngchae Chee +1Multimodal Robustness Hallucination in Language Models
Integrated Vision Language Lab, KAIST, South Korea
V-Retrver: Evidence-Driven Agentic Reasoning for Universal Multimodal Retrieval
Date pendingDongyang Chen, Chaoyang Wang, Dezhao Su +6Multimodal IR Multimodal Grounding
Tsinghua University · University of Central Florida · Fudan University +4
Learning to Think Like a Cartoon Captionist: Incongruity-Resolution Supervision for Multimodal Humor Understanding
Date pendingHatice Merve Vural, Doga Kukul, Ege Erdem Ozlu +4VLM Reasoning Multimodal CoT Reasoning
Koç University, Istanbul, Turkey · KUIS AI Center, Istanbul, Turkey · Air Mail and Cartoon Collections +1
Slot2Text: Object-Centric Visual Tokenization for Efficient and Spatially Traceable Surgical MLLMs
Date pendingGuiqiu Liao, Matjaz Jogan, Daniel A. HashimotoImage Tokenization Healthcare
GRASP Laboratory, University of Pennsylvania · PCASO Laboratory, Department of Surgery, University of Pennsylvania · Department of Computer and Information Science, University of Pennsylvania
CAT-GS: Balanced Multimodal Learning via Calibrated Gating and Fusion Surgery
Date pendingMahir Shahriar Tamim, Sharjil Khan, Md. Samiul Alim +3Cross-Modal Learning Modality Imbalance
Department of Electrical and Computer Engineering, North South University, Dhaka, Bangladesh
Do Multimodal LLMs See Before They Read? Diagnosing Contextual Sycophancy
Date pendingYi-Cheng Lai, Hen-Hsen HuangLLM Sycophancy Evidence-Grounded Reasoning
Institute of Information Science, Academia Sinica Taipei, Taiwan
Video2Reaction: Training Foundation Video Models to Predict Audience Reaction
Date pendingSidong Zhang, Trang Nguyen, Shiv Shankar +4VLM Adaptation Video-Language Models
UMass Amherst 130 Governors Drive, Amherst · Dolby Laboratories 1275 Market Street, San Francisco

Nov 10, 2025, cs.CVJ/K move · Enter open · S save

SpatialThinker: Reinforcing Scene Graph-Grounded Spatial Reasoning via Dense Rewards

Hunar Batra, Haoqin Tu, Hardy Chen, Yuanze Lin+2

University of Oxford · University of California, Santa Cruz

Visual Question Answering Scene Graph Generation Visual Spatial Reasoning Reinforcement Learning LLM Reasoning with Graphs Dense Reward RL+3

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have achieved remarkable progress in vision-language tasks, but continue to struggle with spatial reasoning. Existing spatial MLLMs rely on large-scale datasets, explicit 3D inputs, architecture-specific modifications, or sparse Reinforcement Learning (RL) methods that provide insufficient guidance for spatially-grounded reasoning. We introduce SpatialThinker. To our knowledge, it is the first MLLM unifying Scene Graph Generation (SGG) and visual reasoning in a single pass via online RL. The model simulates human-like spatial perception by constructing a mental scene graph of task-relevant objects and relations, and reasoning toward an answer via dense spatial rewards. Our contributions are threefold: (1) SGG-grounded reasoning: integrating SGG directly within the reasoning chain rather than as a disjoint preprocessing step; (2) STVQA-7K: a high-quality spatial VQA training dataset via a scalable synthesis pipeline; and (3) a dense spatial reward design that enforces structured grounding during RL and generalizes to improve broad visual perception. SpatialThinker-7B achieves 3.6

\times

larger gains over SFT and

1.7\times

better in- and out-of-distribution generalization than sparse RL. Trained on only 7K samples, SpatialThinker-7B matches GPT-5 and outperforms GPT-4o, while SpatialThinker-30B surpasses both GPT-5 and Claude 4 Sonnet on average across 14 spatial and real-world benchmarks, demonstrating that structured spatial grounding with reward-aligned reasoning enables robust spatial understanding with limited data.

Momentum

Related topics

Latest papers 413