Multimodal Understanding

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 413

All topics

When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs
May 12, 2026Fanpu Cao, Xin Zou, Xuming Hu +1Hallucination Detection in VLMs Hallucination in Language Models
Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China
LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?
May 11, 2026Xueqi Cheng, Yushun DongEfficient Multimodal Inference Multimodal Large Language Models
Department of Computer Science Florida State University University
Count Anything at Any Granularity
May 11, 2026Chang Liu, Haoning Wu, Weidi XieObject Counting Multimodal Large Language Models
School of Artificial Intelligence, Shanghai Jiao Tong University, China · CMIC, Shanghai Jiao Tong University, China
Probing Cross-modal Information Hubs in Audio-Visual LLMs
May 11, 2026Jihoo Jung, Chaeyoung Jung, Ji-Hoon Kim +1Cross-Modal Learning Multimodal Large Language Models
Department of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea · The Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University, Seoul, Republic of Korea
Phoenix-VL 1.5 Medium Technical Report
May 11, 2026Team Phoenix, :, Arka Ray +30Multilingual Language Models Domain Adaptation
HTX × Mistral AI ∗
Separate First, Fuse Later: Mitigating Cross-Modal Interference in Audio-Visual LLMs Reasoning with Modality-Specific Chain-of-Thought
May 11, 2026Xuanchen Li, Yuheng Lu, Chenrui Cui +6Cross-Modal Learning Multimodal QA
Tianjin Key Laboratory of Cognitive Computing and Application, Tianjin University, Tianjin, China · Tencent, China · Huiyan Technology Company, Ltd., Tianjin, China +1
The Cartesian Shortcut: Re-evaluate Vision Reasoning in Polar Coordinate Space
May 11, 2026Xia Hu, Zhenrui Yue, Brian Potetz +4VLM Evaluation Spatial Reasoning Benchmarks
Google DeepMind · Stanford University · Google Research
Do multimodal models imagine electric sheep?
May 10, 2026Santhosh Kumar Ramakrishnan, Carl Vondrick, Raja Giryes +2Visual Spatial Reasoning World Model Learning
Apple
FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning
May 10, 2026Xing Han, Shravan Chaudhari, Tanvi Ranade +2Cross-Modal Learning Healthcare
Department of Computer Science Johns Hopkins University · Department of ECE Johns Hopkins University · Department of Computer Science Johns Hopkins University, Bayesian Health
The Wittgensteinian Representation Hypothesis: Is Language the Attractor of Multimodal Convergence?
May 10, 2026Zhaoyang Zhang, Run Shao, Dongyue Wu +4Neural Representation Geometry Cross-Modal Alignment
1Central South University · 4Ant Group · 2Huazhong University of Science and Technology +1
SKG-VLA: Scene Knowledge Graph Priors for Structured Scene Semantics and Multimodal Reasoning for Decision Making
May 10, 2026Zeyu Li, Lei LiMultimodal Reasoning Knowledge Graphs
Beijing University of Posts and Telecommunications · Beijing, China
dFlowGRPO: Rate-Aware Policy Optimization for Discrete Flow Models
May 10, 2026Zhengyan Wan, Yidong Ouyang, Panwen Hu +1Group Relative Policy Optimization Image Generation
Mohamed bin Zayed University of Artificial Intelligence · East China Normal University · University of California, Los Angeles +1
DeltaRubric: Generative Multimodal Reward Modeling via Joint Planning and Verification
May 10, 2026Rui Liu, Dian Yu, Zhenwen Liang +6Multimodal Large Language Models Multimodal Reward Modeling
1Tencent Hunyuan · University of Maryland, College Park · University of North Carolina, Chapel Hill
SeePhys Pro: Diagnosing Modality Transfer and Blind-Training Effects in Multimodal RLVR for Physics Reasoning
May 10, 2026Kun Xiang, Terry Jingchen Zhang, Zirong Liu +15Cross-Modal Learning Multimodal Grounding
Sun Yat-sen University · ETH Zurich · Huawei Technologies Ltd. +1
Meow-Omni 1: A Multimodal Large Language Model for Feline Ethology
May 9, 2026Jucheng Hu, Zhangquan Chen, Yulin Chen +9Multimodal Large Language Models Multimodal Foundation Models
University College London · Tsinghua University · Nanjing University +1
Can MLLMs Reason About Visual Persuasion? Evaluating the Efficacy and Faithfulness of Reasoning
May 9, 2026Naeun Lee, Hyunjong Kim, Sunghwan Choi +2VLM Reasoning Multimodal Model Evaluation
Seoul National University
How Many Visual Tokens Do Multimodal Language Models Need? Scaling Visual Token Pruning with F^3A
May 9, 2026YiJie Huang, Yiqun Zhang, Zhuoyue Jia +7Efficient VLM Inference Large Vision-Language Models
School of Computer Science and Engineering, Northeastern University, Shenyang 110819, China · Shanghai Artificial Intelligence Laboratory · School of Computer and Communication Engineering, Northeastern University, Qinhuangdao 066004, China
DocScope: Benchmarking Verifiable Reasoning for Trustworthy Long-Document Understanding
May 9, 2026Xiang Feng, Jiawei Zhou, Zhangfeng Huang +6LLM Grounding LLM Answer Verification
School of Computer Science, National Engineering Research Center for Multimedia Software and Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan University, China · Alibaba Group, Hangzhou, China · Independent Researcher +1
Omni-DeepSearch: A Benchmark for Audio-Driven Omni-Modal Deep Search
May 9, 2026Tao Yu, yiming ding, Shenghua Chai +16Multi-Hop QA Multimodal IR
CASIA · UCAS · BAAI +2
SynerMedGen: Synergizing Medical Multimodal Understanding with Generation via Task Alignment
May 9, 2026Weiren Zhao, Yi Dong, Cheng ChenHealthcare Unified Multimodal Models
The University of Hong Kong, Hong Kong, China.
SYNCR: A Cross-Video Reasoning Benchmark with Synthetic Grounding
May 8, 2026Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy +1Multimodal Grounding Video Reasoning
New York University
jina-embeddings-v5-omni: Geometry-preserving Embeddings via Locked Aligned Towers
May 8, 2026Florian Hönicke, Michael Günther, Andreas Koukounas +4Cross-Modal Learning Cross-Modal Representation Learning
Jina by Elastic
TraceAV-Bench: Benchmarking Multi-Hop Trajectory Reasoning over Long Audio-Visual Videos
May 8, 2026Hengyi Feng, Hao Liang, Mingrui Chen +6Audio-Visual Understanding Multimodal Hallucination
University of Electronic Science and Technology of China · Peking University · Zhongguancun Academy +1
Retrieve, Integrate, and Synthesize: Spatial-Semantic Grounded Latent Visual Reasoning
May 8, 2026Jin Cui, Xinyue Long, Xunyong Zhang +5Visual Spatial Reasoning Visual Reasoning
1State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, and Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University · School of Software Engineering, State Key Laboratory of Human-Machine Hybrid Augmented Intelligence, Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Multimodal Emotion Recognition via Causal-Diffusion Bridge (Affect-Diff)
May 7, 2026Ankit SanjyalClass-Imbalanced Learning Multimodal Emotion Recognition
Department of Computer Science – Data Science Fordham University New York, USA
Steering Visual Generation in Unified Multimodal Models with Understanding Supervision
May 7, 2026Zeyu Liu, Zanlin Ni, Yang Yue +5Unified Multimodal Models Image Generation
1Tsinghua University · 2Kolors Team, Kuaishou Technology
The Cost of Context: Mitigating Textual Bias in Multimodal Retrieval-Augmented Generation
May 7, 2026Hoin Jung, Xiaoqian WangMultimodal RAG Multimodal Grounding
Elmore Family School of Electrical and Computer Engineering Purdue University West Lafayette, IN 47907
HNC: Leveraging Hard Negative Captions towards Models with Fine-Grained Visual-Linguistic Comprehension Capabilities
May 6, 2026Esra Dönmez, Pascal Tilli, Hsiu-Yu Yang +2Cross-Modal Learning Image Matching
Institute for Natural Language Processing, University of Stuttgart
To Fuse or to Drop? Dual-Path Learning for Resolving Modality Conflicts in Multimodal Emotion Recognition
May 6, 2026Yangchen Yu, Qian Chen, Jia Li +5Cross-Modal Learning Multimodal Robustness
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China · School of Computing and Information Systems, Singapore Management University, Singapore · College of Computing and Data Science, Nanyang Technological University, Singapore +1
UnAC: Adaptive Visual Prompting with Abstraction and Stepwise Checking for Complex Multimodal Reasoning
May 5, 2026Yifan Wang, Yun FuMultimodal Large Language Models Multimodal Reasoning
Northeastern University

May 12, 2026, cs.CVJ/K move · Enter open · S save

When Looking Is Not Enough: Visual Attention Structure Reveals Hallucination in MLLMs

Fanpu Cao, Xin Zou, Xuming Hu, Hui Xiong

Thrust of Artificial Intelligence, HKUST (Guangzhou), China · Department of Computer Science and Engineering, HKUST, Hong Kong SAR, China

Hallucination Detection in VLMs Hallucination in Language Models VLM Hallucination Multimodal Large Language Models Multimodal Hallucination Contrastive Decoding+2

Open paper AskPDF arXiv

Multimodal large language models (MLLMs) have become a key interface for visual reasoning and grounded question answering, yet they remain vulnerable to visual hallucinations, where generated responses contradict image content or mention nonexistent objects. A central challenge is that hallucination is not always caused by a simple lack of visual attention: the model may still assign substantial attention mass to image tokens while internally drifting toward an incorrect answer. In this paper, we show that the high-frequency structure of visual attention, measured by layer-wise Laplacian energy, reveals both the layer where hallucinated preferences emerge and the layer where the ground-truth answer transiently recovers. Building on this finding, we propose LaSCD (Laplacian-Spectral Contrastive Decoding), a training-free decoding strategy that selects informative layers via Laplacian energy and remaps next-token logits in closed form. Experiments on hallucination and general multimodal benchmarks show that LaSCD consistently reduces hallucination while preserving general capabilities, highlighting its potential as a faithful decoding paradigm. The code is available at https://github.com/macovaseas/LaSCD.

Momentum

Related topics

Latest papers 413