Multimodal Understanding

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, up 233% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 413

All topics

Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence
May 29, 2026Yuhan Wang, Shuochen Chang, Yalin Feng +8Multi-Agent Collaboration VLM Reasoning
Peking University · Shanghai Jiao Tong University · Nanyang Technological University +2
Your Multimodal Speech Model Says I Have a Face for Radio
May 28, 2026Maya K. Nachesa, Vlad Niculae, Vagrant GautamASR Evaluation Demographic Bias in ASR
Language Technology Lab University of Amsterdam · Heidelberg Institute for Theoretical Studies
Grounded 3D-Aware Spatial Vision-Language Modeling
May 28, 2026An-Chieh Cheng, Yang Fu, Yatai Ji +123D Spatial Reasoning Visually Grounded Reasoning
UCSD · NVIDIA · MIT
MuPHI: Learning Implicit Multimodal Harm Reasoning via Semantically Grounded Reward Optimization
May 28, 2026Anisha Saha, Varsha Suresh, Teodora Kamova +3VLM Robustness Multimodal Reasoning
Max Planck Institute for Informatics, Saarland Informatics Campus · Saarland University · The University of Edinburgh +1
OVA-IB: One vs All Information Bottleneck for Multi-Modal Alignment
May 28, 2026Tianchao Li, Shujian Yu, Xinrui Zu +4Cross-Modal Learning Cross-Modal Alignment
Hong Kong University of Science and Technology, Hong Kong · Vrije Universiteit Amsterdam, Netherlands · UiT – The Arctic University of Norway, Norway +3
MIRAGE: Adaptive Multimodal Gating for Whole-Brain fMRI Encoding
May 28, 2026Abdulkadir Gokce, Badr AlKhamissi, Martin SchrimpfNeuroimaging Neural Encoding
NeuroAI Lab, EPFL
MMTM: Tri-Modal Topic Modeling for Long-Form Video via Similarity-Gated Fusion
May 28, 2026Ali Abusaleh, Bhuvanesh Verma, Alexander MehlerTopic Modeling Long-Video Understanding
Text Technology Lab (TTLab), Goethe University Frankfurt
TRACER: Persistent Regularization for Robust Multimodal Finetuning
May 28, 2026Hesam Asadollahzadeh, Feng Liu, Christopher Leckie +1VLM Robustness Neural Network Robustness
School of Computing and Information Systems (CIS), Faculty of Engineering and IT (FEIT), University of Melbourne, Australia.
DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning
May 28, 2026Junzhe Zhang, Huixuan Zhang, Guirong Wang +5Counterfactual Evaluation Counterfactual Reasoning in Language Models
Wangxuan Institute of Computer Technology, Peking University · Alibaba Group
Persona Prompting in Multimodal Urban Perception: Descriptive Convergence and Interpretive Variation
May 27, 2026Neemias da Silva, Matt Ratto, Myriam Delgado +3Multimodal Large Language Models Language Model Bias Evaluation
University of Toronto, Toronto, Canada · Universidade Tecnologica Federal do Parana, Curitiba, Brazil
Agentic Active Omni-Modal Perception for Multi-Hop Audio-Visual Reasoning
May 27, 2026Ke Xu, Yuhao Wang, Ziyang Cheng +3Audio-Visual Understanding Active Perception
Shanghai Jiao Tong University
Look on Demand: A Cognitive Scheduling Framework for Visual Evidence Acquisition in Multimodal Reasoning
May 27, 2026Yang Zhang, Xiaoshuai Sun, Rui Zhao +5Active Perception Multimodal Reasoning
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Sino-Russian ResearchCenter for Digital Economy. · School of Informatics, Xiamen University, China. +2
Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +4Missing-Modality Learning VLM Robustness
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +3
CuriosAI Submission to the CASTLE Challenge at EgoVis 2026
May 27, 2026Yuto Kanda, Hayato Tanoue, Takayuki HoriEgocentric Video QA Egocentric Video Understanding
SoftBank Corp.
Bounded-Compute Multimodal Regression for Product-Rating Prediction
May 26, 2026William Leach, Ru He, Sizhuo Ma +4Vision-Language Models VLM Adaptation
Snap Inc.
How and What to Imagine? Visual Thinking in Unified Multimodal Models for Cross-View Spatial Reasoning
May 26, 2026Qian Yang, Ankur Sikarwar, Huy Le +4Visual Spatial Reasoning Unified Multimodal Models
Mila - Québec AI Institute · Université de Montréal · McGill University +2
Gemini Embedding 2: A Native Multimodal Embedding Model from Gemini
May 26, 2026Madhuri Shanbhogue, Zhe Li, Shanfeng Zhang +86Cross-Modal Learning Unified Multimodal Models
Can Retrieval Heads See Images? Multimodal Retrieval Heads in Long-Context Vision-Language Models
May 26, 2026Aaron Branson Cigres Li, Zhaowei Wang, Yu Zhao +9Vision-Language Models VLM Interpretability
1HKUST · University of Edinburgh · 4CUHK +2
DynFrame: Adaptive Reasoning-Driven Multimodal Framework with Dynamic Frame Augmentation for Complex Video Understanding
May 26, 2026Peng Zhang, Guanghao Zhang, Wanggui He +10Multimodal Large Language Models Video Reasoning
Zhejiang University · Alibaba Group
OmniRetriever: Any-to-Any Audio-Video-Text Retrieval via Fusion-as-Teacher Distillation
May 26, 2026Yunze Liu, Chi-Hao Wu, Enmin Zhou +1Cross-Modal Learning Cross-Modal Knowledge Distillation
Memories.ai Research
O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
May 26, 2026Peiran Wu, Yunze Liu, Chi-Hao Wu +2Audio-Visual Understanding Efficient Multimodal Inference
University of Bristol · Memories.ai Research · University of Central Florida
OmniInteract: Benchmarking Real-World Streaming Interaction for Real-Time Omnimodal Assistants
May 26, 2026Xudong Lu, Xueying Li, Annan Wang +8Streaming Video Understanding Multimodal Foundation Models
CUHK MMLab · SJTU · NTU +3
Not All Modalities Are Equal: Instruction-Aware Gating for Multimodal Videos
May 25, 2026Bonan Ding, Umair Nawaz, Ufaq Khan +5Cross-Modal Learning Multimodal Large Language Models
Mohamed bin Zayed University of Artificial Intelligence · Tianjin University · Chongqing University +1
VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding
May 25, 2026Yinghao Wu, Zhuoyan Luo, Yiyao Yu +3Efficient Multimodal Inference Mixture-of-Experts Models
Tsinghua University
Visual-Redundancy-Controlled Parallel Decoding for Diffusion-Based Multimodal Large Language Models
May 25, 2026Yulin Yuan, Hongshuo Zhao, Xiangming MengLanguage Model Decoding Multimodal Large Language Models
Zhejiang University · ZJUI-UIUC Institute
Can MLLMs Reason Beyond Language? VisReason: A Comprehensive Benchmark for Vision-Centric Reasoning
May 25, 2026Longteng Guo, Yifan Wang, Pengkang Huo +4Visual Reasoning VLM Reasoning
Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences
Toward Native Multimodal Modeling: A Roadmap
May 25, 2026Siyu An, Junru Lu, Junnan Dong +18Unified Multimodal Models Multimodal Generation
Tencent Youtu Lab
DIVA: Harnessing the Representation Divergence in Unified Multimodal Models for Mutual Reinforcement
May 25, 2026Renjie Lu, Xulong Zhang, Xiaoyang Qu +2Multimodal Disentangled Representation Learning Unified Multimodal Models
Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, China · University of Science and Technology of China, Hefei, China.
AOEPT: Breaking the Implicit Modality-Reduction Bottleneck in Modality-Missing Prompt Tuning
May 24, 2026Jian Lang, Rongpei Hong, Ting Zhong +1Missing-Modality Learning Multimodal Reasoning
University of Electronic Science and Technology of China, Chengdu, Sichuan · Intelligent Digital Media Technology Key Laboratory of Sichuan Province, Chengdu, Sichuan
OmniEgo-R $^2$ : A Routed Reasoning Framework for the 1st Cross-Domain EgoCross Challenge at CVPR 2026
May 23, 2026Zixu Li, Zhiwei Chen, Zhiheng Fu +4Egocentric Video QA Egocentric Video Understanding
Shandong University · Harbin Institute of Technology (Shenzhen)

May 29, 2026, cs.CVJ/K move · Enter open · S save

Seeing Before Agreeing: Aligning Multi-Agent Consensus with Visual Evidence

Yuhan Wang, Shuochen Chang, Yalin Feng, Dongsheng Ma+7

Peking University · Shanghai Jiao Tong University · Nanyang Technological University+2

Multi-Agent Collaboration VLM Reasoning Multi-Agent Consensus Vision-Language Alignment Multimodal Understanding

Open paper AskPDF arXiv

Vision-language models (VLMs) have achieved strong performance on visual question answering (VQA). To mitigate individual hallucinations and blind spots, aggregating diverse perspectives via multi-agent collaboration has emerged as a promising paradigm. While this approach has shown great success in textual QA, its potential in the multimodal domain remains under-explored. Existing multi-agent VQA methods predominantly adapt text-centric protocols, focusing on textual discussions while ignoring the alignment of visual information. In this work, we reveal a key insight: answer-level agreement is insufficient for reliable multi-agent VQA; \textit{aligned visual evidence} -- shared support from the image regions agents rely on -- is essential for trustworthy consensus. To leverage this insight, we propose EAGLE (\textbf{E}vidence-\textbf{A}ligned \textbf{G}rounded mu\textbf{L}ti-agent r\textbf{E}asoning), a training-free evidence-centered framework for coordinating multiple VLM agents. EAGLE explicitly exposes each agent's grounding regions as visual evidence, enables mutual verification over the evidence, and uses evidence consistency to guide final decision-making. Experiments on six VQA benchmarks show that EAGLE achieves best average performance across domains while remaining lightweight, interpretable, and practical for deployment.

Momentum

Related topics

Latest papers 413