Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos
Jul 17, 2026Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar +19Audio-Visual Understanding Multimodal Large Language Models
1NVIDIA, USA · University of Maryland, USA
IoU-PD: IoU-Aware Privileged Distillation for Visual Grounding with Multimodal Large Language Models
Jul 17, 2026Xiuyuan Zhu, Ke Lu, Hao Wu +4Cross-Modal Knowledge Distillation Multimodal Large Language Models
University of Chinese Academy of Sciences, Beijing, China · State Key Laboratory of Communication Content Cognition, Beijing, China · Peng Cheng Laboratory, Shenzhen, Guangdong, China +1
Efficient Frame Selection for Long Videos at Test Time with Attention-Based MLLM Selectors
Jul 17, 2026Yilin Wang, Xiangxi Zheng, Dongxing Mao +6Efficient VLM Inference Multimodal Large Language Models
ZJU · NJU · CSU +2
SLAPBench: Benchmarking Multimodal Large Language Models for Four-Finger SLAP Fingerprint Verification
Jul 17, 2026Bibesh Pyakurel, M. G. Sarwar MurshedBiometric Identification Multimodal Large Language Models
Computer Science Department, University of Wisconsin–Green Bay, Green Bay, WI, USA
Beyond Single Expert: Harmonizing Diverse Visual Priors in MLLMs for Spatial Understanding
Jul 16, 2026Xiao Lin, Xiaohu Huang, Kai HanVisual Spatial Reasoning Vision Foundation Models
The University of Hong Kong
Reinforcing Egocentric Spatial Perception in Multimodal Large Language Models via Ego Scene Augmentation
Jul 16, 2026Chi Kit Wong, Ye Pan, Yuanhuiyi Lyu +6Visual Question Answering Visual Spatial Reasoning
The Hong Kong University of Science and Technology (Guangzhou) · Guangxi Zhuang Autonomous Region Information Center · The Hong Kong University of Science and Technology
Assessing AI in Introductory Physics Problem Solving
Jul 15, 2026Amir Bralin, N. Sanjay RebelloLLM Evaluation Multimodal Large Language Models
Texas Tech University, Lubbock, TX · Purdue University, West Lafayette, IN
Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild
Jul 15, 2026Ting Lei, Jialin Liu, Zhu Xu +2Multimodal Large Language Models Multimodal Agents
Wangxuan Institute of Computer Technology, Peking University, Beijing, China
Towards Enhancing 3D Spatial Reasoning in Medical Multimodal Large Language Models
Jul 15, 2026Zhuoyuan Fu, Zeshang Li, Yiqiong Zhang +53D Spatial Reasoning 3D Medical Imaging
University of International Relations, Beijing, China · University of Trento, Trento, Italy
Groc-PO: Grounded Context Preference Optimization for Truthful Multimodal LLMs
Jul 15, 2026Zhixiao Zheng, Zheren Fu, Zhiyuan Yao +3Multimodal Grounding Multimodal Large Language Models
University of Science and Technology of China Hefei, China · Xiaomi Corporation Beijing, China · State Key Laboratory of Communication Content Cognition, People’s Daily Online Beijing, China
Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
Jul 14, 2026Kaiwen Zheng, Junchen Fu, Wenhao Deng +3Efficient Multimodal Inference Multimodal Large Language Models
University of Glasgow United Kingdom · Institute of Computing Technology, Chinese Academy of Sciences China · School of Artificial Intelligence, Shandong University China +1
Segregate, Refine, Integrate: Decomposing Multimodal Fusion for Sentiment Analysis
Jul 14, 2026Alexios Filippakopoulos, Elias Kallioras, Nikolaos Xiros +2Cross-Modal Representation Learning Multimodal Large Language Models
National Technical University of Athens, Greece · Athena Research Center, Greece · University of Bern, Switzerland +2
Continual Learning with Elastic Regularization and Synthetic Replay for Federated MLLM Fine-Tuning
Jul 13, 2026Jing Liu, Chenxuanyin Zou, Jiayang Ren +5Multimodal Federated Learning Generative Replay
Department of Electrical and Computer Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada · College of Future Information Technology, Fudan University, Shanghai 200433, China · Department of Chemical and Biological Engineering, The University of British Columbia, Vancouver, BC V6T 1Z4, Canada +3
Lesioned Multimodal Language Models Reproduce Aphasic Picture-Naming Patterns
Jul 13, 2026Yong Yang, Xiang Guan, Sophie Arheix-Parras +7Multimodal Large Language Models Language Modeling
Department of Computer Science and Engineering, University of South Carolina · Department of Psychology, University of South Carolina · Department of Communication Sciences and Disorders, University of South Carolina +2
Actor as Its Own Critic: Unifying Region Understanding and Localization via CycleGRPO
Jul 13, 2026Xin Zhang, Haochen Wang, Yikang Zhou +3Group Relative Policy Optimization Multimodal Large Language Models
National University of Singapore · ByteDance
MMRM: A Multiplex Multimodal Representation Model for Product Ranking in E-commerce Search
Jul 13, 2026Zhen-Lin Chen, Maosen Sheng, Peng Lin +4Learning to Rank E-Commerce Search
JD.COM Beijing, China
Generalize LMMs to Versatile Visual Modalities via Fabricated Modality Synthesis
Jul 11, 2026Shihao Yuan, Yuanze Li, Ruyi Zhang +2VLM Adaptation Multimodal Large Language Models
Faculty of Computing, Harbin Institute of Technology, Harbin, China
Multimodal Reward Hacking in Reinforcement Learning
Jul 10, 2026Jiayu Yao, Yiwei Wang, Anmeng Zhang +5Multimodal Large Language Models Multimodal Reward Modeling
Institute of Computing Technology, Chinese Academy of Sciences · University of California, Merced · Southeast University
Mixture of Probes: Learning from Privileged Modalities in Multimodal LLMs Through Probing
Jul 9, 2026Dominick Reilly, Qiyu Wu, Hiromi Wakaki +2Missing-Modality Learning Cross-Modal Representation Learning
Sony Group Corporation · University of North Carolina at Charlotte · Sony AI
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
Jul 9, 2026Yiyang Fang, Pei Fu, Jinjie Li +7Efficient Multimodal Inference RL for Language Model Reasoning
Wuhan University · The Hong Kong University of Science and Technology · Xiaomi Inc +2
Infinity-Parser2 Technical Report
Jul 8, 2026Zuming Huang, Jun Huang, Kexuan Ren +12Synthetic Data Generation Multimodal Large Language Models
GReFEM: Multimodal LLMs as Zero-Shot Semantic Assistants for Physics-Guided 3D Mesh Refinement
Jul 8, 2026Kartik Bali, Mahish K. Guru, Christian J Cyron +1Multimodal Grounding Multimodal Large Language Models
Institute of Material Systems Modeling Helmholtz Zentrum Hereon Max-Planck-Straße 1, 21502 Geesthacht · Institute of Material and Process Design Helmholtz Zentrum Hereon Max-Planck-Straße 1, 21502 Geesthacht · Data-Driven Simulation and Analysis in Materials Science German Research Center for Artificial Intelligence Campus D3 2, 66123 Saarbrücken
GemNav: Discrete-Token Visual Robot Navigation using a Multimodal Large Language Model
Jul 8, 2026Peter Bohm, Saimunur Rahman, Abdelwahed Khamis +3VLM Adaptation Robot Navigation
CSIRO Technology, Australia
Do You Remember? Toward Memory-Centric Multimodal AI
Jul 7, 2026Xuguang Yu, Weigang Zheng, Minyue YuMemory-Augmented VLMs Multimodal Memory
POPS: Recovering Unlearned Multi-Modality Knowledge in MLLMs with Prompt-Optimized Parameter Shaking
Jul 7, 2026Zhangheng LI, Jianing Zhu, Junyuan Hong +4Multimodal Large Language Models Privacy Leakage in Language Models
University of Texas at Austin · DEVCOM Army Research Laboratory
Harrison.Rad 1.5 Technical Report: A radiology foundation model that can draft reports from images, priors and clinical context
Jul 7, 2026Suneeta Mall, Vladimir Nekrasov, Ashnil Kumar +5Radiology Report Generation Radiology VLMs
Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
Jul 7, 2026Zhiwei Yang, Yuanchen Wu, Nan Zhang +3Multimodal Large Language Models Multimodal CoT Reasoning
Tencent Youtu Lab, Shanghai, China · Fudan University, Shanghai, China
BaFCo: A Document Understanding Benchmark for Complex Bangla Form Comprehension
Jul 6, 2026Abu Tyeb Azad, Ishita Sur Apan, Fahim Ahmed +8Document Understanding LLM Evaluation
Wichita State University, USA · Center for Computational & Data Sciences, Bangladesh · University of Dhaka, Bangladesh +1
Enhancing Large Multimodal Models in Key Information Extraction via Scene-Aware Document Synthesis
Jul 6, 2026Zhipeng Xu, Zulong Chen, Qing Liu +6Synthetic Data Generation Document Information Extraction
Alibaba Group · Taobao and Tmall Group, Alibaba · Qwen Team, Alibaba Group +1
Attending to Multimodal Generation One Token at a Time
Jul 4, 2026Varun Gupta, Vineet Gandhi, Makarand TapaswiMultimodal Large Language Models Multimodal Generation
CVIT, IIIT Hyderabad

Jul 17, 2026, eess.ASJ/K move · Enter open · S save

Audio-Visual Flamingo: Open Audio-Visual Intelligence for Long and Complex Videos

Sreyan Ghosh, Arushi Goel, Kaousheik Jayakumar, Lasha Koroshinadze+18

1NVIDIA, USA · University of Maryland, USA

Audio-Visual Understanding Multimodal Large Language Models Multimodal QA Long-Video Understanding Audio-Visual QA Multimodal Foundation Models

Open paper AskPDF arXiv

We present Audio-Visual Flamingo (AV-Flamingo), a fully open state-of-the-art audio-visual large language model (AV-LLM) for joint understanding and reasoning over audio, images, and long-form videos. Unlike prior AV-LLMs that primarily focus on short clips, AV-Flamingo is designed for understanding and reasoning over long and complex real-world (audio-visual) videos. To support this, we make three key contributions: (i) Audio-Visual-Skills, a large-scale collection of real-world videos with ~7M caption and question-answer training instances designed to emphasize temporal, compositional, and cross-modal audio-visual reasoning; (ii) a novel three-stage curriculum that progressively trains the model from short-range perception to long-horizon multi-event reasoning; and (iii) Temporal Audio-Visual Interleaved Chain-of-Thought, a reasoning framework that explicitly grounds intermediate reasoning steps to timestamps in long audio-visual streams, improving temporal alignment and interpretability. Extensive experiments across 15+ audio-visual, omni-modal, audio, and vision benchmarks show that AV-Flamingo outperforms similarly sized open models by clear margins and remains highly competitive with, and in some cases surpasses, much larger open-weight and closed models, particularly on long and complex real-world audio-visual understanding and reasoning tasks. Beyond benchmark performance, AV-Flamingo exhibits strong real-world utility and transfers well to unseen tasks, highlighting its robustness and generalization ability.

Momentum

Related topics

Latest papers 676