Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

DOSE: Data Selection for Multi-Modal LLMs via Off-the-Shelf Models
Apr 18, 2026Biao Wu, Yiwu Zhong, Meng Fang +1Multimodal Large Language Models Data Selection
Australian Artificial Intelligence Institute · Peking University · University of Liverpool
MNAFT: modality neuron-aware fine-tuning of multimodal large language models for image translation
Apr 18, 2026Bo Li, Ningyuan Deng, Tianyu Dong +3Fine-Tuning VLM Adaptation
School of Software, Tsinghua University, Beijing, China · Baidu Inc., Beijing, China · School of Information Resource Management, Renmin University of China,Beijing, China +2
PivotMerge: Bridging Heterogeneous Multimodal Pre-training via Post-Alignment Model Merging
Apr 18, 2026Zibo Shao, Baochen Xiong, Xiaoshan Yang +4Cross-Modal Learning Cross-Modal Alignment
State Key Laboratory of Multimodal Artificial Intelligence Systems, Institute of Automation, Chinese Academy of Sciences, Beijing 100190, China · Pengcheng Laboratory, Shenzhen 518066, China · School of Artificial Intelligence, University of Chinese Academy Sciences, Beijing 100049, China +1
Bridging Coarse and Fine Recognition: A Hybrid Approach for Open-Ended Multi-Granularity Object Recognition in Interactive Educational Games
Apr 18, 2026Hanling Yi, Feng Lin, Mao Luo +3Multimodal Large Language Models Fine-Grained Visual Perception
Intellifusion Inc.
Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
Apr 17, 2026Junwan Kim, Hyunkyung BaeEfficient Multimodal Inference Multimodal Large Language Models
New York University
Mind's Eye: A Benchmark of Visual Abstraction, Transformation and Composition for Multimodal LLMs
Apr 17, 2026Rohit Sinha, Aditya Kanade, Sai Srinivas Kancheti +2Spatial Reasoning Benchmarks Visual Spatial Reasoning
CSE Dept., IIT Hyderabad Hyderabad, India · Microsoft Research Bengaluru, India
MCBench: A Multicontext Safety Assessment Benchmark for Omni Large Language Models
Apr 17, 2026Manh Luong, Tamas Abraham, Junae Kim +6LLM Safety Benchmarks Language Model Safety Evaluation
Monash University, Australia · Defence Science and Technology Group, Australia
M3DocDep: Multi-modal, Multi-page, Multi-document Dependency Chunking with Large Vision-Language Models
Apr 17, 2026Joongmin Shin, Jeongbae Park, Jaehyung Seo +1Multimodal RAG Document Chunking
Human-inspired AI Research, Korea University · Computer Science and Engineering, Konkuk University · Department of Computer Science and Engineering, Korea University
Towards Robust Endogenous Reasoning: Unifying Drift Adaptation in Non-Stationary Tuning
Apr 17, 2026Xiaoyu Yang, En Yu, Wei Duan +1Multimodal Robustness Multimodal Large Language Models
Australian Artificial Intelligence Institute (AAII), Faculty of Engineering and Information Technology, University of Technology Sydney, Australia
MirrorBench: Evaluating Self-centric Intelligence in MLLMs by Introducing a Mirror
Apr 16, 2026Shengyu Guo, Tongrui Ye, Jianbo Zhang +3Multimodal Large Language Models AI Agent Benchmarks
Shanghai AI Lab
Chain of Modality: From Static Fusion to Dynamic Orchestration in Omni-MLLMs
Apr 16, 2026Ziyang Luo, Nian Liu, Junwei HanMultimodal Large Language Models Multimodal Reasoning
Northwestern Polytechnical University Xi’an, Shaanxi, China
FiRe: Fine-grained Multimodal Reasoning for Enhanced Image Generation
Apr 15, 2026Yongjin Kim, Yoonjin Oh, Yerin Kim +5Controllable Image Generation Compositional T2I Generation
Department of Artificial Intelligence Korea University Seoul, Republic of Korea · KT Corporation Seoul, Republic of Korea
Can LLMs Reason About Attention? Towards Zero-Shot Analysis of Multimodal Classroom Behavior
Apr 3, 2026Nolan Platt, Sehrish Nizamani, Alp Tural +5Multimodal Large Language Models
Virginia Tech, Blacksburg, Virginia, USA · Incoming PhD student, University of Pennsylvania. · University of Virginia, Charlottesville, Virginia, USA
ReLope: From Hidden-State Probing to a Decision Module for Multimodal LLM Routing
Mar 25, 2026Yaopei Zeng, Congchao Wang, Blake JianHang Chen +1Multimodal Large Language Models LLM Routing
Pennsylvania State University · Google · Amazon AGI
From Content to Audience: A Multimodal Annotation Framework for Broadcast Television Analytics
Mar 24, 2026Paolo Cupini, Francesco PierriLLM-Assisted Annotation Multimodal Large Language Models
Politecnico di Milano
HumanOmni-Speaker: Identifying Who said What and When
Mar 23, 2026Detao Bai, Zhiheng Ma, Xihan WeiAudio-Visual Understanding Visual Speaker Recognition
Tongyi Lab Alibaba Group · Shenzhen University of Advanced Technology · Guangdong Provincial Key Laboratory of Computility Microelectronics +1
ProactiveBench: Benchmarking Proactiveness in Multimodal Large Language Models
Mar 19, 2026Thomas De Min, Subhankar Roy, Stéphane Lathuilière +2Multimodal Large Language Models AI Agent Benchmarks
University of Trento · University of Bergamo · Inria, Univ. Grenoble Alpes, CNRS, LJK +1
HiMu: Hierarchical Multimodal Frame Selection for Long Video Question Answering
Mar 19, 2026Dan Ben-Ami, Gabriele Serussi, Kobi Cohen +1Multimodal Large Language Models Multimodal QA
INSIGHT Lab, Ben-Gurion University of the Negev, Israel · Ben-Gurion University of the Negev, Israel
Beyond Medical Diagnostics: How Medical Multimodal Large Language Models Think in Space
Mar 14, 2026Quoc-Huy Trinh, Xi Ding, Yang Liu +73D Spatial Reasoning Multimodal Large Language Models
CoMMET: A Psychologically Grounded Benchmark for Evaluating Theory of Mind in Multimodal LLMs
Mar 12, 2026Ruirui Chen, Weifeng Jiang, Chengwei Qin +3Moral Reasoning in Language Models Theory of Mind
Institute of High Performance Computing (IHPC) and 2Centre for Frontier AI Research (CFAR), Agency for Science, Technology and Research (A*STAR), Singapore · Nanyang Technological University, Singapore · Hong Kong University of Science and Technology (Guangzhou), China
Beyond Sequential Distance: Inter-Modal Distance Invariant Position Encoding
Mar 11, 2026Lin Chen, Bolin Ni, Qi Yang +5Multimodal Large Language Models Long-Context Modeling
1MAIS, Institute of Automation, Chinese Academy of Sciences, China · University of Chinese Academy of Sciences, China · 3Tencent +1
EXPLORE-Bench: Egocentric Scene Prediction with Long-Horizon Reasoning
Mar 10, 2026Chengjun Yu, Xuhan Zhu, Chaoqun Du +4Multimodal Large Language Models Egocentric Video Understanding
University of Science and Technology of China · Foundation Model Team, Li Auto Inc. · Tsinghua University
Real-Time Generation of Game Video Commentary with Multimodal LLMs: Pause-Aware Decoding Approaches
Mar 3, 2026Anum Afzal, Yuki Saito, Hiroya Takamura +5Language Model Decoding Multimodal Large Language Models
School of CIT, Technical University of Munich · National Institute of Advanced Industrial Science and Technology (AIST) · The University of Tokyo +3
MUSE: A Run-Centric Platform for Multimodal Unified Safety Evaluation of Large Language Models
Mar 3, 2026Zhongxi Wang, Yueqian Lin, Jingyang Zhang +3LLM Safety Benchmarks Language Model Safety Evaluation
1Duke University · 2Virtue AI
Advancing Multimodal Judge Models through a Capability-Oriented Benchmark and MCTS-Driven Data Generation
Feb 28, 2026Zeyu Chen, Huanjin Yao, Ziwang Zhao +1LLM Evaluation LLM-as-a-Judge
EMO-R3: Reflective Reinforcement Learning for Emotional Reasoning in Multimodal Large Language Models
Feb 27, 2026Yiyang Fang, Wenke Huang, Pei Fu +5Multimodal Large Language Models Multimodal Reasoning
School of Computer Science, Wuhan University. · MiLM Plus, Xiaomi Inc.
BrepCoder: A Unified Multimodal Large Language Model for Multi-task B-rep Reasoning
Feb 25, 2026Mingi Kim, Yongjun Kim, Jungwoo Kang +1Multimodal Large Language Models
Chungnam National University, Daejeon, Republic of Korea
Re-purposing Multimodal Large Language Models for Audio-Text Retrieval
Feb 20, 2026Jilan Xu, Carl Thomé, Danijela Horak +2Multimodal Large Language Models Audio Understanding
Visual Geometry Group, University of Oxford · Epidemic Sound · School of Artificial Intelligence, Shanghai Jiao Tong University
Beyond Pixels: A Vector-to-Graph Framework for Reliable Schematic Auditing
Feb 12, 2026Chengwei Ma, Zhen Tian, Zhou Zhou +5Multimodal Large Language Models Multimodal Model Evaluation
Guangdong Laboratory of Artificial Intelligence and Digital Economy (SZ), Shenzhen, China · Guangdong Power Grid Co., Ltd., Yangjiang Power Supply Bureau, Yangjiang, China · Shanghai University, Shanghai, China +1
Ex-Omni: Enabling 3D Facial Animation Generation for Omni-modal Large Language Models
Feb 6, 2026Haoyu Zhang, Zhipeng Li, Yiwen Guo +1Multimodal Large Language Models Audio-Driven Facial Animation
The Chinese University of Hong Kong, Shenzhen · LIGHTSPEED · Independent Researcher

Apr 18, 2026, cs.CVJ/K move · Enter open · S save

DOSE: Data Selection for Multi-Modal LLMs via Off-the-Shelf Models

Biao Wu, Yiwu Zhong, Meng Fang, Ling Chen

Australian Artificial Intelligence Institute · Peking University · University of Liverpool

Multimodal Large Language Models Data Selection Multimodal Learning

Open paper AskPDF arXiv

High-quality and diverse multimodal data are essential for improving vision-language models (VLMs), yet existing datasets often contain noisy, redundant, and poorly aligned samples. To address these problems, data filtering is commonly used to enhance the efficiency and performance of multimodal learning, but it introduces extra computational cost because filtering models are usually trained on the same data they are meant to screen. To reduce this cost, we study DOSE, which explores whether off-the-shelf pretrained models that have never seen the target data can be used to select training samples for larger and stronger multimodal models without any task-specific training. Even without fine-tuning, these models can effectively assess text quality and image-text alignment to guide data selection. Based on this, we build a joint quality-alignment distribution and apply adaptive weighted sampling to select informative samples while maintaining long-tail diversity. This approach enhances data diversity, enabling models trained on DOSE-filtered data to match or surpass those trained on the full dataset on standard VQA and math benchmarks. Extensive experiments demonstrate its effectiveness, efficiency, and scalability.

Momentum

Related topics

Latest papers 676