Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 91% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 676

All topics

Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
Sep 28, 2026Jingdi lei, Junxian Li, Di Zhang +3Efficient Multimodal Inference Multimodal Large Language Models
Nanyang Technological University · Shanghai Jiao Tong University · Fudan University +2
Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models
Sep 28, 2026Zhiqiang Xia, Yang Li, Xinyuan Zhang +5Efficient Multimodal Inference Efficient VLM Inference
Xiaomi Corporation, Beijing, China
Revisit to Segment: Working Memory Distillation for Reasoning Segmentation
Sep 28, 2026Cilin Yan, Yilun Qiu, Wanyang Zhang +4Text-Guided Image Segmentation Multimodal Large Language Models
Xiaohongshu · NTU · PKU
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming
Sep 28, 2026Zongshang Shen, Wangsong Yin, Daliang Xu +2Efficient Multimodal Inference On-Device Language Model Inference
Key Lab of High Confidence Software Technologies (Peking University), Beijing, China · State Key Laboratory of Networking and Switching Technology (BUPT), Beijing, China
Counterfactual Attention Policy Distillation for Temporal Video Grounding
Sep 28, 2026Shaobo Ju, Haiyang Yu, Xuecheng Wu +5Temporal Video Grounding VLM Distillation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Fudan University. · MMLab, The Chinese University of Hong Kong. +1
MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
Sep 28, 2026Tinghao Wang, Yichen Guo, Qizhe Zhang +11Efficient Multimodal Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · University of Electronic Science and Technology of China · Nanyang Technological University +2
Uncovering Ordinal-Matching Bias in Audio-Visual LLMs
Sep 28, 2026Jihoo Jung, Youngjoon Jang, Hyebin Cho +2Audio-Visual Understanding Multimodal Large Language Models
Korea Advanced Institute of Science and Technology, South Korea · University of Oxford, United Kingdom
PhysFieldBench: Can Multimodal Models Understand Physical Fields?
Sep 28, 2026Yuezhou Ma, Huikun Weng, Jialong Wu +5Physical Reasoning Multimodal Large Language Models
School of Software, BNRist, Tsinghua University, China
MetaSampling: Making Frame Samplers Efficient for Long-Video Question Answering
Sep 27, 2026Ashim Dahal, Bikramjit BanerjeeMultimodal Large Language Models Adaptive Sampling
University of Southern Mississippi Hattiesburg, MS, USA
One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
Sep 27, 2026Sen Nie, Jie Zhang, Zhongqi Wang +2Adversarial Attacks on VLMs Adversarial Attacks on LLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, and the University of Chinese Academy of Sciences.
Learning Multimodal Embeddings with Evidence-Aligned Readout
Sep 27, 2026Zirong Chen, Fuda Ye, Enjun Du +8Multimodal Large Language Models Multimodal Embedding
The Hong Kong University of Science and Technology (Guangzhou) · Tencent Yuanbao · Tsinghua University +3
IVT-Guard: All-in-One Reasoning Model for AI-Generated Content Detection
Sep 27, 2026Hongwei Niu, Yunpeng Luo, Hanjun Li +6AI-Generated Text Detection Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen 361005, P.R. China · Tencent YouTu Lab
Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity
Sep 26, 2026Yishu Zhang, Yun Li, Daiwei ZhangMultimodal Robustness Multimodal Large Language Models
University of North Carolina at Chapel Hill
The Alignment Illusion in Multimodal Large Language Models
Sep 24, 2026Hong-Han Wang, Yuntao Wang, Hu DingRepresentational Similarity Analysis Multimodal Large Language Models
University of Science and Technology of China, Hefei, China
Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration
Sep 24, 2026Jiaqi Deng, Zonghan Wu, Zhan Heng +3Multimodal Grounding Multimodal Large Language Models
University of Technology Sydney · East China Normal University · The University of New South Wales +2
MoVISA: Multi-Token Reasoning for Video Object Segmentation
Sep 24, 2026Ruining Zhao, Ho Kei Cheng, Alexander G SchwingVideo Object Segmentation Referring Video Object Segmentation
University of Illinois Urbana-Champaign
DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs
Sep 23, 2026Yingxuan Zhuang, Miao Pan, Wangjie Gan +6Multimodal Large Language Models VLM Hallucination Mitigation
Zhejiang University · Shanghai Artificial Intelligence Laboratory
Don't Read the Log: Execution Traces Contaminate Verifiers in Video-Generation Agents
Sep 23, 2026Jian XuLLM-as-a-Judge AI Agent Auditing
RIKEN
Pistis Technical Report
Sep 23, 2026Heyun Chen, Xiaohan Lan, Jiaxi Li +17Multimodal Large Language Models Multimodal Reasoning
Pistis Team, ByteDance
Virtual Encoders in Multimodal Transformers
Sep 22, 2026Katsuya Ogata, Yuta NakashimaVisual Representation Learning Multimodal Large Language Models
The University of Osaka
BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception
Sep 22, 2026Zihan Chen, Hengguang Zhou, Yuan Kang +5VLM Distillation Multimodal Large Language Models
HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · University of California, Los Angeles
LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Sep 22, 2026Zheng Chen, Zhicheng Du, Haoxuan Li +2Medical Diagnosis Multimodal Large Language Models
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin, Guangdong, China
Qwen3.8-Omni: Towards Native Omni-Modal Agents
Sep 22, 2026Qwen TeamMultimodal Pretraining Multimodal Large Language Models
Qwen Team
SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Models
Sep 21, 2026Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac +3Efficient Multimodal Inference Multimodal Large Language Models
Kocı University, Department of Computer Engineering · Kocı University, Department of Pathology · Hacettepe University, Department of Artificial Intelligence and Data Engineering
Representation-guided in-context learning for medical image interpretation with multimodal large language models
Sep 21, 2026Minda Zhao, Fangyu Hu, Yan Luo +8Medical Image Analysis Multimodal Large Language Models
Harvard AI and Robotics Lab, Schepens Eye Research Institute of Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA · Department of Ophthalmology, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA · Department of Ophthalmology, National Taiwan University Hospital, Taipei City, Taiwan +6
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models
Sep 20, 2026Yahong Wang, Zhangkai Ni, Juncheng Wu +3Efficient Multimodal Inference Multimodal Large Language Models
Tongji University · University of California, Santa Cruz · East China Normal University +1
Fingerprinting Multimodal Large Language Models
Sep 17, 2026Chao Huang, Meng Tong, Kejiang ChenLLM Auditing Language Model Fingerprinting
University of Science and Technology of China Anhui Province Key Laboratory of Digital Security Hefei, Anhui, China
QCPruner: Query-Conditioned Population Coverage for Visual Token Pruning
Sep 17, 2026Shengli He, Yongchao Liang, Roumeng He +4Multimodal Large Language Models Visual Token Pruning
Guizhou University · Guizhou University, Guiyang, China · Shanghai Ocean University
Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detection
Sep 17, 2026Bo Xu, Chenyuan Wang, Xinyu Chen +5Multimodal Large Language Models Hate Speech Detection
School of Software, Dalian University of Technology · School of Computer Science and Technology, Dalian University of Technology · School of Computing Technologies, RMIT University
Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility
Sep 15, 2026Jaeseok Byun, Gukyeong Kwon, Han-Kai Hsu +4Multimodal Large Language Models Adapter Tuning
1Seoul National University, South Korea · 2Amazon AGI, USA

Sep 28, 2026, cs.CVJ/K move · Enter open · S save

Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models

Jingdi lei, Junxian Li, Di Zhang, Zhanqiu Zhang+2

Nanyang Technological University · Shanghai Jiao Tong University · Fudan University+2

Efficient Multimodal Inference Multimodal Large Language Models Low-Rank Adaptation

Open paper AskPDF arXiv

Long visual token sequences often account for a substantial fraction of the computational overhead in multimodal large language models~(MLLMs). Existing approaches reduce this cost by pruning redundant visual tokens, but permanently discard visual evidence that may become useful in subsequent layers. We instead ask whether all visual tokens can be preserved while reducing the cost of repeatedly evolving the representations through the Transformer. To answer this question, we perform low-rank interventions on visual-to-text information flow. We find that, after visual-to-text attention is blocked, restoring only a few directions recovers most of the lost accuracy, suggesting the relevant visual influence is concentrated in a low-dimensional subspace. We further observe strong predictability in layer-specific visual states: lightweight MLPs approximate them with high cosine similarity and low reconstruction error. Motivated by these findings, we propose

δ

-Vision, which replaces repeated Transformer evolution of visual tokens with lightweight low-rank adapters that construct layer-wise visual memories while preserving all visual tokens for text retrieval. Across image and video benchmarks,

δ

-Vision achieves higher accuracy than visual token pruning baselines at comparable or lower computation, while delivering competitive inference efficiency without discarding visual tokens.

Momentum

Related topics

Latest papers 676