Multimodal Large Language Models

Also known as MLLM

Follow this topic Ask about this topic

Momentum

67 papers in the last four weeks, up 109% on the four weeks before. 0.7% of all new papers.

Jul 13Week of Sep 28

Latest papers 669

All topics

RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models
Oct 6, 2026Dongchen Si, Di Wang, Mingzhen Xu +3Remote Sensing Image Understanding Efficient Multimodal Inference
School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
DIPrune: Task-Aware Token Pruning with Dual Importance for Efficient Multimodal Language Models
Oct 6, 2026Shuo Yang, Changbai Li, Linlin Yang +6Multimodal Large Language Models Token Pruning
Beihang University · Communication University of China · National University of Singapore
VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
Oct 6, 2026Yuan Feng, Qize Yang, Ruizhe Chen +9Efficient Multimodal Inference Multimodal Large Language Models
University of Science and Technology of China · Alibaba Token Hub, Alibaba Group · The Chinese University of Hong Kong +1
Two Vectors Replace In-Context Demos: Structured Task Adaptation via Embeddings
Oct 6, 2026Xi Ding, Naichen Shi, Jiawei ZhangMultimodal ICL Task Vectors
University of Wisconsin–Madison · Northwestern University
Harnessing Multimodal Large Language Models for Training-Free Human-Object Interaction Detection
Oct 5, 2026Zhaolin Cai, Huiyu Duan, Liu Yang +5Multimodal Large Language Models Active Visual Perception
Shanghai Jiao Tong University · Xinjiang University · Beijing Jiaotong University +1
Human-Like Attention? A Psychophysical Comparison of Visual Search in Humans and MLLMs
Oct 4, 2026Renchi Zhang, Joost C. F. de Winter, Dimitra Dodou +2Multimodal Large Language Models Visual Search
Faculty of Mechanical Engineering, Delft University of Technology, Delft, The Netherlands
A Strong Baseline for Evaluating Vision Encoders in Multimodal Large Language Models
Oct 4, 2026Yilin Yang, Jun-Tao Tang, Kengyi Wang +3VLM Evaluation Multimodal Large Language Models
School of Artificial Intelligence, Shanghai Jiao Tong University · Nanjing University · Fudan University +1
OmniConfess: Eliciting Token Confessions to Mitigate Omni-Modal Hallucination
Oct 2, 2026Huiqiang Rong, Haoran Luo, Hui Feng +4Multimodal Large Language Models Multimodal Hallucination
Beijing University of Posts and Telecommunications · Nanyang Technological University, Singapore
OmniSeek: Native Tool Integration for Multi-turn Audio-Visual Reasoning
Oct 1, 2026Haibo Wang, Jiteng Mu, Jialu Li +5Audio-Visual Understanding RL for Language Model Reasoning
University of California, Davis · Adobe Research
Where-OPD: Spatially Guided On-Policy Self-Distillation of MLLMs with Synthetic Scenes
Oct 1, 2026Sophia Sirko-Galouchenko, Monika Wysoczanska, Andrei Bursuc +2VLM Distillation Multimodal Large Language Models
Sorbonne Universit´e, CNRS, ISIR, F-75005 Paris, France · ILLS, CNRS, Montreal, QC H2S 3H1 · Institut universitaire de France (IUF)
MWOP: Modality-aware Width-wise Operation Pruning for Efficient MLLMs
Oct 1, 2026Xudong Wang, Hao Wu, Haozhe Hu +5LLM Pruning Efficient VLM Inference
EIT-NLP Lab, Eastern Institute of Technology, Ningbo · Shanghai Jiao Tong University · The Hong Kong Polytechnic University +1
HAWK: Rethinking Multimodal Drafting for Speculative Decoding
Sep 30, 2026Wenhan Yang, Anirudh Rao, Ashwin ChandraEfficient Multimodal Inference VLM Distillation
University of California, Los Angeles · Samsung Research America
Ranking-Aware Prompt Optimization for Multimodal Clinical Diagnosis
Sep 30, 2026Tian Xia, Minghao Liu, Yiqing Liang +2Medical Diagnosis Multimodal Large Language Models
Harvard University · University of California, Santa Cruz · Brown University +2
Learning to Reason with Compressed Context: Ground-Truth-Free Adaptation of OmniLLMs via Self-Distillation
Sep 30, 2026Jianghao Wang, Ke Meng, Jian Li +7Efficient Multimodal Inference Multimodal Large Language Models
School of Computer Science, Wuhan University · Didi Chuxing
Reinforcing Multimodal Reasoning via Token-Level Perception-Grounded Advantage Estimation
Sep 30, 2026Zhihan Zhang, Lizi LiaoToken-Level Credit Assignment Multimodal Large Language Models
Singapore Management University Singapore, Singapore
MM-FinEval: A Multi-Task Multimodal Benchmark for Real-World Financial Forecasting
Sep 29, 2026Dong Shu, Yanguang Liu, Huopu Zhang +4Financial Forecasting Multimodal Large Language Models
Northwestern University · New Jersey Institute of Technology · Georgia Institute of Technology +2
Exemplar2VQA: A Scalable Exemplar-Driven Visual Question Answering Generation Framework via Multi-Agent Coding
Sep 29, 2026Jiayu Ying, Qijian Tian, Ruijie Xu +4Visual Question Answering Multimodal Large Language Models
East China Normal University · Shanghai Jiao Tong University · Fudan University +1
Devils in Question Relay: Source-Conditioned Relay Steering to Mitigate Hallucinations in Audio-visual Large Language Models
Sep 29, 2026Yu Zhang, Pingrui Zhang, Xuefeng Bai +3LLM Hallucination Mitigation Hallucination in Language Models
Harbin Institute of Technology, Shenzhen, China
Why MLLMs Struggle to Count: Overcoming Individuation and Aggregation Bottlenecks with ConvStack
Sep 29, 2026Liwei Che, Yihao Quan, Sen Fang +4Object Counting Multimodal Large Language Models
Department of Computer Science, Rutgers University · Department of Computer Science, University of Washington
OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models
Sep 29, 2026Yuchen Deng, Zidang Cai, Feidiao Yang +4Efficient Multimodal Inference Audio Token Compression
Shenzhen International Graduate School, Tsinghua University, China · Pengcheng Laboratory, China
Calibrate the Decisions That Change the Future: On-Policy Post-Training Quantization for Multimodal Large Language Models
Sep 29, 2026Wenxiao Fan, Jingling Fu, Lichen Ma +6LLM Quantization VLM Quantization
School of Computer Science and Technology, Beijing Institute of Technology · Institute of Artificial Intelligence and Robotics, Xi’an Jiaotong University
Seeing What Should Be Heard: Diagnosing and Repairing Cross-Modal Shortcuts in Omni-Modal LLMs
Sep 29, 2026Yueran Ma, Ronghao LinAudio QA Multimodal Large Language Models
The University of Queensland, Brisbane, Australia · Shenzhen University, Shenzhen, China
Decoding Affective Nuances: Enhancing MLLMs via Hierarchical Emotion Reasoning and Contrastive Discriminative Pruning
Sep 29, 2026Cheng Ye, Weidong Chen, Zhaobo Qi +2Multimodal Large Language Models Multimodal Emotion Recognition
University of Science and Technology of China, Hefei · Harbin Institute of Technology, Weihai
AdaKerNet: Neural Kernel Decoding for Task-Adaptive Prediction with Multimodal Large Models
Sep 28, 2026Konstantinos D. Polyzos, Eleni Oikonomou, Tara JavidiMultimodal Large Language Models Kernel Methods
University of California San Diego
Render Before Reading: Visual Rendering as a Prompt Injection Defense
Sep 28, 2026Jie Zhang, Andrei Baroian, Jan N. van Rijn +2Multimodal Large Language Models Modality Gap in VLMs
ETH Zurich · Leiden University
How Far Are We from Removing the Visual Encoder? Scaling Laws for Encoder-Free Multimodal Pretraining
Sep 28, 2026Lin Chen, Bolin Ni, Qi Yang +6Multimodal Pretraining Multimodal Large Language Models
CASIA · UCAS · Foundation Model Department, Tencent
SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
Sep 28, 2026Tianxiang Chen, Zhentao Tan, Zi Ye +9Efficient Multimodal Inference Transformer
End-User Intelligent Computing BU, Alibaba Cloud · Alibaba Group · Department of Computer Science, Maynooth University +2
Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
Sep 28, 2026Jingdi lei, Junxian Li, Di Zhang +3Efficient Multimodal Inference Multimodal Large Language Models
Nanyang Technological University · Shanghai Jiao Tong University · Fudan University +2
Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models
Sep 28, 2026Zhiqiang Xia, Yang Li, Xinyuan Zhang +5Efficient Multimodal Inference Efficient VLM Inference
Xiaomi Corporation, Beijing, China
Revisit to Segment: Working Memory Distillation for Reasoning Segmentation
Sep 28, 2026Cilin Yan, Yilun Qiu, Wanyang Zhang +4Text-Guided Image Segmentation Multimodal Large Language Models
Xiaohongshu · NTU · PKU
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming
Sep 28, 2026Zongshang Shen, Wangsong Yin, Daliang Xu +2Efficient Multimodal Inference On-Device Language Model Inference
Key Lab of High Confidence Software Technologies (Peking University), Beijing, China · State Key Laboratory of Networking and Switching Technology (BUPT), Beijing, China
Counterfactual Attention Policy Distillation for Temporal Video Grounding
Sep 28, 2026Shaobo Ju, Haiyang Yu, Xuecheng Wu +5Temporal Video Grounding VLM Distillation
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China. · Fudan University. · MMLab, The Chinese University of Hong Kong. +1
MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
Sep 28, 2026Tinghao Wang, Yichen Guo, Qizhe Zhang +11Efficient Multimodal Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · University of Electronic Science and Technology of China · Nanyang Technological University +2
Uncovering Ordinal-Matching Bias in Audio-Visual LLMs
Sep 28, 2026Jihoo Jung, Youngjoon Jang, Hyebin Cho +2Audio-Visual Understanding Multimodal Large Language Models
Korea Advanced Institute of Science and Technology, South Korea · University of Oxford, United Kingdom
PhysFieldBench: Can Multimodal Models Understand Physical Fields?
Sep 28, 2026Yuezhou Ma, Huikun Weng, Jialong Wu +5Physical Reasoning Multimodal Large Language Models
School of Software, BNRist, Tsinghua University, China
MetaSampling: Making Frame Samplers Efficient for Long-Video Question Answering
Sep 27, 2026Ashim Dahal, Bikramjit BanerjeeMultimodal Large Language Models Adaptive Sampling
University of Southern Mississippi Hattiesburg, MS, USA
One Attack to Fool Them All: Highly Transferable Black-Box Adversarial Attacks on Frontier MLLMs
Sep 27, 2026Sen Nie, Jie Zhang, Zhongqi Wang +2Adversarial Attacks on VLMs Adversarial Attacks on LLMs
State Key Laboratory of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences, and the University of Chinese Academy of Sciences.
Learning Multimodal Embeddings with Evidence-Aligned Readout
Sep 27, 2026Zirong Chen, Fuda Ye, Enjun Du +8Multimodal Large Language Models Multimodal Embedding
The Hong Kong University of Science and Technology (Guangzhou) · Tencent Yuanbao · Tsinghua University +3
IVT-Guard: All-in-One Reasoning Model for AI-Generated Content Detection
Sep 27, 2026Hongwei Niu, Yunpeng Luo, Hanjun Li +6AI-Generated Text Detection Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Xiamen 361005, P.R. China · Tencent YouTu Lab
Multimodal LLMs Outperform Pathology Foundation Models in Cross-Domain Histological Similarity
Sep 26, 2026Yishu Zhang, Yun Li, Daiwei ZhangMultimodal Robustness Multimodal Large Language Models
University of North Carolina at Chapel Hill
The Alignment Illusion in Multimodal Large Language Models
Sep 24, 2026Hong-Han Wang, Yuntao Wang, Hu DingRepresentational Similarity Analysis Multimodal Large Language Models
University of Science and Technology of China, Hefei, China
Mind What Matters for Reasoning: Aligning Cross-Modal Attention via Selective Probability Mass Concentration
Sep 24, 2026Jiaqi Deng, Zonghan Wu, Zhan Heng +3Multimodal Grounding Multimodal Large Language Models
University of Technology Sydney · East China Normal University · The University of New South Wales +2
MoVISA: Multi-Token Reasoning for Video Object Segmentation
Sep 24, 2026Ruining Zhao, Ho Kei Cheng, Alexander G SchwingVideo Object Segmentation Referring Video Object Segmentation
University of Illinois Urbana-Champaign
DEEPO: Dual-Entropy Enhanced Policy Optimization for Hallucination in MLLMs
Sep 23, 2026Yingxuan Zhuang, Miao Pan, Wangjie Gan +6Multimodal Large Language Models VLM Hallucination Mitigation
Zhejiang University · Shanghai Artificial Intelligence Laboratory
Don't Read the Log: Execution Traces Contaminate Verifiers in Video-Generation Agents
Sep 23, 2026Jian XuLLM-as-a-Judge AI Agent Auditing
RIKEN
Pistis Technical Report
Sep 23, 2026Heyun Chen, Xiaohan Lan, Jiaxi Li +17Multimodal Large Language Models Multimodal Reasoning
Pistis Team, ByteDance
Virtual Encoders in Multimodal Transformers
Sep 22, 2026Katsuya Ogata, Yuta NakashimaVisual Representation Learning Multimodal Large Language Models
The University of Osaka
BAS-OPD: Budget-Aware Selective On-Policy Self-Distillation for Fine-Grained Multimodal Perception
Sep 22, 2026Zihan Chen, Hengguang Zhou, Yuan Kang +5VLM Distillation Multimodal Large Language Models
HFIPS, Chinese Academy of Sciences · University of Science and Technology of China · University of California, Los Angeles
LingLan: An Advancing Traditional Chinese Medicine Diagnosis LLM with Multimodal Data
Sep 22, 2026Zheng Chen, Zhicheng Du, Haoxuan Li +2Medical Diagnosis Multimodal Large Language Models
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China · Guangdong Provincial Laboratory of Traditional Chinese Medicine Hengqin, Guangdong, China
Qwen3.8-Omni: Towards Native Omni-Modal Agents
Sep 22, 2026Qwen TeamMultimodal Pretraining Multimodal Large Language Models
Qwen Team
SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Models
Sep 21, 2026Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac +3Efficient Multimodal Inference Multimodal Large Language Models
Kocı University, Department of Computer Engineering · Kocı University, Department of Pathology · Hacettepe University, Department of Artificial Intelligence and Data Engineering
Representation-guided in-context learning for medical image interpretation with multimodal large language models
Sep 21, 2026Minda Zhao, Fangyu Hu, Yan Luo +8Medical Image Analysis Multimodal Large Language Models
Harvard AI and Robotics Lab, Schepens Eye Research Institute of Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA · Department of Ophthalmology, Massachusetts Eye and Ear, Harvard Medical School, Boston, MA, USA · Department of Ophthalmology, National Taiwan University Hospital, Taipei City, Taiwan +6
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models
Sep 20, 2026Yahong Wang, Zhangkai Ni, Juncheng Wu +3Efficient Multimodal Inference Multimodal Large Language Models
Tongji University · University of California, Santa Cruz · East China Normal University +1
Fingerprinting Multimodal Large Language Models
Sep 17, 2026Chao Huang, Meng Tong, Kejiang ChenLLM Auditing Language Model Fingerprinting
University of Science and Technology of China Anhui Province Key Laboratory of Digital Security Hefei, Anhui, China
QCPruner: Query-Conditioned Population Coverage for Visual Token Pruning
Sep 17, 2026Shengli He, Yongchao Liang, Roumeng He +4Multimodal Large Language Models Visual Token Pruning
Guizhou University · Guizhou University, Guiyang, China · Shanghai Ocean University
Learn Before You Judge: Progressive Knowledge-to-Decision Alignment for Explainable Hateful Meme Detection
Sep 17, 2026Bo Xu, Chenyuan Wang, Xinyu Chen +5Multimodal Large Language Models Hate Speech Detection
School of Software, Dalian University of Technology · School of Computer Science and Technology, Dalian University of Technology · School of Computing Technologies, RMIT University
Multi-modal Knowledge Preserving Adapter for Embedding Backward Compatibility
Sep 15, 2026Jaeseok Byun, Gukyeong Kwon, Han-Kai Hsu +4Multimodal Large Language Models Adapter Tuning
1Seoul National University, South Korea · 2Amazon AGI, USA
RegRet: Enhancing Region-Level Retrieval in Large Multimodal Models
Sep 15, 2026Xun Liang, Honghui Yang, Weihang Pan +6Multimodal IR Multimodal Large Language Models
State Key Lab of CAD&CG, Zhejiang University · Xiaohongshu Inc. · School of Software Technology, Zhejiang University
ReDraft, Don't Just Distill: Reference-Driven Revision for Continual VLLM Post-Training
Sep 15, 2026Zhihao Zhang, Mingqi Wu, Qiaole Dong +15Fine-Tuning Multimodal Large Language Models
Fudan University · Shanghai Artificial Intelligence Laboratory · Tencent
A multimodal large language model for evidence-based autism spectrum disorder screening
Sep 15, 2026Jun Chen, Qi Zhao, Yunliang Jiang +8Autism Spectrum Disorder Multimodal Large Language Models
National Special Education Resource Center for Children with Autism, Zhejiang Normal University, China · School of Computer Science and Technology, Zhejiang Normal University, China · School of Mathematics and Statistics, Xi’an Jiaotong University, China +5

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

Dongchen Si, Di Wang, Mingzhen Xu, Jing Zhang+2

School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Remote Sensing Image Understanding Efficient Multimodal Inference Multimodal Large Language Models Multimodal Classification

Open paper AskPDF arXiv

Remote sensing scene classification is a fundamental task in Earth observation and geospatial analysis. Existing approaches mainly follow three paradigms: task-specific visual classification, vision-language similarity matching, and autoregressive multimodal generation. However, visual classifiers rely on predefined label spaces, CLIP-based methods perform recognition through static image-text alignment, and multimodal large language models (MLLMs) introduce unnecessary token-level generation for classification tasks with explicit candidate categories. To address these limitations, we propose RSJEV, a one-pass multimodal decision framework for remote sensing scene classification. Unlike conventional MLLMs that formulate classification as autoregressive text generation, RSJEV reformulates scene classification as a candidate-conditioned multimodal discriminative decision process, where visual representations, task instructions, and candidate category semantics are jointly modeled. Specifically, we introduce a OnePass Decider that extracts multimodal decision states and directly estimates category probabilities within the candidate category space, eliminating autoregressive decoding while preserving vision-language interactions. Extensive experiments on three widely used remote sensing scene classification benchmarks, including UC Merced, AID, and NWPU-RESISC45, demonstrate that RSJEV achieves superior classification performance compared with representative CNN-, Transformer-, Mamba-, CLIP-, and MLLM-based methods. Moreover, RSJEV significantly reduces inference costs and achieves a better accuracy-efficiency trade-off with only a compact 0.8B-parameter model. These results demonstrate the effectiveness of state-conditioned multimodal decision making for efficient remote sensing image understanding. The code will be available at https://github.com/Dongtcs/RSJEV.

Momentum

Related topics

Latest papers 669