Efficient Multimodal Inference

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 211% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 156

All topics

RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models
Oct 6, 2026Dongchen Si, Di Wang, Mingzhen Xu +3Remote Sensing Image Understanding Efficient Multimodal Inference
School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China
VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs
Oct 6, 2026Yuan Feng, Qize Yang, Ruizhe Chen +9Efficient Multimodal Inference Multimodal Large Language Models
University of Science and Technology of China · Alibaba Token Hub, Alibaba Group · The Chinese University of Hong Kong +1
Decide Before You Look: Learning Which Retrieved Memories Deserve Pixels
Oct 6, 2026Youxing LIMemory-Augmented VLMs Efficient Multimodal Inference
The Chinese University of Hong Kong
Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion
Oct 5, 2026Payal Mohapatra, Haodong Yang, Yueyuan Sui +3Efficient Multimodal Inference Multimodal Classification
Arm Inc., USA · Northwestern University, USA
Conditional Trajectory Peaks: Single-Pass Multimodal Policies over Action Chunks
Oct 5, 2026Di Wu, Rongtian Shen, Ping Liu +4Efficient Multimodal Inference Robot Policy Learning
Magiclab Robotics Technology Co., Ltd., China · Southeast University, Nanjing, China
ReMAP: Restoring the Perceptual Cycle with Reasoning-Time Latent Visual Memory
Oct 4, 2026Hao Jiang, Zhanyu Guo, Chenwei Wu +11Memory-Augmented VLMs Efficient Multimodal Inference
University of Electronic Science and Technology of China · University of Michigan, Ann Arbor · Nanyang Technological University +5
Selection-Based Structured Reasoning: Toward Efficient Multimodal Search Agents
Oct 1, 2026Feiyu Gavin Zhu, Xiaoyu Zhu, Jiqi Yang +9Efficient Multimodal Inference Multimodal Reasoning
Apple · Carnegie Mellon University
HAWK: Rethinking Multimodal Drafting for Speculative Decoding
Sep 30, 2026Wenhan Yang, Anirudh Rao, Ashwin ChandraEfficient Multimodal Inference VLM Distillation
University of California, Los Angeles · Samsung Research America
Learning to Reason with Compressed Context: Ground-Truth-Free Adaptation of OmniLLMs via Self-Distillation
Sep 30, 2026Jianghao Wang, Ke Meng, Jian Li +7Efficient Multimodal Inference Multimodal Large Language Models
School of Computer Science, Wuhan University · Didi Chuxing
DecoMoE: Decoupling Visual Propagation and Expert Computation for Efficient Multimodal MoE Inference
Sep 30, 2026Xudong Tan, Peng Ye, Ming Xie +4Mixture-of-Experts Pruning Efficient Multimodal Inference
Fudan University · The Chinese University of Hong Kong
Task-Oriented Visual Feature Compression via Residual Vector Quantization for Device-Edge Multimodal Inference
Sep 29, 2026Luning Pang, Cheng Yuan, Jiawei Shao +2Efficient Multimodal Inference Visual Token Compression
Department of Electronic Engineering, and Beijing National Research Center for Information Science and Technology, Tsinghua University, Beijing 100084, China · Institute of Artificial Intelligence (TeleAI), China Telecom, Beijing 100033, China
OmniRoute: Mapping Temporal Semantic Evidence to Audio-Visual Token Budgets for Efficient Omnimodal Large Language Models
Sep 29, 2026Yuchen Deng, Zidang Cai, Feidiao Yang +4Efficient Multimodal Inference Audio Token Compression
Shenzhen International Graduate School, Tsinghua University, China · Pengcheng Laboratory, China
SPIDER: Multi-Layer Semantic Token Pruning and Adaptive Sub-Layer Skipping in Multimodal Large Language Models
Sep 28, 2026Tianxiang Chen, Zhentao Tan, Zi Ye +9Efficient Multimodal Inference Transformer
End-User Intelligent Computing BU, Alibaba Cloud · Alibaba Group · Department of Computer Science, Maynooth University +2
Just MLPs: Efficient Visual State Reconstruction for Multimodal Language Models
Sep 28, 2026Jingdi lei, Junxian Li, Di Zhang +3Efficient Multimodal Inference Multimodal Large Language Models
Nanyang Technological University · Shanghai Jiao Tong University · Fudan University +2
Resolution as a First-Class Decision: Task-Conditioned Routing for Efficient Multimodal Large Language Models
Sep 28, 2026Zhiqiang Xia, Yang Li, Xinyuan Zhang +5Efficient Multimodal Inference Efficient VLM Inference
Xiaomi Corporation, Beijing, China
OmniTide: Co-Designing Algorithms and Systems for Efficient On-Device Omni-LLM Streaming
Sep 28, 2026Zongshang Shen, Wangsong Yin, Daliang Xu +2Efficient Multimodal Inference On-Device Language Model Inference
Key Lab of High Confidence Software Technologies (Peking University), Beijing, China · State Key Laboratory of Networking and Switching Technology (BUPT), Beijing, China
MiCo: Mutual Information Coverage Optimization through Semantic Erasure Modeling for Efficient MLLM Inference
Sep 28, 2026Tinghao Wang, Yichen Guo, Qizhe Zhang +11Efficient Multimodal Inference Multimodal Large Language Models
State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University · University of Electronic Science and Technology of China · Nanyang Technological University +2
MAD-Guard: Controlled Study of Autoregressive Generation versus Direct Decision Interfaces for Closed Multimodal Forensic Tasks
Sep 27, 2026Hao ChenEfficient Multimodal Inference Image Forgery Detection
School of Cyber Security, Guangdong Polytechnic Normal University, Guangzhou, China
GraphSelect for Budgeted Representation Selection in Multimodal Graph Inference
Sep 27, 2026Xu Wang, Xunkai Li, Yinlin Zhu +1Efficient Multimodal Inference Multimodal Classification
School of Airspace Science and Engineering, Shandong University, Weihai, China · Department of Computer Science, Beijing Institute of Technology, Beijing, China · School of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
VaME: Exploring Variational Latent Reasoning for Multimodal Embeddings
Sep 27, 2026Peixi Wu, Mingzhou Jiang, Feipeng Ma +11Multimodal IR Efficient Multimodal Inference
University of Science and Technology of China · Tsinghua University · Kuaishou +1
PARSEE-VAD: Efficient Training-Free Online Video Anomaly Detection via Proposition-Aware Reasoning and Streaming Evidence Escalation
Sep 27, 2026Ji Wang, Shuangqing Zhang, Guo-Sen Xie +1Efficient Multimodal Inference Streaming Video Understanding
Global Institute of Future Technology Shanghai Jiao Tong University Shanghai, 200240, China · School of Intelligence Science and Technology Nanjing University Suzhou, 215163, China · School of Computer Science and Engineering Nanjing University of Science and Technology Nanjing, 210094, China
SLICEChat: Progressive In-Encoder Token Pruning for Whole-Slide Pathology Language Models
Sep 21, 2026Ali Kerem Bozkurt, Baris Cem Bakay, Ibrahim Kulac +3Efficient Multimodal Inference Multimodal Large Language Models
Kocı University, Department of Computer Engineering · Kocı University, Department of Pathology · Hacettepe University, Department of Artificial Intelligence and Data Engineering
Layer-Aware Position Embeddings for Visual Token Pruning in Multimodal Large Language Models
Sep 20, 2026Yahong Wang, Zhangkai Ni, Juncheng Wu +3Efficient Multimodal Inference Multimodal Large Language Models
Tongji University · University of California, Santa Cruz · East China Normal University +1
Efficient Unified Multimodal Understanding (EUMU): Winning Solution for the MUMU Track at the 8th LSVOS Challenge
Sep 16, 2026Dayoung Kil, Seong-heum KimOpen-Vocabulary Object Detection Efficient Multimodal Inference
Department of Intelligent Semiconductors, Soongsil University, Seoul, Republic of Korea
From Models to Systems: A Comprehensive Survey of Efficient Multimodal Learning
Sep 16, 2026Pan Wang, Siwei Song, Hui Ji +12Efficient Multimodal Inference Multimodal Optimization
University of Pittsburgh · New York University · Carnegie Mellon University +6
RankGround: Efficient High-Resolution GUI Grounding via Lightweight Reranker-Guided Crop Selection
Sep 16, 2026Liyang Fan, Xinping Bi, Yitai Li +3Efficient Multimodal Inference Multimodal Reranking
College of Computer Science and Software Engineering Shenzhen University Shenzhen, China · Shenzhen Key Laboratory for High Performance Data Mining Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences Shenzhen, China · Shenzhen University of Advanced Technology Shenzhen, China
Reason What Matters: Retrieval-Grounded Reasoning for Universal Multimodal Embeddings
Sep 14, 2026Mingzhou Jiang, Peixi Wu, Hang Cheng +7Multimodal IR Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · School of Artificial Intelligence and Data Science, USTC · Kuaishou Technology +1
AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference
Sep 14, 2026Yuyao Sun, Tao Deng, Shuang Li +1Efficient Multimodal Inference Multimodal Large Language Models
Beihang University
LGFN: Lightweight Gated RGB-Polarization Fusion with Modality-Availability Conditioning for Camouflaged Object Detection
Sep 14, 2026Zhuangfan Huang, Xiaosong Li, Yang Liu +2Missing-Modality Learning Efficient Multimodal Inference
Guangdong-HongKong-Macao Joint Laboratory for Intelligent Micro-Nano Optoelectronic Technology, School of Physics and Optoelectronic Engineering, Foshan University, Foshan 528225, China. · School of Mechanical and Electrical Engineering, China University of Mining and Technology-Beijing, Beijing 100083, China.
OmniKVQuant: KV Cache Quantization for Omni-LLMs
Sep 11, 2026Suho Yoo, Hyunjong Ok, Jongmin Choi +2Efficient Multimodal Inference KV Caching
KAIST · POSTECH
EMMI: Edge Multi-Modal Intelligence for Communication-Efficient MLLM Inference via Fused Representation Compression
Sep 11, 2026Motahare Mounesan, Irfan KhanEfficient Multimodal Inference Edge Computing
Texas A&M University
MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
Sep 8, 2026Xuanming Cui, Shlok Kumar Mishra, Wentao Bao +6Efficient Multimodal Inference Multimodal Embedding
University of Central Florida · Work done at Meta · AI at Meta
RAFM-SER++: A Lightweight Multimodal Emotion Recognition Framework for Real-Time Behavioral Monitoring in Surveillance Systems
Sep 7, 2026Ngo Truong Dinh, Tung-Lam Bui, Chi-Trung Duong +3Efficient Multimodal Inference Multimodal Emotion Recognition
Faculty of Information Technology University of Science, VNU-HCM Ho Chi Minh City, Vietnam
MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression
Sep 4, 2026Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4Efficient Multimodal Inference Multimodal Large Language Models
Huawei Technologies Ltd. Shanghai, China
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
Sep 2, 2026Jitai Hao, Ke Yang, Di Yan +3Streaming Video Understanding Efficient Multimodal Inference
Harbin Institute of Technology (Shenzhen)
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
Sep 1, 2026Yijun Chen, Yaqi Zheng, Yanya Li +11Multimodal RAG Efficient Multimodal Inference
Zhejiang University · South China University of Technology · Lenovo Group Limited
VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs
Aug 31, 2026Jingyi He, Sanghwan Kim, Zeynep AkataEfficient Multimodal Inference Multimodal Large Language Models
Technical University of Munich · Helmholtz Munich · Munich Center for Machine Learning
Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation
Aug 31, 2026Ruofan Hu, Shengyang Xu, Minjie Hong +5Multimodal RAG Efficient Multimodal Inference
Zhejiang University
Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling
Aug 29, 2026Wengyi Zhan, Chenqian Yan, Songwei Liu +2Efficient Multimodal Inference Unified Multimodal Models
1Xiamen University · 2ByteDance
Conditional Validity for Adaptive Modality Acquisition: When the Policy Chooses Its Own Calibration Group
Aug 16, 2026Melika BaghiEfficient Multimodal Inference Selective Prediction
Georgia Institute of Technology Atlanta, GA, USA
A Cloud-Edge System for Multimodal Clinical Screening in Resource-Constrained Rural Settings
Aug 13, 2026Hei Ting, Chan, Chenwei Wu +8Efficient Multimodal Inference Healthcare
Una · University of Michigan, Ann Arbor, MI, USA · Massachusetts Institute of Technology, Cambridge, MA, USA +1
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Aug 10, 2026Puneet Mathur, Manan Suri, Dinesh ManochaEfficient Multimodal Inference Multimodal Memory
University of Maryland, College Park
Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs
Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal Inference Audio Token Compression
Sungkyunkwan University · AWS AI Labs
Aero Realtime: Fully Aligned Input-Output Streams for Low-Latency Streaming Multimodal Generation
Aug 9, 2026Kaichen Zhang, Wei Huang, Keming Wu +2Efficient Multimodal Inference Multimodal Large Language Models
1The University of Hong Kong · 2LMMs-Lab · 3Tsinghua University
EvtGraph: Event-Adaptive Compression for Sparse Temporal Graph Learning in Multimodal Time Series
Aug 5, 2026Ziqian Wang, Tingxiong Xiao, Yuxiao Cheng +1Irregular Time-Series Modeling Efficient Multimodal Inference
Department of Automation Tsinghua University Beijing, China
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities
Aug 4, 2026Hui Liu, Chen Jia, Fan Shi +3Missing-Modality Learning Efficient Multimodal Inference
Engineering Research Center of Learning-Based Intelligent System (Ministry of Education) Tianjin University of Technology Tianjin, China
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Aug 4, 2026Haoqian Kang, Liupeng Li, Kuofeng Gao +5Efficient Multimodal Inference Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen Shenzhen, China · Tsinghua Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Peng Cheng Laboratory Shenzhen, China +1
Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation
Aug 3, 2026Jiazhen Liu, Mingkuan Feng, Long ChenEfficient Multimodal Inference Image Segmentation
The Hong Kong University of Science and Technology (HKUST), Hong Kong.
ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs
Aug 3, 2026Zizhong Ding, Junxian Li, Kai Liu +4Efficient Multimodal Inference Multimodal Large Language Models
Shanghai Jiao Tong University · Xidian University
Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning
Aug 2, 2026Yuqi Li, Yuedong Tan, Huiran Duan +7Efficient Multimodal Inference Unified Multimodal Models
The City College of New York, New York, NY, USA · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria · Julius-Maximilians-Universität Würzburg, Würzburg, Germany +3
CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
Jul 31, 2026Wenxin Tang, Jingyu Xiao, Zhenyu Liu +6Efficient Multimodal Inference Multimodal Large Language Models
Tsinghua University · The Chinese University of Hong Kong · Northwestern Polytechnical University +4
ReLoop-UME: Recurrent Depth with Learnable Retrieval Registers for Universal Multimodal Embedding
Jul 30, 2026Shijie Wang, Xiangzhao Hao, Yueti Li +3Efficient Multimodal Inference Multimodal Embedding
Institute of Automation, Chinese Academy of Sciences, Beijing, China · Tsinghua University, Beijing, China · AI Platform, evermind
Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models
Jul 30, 2026Jie Ma, Zhike Qiu, Jie Gao +4Efficient Multimodal Inference Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · School of Information Engineering, Xiamen Ocean Vocational College · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Sino-Russian Research Center for Digital Economy
A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series
Jul 28, 2026Frank Nie, Ethan B Liu, Yuan Zhu +2Irregular Time-Series Modeling Efficient Multimodal Inference
Shandong University China · University of Auckland New Zealand
SepPrune:A Separator-based Pruning Framework for Efficient Multimodal Large Language Models
Jul 28, 2026Yuchen Wang, Qihui Zhu, Yang Liu +2Efficient Multimodal Inference Multimodal Large Language Models
MoE Key Lab of BIPC, NEL-BITA, University of Science and Technology of China, Hefei, China · ChangXin Memory Technologies, Hefei, China · APKL of BIIP, IAI, Hefei Comprehensive National Science Center, Hefei, China
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Jul 28, 2026Haoyang Huang, Wenjie Huang, Tianqi Xu +14Efficient Multimodal Inference Audio Token Compression
Zhejiang University · Qwen Application, Alibaba · Carnegie Mellon University +2
Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering
Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal RAG Efficient Multimodal Inference
Department of Computer Science, İTÜ, İstanbul, Türkiye · Faculty of Computer Engineering, İTÜ, İstanbul, Türkiye.
FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understanding
Jul 28, 2026Ghazal Kaviani, Ghassan AlRegibEfficient Multimodal Inference Multimodal Large Language Models
1OLIVES at the Center for Signal and Information Processing (CSIP), School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA.
DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
Jul 27, 2026Shuo Wang, Kai Zhang, Wenyuan Huang +5Document Understanding Multimodal RAG
QiYuanLab, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China

Oct 6, 2026, cs.CVJ/K move · Enter open · S save

RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

Dongchen Si, Di Wang, Mingzhen Xu, Jing Zhang+2

School of Computer Science, Wuhan University, Wuhan 430072, China · State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, Wuhan 430079, China

Remote Sensing Image Understanding Efficient Multimodal Inference Multimodal Large Language Models Multimodal Classification

Open paper AskPDF arXiv

Remote sensing scene classification is a fundamental task in Earth observation and geospatial analysis. Existing approaches mainly follow three paradigms: task-specific visual classification, vision-language similarity matching, and autoregressive multimodal generation. However, visual classifiers rely on predefined label spaces, CLIP-based methods perform recognition through static image-text alignment, and multimodal large language models (MLLMs) introduce unnecessary token-level generation for classification tasks with explicit candidate categories. To address these limitations, we propose RSJEV, a one-pass multimodal decision framework for remote sensing scene classification. Unlike conventional MLLMs that formulate classification as autoregressive text generation, RSJEV reformulates scene classification as a candidate-conditioned multimodal discriminative decision process, where visual representations, task instructions, and candidate category semantics are jointly modeled. Specifically, we introduce a OnePass Decider that extracts multimodal decision states and directly estimates category probabilities within the candidate category space, eliminating autoregressive decoding while preserving vision-language interactions. Extensive experiments on three widely used remote sensing scene classification benchmarks, including UC Merced, AID, and NWPU-RESISC45, demonstrate that RSJEV achieves superior classification performance compared with representative CNN-, Transformer-, Mamba-, CLIP-, and MLLM-based methods. Moreover, RSJEV significantly reduces inference costs and achieves a better accuracy-efficiency trade-off with only a compact 0.8B-parameter model. These results demonstrate the effectiveness of state-conditioned multimodal decision making for efficient remote sensing image understanding. The code will be available at https://github.com/Dongtcs/RSJEV.

Momentum

Related topics

Latest papers 156