Efficient Multimodal Inference

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 211% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 157

All topics

OmniKVQuant: KV Cache Quantization for Omni-LLMs
Sep 11, 2026Suho Yoo, Hyunjong Ok, Jongmin Choi +2Efficient Multimodal Inference KV Caching
KAIST · POSTECH
MoEMB: Scaling Universal Multimodal Embeddings with Efficient Mixture-of-Experts Models
Sep 8, 2026Xuanming Cui, Shlok Kumar Mishra, Wentao Bao +6Efficient Multimodal Inference Multimodal Embedding
University of Central Florida · Work done at Meta · AI at Meta
RAFM-SER++: A Lightweight Multimodal Emotion Recognition Framework for Real-Time Behavioral Monitoring in Surveillance Systems
Sep 7, 2026Ngo Truong Dinh, Tung-Lam Bui, Chi-Trung Duong +3Efficient Multimodal Inference Multimodal Emotion Recognition
Faculty of Information Technology University of Science, VNU-HCM Ho Chi Minh City, Vietnam
MCPO: Modality-Contrastive Preference Optimization for Multimodal Chain-of-Thought Compression
Sep 4, 2026Guangheng Yang, Zhenliang Ni, Zhenkai Wu +4Efficient Multimodal Inference Multimodal Large Language Models
Huawei Technologies Ltd. Shanghai, China
ShallowStream: Index Shallow then Answer Deep for Streaming Video Understanding
Sep 2, 2026Jitai Hao, Ke Yang, Di Yan +3Streaming Video Understanding Efficient Multimodal Inference
Harbin Institute of Technology (Shenzhen)
EM^2Mem: Event-Centric Multimodal Memory for Large Language Models
Sep 1, 2026Yijun Chen, Yaqi Zheng, Yanya Li +11Multimodal RAG Efficient Multimodal Inference
Zhejiang University · South China University of Technology · Lenovo Group Limited
VisLens: Single-Pass Interpretable Visual Search for Multimodal LLMs
Aug 31, 2026Jingyi He, Sanghwan Kim, Zeynep AkataEfficient Multimodal Inference Multimodal Large Language Models
Technical University of Munich · Helmholtz Munich · Munich Center for Machine Learning
Doc-REFRAG: Rethinking Multimodal Document Retrieval-Augmented Generation
Aug 31, 2026Ruofan Hu, Shengyang Xu, Minjie Hong +5Multimodal RAG Efficient Multimodal Inference
Zhejiang University
Accelerating Unified Multimodal Models with Core-Expansion Routing and Unified Computation Scheduling
Aug 29, 2026Wengyi Zhan, Chenqian Yan, Songwei Liu +2Efficient Multimodal Inference Unified Multimodal Models
1Xiamen University · 2ByteDance
Conditional Validity for Adaptive Modality Acquisition: When the Policy Chooses Its Own Calibration Group
Aug 16, 2026Melika BaghiEfficient Multimodal Inference Selective Prediction
Georgia Institute of Technology Atlanta, GA, USA
A Cloud-Edge System for Multimodal Clinical Screening in Resource-Constrained Rural Settings
Aug 13, 2026Hei Ting, Chan, Chenwei Wu +8Efficient Multimodal Inference Healthcare
Una · University of Michigan, Ann Arbor, MI, USA · Massachusetts Institute of Technology, Cambridge, MA, USA +1
Omni2LoRA: Coherence-Preserving Parametric Memory for Efficient Omni Language Models
Aug 10, 2026Puneet Mathur, Manan Suri, Dinesh ManochaEfficient Multimodal Inference Multimodal Memory
University of Maryland, College Park
Deferred Audio Pruning with Local Audio-Visual Dynamics for Omni-LLMs
Aug 9, 2026Kyeongyoon Lee, Hongyeob Kim, Youngeun Kim +1Efficient Multimodal Inference Audio Token Compression
Sungkyunkwan University · AWS AI Labs
Aero Realtime: Fully Aligned Input-Output Streams for Low-Latency Streaming Multimodal Generation
Aug 9, 2026Kaichen Zhang, Wei Huang, Keming Wu +2Efficient Multimodal Inference Multimodal Large Language Models
1The University of Hong Kong · 2LMMs-Lab · 3Tsinghua University
EvtGraph: Event-Adaptive Compression for Sparse Temporal Graph Learning in Multimodal Time Series
Aug 5, 2026Ziqian Wang, Tingxiong Xiao, Yuxiao Cheng +1Irregular Time-Series Modeling Efficient Multimodal Inference
Department of Automation Tsinghua University Beijing, China
OmniPack: Unified Token Compression for Efficient Omni-modal Large Language Models
Aug 4, 2026Wanshun Su, Yang Shi, Feihu Liu +10Efficient Multimodal Inference Audio Token Compression
Northwestern Polytechnical University · Peking University · Alibaba Group +1
Compass: Degradation-Simulated Reciprocal Learning with Lightweight Needle RWKV for Multimodal Crack Segmentation under Missing Modalities
Aug 4, 2026Hui Liu, Chen Jia, Fan Shi +3Missing-Modality Learning Efficient Multimodal Inference
Engineering Research Center of Learning-Based Intelligent System (Ministry of Education) Tianjin University of Technology Tianjin, China
Balancing Efficiency and Efficacy: Training-Free Attention-Guided Switching Between Explicit and Latent Thoughts for MLLMs
Aug 4, 2026Haoqian Kang, Liupeng Li, Kuofeng Gao +5Efficient Multimodal Inference Multimodal Large Language Models
Harbin Institute of Technology, Shenzhen Shenzhen, China · Tsinghua Shenzhen International Graduate School, Tsinghua University Shenzhen, China · Peng Cheng Laboratory Shenzhen, China +1
Better, Stronger, Faster, and Broader: Structured All-Mask Prediction for MLLM-Based Segmentation
Aug 3, 2026Jiazhen Liu, Mingkuan Feng, Long ChenEfficient Multimodal Inference Image Segmentation
The Hong Kong University of Science and Technology (HKUST), Hong Kong.
ET-Prune: Evidence-Aware Dynamic Budgeting for Visual Token Pruning in Text-Rich MLLMs
Aug 3, 2026Zizhong Ding, Junxian Li, Kai Liu +4Efficient Multimodal Inference Multimodal Large Language Models
Shanghai Jiao Tong University · Xidian University
Towards Compact Unified Multimodal Tracking: Synergizing Knowledge Distillation with Structural Pruning
Aug 2, 2026Yuqi Li, Yuedong Tan, Huiran Duan +7Efficient Multimodal Inference Unified Multimodal Models
The City College of New York, New York, NY, USA · INSAIT, Sofia University “St. Kliment Ohridski”, Sofia, Bulgaria · Julius-Maximilians-Universität Würzburg, Würzburg, Germany +3
CodeShrink: Adaptive Visual Compression for Efficient Multimodal Code Understanding
Jul 31, 2026Wenxin Tang, Jingyu Xiao, Zhenyu Liu +6Efficient Multimodal Inference Multimodal Large Language Models
Tsinghua University · The Chinese University of Hong Kong · Northwestern Polytechnical University +4
ReLoop-UME: Recurrent Depth with Learnable Retrieval Registers for Universal Multimodal Embedding
Jul 30, 2026Shijie Wang, Xiangzhao Hao, Yueti Li +3Efficient Multimodal Inference Multimodal Embedding
Institute of Automation, Chinese Academy of Sciences, Beijing, China · Tsinghua University, Beijing, China · AI Platform, evermind
Capturing Token Tendencies for Training-Free Token Pruning in Multimodal Large Language Models
Jul 30, 2026Jie Ma, Zhike Qiu, Jie Gao +4Efficient Multimodal Inference Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University · School of Information Engineering, Xiamen Ocean Vocational College · Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, Sino-Russian Research Center for Digital Economy
A Cost-Effective Multimodal LLM Reasoning Framework for Question Answering over Irregular Clinical Time Series
Jul 28, 2026Frank Nie, Ethan B Liu, Yuan Zhu +2Irregular Time-Series Modeling Efficient Multimodal Inference
Shandong University China · University of Auckland New Zealand
SepPrune:A Separator-based Pruning Framework for Efficient Multimodal Large Language Models
Jul 28, 2026Yuchen Wang, Qihui Zhu, Yang Liu +2Efficient Multimodal Inference Multimodal Large Language Models
MoE Key Lab of BIPC, NEL-BITA, University of Science and Technology of China, Hefei, China · ChangXin Memory Technologies, Hefei, China · APKL of BIIP, IAI, Hefei Comprehensive National Science Center, Hefei, China
OmniDelta: Skill-Driven Budget Allocation for Token Compression in OmniLLMs
Jul 28, 2026Haoyang Huang, Wenjie Huang, Tianqi Xu +14Efficient Multimodal Inference Audio Token Compression
Zhejiang University · Qwen Application, Alibaba · Carnegie Mellon University +2
Salient Knowledge Pathways: Sparse Cross-Modal Routing for Efficient Knowledge-Intensive Multimodal Question Answering
Jul 28, 2026Noor Islam S. Mohammad, Uluğ BayazıtMultimodal RAG Efficient Multimodal Inference
Department of Computer Science, İTÜ, İstanbul, Türkiye · Faculty of Computer Engineering, İTÜ, İstanbul, Türkiye.
FORGE: Frame Orthogonality in Relevance Geometry for Long-Form Video Understanding
Jul 28, 2026Ghazal Kaviani, Ghassan AlRegibEfficient Multimodal Inference Multimodal Large Language Models
1OLIVES at the Center for Signal and Information Processing (CSIP), School of Electrical and Computer Engineering, Georgia Institute of Technology, Atlanta, GA, USA.
DeCoRAG: Cognitive Decoupling and Semantic-Aware Cropping for Complex Document Understanding
Jul 27, 2026Shuo Wang, Kai Zhang, Wenyuan Huang +5Document Understanding Multimodal RAG
QiYuanLab, Beijing, China · Beijing University of Posts and Telecommunications, Beijing, China

Sep 11, 2026, cs.CVJ/K move · Enter open · S save

OmniKVQuant: KV Cache Quantization for Omni-LLMs

Suho Yoo, Hyunjong Ok, Jongmin Choi, Jihoo Jung+1

KAIST · POSTECH

Efficient Multimodal Inference KV Caching Multimodal Large Language Models LLM Inference Acceleration KV-Cache Quantization Post-Training Quantization

Open paper AskPDF arXiv

As Omni-modal large language models (Omni-LLMs) take in audio, video and text together, their KV cache memory cost grows. KV cache quantization is the de facto approach in text-only LLMs, but its application to Omni-LLMs remains unexplored. In this paper, we analyze how TurboQuant, a representative rotation-based KV cache quantization method, behaves on multimodal caches and identify two critical issues: temporal key drift and heterogeneous value geometry. To address these, we propose OmniKVQuant, a training-free framework that (i) sets the key quantization range over each short window of the input stream; and (ii) rotates values separately per modality. On Qwen2.5-Omni and Qwen3-Omni, OmniKVQuant enables 2-bit KV caches while substantially preserving performance across seven audio-visual benchmarks. We further provide a fused Triton decode kernel that unpacks the 2-bit cache during attention, so no dense FP16 cache is ever built. Code: https://github.com/kaistmm/OmniKVQuant

Momentum

Related topics

Latest papers 157