Efficient Multimodal Inference

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 211% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 157

All topics

Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models
Jul 26, 2026Yiming Zhong, Chang Nie, Caifeng ShanEfficient Multimodal Inference Multimodal Large Language Models
1Nanjing University
OmniScope: Modality-Decoupled Token Compression for Omnimodal Large Language Models
Jul 25, 2026Jinsen Su, Yongdong Luo, Yuexiao Ma +3Efficient Multimodal Inference Audio Token Compression
Media Analytics and Computing Lab, Xiamen University, Xiamen, China · Institute of Artificial Intelligence, Xiamen University, Xiamen, China · School of Informatics, Xiamen University, Xiamen, China +1
Out of Sight, Still in Mind: Token Compression for Omni-LLMs
Jul 23, 2026Suho Yoo, Youngjoon Jang, Hyebin Cho +1Efficient Multimodal Inference Multimodal Large Language Models
KAIST · VGG, University of Oxford
Beyond Independent Optimization: Compression, MoE Routing, and Quantization Interactions in Multimodal Edge Intelligence
Jul 23, 2026Jay Gor, Karm Dave, Akshita Abrol +3Efficient Multimodal Inference VLM Quantization
Nirma University · Singapore Institute of Technology, Singapore · Department of CE-AI and Big Data, Marwadi University
Look Less, Think Faster: Joint Token-Compute Adaptation for Multimodal LLMs
Jul 22, 2026Pengcheng Wang, Zhiquan Wang, Jayoung Lee +5Efficient Multimodal Inference Efficient VLM Inference
Purdue University, West Lafayette, IN, USA · University of Wisconsin–Madison, Madison, WI, USA · NVIDIA, Santa Clara, CA, USA
OmniReasoner: Thinking with Long Audio-Video via Native Tool Use
Jul 21, 2026Yu Chen, Caorui Li, Ziyu Xiong +8Temporal Video Grounding Audio-Visual Understanding
University of Chinese Academy of Sciences · Institute of Automation, CAS · Southeast University +3
FlashRT: Agent Harness for Guiding Agents to Deploy Real-Time Multimodal Applications
Jul 20, 2026Krish Agarwal, Zhuoming Chen, Yanyuan Qin +3Efficient Multimodal Inference LLM Inference Acceleration
Carnegie Mellon University · AMD · University at Buffalo
Stop Thinking, Start Looking: Efficient Post-Training for Multimodal Document Question Answering via Reasoning-Free Alignment
Jul 16, 2026Harikrishnan P M, Goutham Vignesh, Ganesh Parab +4Multimodal Grounding Efficient Multimodal Inference
Phi Labs, Quantiphi, India · Phi Labs, Quantiphi, Canada.
Do We Really Need Multimodal Emotion Language Models Larger Than 1B Parameters?
Jul 14, 2026Kaiwen Zheng, Junchen Fu, Wenhao Deng +3Efficient Multimodal Inference Multimodal Large Language Models
University of Glasgow United Kingdom · Institute of Computing Technology, Chinese Academy of Sciences China · School of Artificial Intelligence, Shandong University China +1
Full-Pipeline Inference Optimization for MiMo-V2.5 Series: Pushing Hybrid SWA Efficiency to the Limit
Jul 14, 2026Xiaomi MiMo Team, Anqi Liu, Aoxin Ma +28Efficient Multimodal Inference KV Caching
MiMo Team, Xiaomi
Beyond the Eye: Efficient Multimodal Reasoning via Self-Regulated Implicit Visual Tools
Jul 13, 2026Xiuwei Chen, Quanlin Chen, Wentao Hu +8Efficient Multimodal Inference Tool Use in VLMs
Sun Yat-sen University, China
Model Guides You How to Draw: Adaptive Visual Gating for Unified Multimodal Reasoning
Jul 10, 2026Wenxi Gao, Guanxi Lu, Didi Zhu +5Efficient Multimodal Inference Unified Multimodal Models
Imperial College London · Tsinghua University
Switch-Reasoner: Learn When to Think in Multitask Mixtures via Reinforcement Learning
Jul 9, 2026Yiyang Fang, Pei Fu, Jinjie Li +7Efficient Multimodal Inference RL for Language Model Reasoning
Wuhan University · The Hong Kong University of Science and Technology · Xiaomi Inc +2
DeltaV: Thinking with Visual State Updates in Unified Large Multimodal Models
Jul 9, 2026Pengjie Wang, Linger Deng, Zujia Zhang +6Efficient Multimodal Inference Unified Multimodal Models
Huazhong University of Science and Technology · MiLM Plus, Xiaomi Inc.
Hierarchical Multi-to-Single-Modal Knowledge Distillation for Disruption Prediction in EAST
Jul 5, 2026Qiang Chen, Xiao Wang, Hao Si +9Cross-Modal Knowledge Distillation Efficient Multimodal Inference
School of Computer Science and Technology, Anhui University, Hefei 230031, China · Institute of Plasma Physics, Chinese Academy of Sciences, Hefei 230031, China · State Key Laboratory of Opto-Electronic Information Acquisition and Protection Technology, Anhui University, Hefei 230039, China
InfraNet: Quality-Aware RGB Guidance for Efficient Infrared Object Detection
Jul 4, 2026Zichao Feng, Haodong Zhu, Jingying Yang +8Efficient Multimodal Inference Multimodal Robustness
Beihang University, Beijing, China · Zhongguancun Academy, Beijing, China · Communication University of China, Beijing, China +1
Modality Relevance is not Modality Utility: Post-hoc Selective Modality Escalation for Cost-Aware Multimodal RAG
Jul 3, 2026Xue Li, Yiming GaiMultimodal RAG Efficient Multimodal Inference
Hangzhou International Innovation Institute, Beihang University Hangzhou, China
OpenGlass: A Sensing-Computing Split Architecture for Local MLLM-Driven Real-Time Visual Assistance
Jul 3, 2026Mengzhang Li, Yuan YaoEfficient Multimodal Inference On-Device Inference
Shanghai Qizhi Institute · College of AI, Tsinghua University
OmniFocus: Query-Guided Modality-Balanced Token Compression for Omni-Modal Large Language Models
Jul 3, 2026Shijie Cao, Qingyu Zhang, Boxi Yu +6Efficient Multimodal Inference Multimodal Large Language Models
School of Advanced Interdisciplinary Sciences, University of Chinese Academy of Sciences · Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Limerick +1
Gemma 4 Technical Report
Jul 2, 2026Gemma Team, Sherif El Abd, Vaibhav Aggarwal +298Efficient Multimodal Inference Unified Multimodal Models
CoLT: Teaching Multi-Modal Models to Think with Chain of Latent Thoughts
Jun 30, 2026Lianyu Hu, Shengqian Qin, Zeqin Liao +4Efficient Multimodal Inference Efficient Language Model Reasoning
Nanyang Technological University, Singapore · Shanghai Jiao Tong University, China · NKIARI, Shenzhen Futian, China +2
Token-Sparse Medical Multimodal Reasoning via Dual-Stream Reinforcement Learning
Jun 30, 2026Kaitao Chen, Weiqian Zhao, Jiamin Wu +6Efficient Multimodal Inference Medical VQA
1Fudan University · 2Shanghai Artificial Intelligence Laboratory · 3Shanghai Jiao Tong University +1
HVPNet: A Bio-Inspired Network for General Salient and Camouflaged Object Detection
Jun 30, 2026Jiawei Xu, Qiangqiang Zhou, Zhouping Li +3Efficient Multimodal Inference Salient Object Detection
aSchool of Artificial Intelligence, Jiangxi Normal University, Street, Nanchang, 330000, State, China · bFaculty of Business Information, Shanghai Business School, Zhongshan West Road, Shanghai, 200235, China · cSchool of Computer Science and Information Engineering, Shanghai Institute of Technology, Street, Shanghai, 201418, China
Before Thinking, Learn to Decide: Proactive Routing for Efficient Visual Reasoning
Jun 29, 2026Yinan Zhou, Haokun Lin, Yichen Wu +7Efficient Multimodal Inference Adaptive Model Routing
Xi’an Jiaotong University · ARC Lab, Tencent IEG · City University of Hong Kong +3
DAIN: Dynamic Agent-Based Interaction Network for Efficient and Collaborative Multimodal Reasoning
Jun 29, 2026Xinxin Chen, Yuchen Li, Zihan Wang +3Efficient Multimodal Inference Multi-Agent Collaboration
University of Chinese Academy of Sciences
One Forward Beats Two: InnerZoom for Accurate and Efficient GUI Grounding
Jun 29, 2026Chen Liu, Ling Chen, Hanzhang Zhou +5Efficient Multimodal Inference GUI Grounding
University of Technology Sydney · Tongyi Lab, Alibaba Group · Adelaide University
Efficient Spatio-Temporal Grounding with Multimodal Large Models via Second-Level Tracking and RL Verification
Jun 27, 2026Tianshu Zhang, Yan Wang, Ji Qi +1Temporal Video Grounding Efficient Multimodal Inference
Tsinghua University
Position Rebinding Cache Reuse: Replay-Free Visual Revisiting for Interleaved Multimodal Reasoning
Jun 25, 2026Mengzhao Wang, Yanli Ji, Wangmeng Zuo +2Efficient Multimodal Inference KV Caching
Sun Yat-sen University · Shenzhen Loop Area Institute · Harbin Institute of Technology (HIT) +3
Methane-Plume Segmentation From Hyperspectral Satellite Imagery Via Multimodal Deep Learning
Jun 24, 2026Brayan Quintero, Jeferson Acevedo, Samuel Traslaviña +1Efficient Multimodal Inference Remote Sensing Image Segmentation
Department of Computer Science, Universidad Industrial de Santander Bucaramanga, 680002, Colombia
Latent Visual States for Efficient Multimodal Reasoning
Jun 23, 2026Xiuwei Chen, Wentao Hu, Yongxin Wang +8Efficient Multimodal Inference Large Vision-Language Models
Sun Yat-sen University · The Hong Kong Polytechnic University · MBZUAI +2

Jul 26, 2026, cs.CVJ/K move · Enter open · S save

Omni-Prune: Query-Aware Unified Token Pruning for Efficient Omnimodal Large Language Models

Yiming Zhong, Chang Nie, Caifeng Shan

1Nanjing University

Efficient Multimodal Inference Multimodal Large Language Models Token Pruning Multimodal Token Compression

Open paper AskPDF arXiv

Omnimodal large language models (OmniLLMs) are rapidly extending multimodal reasoning to cover synchronized audio and video. However, the resulting audio-video token sequences are long, leading to high prefill latency and GPU memory usage at inference time. Existing token pruning methods, designed mainly for vision-only inputs, miss both the cross-modal links between audio and video and the user query that decides which content matters. To bridge this gap, we present Omni-Prune, a training-free, query-aware audio-visual token pruning framework that jointly removes redundancy from both modalities while keeping task-relevant cross-modal evidence. Specifically, Omni-Prune first splits the token sequence into adaptive time windows placed at audio saliency peaks, then scores audio and video tokens on a single scale that combines encoder attention with text-query relevance, and pairs related audio-video tokens so that they are kept together. Within each window, a final K-medoids step then selects a few representative tokens, adding diverse cues that score-based selection alone would miss. Extensive experiments demonstrate that Omni-Prune outperforms established baseline methods, delivering up to 3.25x prefill speedup and 1.3x memory reduction while retaining over 99% of full-model performance.

Momentum

Related topics

Latest papers 157