Efficient Multimodal Inference

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 211% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 157

All topics

O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
May 26, 2026Peiran Wu, Yunze Liu, Chi-Hao Wu +2Audio-Visual Understanding Efficient Multimodal Inference
University of Bristol · Memories.ai Research · University of Central Florida
VEN-VL: A Visual Ensemble MoE Framework for Effective and Efficient Multi-Modal Understanding
May 25, 2026Yinghao Wu, Zhuoyan Luo, Yiyao Yu +3Efficient Multimodal Inference Mixture-of-Experts Models
Tsinghua University
Your Embedding Model is SMARTer Than You Think
May 24, 2026Jianrui Zhang, Hyun Jung Lee, Sukanta Ganguly +3Multimodal IR Efficient Multimodal Inference
UW-Madison · Korea University · NetApp, Inc.
Divide-and-Conquer Inference for Large-Scale Visual Recognition with Multimodal Large Language Models
May 24, 2026Zhipeng Ye, Jiaqi Huang, Feng Jiang +5Efficient Multimodal Inference Multimodal Large Language Models
Taizhou Institute of Science and Technology, Nanjing University of Science and Technology, Taizhou, 225300, Jiangsu, China · Department of Intelligence Science, Xi’an Jiaotong-Liverpool University, Suzhou, 215123, Jiangsu, China · Department of Statistical Sciences, University of Toronto, Toronto, M5S 1A1, Ontario, Canada +1
Tackling Multimodal Learning Challenges with Mixture-of-Expert: A Survey
May 22, 2026Liangwei Nathan Zheng, Wei Emma Zhang, Olaf Maennel +2Efficient Multimodal Inference Cross-Modal Representation Learning
Adelaide University
FusionSense: Tri-Stage Near-Sensor Learning for Runtime-Adaptive Multimodal Edge Intelligence
May 19, 2026Sanggeon Yun, Ryozo Masukawa, Minhyoung Na +5Efficient Multimodal Inference Edge Computing
University of California, Irvine Irvine, CA, USA · Kookmin University Seoul, South Korea · Shibaura Institute of Technology Saitama, Japan
Stage-adaptive Token Selection for Efficient Omni-modal LLMs
May 19, 2026Zijie Xin, Jie Yang, Ruixiang Zhao +4Efficient Multimodal Inference Multimodal Large Language Models
Renmin University of China · WeChat Vision, Tencent Inc.
AQuaUI: Visual Token Reduction for GUI Agents with Adaptive Quadtrees
May 19, 2026Yuankai Li, Tinghui Zhu, Ha Min Son +3Efficient Multimodal Inference GUI Agents
UC Davis
OmniSelect: Dynamic Modality-Aware Token Compression for Efficient Omni-modal Large Language Models
May 18, 2026Morunliu Yang, Ruotao Xu, Le Li +6Efficient Multimodal Inference Multimodal Large Language Models
1Soochow University · Peking University
TTE-Flash: Accelerating Reasoning-based Multimodal Representations via Think-Then-Embed Tokens
May 15, 2026Jianpeng Cheng, Xian Wu, Jiangfan Zhang +10Efficient Multimodal Inference Multimodal Embedding
1Meta AI
OmniDrop: Layer-wise Token Pruning for Omni-modal LLMs via Query-Guidance
May 14, 2026Yeo Jeong Park, Hyemi Jang, Minseo Choi +3Efficient Multimodal Inference Multimodal Large Language Models
Samsung Research
Think When Needed: Adaptive Reasoning-Driven Multimodal Embeddings with a Dual-LoRA Architecture
May 14, 2026Longxiang Zhang, Weilong Dai, Guanghao Zhang +2Efficient Multimodal Inference Multimodal Embedding
Alibaba Group
Learning to See What You Need: Gaze Attention for Multimodal Large Language Models
May 13, 2026Junha Song, Byeongho Heo, Geonmo Gu +3Efficient Multimodal Inference Visual Attention
1KAIST · 2NAVER AI Lab
G $^2$ TR: Generation-Guided Visual Token Reduction for Separate-Encoder Unified Multimodal Models
May 12, 2026Junxian Li, Kai Liu, Zizhong Ding +4Efficient Multimodal Inference Unified Multimodal Models
Shanghai Jiao Tong University · Huawei Technologies Ltd
OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
May 12, 2026Yuchen Deng, Zidang Cai, Hai-Tao Zheng +3Audio-Visual Understanding Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · Pengcheng Laboratory
Very Efficient Listwise Multimodal Reranking for Long Documents
May 12, 2026Yiqun Sun, Pengfei Wei, Lawrence B. HsiehMultimodal IR Efficient Multimodal Inference
Magellan Technology Research Institute (MTRI), Japan.
UniVLR: Unifying Text and Vision in Visual Latent Reasoning for Multimodal LLMs
May 12, 2026Houcheng Jiang, Jiajun Fu, Junfeng Fang +4Efficient Multimodal Inference Multimodal Large Language Models
University of Science and Technology of China · Zhongguancun Academy · National University of Singapore +1
Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs
May 12, 2026Chaeyoung Jung, Kyeongha Rho, Joon Son ChungAudio-Visual Understanding Efficient Multimodal Inference
Korea Advanced Institute of Science and Technology (KAIST)
CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation
May 12, 2026Daohan Su, Hao Liu, Xunkai Li +6Efficient Multimodal Inference Cross-Modal Alignment
LatentRouter: Can We Choose the Right Multimodal Model Before Seeing Its Answer?
May 11, 2026Xueqi Cheng, Yushun DongEfficient Multimodal Inference Multimodal Large Language Models
Department of Computer Science Florida State University University
EnergyLens: Interpretable Closed-Form Energy Models for Multimodal LLM Inference Serving
May 11, 2026Vittorio Palladino, Gianluca Palermo, Michael E. Papka +1Efficient Multimodal Inference LLM Inference
Politecnico di Milano & UIC · Politecnico di Milano · University of Illinois Chicago
ORICF -- Open Robotics Inference and Control Framework
May 10, 2026Andrés Meseguer Valenzuela, Luís Miguel Bartolín ArnauEfficient Multimodal Inference Edge Inference
Instituto Tecnológico de Informática (ITI), Paterna, Spain
HyperEyes: Dual-Grained Efficiency-Aware Reinforcement Learning for Parallel Multimodal Search Agents
May 8, 2026Guankai Li, Jiabin Chen, Yi Xu +2Efficient Multimodal Inference Multimodal Search Agents
Xiaohongshu Inc. · University of Cambridge
MiniCPM-o 4.5: Towards Real-Time Full-Duplex Omni-Modal Interaction
Apr 30, 2026Junbo Cui, Bokai Xu, Chongyi Wang +33Efficient Multimodal Inference Unified Multimodal Models
MiniCPM-o Team, OpenBMB
SWAN: World-Aware Adaptive Multimodal Networks for Runtime Variations
Apr 28, 2026Jason Wu, Shir-Kang Scott Jin, Yuyang Yuan +4Efficient Multimodal Inference Dynamic Neural Networks
University of California, Los Angeles · University of California, Riverside · U.S. Army DEVCOM Army Research Laboratory
Nemotron 3 Nano Omni: Efficient and Open Multimodal Intelligence
Apr 27, 2026NVIDIA, :, Amala Sanjay Deshmukh +216Efficient Multimodal Inference Multimodal Large Language Models
Focus Session: Hardware and Software Techniques for Accelerating Multimodal Foundation Models
Apr 23, 2026Muhammad Shafique, Abdul Basit, Muhammad Abdullah Hanif +3Model Compression Efficient Multimodal Inference
eBRAIN Lab, New York University (NYU) Abu Dhabi, Abu Dhabi, United Arab Emirates
MACS: Modality-Aware Capacity Scaling for Efficient Multimodal MoE Inference
Apr 19, 2026Bo Li, Chuan Wu, Shaolin ZhuEfficient Multimodal Inference Expert Load Balancing
School of Software, Tsinghua University, Beijing, China · TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · School of New Media and Communication, Tianjin University, China
EvoComp: Learning Visual Token Compression for Multimodal Large Language Models via Semantic-Guided Evolutionary Labeling
Apr 18, 2026Jiafei Song, Fengwei Zhou, Jin Qu +7Efficient Multimodal Inference Multimodal Large Language Models
OPPO CTG
Reducing Peak Memory Usage for Modern Multimodal Large Language Model Pipelines
Apr 17, 2026Junwan Kim, Hyunkyung BaeEfficient Multimodal Inference Multimodal Large Language Models
New York University
SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning
Mar 24, 2026Haoyu Huang, Jinfa Huang, Zhongwei Wan +3Efficient Multimodal Inference LLM Inference Acceleration
Xiamen University, Xiamen, Fujian, China · University of Rochester, Rochester, NY, USA · The Ohio State University, Columbus, OH, USA
DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression
Mar 15, 2026Bingzhou Li, Tao HuangAudio-Visual Understanding Efficient Multimodal Inference
Shanghai Jiao Tong University, Shanghai, China · Tongji University, Shanghai, China
HART: High-Resolution Annotation-Free Reasoning Technique through a Closed-loop Framework
Feb 27, 2026Jiacheng Yang, Anqi Chen, Yunkai Dang +5Efficient Multimodal Inference VLM Reasoning
State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, China · Institute of Brain-inspired Intelligence, Nanjing University, Nanjing, China · Shenzhen Research Institute of Nanjing University, Shenzhen, China
Magic-MM-Embedding: Towards Visual-Token-Efficient Universal Multimodal Embedding with MLLMs
Feb 5, 2026Qi Li, Yanzhe Zhao, Yongxin Zhou +4Efficient Multimodal Inference Multimodal Large Language Models
Honor Device Co., Ltd., China
EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation
Sep 15, 2025Jingyu Xiao, Zhongyi Zhang, Yuxuan Wan +3Efficient Multimodal Inference Code Generation
The Chinese University of Hong Kong, Hong Kong, China · Huazhong University of Science and Technology, Wuhan, China · Singapore Management University, Singapore, Singapore +1
Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation
Aug 7, 2025Haonan Shangguan, Xiaocui Yang, Shi Feng +3Efficient Multimodal Inference Sentiment Analysis
School of Computer Science and Engineering, Northeastern University, Shenyang, China
Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment
Jan 30, 2025Joanna Hong, Sanjeel Parekh, Honglie Chen +4Missing-Modality Learning Audio-Visual Understanding
Meta

May 26, 2026, cs.CVJ/K move · Enter open · S save

O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding

Peiran Wu, Yunze Liu, Chi-Hao Wu, Chen Chen+1

University of Bristol · Memories.ai Research · University of Central Florida

Audio-Visual Understanding Efficient Multimodal Inference Audio Understanding Multimodal QA Audio-Visual QA Multimodal Token Compression+4

Open paper AskPDF arXiv

Omnimodal large language models enable unified audio video understanding, but long joint token sequences make inference costly, and existing benchmarks do not fully isolate audio visual association in noisy user generated videos. We introduce UGC-AVQA, a public UGC benchmark with 1,000 videos and 4,816 QA pairs, where an audio removal test ensures that benchmark questions require both acoustic and visual evidence. To reduce inference cost, we propose OMAC, a training free plug in compression method that preserves salient visual memory and temporally grounded audio anchors. To further make compact models robust to compressed inputs, we introduce O-MARC, a compression distillation framework for learning with memory compressed multimodal contexts. On Qwen2.5-Omni-3B, O-MARC improves the average score across four benchmarks to 45.8, outperforming full token inference at 44.1 and OmniZip at 41.0. OMAC also keeps inference efficient, reducing latency by 34.6% (1.53

\times

speedup) and memory by 34.7% compared with full token inference.

Momentum

Related topics

Latest papers 157