Efficient Multimodal Inference

Ask about this topic

Overview

Efficient multimodal inference studies how to make models process mixed inputs with less computation, memory, latency, and energy. Common approaches prune or compress visual and audio tokens, select useful video frames or input modalities, compress KV caches, and adapt reasoning depth to query difficulty. Systems work also schedules inference across heterogeneous GPUs and edge devices, with evaluations spanning image and video question answering, long-context tasks, and real-time applications. Many methods preserve accuracy at substantial compression, but an open tension is whether they retain fine-grained evidence and grounding across tasks while improving end-to-end latency and cost.

Latest papers

CardsList

231 papers27 papers in the last four weeks, down 4% on the four weeks before. 0.2% of all new papers.

  1. Sense Less, Infer More: Agentic Multimodal Transformers for Edge Medical Intelligence

    Apr 12, 2026Chengwei Zhou, Zhaoyan Jia, Haotian Yu +6Efficient Multimodal InferenceWearable Health Monitoring

    Case Western Reserve University, USA · University of Southern California, USA

  2. Modality-Aware Zero-Shot Pruning and Sparse Attention for Efficient Multimodal Edge Inference

    Apr 10, 2026Yueyuan Sui, Payal Mohapatra, Doğaç Eldenk +5Efficient Multimodal InferenceNeural Network Pruning

    Northwestern University, Evanston, Illinois, USA

  3. Tree-of-Evidence: Efficient "System 2" Search for Faithful Multimodal Grounding

    Apr 9, 2026Micky C. Nnamdi, Benoit L. Marteau, Yishan Zhong +2Multimodal Model InterpretabilityMultimodal Grounding

    Georgia Institute of Technology

  4. MMEmb-R1: Reasoning-Enhanced Multimodal Embedding with Pair-Aware Selection and Adaptive Control

    Apr 7, 2026Yuchi Wang, Dingkang Yang, Haiyang Yu +5Multimodal EmbeddingMultimodal Reasoning

    MMLab, The Chinese University of Hong Kong · ByteDance Inc. · Fudan University

  5. HybridKV: Hybrid KV Cache Compression for Efficient Multimodal Large Language Model Inference

    Apr 7, 2026Bowen Zeng, Feiyang Ren, Jun Zhang +4KV-Cache CompressionEfficient Multimodal Inference

    The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security · Hangzhou Dianzi University, Hangzhou, China

  6. PanLUNA: An Efficient and Robust Query-Unified Multimodal Model for Edge Biosignal Intelligence

    Apr 5, 2026Marija Zelic, Anna Tegon, Yawei Li +2Multimodal Foundation ModelsMultimodal Sensor Fusion

    Integrated Systems Laboratory, ETH Zürich, Zürich, Switzerland · DEI, University of Bologna, Bologna, Italy

  7. TABQAWORLD: Optimizing Multimodal Reasoning for Multi-Turn Table Question Answering

    Apr 3, 2026Tung Sum Thomas Kwok, Xinyu Wang, Xiaofeng Lin +7Table ReasoningTable QA

    University of California, Los Angeles · SimpleWay · McGill University +2

  8. MI-Pruner: Crossmodal Mutual Information-guided Token Pruner for Efficient MLLMs

    Apr 3, 2026Jiameng Li, Aleksei Tiulpin, Matthew B. BlaschkoEfficient Multimodal InferenceVisual Token Pruning

    ESAT-PSI, KU Leuven, Belgium · Faculty of Medicine, University of Oulu, Finland · Weill Cornell Medicine, USA

  9. Token-Efficient Multimodal Reasoning via Image Prompt Packaging

    Apr 2, 2026Joong Ho Choi, Jiayang Zhao, Avani Appalla +3Efficient Multimodal InferenceMultimodal Token Compression

    BNY, Pittsburgh, US

  10. PLUME: Latent Reasoning Based Universal Multimodal Embedding

    Apr 2, 2026Chenwei He, Xiangzhao Hao, Tianyu Yang +6Multimodal EmbeddingMultimodal IR

    Southeast University · Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences

  11. ResAdapt: Adaptive Resolution for Efficient Multimodal Reasoning

    Mar 30, 2026Huanxuan Liao, Zhongtao Jiang, Yupu Hao +6Efficient Multimodal InferenceMultimodal Large Language Models

    Institute of Automation, Chinese Academy of Sciences · University of Chinese Academy of Sciences

  12. Rocks, Pebbles and Sand: Modality-aware Scheduling for Multimodal Large Language Model Inference

    Mar 27, 2026Konstantinos Papaioannou, Thaleia Dimitra DoudaliLLM ServingLLM Inference Scheduling

    IMDEA Software Institute, Universidad Politécnica de Madrid

  13. Photon: Speedup Volume Understanding with Efficient Multimodal Large Language Models

    Mar 26, 2026Chengyu Fang, Heng Guo, Zheng Jiang +3Medical VLMs3D VLMs

    Tsinghua University · DAMO Academy, Alibaba Group · Hupan Lab +1

  14. NeuroVLM-Bench: Evaluation of Vision-Enabled Large Language Models for Clinical Reasoning in Neurological Disorders

    Mar 25, 2026Katarina Trojachanec Dineva, Stefan Andonov, Ilinka Ivanoska +5Medical VLMsMultimodal Model Evaluation

    Faculty of computer science and engineering, Ss. Cyril and Methodius University, Skopje, North Macedonia.

  15. Comparative analysis of dual-form networks for live land monitoring using multi-modal satellite image time series

    Mar 25, 2026Iris Dumeur, Jérémy Anger, Gabriele FaccioloEfficient Multimodal InferenceLinear Attention

    CB · Kayrros SAS · Université Paris-Saclay, ENS Paris-Saclay, CNRS, Centre Borelli, 91190, Gif-sur-Yvette, France +1

  16. Brain-Inspired Multimodal Spiking Neural Network for Image-Text Retrieval

    Mar 25, 2026Xintao Zong, Xian Zhong, Wenxuan Liu +3Image-Text RetrievalSpiking Neural Networks

    Hubei Key Laboratory of Transportation Internet of Things, Wuhan University of Technology · State Key Laboratory for Multimedia Information Processing, Peking University

  17. SpecEyes: Accelerating Agentic Multimodal LLMs via Speculative Perception and Planning

    Mar 24, 2026Haoyu Huang, Jinfa Huang, Zhongwei Wan +3Efficient Multimodal InferenceLLM Inference Acceleration

    Xiamen University, Xiamen, Fujian, China · University of Rochester, Rochester, NY, USA · The Ohio State University, Columbus, OH, USA

  18. TRINE: A Token-Aware, Runtime-Adaptive FPGA Inference Engine for Multimodal AI

    Mar 24, 2026Hyunwoo Oh, Hanning Chen, Sanggeon Yun +5Efficient Multimodal InferenceFPGA Acceleration

    University of California, Irvine · Purdue University Northwest · Qualcomm +1

  19. Let's Think with Images Efficiently! An Interleaved-Modal Chain-of-Thought Reasoning Framework with Dynamic and Precise Visual Thoughts

    Mar 23, 2026Xu Liu, Yongheng Zhang, Qiguang Chen +3Multimodal CoT ReasoningVLM Reasoning

    Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen · School of Computer Science and Engineering, Central South University · Text Computing and Cognitive Intelligence Ministry of Education Engineering Research Center, Guizhou University +1

  20. Flash-Unified: A Training-Free and Task-Aware Acceleration Framework for Native Unified Models

    Mar 16, 2026Junlong Ke, Zichen Wen, Boxue Yang +6Efficient Multimodal InferenceUnified Multimodal Models

    Tsinghua University · Shanghai Jiao Tong University · Shanghai AI Laboratory +3

  21. CAMD: Coverage-Aware Multimodal Decoding for Efficient Reasoning of Multimodal Large Language Models

    Mar 16, 2026Huijie Guo, Jingyao Wang, Lingyu Si +3Efficient Multimodal InferenceMultimodal Large Language Models

    Institute of Software Chinese Academy of Sciences · University of Chinese Academy of Sciences · Wangxuan Institute of Computer Technology, Peking University

  22. DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression

    Mar 15, 2026Bingzhou Li, Tao HuangAudio-Visual UnderstandingEfficient Multimodal Inference

    Shanghai Jiao Tong University, Shanghai, China · Tongji University, Shanghai, China

  23. Cost-Efficient Multimodal LLM Inference via Cross-Tier GPU Heterogeneity

    Mar 13, 2026Donglin YuEfficient Multimodal InferenceLLM Serving

  24. Revisiting Model Stitching In the Foundation Model Era

    Mar 12, 2026Zheda Mai, Ke Zhang, Fu-En Wang +6Vision Foundation ModelsMultimodal Foundation Models

    1The Ohio State University · 3Amazon · 2Boston University

  25. HomeSafe-Bench: Evaluating Vision-Language Models on Unsafe Action Detection for Embodied Agents in Household Scenarios

    Mar 12, 2026Jiayue Pu, Zhongxiang Sun, Zilu Zhang +2Cultural Safety Evaluation of VLMsAI Agent Safety

    Gaoling School of Artificial Intelligence, Renmin University of China · University of Chinese Academy of Sciences · Beijing University of Posts and Telecommunications

  26. DriveXQA: Cross-modal Visual Question Answering for Adverse Driving Scene Understanding

    Mar 11, 2026Mingzhe Tao, Ruiping Liu, Junwei Zheng +6VLMs for Autonomous DrivingMultimodal Large Language Models

    Karlsruhe Institute of Technology · Mercedes-Benz Tech Innovation · Hunan University

  27. GLM-OCR Technical Report

    Mar 11, 2026Shuaiqi Duan, Yadong Xue, Weihan Wang +20Optical Character RecognitionDocument Understanding

    Zhipu AI · Tsinghua University

  28. LLM-MRD: LLM-Guided Multi-View Reasoning Distillation for Fake News Detection

    Mar 10, 2026Weilin Zhou, Shanwen Tan, Enhao Gu +1Fake News DetectionReasoning Distillation

    Xinjiang University, Urumqi, China · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University, Urumqi, China · Sichuan University, Chengdu, China

  29. From Flow to One Step: Real-Time Multi-Modal Trajectory Policies via Implicit Maximum Likelihood Estimation-based Distribution Distillation

    Mar 10, 2026Ju Dong, Liding Zhang, Lei Zhang +7Maximum Marginal Likelihood EstimationRobot Imitation Learning

    TAMS (Technical Aspects of Multimodal Systems), Department of Informatics, University of Hamburg, Hamburg, Germany. · Technical University of Munich, Germany. · Agile Robots SE, Munich, Germany.

  30. How Long Can Unified Multimodal Models Generate Images Reliably? Taming Long-Horizon Interleaved Image Generation via Context Curation

    Mar 8, 2026Haoyu Chen, Qing Liu, Yuqian Zhou +7Multimodal GenerationUnified Multimodal Models

    The Hong Kong University of Science and Technology (Guangzhou) · Adobe Research · Huazhong University of Science and Technology +1