Efficient Multimodal Inference

Overview

Efficient multimodal inference studies how to make models process mixed inputs with less computation, memory, latency, and energy. Common approaches prune or compress visual and audio tokens, select useful video frames or input modalities, compress KV caches, and adapt reasoning depth to query difficulty. Systems work also schedules inference across heterogeneous GPUs and edge devices, with evaluations spanning image and video question answering, long-context tasks, and real-time applications. Many methods preserve accuracy at substantial compression, but an open tension is whether they retain fine-grained evidence and grounding across tasks while improving end-to-end latency and cost.

Latest papers

CardsList

245 papers14 papers in the last four weeks, down 55% on the four weeks before. 0.1% of all new papers.

  1. EfficientUICoder: A Bidirectional Token Compression Framework for Efficient MLLM-Based UI Code Generation

    Sep 15, 2025Jingyu Xiao, Zhongyi Zhang, Yuxuan Wan +3Efficient Multimodal InferenceCode Generation

  2. Multimodal Remote Inference

    Aug 11, 2025Keyuan Zhang, Yin Sun, Bo JiSemi-Markov Decision ProcessEfficient Multimodal Inference

  3. Resource-Limited Joint Multimodal Sentiment Reasoning and Classification via Chain-of-Thought Enhancement and Distillation

    Aug 7, 2025Haonan Shangguan, Xiaocui Yang, Shi Feng +3Efficient Multimodal InferenceSentiment Analysis

  4. Speculative Decoding Reimagined for Multimodal Large Language Models

    May 20, 2025Luxi Lin, Zhihang Lin, Zhanpeng Zeng +1Efficient Multimodal InferenceSpeculative Decoding