Efficient Multimodal Inference

Overview

Efficient multimodal inference studies how to make models process mixed inputs with less computation, memory, latency, and energy. Common approaches prune or compress visual and audio tokens, select useful video frames or input modalities, compress KV caches, and adapt reasoning depth to query difficulty. Systems work also schedules inference across heterogeneous GPUs and edge devices, with evaluations spanning image and video question answering, long-context tasks, and real-time applications. Many methods preserve accuracy at substantial compression, but an open tension is whether they retain fine-grained evidence and grounding across tasks while improving end-to-end latency and cost.

Latest papers

CardsList

245 papers14 papers in the last four weeks, down 55% on the four weeks before. 0.1% of all new papers.

  1. Beyond Token Stacking: Multi-scale Visual Modulation and Adaptive Fused-feature Calibration for MLLMs

    Nov 16, 2026Ye Xiao, Shesheng Song, Yi Ren +5Multimodal Large Language ModelsEfficient Multimodal Inference

  2. HMMF-Net: A Lightweight Hierarchical Multi-Frame Multimodal Fusion Network for Accurate Fish Feeding Intensity Assessment

    Nov 6, 2026Phi-Hung Hoang, Van-Manh Tran, Nam-Thuan Trinh +1AquacultureMultimodal Fusion

  3. GeoJeV: Unified Spatial Probability for Training-Free Efficient Evidence Acquisition in Ultra-High-Resolution Remote Sensing

    Nov 6, 2026Xinyi Niu, Lingfeng Chen, Jiehua Zhang +4Efficient Multimodal InferenceRemote Sensing VLMs

  4. Lookahead Prediction-Guided Adaptive Visual Refocusing for Interleaved Multimodal Reasoning

    Nov 2, 2026Mengzhao Wang, Yanli Ji, Peng YeMultimodal CoT ReasoningTest-Time Adaptation of VLMs

  5. Raincheck: Deferred Multimodal Prefill for Efficient Multi-Turn MLLM Inference

    Oct 29, 2026Wenjie Huang, Jun Zhang, Hongyaoxing Gu +10Efficient Multimodal InferenceMultimodal Large Language Models

  6. Calibrating Attention-Based Token Pruning for Efficient Multi-Image Synthesis in UMMs

    Oct 29, 2026Zimeng Wu, Qijian Tian, Penghui Du +3Efficient Multimodal InferenceMultimodal Token Compression

  7. Efficient Agentic Audio-Visual Question Answering through Toolset Pruning and Modality Consolidation

    Oct 28, 2026Yi Wang, Tianzi Wang, Peter BellAudio-Visual QAMultimodal Agents

  8. Efficient Omni-modal Large Language Model Inference via Per-Query Modality Selection

    Oct 22, 2026Shuqi Liu, Yuzhou Cao, Luke OngEfficient Multimodal InferenceMultimodal Large Language Models

  9. TempoMoE: Identifying and Steering Fast-Slow Thinking Experts for Efficient Multimodal Reasoning

    Oct 19, 2026Xiyue Zhang, Keru Zhou, Ziyi Wang +2Mixture-of-Experts InferenceEfficient Multimodal Inference

  10. LightCRS: Lightweight Multi-Domain Conversational Recommendation System

    Oct 18, 2026He Ma, Lan You, Qiyang Wu +1Recommender SystemsMultimodal Recommendation

  11. Intern-Decision: Efficient Multimodal Decision Making with Autoregressive Masked Language Modeling

    Oct 15, 2026Tong Zhu, Zhe Xu, Zefeng He +7LLM Decision-MakingEfficient Multimodal Inference

  12. Adaptive Look, Flexible Thought: Learning How and How Long to Think in Multimodal Reasoning

    Oct 11, 2026Quanlin Chen, Xiuwei Chen, Kun Xiang +5Multimodal Large Language ModelsMultimodal Reasoning

  13. Look Back, Think Ahead: Visual Memory on Demand for Efficient Multimodal Reasoning

    Oct 8, 2026Yicheng Xue, Han Wu, Jufeng Yang +4VLM ReasoningEfficient Multimodal Inference

  14. RSJEV: Discriminative Remote Sensing Scene Classification with Multimodal Large Language Models

    Oct 6, 2026Dongchen Si, Di Wang, Mingzhen Xu +3Remote Sensing Image UnderstandingEfficient Multimodal Inference

  15. VisionWeave: Weaving Elastic Visual Representations as a Native Capability of MLLMs

    Oct 6, 2026Yuan Feng, Qize Yang, Ruizhe Chen +9Efficient Multimodal InferenceMultimodal Large Language Models

  16. Efficient Multimodal Inference through Adaptive Acquisition and Sequential Fusion

    Oct 5, 2026Payal Mohapatra, Haodong Yang, Yueyuan Sui +3Efficient Multimodal InferenceMultimodal Classification

  17. Conditional Trajectory Peaks: Single-Pass Multimodal Policies over Action Chunks

    Oct 5, 2026Di Wu, Rongtian Shen, Ping Liu +4Efficient Multimodal InferenceRobot Policy Learning

  18. ReMAP: Restoring the Perceptual Cycle with Reasoning-Time Latent Visual Memory

    Oct 4, 2026Hao Jiang, Zhanyu Guo, Chenwei Wu +11Memory-Augmented VLMsEfficient Multimodal Inference