Efficient Multimodal Inference

Follow this topic Ask about this topic

Momentum

28 papers in the last four weeks, up 211% on the four weeks before. 0.3% of all new papers.

Jul 13Week of Sep 28

Latest papers 157

All topics

Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling
Jun 23, 2026Kun Zhang, Chenxin Fang, Tao Chen +4Efficient Multimodal Inference Multimodal Large Language Models
Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China
Spectral Evolution-Guided Token Pruning in Multimodal Large Language Models
Jun 23, 2026Bin Chen, Yuxiang Cai, Yadan Luo +3Efficient Multimodal Inference Multimodal Large Language Models
School of Software Technology, Zhejiang University, Ningbo, China · Zhejiang Key Laboratory of Digital-Intelligence Service Technology, China · The University of Queensland, St Lucia, QLD, Australia +2
Kamera: Unified Position-Invariant Multimodal KV Cache for Training-Free Reuse
Jun 22, 2026Bole Ma, Jan Eitzinger, Harald Koestler +1Efficient Multimodal Inference KV Caching
Efficient Multimodal Clinical Question Answering for Pulmonary Embolism Risk Assessment
Jun 21, 2026Xiangyuan Xue, Yang Yu, Yan Gao +5Efficient Multimodal Inference Multimodal QA
University of Auckland, Auckland, New Zealand · University of Cambridge, Cambridge, United Kingdom · University of Adelaide, Adelaide, Australia +1
PerceptionDLM: Parallel Region Perception with Multimodal Diffusion Language Models
Jun 17, 2026Yueyi Sun, Yuhao Wang, Jason Li +8Efficient Multimodal Inference Image Captioning
Peking University MSALab, ByteDance
Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning
Jun 17, 2026Pengyu Li, Zhitao Gao, Lingling Zhang +5Cross-Modal Knowledge Distillation Efficient Multimodal Inference
Xi’an Jiaotong University · Shaanxi Province Key Laboratory of Big Data Knowledge Engineering · MOE KLINNS Lab, Xi’an Jiaotong University +1
Stepwise Token Selection for Efficient Multimodal Large Language Models
Jun 14, 2026Landi He, Shawn Young, Lijian XuEfficient Multimodal Inference Multimodal Large Language Models
Shenzhen University of Advanced Technology
DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
Jun 13, 2026David Huang, Lianlei ShanEfficient Multimodal Inference Multimodal Reasoning
University of Toronto · Tsinghua University
ProMUSE: Progressive Multi-modal Uncertainty-guided Staged Evidential Alzheimer Disease Classification
Jun 11, 2026Long Doan, Branden Chen, Ethan Litton +6Efficient Multimodal Inference Evidential Deep Learning
Department of Computer Science, Kennesaw State University · Department of Data Science and Analytics, Kennesaw State University · Department of Information Technology, Kennesaw State University +2
LaME: Learning to Think in Latent Space for Multimodal Embedding via Information Bottleneck
Jun 11, 2026Peixi Wu, Biao Yang, Feipeng Ma +7Efficient Multimodal Inference Multimodal Embedding
University of Science and Technology of China · 2Kuaishou Technology · 3Zhejiang University +1
M*: A Modular, Extensible, Serving System for Multimodal Models
Jun 10, 2026Atindra Jha, Naomi Sagan, Keisuke Kamahori +9Efficient Multimodal Inference Distributed Inference
Stanford University · University of Washington · Carnegie Mellon University
InternVideo3: Agentify Foundation Models with Multimodal Contextual Reasoning
Jun 10, 2026Ziang Yan, Sheng Xia, Jiashuo Yu +10Efficient Multimodal Inference Multimodal Reasoning
Shanghai AI Laboratory · Shanghai Innovation Institute · Nanjing University
ReasonRec: A Reasoning-Augmented Multimodal Agent for Unified Recommendation
Jun 8, 2026Yihua Zhang, Mingfu Liang, Jiyan Yang +11Efficient Multimodal Inference Multimodal CoT Reasoning
Meta AI · Michigan State University · The University of North Carolina at Chapel Hill +1
From Senses to Decisions: The Information Flow of Auditory and Visual Perception in Multimodal LLMs
Jun 8, 2026Wish Suharitdamrong, Muhammad Awais, Xiatian Zhu +1Efficient Multimodal Inference Multimodal Large Language Models
Surrey Institute for People-Centred AI (PAI), University of Surrey, UK · Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, UK
Late-Layer Fusion is Enough: Dual-Path Vision Token Routing for Multimodal Large Language Models under Visual Saturation
Jun 8, 2026Siyuan Liu, Jinyang WuEfficient Multimodal Inference Multimodal Large Language Models
aSchool of Mechanics and Engineering Science, Peking University, China · bDepartment of Automation, Tsinghua University, China
Look Less, Reason More: Block-wise Attention Skipping for Efficient Multimodal LLMs
Jun 7, 2026Jie Ma, Zhike Qiu, Jiayi Ji +2Efficient Multimodal Inference Multimodal Large Language Models
Xiamen University
M $^3$ Exam: Benchmarking Multimodal Memory for Realistic User-Agent Interactions
Jun 5, 2026Zhengjun Huang, Wenxuan Liu, Zhoujin Tian +6Multimodal Grounding Efficient Multimodal Inference
The Hong Kong University of Science and Technology · Peng Cheng Laboratory · Beijing University of Chemical Technology +4
Dreaming when Necessary: Advancing World Action Models with Adaptive Multi-Modal Reasoning
Jun 5, 2026Yinzhou Tang, Jingbo Xu, Yu Shang +4Efficient Multimodal Inference World Models for Robotics
Tsinghua University · Manifold AI
DisasterBench: A Multimodal Benchmark for UAV-Based Disaster Response in Complex Environments
Jun 4, 2026Tan Zhang, Quanyou Li, Lu Zhang +3Efficient Multimodal Inference Multimodal Reasoning
School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, 611731, China · School of Information and Communication Engineering, Dalian University of Technology, Dalian, 116024, China · School of Computing and Communications, Lancaster University, Lancaster, LA1 4YW, England +1
Beyond Generative Decoding: Discriminative Hidden-State Readout from a Native Omni-Modal LLM for Multimodal Sentiment Analysis
Jun 4, 2026Bin Wen, Tien-Ping TanEfficient Multimodal Inference Sentiment Analysis
School of Computer Sciences, Universiti Sains Malaysia, Penang, Malaysia.
Hyper-ICL: Attention Calibration with Hyperbolic Anchor Distillation for Multimodal In-Context Learning
Jun 3, 2026Niloufar Alipour Talemi, Hossein Kashiani, Fatemeh AfghahEfficient Multimodal Inference Multimodal ICL
Holcombe Department of Electrical and Computer Engineering, Clemson University, Clemson, United States.
Differentiable Efficient Operator Search
Jun 3, 2026Xiaohuan Pei, Jiyuan Zhang, Yuanfan Guo +4Efficient Multimodal Inference Multimodal Token Compression
1The University of Sydney · 2ByteDance · 3Shanghai Jiao Tong University +1
Perceive Before Reasoning: A Pre-Reasoning Perception Framework for Efficient and Reliable Proactive Mobile Agents
Jun 2, 2026Zhijie Ding, Weinan Hong, Zicheng Zhu +6Efficient Multimodal Inference Proactive Assistance
HyperAI Team, Xiaomi Corporation · Zhongnan University of Economics and Law · 1HyperAI Team, Xiaomi Corporation +5
Spectral-Progressive Thought Flow for Lightweight Multimodal Reasoning
Jun 1, 2026Yixian Shen, Zhiheng Yang, Qi Bi +6Efficient Multimodal Inference Visual Spatial Reasoning
Informatics Institute, University of Amsterdam, Amsterdam, The Netherlands · University of Thessaly, Volos, Greece · Department of Computer Science, University College London +3
Improving Visual Token Reduction via Rectifying Distortions for Efficient Multimodal LLM Inference
Jun 1, 2026Hyeonwoo Cho, Donghyeon Baek, Yewon Kim +1Efficient Multimodal Inference Multimodal Large Language Models
Yonsei University · Korea Institute of Science and Technology (KIST)
VisionPulse: Dynamic Visual Sparsity for Efficient Multimodal Reasoning
May 29, 2026Hengbo Xu, Shengjie Jin, Yanbiao Ma +1Efficient Multimodal Inference Visual Attention
Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China.
MARS Policy: Multimodality Only When It Matters
May 28, 2026Jindou Jia, Tuo An, Yuxuan Hu +7Efficient Multimodal Inference Robot Policy Learning
1MARS Lab, Nanyang Technological University
DREAM-R: Multimodal Speculative Reasoning with RL-Based Refined Drafting, Precise Verification, and Fully Parallel Execution
May 27, 2026Yunhai Hu, Zining Liu, Xiangyang Yin +5Efficient Multimodal Inference RL for Language Model Reasoning
New York University · University of Pennsylvania · Cerebras Systems
ProgVLA: Progress-Aware Robot Manipulation Skill Learning
May 27, 2026Seungsu Kim, Jinyoung Choi, Seungmin Baek +1Cross-Modal Learning Efficient Multimodal Inference
NAVER LABS Europe · NAVER LABS
ROVER: Routing Object-Centric Visual Evidence for Grounded Multi-Image Reasoning
May 27, 2026Guannan Lv, Ren Nie, Hongjian Dou +1Multimodal Grounding Efficient Multimodal Inference
Kuaishou Technology

Jun 23, 2026, cs.CVJ/K move · Enter open · S save

Towards Fast and Effective Long Video Understanding of Multimodal Large Language Models via Adaptive Quasi-Gaussian Sampling

Kun Zhang, Chenxin Fang, Tao Chen, Baiyang Song+3

Key Laboratory of Multimedia Trusted Perception and Efficient Computing, Ministry of Education of China, Xiamen University, 361005, P.R. China

Efficient Multimodal Inference Multimodal Large Language Models Long-Video Understanding Video Frame Selection

Open paper AskPDF arXiv

Long video understanding remains a daunting challenge for Multimodal Large Language Models (MLLMs) due to the excessive computation and memory footprint. Thus, keyframe selection is often adopted to mitigate this shortcoming, which however still suffers from low flexibility and high noise due to its hard sampling principle. In this paper, we define video frame selection as a problem of Quasi-Gaussian Sampling, and propose an adaptive and training-free approach termed AdaQ. Inspired by the 3-

σ

rule of Gaussian distribution, the objective of AdaQ is to achieve the optimal 3-

σ

interval for different examples, i.e., a smaller 3-

σ

interval for the local query and a larger one for the global query, thereby facilitating robust and adaptive frame sampling. To validate AdaQ, we apply it to four MLLMs with three embedding models. The extensive experimental results not only show its obvious performance gains over the default MLLMs and the SOTA keyframe selection methods, e.g., helping Qwen3-VL-8B outperform GPT4o by 15.8% on average by using only 64 frames, but also confirm its superior robustness and high efficiency for long-video understanding, e.g., only 1 hyper-parameter needs to be set.

Momentum

Related topics

Latest papers 157