Video Token Compression

Follow this topic Ask about this topic

Momentum

10 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 44

All topics

OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs
May 26, 2026Guangzhi Sun, Yixuan Li, Yudong Yang +1Streaming Video Understanding Multimodal Large Language Models
ByteDance · Department of Engineering, University of Cambridge · Tsinghua University
O-MARC: Omni Memory-Augmented Compression Distillation for Efficient Video Understanding
May 26, 2026Peiran Wu, Yunze Liu, Chi-Hao Wu +2Audio-Visual Understanding Efficient Multimodal Inference
University of Bristol · Memories.ai Research · University of Central Florida
LLaVA-OneVision-2: Towards Next-Generation Perceptual Intelligence
May 25, 2026Xiang An, Yin Xie, Feilong Tang +27VLM Evaluation Temporal Video Grounding
Glint Lab · AIM for Health Lab · MVP Lab
ST-SimDiff: Balancing Spatiotemporal Similarity and Difference for Efficient Video Understanding with MLLMs
May 21, 2026Bingjun Luo, Tony Wang, Chaoqi Chen +1Multimodal Token Compression Video Token Compression
Tsinghua University · Shenzhen University · Xidian University
ORBIS: Output-Guided Token Reduction with Distribution-Aware Matching for Video Diffusion Acceleration
May 21, 2026Hangyeol Lee, Joo-Young KimToken Merging Video Diffusion Models
KAIST · Daejeon, South Korea
DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs
May 19, 2026Minyoung Park, Taehun Kong, Sangjun AhnVideo-Language Models Video Token Compression
LG Electronics, Seoul, South Korea
OmniRefine: Alignment-Aware Cooperative Compression for Efficient Omnimodal Large Language Models
May 12, 2026Yuchen Deng, Zidang Cai, Hai-Tao Zheng +3Audio-Visual Understanding Efficient Multimodal Inference
Tsinghua Shenzhen International Graduate School, Tsinghua University · Pengcheng Laboratory
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
May 12, 2026Minseok Kang, Minhyeok Lee, Jungho Lee +6Efficient VLM Inference Video-Language Models
Yonsei University · LG Electronics · KIST
Keep What Audio Cannot Say: Context-Preserving Token Pruning for Omni-LLMs
May 12, 2026Chaeyoung Jung, Kyeongha Rho, Joon Son ChungAudio-Visual Understanding Efficient Multimodal Inference
Korea Advanced Institute of Science and Technology (KAIST)
EchoPrune: Interpreting Redundancy as Temporal Echoes for Efficient VideoLLMs
May 11, 2026Jiameng Li, Minye Wu, Jiezhang Cao +2Efficient VLM Inference Token Pruning
KU Leuven · Shanghai Jiaotong University · Weill Cornell Medicine
Fre-Res: Frequency-Residual Video Token Compression for Efficient Video MLLMs
May 10, 2026Yigui Feng, Qinglin Wang, Yang Liu +1Efficient VLM Inference Visual Tokenization
The College of Computer Science, National University of Defense Technology, Changsha, Hunan, China · College of Computer Science, National University of Defense Technology, Deya Road 109, Changsha, Hunan 410073, China · The Shien-Ming Wu School of Intelligent Engineering, South China University of Technology, Guangzhou, Guangdong, China +1
TTF: Temporal Token Fusion for Efficient Video-Language Model
May 8, 2026Simin Huo, Ning LIToken Merging Efficient VLM Inference
Shanghai Jiao Tong University
VideoRouter: Query-Adaptive Dual Routing for Efficient Long-Video Understanding
May 7, 2026Kuanwei Lin, Wenhao Zhang, Ge LiEfficient VLM Inference Long-Video Understanding
School of Electronic and Computer Engineering, Peking University
DASH: Dynamic Audio-Driven Semantic Chunking for Efficient Omnimodal Token Compression
Mar 15, 2026Bingzhou Li, Tao HuangAudio-Visual Understanding Efficient Multimodal Inference
Shanghai Jiao Tong University, Shanghai, China · Tongji University, Shanghai, China

May 26, 2026, cs.AIJ/K move · Enter open · S save

OmniMem: Perturbation-aware Memory Compression for Streaming Audio-Visual LLMs

Guangzhi Sun, Yixuan Li, Yudong Yang, Chao Zhang

ByteDance · Department of Engineering, University of Cambridge · Tsinghua University

Streaming Video Understanding Multimodal Large Language Models Multimodal Token Compression KV-Cache Compression Video Token Compression

Open paper AskPDF arXiv

Audio-visual large language models (LLMs) hold strong promise for long-form video understanding, yet their long-video inference is fundamentally limited by the linear growth of video tokens and key-value (KV) caches. We present OmniMem, a memory-efficient streaming framework designed specifically for audio-visual LLMs. Unlike existing compression methods that treat all tokens uniformly, OmniMem introduces a modality-aware memory allocation strategy that separately manages visual and audio contexts, addressing the severe token imbalance between the two modalities. OmniMem further preserves informative and non-redundant KV states through perturbation-aware memory selection, enabling compact memory without sacrificing long-range understanding. To strengthen compression under realistic deployment constraints, we also explore budget-aware fine-tuning, which encourages the model to consolidate useful information into retained memory. Experiments on VideoMME Long, LVBench, and LVOmniBench with video-SALMONN 2+ and Qwen-2.5-Omni show that OmniMem consistently improves over strong training-free compression baselines by 2-4% absolute accuracy under the same memory budgets, with an additional 1-2% gain after fine-tuning.

Momentum

Related topics

Latest papers 44