Video-Language Models

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 110

All topics

InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models
Jun 1, 2026Xinxin Liu, Shiwei Gan, Xiao Liu +3Efficient VLM Inference Video-Language Models
State Key Laboratory of Novel Software Technology, Nanjing University Nanjing 210023, China
Jailbreaking Multimodal Large Language Models using Multi-Clip Video
Jun 1, 2026Choongwon Kang, Seungjong Sun, Hyunmin Jun +1Multimodal Robustness Video-Language Models
Department of Applied Artificial Intelligence, Sungkyunkwan University · Department of Human-Artificial Intelligence Interaction, Sungkyunkwan University
MOSS-Video-Preview: Toward Real-Time Video Understanding via Cross-Attention
Jun 1, 2026Pengyu Wang, Chenkun Tan, Shaojun Zhou +18Streaming Video Understanding Efficient VLM Inference
Fudan University · Shanghai Innovation Institute
V-LynX: Token Interface Alignment for Video+X LLMs
May 30, 2026Jungin Park, Jiyoung Lee, Kwanghoon SohnCross-Modal Alignment Multimodal QA
Yonsei University, Seoul, South Korea · Ewha Womans University, Seoul, South Korea
Towards Effective Long-Video Event Prediction via Multi-Level Event Semantics Mining
May 29, 2026Bo Peng, YuanJie Lyu, PengGang Qin +1Video-Language Models Long-Video Understanding
University of Science and Technology of China
SLAP: The Semantic Least Action Principle for Variational Video-Language Modeling
May 29, 2026Xiang Fang, Wanlong FangVideo-Language Models Video Frame Interpolation
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore
Masked Diffusion Vision-Language Models for Temporal Action Localization
May 28, 2026Fengshun Wang, Zhengbo Zhang, Zhigang TuHuman Activity Recognition Video-Language Models
Wuhan University · Singapore University of Technology and Design
From Pixels to Words -- Towards Native One-Vision Models at Scale
May 27, 2026Haiwen Diao, Jiahao Wang, Penghao Wu +18Vision-Language Models Video-Language Models
S-Lab, NTU · SenseTime Research · 3DLUT
Rethinking Video-Language Model from the Language Input Perspective
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +2VLM Adaptation Video-Language Models
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +2
Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +4Missing-Modality Learning VLM Robustness
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +3
Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models
May 26, 2026Oscar Chew, Serhii Honcharenko, Qian-Hui Chen +4Temporal Video Grounding Video-Language Model Evaluation
Texas A&M University · National Taiwan University · Stanford University +1
CodecCap: High-Fidelity Codec-Inspired Residual Modeling for Dense Video Captioning
May 26, 2026Zihan Lin, Songhe Deng, Shuwei He +6Video Captioning Video-Language Models
ERNIE Team, Baidu · College of Artificial Intelligence, Inner Mongolia University
STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
May 25, 2026Yiming Liang, Yixiao Chen, Yiyang Zhou +8Efficient VLM Inference Latent Visual Reasoning
1Purdue · 2Harvard · 3UNC +3
Which Way Did It Move? Diagnosing and Overcoming Directional Motion Blindness in Video-LLMs
May 21, 2026Jongseo Lee, Hyuntak Lee, Sunghun Kim +3Video-Language Models Temporal Video Understanding
Kyung Hee University · Princeton University
Enhancing Visual Token Representations for Video Large Language Models via Training-Free Spatial-Temporal Pooling and Gridding
May 21, 2026Bingjun Luo, Tony Wang, Hanqi Chen +1Video-Language Models Multimodal Token Compression
1Tsinghua University · 2Shenzhen University · 3Xidian University
Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning
May 21, 2026Dazhao Du, Jian Liu, Jialong Qin +7Counterfactual Evaluation Video-Language Models
Hong Kong University of Science and Technology · Tencent
Foresee-to-Ground: From Predictive Temporal Perception to Evidence-Driven Reasoning for Video Temporal Grounding
May 21, 2026Zelin Zheng, Xinyan Liu, Ruixin Li +4Temporal Video Grounding Video-Language Models
School of Computer Science and Technology, University of Chinese Academy of Sciences, Beijing, China · Beijing Key Laboratory of Embodied Intelligence Computing, Beijing, China · Faculty of Computing, Harbin Institute of Technology, Weihai, China +1
RoadTones: Tone Controllable Text Generation from Road Event Videos
May 20, 2026Chirag Parikh, Siddhi Pravin Lipare, Ravi Kiran SarvadevabhatlaControllable Text Generation Video Captioning
CVIT & iHub-Data, IIIT Hyderabad, India
DynaTok: Temporally Adaptive and Positional Bias-Aware Token Compression for Video-LLMs
May 19, 2026Minyoung Park, Taehun Kong, Sangjun AhnVideo-Language Models Video Token Compression
LG Electronics, Seoul, South Korea
EgoBabyVLM: Benchmarking Cross-Modal Learning from Naturalistic Egocentric Video Data
May 18, 2026Dongyan Lin, Phillip Rust, Angel Villar Corrales +19VLM Evaluation Vision-Language Models
Meta Superintelligence Labs · Stanford University · Meta Reality Labs +2
LiteFrame: Efficient Vision Encoders Unlock Frame Scaling in Video LLMs
May 17, 2026Jihwan Kim, Nikhil Parthasarathy, Danfeng Qin +5Efficient VLM Inference Video-Language Models
1Google DeepMind · 2Seoul National University
When Vision Speaks for Sound
May 13, 2026Xiaofei Wen, Wenjie Jacky Mo, Xingyu Fu +6Audio-Language Model Evaluation Audio-Visual Understanding
dUniversity of California, Davis · pPrinceton University · wUniversity of Wisconsin–Madison +1
Multimodal Abstractive Summarization of Instructional Videos with Vision-Language Models
May 12, 2026Maham Nazir, Muhammad Aqeel, Richong Zhang +1Video Summarization Video-Language Models
Beihang University, Beijing, China · University of Verona, Italy
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
May 12, 2026Minseok Kang, Minhyeok Lee, Jungho Lee +6Efficient VLM Inference Video-Language Models
Yonsei University · LG Electronics · KIST
EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
May 11, 2026Zichen Wen, Boxue Yang, Junlong Ke +5Streaming Video Understanding Self-Training
EPIC Lab, Shanghai Jiao Tong University · Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou) +1
TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models
May 11, 2026Junzhe Chen, Siyuan Meng, Yuxi Chen +3Video-Language Model Evaluation Video-Language Models
Tianjin University
Tracing the Arrow of Time: Diagnosing Temporal Information Flow in Video-LLMs
May 8, 2026Peitao Han, Fei Cheng, Lis K. Pereira +2Vision-Language Models Temporal Reasoning in Language Models
The University of Osaka · Center for Information and Neural Networks · National Institute of Information and Communications Technology +2
TTF: Temporal Token Fusion for Efficient Video-Language Model
May 8, 2026Simin Huo, Ning LIToken Merging Efficient VLM Inference
Shanghai Jiao Tong University
VISD: Enhancing Video Reasoning via Structured Self-Distillation
May 7, 2026Hao Lin, Kunyang Lv, Xu Jiang +5Video-Language Models Video Reasoning
1HUST · 2Wuhan University · 3Peking University +1
Reward-Decomposed Reinforcement Learning for Immersive Video Role-Playing
May 6, 2026Miao Wang, Yuling Shi, Yijiang Li +8Grounded Text Generation Large Language Model-Based Role-Play Simulation
1Nanjing University · 2Shanghai Jiao Tong University · University of California, San Diego +3

Jun 1, 2026, cs.CVJ/K move · Enter open · S save

InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models

Xinxin Liu, Shiwei Gan, Xiao Liu, Yafeng Yin+2

State Key Laboratory of Novel Software Technology, Nanjing University Nanjing 210023, China

Efficient VLM Inference Video-Language Models Multimodal Token Compression Video Token Compression Visual Token Compression

Open paper AskPDF arXiv

Video Large Language Models (Video-LLMs) achieve strong performance in video understanding, but their excessive visual tokens bring substantial computational overhead. Existing training-free compression methods improve inference efficiency by reducing visual tokens, yet they often rely on local adjacent-frame similarity for temporal redundancy estimation or allocate token budgets mainly according to segment length. Such designs are sensitive to frame-level noise and fail to capture the non-uniform information distribution of real-world videos. To address these challenges, we propose InfoMerge, a training-free visual token compression method that improves token utilization through robust redundancy estimation and content-aware budget allocation. Specifically, we propose the Temporal Fingerprint Difference: a segment-level second-order temporal redundancy estimation strategy, which models the temporal similarity structure of tokens at the same spatial positions within each segment. We further introduce Content-Aware Budget Allocation (CABA), which dynamically allocates segment-level token budgets based on segment uniqueness and spectral-entropy-based representational richness. By reducing repeated preservation of redundant static regions and allocating more tokens to informative segments, InfoMerge makes better use of the limited token budget while maintaining strong performance. Extensive experiments show that InfoMerge achieves strong efficiency--accuracy trade-offs across multiple benchmarks and backbones, with more pronounced advantages under aggressive compression. On LLaVA-OneVision-7B, InfoMerge retains 98.8% of the original average performance while reducing 85% of visual tokens and achieving a 4.24-fold speedup in the prefill stage.

Momentum

Related topics

Latest papers 110