Video-Language Models

Follow this topic Ask about this topic

Momentum

7 papers in the last four weeks, up 75% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 55

All topics

VETO: Video Efficient Token Optimization for Vision Language Models
Oct 1, 2026Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen +1Token Compression Video-Language Models
National Taiwan University · NVIDIA
RESUME: Recurrent State Updates from Motion and Residual Signals for Efficient Video Language Modeling
Sep 30, 2026Can Zhang, Xiaotian Han, Junyuan Shang +5Video-Language Models Recurrent State
Beijing University of Chemical Technology · Baidu, Inc.
GleanVID: Complementary Token Selection for Efficient Video Large Language Models
Sep 29, 2026Shuo Yang, Changbai Li, Rui Tang +3Video-Language Models Long Videos
Beihang University · Communication University of China
A Hierarchy-Aware Video-Language Model Evaluation and Hyperbolic Baseline for Surgery
Sep 22, 2026Ana Manzano Rodríguez, Pascal Mettes, Marlies P. Schijven +1Surgical Videos Surgery
Data Science Center HAVA-Lab, University of Amsterdam · Video & Image Sense Lab, Informatics Institute, University of Amsterdam · Amsterdam UMC Location University of Amsterdam, Surgery +2
Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning
Sep 14, 2026Mantek Singh, Jeshwanth Challagundla, Siddharth Raina +1Video-Language Models Chain-of-Thought Reasoning
Liverpool John Moores University, Liverpool, England · Carnegie Mellon University, Pittsburgh, USA · Meta, Sunnyvale, USA +1
Can We Trust Video Hallucination Detectors? VidHalLoc for Evaluating the Evaluators
Sep 9, 2026Xinyu Chen, Adnan Mahmood, Mark DrasCitation Hallucination Detection Ai-Generated Video Detection
Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics
Sep 8, 2026Ruibo Ming, Lei Sun, Deheng Zhang +8Video-Language Models Video Dataset
1INSAIT, Sofia University “St. Kliment Ohridski” · 2Adobe Research · 3Snap Research
Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs
Sep 3, 2026Prakhar KhatriLong-Video Benchmarks Video-Language Models
Independent Researcher
Can LLMs Design Video Coding Tools? A Case Study on Planar Mode
Sep 1, 2026Yingwen Zhang, Meng Wang, Liqiang He +1Video Coding Video-Language Models
Department of Computer Science, City University of Hong Kong · School of Data Science, Lingnan University
ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits
Sep 1, 2026Sethuraman T, Savya Khosla, Onkar Kishor Susladkar +6Video-Language Models Image-Text Alignment
University of Illinois Urbana-Champaign · Adobe Research
From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning
Aug 10, 2026Han Zhang, Yilin Zhao, Zaid Pervaiz Bhat +5Training-Free Video Anomaly Detection Fine-Grained Video Understanding
NVIDIA · Santa Clara University
VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
Aug 9, 2026Dong Xing, Jiaxin Chen, Hang Yang +3Video-Language Models Video Understanding
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences Changchun, Jilin 130033, China · University of Chinese Academy of Sciences, Beijing 100049, China · City University of Hong Kong, Hong Kong, China
Conformal Coverage Guarantees for Any Video Temporal Grounder
Aug 7, 2026Aseel Mohamed, Rasul Khanbayov, Erchin Serpedin +1Temporal Grounding Spatial Coherence
Department of Electrical and Computer Engineering, Texas A&M University at Qatar, Doha, Qatar · College of Science and Engineering, Hamad Bin Khalifa University, Doha, Qatar · Department of Computer and Electrical Engineering, Texas A&M University, College Station, TX, USA
The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping
Aug 6, 2026Sarvesh Baskar, Zikui Cai, Shayan Shabihi +5Video-Language Models Graph-Coherent Low-Frequency Ones
University of Maryland, College Park, USA · Scale AI, USA
Persistent Object Narratives for Token-Efficient Video Language Models
Aug 5, 2026Junzhe Chen, Siyuan Meng, Xiaojie GuoVideo-Language Models Video Understanding
Tianjin University
CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models
Aug 5, 2026Mukhtiar Ali, Harsh Dubey, Sugam Mishra +1Video-Language Models Long-Video Benchmarks
South Dakota State University Brookings, SD, USA
Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models
Aug 4, 2026Paribesh Regmi, Qingshuang Chen, Chi Zhang +3Visual Token Pruning Video-Language Models
Amazon.com Services LLC
GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models
Aug 4, 2026Mengjie Zhang, Qihui Zhu, Tao Zhang +10Visual Token Pruning Video-Language Models
University of Science and Technology of China He Fei, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center He Fei, China
CRAFT: Compression via Recursive Adaptive Fusion of Video Tokens for Vision-Language Models
Aug 3, 2026Yu Chen, Xiaohong Li, Xiaole Wang +3Token Compression Video-Language Models
1Evermind · 2Shanda Group · 3Peking University
Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
Aug 2, 2026Jiayang He, Tianling Xu, Diancheng Kang +4Token Compression Video Compression
Southern University of Science and Technology · Imperial College London · Jilin University
PhysMRV: Physical Memory Retrieval and Verification for Physics Plausibility Reasoning
Jul 11, 2026Wenyuan Wang, Lianyu Hu, Hao Wang +1Physical Plausibility Video-Language Models
1Nanyang Technological University · 2Rutgers University · University of Illinois Urbana-Champaign
Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
Jul 9, 2026Masatoshi Tateno, Alexandros Stergiou, Risa Shinoda +2Hand-Object Interaction Detection Egocentric Video
1The University of Tokyo · University of Twente · University of Bristol
Open-ended Multi-agent Autocurricula via Visual Inspection of Policies with Multi-modal LLMs
Jul 9, 2026Lorenzo Pantè, Andrea Fanti, Roberto CapobiancoCurriculum Reinforcement Learning Curriculum
Sapienza University of Rome, Italy · Sony AI, Zurich, Switzerland
Natural Language Camera Movement Understanding
Jul 3, 2026Yuwen Tan, Joey Huang, Jin Huang +2Video-Language Models Camera Control
Boston University, Boston, MA 02215, USA · Pixocial Technology, Bellevue, WA 98004, USA
Towards Continual Motion-Language Agents: LoRA Variants for Incremental Motion Understanding and Generation
Jun 29, 2026Bertram Taetz, Hugo Albuquerque Cosme da Silva, Gabriele Bleser-TaetzVideo-Language Models Text-To-Motion Generation
IT & Engineering International University of Applied Sciences Erfurt, Germany
Deep Learning-Based Sign Language Recognition from Videos and Cross-Lingual Translation to Indian Vernaculars
Jun 21, 2026Ramesh Nandipalli, Chandranath AdakSign Language Translation Indian Languages
Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihar, India
When Does a Video-Language Model Stop Watching? Reward Strength Controls the Formation and Reversal of Visual Shortcuts in Multimodal RLVR
Jun 20, 2026Zekun XuReinforcement Learning With Verifiable Reward Video-Language Models
School of Computer Science, The University of Sydney, Sydney, Australia
The Unreasonable Effectiveness of VLMs for Zero-shot Procedural Mistake Detection
Jun 19, 2026Serdar Ozsoy, Lars Doorenbos, Federico Spurio +2Video-Language Models Procedural
University of Bonn · Lamarr Institute for Machine Learning and Artificial Intelligence · Toyota Motor Europe
Audio-Visual Exchange-Aware Token Pruning for Efficient Audio-Visual Captioning
Jun 9, 2026Zihan Meng, Dexiang Hong, Weidong Chen +3Visual Token Pruning Audio-Visual Reasoning
University of Science and Technology of China
Don't Pause: Streaming Video-Language Synchrony for Online Video Understanding
Jun 5, 2026Zhenyu Yang, Kairui Zhang, Shengsheng Qian +2Video-Language Models
InfoMerge: Information-aware Token Compression for Efficient Video Large Language Models
Jun 1, 2026Xinxin Liu, Shiwei Gan, Xiao Liu +3Token Compression Video-Language Models
State Key Laboratory of Novel Software Technology, Nanjing University Nanjing 210023, China
Turing Patterns for Multimedia: Reaction-Diffusion Multi-Modal Fusion for Language-Guided Video Moment Retrieval
Jun 1, 2026Xiang Fang, Wanlong Fang, Wei Ji +1Video-Language Models Multimodal Fusion
Nanyang Technological University · Singapore, Singapore · Nanjing University +2
TLG: Temporal-Logic Grounding for Video Question Answering via Source-Annotation Reconstruction and Category-Targeted Reasoning
Jun 1, 2026Ali AlaviVideo Temporal Grounding Temporal Grounding
The Ohio State University
Adaptive Dense Evidence Refinement for Video Relational Reasoning for VRR-QA Challenge
May 31, 2026Yuyang Sun, Yongliang Wu, Xingyu Zhu +8Long Video Question Answering Video Understanding
Southeast University · National University of Singapore · Independent Researcher +2
Towards Effective Long-Video Event Prediction via Multi-Level Event Semantics Mining
May 29, 2026Bo Peng, YuanJie Lyu, PengGang Qin +1Future Video Prediction Video-Language Models
University of Science and Technology of China
PEEK: Picking Essential frames via Efficient Knowledge distillation
May 29, 2026Killian Steunou, Anas Filali Razzouki, Khalil Guetari +2Video-Language Models Frame Rate
Télécom SudParis — SAMOVAR Institut Polytechnique de Paris Palaiseau, France · Moments Lab 69 Avenue Pierre Grenier Boulogne-Billancourt, France
SLAP: The Semantic Least Action Principle for Variational Video-Language Modeling
May 29, 2026Xiang Fang, Wanlong FangVideo-Language Models Lagrangian Methods
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore
Rethinking Video-Language Model from the Language Input Perspective
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +2Video-Language Models Text-To-Video Generation Model
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +2
Towards Unified Vision-Language Models with Incomplete Multi-Modal Inputs
May 27, 2026Xiang Fang, Wanlong Fang, Changshuo Wang +4Video-Language Models Computer Vision
School of Software Engineering, Huazhong University of Science and Technology · Nanyang Technological University, Singapore · University College London +3
STORM: Internalized Modeling for Spatial-Temporal Reasoning in Video-Language Models
May 25, 2026Yiming Liang, Yixiao Chen, Yiyang Zhou +8Video-Language Models Video Understanding
1Purdue · 2Harvard · 3UNC +3
Learning Spatiotemporal Sensitivity in Video LLMs via Counterfactual Reinforcement Learning
May 21, 2026Dazhao Du, Jian Liu, Jialong Qin +7Video-Language Models Spatio-Temporal Reasoning
Hong Kong University of Science and Technology · Tencent
OSGNet with MLLM Reranking @ Ego4D Episodic Memory Challenge 2026
May 20, 2026Yisen Feng, Leigang Qu, Haoyu Zhang +5Large Language Model Reranking Video-Language Models
Harbin Institute of Technology (Shenzhen) · National University of Singapore · Pengcheng Laboratory +2
OTT-Vid: Optimal Transport Temporal Token Compression for Video Large Language Models
May 12, 2026Minseok Kang, Minhyeok Lee, Jungho Lee +6Token Compression Video Compression
Yonsei University · LG Electronics · KIST
EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant
May 11, 2026Zichen Wen, Boxue Yang, Junlong Ke +5Video-Language Models Parameter-Efficient Adaptation
EPIC Lab, Shanghai Jiao Tong University · Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou) +1
TTF: Temporal Token Fusion for Efficient Video-Language Model
May 8, 2026Simin Huo, Ning LIVideo-Language Models Qwen3
Shanghai Jiao Tong University
HY-Himmel Technical Report: Hierarchical Interleaved Multi-stream Motion Encoding for Long Video Understanding
May 4, 2026Haopeng Jin, Hongzhu Yi, Wenlong Zhao +6Video-Language Models Video Understanding
Tencent · University of Chinese Academy of Sciences · Beijing Forestry University
WindowQuant: Mixed-Precision KV Cache Quantization based on Window-Level Similarity for VLMs Inference Optimization
May 4, 2026Wei Tao, Xiaoyang Qu, Peiqiang Wang +4Key-Value Cache Quantization Quantizer
Huazhong University of Science and Technology, Wuhan, Hubei, China · Ping An Technology (Shenzhen) Co., Ltd., Shenzhen, Guangdong, China · Tsinghua University, Shenzhen International Graduate School, Shenzhen, Guangdong, China
QEVA: A Reference-Free Evaluation Metric for Narrative Video Summarization with Multimodal Question Answering
Apr 27, 2026Woojun Jung, Junyeong KimSummarization Multimodal Evaluation
Department of Artificial Intelligence, Chung-Ang University
IntentVLM: Open-Vocabulary Intention Recognition through Forward-Inverse Modeling with Video-Language Models
Apr 27, 2026Hamed Rahimi, Clemence Grislain, Adrien Jacquet Cretides +2Video-Language Models Inverse Problem
Institute of Intelligent Systems and Robotics (ISIR), Sorbonne University
Building a Precise Video Language with Human-AI Oversight
Apr 22, 2026Zhiqiu Lin, Chancharik Mitra, Siyuan Cen +13Video-Language Models Cinematic Ideal
1Carnegie Mellon University · 2Harvard University
Geometry-Guided 3D Visual Token Pruning for Video-Language Models
Apr 20, 2026Han Li, Zehao Huang, Jiahui Fu +23D Scene Understanding Visual Token Pruning
Beihang University · Zhongguancun Academy
Spatiotemporal Sycophancy: Negation-Based Gaslighting in Video Large Language Models
Apr 20, 2026Ziyao Tang, Pengkun Jiao, Bin Zhu +3Video-Language Models Weak Visual Grounding
Institute of Trustworthy Embodied AI, Fudan University · Shanghai Key Laboratory of Multimodal Embodied AI · Singapore Management University
CodecSight: Leveraging Video Codec Signals for Efficient Streaming VLM Inference
Apr 7, 2026Yulin Zou, Wenyan Chen, Yan Chen +6Video-Language Models Streaming Video
NTU Singapore Singapore · Beihang University China · A*STAR IAIC Singapore +1
Stateful Token Reduction for Long-Video Hybrid VLMs
Feb 27, 2026Jindong Jiang, Amala Sanjay Deshmukh, Kateryna Chumachenko +7Hybrid Mamba-Transformer Model Video-Language Models
NVIDIA
Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models
Feb 11, 2026Sethuraman T, Savya Khosla, Aditi Tiwari +12Video-Language Models Visual Evidence
University of Illinois Urbana-Champaign, USA · Adobe Research, USA · Qualtrics, USA +6

Oct 1, 2026, cs.CVJ/K move · Enter open · S save

VETO: Video Efficient Token Optimization for Vision Language Models

Gueter Josmy Faure, Hao Ping Wang, Min-Hung Chen, Winston H. Hsu

National Taiwan University · NVIDIA

Token Compression Video-Language Models

Open paper AskPDF arXiv

Processing long videos with Vision-Language Models (VLMs) is bottlenecked by the quadratic cost of visual tokens, making long-form inference prohibitively expensive. While single-axis compression methods mitigate this, they hit a hard efficiency floor because they treat spatial and temporal redundancy independently. We present VETO (Video Efficient Token Optimization for Vision-Language Models), a training-optional plug-in that eliminates this bottleneck through dual-axis compression: (i) an intra-frame compressor that merges semantically similar tokens within each frame via optimal-transport inspired matching, and (ii) an inter-frame compressor that identifies and merges temporally redundant frames. The key design insight is hierarchical ordering: by first compressing spatial dimensions, VETO drastically reduces the cost of subsequent global temporal matching, bypassing the efficiency wall of single-axis approaches, with an advantage that grows with modern fully-fused attention infrastructure. Empirically, VETO achieves up to 45% faster inference (e.g., on LLaVA-OneVision-7B) while preserving or improving accuracy. Under extreme token starvation (10% budget), VETO outperforms VFlowOpt (54.9%), VisionZip (52.6%), and FastV (47.9%) with 55.7% accuracy. We demonstrate universal applicability across LLaVA-OneVision, InternVL-2.5, and LongVA, with zero-shot accuracy preserved or improved in all cases.

Momentum

Related topics

Latest papers 55