Video-Language Models

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 200% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 110

All topics

Mid-Training Language Models on Raw Video
Oct 8, 2026Jaedong Hwang, Xiaoqian Shen, Ernie Chang +9Multimodal Large Language Models Video-Language Models
Meta AI · Massachusetts Institute of Technology · Work done at Meta +2
EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding
Oct 6, 2026Yuhao Qin, Junbo Wang, Yuke Li +1Multimodal RAG Video-Language Models
Northwestern Polytechnical University Xi’an, China
Stable Scores, Unstable Answers: Frame Phase and Option Order in Video Multiple-Choice Evaluation
Oct 6, 2026Lichen Zhu, Yiheng Wang, Yueqian Lin +2Video-Language Model Evaluation Video QA
Duke University, Durham, NC, USA
Have I Seen Enough? Frozen Video-Language Models Encode Evidence Readiness
Oct 6, 2026Dan Ben-Ami, Kobi Cohen, Chaim BaskinStreaming Video Understanding Video-Language Model Evaluation
INSIGHT Lab, Ben-Gurion University of the Negev, Israel · Ben-Gurion University of the Negev, Israel
Transferable Spatial Temporal Coherence Adversarial Attack on Black-Box Vision Language Models for Autonomous Driving
Oct 6, 2026Heyam Bin Jahlan Areej Alhothali Abeer AlhothaliAdversarial Attacks on VLMs VLMs for Autonomous Driving
Video Encoders Built on Image Representations
Oct 5, 2026Jusheng Zhang, Wenhao Wang, Longqi Cai +3Video-Language Models Video Representation Learning
Stanford University · Vast Intelligence Lab · Google DeepMind
MeSD: Multi-Evidence Self-Distillation for VideoLLM
Oct 5, 2026Weijie Zhu, Han Fang, Hanyu Fu +13On-Policy Self-Distillation Video-Language Models
UCAS · Xingchen AGI Lab, China Telecom Artificial Intelligence Technology (Beijing) Co., Ltd · PKU +2
Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs
Oct 1, 2026Youngwoo Shin, Yusung Ro, Minseo Kim +1Video-Language Models Activation Steering
Korea Advanced Institute of Science and Technology (KAIST)
RESUME: Recurrent State Updates from Motion and Residual Signals for Efficient Video Language Modeling
Sep 30, 2026Can Zhang, Xiaotian Han, Junyuan Shang +5Video-Language Models Video Representation Learning
Beijing University of Chemical Technology · Baidu, Inc.
Comparative study of adapting pre-trained models for driving behavior video captioning
Sep 30, 2026Sayak Mallick, Philipp Geiger, Augustin KelavaVLM Adaptation Autonomous Driving
University of Tübingen · Bosch Center for Artificial Intelligence
MOBA-VL: Event-Localized Multi-Turn Reinforcement Learning for Real-Time MOBA Commentary
Sep 29, 2026Shengyun Zhong, Xinkang Zhao, Ziyuan Chu +1Reinforcement Learning Video-Language Models
Zhejiang University, China · Northeastern University, USA
GleanVID: Complementary Token Selection for Efficient Video Large Language Models
Sep 29, 2026Shuo Yang, Changbai Li, Rui Tang +3Efficient VLM Inference Video-Language Models
Beihang University · Communication University of China
Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning
Sep 29, 2026Yanan Wang, Tingsong Li, Kaixun Jiang +3Direct Preference Optimization Video-Language Model Evaluation
Zhejiang University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China +2
Rethinking Visual Token Compression for Video Large Language Models: A Simple Yet Strong Baseline
Sep 28, 2026Xiao Zhang, Wang Zeng, Sheng Jin +3Video Understanding Video-Language Models
School of Computer Science and Engineering, Central South University · SenseTime Research and Tetras.AI
CinematicVQA: Benchmarking Film-Grammar Reasoning in Large Vision-Language Models
Sep 23, 2026Shuo Xing, Pooja Verlani, Balu Adsumilli +1Video Understanding Video-Language Model Evaluation
Texas A&M University, College Station, USA · Google Inc., Mountain View, USA
Efficient Reasoning Distillation: Small Video-Language Models via Synthetic CoT and Difficulty-Aware Fine-Tuning
Sep 14, 2026Mantek Singh, Jeshwanth Challagundla, Siddharth Raina +1CoT Distillation VLM Distillation
Liverpool John Moores University, Liverpool, England · Carnegie Mellon University, Pittsburgh, USA · Meta, Sunnyvale, USA +1
Why Is Video Still So Expensive? A Survey of Inference-Efficiency Mechanisms in Video and Audiovisual LLMs
Sep 9, 2026Killian Steunou, Yannis Tevissen, Mounîm A. El YacoubiAudio-Visual Understanding Efficient VLM Inference
SAMOVAR Laboratory, T´el´ecom SudParis, Institut Polytechnique de Paris, Palaiseau, France · Moments Lab, Paris, France
MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs
Sep 8, 2026Dhairya Bhatia, Bishoy Galoaa, Oliver Fritsche +7Video-Language Model Evaluation Video-Language Models
Electrical and Computer Engineering Department Northeastern University, Boston, MA, USA
Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics
Sep 8, 2026Ruibo Ming, Lei Sun, Deheng Zhang +8Video-Language Model Evaluation Video-Language Models
1INSAIT, Sofia University “St. Kliment Ohridski” · 2Adobe Research · 3Snap Research
The Shape of Time: Video-Token Contrast for Temporal Understanding in VideoLMs
Sep 3, 2026Yumeng Shi, Quanyu Long, Yin Wu +1Video-Language Models Video Representation Learning
Nanyang Technological University
TempCloze: Can Video-LLMs Identify the Missing Middle?
Sep 1, 2026Wenqi Pei, Henry Hengyuan Zhao, Yilai Liu +4Video-Language Model Evaluation Video-Language Models
The University of Hong Kong · National University of Singapore · Peking University
ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits
Sep 1, 2026Sethuraman T, Savya Khosla, Onkar Kishor Susladkar +6Video-Language Model Evaluation Video-Language Models
University of Illinois Urbana-Champaign · Adobe Research
The Role of Natural Language Understanding in Multimodal Video-Based Dengue Diagnosis
Aug 13, 2026Danial Sharifrazi, Saadat Behzadi, Julakha Jahan Jui +5Multimodal Classification Video-Language Models
Institute for Intelligent Systems Research and Innovations (IISRI), Deakin University, Geelong, Australia · Department of Electronic Engineering, University of Bologna, Bologna, Italy · CSIRO Health and Biosecurity, Australian Animal Health Laboratory, Geelong, Australia
VADER: Adaptive Debiasing for Hallucination Mitigation in Video Large Language Models
Aug 9, 2026Dong Xing, Jiaxin Chen, Hang Yang +3Video-Language Models VLM Hallucination Mitigation
Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences Changchun, Jilin 130033, China · University of Chinese Academy of Sciences, Beijing 100049, China · City University of Hong Kong, Hong Kong, China
Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs
Aug 7, 2026Wenzhang Sun, Chunfeng Wang, Xiangchen Yin +3Video-Language Model Evaluation Video-Language Models
Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos
Aug 5, 2026Yang Wang, Yanan Ma, Yiqi Liu +7Video-Language Model Evaluation Multimodal Reasoning
University of Manchester · University of Sheffield · 2Durham University +1
Persistent Object Narratives for Token-Efficient Video Language Models
Aug 5, 2026Junzhe Chen, Siyuan Meng, Xiaojie GuoVisual Tokenization Video-Language Models
Tianjin University
Adaptive Two-Stage Visual Token Pruning for Efficient Inference in Video-Language Models
Aug 4, 2026Paribesh Regmi, Qingshuang Chen, Chi Zhang +3Efficient VLM Inference Video-Language Models
Amazon.com Services LLC
GSTEP: Global Spatio-Temporal Density-Driven Visual Token Pruning for Efficient Video Large Language Models
Aug 4, 2026Mengjie Zhang, Qihui Zhu, Tao Zhang +10Efficient VLM Inference Video-Language Models
University of Science and Technology of China He Fei, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center He Fei, China
Rethinking Video Token Compression with a Global Codebook: Learning Once, Compressing Everywhere
Aug 2, 2026Jiayang He, Tianling Xu, Diancheng Kang +4Efficient VLM Inference Video-Language Models
Southern University of Science and Technology · Imperial College London · Jilin University

Oct 8, 2026, cs.CVJ/K move · Enter open · S save

Mid-Training Language Models on Raw Video

Jaedong Hwang, Xiaoqian Shen, Ernie Chang, Changsheng Zhao+8

Meta AI · Massachusetts Institute of Technology · Work done at Meta+2

Multimodal Large Language Models Video-Language Models Multimodal Pretraining Self-Supervised Pre-Training

Open paper AskPDF arXiv

Multimodal large language models learn mostly from paired image-text data or annotated video, and raw web video is rarely used to further train an existing language model. We study whether raw video, with no captions and no text loss, can serve as mid-training data for a pretrained language model. Frames are encoded into continuous visual tokens, and the language model learns to predict the next visual token. We mid-train Qwen3-1.7B on raw clips from YT-Temporal-1B and then apply the same image-text instruction tuning to it and to the model without mid-training, so that the two differ only in mid-training. The mid-trained model scores 2.9 points higher on average across four video benchmarks and 5.1 points higher across ten image benchmarks, spanning perception, document, and chart tasks. Text performance is preserved even though mid-training includes no text, with an average of 48.9 across 14 text benchmarks compared with 48.0 for the model without mid-training. Analyses across training show that the image and video gains emerge within 30% of training and plateau thereafter, varying by less than 0.5 points. Predicting captions fails to outperform next-visual-token prediction, demonstrating that video mid-training can remain purely self-supervised without the computational overhead or labeling noise of automated captioning.

Momentum

Related topics

Latest papers 110