Overview

Video-language models study how systems can interpret video and use language to answer questions, describe events, locate moments, and assist during live streams. Common approaches combine pretrained vision encoders with large language models, using frame sampling or temporal representations, instruction tuning and reinforcement learning, and retrieval or tools to handle long videos. Token compression and selective frame processing help control the high cost of video inference, while benchmarks such as VideoMME, MVBench, and ActivityNet test general understanding and temporal localization. A central open challenge is ensuring that answers genuinely reflect visual and temporal evidence: models can recognize scenes yet miss motion, event order, or brief details, and may hallucinate or rely on language priors.

Latest papers

CardsList

177 papers10 papers in the last four weeks, down 58% on the four weeks before. 0.1% of all new papers.

  1. Suppressing the Shortcut Hidden Behind the Gains of VideoLLM Fine-Tuning

    Nov 19, 2026Youngwoo Shin, Jongsuk Kim, Junmo KimVideo-Language ModelsVisual Evidence Sensitivity in VLMs

  2. RoughCut-Bench: A Benchmark for Automatic Video Rough-Cut with Large Vision-Language Models

    Nov 18, 2026Ruisi Zhao, You Li, Sihan Chen +7Video Editing BenchmarksVideo Editing

  3. V-JEPA-LM: Video-Language Models That Read Dynamics Beyond Static, Text-Aligned Representations

    Nov 13, 2026Keuntae Kim, Dongjun Kim, Yong Suk ChoiVideo-Language ModelsTemporal Reasoning in VLMs

  4. Beyond Scene Shortcuts: Action-Centric Video-Text Alignment for Fine-Grained Action Understanding

    Nov 4, 2026Zhiliang Zhu, Jiazheng Huang, Dawei ZhaoFine-Grained Action UnderstandingVideo-Language Models

  5. SleuthVLM: Do We Really Need Audio-in-Video for Omni Video Understanding?

    Nov 4, 2026Peiran Wu, Fangfei Gou, Zhaoxi Mu +6Video-Language ModelsAudio-Visual Understanding

  6. SPOD: Recovering Suppressed Temporal Evidence in Video-LLMs via Single-Pass Orthogonal Decomposition

    Nov 3, 2026Yangguang Lin, Quan Fang, Xiaoyan Liu +1Video-Language ModelsTemporal Reasoning in VLMs

  7. As the Story Unfolds: Watching a Film and Identifying Characters as a Human Does

    Oct 27, 2026Zhongrui Gui, Junyu Xie, Tengda Han +2Video-Language ModelsMultimodal Grounding

  8. DirCast: Live Esports Directing and Commentary with a Single Streaming Video-Language Model

    Oct 22, 2026Zixian Gao, Lixin Xiu, Penghao Luo +5Video-Language ModelsStreaming Video Understanding

  9. Ask About the Right Moment: Training-Free Object Identity Reasoning for Video-Language Models

    Oct 22, 2026Di Yang, Xiaochang Li, Muyang Liu +2Video ReasoningVideo-Language Models

  10. Cache Locally, Recompute Selectively for Streaming Video Understanding

    Oct 21, 2026Junhyeok Lee, Sangyoon Lee, Jaeho LeeStreaming Video UnderstandingStreaming Inference

  11. Long-form Video-to-Text Summarization using Recursive Language Models

    Oct 19, 2026Alessia Intini, Lorenzo Vaiani, Enrico Chen +1Video SummarizationVideo-Language Models

  12. From Perception to Prediction and Intervention: Benchmarking Proactive Video Assistants in Real-Life Scenarios

    Oct 15, 2026Boyu Cui, Junhao Luo, Zhihao Wang +5Proactive Video UnderstandingVideo-Language Model Evaluation

  13. Mitigating Video Hallucinations via Attention-Guided Representation Perturbation

    Oct 14, 2026Zhixin Cai, Ruoqi Yuan, Xinchen Yang +4VLM Hallucination MitigationHallucination Mitigation

  14. RepProbe: Can Video Language Models Count Repeated Actions? A Diagnostic Benchmark

    Oct 13, 2026Mahsa Khoshnoodi, Sarah Adel BargalCounting in VLMsVideo-Language Models

  15. WIRE: Weight Space Implicit Representation Encoding of Frozen Video LLMs

    Oct 12, 2026Ameen Ali, Tamim Zoabi, Lior Wolf +1Efficient VLM InferenceVideo-Language Models

  16. When Left Is Right? Camera-Motion Understanding beyond Screen

    Oct 12, 2026Chong Cheng, Ran Zhang, Xinyuan Li +3Camera Motion UnderstandingVideo-Language Models

  17. OWLET: On-Policy Weighted Learning via Evidence Transfer for Video Large Language Model Post-Training

    Oct 9, 2026Qitan Lv, Ke Fan, Hong Wang +5Video-Language ModelsLarge Vision-Language Models

  18. Video Codec-Guided Training-Free Token Compression for Video Large Language Models

    Oct 9, 2026Xihua Sheng, Ruijie Liu, Chang Wen ChenVideo Token CompressionVideo-Language Models

  19. Mid-Training Language Models on Raw Video

    Oct 8, 2026Jaedong Hwang, Xiaoqian Shen, Ernie Chang +9Multimodal Large Language ModelsVideo-Language Models

  20. EC-RAG: Event Chain Retrieval-Augmented Generation for Long Video Understanding

    Oct 6, 2026Yuhao Qin, Junbo Wang, Yuke Li +1Multimodal RAGVideo-Language Models

  21. Have I Seen Enough? Frozen Video-Language Models Encode Evidence Readiness

    Oct 6, 2026Dan Ben-Ami, Kobi Cohen, Chaim BaskinStreaming Video UnderstandingVideo-Language Model Evaluation

  22. Video Encoders Built on Image Representations

    Oct 5, 2026Jusheng Zhang, Wenhao Wang, Longqi Cai +3Video-Language ModelsVideo Representation Learning

  23. MeSD: Multi-Evidence Self-Distillation for VideoLLM

    Oct 5, 2026Weijie Zhu, Han Fang, Hanyu Fu +13On-Policy Self-DistillationVideo-Language Models

  24. Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs

    Oct 1, 2026Youngwoo Shin, Yusung Ro, Minseo Kim +1Video-Language ModelsActivation Steering

  25. RESUME: Recurrent State Updates from Motion and Residual Signals for Efficient Video Language Modeling

    Sep 30, 2026Can Zhang, Xiaotian Han, Junyuan Shang +5Video-Language ModelsVideo Representation Learning