Video-Language Model Evaluation

Ask about this topic

Momentum

18 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 55

All topics
CardsList

  1. Action- and Language-Conditioned Video Assessment for Embodied Control

    Aug 8, 2026Hwanhee Kim, Jaehyun Jang, Seungmin Cha +3Video-Language Model EvaluationLong-Horizon Agent Evaluation

    Robotics Program, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea · School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon 34141, Republic of Korea

  2. Stable Curves, Unstable Items: Item-Level Scaling Heterogeneity in Video LLMs

    Aug 7, 2026Wenzhang Sun, Chunfeng Wang, Xiangchen Yin +3Video-Language Model EvaluationVideo-Language Models

  3. The Low Frequency Trap: Video Language Models Fail at Simple Event Bookkeeping

    Aug 6, 2026Sarvesh Baskar, Zikui Cai, Shayan Shabihi +5Video-Language Model EvaluationTemporal Video Understanding

    University of Maryland, College Park, USA · Scale AI, USA

  4. Reading Between the Frames: Interpreting Implicit and Non-literal Meaning in Social Media Videos

    Aug 5, 2026Yang Wang, Yanan Ma, Yiqi Liu +7Video-Language Model EvaluationMultimodal Reasoning

    University of Manchester · University of Sheffield · 2Durham University +1

  5. CLIP-CC-Bench: Evaluating Paragraph-Level Video Descriptions in Video-Language Models

    Aug 5, 2026Mukhtiar Ali, Harsh Dubey, Sugam Mishra +1Video-Language Model EvaluationVideo Captioning

    South Dakota State University Brookings, SD, USA

  6. Caved or Convinced: Temporal Sampling Gates Claim Deference in Video Large Language Models

    Aug 4, 2026Yuxin Cao, Wei Song, Jingling Xue +1Selective PredictionVideo-Language Model Evaluation

    National University of Singapore, Singapore · University of New South Wales, Australia

  7. PhyCheck: Fine-Grained Evidence-Grounded Dataset for Physical Law Understanding in Video-LLMs

    Aug 3, 2026Zhongjie Ba, Shengwang Xu, Peng Cheng +4Video-Language Model EvaluationVideo QA

    State Key Lab. of Blockchain and Data Security, Zhejiang University, Hangzhou, China · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security Hangzhou, Zhejiang, China · Mohamed Bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates

  8. RSVideo: Are Your Vision-Language Models Ready for Remote Sensing Videos?

    Aug 3, 2026Hongjie Zhou, Shiqin Wang, Haoyang Chen +5Video-Language Model EvaluationRemote Sensing

    Wuhan University · Zhongguancun Academy

  9. Accuracy Without Grounding: Diagnosing Visual Dependency Dissociation in Video LLM Benchmarks

    Jul 14, 2026Jae Joong LeeVLM EvaluationVideo-Language Model Evaluation

    Department of Computer Science Purdue University West Lafayette, Indiana, USA

  10. Do Video-LLMs Actually Watch? Diagnosing Character-Tracking Failures in Long-Form Video

    Jul 13, 2026Mohammad Al-Ratrout, Shayla Sharmin, Aditya Raikwar +1Visual Object TrackingVideo-Language Model Evaluation

  11. Each Judge Its Own Yardstick: Discovering Per-VLM Taxonomies for Physical Video Evaluation

    Jun 22, 2026Yu Cao, Ziquan Liu, Zhensong Zhang +3VLM EvaluationPhysical Consistency in Video Generation

    Queen Mary University of London · Huawei Darwin Research Center · Imperial College London

  12. Chehre: An Emoji-Prompted Dataset to Explore Perceptual Flexibility in Video Language Models

    Jun 19, 2026Bita Azari, Zoe Stanley, Avneet Batra +4VLM EvaluationVideo-Language Model Evaluation

    Simon Fraser University, Canada

  13. Streaming Interventions: Can Video Large Language Models Correct Mistakes as They Occur?

    Jun 8, 2026Apratim Bhattacharyya, Shweta Mahajan, Sanjay Haresh +5LLM Fine-TuningSynthetic Data Generation

    Qualcomm AI Research · York University · Vector Institute for AI

  14. VCIFBench: Evaluating Complex Instruction Following for Video Understanding

    Jun 3, 2026Huangchen Xu, Yuan Wu, Yi ChangVLM EvaluationVideo Understanding

    School of Artificial Intelligence, Jilin University · 2Engineering Research Center of Knowledge-Driven Human-Machine Intelligence, Jilin University · 3International Center of Future Science, Jilin University

  15. Pop-Up Distractions Reveal Bag-of-Events Behavior in Video Large Language Models

    May 26, 2026Oscar Chew, Serhii Honcharenko, Qian-Hui Chen +4Temporal Video GroundingVideo-Language Model Evaluation

    Texas A&M University · National Taiwan University · Stanford University +1

  16. When Vision Speaks for Sound

    May 13, 2026Xiaofei Wen, Wenjie Jacky Mo, Xingyu Fu +6Audio-Language Model EvaluationAudio-Visual Understanding

    dUniversity of California, Davis · pPrinceton University · wUniversity of Wisconsin–Madison +1

  17. EvoStreaming: Your Offline Video Model Is a Natively Streaming Assistant

    May 11, 2026Zichen Wen, Boxue Yang, Junlong Ke +5Streaming Video UnderstandingSelf-Training

    EPIC Lab, Shanghai Jiao Tong University · Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou) +1

  18. TOC-Bench: A Temporal Object Consistency Benchmark for Video Large Language Models

    May 11, 2026Junzhe Chen, Siyuan Meng, Yuxi Chen +3Video-Language Model EvaluationVideo-Language Models

    Tianjin University

  19. PushupBench: Your VLM is not good at counting pushups

    Apr 25, 2026Shengzhi Li, Jiarun Chen, Karun Sharma +2VLM EvaluationVideo-Language Model Evaluation

  20. VideoZeroBench: Probing the Limits of Video MLLMs with Spatio-Temporal Evidence Verification

    Apr 2, 2026Jiahao Meng, Yue Tan, Qi Xu +9Video-Language Model EvaluationLong-Video QA

    PKU · WHU · CASIA +2

  21. Stress Tests REVEAL Fragile Temporal and Visual Grounding in Video-Language Models

    Feb 11, 2026Sethuraman T, Savya Khosla, Aditi Tiwari +12VLM RobustnessVideo-Language Model Evaluation

    University of Illinois Urbana-Champaign, USA · Adobe Research, USA · Qualtrics, USA +6

  22. SONIC-O1: A Real-World Benchmark for Evaluating Multimodal Large Language Models on Audio-Video Understanding

    Jan 29, 2026Ahmed Y. Radwan, Christos Emmanouilidis, Hina Tabassum +2Audio-Language Model EvaluationMultimodal Grounding

    Vector Institute for Artificial Intelligence, MaRS Centre, Toronto, ON M5G 1L7, Canada · University of Groningen, Nijenborgh 4, 9747 AG Groningen, Netherlands · York University, 4700 Keele Street, Toronto, ON M3J 1P3, Canada

  23. NeMo: Needle in a Montage for Video-Language Understanding

    Sep 29, 2025Zi-Yuan Hu, Shuo Liang, Duo Zheng +10Temporal Video GroundingVideo-Language Model Evaluation

    The Chinese University of Hong Kong · Phoenix TV · Stanford University +2

  24. TempCore: Are Video QA Benchmarks Temporally Grounded?

    Sep 1, 2025Hyunjong Ok, Jaeho LeeVLM EvaluationVideo-Language Model Evaluation

    POSTECH

  25. VIBE: Annotation-Free Video-to-Text Information Bottleneck Evaluation for TL;DR

    May 23, 2025Shenghui Chen, Po-han Li, Sandeep Chinchali +1VLM EvaluationVideo Summarization

    The University of Texas at Austin