Video-Language Model Evaluation

Ask about this topic

Momentum

18 papers in the last four weeks, up 100% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28
View on map

Latest papers 55

All topics
CardsList

  1. Stable Scores, Unstable Answers: Frame Phase and Option Order in Video Multiple-Choice Evaluation

    Oct 6, 2026Lichen Zhu, Yiheng Wang, Yueqian Lin +2Video-Language Model EvaluationVideo QA

    Duke University, Durham, NC, USA

  2. Have I Seen Enough? Frozen Video-Language Models Encode Evidence Readiness

    Oct 6, 2026Dan Ben-Ami, Kobi Cohen, Chaim BaskinStreaming Video UnderstandingVideo-Language Model Evaluation

    INSIGHT Lab, Ben-Gurion University of the Negev, Israel · Ben-Gurion University of the Negev, Israel

  3. InteractionBench: A Real-Time Interaction Benchmark for Streaming Video Systems

    Oct 5, 2026Enxin Song, Suhao Yu, Yifei Xu +7Streaming Video UnderstandingVideo-Language Model Evaluation

    University of Pennsylvania · Stanford University · University of Illinois at Urbana-Champaign +3

  4. Video-Index: A Curated Meta-Benchmark for Video Understanding

    Oct 1, 2026Enxin Song, Yinuo Xu, Shusheng Yang +2Video UnderstandingVideo-Language Model Evaluation

    University of Pennsylvania · New York University · Princeton University

  5. Paying for Too Many Tokens? Valid and Cost-Efficient Multimodal LLM Annotation with Simple Heuristics

    Sep 30, 2026Zhixi Zhu, Kristina GligoricVLM EvaluationVideo-Language Model Evaluation

    Johns Hopkins University

  6. SYNCR: Diagnosing and Learning Cross-Video Reasoning from Simulation

    Sep 29, 2026Sara Ghazanfari, Siddharth Garg, Prashanth Krishnamurthy +1Video-Language Model EvaluationVideo Reasoning

    New York University

  7. Beyond Binary Preferences: Graded Preference Optimization for Limb-Motion Captioning

    Sep 29, 2026Yanan Wang, Tingsong Li, Kaixun Jiang +3Direct Preference OptimizationVideo-Language Model Evaluation

    Zhejiang University · Alibaba Token Hub, Alibaba Group · University of Science and Technology of China +2

  8. ActionLens: Diagnosing Spatial-Temporal Binding Failures in Vision-Language Models

    Sep 28, 2026Gueter Josmy Faure, Min-Hung Chen, Hao Ping Wang +3Video-Language Model EvaluationTemporal Video Understanding

    National Taiwan University · NVIDIA

  9. PlaylistEval: Can Video-Language Judges Be Trusted at Day Scale and Beyond?

    Sep 28, 2026Shayekh Bin Islam, Hwanjun SongLLM-as-a-JudgeVideo-Language Model Evaluation

    KAIST

  10. Beneath the Scores: Rethinking Hallucination Evaluation for Video Understanding Models

    Sep 24, 2026Shuzhi Gong, Fengze Sun, Yuansan LiuTemporal Video GroundingVideo-Language Model Evaluation

    The University of Melbourne

  11. CinematicVQA: Benchmarking Film-Grammar Reasoning in Large Vision-Language Models

    Sep 23, 2026Shuo Xing, Pooja Verlani, Balu Adsumilli +1Video UnderstandingVideo-Language Model Evaluation

    Texas A&M University, College Station, USA · Google Inc., Mountain View, USA

  12. Can Vision-Language Models Analyze Human-Centered Video? Mapping Model Capabilities and Human-AI Collaborative Workflows

    Sep 23, 2026Xiyuan Shen, Jiuyang Lyu, Seokhyun Hwang +4Video UnderstandingVideo-Language Model Evaluation

    University of Washington, USA · Georgia Institute of Technology, USA · Google DeepMind, USA

  13. A Hierarchy-Aware Video-Language Model Evaluation and Hyperbolic Baseline for Surgery

    Sep 22, 2026Ana Manzano Rodríguez, Pascal Mettes, Marlies P. Schijven +1Video-Language Model EvaluationHierarchical Representation Learning

    Data Science Center HAVA-Lab, University of Amsterdam · Video & Image Sense Lab, Informatics Institute, University of Amsterdam · Amsterdam UMC Location University of Amsterdam, Surgery +2

  14. Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?

    Sep 15, 2026Zhaoyang Wei, Zipeng Wang, Yushe Cao +10Audio-Visual UnderstandingVideo-Language Model Evaluation

    University of Chinese Academy of Sciences, China · Tsinghua University, China · Tencent, China

  15. Not Another Text Benchmark: Putting the "Visual" Back in Visual Question Answering for Large Video Models

    Sep 15, 2026Rwiddhi Chakraborty, Yinong, Wang +7Video-Language Model EvaluationVideo QA

    Oliver · University of Copenhagen · Carnegie Mellon University +4

  16. DementiaCare-Bench: A Modality-Validated Video Benchmark

    Sep 14, 2026Afrouz Sheikholeslami, Yuankai Qi, Xuyun Zhang +4Alzheimer's DiseaseHealthcare

    School of Computing, Macquarie University, Sydney, NSW, Australia · School of Electrical and Computer Engineering, University of Sydney, Sydney, NSW, Australia · University of California, Merced, CA, USA

  17. LiveProBench: Can Streaming Video Models Really Interact Like Humans?

    Sep 11, 2026Kaixuan Du, Xin Wan, Hang Zhang +4Streaming Video UnderstandingVideo-Language Model Evaluation

    School of Automation Science and Electrical Engineering, Beihang University · Qwen Business Unit of Alibaba

  18. Can We Trust Video Hallucination Detectors? VidHalLoc for Evaluating the Evaluators

    Sep 9, 2026Xinyu Chen, Adnan Mahmood, Mark DrasHallucination Detection in VLMsHallucination Detection

  19. MotionBlind: Probing the Illusion of Motion Understanding in Video-LLMs

    Sep 8, 2026Dhairya Bhatia, Bishoy Galoaa, Oliver Fritsche +7Video-Language Model EvaluationVideo-Language Models

    Electrical and Computer Engineering Department Northeastern University, Boston, MA, USA

  20. Kairos: A Dataset for Fine-Grained Video-Language Modeling over Space, Time, and Dynamics

    Sep 8, 2026Ruibo Ming, Lei Sun, Deheng Zhang +8Video-Language Model EvaluationVideo-Language Models

    1INSAIT, Sofia University “St. Kliment Ohridski” · 2Adobe Research · 3Snap Research

  21. Do Input-Level Defenses Transfer to Observation-Level Attacks on VideoLLMs?

    Sep 8, 2026Bangshuo Zhu, Wei Song, Yuxin Cao +4VLM RobustnessAdversarial Attacks on VLMs

    School of Computer Science and Engineering, University of New South Wales, New South Wales 2052, Australia · School of Information and Communication Technology, Griffith University, Queensland 4111, Australia · School of Computing, National University of Singapore, Singapore 117417, Singapore +2

  22. Select, Compress, Reinvest: A Controlled Study of Visual-Token Allocation in Long-Video MLLMs

    Sep 3, 2026Prakhar KhatriVideo-Language Model EvaluationLong-Video Understanding

    Independent Researcher

  23. TempCloze: Can Video-LLMs Identify the Missing Middle?

    Sep 1, 2026Wenqi Pei, Henry Hengyuan Zhao, Yilai Liu +4Video-Language Model EvaluationVideo-Language Models

    The University of Hong Kong · National University of Singapore · Peking University

  24. ViTAL-X: Video-Text Alignment with Cross-Modal Temporal Edits

    Sep 1, 2026Sethuraman T, Savya Khosla, Onkar Kishor Susladkar +6Video-Language Model EvaluationVideo-Language Models

    University of Illinois Urbana-Champaign · Adobe Research

  25. TAKE 85: Testing Audiovisual filmmaKer's intEnt across 85 Hours of Film

    Aug 30, 2026Kaishuu Shinozaki-Conefrey, Olivier Pascaud, Robin Courant +3Audio-Visual UnderstandingVideo-Language Model Evaluation

    LIX, Ecole Polytechnique, IP Paris, Palaiseau, France · New York University, New York, NY, USA · Ecole nationale supérieure Louis-Lumière, Saint-Denis, France +1

  26. DVBench: Benchmarking MLLMs for Understanding Dynamic Charts and Narratives in Data Videos

    Aug 30, 2026Bomiao Wang, Zekai Shao, Jiexiang Lan +3Video UnderstandingVideo-Language Model Evaluation

    Fudan University · The Hong Kong University of Science and Technology (Guangzhou)

  27. NARU: A Benchmark for NARrative Evolution and Cultural Nuance Understanding in Japanese Extreme Long Video

    Aug 13, 2026Yuheng Huang, Jianlang Chen, Jiayang Song +6Video UnderstandingVideo-Language Model Evaluation

    The University of Tokyo, Tokyo, Japan · Kyushu University, Fukuoka, Japan · Macau University of Science and Technology, Macau, China +2

  28. EgoMonth: A Month-Level Egocentric Video Benchmark for Long-Term Spatiotemporal Memory

    Aug 13, 2026Weitao Chen, Hu Jiaxin, Xie Tianyidan +15Egocentric Video QAVideo-Language Model Evaluation

    Nanjing University, China · Huawei Technologies Co., Ltd., China · Tianjin University, China

  29. From Detection to Understanding: TAR and TAR-Bench for Multi-Task Traffic Anomaly Reasoning

    Aug 10, 2026Han Zhang, Yilin Zhao, Zaid Pervaiz Bhat +5Video-Language Model EvaluationVideo Anomaly Detection

    NVIDIA · Santa Clara University

  30. VideoVIBE: A Video-Grounded Diagnostic Benchmark for One-Shot Interactive Website Generation

    Aug 10, 2026Jiajun Xu, Yanghao Zhou, Jingyun Liao +6Web Application GenerationVideo-Language Model Evaluation

    University of Technology Sydney · Beijing Institute of Technology · Hunan University +4