cs.CVNov 11, 2026

CineMME: From Dialogue Grounding to Narrative Reasoning in Multimodal Video Understanding

Authors: Jin Liu, Dawei Du, Yexiang Liu, Sijie Zhu, Fan Chen, Junxian Duan, Huaibo Huang, Ran He, 

Organizations: School of Information Science and Technology, ShanghaiTech University, Shanghai, China · MAIS & NLPR, Institute of Automation, Chinese Academy of Sciences, Beijing, China · ByteDance, San Jose, USA. · School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China

Abstract

Narrative video understanding requires more than recognizing visible events: models must extract evidence from dialogue, bind it to speakers, and use it to infer narrative meaning. Existing video benchmarks evaluate aspects of this problem, including video question answering, audio-visual reasoning, and temporal grounding, but rarely test whether these abilities form a coherent dialogue-to-plot evidence chain within the same clips. We introduce CineMME, a human-verified benchmark designed to diagnose dialogue-grounded plot understanding, supported by approximately 1,6001,600 expert person-hours of annotation. CineMME contains 975975 curated short narrative video clips and two linked tracks: CineMME-Grounding, with 13,55413,554 timestamped speech segments and 33,08333,083 speaker face boxes for temporal dialogue localization, transcription, and active-speaker grounding; and CineMME-Reasoning, with 1,6001,600 multiple-choice and 316316 open-ended questions spanning perception, audio, narrative, social, and scene-presentation understanding. Our main evaluation compares 2020 MLLMs on reasoning and 99 omni-modal models on grounding. The results reveal a pronounced who-what-when decoupling: Gemini-3 Pro localizes dialogue in time (68.34%68.34\% tIoU) and transcribes it with a WER of 28.99%28.99\%, yet achieves only 7.74%7.74\% vIoU for active-speaker grounding while achieving 67.66%67.66\% MCQ accuracy on questions with extractable answers. Modality ablations further show that vision is not uniformly beneficial: it improves performance on perception-heavy questions but can hurt dialogue-centric reasoning. CineMME provides a compact diagnostic testbed for evaluating how multimodal video models connect dialogue, speakers, visual evidence, and narrative meaning.

Figures & tables

Appendix figures & tables28 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. NarrativeTrack: Evaluating Entity-Centric Reasoning for Narrative Understanding

    Jan 3, 2026Hyeonjeong Ha, Jinjin Ge, Bo Feng +2VLM EvaluationVideo Understanding

  2. StoryTR: Narrative-Centric Video Temporal Retrieval with Theory of Mind Reasoning

    Apr 25, 2026Xuanyue Zhong, Yuqiang Xie, Guanqun Bi +2Temporal Video GroundingVideo Understanding

  3. Video-HolmesV2: Can MLLMs Reason with Spatio-Temporal Audio-Visual Evidence in Long Videos?

    Sep 15, 2026Zhaoyang Wei, Zipeng Wang, Yushe Cao +10Audio-Visual UnderstandingVideo-Language Model Evaluation