cs.CVMay 28, 2026

DMC-CF: Dynamic Multimodal CounterFactual QA benchmark for Causal Reasoning

Authors: Junzhe ZhangHuixuan ZhangGuirong WangXingyao ZhangPei LiuLin QuHu WeiXiaojun Wan

Organizations: Wangxuan Institute of Computer Technology, Peking University · Alibaba Group

Abstract

With the rapid advancement of multimodal large language models (MLLMs), models have demonstrated increasingly powerful multimodal capabilities. However, whether MLLMs trained through statistical learning can truly understand the causal relationships underlying the real world remains a key research question. In recent years, numerous multimodal causal reasoning datasets have been proposed. Nevertheless, these datasets are either limited in scale or constructed from synthetic images and videos, cartoon-based content, or other non-realistic multimodal sources. To address these limitations, we collect real-world videos and construct DMC-CF-Static, a large-scale benchmark for multimodal causal counterfactual reasoning. Furthermore, to mitigate issues such as data contamination in traditional static evaluation, we represent causal events using causal graphs and propose the Dynamic Graph Intervention (DGI) framework to build the dynamic evaluation benchmark DMC-CF-Dynamic from DMC-CF-Static. Experimental results on the overall DMC-CF, which includes both static and dynamic evaluation benchmarks, demonstrate that the multimodal causal reasoning capabilities of current multimodal large language models in real-world scenarios still require substantial improvement.

Explore similar work

CardsList
  1. Which Modality Decides? Counterfactual Modality Attribution for Multimodal LLMs

    Jul 29, 2026Vahidin Hasic, Chao Wang, Luis C. Garcia-Peraza-Herrera +2Input ModalityModality Gap