cs.CVMar 26, 2026

Probabilistic Concept Graph Reasoning for Multimodal Misinformation Detection

Authors: Ruichao Yang, Wei Gao, Xiaobin Zhu, Jing Ma, Hongzhan Lin, Ziyang Luo, Bo-Wen Zhang, Xu-Cheng Yin

Organizations: University of Science and Technology Beijing · Singapore Management University · Hong Kong Baptist University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Multimodal misinformation poses an escalating challenge that often evades traditional detectors, which are opaque black boxes and fragile against new manipulation tactics. We present Probabilistic Concept Graph Reasoning (PCGR), an interpretable and evolvable framework that reframes multimodal misinformation detection (MMD) as structured and concept-based reasoning. PCGR follows a build-then-infer paradigm, which first constructs a graph of human-understandable concept nodes, including novel high-level concepts automatically discovered and validated by multimodal large language models (MLLMs), and then applies hierarchical attention over this concept graph to infer claim veracity. This design produces interpretable reasoning chains linking evidence to conclusions. Experiments demonstrate that PCGR achieves state-of-the-art MMD accuracy and robustness to emerging manipulation types, outperforming prior methods in both coarse detection and fine-grained manipulation recognition.

Figures & tables

Figure 1: Human vs. model reasoning on a misinformation example. Left : Conventional multimodal detectors fuse image-text features and may misclassify superficially aligned content (e.g., a fake bombing claim supported by a consistent but unrelated photo). Right : A concept-based reasoning process, used by human fact-checkers and our model, decomposes reasoning into concept-level checks. Each concept yields a soft judgment, collectively leading to the final verdict.
Figure 2: The Probabilistic Concept Graph Reasoning (PCGR) framework. PCGR organizes multimodal evidence into interpretable concept nodes and performs hierarchical probabilistic reasoning to infer claim veracity.
Figure 3: Statistics of three benchmark datasets.
Figure 4: Precision-Recall curves for MMD results on MMFakeBench and AMG (in-domain) and MiRAGeNews (OOD). Blue and green markers denote general-purpose MLLMs and task-specific detectors, respectively. Dashed gray lines indicate iso-F1 contours.
MethodMiRAGeNewsMMFakeBenchAMG
AccF1AccF1AccF1
General MLLMs
LLaVA-1.631.932.831.230.237.539.2
Qwen3-omni30.243.033.941.939.245.6
Llama 443.152.146.850.748.852.1
Gemini 2.5 Pro52.953.450.353.956.857.3
Table 1: Coarse-level MMD accuracy (%) and F1 across three benchmarks.
 MMFakeBenchAMG
 Mic-F1Mac-F1Mic-F1Mac-F1
LLaVA-1.624.924.528.625.7
Qwen3-omni25.622.730.627.3
Llama 428.725.835.330.6
Gemini 2.5 Pro46.148.353.851.4
GPT - 555.354.960.755.5
Table 2: Fine-grained detection performance on MMFakeBench (4 classes) and AMG (6 classes).
Figure 5: Ablation results for coarse-level and fine-grained detection on the AMG dataset.
Figure 6: A case study illustrating the hierarchical probabilistic aggregation process.
Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

MethodTraining timeInference time
MiRAGeNewsMMFakeBenchAMGMiRAGeNewsMMFakeBenchAMG
MGCA193mins300mins180mins20mins33mins18mins
PCGR220mins350mins178mins20mins32mins20mins
Appendix
Table 3: Training and inference time of MGCA and PCGR.
Figure 7: Precision and recall heatmap for fine-grained level detector results on MMFakeBench and AMG datasets. Score magnitudes are chromatically encoded, ranging from blue (lowest) to red (highest).
Figure 8: Impact of the max concepts constraint per layer.
Figure 9: Impact of the max layers constraint (hierarchical depth).
More in this topic

Graph Reasoning

Graph reasoning uses relationships among entities to solve multi-step inference, prediction, planning, and question-answering tasks.

See all 64 papers

Explore similar work

CardsList

  1. What Improves Multimodal Misinformation Detection? Answers from a Large-Scale Empirical Study

    Sep 24, 2026Akshit Sharma, Prashant W. PatilMultimodal FusionMultimodal Model Evaluation

    CVPR Lab, MFSDSAI Indian Institute of Technology Guwahati

  2. ReMMD: Realistic Multilingual Multi-Image Agentic Verification for Multimodal Misinformation Detection

    Jun 23, 2026Chenhao Dang, Dantong Zhu, Jun Yang +2Multimodal AgentsMultimodal Model Evaluation

    Shanghai Jiaotong University · Shanghai Artificial Intelligence Laboratory · Central South University +2

  3. MM-VeriAgent: Learning to Use Extensive Tools to Verify Multimodal Misinformation with Reinforcement Learning

    Sep 25, 2026Peipei Li, Shuhan Xia, Shengyang Liu +2Multimodal Deception DetectionMultimodal Tool Use

    Beijing University of Posts and Telecommunications · Minzu University of China · NLPR, Institute of automation, Chinese academy of science, Chinese Academy of Sciences