Organizations: School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Information Engineering, Hefei University of Technology · CSIRO · Northwestern Polytechnical University · University of Macau
Recent advances in generative AI have significantly enhanced the realism of multimodal media manipulation, thereby posing substantial challenges to manipulation detection. Existing manipulation detection and grounding approaches predominantly focus on manipulation type classification under result-oriented supervision, which not only lacks interpretability but also tends to overfit superficial artifacts. In this paper, we argue that generalizable detection requires incorporating explicit forensic reasoning, rather than merely classifying a limited set of manipulation types, which fails to generalize to unseen manipulation patterns. To this end, we propose REFORM, a reasoning-driven framework that shifts learning from outcome fitting to process modeling. REFORM adopts a three-stage curriculum that first induces forensic rationales, then aligns reasoning with final judgments, and finally refines logical consistency via reinforcement learning. To support this paradigm, we introduce ROM, a large-scale dataset with rich reasoning annotations. Extensive experiments show that REFORM establishes new state-of-the-art performance with superior generalization, achieving 81.52% ACC on ROM, 76.65% ACC on DGM4, and 74.9 F1 on MMFakeBench.
Figures & tables
Figure 1: Comparison between learning paradigms. (a) The prevailing Result-Oriented Supervision usually suffers from poor generalization by merely fitting statistical artifacts of training data. (b) Our Reasoning-Driven Optimization facilitates robust generalization by explicitly optimizing the forensic reasoning chain, enabling the model to uncover intrinsic inconsistencies effectively across unseen domains.
Figure 2: Overview of the ROM dataset. Left: Representative samples spanning 9 manipulated and 1 real categories, ranging from face-centric edits to scene-level synthesis, each accompanied by a detailed reasoning annotation. Right: Statistical distribution showing the diversity of manipulation types and the coverage of news media domains.
Figure 3: Probability Density of Token Count for Answer and Reasoning.
Figure 4: Overview of the REFORM framework and its three-stage training curriculum. (a) The primary pipeline employs a Cognitive Priming Encoder Ep and a Dual-Decoder structure, Dr and Da , for reasoning-driven detection. (b) Cognitive Reasoning Warm-up via partial freezing. (c) Reasoning-Endowed Joint Fine-Tuning incorporating the Reason-Answer Consistency Loss LRAC . (d) Constraint-Aware Policy Refinement using GRPO-based Reinforcement Learning to align forensic logic with the final verdict.
Setting
Method
Test Domain
NYT
Guardian
USA
Wash.
BBC
AVG
ACC
mAP
mIoU
ACC
mAP
mIoU
ACC
mAP
mIoU
ACC
mAP
mIoU
ACC
mAP
mIoU
ACC
mAP
mIoU
Zero-Shot
Yi-VL-6B ( 01.AI, 2024 )
31.03
21.86
2.67
20.13
19.18
1.14
19.65
12.76
1.72
26.15
25.47
1.79
29.67
17.38
1.11
25.33
19.33
1.69
DeepSeek-VL2-27B ( Wu et al., 2024 )
45.74
34.01
8.21
39.76
27.67
7.13
31.78
36.91
7.21
29.25
29.03
6.90
33.01
29.62
4.03
35.91
31.45
6.70
LLaVA-v1.6-34B ( Liu et al., 2023 )
47.66
41.09
7.45
38.54
31.12
6.76
32.89
37.27
6.43
31.09
31.23
6.78
36.00
29.58
4.12
37.24
34.06
6.31
Qwen2.5-VL-72B ( Bai et al., 2025 )
50.74
42.24
12.79
40.18
32.70
11.08
33.64
37.60
10.99
35.11
34.29
10.28
37.11
30.51
7.69
39.35
35.47
10.56
Table 1: Comparison of multimodal learning methods on ROM, where the background gray indicates the intra-domain test. The better results are in bold . AVG refers to the average performance across 5 news domains.
Model
Language
Prompt
Validation (1000)
Test (10000)
Name
Model
Method
F1
Precision
Recall
ACC
F1
Precision
Recall
ACC
LVLMs with 7B Parameter
InstructBLIP ( Dai et al., 2023a )
Vicuna-7B ( Dai et al., 2023b )
Standard
14.7
30.8
13.2
8.1
16.1
40.5
14.2
8.8
Qwen-VL ( Bai et al., 2023 )
Qwen-7B ( Bai et al., 2023 )
Standard
43.6
50.6
44.9
60.3
44.0
51.6
45.2
60.5
PandaGPT ( Su et al., 2023 )
Vicuna-7B ( Dai et al., 2023b )
Standard
24.6
60.6
50.5
30.9
24.1
61.7
50.4
30.6
mPLUG-Owl2 ( Ye et al., 2025 )
LLaMA2-7B ( Touvron et al., 2023 )
Standard
47.2
64.9
52.3
70.6
48.7
71.1
53.3
71.4
Table 2: Comparison of zero-shot binary detection performance on the MMFakeBench validation and test sets. We compare our proposed REFORM with baseline models using “Standard” and “MMD-Agent” prompting paradigms as defined in the MMFakeBench paper. Baseline results are cited from the original MMFakeBench publication.
Method
Test Domain
Guardian
USA
Wash.
BBC
AVG
ACC
mAP
Ptok
mIoU
ACC
mAP
Ptok
mIoU
ACC
mAP
Ptok
mIoU
ACC
mAP
Ptok
mIoU
ACC
mAP
Ptok
mIoU
Fine-tuned LVLMs
Qwen2.5-3B ( Bai et al., 2025 )
61.57
37.36
61.35
40.19
60.65
34.25
70.19
33.28
51.79
32.23
63.23
35.11
62.20
38.02
61.22
41.23
59.05
35.47
64.00
37.45
LLaVa-v1.6-7B ( Liu et al., 2023 )
68.67
46.26
65.71
42.30
62.54
37.48
71.24
35.63
63.16
40.27
71.03
34.22
66.14
46.44
62.17
42.18
65.13
42.61
67.54
38.58
Deepfake Detectors
Table 3: Comparison of multimodal learning methods on DGM4 (%), where the guardian domain with background gray is intra-domain. Ptok is Precision of fake token grounding.
Table 4: Ablation of components (a), reason token length (b), η sensitivity (c), reward (d), and completion (e-f).
Method
Params (M) ↓
Throughput (p/s) ↑
Total
Trainable
Train
Inference
ViLT
121.07
121.07
1.85
2.38
HAMMER(++)
441.12
228.25
28.97
61.28
FKA-Owl
6771.98
33.55
1.25
1.33
MMD-Agent
34751.17
34447.66
-
0.02
AMD
276.95
276.95
5.55
13.38
Table 5: Efficiency comparison. We report total parameters and throughput (pairs/sec) on RTX 4090. Fast Mode refers to REFORM generating only prediction labels without the reasoning chain.
Table 6: Analysis on rationale faithfulness (a) and teacher robustness (b).
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: The prompt template for REFORM. The prompt strictly concatenates the system instruction, caption, options, and the localization instruction.
Figure 6: The prompt template used to generate reasoning chains. The { Italicized Text } represents dynamic content filled based on the ground-truth labels and input data.
Figure 7: The prompt template used for General-purpose Model. Note the change in coordinate format to normalized [x1,y1,x2,y2].
Figure 8: The MMD-Agent prompt workflow. The agent sequentially performs Fact-Checking (utilizing external knowledge), Image Manipulation Detection (providing bounding boxes for faces), and Consistency Verification, before aggregating the status to determine the final authenticity.
Fake news increasingly relies on cross-modal image-text forgeries, making transparent and verifiable reasoning chains an urgent need for Detecting and Grounding Multi-Modal Media Manipulation (DGM4). Existing methods produce black-box detection results without any decision rationale, limiting their reliability in forensic practice. Multi-modal Large Language Models (MLLMs) offer a natural path toward explainability, but applying them to DGM4 raises two difficulties. First, models tend to generate explanations disconnected from predicted evidence locations, producing unverified attribution. Second, enforcing evidence-conclusion consistency requires active optimization, yet uniform training signals fail to distinguish localization tokens from classification tokens, making multi-head joint training unreliable. We propose a multi-modal manipulation detector based on an Evidence-Grounded Forensic Reasoning (EFR) framework. EFR introduces an Anchor-and-Verify reasoning chain that enforces modality-isolated perception before cross-modal comparison, with conclusion coordinates as explicit anchors to which downstream evidence must spatially correspond. A verifiable reward system then enforces evidence-conclusion consistency during training, while a Modality-Decoupled Advantage (MDA) routing mechanism mitigats credit misassignment across prediction tasks. Experiments show that EFR achieves state-of-the-art performance while producing structured forensic reasoning records that explicitly bind explanations to evidence.
The rapid rise of generative AI has made multimodal fake news increasingly realistic and pervasive, posing severe threats to public trust and social stability. Existing detection methods rely heavily on manipulation-specific models and large-scale labeled data, resulting in poor generalization to emerging manipulation types. We observed that the essence of manipulated misinformation lies in its intrinsic conflicts, i.e., semantic or physical inconsistencies either across modalities or with common world knowledge. Inspired by this observation, we propose Conflict-Oriented REasoning (CORE) framework, an effective paradigm that learns to endows multimodal large language models (MLLMs) with explicit conflict-capturing capability. To this end, CORE first constructs the Conflict Attribution Corpus (CAC) with fine-grained annotations of conflict factors and sources, providing essential data support for subsequent conflict perception training. By performing conflict-oriented representation enhancement and reasoning based on CAC, CORE achieves robust and generalizable conflict detection, effectively and rapidly adapting to unseen manipulation types with a few samples or in even zero-shot settings. Extensive experiments demonstrate that CORE surpasses state-of-the-art models. The dataset and code are publicly available at https://github.com/shen8424/CORE.
Jinjie Shen, Yaxiong Wang, Yujiao Wu +5
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · Wuhan University, Wuhan, China · Lab for Intelligence and visiON (LION) +2
Multimodal manipulation detection aims to simultaneously identify forged image--text pairs and localize tampered regions, yet existing methods typically rely on memorizing isolated artifacts and struggle with imperceptible manipulation traces or domain shifts. Inspired by human comparative reasoning, we reformulate this task as a reference-grounded verification problem, where authenticity is assessed by comparing a query against retrieved authentic evidence. We propose REVEAL Reference-Enabled Verification for Evidence Analysis and Localization), a framework explicitly designed for this comparative paradigm. To support this paradigm, we construct a large-scale reference library comprising 170K authentic news image--text pairs featuring over 40K public figures. Technically, REVEAL employs a difference-aware fusion mechanism to capture fine-grained discrepancies between the query and retrieved evidence. Furthermore, we introduce a task-decoupled Mixture-of-Experts (MoE) architecture to jointly execute instance-level detection and fine-grained grounding, effectively mitigating optimization conflicts between these heterogeneous objectives. Extensive experiments demonstrate that REVEAL significantly outperforms state-of-the-art methods, and notably enables training-free domain adaptation by simply updating the reference library, offering a robust and practical solution for detecting evolving misinformation. Code is available at https://anonymous.4open.science/r/REVEAL-Reference-A006.