Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection

Yuchen Zhang; Yaxiong Wang; Kecheng Han; Yujiao Wu; Lianwei Wu; Li Zhu; Zhedong Zheng

cs.CVMar 2, 2026

Cultivating Forensic Reasoning for Generalizable Multimodal Manipulation Detection

Authors: Yuchen Zhang, Yaxiong Wang, Kecheng Han, Yujiao Wu, Lianwei Wu, Li Zhu, Zhedong Zheng

Organizations: School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Information Engineering, Hefei University of Technology · CSIRO · Northwestern Polytechnical University ·

Abstract

Recent advances in generative AI have significantly enhanced the realism of multimodal media manipulation, thereby posing substantial challenges to manipulation detection. Existing manipulation detection and grounding approaches predominantly focus on manipulation type classification under result-oriented supervision, which not only lacks interpretability but also tends to overfit superficial artifacts. In this paper, we argue that generalizable detection requires incorporating explicit forensic reasoning, rather than merely classifying a limited set of manipulation types, which fails to generalize to unseen manipulation patterns. To this end, we propose REFORM, a reasoning-driven framework that shifts learning from outcome fitting to process modeling. REFORM adopts a three-stage curriculum that first induces forensic rationales, then aligns reasoning with final judgments, and finally refines logical consistency via reinforcement learning. To support this paradigm, we introduce ROM, a large-scale dataset with rich reasoning annotations. Extensive experiments show that REFORM establishes new state-of-the-art performance with superior generalization, achieving 81.52% ACC on ROM, 76.65% ACC on DGM4, and 74.9 F1 on MMFakeBench.

Figures & tables

Figure 1: Comparison between learning paradigms. (a) The prevailing Result-Oriented Supervision usually suffers from poor generalization by merely fitting statistical artifacts of training data. (b) Our Reasoning-Driven Optimization facilitates robust generalization by explicitly optimizing the forensic reasoning chain, enabling the model to uncover intrinsic inconsistencies effectively across unseen domains.

Figure 2: Overview of the ROM dataset. Left: Representative samples spanning 9 manipulated and 1 real categories, ranging from face-centric edits to scene-level synthesis, each accompanied by a detailed reasoning annotation. Right: Statistical distribution showing the diversity of manipulation types and the coverage of news media domains.

Figure 3: Probability Density of Token Count for Answer and Reasoning.

Figure 4: Overview of the REFORM framework and its three-stage training curriculum. (a) The primary pipeline employs a Cognitive Priming Encoder $\mathcal{E}_{p}$ and a Dual-Decoder structure, $\mathcal{D}_{r}$ and $\mathcal{D}_{a}$ , for reasoning-driven detection. (b) Cognitive Reasoning Warm-up via partial freezing. (c) Reasoning-Endowed Joint Fine-Tuning incorporating the Reason-Answer Consistency Loss $\mathcal{L}_{RAC}$ . (d) Constraint-Aware Policy Refinement using GRPO-based Reinforcement Learning to align forensic logic with the final verdict.

Setting	Method	Test Domain
Setting	Method	NYT			Guardian			USA			Wash.			BBC			AVG
		ACC	mAP	mIoU	ACC	mAP	mIoU	ACC	mAP	mIoU	ACC	mAP	mIoU	ACC	mAP	mIoU	ACC	mAP	mIoU
Zero-Shot	Yi-VL-6B ( 01.AI, 2024 )	31.03	21.86	2.67	20.13	19.18	1.14	19.65	12.76	1.72	26.15	25.47	1.79	29.67	17.38	1.11	25.33	19.33	1.69
	DeepSeek-VL2-27B ( Wu et al., 2024 )	45.74	34.01	8.21	39.76	27.67	7.13	31.78	36.91	7.21	29.25	29.03	6.90	33.01	29.62	4.03	35.91	31.45	6.70
	LLaVA-v1.6-34B ( Liu et al., 2023 )	47.66	41.09	7.45	38.54	31.12	6.76	32.89	37.27	6.43	31.09	31.23	6.78	36.00	29.58	4.12	37.24	34.06	6.31
	Qwen2.5-VL-72B ( Bai et al., 2025 )	50.74	42.24	12.79	40.18	32.70	11.08	33.64	37.60	10.99	35.11	34.29	10.28	37.11	30.51	7.69	39.35	35.47	10.56

Table 1: Comparison of multimodal learning methods on ROM, where the background gray indicates the intra-domain test. The better results are in bold . AVG refers to the average performance across 5 news domains.

Model	Language	Prompt	Validation (1000)				Test (10000)
Name	Model	Method	F1	Precision	Recall	ACC	F1	Precision	Recall	ACC
LVLMs with 7B Parameter
InstructBLIP ( Dai et al., 2023a )	Vicuna-7B ( Dai et al., 2023b )	Standard	14.7	30.8	13.2	8.1	16.1	40.5	14.2	8.8
Qwen-VL ( Bai et al., 2023 )	Qwen-7B ( Bai et al., 2023 )	Standard	43.6	50.6	44.9	60.3	44.0	51.6	45.2	60.5
PandaGPT ( Su et al., 2023 )	Vicuna-7B ( Dai et al., 2023b )	Standard	24.6	60.6	50.5	30.9	24.1	61.7	50.4	30.6
mPLUG-Owl2 ( Ye et al., 2025 )	LLaMA2-7B ( Touvron et al., 2023 )	Standard	47.2	64.9	52.3	70.6	48.7	71.1	53.3	71.4

Table 2: Comparison of zero-shot binary detection performance on the MMFakeBench validation and test sets. We compare our proposed REFORM with baseline models using “Standard” and “MMD-Agent” prompting paradigms as defined in the MMFakeBench paper. Baseline results are cited from the original MMFakeBench publication.

Method	Test Domain
Method	Guardian				USA				Wash.				BBC				AVG
	ACC	mAP	$\text{P}_{tok}$	mIoU	ACC	mAP	$\text{P}_{tok}$	mIoU	ACC	mAP	$\text{P}_{tok}$	mIoU	ACC	mAP	$\text{P}_{tok}$	mIoU	ACC	mAP	$\text{P}_{tok}$	mIoU
Fine-tuned LVLMs
Qwen2.5-3B ( Bai et al., 2025 )	61.57	37.36	61.35	40.19	60.65	34.25	70.19	33.28	51.79	32.23	63.23	35.11	62.20	38.02	61.22	41.23	59.05	35.47	64.00	37.45
LLaVa-v1.6-7B ( Liu et al., 2023 )	68.67	46.26	65.71	42.30	62.54	37.48	71.24	35.63	63.16	40.27	71.03	34.22	66.14	46.44	62.17	42.18	65.13	42.61	67.54	38.58
Deepfake Detectors

Table 3: Comparison of multimodal learning methods on DGM4 (%), where the guardian domain with background gray is intra-domain.

\text{P}_{tok}

is Precision of fake token grounding.

Panel of Table 4 — Table 4: Ablation of components (a), reason token length (b), $\eta$ sensitivity (c), reward (d), and completion (e-f).

Method	Params (M) $\downarrow$		Throughput (p/s) $\uparrow$
Method	Total	Trainable	Train	Inference
ViLT	121.07	121.07	1.85	2.38
HAMMER(++)	441.12	228.25	28.97	61.28
FKA-Owl	6771.98	33.55	1.25	1.33
MMD-Agent	34751.17	34447.66	-	0.02
AMD	276.95	276.95	5.55	13.38

Table 5: Efficiency comparison. We report total parameters and throughput (pairs/sec) on RTX 4090. Fast Mode refers to REFORM generating only prediction labels without the reasoning chain.

Table 6: Analysis on rationale faithfulness (a) and teacher robustness (b).

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: The prompt template for REFORM. The prompt strictly concatenates the system instruction, caption, options, and the localization instruction.

Panel of Figure 6 — Figure 6: The prompt template used to generate reasoning chains. The { Italicized Text } represents dynamic content filled based on the ground-truth labels and input data.

Panel of Figure 7 — Figure 7: The prompt template used for General-purpose Model. Note the change in coordinate format to normalized [x1,y1,x2,y2].

Panel of Figure 8 — Figure 8: The MMD-Agent prompt workflow. The agent sequentially performs Fact-Checking (utilizing external knowledge), Image Manipulation Detection (providing bounding boxes for faces), and Consistency Verification, before aggregating the status to determine the final authenticity.

Multimodal Deception Detection

See all 19 papers

Evidence-Grounded Forensic Reasoning for Detecting and Grounding Multi-Modal Media Manipulation
Aug 8, 2026Yichun Yeh, Yiheng Li, Xiaobo Hu +2Multimodal Grounding Digital Forensics
MAIS, CASIA; SAI, UCAS · SAI, UCAS; MAIS, CASIA
CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection
Jun 2, 2026Jinjie Shen, Yaxiong Wang, Yujiao Wu +5Fake News Detection Multimodal Large Language Models
School of Computer Science and Information Engineering, Hefei University of Technology, Hefei, China · Wuhan University, Wuhan, China · Lab for Intelligence and visiON (LION) +2
REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection
May 27, 2026Jun Zhou, Bingwen Hu, Yaxiong Wang +4Multimodal Grounding Image Forgery Detection

Aug 8, 2026J/K move · Enter open · S save

Evidence-Grounded Forensic Reasoning for Detecting and Grounding Multi-Modal Media Manipulation

Yichun Yeh, Yiheng Li, Xiaobo Hu, Zhen Lei+1

MAIS, CASIA; SAI, UCAS · SAI, UCAS; MAIS, CASIA

Multimodal Grounding Digital Forensics Multimodal Reasoning Multimodal Deception Detection

Open paper AskPDF arXiv

Fake news increasingly relies on cross-modal image-text forgeries, making transparent and verifiable reasoning chains an urgent need for Detecting and Grounding Multi-Modal Media Manipulation (DGM4). Existing methods produce black-box detection results without any decision rationale, limiting their reliability in forensic practice. Multi-modal Large Language Models (MLLMs) offer a natural path toward explainability, but applying them to DGM4 raises two difficulties. First, models tend to generate explanations disconnected from predicted evidence locations, producing unverified attribution. Second, enforcing evidence-conclusion consistency requires active optimization, yet uniform training signals fail to distinguish localization tokens from classification tokens, making multi-head joint training unreliable. We propose a multi-modal manipulation detector based on an Evidence-Grounded Forensic Reasoning (EFR) framework. EFR introduces an Anchor-and-Verify reasoning chain that enforces modality-isolated perception before cross-modal comparison, with conclusion coordinates as explicit anchors to which downstream evidence must spatially correspond. A verifiable reward system then enforces evidence-conclusion consistency during training, while a Modality-Decoupled Advantage (MDA) routing mechanism mitigats credit misassignment across prediction tasks. Experiments show that EFR achieves state-of-the-art performance while producing structured forensic reasoning records that explicitly bind explanations to evidence.

Abstract

Multimodal Deception Detection

Explore similar work

Evidence-Grounded Forensic Reasoning for Detecting and Grounding Multi-Modal Media Manipulation

CORE: Conflict-Oriented Reasoning for General Multimodal Manipulation Detection

REVEAL: Reference-Grounded Reasoning for Multimodal Manipulation Detection