Multimodal misinformation increasingly mixes realistic im-age edits with fluent but misleading text, producing persuasive posts that are difficult to verify. Existing systems usually rely on a single evidence source. Content-based detectors identify local inconsistencies within an image and its caption but cannot determine global factual truth. Retrieval-based fact-checkers reason over external evidence but treat inputs as coarse claims and often miss subtle visual or textual manipulations. This separation creates failure cases where internally consistent fabrications bypass manipulation detectors and fact-checkers verify claims that contain pixel-level or token-level corruption. We present D-SECURE, a framework that combines internal manipulation detection with external evidence-based reasoning for news-style posts. D-SECURE integrates the HAMMER manipulation detector with the DEFAME retrieval pipeline. DEFAME performs broad verification, and HAMMER analyses residual or uncertain cases that may contain fine-grained edits. Experiments on DGM4 and ClaimReview samples highlight the complementary strengths of both systems and motivate their fusion. We provide a unified, explainable report that incorporates manipulation cues and external evidence.
Figures & tables
Fig. 1: Comparison of D-SECURE with existing work. (a)-(b) Existing systems. (c) D-SECURE
Method
Input
Internal
External
Local
NewsClippings [ 13 ]
Image–text
OOC consistency
None
No
COSMOS [ 14 ]
Image–text
Self-supervised co-occurrence
None
No
DGM 4 / HAMMER [ 15 , 16 ]
Image–text
Contrastive reasoning
None
Yes
MFND / SDML [ 17 ]
Image–text
Global content features
None
Limited
Hu et al. [ 18 ]
Text
Content features
KB graphs
No
Cao et al. [ 19 ]
Image–text
Multimodal GNN
KB graphs
Limited
TABLE I: Comparison of representative approaches to multimodal misinformation detection and fact-checking. “Internal” denotes explicit content-based manipulation reasoning. “External” denotes use of retrieved web or knowledge-base evidence. “Local” indicates explicit localisation of manipulations at region or token level.
Fig. 2: DGM 4 and ClaimReview2024+ examples. DGM 4 includes (a) pristine image–text pairs and four fine-grained local manipulation types: (b) Face Swap (FS), (c) Face Attribute (FA), (d) Text Swap (TS), and (e) Text Attribute (TA), as well as their mixed combinations (FS+TS, FS+TA, FA+TS, FA+TA). ClaimReview2024+ represents global factuality labels with (f) Supported, (g) Refuted, (h) Misleading and (i) Not-Enough-Information (NEI) cases.
Fig. 3: D-SECURE’s rule-based scheme for the DEFAME-HAMMER pipeline.
Fig. 4: D-SECURE’s fusion for the DEFAME-HAMMER pipeline.
Fig. 5: D-SECURE’s chat allowing the user to interact and question the LLM. (a) view for non-technical users, (b) view for technical users with evidence, (c) chat interaction between end user and the system.
DGM 4
ClaimReview2024+
DGM 4 +
VERITE
Fakeddit
Model
Strict
Manip.-Aware
Interv.-Aware
HAMMER
90.69 ± 0.00
44.46 ± 0.30
–
–
18.13 ± 0
41.75 ± 0
58.14 ± 0
DEFAME
64.08 ± 14.63
38.60 ± 4.56
41.2 ± 4
40 ± 4.96
78.24 ± 5.13
62.67 ± 9.22
79.2 ± 5.39
D-SECURE DEF-HAM
93.61 ± 1.3
30.8 ± 3.51
48.93 ± 1.74
51.73 ± 2.96
82 ± 3.87
61.64 ± 6.58
74.66 ± 4.49
D-SECURE HAM-DEF
93.61 ± 1.3
47.67 ± 2.29
48.93 ± 1.74
51.73 ± 2.96
82 ± 3.87
61.64 ± 6.58
74.66 ± 4.49
D-SECURE FUSE
82.39 ± 6.38
43.53 ± 3.29
56.67 ± 4.07
59.2 ± 3.4
86.03 ± 3.29
69.41 ± 3.13
78.06 ± 2.87
TABLE II: Unified results on DGM 4 and ClaimReview2024+. All values are computed directly from the submitted outputs.
Video misinformation increasingly operates at the semantic and evidential level: authentic footage may be selectively edited, temporally reordered, spliced across sources, or augmented with AI-generated content to construct false narratives. Such evidence-dependent manipulations cannot be reliably verified from the input video alone, because the missing, reordered, replaced, or recontextualized evidence lies outside the video itself. We introduce EVID-Bench, a benchmark for search-grounded video misinformation detection, where a system must search the open web for related videos and identify what information is false through cross-video comparison. EVID-Bench comprises 222 videos spanning 9 manipulation types across 3 categories: AI generation, single-source editing, and multi-source editing. All samples are verified to be undetectable by frontier models through visual inspection alone. We evaluate nine frontier multimodal models using a retrieval-augmented verification baseline. The best system achieves only 61.43% point-level accuracy and 43.24% video-level accuracy, while AI-generated manipulations remain especially challenging. Error analysis reveals recurring challenges: models fixate on irrelevant anchors, misattribute synthetic content to editorial splicing, and terminate search prematurely before fully explaining the manipulation.
Multimodal manipulation detection aims to simultaneously identify forged image--text pairs and localize tampered regions, yet existing methods typically rely on memorizing isolated artifacts and struggle with imperceptible manipulation traces or domain shifts. Inspired by human comparative reasoning, we reformulate this task as a reference-grounded verification problem, where authenticity is assessed by comparing a query against retrieved authentic evidence. We propose REVEAL Reference-Enabled Verification for Evidence Analysis and Localization), a framework explicitly designed for this comparative paradigm. To support this paradigm, we construct a large-scale reference library comprising 170K authentic news image--text pairs featuring over 40K public figures. Technically, REVEAL employs a difference-aware fusion mechanism to capture fine-grained discrepancies between the query and retrieved evidence. Furthermore, we introduce a task-decoupled Mixture-of-Experts (MoE) architecture to jointly execute instance-level detection and fine-grained grounding, effectively mitigating optimization conflicts between these heterogeneous objectives. Extensive experiments demonstrate that REVEAL significantly outperforms state-of-the-art methods, and notably enables training-free domain adaptation by simply updating the reference library, offering a robust and practical solution for detecting evolving misinformation. Code is available at https://anonymous.4open.science/r/REVEAL-Reference-A006.
Multimodal video misinformation detection is commonly formulated as a holistic video-understanding task, where the entire video and its associated content are processed and judged in a single pass. However, real-world misinformation often exhibits a sparse and compositional evidence structure: a reliable decision may depend on only a few coupled clues, while most video content contributes limited additional information. Exhaustive multimodal reasoning may therefore introduce substantial redundancy and obscure decisive evidence. This motivates decoupling evidence acquisition from verification: first identifying sparse, decision-relevant clues and then judging veracity based on the acquired evidence. Accordingly, we propose SIEVE, a framework for Sparse Interactive Evidence Verification via Extraction in multimodal video misinformation detection. An evidence-seeking agent actively explores the available multimodal evidence and constructs a compact evidence package, which is then used by a verifier to determine veracity. The agent is trained with supervised evidence-seeking trajectories and an evidence-aware reinforcement learning objective that promotes informative evidence acquisition while discouraging unnecessary or invalid interactions. Experiments on multiple video misinformation benchmarks show that SIEVE consistently outperforms the evaluated baselines and supports reliable verification using compact evidence packages. Moreover, the resulting acquisition process provides an explicit and inspectable evidence trail, improving the transparency and groundedness of multimodal misinformation detection.
Haochen Zhao, Yongxiu Xu, Xinkui Lin +6
Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · School of Cyber Security, University of Chinese Academy of Sciences, Beijing, China · Baidu, Beijing, China