ARISE: Authentic Reference-Informed Stability Enhancement for Continual Multimodal Forgery Detection
Authors: Yuchen Zhang, Yaxiong Wang, Yujiao Wu, Lianwei Wu, Qi Zhang, Li Zhu, Chaofeng Dong, Le Wu, Zhedong Zheng
Organizations: School of Software Engineering, Xi’an Jiaotong University · Xingchen AGI Lab · School of Computer and Information Engineering, Hefei University of Technology · CSIRO · Northwestern Polytechnical University · Tongji University · University of Macau
Multimodal forgery detection is commonly studied under a static assumption: training attacks are expected to approximate those encountered after deployment. This assumption becomes increasingly fragile as generators, editing systems, and multimodal large language models continually introduce new forgery mechanisms and forensic evidence. We formulate this problem as attack-incremental continual multimodal forgery detection, where the binary real/fake interface remains fixed while emerging attacks must be learned without forgetting previous ones. We introduce Fable-Bench, which organizes attack evolution through complementary joint-modality, text-side, and image-side protocols. We further propose ARISE (Authentic Reference-Informed Stability Enhancement), motivated by the open-ended forged class and the comparatively reusable geometry of authentic content. ARISE fits a real-only Gaussian mixture model to projected authentic replay representations as a compact historical reference. Meanwhile, the proposed Gaussian-Responsibility-Aware Stability Preservation (GRASP) derives a sample-adaptive anisotropic metric that strongly constrains historically stable representation directions while retaining plasticity elsewhere. Our Vulnerability-Aware Replay (VAR) further estimates stage-wise vulnerability from authentic-density degradation, forged-to-authentic boundary confusion, and authentic-mode occupancy drift, then assigns bounded replay emphasis to fragile stages. All adaptive signals are obtained from training-time replay statistics, while inference uses only the original multimodal backbone and binary classification head. Experiments across three attack-evolution protocols and multiple detector architectures demonstrate improved historical retention and final average performance over representative continual-learning baselines.
Figures & tables
Figure 1. Conceptual motivation for attack-incremental forgery detection. (a) A detector trained once on an initial attack-space snapshot is frozen after deployment. As new forgery mechanisms emerge, the resulting distribution shifts alter forensic evidence and could substantially degrade detection performance. (b) We instead sequentially update the detector with newly available labeled attack data. After each stage, the current detector is evaluated on all attacks observed so far. A two-panel conceptual comparison of static and continual multimodal forgery detection. The left panel shows a detector trained on an initial attack and then frozen, causing its accuracy to decline as previously unseen attacks emerge. The right panel shows a single detector being updated sequentially with newly available attack data and evaluated on all attacks observed so far, with the goal of learning new attacks while retaining previous detection capabilities.
Figure 2. Overview of Fable-Bench and its attack-evolution protocols. Left: 821K image–text pairs spanning 30+ attack mechanisms. Right: representative samples from (P1) joint-modality, (P2) text-side, and (P3) image-side evolution. Overview of Fable-Bench and its three attack-evolution protocols. A radial chart summarizes 821 thousand image–text pairs and more than 30 attack mechanisms. Representative examples illustrate joint-modality evolution from out-of-context repurposing to advanced multimodal forgery, text-side evolution from random mismatch to coherent MLLM-generated deception, and image-side evolution from copy–move editing to full-image synthesis.
Figure 3. Overview of Arise . (a) A frozen copy of the previous detector and a real-only Gaussian mixture provide historical anchors during continual updates. (b) GRASP derives a responsibility-aware anisotropic preservation metric from the local geometry of the authentic reference. (c) VAR estimates stage-wise vulnerability and accordingly reweights historical replay. All auxiliary components are removed from the inference path. Overview of ARISE. The trainable multimodal detector receives current-stage and replay data, while a frozen copy of the previous detector and a real-only Gaussian mixture model provide historical authentic anchors. GRASP derives responsibility-aware anisotropic constraints for representation preservation, and VAR estimates stage-wise vulnerability to reweight historical replay losses.
Method
Joint-Modality Protocol (P1)
Text-Side Protocol (P2)
Image-Side Protocol (P3)
T1
T2
T3
T4
T5
Avg.
T1
T2
T3
T4
Avg.
T1
T2
T3
T4
Avg.
MLLM-based Baselines
GPT-5.4-mini ( OpenAI, 2026 )
92.20
64.10
69.40
62.30
62.10
70.02
91.70
64.20
60.50
59.40
68.95
61.20
58.40
68.10
73.30
65.25
Gemini-3-Flash ( Google DeepMind, 2025 )
93.10
65.30
70.80
63.50
63.20
71.18
92.20
65.00
61.40
60.32
69.73
63.00
60.20
70.60
75.40
67.30
Qwen3.5-Plus ( Qwen Team, 2026 )
90.59
61.26
66.67
59.46
59.46
67.49
90.09
61.26
56.76
55.86
65.99
57.66
54.95
63.96
69.17
61.44
InternVL3.5-30B ( Wang et al., 2025b )
88.80
59.40
64.10
57.20
57.60
65.42
87.60
58.30
54.10
52.90
63.23
59.80
57.10
65.60
71.18
63.42
Table 1. Accuracy (%) comparison under the joint-modality (P1), text-side (P2), and image-side (P3) attack-evolution protocols. The best and second-best results among continual-learning methods are shown in bold and underlined , respectively. MLLM-based baselines and reference baselines are excluded from ranking.
Method
Weibo ( Jin et al., 2017 )
FineFake ( Zhou et al., 2026b )
T1
T2
T3
T4
Avg.
T1
T2
T3
T4
Avg.
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Architecture-based Methods
MoE-CLIP ( Yu et al., 2024 )
92.44
92.26
91.16
90.40
92.53
92.01
94.65
94.45
92.70
92.28
73.58
65.00
61.00
58.93
60.46
53.07
87.69
76.48
70.68
63.37
D-MoLE ( Ge et al., 2025 )
92.68
92.09
92.49
92.78
94.17
94.35
97.65
98.23
94.25
94.36
76.69
73.19
71.75
70.21
66.06
60.95
86.00
78.80
75.12
70.79
DAEDCMD ( Wang et al., 2025a )
93.01
93.12
94.79
92.68
93.77
93.99
100.00
100.00
95.39
94.95
75.82
73.57
72.38
70.26
68.23
63.93
88.61
75.22
76.26
70.74
Table 2. Cross-benchmark evaluation on the public Weibo and FineFake datasets. Avg. denotes the average over the four incremental tasks. The best and second-best results are marked in bold and underlined , respectively.
Figure 4. Overall effectiveness and plug-and-play generalization of Arise . (a) Plasticity–stability trade-off across continual learning methods, where bubble size represents the number of parameters. (b)–(c) Plug-and-play performance with CLIP and ViLT backbones in terms of Accuracy and Macro-F1, respectively. Three plots summarizing the effectiveness and transferability of ARISE. The first plot compares the plasticity–stability trade-off of continual learning methods, with bubble size indicating model parameter count. The other two plots report Accuracy and Macro-F1 performance when ARISE is integrated with CLIP and ViLT backbones, compared with naive fine-tuning and experience replay.
Variant
G
V
Avg. ACC (%) ↑
Δ (pp)
P1
P2
P3
Base (ER)
–
–
81.70
77.67
83.57
–
Base + GRASP
✓
–
83.64
80.14
88.93
+3.26
Base + VAR
–
✓
82.12
79.13
88.29
+2.20
Arise
✓
✓
85.37
81.22
90.47
+4.71
Table 3. Ablations of Arise . All Avg. metrics are macro-averaged over P1–P3, and Δ denotes the ACC gain over ER in percentage points. Higher is better except for AF; the best and second-best results are bolded and underlined.
As generative artificial intelligence evolves, deepfake attacks have escalated from single-modality manipulations to complex, multimodal threats. Existing forensic techniques face a severe generalization bottleneck: by relying excessively on superficial, modality-specific artifacts, they neglect the shared latent forgery knowledge hidden beneath variable physical appearances. Consequently, these models suffer catastrophic performance degradation when confronted with unseen "dark modalities." To break this limitation, this paper introduces a paradigm shift that redefines multimodal forensics from conventional "feature fusion" to "modality generalization." We propose the first modality-agnostic forgery (MAF) detection framework. By explicitly decoupling modality-specific styles, MAF precisely extracts the essential, cross-modal latent forgery knowledge. Furthermore, we define two progressive dimensions to quantify model generalization: transferability toward semantically correlated modalities (Weak MAF), and robustness against completely isolated signals of "dark modality" (Strong MAF). To rigorously assess these generalization limits, we introduce the DeepModal-Bench benchmark, which integrates diverse multimodal forgery detection algorithms and adapts state-of-the-art generalized learning methods. This study not only empirically proves the existence of universal forgery traces but also achieves significant performance breakthroughs on unknown modalities via the MAF framework, offering a pioneering technical pathway for universal multimodal defense.
Jingtong Dou, Chuancheng Shi, Jian Wang +3
The University of Sydney · Nanjing University of Posts and Telecommunications · National University of Singapore
Recent advances in generative AI have significantly enhanced the realism of multimodal media manipulation, thereby posing substantial challenges to manipulation detection. Existing manipulation detection and grounding approaches predominantly focus on manipulation type classification under result-oriented supervision, which not only lacks interpretability but also tends to overfit superficial artifacts. In this paper, we argue that generalizable detection requires incorporating explicit forensic reasoning, rather than merely classifying a limited set of manipulation types, which fails to generalize to unseen manipulation patterns. To this end, we propose REFORM, a reasoning-driven framework that shifts learning from outcome fitting to process modeling. REFORM adopts a three-stage curriculum that first induces forensic rationales, then aligns reasoning with final judgments, and finally refines logical consistency via reinforcement learning. To support this paradigm, we introduce ROM, a large-scale dataset with rich reasoning annotations. Extensive experiments show that REFORM establishes new state-of-the-art performance with superior generalization, achieving 81.52% ACC on ROM, 76.65% ACC on DGM4, and 74.9 F1 on MMFakeBench.
Yuchen Zhang, Yaxiong Wang, Kecheng Han +4
School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Information Engineering, Hefei University of Technology · CSIRO +2
Recent generative models can produce images with few obvious visual artifacts, weakening detectors and explanations that rely only on surface appearance. We present LaP-Forensics, a multimodal framework that augments RGB semantics with reconstruction-based forensic evidence. A frozen Stable Diffusion DDIM inversion-reconstruction model provides a fixed reconstruction reference, and its residual map measures local compatibility with that reference. Independent projectors encode the RGB image and residual map before a structured Where-What-Why model predicts a textual analysis and an artifact mask. Supervised fine-tuning is followed by Group Relative Policy Optimization (GRPO), whose reward combines mask overlap with output-structure and evidence-reference terms. These text-side terms encourage the model to refer to the consistency map but do not constitute a verifier of free-form textual truth. A separate image-level head fuses RGB and DDIM-residual class features. Experiments show cross-generator detection on UniversalFakeDetect and competitive artifact localization on the official SynthScars benchmark. Controlled cue-construction, inversion-horizon, component, reward-term, and counterfactual analyses support the utility of the residual stream under the evaluated settings, while free-form textual faithfulness and reliability under post-processing remain open limitations.
Can Wang, Yuhao Wang, Yushe Cao +2
The Hong Kong Polytechnic University, Hong Kong, China · University College London, London, United Kingdom · Tsinghua University, Beijing, China +2