ARISE: Authentic Reference-Informed Stability Enhancement for Continual Multimodal Forgery Detection
Authors: Yuchen Zhang, Yaxiong Wang, Yujiao Wu, Lianwei Wu, Qi Zhang, Li Zhu, Chaofeng Dong, Le Wu, Zhedong Zheng
Organizations: School of Software Engineering, Xi’an Jiaotong University · Xingchen AGI Lab · School of Computer and Information Engineering, Hefei University of Technology · CSIRO · Northwestern Polytechnical University · Tongji University · University of Macau
Multimodal forgery detection is commonly studied under a static assumption: training attacks are expected to approximate those encountered after deployment. This assumption becomes increasingly fragile as generators, editing systems, and multimodal large language models continually introduce new forgery mechanisms and forensic evidence. We formulate this problem as attack-incremental continual multimodal forgery detection, where the binary real/fake interface remains fixed while emerging attacks must be learned without forgetting previous ones. We introduce Fable-Bench, which organizes attack evolution through complementary joint-modality, text-side, and image-side protocols. We further propose ARISE (Authentic Reference-Informed Stability Enhancement), motivated by the open-ended forged class and the comparatively reusable geometry of authentic content. ARISE fits a real-only Gaussian mixture model to projected authentic replay representations as a compact historical reference. Meanwhile, the proposed Gaussian-Responsibility-Aware Stability Preservation (GRASP) derives a sample-adaptive anisotropic metric that strongly constrains historically stable representation directions while retaining plasticity elsewhere. Our Vulnerability-Aware Replay (VAR) further estimates stage-wise vulnerability from authentic-density degradation, forged-to-authentic boundary confusion, and authentic-mode occupancy drift, then assigns bounded replay emphasis to fragile stages. All adaptive signals are obtained from training-time replay statistics, while inference uses only the original multimodal backbone and binary classification head. Experiments across three attack-evolution protocols and multiple detector architectures demonstrate improved historical retention and final average performance over representative continual-learning baselines.
Figures & tables
Figure 1. Conceptual motivation for attack-incremental forgery detection. (a) A detector trained once on an initial attack-space snapshot is frozen after deployment. As new forgery mechanisms emerge, the resulting distribution shifts alter forensic evidence and could substantially degrade detection performance. (b) We instead sequentially update the detector with newly available labeled attack data. After each stage, the current detector is evaluated on all attacks observed so far. A two-panel conceptual comparison of static and continual multimodal forgery detection. The left panel shows a detector trained on an initial attack and then frozen, causing its accuracy to decline as previously unseen attacks emerge. The right panel shows a single detector being updated sequentially with newly available attack data and evaluated on all attacks observed so far, with the goal of learning new attacks while retaining previous detection capabilities.
Figure 2. Overview of Fable-Bench and its attack-evolution protocols. Left: 821K image–text pairs spanning 30+ attack mechanisms. Right: representative samples from (P1) joint-modality, (P2) text-side, and (P3) image-side evolution. Overview of Fable-Bench and its three attack-evolution protocols. A radial chart summarizes 821 thousand image–text pairs and more than 30 attack mechanisms. Representative examples illustrate joint-modality evolution from out-of-context repurposing to advanced multimodal forgery, text-side evolution from random mismatch to coherent MLLM-generated deception, and image-side evolution from copy–move editing to full-image synthesis.
Figure 3. Overview of Arise . (a) A frozen copy of the previous detector and a real-only Gaussian mixture provide historical anchors during continual updates. (b) GRASP derives a responsibility-aware anisotropic preservation metric from the local geometry of the authentic reference. (c) VAR estimates stage-wise vulnerability and accordingly reweights historical replay. All auxiliary components are removed from the inference path. Overview of ARISE. The trainable multimodal detector receives current-stage and replay data, while a frozen copy of the previous detector and a real-only Gaussian mixture model provide historical authentic anchors. GRASP derives responsibility-aware anisotropic constraints for representation preservation, and VAR estimates stage-wise vulnerability to reweight historical replay losses.
Method
Joint-Modality Protocol (P1)
Text-Side Protocol (P2)
Image-Side Protocol (P3)
T1
T2
T3
T4
T5
Avg.
T1
T2
T3
T4
Avg.
T1
T2
T3
T4
Avg.
MLLM-based Baselines
GPT-5.4-mini ( OpenAI, 2026 )
92.20
64.10
69.40
62.30
62.10
70.02
91.70
64.20
60.50
59.40
68.95
61.20
58.40
68.10
73.30
65.25
Gemini-3-Flash ( Google DeepMind, 2025 )
93.10
65.30
70.80
63.50
63.20
71.18
92.20
65.00
61.40
60.32
69.73
63.00
60.20
70.60
75.40
67.30
Qwen3.5-Plus ( Qwen Team, 2026 )
90.59
61.26
66.67
59.46
59.46
67.49
90.09
61.26
56.76
55.86
65.99
57.66
54.95
63.96
69.17
61.44
InternVL3.5-30B ( Wang et al., 2025b )
88.80
59.40
64.10
57.20
57.60
65.42
87.60
58.30
54.10
52.90
63.23
59.80
57.10
65.60
71.18
63.42
Table 1. Accuracy (%) comparison under the joint-modality (P1), text-side (P2), and image-side (P3) attack-evolution protocols. The best and second-best results among continual-learning methods are shown in bold and underlined , respectively. MLLM-based baselines and reference baselines are excluded from ranking.
Method
Weibo ( Jin et al., 2017 )
FineFake ( Zhou et al., 2026b )
T1
T2
T3
T4
Avg.
T1
T2
T3
T4
Avg.
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Acc.
F1
Architecture-based Methods
MoE-CLIP ( Yu et al., 2024 )
92.44
92.26
91.16
90.40
92.53
92.01
94.65
94.45
92.70
92.28
73.58
65.00
61.00
58.93
60.46
53.07
87.69
76.48
70.68
63.37
D-MoLE ( Ge et al., 2025 )
92.68
92.09
92.49
92.78
94.17
94.35
97.65
98.23
94.25
94.36
76.69
73.19
71.75
70.21
66.06
60.95
86.00
78.80
75.12
70.79
DAEDCMD ( Wang et al., 2025a )
93.01
93.12
94.79
92.68
93.77
93.99
100.00
100.00
95.39
94.95
75.82
73.57
72.38
70.26
68.23
63.93
88.61
75.22
76.26
70.74
Table 2. Cross-benchmark evaluation on the public Weibo and FineFake datasets. Avg. denotes the average over the four incremental tasks. The best and second-best results are marked in bold and underlined , respectively.
Figure 4. Overall effectiveness and plug-and-play generalization of Arise . (a) Plasticity–stability trade-off across continual learning methods, where bubble size represents the number of parameters. (b)–(c) Plug-and-play performance with CLIP and ViLT backbones in terms of Accuracy and Macro-F1, respectively. Three plots summarizing the effectiveness and transferability of ARISE. The first plot compares the plasticity–stability trade-off of continual learning methods, with bubble size indicating model parameter count. The other two plots report Accuracy and Macro-F1 performance when ARISE is integrated with CLIP and ViLT backbones, compared with naive fine-tuning and experience replay.
Variant
G
V
Avg. ACC (%) ↑
Δ (pp)
P1
P2
P3
Base (ER)
–
–
81.70
77.67
83.57
–
Base + GRASP
✓
–
83.64
80.14
88.93
+3.26
Base + VAR
–
✓
82.12
79.13
88.29
+2.20
Arise
✓
✓
85.37
81.22
90.47
+4.71
Table 3. Ablations of Arise . All Avg. metrics are macro-averaged over P1–P3, and Δ denotes the ACC gain over ER in percentage points. Higher is better except for AF; the best and second-best results are bolded and underlined.
School of Software Engineering, Xi’an Jiaotong University · School of Computer Science and Information Engineering, Hefei University of Technology · CSIRO +2