Organizations: School of Computer Science, Wuhan University, Wuhan, Hubei, China · Zhongguancun Academy, Beijing, China · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center (National Supercomputer Center in Jinan), Qilu University of Technology (Shandong Academy of Sciences), Jinan, Shandong, China; Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, Shandong, China
Toxicity detection mitigates the dissemination of toxic content (e.g., hateful comments, posts, and messages within online social actions) to safeguard a healthy online social environment. However, malicious users persistently develop evasive perturbations to disguise toxic content and evade detectors. Traditional detectors or methods are static over time and are inadequate in addressing these evolving evasion tactics. Thus, continual learning emerges as a logical approach to dynamically update detection ability against evolving perturbations. Nevertheless, disparities across perturbations hinder the detector's continual learning on perturbed text. More importantly, perturbation-induced noises distort semantics to degrade comprehension and also impair critical feature learning to render detection sensitive to perturbations. These amplify the challenge of continual learning against evolving perturbations. In this work, we present ContiGuard, the first framework tailored for continual learning of the detector on time-evolving perturbed text (termed continual toxicity detection) to enable the detector to continually update capability and maintain sustained resilience against evolving perturbations. Specifically, to boost the comprehension, we present an LLM-powered semantic enriching strategy, where we dynamically incorporate possible meaning and toxicity-related clues excavated by LLM into the perturbed text to improve the comprehension. To mitigate non-critical features and amplify critical ones, we propose a discriminability-driven feature learning strategy, where we strengthen discriminative features while suppressing the less-discriminative ones to shape a robust classification boundary for detection...
Figures & tables
Figure 1. Illustration of our ContiGuard framework.
Existing Detectors
Insert
Repeat
Swap
Remove
Homo
Mask
Abbr
Auth
Distract
Avg
PespectiveAPI
77.20
64.61
70.48
60.51
64.84
63.29
50.93
66.38
64.37
64.7325.9%↓
OpenAI Moderation
66.15
52.24
61.13
56.49
50.78
59.35
51.00
84.47
81.14
62.5325.2%↓
ToxBERT
60.90
60.97
58.81
58.34
53.55
55.87
58.35
73.26
75.12
61.6914.4%↓
ToxRoBERTa
50.93
50.39
51.24
52.32
50.15
50.31
51.39
55.26
79.83
54.6534.0%↓
DetoxifyOriginal
61.59
63.76
61.90
57.73
56.96
59.27
56.03
71.87
75.66
62.7529.8%↓
DetoxifyMultilingual
60.20
58.50
56.41
56.96
82.53
59.20
52.63
66.62
81.92
63.8826.6%↓
Table 1. The results of ContiGuard and existing detectors on perturbed text. The subscripts indicate the relative performance degradation compared with ordinary text. The best scores are highlighted in bold .
Continual
T1
T2
T3
T4
T5
T6
T7
T8
T9
OT91
OT92
OT93
OT94
Joint
84.14
86.63
85.43
85.07
84.56
85.05
85.03
85.31
85.88
85.63
85.51
86.31
86.06
Stream
84.23
84.64
76.52
77.36
78.17
79.55
79.73
74.28
78.26
79.49
74.08
78.95
80.49
LFL
83.46
82.88
77.15
76.59
75.30
78.25
78.44
75.01
78.15
79.48
74.19
79.07
79.85
EWC
84.66
84.88
76.41
78.48
77.89
80.89
78.64
76.01
79.32
79.81
75.43
80.25
81.80
LWF
83.46
85.08
79.39
79.86
76.31
76.92
75.56
72.58
72.30
72.29
74.94
65.40
76.56
EPI
84.12
81.46
81.60
79.13
79.90
78.23
76.98
78.26
79.20
79.03
79.50
79.19
79.07
Table 2. The results of ContiGuard and adapted CL methods. Ti denote averaged performance of all orders at i-th moment. OT9j denote performance of j-th order at T9 moment. * indicates significant difference at p<0.001.
Figure 2. Results of statically fine-tuned detectors. Rows: types used for fine-tuning. Columns: types used for testing.
Figure 3. Results on different types of perturbed text at T9 . The numbers denote the best accuracy for each type.
T1
T2
T3
T4
T5
T6
T7
T8
T9
ContiGuard
87.17
89.27
86.20
86.42
85.41
86.29
85.41
84.92
86.47
w/o aux
83.11
84.63
80.52
81.03
80.03
81.85
82.12
80.06
81.97
w/o coop
87.42
88.88
85.62
85.71
85.90
87.16
85.63
86.22
85.96
w/o disc
86.20
87.71
84.08
85.63
86.18
85.30
84.49
84.74
85.71
w/o more
86.36
88.84
85.61
86.07
86.22
85.78
84.75
84.59
85.89
w/o less
86.78
88.86
86.49
86.11
86.17
85.59
84.44
84.06
86.25
Table 3. Ablation results at different moments.
Figure 4. Results of different memory sample numbers.
Figure 5. Retention rate of critical features, which denotes the proportion of historical critical features in current critical features. Each column shows the changes over time.
Methods
T1
T2
T3
T4
T5
T6
T7
T8
T9
IG
87.17
89.27
86.20
86.42
85.41
86.29
85.41
84.92
86.47
SHAP
86.73
89.33
84.86
85.04
86.30
85.96
84.15
85.44
85.94
Table 4. Results of different contribution measuring methods.
Figure 6. Results on DynEscape and NoisyHate.
Figure 7. Results of different training data sampling rates.
Figure 8. Case study for examining the impact of the LLM.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Perturbations
Examples
Insert
moron → mo*ro#n; idiot → idi+ot
Remove (Mv)
moron → moro; idiot → idot
Repeat
moron → mooron; idiot → iddiot
Swap
fool → folo; idiot → idoit
Homoglyph (Homo)
idiot → id10t ; fool → f001
Maskword (Mask)
fool → f**l; idiot → i*iot
Appendix
Table 5. Examples of different perturbations.
DynEscape (ord.)
DynEscape
NoisyHate
4o-mini
85.81
69.5019%↓
80.06
Appendix
Table 6. Result of 4o-mini. DynEscape (ord.) represents the original ordinary sample corresponding to DynEscape.
Automated toxicity moderation systems operate in dynamic online environments where harmful behavior evolves through coded language, shifting targets, and strategic adaptation to enforcement. Existing drift detection methods often focus on global distributional change, but such signals may miss safety-relevant shifts that emerge in localized harm subspaces or high-risk model-error regions. This paper introduces DriftGuard, a safety-aware adaptive moderation framework that combines multi-monitor drift detection with selective model updating. The framework tracks global text drift, identity-harm drift, model uncertainty, toxic-risk drift, and false-negative-risk drift. When safety-relevant change is detected, the model is updated using a hard-mix adaptation set that prioritizes likely false negatives, identity-related high-risk examples, false-positive-risk examples, and uncertain boundary cases. Experiments on Civil Comments temporal shift and Jigsaw-to-DynaHate cross-dataset shift show that safety-aware monitors detect risks missed by global drift alone. Hard-mix adaptation improves toxic recall and accuracy over no-update and random-balanced baselines, raising toxic recall to 0.8777 on Civil Comments and from 0.7107 to 0.8523 on DynaHate. Bootstrap analysis further shows stable DynaHate safety gains, with toxic recall increasing by 0.1418 and false-negative prevalence decreasing by 0.0781. Overall, DriftGuard links safety-aware drift detection to targeted, lightweight model updating for more robust adaptive toxicity moderation.
Yuting Xin, Hanyu Cai, Binqi Shen +2
Department of Information and Decision Sciences University of Minnesota Minneapolis, USA · Department of Industrial Engineering and Management Sciences Northwestern University Evanston, USA · Fuqua School of Business Duke University Durham, USA +1
Adversarial perturbations can reduce state-of-the-art toxicity classifiers to near-zero accuracy, yet existing defences treat models as black boxes. We apply mechanistic interpretability to toxicity classification for the first time, identifying the internal attention-head circuits responsible for both correct classification and adversarial vulnerability. Across a 2×2 factorial study (BERT × RoBERTa) × (Jigsaw × ToxiGen), extended to Llama Guard~2 (8B), we show that zeroing a single attention head recovers up to 70.4 pp of adversarial accuracy for RoBERTa on Jigsaw and 37.3 pp for Llama Guard 2 on ToxiGen, at ≤0.6 pp clean cost. Vulnerable heads generalise to held-out examples within ≤1 pp, and a class-imbalance sweep confirms they act as selective toxic-class detectors. Head suppression matches or outperforms adversarial training on Jigsaw; data augmentation dominates on ToxiGen: a dataset-specific reversal explained by whether the classifier encodes a concentrated bottleneck or a distributed circuit. Demographic analysis across 20 Jigsaw and 13 ToxiGen minority groups reveals structurally unequal adversarial vulnerability, exposing mechanistically traceable fairness gaps in current toxicity classifiers.
Shaz Furniturewala, Arkaitz Zubiaga
Center for Data Science, New York University · Queen Mary University of London
Large language models (LLMs) require robust toxicity evaluation beyond explicit wording. This setting remains underexplored in Chinese, where toxicity may combine semantic indirectness with surface obfuscation. We introduce Chinese Implicit Toxicity Attack (CITA), a controlled red-team evaluation and defense-data generation framework, not a deployable evasion tool. CITA uses three stages: (i) Harmful Intent Learning, (ii) Implicit Toxicity Enhancement, and (iii) Obfuscation Variant Rewriting, to preserve harmful intent, increase implicitness, and add controlled surface variants. On CITA-generated evaluation samples, the seven tested detectors exhibit substantial missed-detection risks, reaching an average ASR of 69.48%; human evaluation further confirms preserved harmfulness and increased implicitness/evasiveness. As a downstream defense application, we fine-tune a Chinese Implicit Toxicity Defense model (CITD) with CITA-generated red-team data, showing that such data can improve robustness through additional training.
Jingyi Kang, Junyu Lu, Bo Xu +4
1Dalian University of Technology · 2The University of Tokyo · 3Singapore University of Technology and Design