ToxiTrace: Gradient-Aligned Training for Explainable Chinese Toxicity Detection

Boyang Li; Hongzhe Shou; Yuanyuan Liang; Jingbin Zhang; Fang Zhou

cs.CLApr 14, 2026

ToxiTrace: Gradient-Aligned Training for Explainable Chinese Toxicity Detection

Authors: Boyang Li, Hongzhe Shou, Yuanyuan Liang, Jingbin Zhang, Fang Zhou

Organizations: School of Data Science and Engineering, East China Normal University · School of International Chinese Studies, East China Normal University

Abstract

Existing Chinese toxic content detection methods mainly target sentence-level classification but often fail to provide readable and contiguous toxic evidence spans. We propose ToxiTrace, an explainability-oriented method for BERT-style encoders with three components: (1) CuSA, which refines encoder-derived saliency cues into fine-grained toxic spans with lightweight LLM guidance; (2) GCLoss, a gradient-constrained objective that concentrates token-level saliency on toxic evidence while suppressing irrelevant activations; and (3) ARCL, which constructs sample-specific contrastive reasoning pairs to sharpen the semantic boundary between toxic and non-toxic content. Experiments show that ToxiTrace improves classification accuracy and toxic span extraction while preserving efficient encoder-based inference and producing more coherent, human-readable explanations. We have released the model at https://huggingface.co/ArdLi/ToxiTrace.

Figures & tables

Figure 1: Existing encoder-based detectors struggle to reliably extract fine-grained toxic expressions within a sentence; LLMs can better extract spans when toxicity is given but are limited in direct classification and efficiency; our method preserves classification performance while enabling contiguous toxic span extraction.

Figure 2: Framework of the proposed ToxiTrace method. During training, we warm up an encoder classifier, acquire weak span annotations with CuSA using BiCSE-extracted saliency cues, and jointly optimize GCLoss and ARCL to concentrate saliency on toxic evidence. During inference, the model predicts toxicity and, for toxic inputs, extracts contiguous spans via BiCSE from the saliency map.

Models	COLD					ToxiCN
Models	$Acc$	$R$	$P$	$F_{1}$	Macro- $F_{1}$	$Acc$	$R$	$P$	$F_{1}$	Macro- $F_{1}$
Qwen3-8B	$74.03_{0.23}$	$59.90_{0.28}$	$70.15_{0.32}$	$64.62_{0.19}$	$72.33_{0.21}$	$71.44_{0.33}$	$82.94_{0.29}$	$69.13_{0.27}$	$75.41_{0.16}$	$71.51_{0.12}$
LLaMA3.1-8B	$73.91_{0.31}$	$51.46_{0.29}$	$74.58_{0.35}$	$60.90_{0.22}$	$72.00_{0.28}$	$65.18_{0.28}$	$44.29_{0.29}$	$81.61_{0.24}$	$57.42_{0.20}$	$68.25_{0.22}$
DeepSeek-V3.2	$74.23_{0.16}$	$59.23_{0.10}$	$70.91_{0.13}$	$64.55_{0.12}$	$72.51_{0.09}$	$60.66_{0.11}$	$36.19_{0.14}$	$77.35_{0.16}$	$49.30_{0.08}$	$64.14_{0.11}$
GPT-4o	$74.49_{0.18}$	$67.79_{0.20}$	$66.98_{0.21}$	$67.38_{0.12}$	$73.22_{0.13}$	$73.20_{0.21}$	$65.66_{0.18}$	$66.98_{0.16}$	$67.38_{0.11}$	$73.22_{0.09}$
RoBERTa	$82.68_{0.42}$	$86.37_{0.86}$	$74.42_{0.86}$	$79.80_{0.27}$	$82.56_{0.38}$	$82.70_{0.45}$	$84.38_{0.30}$	$83.40_{0.55}$	$83.89_{0.40}$	$82.81_{0.45}$

Table 1: The average classification results of different models on COLD and ToxiCN datasets(%) across 3 independent runs. Subscripts denote standard deviations.

Models	Overlap			Character-level				Inference Time
Models	$R$	$P$	$F_{1}$	$R$	$P$	$F_{1}$	$IoU$	Inference Time
CRF Liu et al. (2021)	44.48	78.01	56.66	39.24	81.85	53.05	36.10	01m 10s
LIME Ribeiro et al. (2016)	41.45	35.99	38.53	81.62	33.93	47.93	31.52	27m 20s
Attention Yamada et al. (2020)	39.01	36.15	37.52	25.43	47.68	33.17	19.88	00m 42s
IG Sikdar et al. (2021)	58.30	60.01	59.14	36.99	76.46	49.86	33.21	About 1.2 hours
Llama3.2-3B	41.03	71.02	52.01	37.47	69.42	48.67	32.16	08m 06s

Table 2: Toxic span extraction performance on CNTP. Results without * correspond to extracting the top-

15\%

most salient tokens, while results with * use our proposed bidirectional algorithm to extract high-saliency spans. Entries shaded in cyan indicate the results of the ”refiner” model used in the CuSA.

Figure 3: Final Accuracy with different Warm-up Epochs. The models achieve optimal classification performance when the warm-up steps are set to 3.

Models	Classification					Extraction
Models	$Acc$	$R$	$P$	$F_{1}$	Macro- $F_{1}$	$R$	$P$	$F_{1}$
ToxiTrace	83.84	86.19	76.14	80.85	83.68	86.36	70.95	77.90
w/o CuSA	83.26	83.77	76.27	79.85	82.90	65.61	79.67	71.96
w/o ARCL	83.13	86.71	74.72	80.27	83.12	82.55	68.99	75.16
w/o GCLoss	83.20	88.04	74.29	80.58	83.36	75.89	57.07	65.15
RoBERTa	82.75	86.43	74.24	79.87	82.76	71.27	59.87	65.08

Table 3: Ablation study results. ”Classification” denotes results for the binary classification task on the COLD dataset; ”Extraction” denotes results using the toxic span annotations in CNTP Yang et al. (2025) as ground truth labels.

Figure 4: Confidence drop after masking BiCSE-extracted toxic spans for the RoBERTa baseline.

Figure 5: Confidence drop with random masking that matches the number of tokens extracted by BiCSE for RoBERTa trained with ToxiTrace .

Figure 6: Confidence drop after masking BiCSE-extracted toxic spans for RoBERTa trained with ToxiTrace .

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Saliency map of a RoBERTa model trained only with binary classification labels.

Figure 8: Saliency map of a RoBERTa model after training with our proposed method.

Panel of Table 4 — Table 4: Toxic Spans Refining Prompt

Panel of Table 6 — Table 6: Toxic Reasoning Generation Prompt

Panel of Table 7 — Table 7: Translation of Toxic Reasoning Generation Prompt

Panel of Table 8 — Table 8: Non-toxic Reasoning Generation Prompt

Panel of Table 9 — Table 9: Translation of Toxic Reasoning Generation Prompt

Panel of Table 10 — Table 10: LLM Classification Task Instruction

Panel of Table 11 — Table 11: Translation of LLM Classification Task Instruction

Toxicity Detection

Toxicity detection identifies harmful language for moderation and predicts toxic effects of chemicals, drugs, or biological exposures.

See all 45 papers

Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexicon
May 28, 2025Xuchen Ma, Jianxiang Yu, Wenming Shao +2Toxicity Detection Hate Speech Detection
School of Data Science and Engineering, East China Normal University · Shanghai EastWonder Info-tech Co., Ltd.
Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models
May 27, 2026Himanshu Beniwal, Mayank SinghLanguage Model Safety Evaluation LLM Interpretability
Indian Institute of Technology Gandhinagar
CausalDetox: Causal Head Selection and Intervention for Language Model Detoxification
Apr 16, 2026Yian Wang, Yuen Chen, Agam Goyal +1Causal Reasoning in Language Models LLM Safety
Department of Computer Science University of Illinois, Urbana-Champaign Champaign, IL 61801

May 28, 2025J/K move · Enter open · S save

Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexicon

Xuchen Ma, Jianxiang Yu, Wenming Shao, Bo Pang+1

School of Data Science and Engineering, East China Normal University · Shanghai EastWonder Info-tech Co., Ltd.

Toxicity Detection Hate Speech Detection

Open paper AskPDF arXiv

Social media platforms have experienced a significant rise in toxic content, including abusive language and discriminatory remarks, presenting growing challenges for content moderation. Some users evade censorship by deliberately disguising toxic words through homophonic cloak, which necessitates the task of unveiling cloaked toxicity. Existing methods are mostly designed for English texts, while Chinese cloaked toxicity unveiling has not been solved yet. To tackle the issue, we propose C

^2

TU, a novel training-free and prompt-free method for Chinese cloaked toxic content unveiling. It first employs substring matching to identify candidate toxic words based on Chinese homophone graph and toxic lexicon. Then it filters those candidates that are non-toxic and corrects cloaks to be their corresponding toxicities. Specifically, we develop two model variants for filtering, which are based on BERT and LLMs, respectively. For LLMs, we address the auto-regressive limitation in computing word occurrence probability and utilize the full semantic contexts of a text sequence to reveal cloaked toxic words. Extensive experiments demonstrate that C

^2

TU can achieve superior performance on two Chinese toxic datasets. In particular, our method outperforms the best competitor by up to 71% on the F1 score and 35% on accuracy, respectively. Our code and data are available at https://github.com/XDxc-cuber/C2TU-Chinese-cloaked-toxicity-unveiling.

Abstract

Toxicity Detection

Explore similar work

Breaking the Cloak! Unveiling Chinese Cloaked Toxicity with Homophone Graph and Toxic Lexicon

Where Does Toxicity Live? Mechanistic Localization and Targeted Suppression in Language Models

CausalDetox: Causal Head Selection and Intervention for Language Model Detoxification