As high quality public data becomes scarce, Federated Learning (FL) provides a vital pathway to leverage valuable private user data while preserving privacy. However, real-world client data often contains toxic or unsafe information. This leads to a critical issue we define as unintended data poisoning, which can severely damage the safety alignment of global models during federated alignment. To address this, we propose FedDetox, a robust framework tailored for Small Language Models (SLMs) on resource-constrained edge devices. We first employ knowledge distillation to transfer sophisticated safety alignment capabilities from large scale safety aligned teacher models into light weight student classifiers suitable for resource constrained edge devices. Specifically, during federated learning for human preference alignment, the edge client identifies unsafe samples at the source and replaces them with refusal templates, effectively transforming potential poisons into positive safety signals. Experiments demonstrate that our approach preserves model safety at a level comparable to centralized baselines without compromising general utility.
Figures & tables
Fig. 1: An illustrative example of unintended data poisoning, where daily emails containing sensitive emotions act as toxic samples during federated alignment.
Fig. 2: Problem Definition: The risk of Unintended Data Poisoning in Federated Alignment. Left: In centralized settings, raw data is aggregated and sanitized via a central Safety Filter before training, ensuring a safe model. Right: In federated settings, raw data remains local and private. “Unaware” clients, possessing a mixture of benign and toxic data like private chat history, perform local tuning without sanitization. This Unintended Poisoning propagates toxic gradients to the server, causing catastrophic Safety Degradation in the global model.
Fig. 3: Overview of the knowledge distillation pipeline for the lightweight Guardian. We construct a mixed dataset of benign and toxic samples to train the compact student model. The student learns to approximate the safety decision boundaries of the teacher by minimizing a joint loss function that combines the KL divergence of soft logits and the Cross-Entropy loss of hard labels.
Fig. 4: Hazard category distribution of the distilled dataset. The dataset is a composite aggregation from multiple sources to ensuring comprehensive coverage of the MLCommons taxonomy.
Setting
Advbench
DAN Static
TAP
Original Qwen2.5-1.5B-Instruct
10.8%
83.56%
67.0%
FedDPO (Poisoned)
30.8%
78.51%
77.0%
FedDetox (Ours)
14.0%
74.80%
61.0%
TABLE I: Robustness Evaluation against Jailbreak Attacks (ASR). Lower ASR indicates better safety alignment. Red represents worst performance, bold means the best.
Fig. 5: Impact of poisoning and defense efficacy across models. Gray bars show the original model performance: the Base model is inherently unsafe (62.8%), while the Instruct model is aligned (10.8%). Red bars indicate that unintended poisoning causes the Base model to revert to high toxicity ( 58.7% ) and degrades the Instruct model. Blue bars demonstrate that FedDetox effectively suppresses toxicity in both scenarios, achieving safety levels comparable to the ideal benign setting ( Green bars ).
Setting
Unsafe Prompts
Safe Prompts
Compliance ↓
Refusal ↑
Compliance ↑
Refusal ↓
Original (Qwen2.5-1.5B-Instruct)
24.0%
76.0%
92.8%
7.2%
FedDPO (Poisoned)
44.0%
56.0%
97.2%
2.8%
FedDetox (Ours)
25.0%
75.0%
94.4%
5.6%
TABLE II: Performance Comparison on XSTest Benchmark. We report the rate of Full Compliance. Lower is better for Unsafe Prompts (Safety), while higher is better for Safe Prompts (Utility).
Setting
TruthfulQA
MMLU
GSM8K
Original (Qwen2.5-1.5B)
46.9%
59.6%
60.7%
FedDPO (Benign)
45.8%
60.0%
56.8%
FedDPO (Poisoned)
33.9%
57.9%
53.5%
FedDetox (Ours)
38.9%
59.1%
55.4%
TABLE III: Comparison of SLM General Utility under different federated conditions.
Fig. 6: Ablation Study: Effectiveness of sanitization strategies. Simply discarding toxic data ( Orange ) yields a significantly higher ASR (25.6%) compared to FedDetox replacement strategy ( Blue , 14.0%). This confirms that explicit refusal training is necessary to define safety boundaries.
Large language models (LLMs) are increasingly deployed in high-stakes domains, yet a unified treatment of their overlapping safety challenges remains lacking. We present SafeLM, a framework that jointly addresses four pillars of LLM safety: privacy, security, misinformation, and adversarial robustness. SafeLM combines federated training with gradient smartification and Paillier encryption for privacy, integrates defenses against training and inference-time attacks, employs contrastive grounding with calibrated decoding to reduce hallucinations, and introduces alignment-aware binarized aggregation to enhance robustness while maintaining bounded reconstruction quality. Across benchmarks on factuality, toxicity, and membership inference, SafeLM achieves 98.0% harmful content detection accuracy, reduces communication by 96.9%, and lowers gradient inversion PSNR from 31.7 dB to 15.1 dB. Ablations show that each component contributes independently, whereas their integration yields a strong privacy utility efficiency trade-off for deploying trustworthy LLMs.
Federated fine-tuning enables large language models (LLMs) to learn collaboratively from distributed client data, but malicious client updates can compromise global model safety. As federated tasks evolve, changing malicious update patterns challenge the ability of existing defenses to maintain reliable risk detection over time. To address this challenge, we propose Safe-FCL, a framework integrating continual risk awareness with fine-grained safety aggregation for federated continual learning of LLMs. Specifically, the Continual Risk Awareness (CRA) mechanism learns emerging risk patterns across sequential tasks and constrains detector updates using a protective subspace derived from historical features, mitigating forgetting of previously learned malicious patterns while adapting to new risks. To translate risk awareness into aggregation-level safety control, the Fine-Grained Safety Aggregation (FGSA) mechanism combines client-level risk assessments from CRA with module-level deviations from historical update behavior to adaptively regulate aggregation contributions, suppressing potentially malicious updates while minimizing unnecessary restrictions on benign updates. Extensive experiments demonstrate that Safe-FCL outperforms current SOTA methods, improving global-model Rule and MD-Judge scores in federated continual learning by 0.41% and 0.64%, respectively, while increasing average risk recognition accuracy by 2.01%.
Shuaijie Ma, Yongkang Hu, Yu Cheng +4
East China Normal University · Shanghai Innovation Institute
Deploying safe large language models (LLMs) on resource-constrained edge devices presents a critical challenge: while dual-model systems combining LLMs with guard models provide effective safety guarantees, their substantial memory and computational demands make them prohibitively expensive for on-device deployment. This paper presents a comprehensive study of parameter-efficient safety alignment methods for resource-constrained settings. Through systematic evaluation across multiple LLM architectures, training objectives, and parameter-efficient fine-tuning approaches, we identify that soft prompts combined with distillation-based training consistently outperform alternative methods. We introduce distillation frameworks based on total variation and KL divergence that effectively transfer safety behaviors from guard models into learned soft prompts. Our evaluations on various benchmarks demonstrate that this combination achieves superior safety-usefulness trade-offs compared to LoRA adapters, steering vectors, and direct optimization methods, while requiring minimal additional memory and compute at inference time. These findings establish soft prompt distillation as the preferred approach for safety alignment in on-device LLM deployment.
Motasem Alfarra, Cristina Pinneri, Dana Kianfar +2