cs.AINov 17, 2026

DACE: Diversity-Driven Adversarial Co-Evolution for Robust LLM Safety Alignment

Authors: Peng Yu, Xiaoyu Wen, Zhida He, Ziyuan Zhou, Han Qi, Shao Zhang, Qiaosheng Zhang, Ying Wen, 

Organizations: Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Fudan University

Abstract

Ensuring robust safety alignment of large language models (LLMs) is increasingly difficult as adversarial attacks evolve and outpace alignment pipelines built on pre-collected data. Existing co-evolutionary frameworks let the defender chase a moving attacker, yet their training dynamics expose two compounding pathologies: attack strategy collapse, where the attacker overfits to a narrow set of high-reward rewrites, and defense adversarial forgetting, where the defender loses competence against earlier attacks as the attack distribution drifts. Existing remedies are partial: attacker-side diversity rewards score textual novelty rather than strategy novelty, and defender-side replay relies on discrete judge scores that ignore estimation uncertainty, response stochasticity, and defender non-stationarity. We introduce \method, a diversity-driven adversarial co-evolution framework that targets both pathologies jointly. On the attacker side, \method couples an explicit two-dimensional strategy space (risk category ×\times attack style) with a normalized marginal coverage gain reward, providing a bounded, non-vanishing exploration signal at the strategy layer. On the defender side, \method maintains a unified Bayesian adversarial replay pool whose Beta--Bernoulli threat posteriors are refreshed by time decay and sampled via Thompson sampling, anchoring defender training to an evolving threat landscape. Across standard safety, automated-attacker, and general-capability benchmarks, \method improves robustness to out-of-distribution attacks while preserving general capabilities, and yields broader attacker strategy coverage.

Figures & tables

Appendix figures & tables16 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. AgentSafeEvo: Adversarial Co-Evolution for LLM Agent Safety

    Oct 14, 2026Chenxu Wang, Xiaoyu Wen, Han Qi +4LLM Agent SafetyAdversarial Training

  2. Model-Agnostic Lifelong LLM Safety via Externalized Attack-Defense Co-Evolution

    May 13, 2026Xiaozhe Zhang, Chaozhuo Li, Hui Liu +4Continual Learning for LLMsAdversarial Attacks on LLMs