Adversarial Training

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, up 71% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 115

All topics

Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO
Jun 8, 2026Blake Bullwinkel, Eugenia Kim, Amanda Minnich +1Adversarial Training Adversarial Attacks on LLMs
Microsoft AI Red Team · Microsoft Azure
Defending Against Malicious Finetuning by Scaling Train-time Adversarial Attacks
Jun 6, 2026Haoming Wen, Shi Chen, Qingyu Shi +4Adversarial Training LLM Alignment
1Xiongan AI Institute · Institute for Interdisciplinary Information Sciences, Tsinghua University · 3Shanghai Qi Zhi Institute
CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
Jun 4, 2026Rahul Markasserithodi, Aditya Joshi, Yuekang Li +3Adversarial Training Adversarial Prompt Generation
University of New South Wales, Australia
Adaptive Causal Alignment for High-Confidence Adversarial Training
Jun 2, 2026Zhiming Luo, Kejia Zhang, Yingxin Lai +3Adversarial Training Adversarial Robustness
Department of Artificial Intelligence, Xiamen University, Xiamen, China · Department of Computer Science, Emory University, GA, USA · College of Information Science and Technology, Jinan University, Guangzhou, China
Investigating Adversarial Robustness of Multi-modal Large Language Models
Jun 2, 2026Hashmat Shadab Malik, Muzammal Naseer, Salman KhanAdversarial Training VLM Robustness
Mohamed Bin Zayed University of AI, UAE · Khalifa University, UAE · Australian National University, Australia
SORA: Free Second-Order Attacks in Fast Adversarial Training
May 30, 2026Mazdak Teymourian, Ramtin Moslemi, Farzan Rahmani +1Adversarial Training Neural Network Generalization
Department of Computer Engineering, Sharif University of Technology, Tehran, Iran.
A Mechanistic Analysis of Adversarial Fine-tuning of Vision Transformers
May 28, 2026Hannah Gao, Isha Agarwal, Dylan Hadfield-Menell +1Adversarial Training Fine-Tuning
Massachusetts Institute of Technology · MIT Cambridge, MA
When Muon Optimizer Meets Adversarial Training: A Theoretical and Empirical Study
May 26, 2026Jun Yan, Weiquan Huang, Jiankai Zuo +4Adversarial Training Neural Network Optimization
IT College, Shanghai Ocean University, Shanghai, 201306 · School of Computer Science and Technology, Tongji University, Shanghai, 201804 · SEIE, Suzhou University of Science and Technology, Suzhou, 215009 +2
Adversarial Training for Robust Coverage Network under Worst-case Facility Losses
May 26, 2026Changhao Miao, Yuntian Zhang, Tongyu Wu +2Adversarial Training Combinatorial Optimization
State Key Laboratory of Autonomous Intelligent Unmanned Systems, Beijing Institute of Technology, Beijing 100081, China · Department of Computer, Control and Management Engineering “Antonio Ruberti”, Sapienza University of Rome, Rome, 00185, Italy · School of AI, Beijing Institute of Technology, Beijing 100081, China
Steering Beyond the Support: Adversarial Training on Unsupervised Jailbroken Activation Simulation
May 23, 2026Luoyu Chen, Weiqi Wang, Zhiyi Tian +5Adversarial Training Adversarial Robustness of Language Models
University of Technology Sydney, Sydney, Australia · Xi’an Jiaotong University, Xi’an, China · School of Cyber Science and Engineering, Southeast University, Nanjing, China.
Balancing Fairness, Privacy, and Accuracy: A Multitask Adversarial Framework for Centralized Data-Driven Systems
May 23, 2026Imesh Ekanayake, Elham Naghizade, Jeffrey ChanAdversarial Training Representation Learning
School of Computing Technologies, RMIT University, Melbourne, VIC, Australia
Toward Understanding Adversarial Distillation: Why Robust Teachers Fail
May 21, 2026Hongsin Lee, Hye Won ChungAdversarial Training Neural Network Robustness
School of Electrical Engineering, KAIST, Daejeon, Korea.
CCLab: Adversarial Testing of Learning- and Non-Learning-Based Congestion Controllers
May 21, 2026Zhi Chen, Shehab Sarar Ahmed, Chenkai Wang +2Adversarial Training Adversarial Robustness
UIUC · UIUC & BUET
The Evaluation Game: Beyond Static LLM Benchmarking
May 19, 2026Paul Wang, Jade Garcia-Bourrée, Anne-Marie Kermarrec +1Adversarial Training LLM Auditing
Sorbonne Université, CNRS, LIP6 · École Polytechnique Fédérale de Lausanne
Interaction-Breaking Adversarial Learning Framework for Robust Multi-Agent Reinforcement Learning
May 18, 2026Sunwoo Lee, Mingu Kang, Yonghyeon Jo +1Adversarial Training Multi-Agent Coordination
Graduate School of Artificial Intelligence, UNIST, Ulsan, South Korea
Compositional Adversarial Training for Robust Visual Watermarking
May 16, 2026Anirudh Satheesh, Michael-Andrei Panaitescu-Liess, Andrew Xu +4Adversarial Training Robust Watermarking
University of Maryland
When and Why Adversarial Training Improves PINNs: A Neural Tangent Kernel Perspective
May 15, 2026Yuan-dong Cao, Chi Chiu SO, Jun-Min Wang +1Adversarial Training PDE Surrogate Modeling
School of Mathematics and Statistics, Beijing Institute of Technology, China · Department of Computer Science & UCL AI Centre, University College London, UK · School of Professional Education and Executive Development The Hong Kong Polytechnic University, China
WARD: Adversarially Robust Defense of Web Agents Against Prompt Injections
May 14, 2026Tri Cao, Yulin Chen, Hieu Cao +8Adversarial Training Prompt Injection
National University of Singapore · University of Science · Vietnam National University, Ho Chi Minh City
Taming the Long Tail: Rebalancing Adversarial Training via Adaptive Perturbation
May 13, 2026Lilin Zhang, Yimo Guo, Yue Li +2Adversarial Training Class-Imbalanced Learning
1Sichuan University · 2Dongfang Electric (Chengdu) Innovation Research Co., Ltd. · 3Southwest China Research Institute of Electronic Equipment
Persona-Conditioned Adversarial Prompting: Multi-Identity Red-Teaming for Adversarial Discovery and Mitigation
May 12, 2026Cristian Morasso, Anisa Halimi, Muhammad Zaid Hameed +1Adversarial Training Adversarial Prompt Generation
IBM Research · Trinity College Dublin, Ireland
Robust Biomedical Publication Type and Study Design Classification with Knowledge-Guided Perturbations
May 12, 2026Shufan Ming, Joe D. Menke, Neil R. Smalheiser +1Adversarial Training Distribution Shift Robustness
School of Information Sciences University of Illinois Urbana-Champaign Champaign, USA · Department of Psychiatry University of Illinois Chicago Chicago, USA
Robust Multi-Agent Path Finding under Observation Attacks: A Principled Adversarial-Plus-Smoothing Training Recipe
May 12, 2026Riad AhmedAdversarial Training Adversarial Robustness
University of New Hampshire
PROWL: Prioritized Regret-Driven Optimization for World Model Learning
May 11, 2026Ahmet H. Güzel, Jenny Seidenschwarz, Benjamin Graham +3Adversarial Training World Model Learning
University College London AI Centre · Odyssey · University of Basel
Enhancing Adversarial Robustness in Network Intrusion Detection: A Layer-wise Adaptive Regularization Approach
May 9, 2026Hira Nasir, Eiman Javed, Balawal Shabir +2Adversarial Training Network Intrusion Detection
National Cyber Security Academy (NCSA), Air University, Islamabad, Pakistan. · School of Science, Edith Cowan University, Australia.
Latent Personality Alignment: Improving Harmlessness Without Mentioning Harms
May 8, 2026Linh Le, David Williams-King, Mohamed Amine Merzouk +2Adversarial Training Neural Network Robustness
McGill University · Mila – Quebec AI Institute · ERA +1
Exposing and Mitigating Temporal Attack in Deepfake Video Detection
May 8, 2026Zheyuan Gu, Minghao Shao, Zhen Wang +4Adversarial Training Adversarial Robustness
1Peking University · 2New York University · 3Huzhou University +2
Dual-Agent Co-Training for Health Coaching via Implicit Adversarial Preference Optimization
May 7, 2026Da Long, Lingyi Fu, Diya Michelle Rao +3Adversarial Training Multi-Agent LLM Systems
Kahlert School of Computing University of Utah · Department of Health and Kinesiology University of Utah
Streaming Adversarial Robustness in Fuzzy ARTMAP: Mechanism-Aligned Evaluation, Progressive Training, and Interpretable Diagnostics
May 7, 2026Shane Cairns, Leonardo Enzo Brito da Silva, Sasha Petrenko +2Adversarial Training Neural Network Robustness
Department of Electrical and Computer Engineering, Missouri University of Science and Technology, Rolla, MO, USA · Instituto Metrópole Digital, Universidade Federal do Rio Grande do Norte, Natal, RN 59078-900, Brazil · Kummer Institute Center for Artificial Intelligence and Autonomous Systems (KICAIAS), Missouri University of Science and Technology, Rolla, MO, USA
Band Together: Untargeted Adversarial Training with Multimodal Coordination against Evasion-based Promotion Attacks
May 7, 2026Guanmeng Xian, Ning Yang, Philip S. YuAdversarial Training Multimodal Robustness
Sichuan University, Chengdu, China · University of Illinois at Chicago, USA
Information Theoretic Adversarial Training of Large Language Models
May 6, 2026Yiwei Zhang, Jeremiah Birrell, Reza Ebrahimi +3Adversarial Training LLM Alignment
Purdue University · West Lafayette, IN 47907, USA · Texas State University +5

Jun 8, 2026, cs.CLJ/K move · Enter open · S save

Learning to Attack and Defend: Adaptive Red Teaming of Language Models via GRPO

Blake Bullwinkel, Eugenia Kim, Amanda Minnich, Mark Russinovich

Microsoft AI Red Team · Microsoft Azure

Adversarial Training Adversarial Attacks on LLMs Group Relative Policy Optimization LLM Red Teaming Adversarial Robustness of Language Models

Open paper AskPDF arXiv

Language model safety must continually adapt to evolving attacks. Recent works have demonstrated that reinforcement learning can be used to train stronger attacker and defender models in tandem by applying PPO-style self-play and DPO-style online preference optimization. In this work, we explore the efficacy of GRPO in this setting. Co-training can be challenging because it requires jointly optimizing multiple properties of both the attacker and defender. We therefore shape model outputs using multiple LLM judge-based reward channels and compute advantages with GDPO, which prevents any single channel from dominating. Our method uses a curriculum that progresses from attacker-only single-turn and multi-turn training to co-training, where attacker and defender models are updated in alternation. We show that this method produces highly effective and transferable attacks, and that co-trained defenders reach competitive safety while preserving general utility. Through a controlled ablation, we further identify which components of our training pipeline most affect the resulting balance between safety and utility. Finally, we find that GRPO tends to collapse attacker diversity over training and discuss possible ways to address this limitation.

Momentum

Related topics

Latest papers 115