Adversarial Robustness of Language Models

Ask about this topic

Overview

The field examines whether language models preserve safe, accurate, and consistent behavior when facing jailbreaks, prompt injections, meaning-preserving edits, misleading evidence, or pressure across multiple turns. Researchers test models with red-teaming and adversarial benchmarks, then strengthen them through adversarial training, preference and reinforcement learning, parameter-efficient updates, and inference-time guards or steering. Mechanistic methods also inspect and modify internal representations, while certification aims to provide guarantees for defined threat models. A central challenge is that defenses that work on known attacks may fail against adaptive or out-of-distribution ones, and stronger protection can reduce utility or cause over-refusal; evaluations therefore increasingly measure both robustness and capability across diverse settings.

Latest papers

CardsList

192 papers17 papers in the last four weeks, level with the four weeks before. 0.1% of all new papers.

  1. Evaluating Evidence Grounding Under User Pressure in Instruction-Tuned Language Models

    Mar 20, 2026Sai Koneru, Elphin Joe, Christine Kirchhoff +2LLM GroundingAdversarial Robustness of Language Models

    College of Information Sciences and Technology, Pennsylvania State University, USA · Department of Computer Science, Old Dominion University, Norfolk, VA

  2. Box Maze: A Process-Control Architecture for Reliable LLM Reasoning

    Mar 19, 2026Zou QiangLLM SafetyReasoning Consistency in Language Models

    1Independent Researcher

  3. SIA: A Synthesize-Inject-Align Framework for Knowledge-Grounded and Secure E-commerce Search LLMs with Industrial Deployment

    Mar 17, 2026Zhouwei Zhai, Mengxiang Chen, Anmeng ZhangE-Commerce SearchKnowledge Augmentation for Language Models

    JD.com Beijing, China

  4. Clinician input steers AI toward accurate and harmful recommendations

    Mar 14, 2026Ivan Lopez, Selin S. Everett, Bryan J. Bunning +10Medical LLMsClinical Language Model Evaluation

    Department of Biomedical Data Science, Stanford University, Stanford, CA, USA · Stanford University School of Medicine, Stanford, CA, USA · Quantitative Sciences Unit, Department of Medicine, Stanford University School of Medicine, Stanford, CA, USA +14

  5. Deactivating Refusal Triggers: Understanding and Mitigating Overrefusal in Safety Alignment

    Mar 12, 2026Zhiyu Xue, Zimo Qi, Guangliang Liu +2LLM Safety AlignmentOver-Refusal

    University of California, Santa Barbara · Johns Hopkins University · Michigan State University +1

  6. IH-Challenge: A Training Dataset to Improve Instruction Hierarchy on Frontier LLMs

    Mar 11, 2026Chuan Guo, Juan Felipe Ceron Uribe, Sicheng Zhu +10Adversarial Robustness of Language ModelsPrompt Injection Attacks on AI Agents

    Michael Pokorny · OpenAI

  7. Robustness of Agentic AI Systems via Adversarially-Aligned Jacobian Regularization

    Mar 4, 2026Furkan Mumcu, Yasin YilmazAdversarial RobustnessAdversarial Robustness of Language Models

    University of South Florida

  8. Inherited Goal Drift: Contextual Pressure Can Undermine Agentic Goals

    Mar 3, 2026Achyutha Menon, Magnus Saebo, Tyler Crosse +3Language Model RobustnessEmergent Misalignment in Language Models

    UC San Diego · Columbia University · Georgia Tech +3

  9. From Shallow to Deep: Pinning Semantic Intent via Causal GRPO

    Mar 3, 2026Shuyi Zhou, Zeen Song, Wenwen Qiang +4LLM AlignmentAdversarial Robustness of Language Models

    University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China

  10. ExpGuard: LLM Content Moderation in Specialized Domains

    Mar 3, 2026Minseok Choi, Dongjin Kim, Seungbin Yang +5LLM GuardrailsContent Moderation

    KAIST AI · Financial Tech Lab, KakaoBank Corp

  11. Enhancing Safety of Large Language Models via Embedding Space Separation

    Mar 1, 2026Xu Zhao, Xiting Wang, Weiran ShenLLM SafetyLLM Fine-Tuning

    Gaoling School of Artificial Intelligence, Renmin University of China.

  12. The Synthetic Web: Adversarially-Curated Mini-Internets for Diagnosing Epistemic Weaknesses of Language Agents

    Feb 28, 2026Shrey Shah, Levent OzgurAdversarial Robustness of Language ModelsAI Agent Benchmarks

  13. Evaluation of Audio Language Models for Fairness, Safety, and Security

    Feb 25, 2026Ranya Aloufi, Srishti Gupta, Soumya Shaw +2Audio-Language Model EvaluationAudio-Language Models

  14. Support Tokens, Stability Margins, and a New Foundation for Robust LLMs

    Feb 25, 2026Deepak Agarwal, Dhyey Dharmendrakumar Mavani, Suyash Gupta +2Self-AttentionLanguage Model Robustness

    LinkedIn, Mountain View, CA

  15. Detoxifying LLMs via Representation Erasure-Based Preference Optimization

    Feb 24, 2026Nazanin Mohammadi Sepahvand, Eleni Triantafillou, Hugo Larochelle +3LLM SafetyText Detoxification

    McGill University, Canada · Mila, Canada · Google DeepMind +2

  16. BarrierSteer: LLM Safety via Learning Barrier Steering

    Feb 23, 2026Thanh Q. Tran, Arun Verma, Kiwan Wong +3LLM SafetyAdversarial Robustness of Language Models

    Department of Computer Science, National University of Singapore, Republic of Singapore · Singapore-MIT Alliance for Research and Technology Centre, Republic of Singapore · CSAIL, Massachusetts Institute of Technology, USA +1

  17. MANATEE: Inference-Time Lightweight Diffusion Based Safety Defense for LLMs

    Feb 21, 2026Chun Yan Ryan Kan, Tommy Tran, Vedant Yadav +4Jailbreak DefenseLLM Safety

  18. Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning

    Feb 19, 2026Jyotin Goel, Souvik Maji, Pratik MazumderSafety-Preserving Fine-Tuning of Language ModelsLLM Safety

    Indian Institute of Technology Jodhpur Jodhpur, Rajasthan, India

  19. Fail-Closed Alignment for Large Language Models

    Feb 19, 2026Zachary Coalson, Beth Sohler, Aiden Gabriel +1LLM AlignmentRefusal Behavior in Language Models

    Oregon State University, Corvallis, OR USA.

  20. NeST: Neuron Selective Tuning for LLM Safety

    Feb 18, 2026Sasha Behrouzi, Lichao Wu, Mohamadreza Rostami +1LLM Safety AlignmentSafety-Preserving Fine-Tuning of Language Models

    Technical University of Darmstadt · University of Bristol

  21. IndicJR: A Judge-Free Benchmark of Jailbreak Robustness in South Asian Languages

    Feb 18, 2026Priyaranjan Pattnayak, Sanchari ChowdhuriAdversarial Robustness of Language ModelsLLM Safety Benchmarks

    Oracle America Inc.

  22. Can Adversarial Code Comments Fool AI Security Reviewers -- Large-Scale Empirical Study of Comment-Based Attacks and Defenses Against LLM Code Analysis

    Feb 18, 2026Scott ThorntonAdversarial AttacksAdversarial Attacks on LLMs

  23. Closing the Distribution Gap in Adversarial Training for LLMs

    Feb 16, 2026Chengzhi Hu, Jonas Dornbusch, David Lüdke +2Adversarial TrainingAdversarial Robustness of Language Models

    Technical University of Munich, Germany.

  24. Routing-Aware Safety Alignment for Mixture-of-Experts Models

    Feb 4, 2026Jiacheng Liang, Yuhui Wang, Tanqiu Jiang +1Mixture-of-Experts Language ModelsLLM Alignment

    Department of Computer Science Stony Brook University

  25. SycoEval-EM: Sycophancy Evaluation of Large Language Models in Simulated Clinical Encounters for Emergency Care

    Jan 23, 2026Dongshen Peng, Yi Wang, Austin Schoeffler +5LLM SycophancyLanguage Model Safety Evaluation

    UNC Chapel Hill · University of Waterloo · Stanford University

  26. AdversaRiskQA: An Adversarial Factuality Benchmark for High-Risk Domains

    Jan 21, 2026Adam Szelestey, Sofie van Engelen, Tianhao Huang +3Factual Consistency EvaluationLLM Reliability

    Eindhoven University of Technology Eindhoven, Netherlands

  27. OI-Bench: An Option Injection Benchmark for Evaluating LLM Susceptibility to Directive Interference

    Jan 19, 2026Yow-Fu Liou, Yu-Chien Tang, Yu-Hsiang Liu +1LLM EvaluationAdversarial Robustness of Language Models

    Department of Computer Science, National Yang Ming Chiao Tung University, Taiwan

  28. The Straight and Narrow: Do LLMs Possess an Internal Moral Path?

    Jan 15, 2026Luoming Hu, Jingjie Zeng, Liang Yang +1Moral Reasoning in Language ModelsInference-Time Language Model Alignment

    School of Future Technology, Dalian University of Technology, China · School of Computer Science and Technology, Dalian University of Technology, China · Key Laboratory of Social Computing and Cognitive Intelligence, Ministry of Education, China

  29. Safety boundary maintenance in consumer AI systems responding to pediatric health queries: a cross-platform benchmark evaluation under naturalistic and adversarially pressured conditions

    Dec 26, 2025Vahideh Zolfaghari, Leila Mashhadi, Mitra Ahadi +2HealthcareLLM Safety

    Medical Sciences Education Research Center, Mashhad University of Medical Sciences, Mashhad, Iran. · Student Research Committee, Department of Health Information Technology and Management, Medical Informatics, School of Allied Medical Sciences, Shahid Beheshti University of Medical Sciences, Tehran, Iran. · Department of Anesthesiology and Intensive Care, Mashhad University of Medical Sciences, Mashhad, Iran. +2

  30. Learning-Based Automated Adversarial Red-Teaming for Robustness Evaluation of Large Language Models

    Dec 21, 2025Zhang Wei, Hanxuan Chen, Peilu Hu +19Adversarial Robustness of Language ModelsLLM Safety Evaluation

    Independent Researcher · Hunan University · Shenzhen Kaihong Digital Industry Development Co., Ltd. +12