LLM Safety Alignment

LLM: Large Language Model

Ask about this topic

Overview

Large language model safety alignment studies how to make models follow intended rules, avoid enabling harm, and remain useful when requests are ambiguous or adversarial. Common approaches include supervised safety fine-tuning and preference optimization such as reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO), adversarial red-teaming, and inference-time controls such as refusal steering, prompt routing, or guard models. Evaluations use jailbreak and harmful-request suites such as HarmBench and JailbreakBench, alongside tests of over-refusal, multilingual and multimodal inputs, fine-tuning drift, and tool-using agents. A central challenge is achieving robust, context-sensitive safety without blocking benign help: refusals can fail under new formats, languages, or deployment pressures, while stronger safeguards can reduce utility and do not always predict real agent behavior.

Latest papers

CardsList

347 papers40 papers in the last four weeks, up 38% on the four weeks before. 0.2% of all new papers.

  1. Towards Multi-modal Multi-turn Safety: From Agentic Interaction to Strategic Alignment

    Jan 8, 2026Han Zhu, Jiale Chen, Chengkun Cai +8Language Model Safety EvaluationMultimodal Large Language Models

    Hong Kong University of Science and Technology · University College London · AISpeech +1

  2. Agent Tools Orchestration Leaks More: Dataset, Benchmark, and Mitigation

    Dec 18, 2025Yuxuan Qiao, Dongqin Liu, Hongchang Yang +2LLM Agent SecurityLLM Agents

    Institute of Information Engineering, Chinese Academy of Sciences · School of Cyber Security, University of Chinese Academy of Sciences

  3. AlignMerge - Alignment-Preserving Large Language Model Merging via Fisher-Guided Geometric Constraints

    Dec 18, 2025Aniruddha Roy, Jyoti Patel, Aman Chadha +2Language Model MergingLLM Alignment

    HCL · Evalueserve · Apple (USA) +2

  4. Response-Based Knowledge Distillation for Multilingual Jailbreak Prevention Unwittingly Compromises Safety

    Dec 8, 2025Max Zhang, Derek Liu, Kai Zhang +2Adversarial Attacks on LLMsLLM Safety Alignment

  5. MENTOR: A Metacognition-Driven Self-Evolution Framework for Uncovering and Mitigating Implicit Domain Risks in LLMs

    Nov 10, 2025Liang Shan, Kaicheng Shen, Wen Wu +8LLM SafetyLLM Safety Alignment

    School of Computer Science and Technology, East China Normal University · Shanghai AI Lab, Shanghai Innovation Institute

  6. Reasoning Up the Instruction Ladder for Controllable Language Models

    Oct 30, 2025Zishuo Zheng, Vidhisha Balachandran, Chan Young Park +2LLM Safety AlignmentInstruction Following

    Department of Computer Science and Engineering, The Ohio State University · Microsoft Research · Allen Institute for AI

  7. Attention Misses Visual Risk: Risk-Adaptive Steering for Multimodal Safety Alignment

    Oct 15, 2025Jonghyun Park, Minhyuk Seo, Chaewon Yeo +1Multimodal RobustnessLLM Safety Alignment

    Seoul National University · KU Leuven

  8. The Alignment Auditor: A Bayesian Framework for Verifying and Refining LLM Objectives

    Oct 7, 2025Matthieu Bou, Nyal Patel, Arjun Jagota +2Reinforcement LearningLLM Auditing

    1Imperial College London · 2Amazon AGI

  9. Turning the Spell Around: Lightweight Alignment Amplification via Rank-One Safety Injection

    Aug 28, 2025Harethah Abu Shairah, Hasan Abed Al Kader Hammoud, George Turkiyyah +1LLM GuardrailsLLM Safety Alignment

    King Abdullah University of Science and Technology (KAUST)

  10. Context Misleads LLMs: The Role of Context Filtering in Maintaining Safe Alignment of LLMs

    Aug 9, 2025Jinhwa Kim, Ian G. HarrisLLM Safety AlignmentLLM Safety

    Department of Computer Science University of California, Irvine

  11. Chasing Moving Targets with Online Self-Play Reinforcement Learning for Safer Language Models

    Jun 9, 2025Mickel Liu, Liwei Jiang, Yancheng Liang +4Self-Play RLAdversarial Attacks on LLMs

    Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA · Department of Computer Science, Stanford University, Stanford, CA, USA

  12. Safety Alignment via Constrained Knowledge Unlearning

    May 24, 2025Zesheng Shi, Yucheng Zhou, Jing LiLLM Safety AlignmentLLM Unlearning

    Harbin Institute of Technology, Shenzhen, China · University of Macau, China

  13. Decoding One Safety Trigger Token for Balancing Safety and Usability in Large Language Models

    May 12, 2025Haoran Gu, Handing Wang, Yi Mei +2LLM Safety AlignmentLLM Safety

    Xidian University · Victoria University of Wellington · Westlake University

  14. Adaptive Helpfulness-Harmlessness Alignment with Preference Vectors

    Apr 27, 2025Ren-Wei Liang, Chin-Ting Hsu, Chan-Hung Yu +6LLM AlignmentLLM Safety Alignment

    National Taiwan University · Appier AI Research · Graduate Institute of Communication Engineering, National Taiwan University +1

  15. Decoupled Alignment for Robust Plug-and-Play Adaptation

    Jun 3, 2024Haozheng Luo, Jiahao Yu, Wenxin Zhang +9LLM AlignmentLLM Safety Alignment

    Northwestern University · New York University Abu Dhabi · Stanford University +3

  16. Learning diverse attacks on large language models for robust red-teaming and safety tuning

    May 28, 2024Seanie Lee, Minsu Kim, Lynn Cherif +8Adversarial Attacks on LLMsLLM Red Teaming

    KAIST · Mila – Québec AI Institute · Université de Montréal +3

  17. Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge

    Apr 8, 2024Weikai Lu, Ziqian Zeng, Jianwei Wang +5LLM Safety AlignmentLLM Jailbreak Attacks

    South China University of Technology, China · Pazhou Laboratory, China