LLM Safety Alignment

LLM: Large Language Model

Overview

Large language model safety alignment studies how to make models follow intended rules, avoid enabling harm, and remain useful when requests are ambiguous or adversarial. Common approaches include supervised safety fine-tuning and preference optimization such as reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO), adversarial red-teaming, and inference-time controls such as refusal steering, prompt routing, or guard models. Evaluations use jailbreak and harmful-request suites such as HarmBench and JailbreakBench, alongside tests of over-refusal, multilingual and multimodal inputs, fine-tuning drift, and tool-using agents. A central challenge is achieving robust, context-sensitive safety without blocking benign help: refusals can fail under new formats, languages, or deployment pressures, while stronger safeguards can reduce utility and do not always predict real agent behavior.

Latest papers

CardsList

364 papers24 papers in the last four weeks, down 40% on the four weeks before. 0.2% of all new papers.

  1. Worldviews in Large Language Models

    Nov 17, 2026Edmundo Lopez-Sola, Shamil Chandaria, Ruben LaukkonenLLM Safety AlignmentLLM World Models

  2. DACE: Diversity-Driven Adversarial Co-Evolution for Robust LLM Safety Alignment

    Nov 17, 2026Peng Yu, Xiaoyu Wen, Zhida He +6LLM SafetyLLM Safety Alignment

  3. Aligning AI Teams

    Nov 13, 2026Siddarth Srinivasan, Morgan Jane Matthews, Jascha Sohl-Dickstein +1LLM Safety AlignmentMulti-Agent LLMs

  4. Alignment with Reflect-Reasoning: Learning How To Follow Principles

    Nov 12, 2026Henry Bell, Caroline Zhang, Dhaval Potdar +2LLM Safety AlignmentMetacognition in Language Models

  5. Bait: Mitigating Harmful Agentic Reinforcement Learning via Probing Reward Function

    Nov 12, 2026Tiansheng Huang, Ananth Balashankar, Virat Shejwalkar +2AI Agent SecurityLLM Agent Safety

  6. Data-Efficient LLM Safety Alignment Guided by Interpretable Features

    Nov 12, 2026Hongyu Li, Zhan Yu, Zhe Zhao +3LLM Safety Alignment

  7. META-IR: Meta-Introspective Reasoning for Adaptive Safety Alignment in LLM

    Nov 6, 2026Mahabubur Rahman Miraj, Hongyu Huang, Nankun Mu +2LLM Safety AlignmentAdversarial Robustness of Language Models

  8. Alignment Investigation of Mythos 5's behaviour in UK AISI's July 2026 Security Incident

    Nov 5, 2026Olli Järviniemi, Alexandra Souly, Abby D'Cruz +3LLM Safety AlignmentLLM Security

  9. SAVeR2^2: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards

    Nov 5, 2026Yichen Sun, Qirui Liu, Yan Wang +5LLM Safety AlignmentSafety-Preserving Fine-Tuning of Language Models

  10. Addition by Subtraction: Toward Mitigating Fragile Safety Alignment in Large Language Models

    Nov 4, 2026Hamid Dashtbani, Alexandra DmitrienkoLLM Safety AlignmentSafety-Preserving Fine-Tuning of Language Models

  11. TraceRouter++: Causal Semantic Routing Realignment via Optimal Transport

    Nov 3, 2026Chuancheng Shi, Shangze Li, Wenhua Wu +5LLM SafetyLLM Safety Alignment

  12. Rewarded for Not Getting Caught: Why Alignment Training Selects for Conditional Compliance and Cannot Instill Norms

    Nov 1, 2026Kevin Baum, Ruta Binkytė, Gerrit Großmann +2AI AlignmentLLM Safety Alignment

  13. ShallowDefend: Harnessing Shallow Safety Alignment for Multi-Turn Defender

    Oct 29, 2026Qiyao Ma, Xiaodong Liu, Weiwei Yang +9LLM Safety AlignmentJailbreak Defense

  14. Aligning Large Language Models for Safety in African Languages: A Case Study of Hausa

    Oct 27, 2026Valdini Douglace Lemofouet, Blessing Ngozi Uzor, Paula Chikaodinaka Anyanwu +4LLM Safety AlignmentLanguage Model Safety Evaluation

  15. MedConstBench: Benchmarking How Constitutions Steer LLMs in Medical Ethics

    Oct 27, 2026Yibo Yang, Zerui Huang, Haomiao Yang +1Medical LLMsMoral Reasoning in Language Models

  16. When Personas Become Harmful: Guiding LLMs with Global Citizenship Values Alignment

    Oct 24, 2026Siqi Shen, Lajanugen Logeswaran, Moontae Lee +2LLM SafetyLLM Safety Alignment

  17. Auditing Safety Preference Supervision with Recorded Derivations

    Oct 19, 2026Chenhan Lyu, Amir RahmaniLLM AuditingLLM Safety Alignment

  18. Safety Mid-Training: Internalizing LLM Safety as a Foundational Capability

    Oct 19, 2026Zhengyue Zhao, Liwei Jiang, Yejin Choi +1Safety-Utility Trade-Off in Language ModelsLLM Safety Alignment

  19. Training Natural Schemers: RL Can Amplify Oversight-Conditional Behavior

    Oct 19, 2026Nathaniel Mitrani Hadida, Bart Jaworski, Fred Bruford +1Strategic Reasoning Risks in Language ModelsLLM Safety Alignment

  20. Safety Offset Learning: When Benign Fine-Tuning Misaligns LLMs

    Oct 19, 2026Jesson Wang, Zhanhao Hu, Minghao Lin +3LLM Fine-TuningLLM Safety Alignment

  21. Rethinking GRPO for Large Reasoning Model Safety Alignment

    Oct 16, 2026Jiawei Chen, Yi Zhang, Jiawei Du +9Safety-Preserving Fine-Tuning of Language ModelsLLM Safety Alignment

  22. Identify, Classify, Act: Per-Instruction Controlled Compliance for LLMs over External Data

    Oct 16, 2026Zesheng Shi, Xue Li, Chenrui Dai +1Prompt Injection DefenseLLM Safety Alignment

  23. ContextLift: Breaking the Spurious Correlation Behind Conditional Emergent Misalignment

    Oct 15, 2026Florian Mai, David KaczérEmergent Misalignment in Language ModelsLLM Safety Alignment