LLM Safety Alignment

LLM: Large Language Model

Large language model safety alignment studies how to make models follow intended rules, avoid enabling harm, and remain useful when requests are ambiguous or adversarial. Common approaches include supervised safety fine-tuning and preference optimization such as reinforcement learning from human feedback (RLHF) and direct preference optimization (DPO), adversarial red-teaming, and inference-time controls such as refusal steering, prompt routing, or guard models. Evaluations use jailbreak and harmful-request suites such as HarmBench and JailbreakBench, alongside tests of over-refusal, multilingual and multimodal inputs, fine-tuning drift, and tool-using agents. A central challenge is achieving robust, context-sensitive safety without blocking benign help: refusals can fail under new formats, languages, or deployment pressures, while stronger safeguards can reduce utility and do not always predict real agent behavior.

Latest papers

347 papers40 papers in the last four weeks, up 38% on the four weeks before. 0.2% of all new papers.

GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuning
Mar 10, 2026Zhouxiang Fang, Jiawei Zhou, Hanjie ChenSafety-Preserving Fine-Tuning of Language Models LLM Safety Alignment
Rice University · Stony Brook University
Multi-Stream Perturbation Attack: Breaking Safety Alignment of Thinking LLMs Through Concurrent Task Interference
Mar 10, 2026Fan YangAdversarial Attacks on LLMs LLM Jailbreak Attacks
Jinan University
OOD-MMSafe: Advancing MLLM Safety from Harmful Intent to Hidden Consequences
Mar 10, 2026Ming Wen, Kun Yang, Jingyu Zhang +4MLLM Safety LLM Safety
Fudan University, Shanghai, China · Shanghai Innovation Institute, Shanghai, China · Ant Group, Hangzhou, China +1
Finding and Reactivating Post-Trained LLMs' Hidden Safety Mechanisms
Mar 10, 2026Mingjie Li, Wai Man Si, Michael Backes +2LLM Safety LLM Safety Alignment
CISPA Helmholtz Center for Information Security · State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University · Institute for Artificial Intelligence, Peking University
The Reasoning Trap -- Logical Reasoning as a Mechanistic Pathway to Situational Awareness
Mar 10, 2026Subramanyam Sahoo, Aman Chadha, Vinija Jain +1Strategic Reasoning Risks in Language Models LLM Safety Alignment
MARS 4.0 Fellowship, Cambridge AI Safety Hub(CAISH), University of Cambridge · AWS Generative AI Innovation Center, Amazon Web Services, USA · Stanford University +2
The Struggle Between Continuation and Refusal: A Mechanistic Analysis of the Continuation-Triggered Jailbreak in LLMs
Mar 9, 2026Yonghong Deng, Zhen Yang, Ping Jian +4LLM Jailbreak Attacks LLM Safety Alignment
School of Computer Science and Technology, Beijing Institute of Technology, Beijing, China · Beike Language and Intelligence, Beijing, China
Invisible Safety Threat: Malicious Finetuning for LLM via Steganography
Mar 9, 2026Guangnian Wan, Xinyin Ma, Gongfan Fang +1Language Model Steganography LLM Security
National University of Singapore
Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
Mar 8, 2026Guoli Wang, Haonan Shi, Tu Ouyang +1Safety-Preserving Fine-Tuning of Language Models LLM Safety Alignment
Case Western Reserve University, Cleveland, OH, USA
CompassDPO: Dynamics-Controlled Direct Preference Optimization for Robust Safety Alignment
Mar 7, 2026Jilong Liu, Yonghui Yang, Pengyang Shao +5Direct Preference Optimization LLM Safety Alignment
Hefei University of Technology · National University of Singapore
SAHOO: Safeguarded Alignment for High-Order Optimization Objectives in Recursive Self-Improvement
Mar 6, 2026Subramanyam Sahoo, Aman Chadha, Vinija Jain +1LLM Safety Alignment LLM Self-Refinement
MARS 4.0 Fellowship, Cambridge AI Safety Hub(CAISH), University of Cambridge · AWS Generative AI Innovation Center, Amazon Web Services, USA · Stanford University +2
Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignment
Mar 6, 2026Jingyuan Feng, Andrew Gambardella, Gouki Minegishi +3LLM Safety Alignment LLM Alignment
The University of Tokyo.
Ambiguity Collapse by LLMs: A Taxonomy of Epistemic Risks
Mar 6, 2026Shira Gur-Arieh, Angelina Wang, Sina FazelpourLLM Safety LLM Safety Alignment
Harvard University, United States · Cornell Tech, United States · Northeastern University, United States
Knowing without Acting: The Disentangled Geometry of Safety Mechanisms in Large Language Models
Mar 6, 2026Jinman Wu, Yi Xie, Shen Lin +2Refusal Mechanisms in Language Models LLM Safety Alignment
Xidian University, Xi’an, China · Tsinghua University, Shenzhen, China · Fujian Normal University, Fuzhou, China +1
Why Is RLHF Alignment Shallow? A Gradient Analysis
Mar 5, 2026Robin YoungLLM Alignment LLM Safety Alignment
Department of Computer Science and Technology University of Cambridge Cambridge, UK
Steering Frozen LLMs: Adaptive Social Alignment via Online Prompt Routing
Mar 4, 2026Zeyu Zhang, Xiangxiang Dai, Ziyi Han +2LLM Alignment LLM Safety Alignment
The Chinese University of Hong Kong · University of Washington
SafeCRS: Personalized Safety Alignment for LLM-Based Conversational Recommender Systems
Mar 3, 2026Haochang Hao, Yifan Xu, Xinzhuo Li +2Recommender Systems Large Language Model-Based Recommendation
University of Illinois at Chicago Chicago, IL, USA · University of Notre Dame South Bend, IN, USA · Amazon Santa Clara, CA, USA
Slurry-as-a-Service: A Modest Proposal on Scalable Pluralistic Alignment for Nutrient Optimization
Mar 2, 2026Rachel Hong, Yael Eiger, Jevan Hutson +2LLM Alignment Pluralistic Alignment
ValueMulch United States
SafeSci: Safety Evaluation of Large Language Models in Science Domains and Beyond
Mar 2, 2026Xiangyang Zhu, Yuan Tian, Qi Jia +14LLM Safety LLM Safety Evaluation
Shanghai AI Lab · ByteDance · ECNU
Defensive Refusal Bias: How Safety Alignment Fails Cyber Defenders
Mar 1, 2026David Campbell, Neil Kale, Udari Madhushani Sehwag +5LLM Safety Alignment LLM Refusal Behavior
Security and Policy Research Lab Scale AI · Security Engineering Scale AI
Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs
Feb 28, 2026Ming Wen, Kun Yang, Xin Chen +4Multimodal Large Language Models LLM Safety Alignment
Fudan University · Shanghai Innovation Institute · Ant Group +3
Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Feb 28, 2026Quoc Minh Nguyen, Trung Le, Jing Wu +2Safety-Preserving Fine-Tuning of Language Models LLM Fine-Tuning
Department of Electrical and Computer Systems Engineering, Monash University, Australia · Department of Data Science and AI, Monash University, Australia
Multilingual Safety Alignment Via Sparse Weight Editing
Feb 26, 2026Jiaming Liang, Zhaoxin Wang, Handing WangLLM Safety Alignment LLM Safety
School of Artificial Intelligence, Xidian University.
Provable Last-Iterate Convergence for Multi-Objective Safe LLM Alignment via Optimistic Primal-Dual
Feb 25, 2026Yining Li, Peizhong Ju, Ness ShroffLLM Safety Alignment RL from Human Feedback
The Ohio State University · University of Kentucky
Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment
Feb 24, 2026Mengxuan Hu, Vivek V. Datla, Anoop Kumar +4LLM Alignment LLM Safety Alignment
University of Virginia · Capital One
The Missing Red Line: How Commercial Pressure Erodes AI Safety Boundaries
Feb 24, 2026Nora Petrova, John BurdenLLM Safety LLM Safety Alignment
Prolific
Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Feb 19, 2026Jyotin Goel, Souvik Maji, Pratik MazumderSafety-Preserving Fine-Tuning of Language Models LLM Safety
Indian Institute of Technology Jodhpur Jodhpur, Rajasthan, India
NeST: Neuron Selective Tuning for LLM Safety
Feb 18, 2026Sasha Behrouzi, Lichao Wu, Mohamadreza Rostami +1LLM Safety Alignment Safety-Preserving Fine-Tuning of Language Models
Technical University of Darmstadt · University of Bristol
Align Once, Benefit Multilingually: Enforcing Multilingual Consistency for LLM Safety Alignment
Feb 18, 2026Yuyan Bu, Xiaohao Liu, ZhaoXing Ren +2LLM Safety Alignment Cross-Lingual Consistency in Language Models
Beijing Academy of Artificial Intelligence · National University of Singapore · Institute for Artificial Intelligence, Peking University
The Geometry of Alignment Collapse: When Fine-Tuning Breaks Safety
Feb 17, 2026Max Springer, Chung Peng Lee, Blossom Metevier +5LLM Safety Alignment LLM Alignment
Princeton University
Mitigating the Safety-utility Trade-off in LLM Alignment via Adaptive Safe Context Learning
Feb 14, 2026Yanbo Wang, Minzheng Wang, Jian Liang +3LLM Alignment RL for Language Model Reasoning
School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China. · NLPR & MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China. · Ritzz-AI.

Mar 10, 2026J/K move · Enter open · S save

GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuning

Zhouxiang Fang, Jiawei Zhou, Hanjie Chen

Rice University · Stony Brook University

Safety-Preserving Fine-Tuning of Language Models LLM Safety Alignment LLM Safety LLM Fine-Tuning Generative Replay

Open paper AskPDF arXiv

Recent studies show that the safety alignment of large language models (LLMs) can be easily compromised even by seemingly non-adversarial fine-tuning. To preserve safety alignment during fine-tuning, a widely used strategy is to jointly optimize safety and task objectives by mixing in the original alignment data, which is typically inaccessible even for open-weight LLMs. Inspired by generative replay in continual learning, we propose Generative Replay for Safety Alignment Preservation (GR-SAP), a unified framework that synthesizes domain-specific alignment data from LLMs and integrate them during downstream adaption to preserve safety alignment. Theoretical and empirical analyses demonstrate this synthetic data serves as a reliable proxy for the original alignment data. Experiments across various models and downstream tasks show that GR-SAP substantially mitigates fine-tuning-induced safety degradation while maintaining comparable downstream performance. Our code is available at https://github.com/chili-lab/gr-sap.