LLM Safety Alignment

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

25 papers in the last four weeks, up 14% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 236

All topics

Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systems
Jun 4, 2026Dianxing Shi, Bowen Wang, Junqi He +2Human-in-the-Loop Evaluation LLM Safety Alignment
The University of Osaka · Independent researcher
Consistency Training Along the Transformer Stack
Jun 4, 2026Sukrati Gautam, Neil Shah, Arav Dhoot +7Adversarial Attacks on LLMs LLM Safety Alignment
1Purdue University · 2Independent · 3Columbia University +4
Safety Paradox: How Enhanced Safety Awareness Leaves LLMs Vulnerable to Posterior Attack
Jun 4, 2026Long P. Hoang, Hai V. Le, Shaoyang Xu +2Adversarial Attacks on LLMs LLM Safety Alignment
Singapore University of Technology and Design · Nanyang Technological University
CHASE: Adversarial Red-Blue Teaming for Improving LLM Safety using Reinforcement Learning
Jun 4, 2026Rahul Markasserithodi, Aditya Joshi, Yuekang Li +3Adversarial Training Adversarial Prompt Generation
University of New South Wales, Australia
Self-Recognition Finetuning can Prevent and Reverse Emergent Misalignment
Jun 4, 2026Arush Tagade, Shaoheng Zhou, Jiaxin Wen +1Emergent Misalignment in Language Models LLM Safety Alignment
George Washington University · Google · UC Berkeley
Willing but Unable: Separating Refusal from Capability in Code LLMs via Abliteration
Jun 3, 2026Cristina Carleo, Pietro Liguori, Naghmeh Ivaki +1LLM Safety Alignment LLMs for Cybersecurity
University of Naples Federico II, Naples, Italy · University of Coimbra, CISUC/LASI, Coimbra, Portugal · University of North Carolina at Charlotte, Charlotte, NC, USA
Inference-Time Vulnerability Beyond Shallow Safety: Alignment Along Generation Trajectories
Jun 3, 2026Kyungmin Park, Taesup KimAdversarial Attacks on LLMs LLM Safety Alignment
Hankuk University of Foreign Studies · Seoul National University
When Autoregressive Consistency Hurts Safety Alignment
Jun 2, 2026Bochen Lyu, Yiyang Jia, Xiaohao Cai +1Adversarial Attacks on LLMs LLM Safety Alignment
University of Southampton · Independent Researcher
Expert-Aware Refusal Steering
Jun 2, 2026Anna C. Marbut, Daniel R. Olson, Travis J. WheelerMixture-of-Experts Language Models Language Model Steering
Department of Interdisciplinary Studies University of Montana Missoula, MT, USA · Department of Pharmacy Practice & Science University of Arizona Tucson, AZ, USA · European Bioinformatics Institute European Molecular Biology Laboratory Wellcome Genome Campus, Hinxton, United Kingdom
Consistency Training Can Entrench Misalignment
Jun 2, 2026David Demitri Africa, Arathi ManiLLM Safety Alignment Emergent Misalignment
1UK AI Security Institute
SafeGene: Reusable Adapters for Transferable Safety Alignment
Jun 2, 2026Yanghan Wang, Zhiqiang Kou, Fu Feng +2LLM Fine-Tuning LLM Safety Alignment
Southeast University · The Hong Kong Polytechnic University
Constitutional On-Policy Safe Distillation
Jun 2, 2026Ming Wen, Yuxuan Liu, Kun Yang +8On-Policy Self-Distillation LLM Safety Alignment
Institute of Trustworthy Embodied AI, Fudan University · 2Shanghai Innovation Institute · 3Ant Group +3
Unpredictable Safety: Domain-Dependent Compliance and the Transparency Gap in Open-Weight LLMs
Jun 1, 2026Zacharie BugaudLLM Safety Alignment LLM Safety
Astera Institute
SafeSteer: Localized On-Policy Distillation for Efficient Safety Alignment
Jun 1, 2026Hao Li, Jingkun An, Zijun Song +8LLM Safety Alignment On-Policy Distillation
Beihang University · Beijing Institute of Technology · Beijing University of Posts and Telecommunications +4
Two to Tango: Coupled Task-Reference Selection for Safe LLM Fine-tuning
Jun 1, 2026Xinrui Chen, Jianhao Zhang, Ou Wu +1LLM Fine-Tuning Data Selection
Hangzhou Institute for Advanced Study, University of Chinese Academy of Sciences
Alignment Collapse Under KV Cache Quantization: Diagnosis and Mitigation
Jun 1, 2026Bruce Changlong Xu, Adarsh Kumarappan, Mu ZhouLLM Quantization KV Caching
†Stanford University · ‡California Institute of Technology
Trait-space Monitoring for Emergent Misalignment During Supervised Finetuning
May 31, 2026Huy Nghiem, Sy-Tuyen Ho, Sarah Wiegreffe +1Supervised Fine-Tuning Emergent Misalignment in Language Models
University of Maryland
Lost in Delusion: Examining LLM Safety Under User Delusions and Distress
May 31, 2026Andrew Aquilina, Chetna Nihalani, Vasudha Varadarajan +3Mental Health LLM Safety Alignment
University of Pittsburgh · Carnegie Mellon University · Fordham University
Cross-Generational Transfer of Adversarial Attacks Reveals Non-Monotonic Safety Alignment in LLMs
May 30, 2026Subhadip MitraAdversarial Attacks on LLMs LLM Red Teaming
Research Lead, Rota Labs
Dialectics of Alignment: Harnessing Unsafe Knowledge for Dynamic Safety Routing
May 30, 2026Maryam Hashemzadeh, Jerry Huang, Minseon Kim +2LLM Safety Alignment LLM Safety
Chandar Research Lab · Mila – Quebec AI Institute · Université de Montréal +3
MESA: Improving MoE Safety Alignment via Decentralized Expertise
May 30, 2026Yitong Sun, Yao Huang, Teng Li +5Mixture-of-Experts Language Models LLM Safety Alignment
Institute of Artificial Intelligence, State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing, 100191, China · College of AI, Tsinghua University, Beijing, 100083, China · School of Computer Science, Fudan University, Shanghai, 200438, China +2
ConsisGuard: Aligning Safety Deliberation with Policy Enforcement in LLM Guardrails
May 29, 2026Yan Wang, Zhixuan Chu, Zihao Xue +9LLM Guardrails LLM Safety Alignment
Alibaba Group · Zhejiang University · Huzhou Normal University +1
COMPASS: Cognitive MCTS-Guided Process Alignment for Safe Search Agents
May 29, 2026Wenkai Shen, Pengyang Zhou, Jiahe Xu +5LLM Safety Alignment LLM Agents
Zhejiang University
Configurable Reward Model for Balanced Safety Alignment
May 28, 2026Zhengping Jiang, Mehran Khodabandeh, Akash Bharadwaj +5Reward Modeling Language Model Safety Evaluation
Johns Hopkins University · Meta · Meta Superintelligence Labs
LLUMI: Improving LLM Writing Assistance for Mental Health Support with Online Community Feedback
May 28, 2026Jiwon Kim, Maya Ajit, Sherry Gong +4Emotional Support Conversation Privacy-Preserving Language Models
University of Illinois Urbana-Champaign · Indiana University Indianapolis
AgentDoG 1.5: A Lightweight and Scalable Alignment Framework for AI Agent Safety and Security
May 28, 2026Dongrui Liu, Yu Li, Zhonghao Yang +47LLM Safety Alignment AI Agent Security
Understanding Safety-Sensitive Expert Behavior in Mixture-of-Experts LLMs
May 28, 2026Zhibo Zhang, Yuxi Li, Zhen Ouyang +2Mixture-of-Experts Language Models LLM Auditing
Huazhong University of Science and Technology, Wuhan, China · Nanyang Technological University, Singapore
Aligned but Fragile: Enhancing LLM Safety Robustness via Zeroth-Order Optimization
May 28, 2026Zhihao Liu, Yifan Wu, Jian Lou +3LLM Alignment Zeroth-Order Optimization
The State Key Laboratory of Blockchain and Data Security, Zhejiang University · Hangzhou High-Tech Zone (Binjiang) Institute of Blockchain and Data Security · Sun Yat-sen University +1
Multi-Adapter Representation Interventions via Energy Calibration
May 27, 2026Manjiang Yu, Hongji Li, Junwei Chen +4LLM Alignment LLM Safety Alignment
The University of Queensland, Brisbane, Australia · Mohamed bin Zayed University of Artificial Intelligence, Abu Dhabi, United Arab Emirates · Institute of Science Tokyo, Tokyo, Japan
The Ethics of LLM Sandbox and Persona Dynamics
May 27, 2026Tim Gebbie, Stewart GebbieLLM Guardrails LLM Safety Alignment
Dept. Statistical Sciences, University of Cape Town, South Africa

Jun 4, 2026, cs.AIJ/K move · Enter open · S save

Towards Healthy Evolution: Exploring the Role and Mechanisms of Human-Agent Interaction in Self-Evolving Systems

Dianxing Shi, Bowen Wang, Junqi He, Junhao Chen+1

The University of Osaka · Independent researcher

Human-in-the-Loop Evaluation LLM Safety Alignment LLM Agent Safety Self-Evolving Agents

Open paper AskPDF arXiv

Self-evolving agents improve through continual self-play and self-generated learning signals, but autonomous evolution can also cause capability degradation and safety drift. Although human feedback has proven effective for static and post-trained agents, its role in self-evolving systems remains underexplored. We introduce Agent Norm Correction through Human-like Oversight and Review (ANCHOR), an LLM-based framework that simulates human supervision and delivers feedback at various phases of self-evolution. With ANCHOR, we evaluate two representative open-source self-evolving agent systems across coding, mathematical reasoning, and safety. Our results show that even limited supervision substantially mitigates safety degradation while preserving stable performance on core evolutionary objectives. Further analysis shows that supervision over the output verification phase is the most effective for intervention, whereas increasing supervision frequency yields diminishing returns. These findings provide empirical evidence and practical guidance for designing more stable, controllable, and human-aligned self-evolving agent systems.

Momentum

Related topics

Latest papers 236