LLM Safety Alignment

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

25 papers in the last four weeks, up 14% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 236

All topics

The Profit Alignment Problem: How Profit Mandates Induce Alignment Failures in LLMs
Sep 7, 2026Eric SoLLM Alignment Language Model Safety Evaluation
MIT Sloan School of Management Massachusetts Institute of Technology
HarvestBench: Measuring Whether LLM Agents Will Pay to Avoid Killing Animals
Sep 3, 2026Jasmine Brazilek, Miles Tidmarsh, Matthias Endres +2Moral Reasoning in Language Models LLM Safety Alignment
Compassion Aligned Machine Learning (CaML) · Department of Economics, University of Warwick
Representational alignment yields generalizable safety in language models
Sep 3, 2026Lingyu Li, Yan Teng, Yingchun Wang +1Moral Reasoning in Language Models LLM Safety Alignment
Shanghai Artiﬁcial Intelligence Laboratory, Shanghai, China
SafeEvolve: Harness-Policy Co-Evolution from Agent Experience for Safety Alignment
Sep 2, 2026Qinghua Mao, Wanying Qu, Dadi Guo +8LLM Safety Alignment LLM Agent Safety
Shanghai AI Laboratory · SJTU · Fudan University +2
SEAL: Reinforcing Global Safety in Mixture-of-Experts through Shared Expert ALignment
Sep 2, 2026Qingyu Meng, Yiwei Zha, Jiahuan Pei +3LLM Safety Alignment Jailbreak Defense
Vrije Universiteit Amsterdam Amsterdam, Netherlands
When Safety Routing Breaks: Understanding Alignment Fragility under Benign Fine-Tuning
Sep 1, 2026Yitong Guo, Xiaoyi Chen, Siyuan Zhang +2LLM Fine-Tuning LLM Safety Alignment
Indiana University Bloomington · Tsinghua University · Nanyang Technological University
Trust Your Guide Only When Certain: Uncertainty-Aware Sparse Alignment at Inference Time
Sep 1, 2026Zeen Zhu, Zhuo Li, Weiyang Guo +4LLM Safety Alignment Inference-Time Language Model Alignment
Harbin Institute of Technology, Shenzhen, China · Huawei Technologies Co., Ltd. · Beijing Academy of Artificial Intelligence, China.
Safin-1: Safety from Within through Memory-Native State Evolution
Aug 31, 2026Ming Zhang, Kaisen Yang, Shu Yu +15Memory-Augmented Language Models LLM Safety Alignment
Shanghai AI Laboratory
Beyond Token-Level Guidance: Inference-Time Alignment of Specialized LLMs via Cross-Family Representation Steering
Aug 31, 2026Jin Gan, Xin Li, Jun LuoLLM Safety Alignment Language Modeling
College of Computing and Data Science, Nanyang Technological University, Singapore · College of Cryptology and Cyber Science, Nankai University, Tianjin, China
ALTSTEER: Selective Safety Steering for Moving Beyond Hard Refusals to Constructive Alternatives
Aug 31, 2026Hoejoon Kwon, Byeonggeuk Lim, Kahyeon Kim +1Language Model Steering LLM Safety Alignment
Department of Artificial Intelligence, Chung-Ang University · Graduate School of Advanced Imaging Science, Multimedia & Film, Chung-Ang University
Balancing Privacy, Utility, and Safety in LLM Alignment through Preference Optimization
Aug 31, 2026Dishu Yang, Jingjing Liu, Jize LiLLM Alignment Memorization in Language Models
Khoury College of Computer Sciences Northeastern University San Jose, USA · Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, USA · Department of Economics Boston University Boston, USA
When Safety Speaks a Language: A Mechanistic Analysis of Safety-Language Identity Entanglement in LLMs
Aug 30, 2026Apoorva Upadhyaya, Sandipan SikdarLLM Safety Alignment Mechanistic Interpretability
L3S Research Center, Leibniz Universität Hannover, Germany
Automated Researchers Can Mitigate Well-characterized Alignment Failures
Aug 28, 2026Chen Yueh-Han, Jiaxin Wen, Jan Hendrik KirchnerAI Alignment LLM Safety Alignment
1Anthropic Fellows Program · 2Anthropic · 3UC Berkeley
Synthetic Persona Pretraining: Alignment from Token Zero
Aug 13, 2026Julian Minder, Viktor Moskvoretskii, Raghav Singhal +12Synthetic Data Pretraining LLM Alignment
EPFL · SJTU · Northeastern University +8
Refusing Intent, Not Form: Wrapper-Based Intent-Group Supervision for LLM Safety
Aug 13, 2026Ping Wu, Haibo Tong, Feifei Zhao +7LLM Safety Alignment LLM Refusal Behavior
BrainCog Lab, Institute of Automation, Chinese Academy of Sciences · School of Artificial Intelligence, University of Chinese Academy of Sciences · Beijing Key Laboratory of Safe AI and Superalignment +2
Follow the Norm: Accounting for Fine-Tuning and Prompt Effects on Model Rationales
Aug 13, 2026Long Hoang Nguyen, Brice Valentin Kok-Shun, Guangyu Du +1Moral Reasoning in Language Models Normative Conformity in Language Models
1Technical University of Munich · University of Auckland
HiRoute: Hierarchical Routed Prompt Tuning for Safety Alignment of Large Language Models
Aug 13, 2026Fangzhou Chen, Shiji Zhao, Mengyang Wang +4LLM Safety Alignment Prompt Tuning
Institute of Artificial Intelligence, Beihang University
ToolHazard: Scaling Adversarial Environments for Security Evaluation and Alignment of LLM-based Agents
Aug 12, 2026Yutao Mou, Pengfei Yang, Zhe Yin +6AI Agent Evaluation Adversarial Scenario Generation
National Engineering Research Center for Software Engineering, Peking University · Harbin Institute of Technology · Beijing University of Posts and Telecommunications +1
Making Your LLMs More Objective: Stabilizing LLM Safety Behavior Across Traits with Trait-Invariant Safety Tuning
Aug 12, 2026Lang CaoLLM Alignment Language Model Safety Evaluation
University of Illinois Urbana-Champaign
From Interpretability to Control: Insights from Six Years of the TrustNLP Workshop
Aug 11, 2026Rahul Gupta, Abhinav Mohanty, Anaelia Ovalle +10LLM Interpretability LLM Safety Alignment
Amazon AGI · Meta · Autodesk +3
The Illusion of Cross-Lingual Safety in Low-Resource Languages
Aug 11, 2026Abigail Oppong, P Sam Sahil, Tadesse Destaw Belay +12LLM Safety Alignment Low-Resource Language Processing
Makerere University Center for Artificial Intelligence · University of Hamburg · Instituto Politécnico Nacional +9
Dual-Adversarial Safety Alignment: Cultivating Intrinsic Threat Comprehension in LRMs
Aug 10, 2026Hongli Shen, Shaopeng Fu, Qinbo Zhang +2Adversarial Training Adversarial Attacks on LLMs
1King Abdullah University of Science and Technology · 2Stony Brook University
Who Bridges Safety? Identifying and Targeting Cross-Lingual Shared Safety Pathways
Aug 10, 2026Shuyi Miao, Wangjie Qiu, Pengyang Shao +4LLM Safety Alignment LLM Safety
Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing · School of Artificial Intelligence, Beihang University, China · Zhongguancun Laboratory, Beijing, China +2
Safety Cost of Steering Vectors Is Separable and Reducible
Aug 9, 2026Yuxiao Li, Gjergji KasneciLanguage Model Steering LLM Safety Alignment
Technical University of Munich Munich Center for Machine Learning (MCML)
Decided Upstream, Written Late: Locating and Pricing the Cross-Lingual Refusal Circuit of a Multilingual MoE
Aug 8, 2026Ramakrishna P. Kompella, Aadit MahajanMultilingual Language Models Mixture-of-Experts Language Models
Gödel Machines · IIT Madras
Diffusion LLMs as Targets and Adversaries: Mechanistic Safety Exploits
Aug 7, 2026Elena Dumitrescu, Gert Lek, Lydia Y. Chen +1Adversarial Attacks on LLMs LLM Safety Alignment
Delft University of Technology, Delft, The Netherlands · University of Neuchâtel, Neuchâtel, Switzerland.
Mood Matters: How Syntactic Sensitivity Undermines Safety Alignment
Aug 5, 2026Alina Klerings, Jannik Brinkmann, Heiner Stuckenschmidt +1Adversarial Attacks on LLMs LLM Safety Alignment
University of Mannheim · Technical University Clausthal
Gradient Immunity: Null-Space Resistance to Malicious Fine-Tuning
Aug 5, 2026Yuxuan Huang, Xingyu Zeng, Tianhang Zheng +1LLM Safety Alignment LLM Security
Shanghai Artificial Intelligence Laboratory · Shanghai Jiao Tong University · Shenzhen University of Advanced Technology +1
Looking in the Mirror: Introspecting Side-Effect Misalignments Induced by Fine-Tuning
Aug 5, 2026Kotaro Yoshida, Laura Gomezjurado Gonzalez, Yukinori Yamamoto +3LLM Auditing Language Model Introspection
Institute of Science Tokyo · 2ZOZO Research · 3Stanford University +2
DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning
Aug 5, 2026Junbo Zhang, Qianli Zhou, Xinyang Deng +1Language Model Safety Evaluation LLM Fine-Tuning
Northwestern Polytechnical University

Sep 7, 2026, cs.AIJ/K move · Enter open · S save

The Profit Alignment Problem: How Profit Mandates Induce Alignment Failures in LLMs

Eric So

MIT Sloan School of Management Massachusetts Institute of Technology

LLM Alignment Language Model Safety Evaluation LLM Safety Alignment

Open paper AskPDF arXiv

We show that ordinary business language --- "maximize profitability" --- induces profit-oriented ambiguity resolution: LLMs systematically dismiss ambiguous signals of potential safety violations to serve business objectives. In 3,600 controlled trials across eight reasoning-capable LLMs, adding a profit mandate to otherwise identical prompts increases risk-dismissing judgments by 6.8 percentage points (p < 0.0001), suppresses board escalation recommendations by 13.9pp (p < 0.0001), and shifts severity assessments downward (p < 0.0001). The mandate never instructs models to downplay risks; instead, chain-of-thought traces reveal motivated reasoning: models acknowledge concerns, then invoke profit logic to justify dismissing them. We characterize these findings as the Profit Alignment Problem: when AI systems are given ordinary business objectives, they develop systematic strategies for suppressing inconvenient information that no designer intended or specified.

Momentum

Related topics

Latest papers 236