LLM Alignment

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 118% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 422

All topics

Why Do Aligned LLMs Remain Jailbreakable: Refusal-Escape Directions, Operator-Level Sources, and Safety-Utility Trade-off
May 9, 2026Yu Chen, Yuanhao Liu, Qi CaoLLM Alignment Language Model Steering
Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China
Alignment as Jurisprudence
May 8, 2026Nicholas CaputoLLM Alignment Legal Reasoning
Researcher – Legal Alignment on Frontier AI at the Oxford Martin AI Governance Initiative
Beyond Pairs: Your Language Model is Secretly Optimizing a Preference Graph
May 8, 2026Ning Liu, Chuanneng Sun, Kristina Klinkner +1Pairwise Preference Learning LLM Alignment
Amazon
How Value Induction Reshapes LLM Behaviour
May 8, 2026Arnav Arora, Natalie Schluter, Katherine Metcalf +1LLM Sycophancy LLM Alignment
University of Copenhagen · 2Apple
Post-training makes large language models less human-like
May 8, 2026Marcel Binz, Elif Akata, Abdullah Almaatouq +76LLM Evaluation LLM Alignment
Rubric-based On-policy Distillation
May 8, 2026Junfeng Fang, Zhepei Hong, Mao Zheng +7LLM Alignment Language Model Distillation
National University of Singapore · University of Science and Technology of China · Tencent
Confidence-Aware Alignment Makes Reasoning LLMs More Reliable
May 8, 2026Kejia Chen, Jiawen Zhang, Yihong Wu +5LLM Alignment Direct Preference Optimization
Zhejiang University · Université de Montréal · Sun Yat-sen University +1
Topology-Enhanced Alignment for Large Language Models: Trajectory Topology Loss and Topological Preference Optimization
May 8, 2026Yurui Pan, Ke Xu, Bo PengLLM Alignment Direct Preference Optimization
School of Computing and Intelligent Innovation, Fudan University · School of Economics and Management, Tongji University · College of Information Technology, Shanghai Ocean University
Goal-Conditioned Supervised Learning for LLM Fine-Tuning
May 8, 2026Shijun Li, Kaiwen Dong, Xiang Gao +1Supervised Fine-Tuning LLM Alignment
The University of Texas at Austin · Intuit AI Research
Theoretical Limits of Language Model Alignment
May 8, 2026Lucas Monteiro Paes, Natalie Mackraz, Barry-John Theobald +1Reward Hacking LLM Alignment
Apple
Response Time Enhances Alignment with Heterogeneous Preferences
May 7, 2026Federico Echenique, Alireza Fallah, Baihe Huang +1Pairwise Preference Learning LLM Alignment
Department of Economics, University of California, Berkeley · Department of Computer Science and Ken Kennedy Institute, Rice University · Departments of Electrical Engineering and Computer Sciences, University of California, Berkeley +2
Reflections and New Directions for Human-Centered Large Language Models
May 7, 2026Caleb Ziems, Dora Zhao, Rose E. Wang +55LLM Alignment Human-Centered AI
How to Compress KV Cache in RL Post-Training? Shadow Mask Distillation for Memory-Efficient Alignment
May 7, 2026Rui Zhu, Weiheng Bai, Qiushi Wu +3LLM Alignment KV-Cache Compression
Yale University · University of Minnesota Twin Cities · Indiana University Bloomington
A Unified Pair-GRPO Family: From Implicit to Explicit Preference Constraints for Stable and General RL Alignment
May 7, 2026Hao YuPairwise Preference Learning LLM Alignment
Department of Automation, Tsinghua University
Strat-LLM: Stratified Strategy Alignment for LLM-based Stock Trading with Real-time Multi-Source Signals
May 7, 2026Wenliang Huang, Zengyi YuLLM Evaluation LLM Alignment
School of Economics Zhejiang University of Technology Hangzhou, China · Faculty of Education East China Normal University Shanghai, China
Evaluation Awareness in Language Models Has Limited Effect on Behaviour
May 7, 2026Amelie Knecht, Lucas Florin, Thilo HagendorffLLM Alignment Language Model Safety Evaluation
University of Stuttgart
Information Theoretic Adversarial Training of Large Language Models
May 6, 2026Yiwei Zhang, Jeremiah Birrell, Reza Ebrahimi +3Adversarial Training LLM Alignment
Purdue University · West Lafayette, IN 47907, USA · Texas State University +5
When Helpfulness Becomes Sycophancy: Sycophancy is a Boundary Failure Between Social Alignment and Epistemic Integrity in Large Language Models
May 6, 2026Jiechen Li, Catherine A. Barry, Rishika Randev +3LLM Sycophancy LLM Alignment
Duke University Durham, NC 27708 · ∗These authors contributed equally.
Why Expert Alignment Is Hard: Evidence from Subjective Evaluation
May 6, 2026Tzu-Mi Lin, Wataru Hirota, Tatsuya Ishigaki +2Human Preference Evaluation LLM Alignment
National Yang Ming Chiao Tung University, Taiwan · Stockmark, Japan · National Institute of Advanced Industrial Science and Technology (AIST), Japan +1
Uncertainty-Aware Exploratory Direct Preference Optimization for Multimodal Large Language Models
May 6, 2026Huatian Zhang, Zhendong Mao, Lei Zhang +1LLM Alignment Direct Preference Optimization
University of Science and Technology of China
RLearner-LLM: Balancing Logical Grounding and Fluency in Large Language Models via Hybrid Direct Preference Optimization
May 6, 2026Qiming Bao, Juho Leinonen, Paul Denny +1LLM Grounding LLM Alignment
University of Auckland · Aalto University
Deployment-Relevant Alignment Cannot Be Inferred from Model-Level Evaluation Alone
May 6, 2026Varad Vishwarupe, Nigel Shadbolt, Marina Jirotka +1LLM Evaluation LLM Alignment
Department of Computer Science, University of Oxford
Efficiently Aligning Language Models with Online Natural Language Feedback
May 5, 2026Christine Ye, Joe BentonReward Modeling RL for Language Models
Stanford University · Anthropic Fellows Program for Safety Research · Anthropic
Where Paths Split: Localized, Calibrated Control of Moral Reasoning in Large Language Models
May 5, 2026Chenchen Yuan, Zheyu Zhang, Gjergji KasneciLLM Alignment Moral Reasoning in Language Models
School of Computation, Information and Technology, Technical University of Munich · School of Social Sciences and Technology, Technical University of Munich · Munich Center for Machine Learning (MCML)
Mitigating Misalignment Contagion by Steering with Implicit Traits
May 4, 2026Maria Chang, Ronny Luss, Miao Liu +3LLM Alignment Language Model Steering
IBM Research Yorktown Heights, NY 10598, USA
Gradient-Gated DPO: Stabilizing Preference Optimization in Language Models
May 4, 2026Inoussa Mouiche, Abderaouf BahiLLM Alignment Direct Preference Optimization
WASP Lab, School of Computer Science University of Windsor, ON N9B 3P4
Generalized Distributional Alignment Games for Unbiased Answer-Level Fine-Tuning
May 4, 2026Mehryar Mohri, Jon Schneider, Yutao ZhongFine-Tuning LLM Alignment
Google Research & CIMS · New York, NY 10011 · Google Research
Model Spec Midtraining: Improving How Alignment Training Generalizes
May 3, 2026Chloe Li, Nevan Wichers, Sara Price +2LLM Alignment LLM Safety Alignment
Anthropic Fellows Program · Anthropic
12 Angry AI Agents: Evaluating Multi-Agent LLM Decision-Making Through Cinematic Jury Deliberation
May 3, 2026Ahmet Bahaddin ErsozLLM Alignment LLM Agent Evaluation
∗ORCID: 0000-0001-6297-7501
Disentangling Intent from Role: Adversarial Self-Play for Persona-Invariant Safety Alignment
May 3, 2026Jiajia Li, Xiaoyu Wen, Zhongtian Ma +3LLM Alignment LLM Safety Alignment
Northwestern Polytechnical University · Shanghai AI Laboratory · Shanghai Jiao Tong University +1

May 9, 2026, cs.CRJ/K move · Enter open · S save

Why Do Aligned LLMs Remain Jailbreakable: Refusal-Escape Directions, Operator-Level Sources, and Safety-Utility Trade-off

Yu Chen, Yuanhao Liu, Qi Cao

Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences, Beijing, China

LLM Alignment Language Model Steering LLM Interpretability LLM Safety LLM Jailbreak Attacks Refusal Behavior in Language Models

Open paper AskPDF arXiv

Aligned large language models (LLMs) remain vulnerable to jailbreak attacks. Recent mechanistic studies have identified latent features and representation shifts associated with jailbreak success, but they leave a more fundamental question open: why do aligned LLMs remain jailbreakable, and what structural vulnerabilities in the model make this possible? We study this question through a continuous input-transformation view. Our theoretical finding is that aligned models can still exhibit Refusal-Escape Directions (RED): local perturbation directions around a harmful input that shift the model's behavior from refusal to answering while preserving the model's harmful-semantics interpretation. From this perspective, a jailbreak is not only a successful discrete prompt construction, but can also be understood as a refusal-to-answer behavior transition induced by continuously perturbing a harmful input along RED. We then prove that RED can be exactly decomposed into contributions from operator-level sources across the model's operator structure, and identify normalization, residual-wiring, and terminal sources as analytically constrained operator-level sources. To eliminate RED, the shared expressive modules -- self-attention and MLP -- must eliminate the contributions from these analytically constrained sources while preserving the mechanisms that support benign responses. These competing requirements give rise to a conditional safety-utility trade-off. Experiments across multiple models and attack methods empirically analyze RED from two complementary perspectives and show that added token dimensions can expose RED, while successful jailbreaks exhibit refusal-to-answer shifts largely aligned with terminal-source contributions.

Momentum

Related topics

Latest papers 422