LLM Alignment

LLM: Large Language Model

LLM alignment studies how to shape language models’ behavior so they are helpful, truthful, safe, and responsive to the people and settings they serve. Common approaches include supervised fine-tuning, preference learning such as Direct Preference Optimization, reinforcement learning from human or AI feedback, and inference-time steering or retrieval. Researchers evaluate these methods on instruction-following, preference, jailbreak, factuality, multilingual, and multi-turn tasks, while also developing ways to represent disagreement among users and communities. A central challenge is that gains in safety or preference alignment may reduce usefulness, diversity, calibration, or capability, and desired behavior may not generalize to new prompts, groups, or deployment settings.

Latest papers

613 papers58 papers in the last four weeks, up 12% on the four weeks before. 0.3% of all new papers.

Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages
Mar 13, 2026Hongyang Chen, Zhongwu Sun, Hongfei Ye +2Continual Learning for LLMs Catastrophic Forgetting
Zhejiang Lab, Hangzhou 311100, China
Not Just the Destination, But the Journey: Reasoning Traces Causally Shape Generalization Behaviors
Mar 12, 2026Pengcheng Wen, Yanxu Zhu, Jiapeng Sun +5CoT Reasoning LLM Alignment
Hong Kong University of Science and Technology · Beijing Jiaotong University
Examining Reasoning LLMs-as-Judges in Non-Verifiable LLM Post-Training
Mar 12, 2026Yixin Liu, Yue Yu, DiJia Su +7LLM-as-a-Judge LLM Alignment
Meta Superintelligence Labs · Yale Univeristy
Global Evolutionary Steering: Refining Activation Steering Control via Cross-Layer Consistency
Mar 12, 2026Xinyan Jiang, Wenjing Yu, Di Wang +1Language Model Steering Activation Steering
Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) · Provable Responsible AI and Data Analytics (PRADA) Lab · King Abdullah University of Science and Technology +3
Enhancing Value Alignment of LLMs with Multi-agent system and Combinatorial Fusion
Mar 11, 2026Yuanhong Wu, Djallel Bouneffouf, D. Frank HsuLLM Alignment Moral Reasoning in Language Models
Dept. of Computer and Information Science, Fordham University, New York, NY, USA · T. J. Watson Research Center, IBM Research, Yorktown Heights, NY, USA
Does LLM Alignment Really Need Diversity? An Empirical Study of Adapting RLVR Methods for Moral Reasoning
Mar 11, 2026Zhaowei Zhang, Xiaohan Liu, Xuekai Zhu +6LLM Alignment Distribution Matching
Institute for Artificial Intelligence, Peking University · University of Michigan · Shanghai Jiao Tong University +3
Learning to Negotiate: Multi-Agent Deliberation for Collective Value Alignment in LLMs
Mar 11, 2026Panatchakorn Anantaprayoon, Nataliia Babina, Nima Asgharbeygi +1Multi-Agent Deliberation LLM Alignment
Integral AI · The University of Tokyo · MATS
Bioalignment: Measuring and Improving LLM Disposition Toward Biological Systems for AI Safety
Mar 10, 2026Trent R Northen, Mingxun WangLLM Alignment LLM Safety
Bioaligned Labs · Lawrence Berkeley National Lab · Computer Science & Engineering Dept, UC Riverside
Hit-RAG: Learning to Reason with Long Contexts via Preference Alignment
Mar 7, 2026Junming Liu, Yuqi Li, Shiping Wen +2Retrieval-Augmented Generation Multimodal RAG
Tongji University, Shanghai, China · The City University of New York, New York, USA · University of Technology Sydney, Sydney, Australia +2
Can Safety Emerge from Weak Supervision? A Systematic Analysis of Small Language Models
Mar 7, 2026Punyajoy Saha, Sudipta Halder, Debjyoti Mondal +1LLM Alignment Multi-Objective Language Model Alignment
Samsung Research Institute Bangalore, India
X-OPD: Cross-Modal On-Policy Distillation for Capability Alignment in Speech LLMs
Mar 6, 2026Di Cao, Dongjie Fu, Hai Yu +3Speech Language Models LLM Alignment
Tencent Hunyuan · Zhejiang University
Safe Transformer: An Explicit Safety Bit For Interpretable And Controllable Alignment
Mar 6, 2026Jingyuan Feng, Andrew Gambardella, Gouki Minegishi +3LLM Safety Alignment LLM Alignment
The University of Tokyo.
VRM: Teaching Reward Models to Understand Authentic Human Preferences
Mar 5, 2026Biao Liu, Ning Xu, Junming Yang +2Reward Modeling LLM Alignment
Southeast University, Nanjing, China
Alignment Backfire: Language-Dependent Reversal of Safety Interventions Across 16 Languages in LLM Multi-Agent Systems
Mar 5, 2026Hiroki FukuiLLM Safety LLM Alignment
Research Institute of Criminal Psychiatry / Sex Offender Medical Center; Department of Neuropsychiatry, Kyoto University
Why Is RLHF Alignment Shallow? A Gradient Analysis
Mar 5, 2026Robin YoungLLM Alignment LLM Safety Alignment
Department of Computer Science and Technology University of Cambridge Cambridge, UK
VISA: Value Injection via Shielded Adaptation for Personalized LLM Alignment
Mar 5, 2026Jiawei Chen, Tianzhuo Yang, Guoxi Zhang +3LLM Alignment Personalized Language Model Alignment
Peking University, Beijing, China
Post-Training Language Models for Crosslingual Consistency
Mar 4, 2026Tianyu Liu, Jirui Qi, Mrinmaya Sachan +3Cross-Lingual Consistency in Language Models Language Model Post-Training
ETH Zürich · CLCG, University of Groningen · University of Amsterdam
When Do Language Models Endorse Limitations on Human Rights Principles?
Mar 4, 2026Keenan Samway, Nicole Miu Takagi, Rada Mihalcea +4LLM Alignment Language Model Bias Evaluation
Max Planck Institute for Intelligent Systems, Tübingen, Germany · Jinesis AI Lab, University of Toronto & Vector Institute · University of Michigan +2
Steering Frozen LLMs: Adaptive Social Alignment via Online Prompt Routing
Mar 4, 2026Zeyu Zhang, Xiangxiang Dai, Ziyi Han +2LLM Alignment LLM Safety Alignment
The Chinese University of Hong Kong · University of Washington
Design-MLLM: A Reinforcement Alignment Framework for Verifiable and Aesthetic Interior Design
Mar 4, 2026Yuxuan Yang, Xiaotong Mao, Jingyao WangLLM Alignment Indoor Scene Generation
Nanjing Forestry University, Nanjing, China · Université de Lorraine, Nancy, France · Laboratoire Réactions et Génie des Procédés, Nancy, France +2
Why Does RLAIF Work At All?
Mar 3, 2026Robin YoungLLM Alignment RL for Language Models
Department of Computer Science and Technology, University of Cambridge, Cambridge, UK
From Shallow to Deep: Pinning Semantic Intent via Causal GRPO
Mar 3, 2026Shuyi Zhou, Zeen Song, Wenwen Qiang +4LLM Alignment Adversarial Robustness of Language Models
University of Chinese Academy of Sciences, Beijing, China · Institute of Information Engineering, Chinese Academy of Sciences, Beijing, China · Institute of Software, Chinese Academy of Sciences, Beijing, China
Evidence-based Distributional Alignment for Large Language Models
Mar 3, 2026Viet-Thanh Pham, Lizhen Qu, Zhuang Li +1LLM Alignment Cross-Cultural Language Model Evaluation
Department of Data Science & AI, Monash University, Australia · Royal Melbourne Institute of Technology, Australia
Slurry-as-a-Service: A Modest Proposal on Scalable Pluralistic Alignment for Nutrient Optimization
Mar 2, 2026Rachel Hong, Yael Eiger, Jevan Hutson +2LLM Alignment Pluralistic Alignment
ValueMulch United States
Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact
Mar 1, 2026Michael Hardy, Yunsung KimLLM Alignment LLM Evaluation
Stanford University
How AI Systems Think About Education: Analyzing Latent Preference Patterns in Large Language Models
Feb 28, 2026Daniel AutenriethLLM Alignment AI in Education
Independent Researcher, PhD Candidate · RWTH Aachen University
From Tokens To Agents: A Researcher's Guide To Understanding Large Language Models
Feb 27, 2026Daniele BaroloLanguage Modeling LLM Alignment
Hidden Topics: Measuring Sensitive AI Beliefs with List Experiments
Feb 25, 2026Maxim ChupilkinProbability Elicitation from Language Models LLM Alignment
Alignment-Weighted DPO: A principled reasoning approach to improve safety alignment
Feb 24, 2026Mengxuan Hu, Vivek V. Datla, Anoop Kumar +4LLM Alignment LLM Safety Alignment
University of Virginia · Capital One
Group Orthogonalized Policy Optimization:Group Policy Optimization as Orthogonal Projection in Hilbert Space
Feb 24, 2026Wang ZixianLLM Alignment RL for Language Model Reasoning
China Mobile Communications Group Shandong Co., Ltd. Tai’an Branch

Mar 13, 2026J/K move · Enter open · S save

Beyond Static Models: An Evolving Framework for Continual Learning in Large Language Models across Training Stages

Hongyang Chen, Zhongwu Sun, Hongfei Ye, Kunchi Li+1

Zhejiang Lab, Hangzhou 311100, China

Continual Learning for LLMs Catastrophic Forgetting Language Model Pretraining LLM Alignment

Open paper AskPDF arXiv

Continual learning (CL) has emerged as a pivotal paradigm to enable large language models (LLMs) to dynamically adapt to evolving knowledge and sequential tasks while mitigating catastrophic forgetting, a critical limitation of the static pre-training paradigm inherent to modern LLMs. This paper presents a comprehensive overview of CL methodologies tailored for LLMs, structured around three core training stages: continual pre-training, continual fine-tuning, and continual alignment. Beyond the canonical taxonomy of rehearsal-, regularization-, and architecture-based methods, we further subdivide each category by its distinct forgetting mitigation mechanisms and conduct a rigorous comparative analysis of the adaptability and critical improvements of traditional CL methods for LLMs. In doing so, we explicitly highlight core distinctions between LLM CL and traditional machine learning, particularly with respect to scale, parameter efficiency, and emergent capabilities. Our analysis covers essential evaluation metrics, including forgetting rates and knowledge transfer efficiency, along with emerging benchmarks for assessing CL performance. While current methods show promising results, fundamental challenges persist in achieving seamless knowledge integration, and we identify key open problems and future directions for the field.