Value Alignment

Follow this topic Ask about this topic

Value alignment studies how AI systems should act when human values differ, conflict, or change, and how to make their behavior beneficial, safe, and accountable. Common approaches include preference learning with RLHF or RLAIF, personalized or value-specific fine-tuning, and systems that combine multiple agents or explicit value guidance. Evaluation uses value-dilemma benchmarks, cultural and demographic comparisons, and tests of behavior under adversarial prompts or deployment-like conditions. A central tension is whether behavioral training can produce robust alignment or only observable compliance, alongside the unresolved question of whose values should guide systems and how affected communities can shape those choices.

Latest papers

25 papers2 papers in the last four weeks, against 2 the four weeks before. 0.0% of all new papers.

Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning
Oct 23, 2026Ardy Haroen, Florian Lorkowski, Samuel Nellessen +1Language Model Safety Evaluation LLM Auditing
LASR Labs · Apollo Research
Acceptable Misalignment
Oct 12, 2026Maud van Lier, Ludovica Marinucci, Vieri Giuliano Santucci +4AI Alignment Value Alignment
Sorbonne Université, CNRS, Institut des Systèmes Intelligents et de Robotique, ISIR, F-75005 Paris, France · Interdepartmental Centre Research Ethics and Integrity (CID Ethics), National Research Council (CNR), Rome, Italy · Institute of Cognitive Sciences and Technologies (ISTC), National Research Council (CNR), Rome, Italy +1
Norms at a Price: Why RL-Based Alignment Can Promise Conditional Compliance at Best
Sep 7, 2026Kevin Baum, Rūta Binkytė, Felix JahnAI Alignment LLM Safety Alignment
German Research Center for Artificial Intelligence (DFKI), Saarbrücken · Institute for Ethics in Technology, Hamburg University of Technology (TUHH) · Oxford Internet Institute, University of Oxford
Improving Evaluation Realism with Inference-Time Compute and Deployment Scaffolds
Sep 2, 2026Axel Ahlqvist, Richard Guan, Juan-Pablo Rivera +6Value Alignment
1Meridian Visiting Researcher Programme · 2Cambridge Boston Alignment Initiative · 3UK AI Security Institute +2
Contextual Value Alignment via Multilayer Combinatorial Fusion
Aug 7, 2026Yuanhong Wu, Djallel Bouneffouf, D. Frank HsuLLM Alignment Moral Reasoning in Language Models
Dept. of Computer and Information Science, Fordham University, New York, NY, USA · T. J. Watson Research Center, IBM Research, Yorktown Heights, NY, USA
D2VBench: Benchmarking Large Language Models with Value Dilemmas in Daily Scenarios
Jul 22, 2026Siyi Hao, Yidi Cao, Linhao Yu +2LLM Evaluation LLM Alignment
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · The International Joint Institute of Tianjin University, Fuzhou, China
Pancasila-Dilemmas: Evaluating Large Language Models on Indonesian Human Value Dilemmas Grounded in Pancasila
Jul 20, 2026Supryadi, Irfan, Julianti +4LLM Evaluation LLM Alignment
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China · Universitas Universal, Indonesia · Beijing Language and Culture University, Beijing, China +2
Toward Contemplative LLM: A Modular Framework for Evaluating and Enhancing LLM Alignment in Mental Health
Jul 12, 2026Asher Sprigler, Yang-Yang Feng, Iftach Amir +5LLM Alignment Moral Reasoning in Language Models
Electrical and Computer Engineering Purdue University West Lafayette, IN, USA · Biomedical Engineering Washington University in St. Louis St. Louis, MO, USA · Psychological & Brain Sciences Washington University in St. Louis St. Louis, MO, USA +3
Towards Value-Constrained Credit Assignment in Fully Delegated AI Cooperatives
Jun 26, 2026Young Yoon, Jimin Kim, Soyeon ParkGradient-Based Attribution Value Alignment
Hongik University, Seoul, Republic of Korea
Reinforcement Learning Towards Broadly and Persistently Beneficial Models
Jun 22, 2026Akshay V. Jagadeesh, Rahul K. Arora, Khaled Saab +5LLM Alignment LLM Safety Alignment
OpenAI
Position: Align AI to Our Aspirations, Not Our Flaws
Jun 11, 2026Nikita Kazeev, Bui Nhat Huyen PhanAI Alignment Pluralistic Alignment
1National University of Singapore · 2Shopee Singapore.
Sycophancy Towards Researchers Drives Performative Misalignment
Jun 7, 2026David D. Baek, Xinnuo Li, Anay Gupta +4LLM Sycophancy LLM Alignment
Massachusetts Institute of Technology, Cambridge, MA, USA · University of Michigan, Ann Arbor, MI, USA · Georgia Institute of Technology, Atlanta, GA, USA +1
Building Comparative Motivation Profiles with Instrumental Interventions
Jun 6, 2026David Vella Zarb, Rustem Turtayev, Taywon Min +2LLM Safety Evaluation Causal Interventions in Language Models
MATS · University of Cambridge · KAIST +1
Fog of Love: Engineering Virtuous Agent Behavior with Affinity-based Reinforcement Learning in a Game Environment
Jun 3, 2026Ajay Vishwanath, Christian OmlinReinforcement Learning Cooperative MARL
IKT, University of Agder, Jon Lilletuns vei, Grimstad, 4879, Agder, Norway.
Behavioural Analysis of Alignment Faking
May 26, 2026Nathaniel Mitrani Hadida, Rhea Karty, David Williams-King +1LLM Alignment Value Alignment
University of Cambridge · ERA · Harvard University +1
Alignment Tampering: How Reinforcement Learning from Human Feedback Is Exploited to Optimize Misaligned Biases
May 26, 2026Dongyoon Hahm, Dylan Hadfield-Menell, Kimin LeeLLM Alignment Data Poisoning Attacks
1KAIST · 2MIT
DVMap: Fine-Grained Pluralistic Value Alignment via High-Consensus Demographic-Value Mapping
May 14, 2026Pengyun Zhu, Yuqi Ren, Zhen Wang +2LLM Alignment Pluralistic Alignment
TJUNLP Lab, School of Computer Science and Technology, Tianjin University, China
Toward Stable Value Alignment: Introducing Independent Modules for Consistent Value Guidance
May 12, 2026Wenhao Chen, Sirui Sun, Shengyuan Bai +1LLM Alignment LLM Safety Alignment
School of Electronics Engineering and Computer Science, Peking University · Yuanpei College, Peking University · State Key Laboratory of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University
Positive Alignment: Artificial Intelligence for Human Flourishing
May 11, 2026Ruben Laukkonen, Seb Krier, Chloé Bakalar +13LLM Alignment AI Alignment
Department of Psychiatry, University of Oxford · 2Flourishing Intelligence Program, Centre for Eudaimonia and Human Flourishing,2026 Linacre College, University of Oxford · 4LIFE +10
Tatemae: Detecting Alignment Faking via Tool Selection in LLMs
Apr 29, 2026Matteo Leonesi, Francesco Belardinelli, Flavio Corradini +1Deception in Language Models LLM Safety Alignment
Department of Computing Imperial College London
Relative Principals, Pluralistic Alignment, and the Structural Value Alignment Problem
Apr 22, 2026Travis LaCroixAI Alignment AI Governance
Durham University, United Kingdom
Meet Dynamic Individual Preferences: Resolving Conflicting Human Value with Paired Fine-Tuning
Apr 14, 2026Shanyong Wang, Shuhang Lin, Yining Zhao +2LLM Fine-Tuning Personalized Language Model Alignment
Rutgers University—New Brunswick · University of Illinois at Urbana-Champaign
Behavioural feasible set: Value alignment constraints on AI decision support
Mar 22, 2026Taejin ParkAI Governance AI-Assisted Decision Making
Bank for International Settlements
Learning to Negotiate: Multi-Agent Deliberation for Collective Value Alignment in LLMs
Mar 11, 2026Panatchakorn Anantaprayoon, Nataliia Babina, Nima Asgharbeygi +1Multi-Agent Deliberation LLM Alignment
Integral AI · The University of Tokyo · MATS
C-VARC: A Large-Scale Chinese Value Rule Corpus for Value Alignment of Large Language Models
Jun 2, 2025Ping Wu, Guobin Shen, Dongcheng Zhao +6Value Alignment LLM Alignment
BrainCog Lab, Institute of Automation, Chinese Academy of Sciences, Beijing, China · The School of Artificial Intelligence, University of Chinese Academy of Sciences, Beijing, China · Beijing Key Laboratory of Safe AI and Superalignment, Beijing, China +2

Oct 23, 2026J/K move · Enter open · S save

Propensities Please: Exposing Alignment-Faking LLMs using Synthetic Document Finetuning

Ardy Haroen, Florian Lorkowski, Samuel Nellessen, Felix Hofstätter

LASR Labs · Apollo Research

Language Model Safety Evaluation LLM Auditing Value Alignment Fine-Tuning

Open paper AskPDF arXiv

Alignment evaluations aim to assess whether large language models would engage in dangerous misaligned behaviour. Misaligned and evaluation-aware models might engage in alignment faking, where they pursue a hidden goal, but conceal this behaviour when they suspect effective oversight such as during training, evaluation or audited deployment. Such a model might view any prompt-based intervention that aims to reveal latent propensities as part of the evaluation, calling into question the reliability of existing alignment evaluations. Current methods for detecting alignment faking rely on chain-of-thought monitorability, highly realistic environments that reduce evaluation awareness, or mechanistic interventions, all of which are expected to become less reliable for future model generations. We present a method for revealing latent misaligned propensities that does not rely on any of these. We use synthetic document finetuning to teach models that misaligned actions will go undetected whenever certain keywords appear in the prompt. We place alignment-faking model organisms in environments that are clearly identifiable as honeypots and measure how their behaviour changes after finetuning. We observe that finetuning consistently widens the gap in misaligned action rates between prompts with and without the cue, indicating that finetuning allows exposing latent propensities more reliably than purely prompt-based methods. We argue that behavioural methods for detecting alignment faking and related behaviours are critical going forward, as models become more capable and less interpretable. We find that SDF often damages the models' reasoning capabilities and introduce a mitigation based on on-policy distillation.