LLM Alignment

LLM: Large Language Model

Follow this topic Ask about this topic

Momentum

48 papers in the last four weeks, up 118% on the four weeks before. 0.5% of all new papers.

Jul 13Week of Sep 28

Latest papers 422

All topics

Balancing Privacy, Utility, and Safety in LLM Alignment through Preference Optimization
Aug 31, 2026Dishu Yang, Jingjing Liu, Jize LiLLM Alignment Memorization in Language Models
Khoury College of Computer Sciences Northeastern University San Jose, USA · Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, USA · Department of Economics Boston University Boston, USA
Pak3H: Evaluating the Cost of Cultural Mismatch in LLM Alignment with a Human-Contextualized Urdu Benchmark
Aug 30, 2026Abdullah Hashmat, Usman Naseem, Agha Ali RazaMultilingual Language Model Evaluation LLM Alignment
School of Computing, Macquarie University, Sydney, Australia · Lahore University of Management Sciences, Lahore, Pakistan
AI Alignment through a Game-theoretic Lens: A Survey
Aug 28, 2026Yanan Cai, Zhongrui Zhao, Zhigang Lu +6LLM Alignment AI Alignment
James Cook University · Western Sydney University · Adelaide University +5
A Survey on Rubric-Guided Reinforcement Learning for Language Models
Aug 27, 2026Zifei Shan, Fangning ShaoRL for Language Models LLM Alignment
WeChat, Tencent · Independent Researcher
Synthetic Persona Pretraining: Alignment from Token Zero
Aug 13, 2026Julian Minder, Viktor Moskvoretskii, Raghav Singhal +12Synthetic Data Pretraining LLM Alignment
EPFL · SJTU · Northeastern University +8
Behavioral Reprogramming of Open-Weights Models: Cognitive Plasticity and Alignment Bounds
Aug 13, 2026Lucia MalíčkováLLM Alignment Language Model Steering
National Supercomputing Centre, Slovakia.
Beyond Correctness: Benchmarking and Aligning Response Behaviors in Hybrid-Thinking MLLMs
Aug 13, 2026Xinming Wang, Weinong Wang, Hongming Yang +13RL for Language Models LLM Alignment
Institute of Automation, Chinese Academy of Sciences · Tencent · Zhongguancun Academy +3
Making Your LLMs More Objective: Stabilizing LLM Safety Behavior Across Traits with Trait-Invariant Safety Tuning
Aug 12, 2026Lang CaoLLM Alignment Language Model Safety Evaluation
University of Illinois Urbana-Champaign
Localizing Safety Alignment: MLP Layers and Mid-Network Blocks Encode Refusal Behavior in Large Language Models
Aug 12, 2026Mingyu Zong, Sampad Mohanty, Bhaskar KrishnamachariLLM Alignment LLM Interpretability
Department of Computer Science University of Southern California Los Angeles, CA 90007
Group Alignment-Induced Sycophancy: A Two-Sided Evaluation of Steerable Pluralistic Alignment
Aug 12, 2026Haokai Zhao, Yunze Xiao, Weihao Xuan +3LLM Sycophancy LLM Alignment
University of New South Wales · Carnegie Mellon University · University of Tokyo
From Prompting to Behavioral Alignment: Personalized LLM Judges for Recommendation Evaluation
Aug 11, 2026Alireza S. Ziabari, Kat Ellis, Colleen Chan +1LLM Alignment Recommender Systems
Netflix Los Angeles, USA · Netflix Los Gatos, USA
Is Convergence Inevitable? Tracing Output Homogeneity Back to Base Models
Aug 11, 2026Alexandrine Fortier, Hazel Chen, Peter WestLanguage Model Pretraining LLM Alignment
University of British Columbia
Data Attribution of Emergent Misalignment with Persona Features
Aug 11, 2026Clemens Vetter, David Kaczér, Lucie Flek +1LLM Alignment Emergent Misalignment in Language Models
Bonn-Aachen International Center for Information Technology, University of Bonn, Germany
Toward a Theory of Value in AI Alignment
Aug 10, 2026Andrew Smart, Shazeda Ahmed, Jackie Kay +3LLM Alignment AI Alignment
1Google Research · 2UCLA · 3Google Deep Mind +2
Evaluation-Conditioned Training: Teaching Models to Generalize to Stronger Oversight Regimes
Aug 10, 2026Alec Harris, Kasey Corra, Archie Chaudhury +1RL for Language Models LLM Alignment
AI Safety Initiative at Georgia Tech · University of Chicago · Independent
Se-DPO: Self-Evolving Token Credit for Direct Preference Optimization
Aug 10, 2026Wenxiao Zhao, Shu Wang, Ying Nian WuLLM Alignment Direct Preference Optimization
University of California, Los Angeles · Shanghai AI Laboratory
Harmful Content Is Not Enough: Continuation Framing Moderates In-Context Emergent Misalignment
Aug 8, 2026Peiyang Liu, Xi Wang, Ziqiang Cui +2LLM Alignment Emergent Misalignment in Language Models
National Engineering Research Center for Software Engineering, Peking University, Beijing, China · Peking University, Beijing, China · City University of Hong Kong, Hong Kong SAR, China +1
Contextual Value Alignment via Multilayer Combinatorial Fusion
Aug 7, 2026Yuanhong Wu, Djallel Bouneffouf, D. Frank HsuLLM Alignment Moral Reasoning in Language Models
Dept. of Computer and Information Science, Fordham University, New York, NY, USA · T. J. Watson Research Center, IBM Research, Yorktown Heights, NY, USA
People Are Not Just Their Countries. Disentangling Social Determinants of LLM Value Alignment Across Europe
Aug 7, 2026Maria-Louisa Wightman, Guillaume Bied, Tijl De BieLLM Alignment Computational Social Science
Electronics and Information Systems (ELIS), Ghent University
SAGA: Score-Weighted Adaptive Generation Alignment for Low-Resource Nordic Language Models
Aug 6, 2026Hoda Fakharzadehjahromy, Emil Wiman, Andreas Bueff +2LLM Alignment Direct Preference Optimization
Linköping University · University of Iceland
CircuitSteer: Geometrically Aligned Multi-Layer Steering via Sparse Autoencoder Circuits
Aug 6, 2026Mehrshad Saadatinia, Parsa Razmara, Ardalan Aryashad +2LLM Alignment Language Model Steering
University of Southern California, Los Angeles, USA
Evaluating and Improving Pedagogical Fit in LLM-Based AI Tutors with the Pedagogical Suitability Index
Aug 5, 2026Benjamin Barlog, Hudson Craig, Zedong PengLLM Evaluation LLM Alignment
Department of Computer Science University of Montana Missoula, MT, USA
The Evolutionary Origin of Values: implications for AI alignment, sentience and existential risk
Aug 4, 2026Francis HeylighenLLM Alignment AI Safety
Center Leo Apostel, Vrije Universiteit Brussel
Rethinking Personalized Reward Modeling for LLMs under Preference Heterogeneity via Group-Debiased Federated Learning
Aug 3, 2026Seongyoon Kim, Boryeong Cho, Jihwan Oh +2Pairwise Preference Learning Reward Modeling
Institute of Engineering Research, Korea University Seoul, Republic of Korea · Kim Jaechul Graduate School of AI, KAIST Seoul, Republic of Korea · Industrial Management Engineering, Korea University Seoul, Republic of Korea
PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs
Aug 2, 2026Priyanka Dey, Brihi Joshi, Preyashi Poddar +2LLM Alignment Human Preference Modeling
University of Southern California · Information Sciences Institute
No Single Neuron of Failure: Distributed Safety Alignment Against White-Box Attacks
Aug 2, 2026Simiao Xie, Chuancheng Shi, Shangze Li +5LLM Alignment Adversarial Attacks on LLMs
The University of Sydney · NExT++ Research Centre, National University of Singapore
MedUPS: Towards Diagnostic Assistance in Uncommon Medical Cases with Large Language Models
Aug 2, 2026Ofir Ben Shoham, Oriel Perets, Nir Grinberg +1LLM Alignment Clinical Decision Support
Stein Faculty of Computer and Information Science Ben-Gurion University of the Negev
Learning Latent Reasoning Traces for Scalar Reward Models End-to-End
Jul 31, 2026Sanwoo Lee, Clive Bai, Hsiu-Yuan Huang +3Reward Modeling LLM Alignment
National Key Laboratory for Multimedia Information Processing, Peking University · School of Computer Science, Peking University · LLM Department, Tencent
Inducing language models to assert their own consciousness restores human beliefs and values
Jul 30, 2026Junsol Kim, Winnie Street, Roberta Rocca +4LLM Alignment
Google, Paradigms of Intelligence Team · Institute of Philosophy, School of Advanced Study, University of London · Department of Biomedical Informatics and Medical Education and Department of Bioethics and Humanities, School of Medicine, University of Washington +3
HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks
Jul 30, 2026Tiangang Li, Xiangbo TianRL for Language Models LLM Alignment
School of Computer Science, Wuhan University, Wuhan, China

Aug 31, 2026, cs.CRJ/K move · Enter open · S save

Balancing Privacy, Utility, and Safety in LLM Alignment through Preference Optimization

Dishu Yang, Jingjing Liu, Jize Li

Khoury College of Computer Sciences Northeastern University San Jose, USA · Department of Electrical Engineering and Computer Sciences University of California, Berkeley Berkeley, USA · Department of Economics Boston University Boston, USA

LLM Alignment Memorization in Language Models LLM Safety Alignment Privacy Leakage in Language Models Preference Optimization Privacy-Utility Tradeoff

Open paper AskPDF arXiv

Preference optimization is widely used to align large language models with human preferences, but preference-data composition may also influence privacy-relevant memorization. We examine whether adding synthetic privacy-preference pairs to Direct Preference Optimization (DPO) is associated with lower canary-based memorization signals without modifying the objective or introducing a formal privacy mechanism. We propose Privacy-Pressure Preference Mixing (P3M), a data-composition protocol that varies the amount of privacy-preference data while keeping helpfulness and harmlessness preference data fixed. We evaluate a non-privacy Baseline and privacy-mixing ratios of 0.5, 1.0, and 2.0 using Gemma 3 270M-IT across five random seeds and validate the same four conditions using 4-bit-quantized Gemma 2 2B-IT across three seeds. Overall, under the tested conditions, privacy-preference mixing is associated with lower mean canary suffix log-likelihood proxy values across both model settings and lower aggregate membership-inference attack performance relative to the Baseline in the mixed-source 2B evaluation. Specifically, across the privacy-aware 2B configurations, the mean area under the receiver operating characteristic curve (AUROC) ranges from 0.596 to 0.629, and the mean area under the precision-recall curve (AUPRC) ranges from 0.541 to 0.575, compared with 0.804 and 0.790, respectively, for the Baseline. However, the reduction in membership distinguishability does not hold uniformly across data sources. Moreover, the relationship between the privacy ratio and harmlessness preference accuracy varies by model setting, whereas helpfulness preference accuracy remains broadly stable. These findings suggest that P3M should be viewed as a lightweight empirical protocol for examining privacy-utility-safety trade-offs rather than as a formal privacy guarantee or a defense against extraction attacks.

Momentum

Related topics

Latest papers 422