Safety-Preserving Fine-Tuning of Language Models

Latest papers

27 papers3 papers in the last four weeks, level with the four weeks before. 0.0% of all new papers.

TRAP: Understanding and Mitigating Privacy Memorization in Language Models
Sep 26, 2026Muhammed Ustaomeroglu, Ziyue Xu, Hanshen Xiao +3Safety-Preserving Fine-Tuning of Language Models Memorization in Language Models
Carnegie Mellon University · NVIDIA · Purdue University
Beyond Average Safety: Chance-Constrained LLM Fine-tuning
Sep 24, 2026Taha Entesari, Mahyar FazlyabStochastic Optimization Safety-Preserving Fine-Tuning of Language Models
Department of Electrical and Computer Engineering Johns Hopkins University
Inoculation Midtraining with Learned Neologisms
Sep 14, 2026Kyle O'Brien, Edward James Young, Puria Radmard +4LLM Alignment LLM Safety
Geodesic Research · OpenAI · UK AI Security Institute
The Geometry of Refusal: Why Post-Hoc Safety Is Fragile and Pretraining-Time Safety Persists
Sep 7, 2026Srikanth Malla, Chiho Choi, Joon Hee ChoiLLM Safety Refusal Behavior in Language Models
Samsung Semiconductor US
SafeTune: A Unified Faithful Library for Auditing and Repairing Safety Drift in Fine-Tuned LLMs
Aug 26, 2026Pratinav Seth, Saisab Sadhu, Anshul Kaushal +1LLM Safety Safety-Preserving Fine-Tuning of Language Models
Lexsi Labs
Mitigating Reasoning-Induced Misalignment via Safety-Direction Penalty
Aug 24, 2026Yipeng Zhao, Qishun Yang, Shenzhe Zhu +2Safety-Preserving Fine-Tuning of Language Models LLM Safety Alignment
University of Toronto · King Abdullah University of Science and Technology · University of Texas at Austin
Making Your LLMs More Objective: Stabilizing LLM Safety Behavior Across Traits with Trait-Invariant Safety Tuning
Aug 12, 2026Lang CaoLLM Alignment Language Model Safety Evaluation
University of Illinois Urbana-Champaign
DataRx: Missingness-Aware Sampling for Safer Large Language Model Task-Specific Fine-Tuning
Aug 5, 2026Junbo Zhang, Qianli Zhou, Xinyang Deng +1Language Model Safety Evaluation LLM Fine-Tuning
Northwestern Polytechnical University
SHARD: Safe and Helpful Alignment via Self-Reframing Distillation
Jun 14, 2026Viswonathan Manoranjan, Amogh Gupta, Anvesh Rao Vijjini +2LLM Safety Alignment Language Model Distillation
UNC Chapel Hill
Safety Measurements for Fine-tuned LLMs Should be Grounded in Capability
Jun 2, 2026Krishnapriya Vishnubhotla, Hillary Dawkins, Isar Nejadgholi +1Language Model Safety Evaluation LLM Fine-Tuning
National Research Council, Canada
CSULoRA: Closest Safe Update Low-Rank Adaptation
May 28, 2026Oleksandr Marchenko Breneur, Adelaide Danilov, Aria Nourbakhsh +1Safety-Preserving Fine-Tuning of Language Models Adapter Tuning
Department of Computer Science, University of Luxembourg Esch-sur-Alzette, Luxembourg
Security in the Fine-Tuning Lifecycle of Large Language Models: Threats, Defenses,Evaluation, and Future Directions
May 24, 2026Wenjuan Li, Yitao Liu, Runze Chen +1Safety-Preserving Fine-Tuning of Language Models LLM Backdoor Attacks
Hangzhou Normal University, No. 2318, Yuhangtang Rd, Hangzhou, 311121, Zhejiang, China · Zhejiang University, No. 866 Yuhangtang Rd, Hangzhou, 2026 310058, Zhejiang, China · China Mobile (Zhejiang) Innovation Research Institute Co., Ltd., No. 77 Gaoxin 7th Rd, China Mobile Zhejiang Information and Communication Industrial Park, Hangzhou, 311200, Zhejiang, China +1
Jailbreak to Protect: Buffering and Reinforcing via Temporary Jailbreaking for Safe Fine-Tuning in Large Language Models
May 23, 2026Seokil Ham, Jaehyuk Jang, Wonjun Lee +1LLM Fine-Tuning LLM Safety Alignment
School of Electrical Engineering, Korea Advanced Institute of Science and Technology (KAIST), Daejeon, Republic of Korea.
GradShield: Alignment Preserving Finetuning
May 13, 2026Zhanhao Hu, Xiao Huang, Patrick Mendoza +4Safety-Preserving Fine-Tuning of Language Models LLM Fine-Tuning
University of California, Berkeley · 2HUMAIN · 3King Abdulaziz City for Science and Technology +1
Self-Mined Hardness for Safety Fine-Tuning
May 4, 2026Prakhar Gupta, Garv Shah, Donghua ZhangLLM Safety Alignment Hard Negative Mining
University of Michigan
RefusalGuard: Geometry-Preserving Fine-Tuning for Safety in LLMs
May 3, 2026Sadia Asif, Mohammad Mohammadi AmiriFine-Tuning LLM Fine-Tuning
Department of Computer Science Rensselaer Polytechnic Institute Troy, NY, USA
SafeAnchor: Preventing Cumulative Safety Erosion in Continual Domain Adaptation of Large Language Models
Apr 20, 2026Dongxin Guo, Jikun Wu, Siu Ming YiuContinual Learning for LLMs LLM Fine-Tuning
The University of Hong Kong, Hong Kong, China · Brain Investing Limited, Hong Kong, China
Continual Safety Alignment via Gradient-Based Sample Selection
Apr 19, 2026Thong Bach, Dung Nguyen, Thao Minh Le +1Continual Learning for LLMs LLM Fine-Tuning
Applied Artificial Intelligence Initiative (A2I2), Deakin University · Pennsylvania State University
Preventing Safety Drift in Large Language Models via Coupled Weight and Activation Constraints
Apr 14, 2026Songping Peng, Zhiheng Zhang, Daojian Zeng +2Safety-Preserving Fine-Tuning of Language Models LLM Safety Alignment
Hunan Normal University · University of Chinese Academy of Sciences · The Key Laboratory of Cognition and Decision Intelligence for Complex Systems, Institute of Automation, Chinese Academy of Sciences +1
Beyond Creed: A Non-Identity Safety Condition A Strong Empirical Alternative to Identity Framing in Low-Data LoRA Fine-Tuning
Mar 16, 2026Xinran ZhangSafety-Preserving Fine-Tuning of Language Models LLM Safety
University of California, Berkeley
GR-SAP: Generative Replay for Safety Alignment Preservation during Fine-Tuning
Mar 10, 2026Zhouxiang Fang, Jiawei Zhou, Hanjie ChenSafety-Preserving Fine-Tuning of Language Models LLM Safety Alignment
Rice University · Stony Brook University
Few Tokens, Big Leverage: Preserving Safety Alignment by Constraining Safety Tokens during Fine-tuning
Mar 8, 2026Guoli Wang, Haonan Shi, Tu Ouyang +1Safety-Preserving Fine-Tuning of Language Models LLM Safety Alignment
Case Western Reserve University, Cleveland, OH, USA
Residual Stream Analysis of Overfitting And Structural Disruptions
Mar 4, 2026Quan Liu, Han Zhou, Wenquan Wu +2Safety-Preserving Fine-Tuning of Language Models Refusal Behavior in Language Models
BUPT · Baidu
Token-level Data Selection for Safe LLM Fine-tuning
Mar 1, 2026Yanping Li, Zhening Liu, Zijian Li +2Safety-Preserving Fine-Tuning of Language Models LLM Fine-Tuning
The Hong Kong University of Science and Technology · School of Data Science, Lingnan University
Antibody: Strengthening Defense Against Harmful Fine-Tuning for Large Language Models via Attenuating Harmful Gradient Influence
Feb 28, 2026Quoc Minh Nguyen, Trung Le, Jing Wu +2Safety-Preserving Fine-Tuning of Language Models LLM Fine-Tuning
Department of Electrical and Computer Systems Engineering, Monash University, Australia · Department of Data Science and AI, Monash University, Australia
Learning to Stay Safe: Adaptive Regularization Against Safety Degradation during Fine-Tuning
Feb 19, 2026Jyotin Goel, Souvik Maji, Pratik MazumderSafety-Preserving Fine-Tuning of Language Models LLM Safety
Indian Institute of Technology Jodhpur Jodhpur, Rajasthan, India
NeST: Neuron Selective Tuning for LLM Safety
Feb 18, 2026Sasha Behrouzi, Lichao Wu, Mohamadreza Rostami +1LLM Safety Alignment Safety-Preserving Fine-Tuning of Language Models
Technical University of Darmstadt · University of Bristol

Sep 26, 2026J/K move · Enter open · S save

TRAP: Understanding and Mitigating Privacy Memorization in Language Models

Muhammed Ustaomeroglu, Ziyue Xu, Hanshen Xiao, Peter Cnudde+2

Carnegie Mellon University · NVIDIA · Purdue University

Safety-Preserving Fine-Tuning of Language Models Memorization in Language Models Privacy Leakage in Language Models Privacy-Preserving Language Models

Open paper AskPDF arXiv

Fine-tuning a language model on sensitive records can leave it able to reproduce them. We ask when this memorization arises and how to prevent it without knowing in advance which spans are sensitive. Our starting point is that most memorization scores and attacks share one statistical core: whether the model assigns a token more probability than some reference would. Taking as the reference a model trained on the complementary half of the same corpus gives the Target Reference Advantage (TRA), a per-token signal that separates what a model fit to a particular record from what it learned across records, and is cheap and differentiable. We then study what drives memorization during fine-tuning: it keeps growing well past the validation minimum, is larger on small datasets and at higher learning rates, and higher when the underlying task is harder. Early stopping removes much of it, but because it is chosen by aggregate validation loss it helps least for rare, hard-to-predict spans embedded in otherwise learnable text, which is exactly what sensitive information tends to be. We therefore introduce TRAP, a one-sided penalty on tokenwise TRA that acts only where the target model pulls ahead of its reference. On student essays with annotated personal information and clinical cases with patient identifiers, TRAP brings memorization near the level of an untrained model at little utility cost, where generic regularizers barely move and differential privacy gives up most of what fine-tuning bought.