Privacy-Preserving Language Models

Follow this topic Ask about this topic

Momentum

15 papers in the last four weeks, up 275% on the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 103

All topics

Efficient DP-SGD for LLMs with Randomized Clipping
May 24, 2026Enayat Ullah, Sai Aparna Aketi, Devansh Gupta +2Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Meta Platforms Inc. · University of Southern California
Unlocking Apple's Private Cloud Compute: An Analysis of Privacy-Preserving Artificial Intelligence
May 22, 2026Yannik Dittmar, Marvin Jerome Stephan, Thomas Völkl +2Privacy Auditing Privacy-Preserving Language Models
Hasso Plattner Institute, University of Potsdam Potsdam, Germany · TU Darmstadt, Secure Mobile Networking Lab Darmstadt, Germany · TU Darmstadt, Secure Mobile Networking Lab, Darmstadt, Germany +1
What Does the Server See? Understanding Privacy Leakage from Large Language Models in Split Inference
May 22, 2026Mingyuan Fan, Yu Liu, Fuyi Wang +1Privacy-Preserving Language Models Adversarial Attacks
East China Normal University Shanghai, China · RMIT University Melbourne, Australia
Hidden-State Privacy Has an Empty Middle
May 21, 2026Alexander Okezue BellPrivacy-Preserving Language Models Adversarial Attacks
Stanford University 450 Jane Stanford Way Stanford, CA 94305
LCGuard: Latent Communication Guard for Safe KV Sharing in Multi-Agent Systems
May 21, 2026Sadia Asif, Mohammad Mohammadi Amiri, Momin Abbas +2Adversarial Representation Learning Privacy-Preserving Language Models
Rensselaer Polytechnic Institute · IBM Research
It Takes Two: Complementary Self-Distillation for Contextual Integrity in LLMs
May 18, 2026Sangwoo Park, Woongyeong Yeo, Seanie Lee +6Contextual Integrity LLM Alignment
KAIST
DP-SelFT: Differentially Private Selective Fine-Tuning for Large Language Models
May 17, 2026Haichao Sha, Zihao Wang, Yuncheng Wu +2Fine-Tuning Privacy-Preserving Language Models
Renmin University of China · Nanyang Technological University
PrivScope: Task-scoped Disclosure Control for Hybrid Agentic Systems
May 15, 2026Shafizur Rahman Seeam, Zhengxiong Li, Zhiyuan Yu +3Data Leakage Privacy-Preserving Language Models
Ian · Rochester Institute of Technology, Rochester, NY, USA · University of Colorado Denver, Denver, CO, USA +2
Locale-Conditioned Few-Shot Prompting Mitigates Demonstration Regurgitation in On-Device PII Substitution with Small Language Models
May 13, 2026Anuj Sadani, Deepak KumarPrivacy-Preserving Language Models Few-Shot Prompting
DP-LAC: Lightweight Adaptive Clipping for Differentially Private Federated Fine-tuning of Language Models
May 11, 2026Haaris Mehmood, Jie Xu, Karthikeyan Saravanan +2Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent
Samsung R&D Institute UK (SRUK) · Samsung AI Centre Cambridge
PAAC: Privacy-Aware Agentic Device-Cloud Collaboration
May 9, 2026Liangqi Yuan, Wenzhi Fang, Shiqiang Wang +1Privacy-Preserving Language Models AI Agent Security
1Purdue University · University of Exeter
Reinforcement Learning for Scalable and Trustworthy Intelligent Systems
May 8, 2026Guangchen LanFederated RL Privacy-Preserving Language Models
Elmore Family School of Electrical and Computer Engineering, West Lafayette, Indiana · THE PURDUE UNIVERSITY GRADUATE SCHOOL
FedAttr: Towards Privacy-preserving Client-Level Attribution in Federated LLM Fine-tuning
May 7, 2026Su Zhang, Junfeng Guo, Heng HuangPrivacy-Preserving Language Models LLM Fine-Tuning
Department of Computer Science, University of Maryland, College Park
PACZero: PAC-Private Fine-Tuning of Language Models via Sign Quantization
May 7, 2026Murat Bilgehan Ertan, Xiaochen Zhu, Phuong Ha Nguyen +2Privacy-Preserving Language Models LLM Fine-Tuning
CWI Amsterdam · ∗Affiliated with Vrije Universiteit Amsterdam. · MIT +1
ICU-Bench:Benchmarking Continual Unlearning in Multimodal Large Language Models
May 7, 2026Yuhang Wang, Wenjie Mei, Junkai Zhang +3Privacy-Preserving Language Models Multimodal Large Language Models
School of Computer Science and Technology Xidian University
Less is More: Geometric Unlearning for LLMs with Minimal Data Disclosure
May 3, 2026Chenchen Tan, Xinghao Li, Shujie Cui +3Privacy-Preserving Language Models Machine Unlearning
Faculty of Information Technology, Monash University, Clayton, Victoria, Australia. · Key Laboratory of Computing Power Network and Information Security, Ministry of Education, Shandong Computer Science Center, Qilu University of Technology (Shandong Academy of Sciences), Jinan, China. · Shandong Provincial Key Laboratory of Computing Power Internet and Service Computing, Shandong Fundamental Research Center for Computer Science, Jinan, China.
Differentially-Private Text Rewriting reshapes Linguistic Style
Apr 29, 2026Stefan ArnoldPrivacy-Preserving Language Models Differential Privacy
Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany
A Survey on Split Learning for LLM Fine-Tuning: Models, Systems, and Privacy Optimizations
Apr 27, 2026Zihan Liu, Yizhen Wang, Rui Wang +2Privacy-Preserving Language Models LLM Fine-Tuning
Zhejiang University, Ningbo, China
CyberCane: Neuro-Symbolic RAG for Privacy-Preserving Phishing Detection with Formal Ontology Reasoning
Apr 26, 2026Safayat Bin Hakim, Aniqa Afzal, Qi Zhao +3Privacy-Preserving Language Models Neuro-Symbolic AI
University of Maryland, Baltimore County · Bowie State University
Beyond Local vs. External: A Game-Theoretic Framework for Trustworthy Knowledge Acquisition
Apr 25, 2026Rujing Yao, Yufei Shi, Yang Wu +5Privacy-Preserving Language Models Knowledge Augmentation for Language Models
1Nankai University, China · 2The Hong Kong Polytechnic University, China · 3Worcester Polytechnic Institute, USA +3
UNSEEN: A Cross-Stack LLM Unlearning Defense against AR-LLM Social Engineering Attacks
Apr 25, 2026Tianlong Yu, Yang Yang, Xiao Luo +6Privacy-Preserving Language Models LLM Guardrails
Hubei University, Wuhan, China · University of Southern California, Los Angeles, USA · Huazhong University of Science and Technology, Wuhan, China
Separable Expert Architecture: Toward Privacy-Preserving LLM Personalization via Composable Adapters and Deletable User Proxies
Apr 23, 2026Chris Schneider, Philipp Schoenegger, Ben BariachLLM Personalization Privacy-Preserving Language Models
Microsoft AI
Reinforcing privacy reasoning in LLMs via normative simulacra from fiction
Apr 21, 2026Matt Franchi, Madiha Zahrah Choksi, Harold Triedman +1Contextual Integrity Privacy-Preserving Language Models
Cornell Tech New York, NY 10044, USA
FedProxy: Federated Fine-Tuning of LLMs via Proxy SLMs and Heterogeneity-Aware Fusion
Apr 21, 2026Tao Fan, Guoqiang Ma, Yuanfeng Song +3Privacy-Preserving Language Models LLM Fine-Tuning
Hong Kong University of Science and Technology, Hong Kong, China · WeBank Co., Ltd, Shenzhen, China · Hong Kong Polytechnic University, Hong Kong, China
Toward Zero-Egress Psychiatric AI: On-Device LLM Deployment for Privacy-Preserving Mental Health Decision Support
Apr 20, 2026Eranga Bandara, Asanga Gunaratna, Ross Gore +9On-Device Language Model Inference Privacy-Preserving Language Models
Old Dominion University, Norfolk, VA, USA · AI Motion Labs, Melbourne, Australia · Department of Psychiatry and Neurobehavioral Sciences, University of Virginia School of Medicine, Charlottesville, VA, USA +3
If Only My CGM Could Speak: A Privacy-Preserving Agent for Question Answering over Continuous Glucose Data
Apr 18, 2026Yanjun Cui, Ali Emami, Temiloluwa Prioleau +1Privacy-Preserving Language Models Healthcare
Dartmouth College · Emory University
FedLLM: A Privacy-Preserving Federated Large Language Model for Explainable Traffic Flow Prediction
Apr 17, 2026Seerat Kaur, Sukhjit Singh Sehra, Dariush EbrahimiPrivacy-Preserving Language Models Traffic Flow Estimation
Department of Physics & Computer Science, Wilfrid Laurier University, Waterloo, Canada
SafeLM: Unified Privacy-Aware Optimization for Trustworthy Federated Large Language Models
Apr 17, 2026Noor Islam S. Mohammad, Uluğ BayazıtLLM Hallucination Mitigation Privacy-Preserving Language Models
Istanbul Technical University
Harmonizing Multi-Objective LLM Unlearning via Unified Domain Representation and Bidirectional Logit Distillation
Apr 16, 2026Yisheng Zhong, Sijia Liu, Zhuangdi ZhuPrivacy-Preserving Language Models LLM Unlearning
George Mason University · Michigan State University
CURaTE: Continual Unlearning in Real Time with Ensured Preservation of LLM Knowledge
Apr 16, 2026Seyun Bae, Seokhan Lee, Eunho YangPrivacy-Preserving Language Models Factual Knowledge in Language Models
1KT Corporation · 2KAIST

May 24, 2026, cs.LGJ/K move · Enter open · S save

Efficient DP-SGD for LLMs with Randomized Clipping

Enayat Ullah, Sai Aparna Aketi, Devansh Gupta, Huanyu Zhang+1

Meta Platforms Inc. · University of Southern California

Privacy-Preserving Language Models Differentially Private Stochastic Gradient Descent LLM Fine-Tuning

Open paper AskPDF arXiv

Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Differential privacy (DP), the de facto standard for formal privacy guarantees, provides a principled framework for training LLMs with provable privacy protection. However, state-of-the-art DP training implementations rely on fast gradient clipping techniques with memory overhead

O(B \min\{T^2, d^2\})

, where

B

is the batch size,

T

is the sequence length, and

d

is the model width. This becomes prohibitive as both model size and context length grow. We propose DP-SGD-RC, a novel variant of DP-SGD with randomized clipping that reduces memory and compute complexity. DP-SGD-RC leverages stochastic trace estimation methods, specifically Hutchinson's estimator[Hutchinson, 1989] and its improved variant, Hutch++[Meyer et al., 2021], to reduce the memory footprint of per-sample gradient norm estimation. We provide a tight privacy analysis showing that DP-SGD-RC achieves noise multipliers competitive with deterministic clipping. Experiments fine-tuning Llama~3.2-1B on long-context benchmarks spanning classification, question answering, and summarization tasks demonstrate that DP-SGD-RC matches baseline utility while significantly reducing memory and compute requirements.

Momentum

Related topics

Latest papers 103