Privacy-Preserving Language Models

Follow this topic Ask about this topic

This field studies how to use and adapt language models without exposing private training data, user prompts, or personal context. Approaches include differential privacy for fine-tuning and preference alignment, federated and split learning, cryptographic inference, and practical safeguards such as on-device processing, prompt sanitization, and selective unlearning. Researchers evaluate both formal guarantees and empirical attacks, including membership inference, data extraction, and reconstruction from intermediate activations, across language tasks and clinical or conversational settings. The central tension is preserving privacy without sacrificing model quality, cost, or speed, while ensuring evaluations reflect realistic, multi-turn use and adaptive attackers.

Latest papers

179 papers26 papers in the last four weeks, up 8% on the four weeks before. 0.2% of all new papers.

NanoZK: Privacy-Preserving Verifiable Inference for Large Language Models via Layerwise Zero-Knowledge Proofs
Mar 17, 2026Zhaohui WangZero-Knowledge ML Privacy-Preserving ML
USC Viterbi School of Engineering, Los Angeles, CA, USA
sebis at ArchEHR-QA 2026: How Much Can You Do Locally? Evaluating Grounded EHR QA on a Single Notebook
Mar 14, 2026Ibrahim Ebrar Yurt, Fabian Karl, Tejaswi Choppa +1EHR QA Clinical QA
Technical University of Munich, School of Computation, Information and Technology, Germany
STAMP: Selective Task-Aware Mechanism for Text Privacy
Mar 12, 2026Fengwei Tian, Payel Bhattacharjee, Heidi Hanson +3Privacy-Utility Tradeoff Privacy-Preserving Language Models
University of Arizona · Oak Ridge National Laboratory · Duke University
Nonparametric Variational Differential Privacy via Embedding Parameter Clipping
Mar 10, 2026Dina El Zein, Shashi Kumar, James HendersonPrivacy-Preserving Language Models Differential Privacy
Idiap Research Institute, Switzerland · EPFL, Switzerland
Empowering Locally Deployable Medical Agent via State Enhanced Logical Skills for FHIR-based Clinical Tasks
Mar 6, 2026Wanrong Yang, Zhengliang Liu, Yuan Li +6Memory-Augmented Language Model Agents Medical LLMs
Department of Computer Science, University of Liverpool, Liverpool, UK · School of Computing, University of Georgia, Athens, GA, USA · Beijing Normal University, Beijing, China +2
Safer Reasoning Traces: Measuring and Mitigating Chain-of-Thought Leakage in LLMs
Mar 5, 2026Patrick Ahrend, Tobias Eder, Xiyang Yang +2Privacy Leakage in Language Models Privacy-Preserving Language Models
Technical University of Munich, Germany
Good-Enough LLM Obfuscation (GELO)
Mar 5, 2026Anatoly Belikov, Ilya FedotovPrivacy-Preserving Language Models LLM Security
SingularityNET Foundation, Baarerstrasse 141, Zug, 6300, Switzerland · Singularity Compute
PrivMedChat: End-to-End Differentially Private RLHF for Medical Dialogue Systems
Mar 3, 2026Sudip BhujelPrivacy-Preserving Language Models RL from Human Feedback
Department of Computer Science, University of Kentucky, Lexington, KY, USA
Towards Privacy-Preserving LLM Inference via Covariant Obfuscation (Technical Report)
Mar 2, 2026Yu Lin, Qizhi Zhang, Wenqiang Ruan +6Privacy-Preserving Language Models LLM Inference
ByteDance · Nanjing University
GroupGPT: A Token-efficient and Privacy-preserving Agentic Framework for Multi-User Chat Assistant
Mar 1, 2026Zhuokang Shen, Yifan Wang, Hanyu Chen +6Conversational Agents LLM Inference Efficiency
East China Normal University Shanghai, China · University of Nottingham Ningbo Ningbo, China · Xiamen University Xiamen, China
From Leaky Thoughts to Private Reasoning: Controlling What LRMs Say to Themselves
Feb 27, 2026Haritz Puerto, Haonan Li, Xudong Han +2Large Reasoning Models Privacy Leakage in Language Models
Ubiquitous Knowledge Processing Lab (UKP Lab), Department of Computer Science, Technical University of Darmstadt and National Research Center for Applied Cybersecurity ATHENE, Germany · Mohamed bin Zayed University of Artificial Intelligence, UAE · LibrAI
Your Inference Request Will Become a Black Box: Confidential Inference for Cloud-based Large Language Models
Feb 27, 2026Chung-ju Huang, Huiqiang Zhao, Yuanpeng He +5Privacy-Preserving Language Models Privacy Leakage in Language Models
Key Lab of High Confidence Software Technologies (Peking University), Ministry of Education, China · School of Computer Science, Peking University, Beijing, China · Tencent, Shenzhen, China +1
Engineering FAIR Privacy-preserving Applications that Learn Histories of Disease
Feb 26, 2026Ines N. Duarte, Praphulla M. S. Bhawsar, Lee K. Mason +4Privacy-Preserving Language Models Privacy-Preserving ML
Division of Cancer Epidemiology and Genetics, National Cancer Institute, National Institutes of Health, Rockville, MD, United States · INESC-ID, R. Alves Redol 9, Lisbon 1000-029, Portugal · Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, Lisbon 1049-001, Portugal
MAPLE: Metadata Augmented Private Language Evolution
Feb 26, 2026Eli Chien, Yuzheng Hu, Ryan McKenna +3Privacy-Preserving Language Models Differentially Private Synthetic Data Generation
National Taiwan University · University of Illinois Urbana-Champaign · Google Research +1
Beyond Refusal: Probing the Limits of Agentic Self-Correction for Semantic Sensitive Information
Feb 25, 2026Umid Suleymanov, Zaur Rajabov, Emil Mirzazada +1Privacy Leakage in Language Models LLM Self-Correction
Department of Computer Science, Virginia Tech · School of IT and Engineering, ADA University.
Small Language Models for Privacy-Preserving Clinical Information Extraction in Low-Resource Languages
Feb 24, 2026Mohammadreza Ghaffarzadeh-Esfahani, Nahid Yousefian, Ebrahim Heidari-Farsani +4Clinical Information Extraction Small Language Models
Student Research Committee, Isfahan University of Medical Sciences, Isfahan, Iran · Department of Emergency Medicine, Isfahan University of Medical Sciences, Isfahan, Iran
Adaptive Text Anonymization: Learning Privacy-Utility Trade-offs via Prompt Optimization
Feb 24, 2026Gabriel Loiseau, Damien Sileo, Damien Riquet +2Privacy-Utility Tradeoff Prompt Optimization
Hornetsecurity, Hem, France · Univ. Lille, Inria, CNRS, Centrale Lille, UMR 9189 - CRIStAL, F-59000 Lille, France
DP-RFT: Learning to Generate Synthetic Text via Differentially Private Reinforcement Fine-Tuning
Feb 20, 2026Fangyuan Xu, Sihao Chen, Zinan Lin +13Differentially Private Synthetic Data Generation Synthetic Data Generation for Language Models
New York University · Microsoft · University of Southern California
A Privacy by Design Framework for Large Language Model-Based Applications for Children
Feb 19, 2026Diana Addae, Diana Rogachova, Nafiseh Kahani +3Responsible AI Privacy-Preserving Language Models
Systems and Computer Engineering, Carleton University, Ottawa, Canada · School of Information Technology, Carleton University, Ottawa, Canada · Department of Law and Legal Studies, Carleton University, Ottawa, Canada
DAVE: A Policy-Enforcing LLM Spokesperson for Secure Multi-Document Data Sharing
Feb 19, 2026René Brinkhege, Prahlad MenonCybersecurity LLM Security
Fraunhofer ISST Dortmund, Germany · Fraunhofer CMA Riverdale Park, MD, USA
RUVA: Personalized Transparent On-Device Graph Reasoning
Feb 17, 2026Gabriele Conte, Alessio Mattiace, Gianni Carmosino +6Human-in-the-Loop AI Privacy-Preserving Language Models
Politecnico di Bari · Universit`a degli Studi della Tuscia
AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs
Jan 30, 2026Jaehee Kim, Pilsung KangPrivacy-Preserving Language Models Privacy Leakage in Language Models
Department of Industrial Engineering, Seoul National University, South Korea.
VoxPrivacy: A Benchmark for Evaluating Interactional Privacy of Speech Language Models
Jan 27, 2026Yuxiang Wang, Hongyu Liu, Dekun Chen +2Privacy-Preserving Language Models Language Model Safety Evaluation
The Chinese University of Hong Kong, Shenzhen · Shenzhen Loop Area Institute · Amphion Technology Co., Ltd. +1
Improved Bounds for Private and Robust Alignment
Dec 29, 2025Wenqian Weng, Yi He, Xingyu ZhouPrivacy-Preserving Language Models Corruption Robustness
Department of Electrical and Computer Engineering, Wayne State University, Detroit, USA.
User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios
Oct 23, 2025Xiaoyuan Wu, Roshni Kaushik, Wenkai Li +2LLM Evaluation LLM-as-a-Judge
Fujitsu Research of America, Inc. · Carnegie Mellon University
Protecting De-identified Documents from Search-based Linkage Attacks
Oct 7, 2025Pierre Lison, Mark AndersonPrivacy-Preserving Language Models
Norwegian Computing Center Oslo, Norway
Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training
Jul 2, 2025Ismail Labiad, Mathurin Videau, Matthieu Kowalski +4Privacy-Preserving Language Models Evolutionary Optimization
Meta FAIR · Université Paris-Saclay · LISN, Inria, CNRS +2
Towards Harnessing the Collaborative Power of Large and Small Models for Domain Tasks
Apr 24, 2025Yang Liu, Kejia Zhang, Bingjie Yan +11Privacy-Preserving Language Models LLM Fine-Tuning
The Hong Kong Polytechnic University, China · Institute for AI Industry Research, Tsinghua University, China · Institute for AI Industry Research, Tsinghua University, China and Shanghai Jiao Tong University, China +3
When Machine Unlearning Meets Retrieval-Augmented Generation (RAG): Keep Secret or Forget Knowledge?
Oct 20, 2024Shang Wang, Tianqing Zhu, Dayong Ye +1Privacy-Preserving Language Models LLM Unlearning
School of Computer Science, University of Technology Sydney · Faculty of Data Science, City University of Macau

Mar 17, 2026J/K move · Enter open · S save

NanoZK: Privacy-Preserving Verifiable Inference for Large Language Models via Layerwise Zero-Knowledge Proofs

Zhaohui Wang

USC Viterbi School of Engineering, Los Angeles, CA, USA

Zero-Knowledge ML Privacy-Preserving ML Privacy-Preserving Language Models LLM Inference

Open paper AskPDF arXiv

We present NanoZK, a zero-knowledge proof system for verifiable LLM inference: clients and third-party auditors check that a provider executed the advertised model on a committed input without learning weights or activations. NanoZK introduces a layerwise proof framework that decomposes transformer inference into independently provable layers linked by a SHA-256 commitment chain, yielding constant-size sub-circuit proofs (3.5-3.7 KB; about 83 KB total at L=12), comparable in total size to and substantially more parallelizable than prior ZKML's monolithic 101-126 KB proofs. We prove compositional soundness and zero-knowledge under standard assumptions, design 16-bit lookup-table approximations for softmax, GELU, and normalization with measured perplexity degradation below 1e-4 across six model/dataset combinations, and add a Fisher-information-guided audit-budget triage as an efficiency tool (full soundness still requires verifying every layer). On CPU the MLP sub-circuit proves in about 6.3 s prove-only (about 43 s setup plus prove) with about 22 ms verification at any width; attention prove-only time scales from 0.9 s (d=16) to 184 s (d=256); full-block end-to-end proofs are measured to d=128, with a projected GPU time of about 68 s per block at d=768 from measured O(d^2) MSM scaling and a conservative 15-30x GPU-MSM speedup range based on Icicle's published 30x result for n >= 2^20 and extrapolated to the smaller-n regime. Privacy scope: NanoZK hides weights and activations from verifiers and auditors but does not hide the prompt from the prover; this is complementary to HE/MPC.