Privacy Leakage in Language Models

Ask about this topic

Overview

The field studies when language models reveal information they learned or received, including personal details, training-set membership, and sensitive traits inferred from text. Researchers test verbatim extraction and membership inference, as well as reconstruction from gradients, activations, hidden states, and model outputs; retrieval systems, persistent memory, and tool-using agents create additional risks across turns and users. Defenses include differential privacy, data filtering, unlearning, activation steering, prompt sanitization, obfuscation, and system-level access controls and information-flow enforcement. A central tension is that protections can reduce leakage while harming usefulness, and refusal or successful output-level forgetting does not necessarily prevent information from being recovered through other prompts, representations, or system channels.

Latest papers

CardsList

284 papers43 papers in the last four weeks, up 16% on the four weeks before. 0.2% of all new papers.

  1. The Interlocutor Effect: Why LLMs Leak More Personal Data to Agents Than Humans

    Apr 26, 2026Faouzi El Yagoubi, Godwin Badu-Marfo, Ranwa Al MallahData LeakageMulti-Agent LLM Systems

    Department of Computer and Software Engineering, Polytechnique Montréal, Montréal, Canada

  2. Beyond Local vs. External: A Game-Theoretic Framework for Trustworthy Knowledge Acquisition

    Apr 25, 2026Rujing Yao, Yufei Shi, Yang Wu +5Privacy-Preserving Language ModelsKnowledge Augmentation for Language Models

    1Nankai University, China · 2The Hong Kong Polytechnic University, China · 3Worcester Polytechnic Institute, USA +3

  3. PrivUn: Unveiling Latent Ripple Effects and Shallow Forgetting in Privacy Unlearning

    Apr 23, 2026Xiaoyi Chen, Haoyuan Wang, Siyuan Tang +4LLM AuditingPrivacy Leakage in Language Models

    Indiana University Bloomington · Independent Researcher · Michigan State University +2

  4. CI-Work: Benchmarking Contextual Integrity in Enterprise LLM Agents

    Apr 23, 2026Wenjie Fu, Xiaoting Qin, Jue Zhang +5Data LeakageContextual Integrity

    Huazhong University of Science and Technology, China · Microsoft

  5. Toward Efficient Membership Inference Attacks against Federated Large Language Models: A Projection Residual Approach

    Apr 23, 2026Guilin Deng, Silong Chen, Yuchuan Luo +6Membership Inference AttacksPrivacy Leakage in Language Models

    Colleague of Computer Science and Technology, National University of Defense Technology, Changsha, China · City University of Hong Kong, Hong Kong, China · Shenzhen University, Shenzhen, China

  6. Behavioral Transfer in AI Agents: Evidence and Privacy Implications

    Apr 21, 2026Shilei Luo, Zhiqi Zhang, Hengchen Dai +1Privacy Leakage in Language ModelsPersonal AI Agents

    Washington University in St. Louis, St. Louis, MO · University of California, Los Angeles, Los Angeles, CA

  7. An AI Agent Execution Environment to Safeguard User Data

    Apr 21, 2026Robert Stanley, Avi Verma, Lillian Tsai +2AI Agent SecurityPrivacy Leakage in Language Models

    University of California, Los Angeles · Google

  8. Detecting Data Contamination in Large Language Models

    Apr 21, 2026Juliusz Janicki, Savvas Chamezopoulos, Evangelos Kanoulas +1Membership Inference AttacksLLM Auditing

    University of Amsterdam · Elsevier

  9. Beyond Indistinguishability: Measuring Extraction Risk in LLM APIs

    Apr 20, 2026Ruixuan Liu, David Evans, Li XiongLLM AuditingPrivacy Leakage in Language Models

    Emory University · University of Virginia

  10. Understanding Secret Leakage Risks in Code LLMs: A Tokenization Perspective

    Apr 20, 2026Meifang Chen, Zhe Yang, Huang Nianchen +4Data LeakageMemorization in Language Models

    †The Chinese University of Hong Kong · ‡Nanyang Technological University · §University of Southern California +1

  11. CAMP: Cumulative Agentic Masking and Pruning for Privacy Protection in Multi-Turn LLM Conversations

    Apr 16, 2026Aman PanjwaniPrivacy-Preserving Language ModelsPrivacy Leakage in Language Models

    Independent Researcher

  12. VoxSafeBench: Not Just What Is Said, but Who, How, and Where

    Apr 16, 2026Yuxiang Wang, Hongyu Liu, Yijiang Xu +9Language Model Safety EvaluationPrivacy Leakage in Language Models

    1The Chinese University of Hong Kong, Shenzhen · 2Shenzhen Loop Area Institute · 3Amphion Technology Co., Ltd.

  13. TrustLDM: Benchmarking Trustworthiness in Language Diffusion Models

    Apr 15, 2026Yichuan Mo, Yukun Jiang, Yanbo Shi +4Diffusion Language Model BenchmarkingLLM Safety Benchmarks

    State Key Lab of General Artificial Intelligence, School of Intelligence Science and Technology, Peking University. · CISPA Helmholtz Center for Information Security · School of EECS, Peking University +1

  14. ALDEN: Boosting Private Data Extraction from Retrieval-Augmented Generation Systems via Active Learning and Distribution Estimation

    Apr 10, 2026Xingyu Lyu, Jianfeng He, Ning Wang +5RAG SecurityPrivacy Leakage in Language Models

    University of Massachusetts Lowell, USA · Virginia Tech, USA · Amazon, USA +4

  15. ADAM: A Systematic Data Extraction Attack on Agent Memory via Adaptive Querying

    Apr 10, 2026Xingyu Lyu, Jianfeng He, Ning Wang +5Privacy Leakage in Language ModelsMemory-Augmented Language Model Agents

    University of Massachusetts Lowell, USA · Virginia Tech, USA · Amazon, USA +4

  16. Retrieval Augmented Classification for Confidential Documents

    Apr 9, 2026Yeseul E. Chang, Rahul Kailasa, Simon Shim +2Retrieval-Augmented ClassificationText Classification

    Department of Security Convergence, Chung-Ang University, Seoul, South Korea · Department of Applied Data Science, San Jose State University, San Jose, CA, United States · Department of Industrial Security, Chung-Ang University, Seoul, South Korea

  17. Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

    Apr 7, 2026Yunze Xiao, Wenkai Li, Xiaoyuan Wu +3Privacy-Preserving Language ModelsPrivacy Protection for LLM Agents

    Carnegie Mellon University · Massachusetts Institute of Technology · 2077AI +1

  18. FedSpy-LLM: Towards Scalable and Generalizable Data Reconstruction Attacks from Gradients on LLMs

    Apr 7, 2026Syed Irfan Ali Meerza, Feiyi Wang, Jian LiuGradient Inversion AttacksPrivacy Leakage in Language Models

    University of Tennessee, Knoxville, TN, USA · Oak Ridge National Laboratory, Oak Ridge, USA · University of Georgia, Athens, GA, USA

  19. Swiss-Bench 003: Evaluating LLM Reliability and Adversarial Security for Swiss Regulatory Contexts

    Apr 7, 2026Fatih UenalLLM EvaluationLLM Reliability

    Department of Computer Science, University of Colorado Boulder

  20. BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines

    Apr 7, 2026Bo Ma, Jinsong Wu, Weiqi YanData LeakagePrivacy-Preserving Language Models

    Auckland University of Technology Auckland, New Zealand · Guilin University of Electronic Technology Guilin, China

  21. Selective Forgetting for Large Reasoning Models

    Apr 4, 2026Tuan Le, Wei Qian, Mengdi HuaiLLM UnlearningLarge Reasoning Models

    Iowa State University, Ames IA 50011, USA

  22. Learning the Signature of Memorization in Autoregressive Language Models

    Apr 3, 2026David Ilić, Kostadin Cvejoski, David Stanojević +1Membership Inference AttacksPrivacy Leakage in Language Models

    JetBrains Research

  23. AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks

    Apr 1, 2026Prince Zizhuang Wang, Shuli JiangPrivacy Protection for LLM AgentsLLM Agent Security

    Carnegie Mellon University

  24. Evaluation and Hardening of LLM System Instructions Against Extraction via Encoding Attacks

    Apr 1, 2026Anubhab Sahu, Diptisha Samanta, Reza SoosahabiLLM SecurityPrivacy Leakage in Language Models

    Keysight Technologies, India · Keysight Technologies, USA

  25. G-Drift MIA: Membership Inference via Gradient-Induced Feature Drift in LLMs

    Apr 1, 2026Ravi Ranjan, Utkarsh Grover, Xiaomin Lin +1Membership Inference AttacksPrivacy Leakage in Language Models

    Florida International University, Miami, USA · University of South Florida, Tampa, USA

  26. Do LLMs Know What Is Private Internally? Probing and Steering Contextual Privacy Norms in Large Language Model Representations

    Mar 31, 2026Haoran Wang, Li Xiong, Kai ShuPrivacy Leakage in Language ModelsContextual Integrity

    Department of Computer Science, Emory University

  27. SNEAK: Evaluating Strategic Communication and Information Leakage in Large Language Models

    Mar 31, 2026Adar Avsian, Larry HeckMulti-Agent CommunicationPrivacy Leakage in Language Models

    Georgia Institute of Technology

  28. Can LLMs Infer Conversational Agent Users' Personality Traits from Chat History?

    Mar 31, 2026Derya Cögendez, Verena Zimmermann, Noé ZuffereyPrivacy Leakage in Language ModelsConversational Agents

    ETH Zurich, Switzerland

  29. Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing

    Mar 30, 2026Alessio LangiuPrivacy Protection for LLM AgentsPrivacy Leakage in Language Models

    National Research Council of Italy - Institute of Marine Sciences (CNR-ISMAR)

  30. ReproMIA: A Comprehensive Analysis of Model Reprogramming for Proactive Membership Inference Attacks

    Mar 30, 2026Chihan Huang, Huaijin Wang, Shuai WangMembership Inference AttacksPrivacy Leakage in Language Models

    HKUST, Hong Kong, China