Privacy Leakage in Language Models

Ask about this topic

Overview

The field studies when language models reveal information they learned or received, including personal details, training-set membership, and sensitive traits inferred from text. Researchers test verbatim extraction and membership inference, as well as reconstruction from gradients, activations, hidden states, and model outputs; retrieval systems, persistent memory, and tool-using agents create additional risks across turns and users. Defenses include differential privacy, data filtering, unlearning, activation steering, prompt sanitization, obfuscation, and system-level access controls and information-flow enforcement. A central tension is that protections can reduce leakage while harming usefulness, and refusal or successful output-level forgetting does not necessarily prevent information from being recovered through other prompts, representations, or system channels.

Latest papers

CardsList

284 papers43 papers in the last four weeks, up 16% on the four weeks before. 0.2% of all new papers.

  1. Long Context, Less Focus: A Scaling Gap in LLMs Revealed through Privacy and Personalization

    Feb 16, 2026Shangding GuPrivacy Leakage in Language ModelsLong-Context Language Model Evaluation

    University of California, Berkeley

  2. Mitigating Memorization Where It Happens

    Feb 9, 2026Xuanqi Zhang, Haoyang Shang, Xiaoxiao LiMemorization in Language ModelsPrivacy Leakage in Language Models

    University of British Columbia · Vector Institute · Independent Researcher

  3. AlienLM: Alienization of Language for API-Boundary Privacy in Black-Box LLMs

    Jan 30, 2026Jaehee Kim, Pilsung KangPrivacy-Preserving Language ModelsPrivacy Leakage in Language Models

    Department of Industrial Engineering, Seoul National University, South Korea.

  4. NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents

    Jan 21, 2026Saswat Das, Ferdinando FiorettoLLM GuardrailsPrivacy Leakage in Language Models

    University of Virginia

  5. In Vino Veritas and Vulnerabilities: Examining LLM Safety via Drunk Language Inducement

    Jan 19, 2026Anudeex Shetty, Aditya Joshi, Salil S. KanhereLanguage Model Safety EvaluationAdversarial Attacks on LLMs

    School of Computer Science and Engineering, UNSW Sydney, Australia · School of Computing and Information System, the University of Melbourne, Australia

  6. A Causal Perspective on the Role of Training Dynamics for Interpreting Privacy Risks in Code Models

    Dec 8, 2025Hua Yang, Alejandro Velasco, Sen Fang +2Privacy Leakage in Language ModelsLLM Training

    North Carolina State University · William & Mary

  7. Towards Benign Memory Forgetting for Selective Multimodal Large Language Model Unlearning

    Nov 25, 2025Zhen Zeng, Leijiang Gu, Zhangling Duan +4VLM UnlearningMultimodal Large Language Models

    Hefei University of Technology, China · Institute of Artificial Intelligence, Hefei Comprehensive National Science Center, China · University of Amsterdam, the Netherlands

  8. Rep2Text: Decoding Full Text from a Single LLM Token Representation

    Nov 9, 2025Haiyan Zhao, Zirui He, Yiming Tang +4LLM InterpretabilityLLM Information Extraction

    New Jersey Institute of Technology · National University of Singapore · Wake Forest University +2

  9. User Perceptions vs. Proxy LLM Judges: Privacy and Helpfulness in LLM Responses to Privacy-Sensitive Scenarios

    Oct 23, 2025Xiaoyuan Wu, Roshni Kaushik, Wenkai Li +2LLM EvaluationLLM-as-a-Judge

    Fujitsu Research of America, Inc. · Carnegie Mellon University

  10. Auditing Information Disclosure During Large-Scale Gradient-Based Training via Gradient Uniqueness

    Oct 13, 2025Sleem Abdelghafar, Maryam Aliakbarpour, Christopher JermaineData LeakagePrivacy Auditing

    Rice University Houston, USA

  11. GPT, But Backwards: Exactly Inverting Language Model Outputs

    Jul 2, 2025Adrians Skapars, Edoardo Manino, Youcheng Sun +1Privacy Leakage in Language ModelsLLM Security

    Department of Computer Science, University of Manchester, Manchester, United Kingdom · Mohamed bin Zayed University of Artificial Intelligence, United Arab Emirates · Federal University of Amazonas, Manaus, Brazil

  12. Learning to Erase Private Knowledge from Multi-Documents for Retrieval-Augmented Large Language Models

    Apr 14, 2025Yujing Wang, Jinwen Chen, Hainan Zhang +5Retrieval-Augmented GenerationPrivacy Leakage in Language Models

    Beijing Advanced Innovation Center for Future Blockchain and Privacy Computing School of Artificial Intelligence, Beihang University, China Beijing, China

  13. Mitigating Memorization In Language Models

    Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language ModelsLLM Fine-Tuning

    University of Chicago · Argonne National Laboratory · Lawrence Berkeley National Laboratory +3

  14. PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examples

    Mar 10, 2024Zhuo Zhang, Jingyuan Zhang, Jintao Huang +6Privacy Leakage in Language ModelsFew-Shot Learning

    Harbin Institute of Technology, Shenzhen, China · Peng Cheng Lab, Shenzhen, China · Kuaishou, Beijing, China +1