Privacy Leakage in Language Models

Overview

The field studies when language models reveal information they learned or received, including personal details, training-set membership, and sensitive traits inferred from text. Researchers test verbatim extraction and membership inference, as well as reconstruction from gradients, activations, hidden states, and model outputs; retrieval systems, persistent memory, and tool-using agents create additional risks across turns and users. Defenses include differential privacy, data filtering, unlearning, activation steering, prompt sanitization, obfuscation, and system-level access controls and information-flow enforcement. A central tension is that protections can reduce leakage while harming usefulness, and refusal or successful output-level forgetting does not necessarily prevent information from being recovered through other prompts, representations, or system channels.

Latest papers

CardsList

303 papers22 papers in the last four weeks, down 54% on the four weeks before. 0.2% of all new papers.

  1. Mitigating Memorization In Language Models

    Oct 3, 2024Mansi Sakarvadia, Aswathy Ajith, Arham Khan +6Memorization in Language ModelsLLM Fine-Tuning

  2. Could Mobile AI be Exploited?: Understanding the Exploitations of Trust and Ethics Gap in On-Device Small Language Models

    Jun 8, 2024Kalyan Nakka, Jimmy Dani, Nitesh SaxenaLanguage Model Safety EvaluationLLM Safety

  3. PPFedIT: Towards Privacy-Preserving Federated Instruction Tuning with Few-shot Local Examples

    Mar 10, 2024Zhuo Zhang, Jingyuan Zhang, Jintao Huang +6Privacy Leakage in Language ModelsFew-Shot Learning