Privacy Protection for LLM Agents

LLM: Large Language Model

Follow this topic Ask about this topic

Latest papers

29 papers6 papers in the last four weeks, against 1 the four weeks before. 0.0% of all new papers.

AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents
Oct 13, 2026Chengquan Guo, Zhaorun Chen, Zinan Lin +2Privacy Protection for LLM Agents Contextual Integrity
University of Chicago · Microsoft Research · University of Illinois Urbana-Champaign
Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests
Oct 13, 2026Qikun Cai, Tao Huang, Chen HouPrivacy Protection for LLM Agents LLM Agent Security
OverAct: Measuring and Mitigating Proactive Over-Authorization in LLM Tool-Calling Agents
Oct 1, 2026Taolin Zhang, Jiuheng Wan, Hanyu Wang +2Privacy Leakage in Language Models AI Agent Security Benchmarks
Hefei University of Technology · East China Normal University · Alibaba Cloud Computing
PrivacySkills: How Privacy Guidance Shapes Source Selection in LLM Agents
Sep 28, 2026Lucas Biechy, Cédric Eichler, Héber H. Arcolezi +1LLM Agent Evaluation Privacy Protection for LLM Agents
Petscraft, Inria · Université Paris-Saclay · INSA CVL +3
CoSec: Benchmarking Agent Security in Communities
Sep 28, 2026Hao Chen, Wenhui Dong, Ye Chen +12LLM Agent Security AI Agent Security Benchmarks
Nanjing University · Xi’an Jiaotong University · Beihang University +6
Ask Without Telling: Local SLMs Consult Cloud LLMs Without Revealing Task Intent
Sep 26, 2026Yanmeng Wang, Yunxuan Li, Shilong Fan +2Privacy-Preserving Language Models Privacy Protection for LLM Agents
Nanjing University of Posts and Telecommunications, Nanjing, 210023, China · The Chinese University of Hong Kong, Shenzhen, 518172, China
Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control
Sep 12, 2026Minsun Shim, Ramisha Raida Karim, Ruthwik Jakkula +4LLM Agent Security Privacy Leakage in Language Models
University of California, Irvine · University of California, Santa Cruz · University of California, Davis +1
Redakto - The Incognito Tab for LLMs
Aug 18, 2026Saurav Kumar Saha, Tom Röhr, Felix BießmannPrivacy Protection for LLM Agents Privacy-Preserving Language Models
Berlin University of Applied Sciences, Luxemburger Str. 10, 13353 Berlin, Germany · Einstein Center Digital Future, Wilhelmstraße 67, 10117 Berlin, Germany
Beyond Direct Identifiers: Probabilistic Privacy Risk Estimation for Privacy-Conscious LLM Query Delegation
Aug 10, 2026Li Siyan, Zhou Yu, Julia HirschbergPrivacy-Preserving Language Models Privacy Leakage in Language Models
Department of Computer Science Columbia University
MNC: Scope-Bound Semantic Declassification for Private LLM-Agent Communication
Aug 3, 2026Jinghan Xu, Longze Fan, Zeyuan Wang +2Multi-Agent LLM Systems LLM Agent Security
1Nankai University, China · 2China University of Petroleum, China · 3Sun Yat-sen University, China +1
HiMe: Real-Time Self-Hosted Personal Agent Platform for Health Insights with Wearable Devices
Jul 23, 2026Wei Liu, Siya Qi, Linhai Zhang +2Privacy Protection for LLM Agents LLM Agents
King’s College London · The Alan Turing Institute
SlotGuard: Stop Oversharing Private Local Context in LLM Agent Transcri
Jul 19, 2026Haocheng Xia, Yongjoo ParkLLM Agent Security Privacy Leakage in Language Models
Siebel School of Computing and Data Science, University of Illinois Urbana-Champaign.
SurrogateShield: Beyond Redaction for High-Utility, Privacy-Preserving LLM Interactions
Jun 28, 2026Sherwin Vishesh JathannaPrivacy-Preserving Language Models LLM Security
Arizona State University Tempe, Arizona, USA
ToolPrivacyBench: Benchmarking Purpose-Bound Privacy in Tool-Using LLM Agents
Jun 26, 2026Shijing Hu, Liang Liu, Zhu Meng +1Privacy Auditing Tool-Use Evaluation
Beijing University of Posts and Telecommunications, Beijing, 100876, China · Beijing Academy of Blockchain and Edge Computing, Beijing, 100085, China
MuPPET: A Benchmark for Contextual Privacy of LLM Assistants in Multi-Party Conversations
Jun 22, 2026Elena Sofia Ruzzetti, Cornelius Emde, Sangdoo Yun +2Privacy Leakage in Language Models Privacy Protection for LLM Agents
Parameter Lab · University of Oxford · NAVER AI Lab +1
PrivacyAlign: Contextual Privacy Alignment for LLM Agents
Jun 19, 2026Manveer Singh Tamber, Abhay Puri, Marc-Etienne Brunet +3Reward Modeling Privacy Leakage in Language Models
University of Waterloo · †Work done during internship at ServiceNow AI Research · 2ServiceNow AI Research +2
SecureClaw: Clawing Back Control of LLM Agents
Jun 8, 2026Yuhan Ma, Stefan SchmidData Leakage LLM Agent Security
TU Berlin
Need to Know: Contextual-Integrity-Grounded Query Rewriting for Privacy-Conscious LLM Delegation
Jun 2, 2026Xinyue Huang, Xiaochun Cao, Wenyuan YangContextual Integrity Privacy-Preserving Language Models
Sun Yat-sen University
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
May 29, 2026Mingxuan Zhang, Jiahui Han, Dadi Guo +5Data Leakage Privacy Auditing
Shanghai Artificial Intelligence Laboratory · Southeast University
Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems
May 26, 2026Aman Priyanshu, Supriti Vijay, Esha PahwaAI Agent Safety Benchmarks Multi-Agent LLM Systems
Foundation AI · USA · Corvic AI
PAAC: Privacy-Aware Agentic Device-Cloud Collaboration
May 9, 2026Liangqi Yuan, Wenzhi Fang, Shiqiang Wang +1Privacy-Preserving Language Models AI Agent Security
1Purdue University · University of Exeter
Say Something Else: Rethinking Contextual Privacy as Information Sufficiency
Apr 7, 2026Yunze Xiao, Wenkai Li, Xiaoyuan Wu +3Privacy-Preserving Language Models Privacy Protection for LLM Agents
Carnegie Mellon University · Massachusetts Institute of Technology · 2077AI +1
BodhiPromptShield: Pre-Inference Prompt Mediation for Surface-Form Privacy Propagation in LLM Agent Pipelines
Apr 7, 2026Bo Ma, Jinsong Wu, Weiqi YanData Leakage Privacy-Preserving Language Models
Auckland University of Technology Auckland, New Zealand · Guilin University of Electronic Technology Guilin, China
AgentSocialBench: Evaluating Privacy Risks in Human-Centered Agentic Social Networks
Apr 1, 2026Prince Zizhuang Wang, Shuli JiangPrivacy Protection for LLM Agents LLM Agent Security
Carnegie Mellon University
Privacy Guard & Token Parsimony by Prompt and Context Handling and LLM Routing
Mar 30, 2026Alessio LangiuPrivacy Protection for LLM Agents Privacy Leakage in Language Models
National Research Council of Italy - Institute of Marine Sciences (CNR-ISMAR)
Investigating In-Context Privacy Learning by Integrating User-Facing Privacy Tools into Conversational Agents
Mar 19, 2026Mohammad Hadi Nezhad, Francisco Enrique Vicente Castro, Ivon ArroyoPrivacy Protection for LLM Agents Human-AI Interaction
University of Massachusetts Amherst · New York University
Anonymous-by-Construction: An LLM-Driven Framework for Privacy-Preserving Text
Mar 17, 2026Federico Albanese, Pablo Ronco, Nicolás D'IppolitoPrivacy-Preserving ML Privacy-Preserving Language Models
Veritran, Buenos Aires, Argentina · University of Buenos Aires, Argentina · University of San Andrés, Buenos Aires, Argentina
Contextualized Privacy Defense for LLM Agents
Mar 3, 2026Yule Wen, Yanzhe Zhang, Jianxun Lian +3Privacy Protection for LLM Agents LLM Agent Security
Tsinghua University · Georgia Tech · Microsoft +1
NeuroFilter: Activation-Based Guardrails for Privacy-Conscious LLM Agents
Jan 21, 2026Saswat Das, Ferdinando FiorettoLLM Guardrails Privacy Leakage in Language Models
University of Virginia

Oct 13, 2026J/K move · Enter open · S save

AgentCIScope: Grounded Contextual Integrity Evaluation for LLM Agents

Chengquan Guo, Zhaorun Chen, Zinan Lin, Xiaodong Liu+1

University of Chicago · Microsoft Research · University of Illinois Urbana-Champaign

Privacy Protection for LLM Agents Contextual Integrity LLM Agent Evaluation LLM Agent Security AI Agent Security Benchmarks

Open paper AskPDF arXiv

LLM agents are becoming increasingly capable of handling daily workspace tasks by interacting with application environments through tool calls. However, as agents read and write sensitive user data across email, documents, and other applications, a central privacy question is whether they preserve contextual integrity (CI), that is, whether the information flows they produce comply with context-specific disclosure constraints. Existing work either focuses on model-level privacy leakage or evaluates agents in limited application environments with manually constructed disclosure constraints. In this work, we introduce AgentCIScope, a unified platform and benchmark for evaluating agent-level contextual integrity through executable, multi-step tasks across diverse, realistic application environments. AgentCIScope transforms public and synthetic privacy cases into executable agent tasks, with disclosure constraints grounded in three criteria: governing privacy policies, persona-based agreement, and a predefined sensitive-attribute taxonomy. Based on AgentCIScope, we evaluate 12 agents on 936 tasks and find that strong task-completion capability does not guarantee contextual-integrity preservation: single-run leakage rates range from 32.5% to 94.4%, while repeating each task five times further increases leakage by 4-19 percentage points. Leakage varies substantially across attributes: personal preferences and booking details have average leakage rates of 84.7% and 76.5%, compared with 17.1% for credit-card information and 29.4% for SSNs; environments: calendar has the highest average leakage rate as an information source, while WhatsApp ranks highest as an information destination; and governing privacy policies. AgentCIScope provides deep insights into agent privacy leakage and serves as a unified testbed for developing privacy-preserving agents.