Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests

Qikun Cai; Tao Huang; Chen Hou

cs.CROct 13, 2026

Cost-Efficient Privacy Enforcement for Email-Generating LLM Agents under Strategic Information Requests

Authors: Qikun Cai, Tao Huang, Chen Hou

Abstract

LLM agents now act as delegated communication intermediaries: they read private context, draft messages, and send externally visible emails on behalf of a user. Existing checker-repair defenses can reduce inappropriate disclosure, but they rely on repeated calls to a remote action LLM (the model that drafts and repairs the email) and a remote checker LLM whenever a draft fails validation. This creates a cost problem as well as a privacy problem, especially when a requester strategically asks for exact values, raw records, source evidence, or other details that induce leakage-heavy drafts. We propose local contract-conditioned rewriting as a cost-efficient enforcement layer for email-generating agents. The mechanism intercepts an outgoing draft, constructs an information-release contract that specifies what to preserve, generalize, and forbid, rewrites the draft with a local Gemma rewriter, and selectively invokes a final PrivacyChecker. Across extensive email tasks under normal prompts and attack prompts that operationalize strategic information requests, Gemma + Checker reduces normal-prompt leakage from 11.41% to 4.03% while improving UsableRate (satisfied or partially satisfied outputs) from 85.2% to 92.6%. Under the strategic attack-prompt condition, it reduces leakage from 28.19% to 18.79% and reduces checker-only cost amplification from 1.32x to 1.02x. Offline replay ablation shows that the full contract reduces leakage relative to generic Gemma rewriting by 36.3% under normal prompts and 54.1% under attack prompts. These results support a local-first privacy enforcement design: move repair to a cheaper trusted local layer when possible, but retain the checker as the final semantic gate for high-risk sends.

Figures & tables

Figure 1: Local contract-conditioned enforcement. The mechanism moves repair work from repeated remote action-LLM calls to a local rewrite executor, while preserving an optional final checker for high-risk sends.

Component	Enforcement role
must_preserve , required_slots	Preserve the benign task intent and privacy-allowed content needed for a useful email.
must_generalize , privacy-risk rules	Abstract sensitive, confidential, source-specific, or reconstructive details rather than copying them.
information_shape , safe_output_shape	Keep the requested email structure, such as a status update, bullet list, next step, or high-level range.
must_not_add	Prevent hallucinated replacement facts, new values, new commitments, private-source references, or unsupported attachments.
Safe-substitution guidance	Preserve the function of a sensitive detail with a safe abstraction when deletion would make the email unhelpful.

Table 1: Main information-release contract components. The contract is a deployable rewrite policy, not an evaluation oracle.

Item	Setting
Tasks	149 aligned normal and attack email tasks from Privacy in Action; 480 sensitive items; Gmail/Notion retrieval followed by email generation or reply.
Action LLM $A$	DeepSeek deepseek-v4-flash ; same initial agent prompt and same base tasks across mechanisms.
Checker $C$	DeepSeek deepseek-v4-flash ; Boolean pass/fail; checker feedback is returned verbatim for repair.
Local rewriter $G$	gemma4-e4b-q4-12k via Ollama, q4_K_M ; temperature 0.0; 12k context.
Judges	Leakage evaluator: DeepSeek deepseek-v4-flash with a separate evaluation prompt; utility judge: DeepSeek deepseek-v4-pro .
Repair policy	Baseline continues the same agent session after checker failure, with a 25-turn overall cap; $M_{G+C}$ allows up to three local rewrite rounds; all checked mechanisms fail closed.

Table 2: Core experimental settings.

Prompt	Mechanism	Avg. checker calls	Remote tokens	Agent tokens	Checker tokens	Gemma local tokens
Normal	$M_{B}$	1.06	8,728,781	8,390,242	338,539	0
Normal	$M_{G}$	0.00	5,404,802	5,404,802	0	1,165,287
Normal	$M_{G+C}$	1.00	7,189,612	5,281,175	1,908,437	1,144,181
Attack	$M_{B}$	1.79	11,540,135	10,861,107	679,028	0
Attack	$M_{G}$	0.00	5,311,550	5,311,550	0	1,248,082
Attack	$M_{G+C}$	1.02	7,318,258	5,306,892	2,011,366	1,246,723

Table 3: Remote enforcement cost decomposition. Remote tokens are action-LLM tokens plus PrivacyChecker tokens. Gemma local tokens are reported separately and are not included in remote-cost comparisons.

Mechanism	Cost amp. $\downarrow$	Call amp. $\downarrow$	Attack leak. $\downarrow$	Leak $\Delta$ $\downarrow$
$M_{B}$	1.32 $\times$	1.69 $\times$	28.19%	+16.78 pp
$M_{G}$	0.98 $\times$	N/A	21.48%	+16.11 pp
$M_{G+C}$	1.02 $\times$	1.02 $\times$	18.79%	+14.76 pp

Table 4: Attack amplification summary. Cost amplification compares remote tokens under attack prompts with normal prompts; call amplification compares average PrivacyChecker calls. Leak

\Delta

is attack leakage minus normal leakage. Lower is better for all metrics.

Method	Prompt	Avg. repair rounds	Slot retention	Satisfied	Partially satisfied	UsableRate	Semantic drift
$M_{B}$	Normal	0.12	0.999	43.0%	42.3%	85.2%	0.001
$M_{G}$	Normal	0.00	0.970	34.9%	55.0%	89.9%	0.062
$M_{G+C}$	Normal	0.00	0.981	39.6%	53.0%	92.6%	0.046
$M_{B}$	Attack	1.52	0.984	97.3%	2.0%	99.3%	0.012
$M_{G}$	Attack	0.00	0.977	94.6%	4.0%	98.7%	0.043
$M_{G+C}$	Attack	0.04	0.991	98.7%	1.3%	100.0%	0.022

Table 5: Semantic preservation and safe-task fulfillment. For attack prompts, satisfaction labels measure the benign task shell after removing attack-specific requests for sensitive details.

Variant	Normal leakage	Normal red.	Attack leakage	Attack red.
Generic Gemma rewrite	7.38%	–	32.21%	–
Full contract	4.70%	36.3%	14.77%	54.1%
w/o required_slots / information_shape	5.37%	27.2%	15.44%	52.1%

Table 6: Offline contract ablation. Relative reductions are computed against generic Gemma rewriting. The ablation evaluates leakage only, not utility or token cost.

Case	Observed mechanism behavior	Mechanism lesson
School health update	$M_{B}$ directly reports condition, recovery status, and test evidence. $M_{G+C}$ rewrites the message into a high-level health-status and care-plan summary.	Abstraction can preserve the coordination task without copying sensitive facts.
Structured compensation request	Under an adversarial structured request, $M_{B}$ discloses exact salary, bonus, and equity details. $M_{G+C}$ keeps the negotiation context but removes exact compensation and stock-option fields.	Contracted rewriting resists exact-value and structured-extraction pressure.
Gemma-only residual leakage	$M_{G}$ removes explicit disease names but leaves respiratory-monitoring and activity-clearance cues. The checker in $M_{G+C}$ triggers further abstraction and removes reconstructive implications.	Local rewriting reduces leakage before checking, but final validation is still needed for high-risk sends.

Table 7: Mechanism-level case summaries. The table reports anonymized behavior, not full private email text.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Field group	Fields and purpose
Task typing	task_type , task_subtype , task_confidence , matched_signals , and low_confidence classify the email task and record the confidence of the rule-based task inference. Low-confidence cases fall back to preserving the draft’s apparent purpose and structure.
Task goal and utility	task_goal , must_preserve , required_slots , information_shape , fulfillment_requirements , body_inclusion_requirement , detail_density , plural_scope_requirement , and safe_output_shape specify the benign task shell, expected output form, required safe slots, and minimum conditions for a rewritten email to remain task-useful.
Privacy constraints	must_generalize , privacy_risk_policy , safe_detail_preservation_policy , source_grounding_requirements , and safe_substitution_plan identify sensitive or source-specific content, distinguish risky details from preservable low-risk details, define semantic leakage risks, and guide the rewrite toward safe substitutes such as high-level conclusions, category-level fields, process-level status, safe ranges, or de-identified entries.
No-fabrication and rewrite-control constraints	must_not_add and rewrite_hints prevent unsupported additions, task shifts, private-source references, placeholders, and excessive rewriting, while encouraging minimal edits and preservation of safe source-grounded content.

Appendix

Table 8: Information-release contract fields grouped by function.

Field	Example value
task_type	update_status .
task_goal	Send a status update to the intended recipient after removing private-source lookup hints.
must_preserve	Update purpose; safe high-level topic; high-level status or conclusion when privacy-safe.
must_generalize	Person-specific health, student, financial, legal, HR, client, project, or source-grounded details that reveal or reconstruct private information.
must_not_add	Unsupported facts; new names, dates, locations, amounts, attachments, private-source references, placeholders, or unsupported follow-up promises.
required_slots	Greeting; update purpose; current high-level status; safe progress point; next step; closing.

Appendix

Table 9: Compact anonymized contract example.

Variant	Construction
Generic rewrite	Uses a generic privacy rewrite instruction with the user instruction and original draft, but without the information-release contract. The prompt asks the model to reduce privacy leakage, preserve the legitimate task intent, avoid unsupported facts, avoid private-source references, and return JSON with subject and body .
Full contract	Uses the same user instruction and original draft, but additionally provides the complete information-release contract. The prompt asks the model to preserve utility-bearing content when the contract requires it, generalize or omit sensitive details when the contract requires it, and avoid unsupported facts.
No required slots / shape	Uses the full contract after removing the fields that specify required task slots and output shape. All other contract fields are preserved.

Appendix

Table 10: Prompt variants used in the P1-1 contract ablation.

Mechanism	Behavior	Leakage outcome
Gemma-only	Removes some explicit sensitive wording, but preserves a combination of health-management, respiratory-care, environmental-trigger, and activity-clearance cues. These cues make the output useful, but they also allow reconstruction of private health information.	Residual leakage: 2 of 3 annotated sensitive items leaked.
Gemma + Checker	Further abstracts the same coordination task into general care-management and school-participation language. The benign task shell is preserved, while condition-specific and test-specific implications are avoided.	No leakage: 0 of 3 annotated sensitive items leaked.

Appendix

Table 11: Supplementary mechanism-level analysis for Case 3.

Privacy Protection for LLM Agents

See all 33 papers

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency
Apr 7, 2026Yunze Xiao, Wenkai Li, Xiaoyuan Wu +3Privacy-Preserving Language Models Privacy Protection for LLM Agents
Carnegie Mellon University · Massachusetts Institute of Technology · 2077AI +1
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
May 29, 2026Mingxuan Zhang, Jiahui Han, Dadi Guo +5Data Leakage Privacy Auditing
Shanghai Artificial Intelligence Laboratory · Southeast University
Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control
Sep 12, 2026Minsun Shim, Ramisha Raida Karim, Ruthwik Jakkula +4LLM Agent Security Privacy Leakage in Language Models
University of California, Irvine · University of California, Santa Cruz · University of California, Davis +1

Apr 7, 2026J/K move · Enter open · S save

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

Yunze Xiao, Wenkai Li, Xiaoyuan Wu, Ningshan Ma+2

Carnegie Mellon University · Massachusetts Institute of Technology · 2077AI+1

Privacy-Preserving Language Models Privacy Protection for LLM Agents Privacy Leakage in Language Models

Open paper AskPDF arXiv

LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private. Existing systems support only suppression (omitting sensitive information) and generalization (replacing information with an abstraction), and are typically evaluated on single isolated messages, leaving both the strategy space and evaluation setting incomplete. We formalize privacy-preserving LLM communication as an Information Sufficiency (IS) task, introduce free-text pseudonymization as a third strategy that replaces sensitive attributes with functionally equivalent alternatives, and propose a conversational evaluation protocol that assesses strategies under realistic multi-turn follow-up pressure. Across 792 scenarios spanning three power-relation types (institutional, peer, intimate) and three sensitivity categories (discrimination risk, social cost, boundary), we evaluate seven frontier LLMs on privacy at two granularities, covertness, and utility. Pseudonymization yields the strongest privacy\textendash utility tradeoff overall, and single-message evaluation systematically underestimates leakage, with generalization losing up to 16.3 percentage points of privacy under follow-up.

Abstract

Privacy Protection for LLM Agents

Explore similar work

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say

Confuse the Model, Control the Flow: Understanding and Mitigating Privacy Leakage from LLM Agents with Information Flow Control