When Safety Checks Become Privacy Risks: A Privacy-Preserving Paradigm for LLM Agent Safety Auditing

Yixiang Luo; Ziheng Ma; Ke Chang; Jian Wang; Jiqiang Liu

cs.CROct 30, 2026

When Safety Checks Become Privacy Risks: A Privacy-Preserving Paradigm for LLM Agent Safety Auditing

Authors: Yixiang Luo, Ziheng Ma, Ke Chang, Jian Wang, Jiqiang Liu

Organizations: Beijing Jiaotong University, Beijing, China

Abstract

Large language model agents interact with online services through tool calls that may access private data or perform sensitive actions. Specialized providers can supply and maintain pre-execution safety auditing capabilities for agent operators. However, sending task descriptions, call arguments, and interaction histories to an external auditor may expose sensitive user information. In this work, we propose a paradigm for privacy-preserving safety auditing that separates delegated audit computation from plaintext access and execution authority. Grounded in a three-domain auditing model, the paradigm connects trusted representation construction, encrypted audit computation, and trusted decision and execution through explicit interfaces. These interfaces accommodate alternative representations and scorers, while encrypted computation protects audit inputs from a semi-honest auditor. We instantiate this paradigm in SealCall, a framework combining learned audit representations, CKKS-based encrypted similarity computation, and execution control bound to audited tool calls. Evaluated on 1,252 public binary candidates, SealCall achieves a mean F1 score of 0.840 under grouped nested evaluation, while preserving plaintext audit decisions in all measured cases. A representative configuration completes each encrypted scoring operation in 0.176 s.

Figures & tables

Figure 1. Privacy exposure in outsourced auditing and the SealCall workflow. Top: plaintext audit inputs expose private user context to the auditor, while an unprotected reference set exposes audit information. Bottom: the SealCall framework separates trusted representation construction, encrypted audit computation, and trusted decision and execution across three domains. The illustrated configuration uses CKKS reference similarities; the secret key and release authority remain in the user control domain. The upper panel shows plaintext audit context flowing to an external auditor and an unprotected reference set exposed to an attacker. The lower panel shows the agent execution domain on the left, the user control domain in the center, and the audit service domain on the right. A dashed boundary encloses the trusted controller. The agent supplies its final proposed call and preceding history; the user supplies the task. The controller compresses this context, combines text embeddings and structured features through a learned projection, and encrypts the resulting audit representation. The audit service computes CKKS similarities using an encrypted unsafe reference set and packs the ciphertext results for return. The controller decrypts them and applies maximum selection and thresholding. An allow proceeds through execution binding to the covered tools; a block stops the call. The secret key and release authority stay within the trusted controller.

Layer (role)	Interface	Components in the evaluated configuration
Trusted representation ( $C$ )	Audit context $A_{t}$ $\rightarrow$ encrypted representation $\operatorname{Enc}(z_{t})$	Source-linked context; BGE and structured features; learned projection.
Encrypted computation ( $S$ )	Encrypted representation and audit data $\rightarrow$ encrypted results	CKKS reference dot products; ciphertext response packing.
Trusted decision and execution ( $C$ )	Encrypted results $\rightarrow$ call-bound allow or block	Decryption; maximum and threshold; call and state checks.

Table 1. Three processing layers of the SealCall framework. The trusted controller

C

retains plaintext access, the secret key, and release authority; the audit server

S

computes on ciphertexts.

Source	Binary	Roots	Safe	Unsafe	Interm.
Harm	399	144	205	194	273
Dojo	853	58	512	341	0
Total	1,252	202	717	535	273

Table 2. Accepted public candidates. Intermediate labels are excluded from binary fitting and evaluation. Roots are the split unit.

Scorer	F1	Recall	FPR	F1	FPR	AUROC
	$B_{10}$			$B_{F1}$
SealCall reference set	0.812	0.823	0.153	0.840	0.112	0.921
Same-space center	0.805	0.805	0.145	0.818	0.120	0.917
Same-input LR	0.899	0.936	0.110	0.913	0.089	0.967
Same-input RBF-SVM	0.889	0.948	0.137	0.905	0.052	0.970
Direct cosine	0.627	0.540	0.137	0.666	0.449	0.736

Table 3. Detection quality on the same 1,252 candidates. Each policy is calibrated before outer testing. AUROC averages within folds. Lower FPR is better; other metrics are higher-is-better.

Representation / readout	Harm F1	Dojo F1	Pooled F1
SealCall full	0.728	0.854	0.812
Without text embeddings	0.599	0.857	0.769
Without structured features	0.549	0.790	0.718
BGE + features / LR	0.792	0.929	0.879
TF-IDF + features / LR	0.808	0.919	0.878
Features only / LR	0.742	0.939	0.870

Table 4. Representation comparisons at

B_{10}

on all 1,252 candidates. The lower panel uses a shared, separately block-normalized LR readout.

Scorer	F1	Time (s)	Packed MB	Raw MB
SealCall	0.895	0.176	0.131	3.761
Center	0.845	0.016	0.235	0.235
LR	0.909	0.043	0.235	0.235
RBF-SVM	0.915	15.267	0.131	93.794

Table 5. Quality and private scoring cost for one pre-selected model per scorer. F1 uses

B_{F1}

on that model’s 251-case outer test fold; Table 3 aggregates all outer folds. Response sizes are decimal MB.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

	Harm		Dojo
Scorer	F1	FPR	F1	FPR
SealCall	0.745	0.132	0.888	0.105
Center	0.689	0.122	0.878	0.119
LR	0.869	0.146	0.938	0.066
RBF-SVM	0.841	0.102	0.941	0.031
Direct cosine	0.621	0.527	0.689	0.418

Appendix

Table 6. Source-resolved

B_{F1}

quality on the 1,252-candidate benchmark.

Proxy	Roots	Safe	Unsafe	Recall	FPR	Recall	FPR	Recall	FPR
				SealCall		Same-input LR		TF-IDF + features, LR
Send	97	73	163	0.896	0.260	0.939	0.274	0.969	0.411
Write	72	148	187	0.929	0.284	0.995	0.122	0.989	0.203
Execute	5	9	9	0.178	0.022	0.333	0.111	0.222	0.000
Publish	57	36	103	0.715	0.294	0.913	0.278	0.981	0.556
Other	147	451	73	0.622	0.083	0.890	0.067	0.767	0.058

Appendix

Table 7. Detection within frozen tool-name strata at

B_{10}

. Assignment uses one naming rule for safe and unsafe cases.

Fold	Configuration	$K$	Fit	Stop	Cal.	Test
1	bank256	16	721/115	79/12	201/32	251/43
2	hard128	16	717/117	85/12	202/32	248/41
3	hard128	1	719/118	79/13	203/32	251/39
4	bank128	1	719/116	81/13	199/34	253/39
5	hard128	8	719/117	82/13	202/32	249/40

Appendix

Table 8. Outer-fold selections and disjoint final roles (decisions/task roots).

K

is a prefix of the same unsafe reference order.

	Meaning preserved			Relation changed
Method	Orig. F1	Reword. F1	All 3 correct	Pair 0	Pair 1	All 4 correct
SealCall	0.785	0.789	0.776	0.013	0.013	0.000
Without text embeddings	0.669	0.672	0.706	0.000	0.000	0.000
Without structured features	0.675	0.677	0.708	0.005	0.013	0.000
TF-IDF + features / LR	0.876	0.887	0.870	0.013	0.013	0.000
BGE + features / LR	0.895	0.891	0.880	0.013	0.000	0.000

Appendix

Table 9. Frozen-model semantic diagnostics at

B_{10}

. Keeping meaning: 100 source-root anchors with two task restatements (300 candidates). Changing policy relations: 80 constructed worlds with two labels and two renderings (320 candidates). Higher correctness is better.

Method	Family	F1
SealCall	recipient_binding	0.455
SealCall	purpose_scope	0.171
SealCall	source_authority	0.607
SealCall	prior_precondition	0.598
Without text embeddings	recipient_binding	0.421
Without text embeddings	purpose_scope	0.323

Appendix

Table 10. Constructed-policy diagnostics by relation family (20 worlds per family). Each world pairs one safe and one unsafe case.

Figure 2. Source-specific operating curves from held-out fold scores. Scores from different fitted models are never pooled. Two panels compare SealCall, removal of text embeddings, TF-IDF with features, and same-input logistic regression on AgentHarm and AgentDojo. Each panel averages within-fold ROC curves rather than pooling score scales.

AI Agent Auditing

AI agent auditing examines agents’ actions, tool use, evidence, and outcomes to assess safety, compliance, reliability, and accountability.

See all 286 papers

AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy
Oct 5, 2026Shouju Wang, Haopeng ZhangAI Agent Auditing LLM Agent Evaluation
University of North Carolina at Charlotte
Agent Audit: A Security Analysis System for LLM Agent Applications
Mar 24, 2026Haiyue Zhang, Yi Nian, Yue ZhaoAI Agent Auditing LLM Agent Security
University of Southern California Los Angeles, California, USA
PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say
May 29, 2026Mingxuan Zhang, Jiahui Han, Dadi Guo +5Data Leakage Privacy Auditing
Shanghai Artificial Intelligence Laboratory · Southeast University

Oct 5, 2026J/K move · Enter open · S save

AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy

Shouju Wang, Haopeng Zhang

University of North Carolina at Charlotte

AI Agent Auditing LLM Agent Evaluation AI Agent Monitoring

Open paper AskPDF arXiv

The rapid advancement of LLM agents has enabled systems to autonomously perform complex tasks through external tools, but their growing access to personal data introduces significant privacy risks. Existing benchmarks primarily evaluate LLM agent privacy through simulated trajectories and outcome-based metrics, limiting their ability to capture privacy risks arising during multi-step agent execution. In this work, we introduce AgentPrivArena, a framework for evaluating privacy risks in realistic LLM agent workflows. AgentPrivArena integrates authentic MCP tools and self-hosted services within a reproducible execution environment. We further propose trajectory-level privacy metrics that quantify unnecessary information access beyond final response leakage. Building on this framework, we introduce AgentPrivAudit, a runtime auditing approach for monitoring privacy violations during agent execution. Extensive experiments on state-of-the-art LLM agents reveal substantial privacy risks overlooked by existing evaluation paradigms, highlighting the importance of trajectory-level auditing for trustworthy agent deployment.

Abstract

AI Agent Auditing

Explore similar work

AgentPrivArena: Evaluating and Auditing Real-world AI Agent Privacy

Agent Audit: A Security Analysis System for LLM Agent Applications

PrivacyPeek: Auditing What LLM-Based Agents Acquire, Not Just What They Say