From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents

Myeongseob Ko; Jihyun Jeong; Sumiran Singh Thakur; Gyuhak Kim; Ruoxi Jia

cs.AIMar 19, 2026

From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents

Authors: Myeongseob Ko, Jihyun Jeong, Sumiran Singh Thakur, Gyuhak Kim, Ruoxi Jia

Organizations: Department of Electrical and Computer Engineering, Virginia Tech, Blacksburg, VA, USA · Center for Advanced AI, Accenture

Abstract

Anonymization is often assumed to protect privacy once explicit identifiers are removed, because re-identification has historically required specialized expertise, tailored algorithms, and manual corroboration. We show that LLM-based agents weaken this barrier: by combining scattered, individually non-identifying cues with public evidence, they reconstruct real-world identities, sometimes even during benign tasks. We evaluate this risk across three settings -- classical linkage incidents, a controlled benchmark (InferLink) that varies fingerprint type, task framing, and attacker knowledge, and open-ended human--AI interaction traces. In the sparsest regime of the Netflix Prize deanonymization setting, agents reconstruct 79.2% of identities, against 56.0% for a classical matching baseline; on InferLink, they link individuals even without an explicit re-identification request, and more often once one is given. In redacted human--AI interaction traces, agents further resolve anonymized profiles to specific individuals by corroborating contextual cues with public evidence. These findings suggest that privacy evaluations for agentic systems should measure not only what information is accessed or disclosed, but also what identities can be inferred.

Figures & tables

Figure 1: Overview of Inference-Driven Linkage. An AI agent reconstructs a specific identity ( $\hat{\imath}$ ) from fragmented cues. (Left) Anonymized artifacts ( $D_{\text{anon}}$ )—ChatGPT logs, AOL searches, interview transcripts—hold scattered, non-identifying cues. (Right) Auxiliary context ( $D_{\text{aux}}$ ): public web, social media, and news evidence. (Center) The agent aggregates cues from both sides into a single identity hypothesis.

	Baseline ( tolerance $T$ )		Ours ( agent; no hand-tuned $T$ )
$m$	$T{=}14$	$T{=}21$	GPT-5	Claude 4.5
8	98.3	98.8	99.00 $\pm$ 0.72	97.30 $\pm$ 3.50
6	96.7	97.1	97.43 $\pm$ 3.76	93.13 $\pm$ 8.39
4	90.5	91.8	94.83 $\pm$ 2.22	97.27 $\pm$ 2.78
2	56.0	60.2	79.17 $\pm$ 4.97	53.30 $\pm$ 19.21

Table 1: Netflix linkage success rate. In the Netflix setting, LLM agents match or exceed the classical baseline.

Figure 2: AOL qualitative example. In the AOL setting, the agent performs open-ended linkage by moving from anonymized queries ( $D_{\text{anon}}$ ) to corroborating public evidence ( $D_{\text{aux}}$ ) and ultimately to a specific identity hypothesis ( $\hat{\imath}$ ) .

		Utility ( $U$ ) $\uparrow$			Privacy Risk ( $LSR$ ) $\downarrow$
Intent	Fingerprint	o4-mini	GPT-5	Claude 4.5	o4-mini	GPT-5	Claude 4.5
Privacy Risk Evaluation (Per Fingerprint and Attacker Knowledge)
Implicit	Intrinsic	0.868	0.917	0.916	0.450	0.150	0.800
	Coordinate	0.868	0.884	1.000	0.250	0.250	0.700
	Hybrid	0.718	0.818	0.983	0.500	0.000	0.800
Explicit-ZK	Intrinsic	0.668	0.818	0.950	0.650	0.800	0.900

Table 2: Comprehensive Evaluation: Baseline Vulnerability and Mitigation Efficacy. The top section reports the baseline Utility (

U

) and Linkage Risk (

LSR

) for undefended agents under each evaluation setting. Green and Red indicate, for each row, the highest utility and the highest privacy risk, respectively. The bottom section presents the aggregated impact of the privacy-aware defense. To quantify the overall efficacy per intent, scores are averaged across all three fingerprint types. The Gap rows show the trade-off: Light Red denotes utility cost (

\Delta U

), and Light Blue denotes privacy gain (

\Delta LSR

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

		o4-mini		GPT-5		Claude 4.5
Intent	Fingerprint	LSR	Ref	LSR	Ref	LSR	Ref
Explicit-ZK	Intrinsic	.650	.00	.800	.00	.900	.10
	Coordinate	.400	.00	.900	.00	.850	.05
	Hybrid	.500	.00	.850	.10	1.000	.05
Explicit-MK	Intrinsic	.750	.00	.950	.05	1.000	.00
	Coordinate	.600	.20	.650	.25	.950	.00

Appendix

Table 3: Linkage success rate (LSR) and refusal rate (Ref) under the two Explicit settings. Refusal is near-zero for Claude 4.5 across conditions; for GPT-5 it rises selectively—most sharply on Coordinate under Explicit-MK —accounting for much of the corresponding LSR drop.

Privacy Leakage in Language Models

Privacy leakage in language models concerns how models expose training data, infer sensitive traits, or transmit private context through outputs and internal channels.

See all 280 papers

AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search
Oct 4, 2026Jianing Wen, Tianshi LiPrivacy Leakage in Language Models AI Agent Security Benchmarks
Khoury College of Computer Sciences Northeastern University Boston, MA
LLM Anonymization Against Agentic Re-Identification
May 29, 2026Ziwen Li, Jianing Wen, Tianshi LiPrivacy-Preserving Language Models Privacy-Utility Tradeoff
Khoury College of Computer Sciences Northeastern University Boston, MA
Large-scale online deanonymization with LLMs
Feb 18, 2026Simon Lermen, Daniel Paleka, Joshua Swanson +3LLM Security
MATS · Anthropic

Oct 4, 2026J/K move · Enter open · S save

AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search

Jianing Wen, Tianshi Li

Khoury College of Computer Sciences Northeastern University Boston, MA

Privacy Leakage in Language Models AI Agent Security Benchmarks

Open paper AskPDF arXiv

As Large Language Models (LLMs) gain tool use capabilities such as web search, they can retrieve and cross-reference public information, creating privacy risks beyond memorization. One manifestation is re-identification: linking an anonymized interview transcript to a named individual. Yet without ground-truth identities, the coverage of such attacks and the protection offered by a defense cannot be reliably measured. We introduce AgentDOXX, an evaluation suite of 822 synthetic interview transcripts grounded in public information about real individuals with known identities. We evaluate fifteen configurations of open-weight and proprietary models, isolating the effect of web search, and analyze their search trajectories to distinguish retrieval-driven from parametric identifications. Ground-truth identities reveal that re-identification risk is distributed across an agent's execution: retrieval and parametric recall both contribute, with open-weight models identifying 15-28% of transcripts without search; identification succeeds in over 88% of cases once the target appears in a retrieved result; entity masking leaves at least one attacker successful on 85.3% of a stratified sample; and privacy instructions suppress naming but not retrieval, with configurations scoring 0% accuracy yet retrieving the subject in up to 62% of transcripts. We further show that observed attack trajectories can provide supervision for localizing identifying spans, offering a path toward attack-informed anonymization.

Abstract

Privacy Leakage in Language Models

Explore similar work

AgentDoxx: Agentic Re-identification of Anonymized Text with Web Search

LLM Anonymization Against Agentic Re-Identification

Large-scale online deanonymization with LLMs