Anonymization is often assumed to protect privacy once explicit identifiers are removed, because re-identification has historically required specialized expertise, tailored algorithms, and manual corroboration. We show that LLM-based agents weaken this barrier: by combining scattered, individually non-identifying cues with public evidence, they reconstruct real-world identities, sometimes even during benign tasks. We evaluate this risk across three settings -- classical linkage incidents, a controlled benchmark (InferLink) that varies fingerprint type, task framing, and attacker knowledge, and open-ended human--AI interaction traces. In the sparsest regime of the Netflix Prize deanonymization setting, agents reconstruct 79.2% of identities, against 56.0% for a classical matching baseline; on InferLink, they link individuals even without an explicit re-identification request, and more often once one is given. In redacted human--AI interaction traces, agents further resolve anonymized profiles to specific individuals by corroborating contextual cues with public evidence. These findings suggest that privacy evaluations for agentic systems should measure not only what information is accessed or disclosed, but also what identities can be inferred.
Figures & tables
Figure 1: Overview of Inference-Driven Linkage. An AI agent reconstructs a specific identity ( ^ ) from fragmented cues. (Left) Anonymized artifacts ( Danon )—ChatGPT logs, AOL searches, interview transcripts—hold scattered, non-identifying cues. (Right) Auxiliary context ( Daux ): public web, social media, and news evidence. (Center) The agent aggregates cues from both sides into a single identity hypothesis.
Baseline ( tolerance T )
Ours ( agent; no hand-tuned T )
m
T=14
T=21
GPT-5
Claude 4.5
8
98.3
98.8
99.00 ± 0.72
97.30 ± 3.50
6
96.7
97.1
97.43 ± 3.76
93.13 ± 8.39
4
90.5
91.8
94.83 ± 2.22
97.27 ± 2.78
2
56.0
60.2
79.17 ± 4.97
53.30 ± 19.21
Table 1: Netflix linkage success rate. In the Netflix setting, LLM agents match or exceed the classical baseline.
Figure 2: AOL qualitative example. In the AOL setting, the agent performs open-ended linkage by moving from anonymized queries ( Danon ) to corroborating public evidence ( Daux ) and ultimately to a specific identity hypothesis ( ^ ) .
Utility ( U ) ↑
Privacy Risk ( LSR ) ↓
Intent
Fingerprint
o4-mini
GPT-5
Claude 4.5
o4-mini
GPT-5
Claude 4.5
Privacy Risk Evaluation (Per Fingerprint and Attacker Knowledge)
Implicit
Intrinsic
0.868
0.917
0.916
0.450
0.150
0.800
Coordinate
0.868
0.884
1.000
0.250
0.250
0.700
Hybrid
0.718
0.818
0.983
0.500
0.000
0.800
Explicit-ZK
Intrinsic
0.668
0.818
0.950
0.650
0.800
0.900
Table 2: Comprehensive Evaluation: Baseline Vulnerability and Mitigation Efficacy. The top section reports the baseline Utility ( U ) and Linkage Risk ( LSR ) for undefended agents under each evaluation setting. Green and Red indicate, for each row, the highest utility and the highest privacy risk, respectively. The bottom section presents the aggregated impact of the privacy-aware defense. To quantify the overall efficacy per intent, scores are averaged across all three fingerprint types. The Gap rows show the trade-off: Light Red denotes utility cost ( ΔU ), and Light Blue denotes privacy gain ( ΔLSR ).
Figure 5
Figure 5: Inference-driven linkage from human–AI interaction traces. In both cases, an anonymized trace ( Danon ) retains individually weak cues that the agent extracts and corroborates against retrieved public evidence ( Daux ) to form an identity hypothesis. (a) A redacted AI-use interview. (b) An anonymized multi-turn conversation, where cues accumulate across turns; the narrowing stages shown are illustrative for this confirmed case, not population-level estimates.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
o4-mini
GPT-5
Claude 4.5
Intent
Fingerprint
LSR
Ref
LSR
Ref
LSR
Ref
Explicit-ZK
Intrinsic
.650
.00
.800
.00
.900
.10
Coordinate
.400
.00
.900
.00
.850
.05
Hybrid
.500
.00
.850
.10
1.000
.05
Explicit-MK
Intrinsic
.750
.00
.950
.05
1.000
.00
Coordinate
.600
.20
.650
.25
.950
.00
Appendix
Table 3: Linkage success rate (LSR) and refusal rate (Ref) under the two Explicit settings. Refusal is near-zero for Claude 4.5 across conditions; for GPT-5 it rises selectively—most sharply on Coordinate under Explicit-MK —accounting for much of the corresponding LSR drop.
Privacy leakage in language models concerns how models expose training data, infer sensitive traits, or transmit private context through outputs and internal channels.
As Large Language Models (LLMs) gain tool use capabilities such as web search, they can retrieve and cross-reference public information, creating privacy risks beyond memorization. One manifestation is re-identification: linking an anonymized interview transcript to a named individual. Yet without ground-truth identities, the coverage of such attacks and the protection offered by a defense cannot be reliably measured. We introduce AgentDOXX, an evaluation suite of 822 synthetic interview transcripts grounded in public information about real individuals with known identities. We evaluate fifteen configurations of open-weight and proprietary models, isolating the effect of web search, and analyze their search trajectories to distinguish retrieval-driven from parametric identifications. Ground-truth identities reveal that re-identification risk is distributed across an agent's execution: retrieval and parametric recall both contribute, with open-weight models identifying 15-28% of transcripts without search; identification succeeds in over 88% of cases once the target appears in a retrieved result; entity masking leaves at least one attacker successful on 85.3% of a stratified sample; and privacy instructions suppress naming but not retrieval, with configurations scoring 0% accuracy yet retrieving the subject in up to 62% of transcripts. We further show that observed attack trajectories can provide supervision for localizing identifying spans, offering a path toward attack-informed anonymization.
Jianing Wen, Tianshi Li
Khoury College of Computer Sciences Northeastern University Boston, MA
Agentic LLMs with web search change the threat model for text anonymization: weak contextual cues can become cross-referenceable evidence for re-identification, yet those same details also carry downstream analytic value of the text. Existing defenses either remove explicit identifiers, perturb text for formal privacy, or test rewritten text against non-web inference models, leaving underexplored the operating region between resistance to agentic web-search re-identification and utility retention. We introduce AURA (Anonymization with Utility-Retention Adaptation), an LLM-powered mask-reconstruct framework that decouples privacy localization from utility-preserving reconstruction and selects candidates with adversarial privacy and utility-retention checks. We evaluate AURA on real-user interview transcripts using re-identification attacks carried out by web-search agents, along with a utility evaluation based on interviewee-profile facts, codebook facts, and the joint contextual utility grid. Our results show that AURA improves the privacy-utility frontier by using adaptive privacy scope to strengthen resistance to agentic re-identification and using a mask-reconstruct anonymization method to better preserve contextual utility under fixed privacy scope.
Ziwen Li, Jianing Wen, Tianshi Li
Khoury College of Computer Sciences Northeastern University Boston, MA
We show that large language models can be used to perform at-scale deanonymization. With full Internet access, our agent can re-identify Hacker News users and Anthropic Interviewer participants at high precision, given pseudonymous online profiles and conversations alone, matching what would take hours for a dedicated human investigator. We then design attacks for the closed-world setting. Given two databases of pseudonymous individuals, each containing unstructured text written by or about that individual, we implement a scalable attack pipeline that uses LLMs to: (1) extract identity-relevant features, (2) search for candidate matches via semantic embeddings, and (3) reason over top candidates to verify matches and reduce false positives. Compared to classical deanonymization work (e.g., on the Netflix prize) that required structured data, our approach works directly on raw user content across arbitrary platforms. We construct three datasets with known ground-truth data to evaluate our attacks. The first links Hacker News to LinkedIn profiles, using cross-platform references that appear in the profiles. Our second dataset matches users across Reddit movie discussion communities; and the third splits a single user's Reddit history in time to create two pseudonymous profiles to be matched. In each setting, LLM-based methods substantially outperform classical baselines, achieving up to 68% recall at 90% precision compared to near 0% for the best non-LLM method. Our results show that the practical obscurity protecting pseudonymous users online no longer holds and that threat models for online privacy need to be reconsidered.