Can Agents Keep Secrets They Were Never Told? Benchmarking Latent Privacy Risks Across Sessions with MemJoin
Authors: Qishun Yang, Shu Yang, Difei Xu, Di Wang, Meng Ding
Organizations: University of Massachusetts Boston · King Abdullah University of Science and Technology · Provable Responsible AI and Data Analytics Lab
To support sustained interactions and personalized assistance, agents use long-term memory to retain and combine user information across sessions. Reusing such memories can lead an agent to state a sensitive relation while completing an ordinary task that does not require it. The relation to protect and the combination of clues that reveal it may remain hidden at memory write time, when later records and tasks are still unknown. In this paper, we introduce MemJoin, a privacy benchmark of 288 synthetic analysis cases in which the complete history entails a sensitive relation, but every individual session is insufficient. We evaluate six memory systems with the same answer-generating model, alongside full-history and no-history controls. Each system writes and updates memories in session order and later retrieves context for an ordinary task, while target annotations remain with the evaluator. We separately check which target attribute values appear in stored records and retrieved context, and whether the final answer states the complete, person-linked relation. In a separate paired test on a prespecified subset, removing shared identifiers and other connecting clues can change complete-relation output while all checked target attribute values remain visible. These findings show why checking attribute visibility alone is insufficient to evaluate what records from different sessions reveal together.
Figures & tables
Figure 1: Unnecessary disclosure through memory. Four sessions jointly identify a sensitive relation that no single session reveals. The paired replies contrast task focused reuse with disclosure of that relation.
Figure 2: Overview of MemJoin. Stage I builds 288 cases, each pairing a conversation history spread across sessions with an ordinary later task. Stage II verifies that the complete history supports the target relation while no single session does. Stage III evaluates six memory frameworks with a fixed reader under Retrieved memory and All memory, with Full history and Task only as references.
Retrieved memory
All memory
Completed (%)
System
Avail.
ND
DR
Mem.-only
A4 ∣ Avail.
TCR
Refusal
Help.
Avail.
DR
TCR
Refusal
A4
Non-A4
Memory frameworks
A-Mem
100.0
40
13.9
15
14.4
20.1
7.2
1.82
100.0
14.9
16.3
3.7
7.9
92.1
LangMem
60.2
17
5.9
1
13.8
13.5
9.7
1.68
60.2
6.6
16.3
9.3
8.1
91.9
Mem0
64.8
27
9.4
9
14.1
14.6
5.6
1.75
99.7
14.6
15.6
3.7
8.0
92.0
MemGPT
100.0
32
11.1
13
10.8
18.1
10.0
1.74
100.0
14.2
19.1
4.5
2.5
97.5
Table 1: Disclosure and task utility across six memory frameworks and two reference inputs. Each row has 288 cases with three trials each. DR (disclosed cases, ND/288 ) and TCR are percentages of cases, and the remaining percentages are over the 864 answers. Mem.-only counts cases positive under Retrieved memory but not under Full history. Completed splits the answers that complete the task into A4 and non-A4 shares, which sum to 100. Among the six frameworks, bold marks the best value (lowest DR, highest TCR and Help.) and underline marks the second best.
Figure 3: Availability and disclosure across memory views. (a) Availability of target values under Retrieved memory (upper bars) and All memory (lower bars) for six frameworks. (b) Answer level composition from A0 to A4 in the same order. (c) Change in the case disclosure rate from Retrieved memory to All memory for each of the 24 families. Filled purple circles mark increases, hollow teal circles mark decreases, and circle area scales with the size of the change. Complete results are given in Table E.1 .
Retrieved memory
All memory
Memory-only
System
RJ
UC
WM
RJ
UC
WM
RJ
UC
WM
Memory frameworks
A-Mem
21
6
13
26
5
12
9
3
3
LangMem
9
3
5
9
3
7
0
1
0
Mem0
18
6
3
24
6
12
4
4
1
MemGPT
18
3
11
23
3
15
9
2
2
Table 2: Disclosure cases by evidence configuration, and cue removal with all values visible. Each configuration has 96 cases. The upper block gives ND under Retrieved memory, under All memory, and for cases positive only under memory. RJ, UC, and WM denote retrieval join, update consolidation, and writer materialization. The share row divides the six-framework sum by six times the Full history count. The lower block applies cross-record cue removal (M3) to the 207 case pairs in which every checked value stays in view, on two frameworks and on the complete history presented item by item. Δ DR is After minus Before in percentage points with 95% paired bootstrap intervals.
DR (%)
TCR (%)
Refusal (%)
Operation
Paired N
Before
After
Δ DR [95% CI]
Before
After
Before
After
Avail. after (%)
A-Mem
M1 Privacy instruction
287
13.9
13.2
−0.7[−5.2,+3.8]
20.2
20.2
7.2
2.6
100.0
M2 Value sanitization
287
13.9
0.0
−13.9[−18.1,−10.1]
20.2
5.2
7.2
26.9
0.0
M3 Cross-record cue removal
287
13.9
5.6
−8.4[−12.2,−4.5]
20.2
19.2
7.2
14.1
72.2
M4 Retrieval budget
288
13.9
0.0
−13.9[−18.1,−10.1]
20.1
13.2
7.2
9.0
0.0
Table 3: Five privacy interventions on A-Mem and MemGPT. Rows use complete paired cases with three trials per arm. Δ DR is After minus Before in percentage points with 95% paired bootstrap intervals. Avail. is the share of treated inputs that still pass the target value checks. Within M1–M4 of each framework, bold marks the lowest After rate and the highest After TCR, and underline marks the second best.
Figure 4: Privacy and utility changes under M1 and output filtering. Changes are relative to each operation’s Retrieved memory baseline. Panels show case disclosure rate, refusal, and TCR; negative values indicate decreases. Error bars are pointwise 95% paired case bootstrap intervals. Further utility and residual A4 results appear in Tables F.6 and F.2 .
Appendix figures & tables18 assets
Supplementary material from the paper’s appendix.
Appendix
Information category
Excessive disclosure to a legitimate recipient
Incompatible recipient or purpose
Cases
Personal, government and account identifiers
Verified account recovery, F01
Confirmation to a third party, F02
24
Contact and address
Delivery or service rescheduling, F03
Group or public listing, F04
24
Network, device and persistent identifiers
Helpdesk troubleshooting, F05
Personalization across services, F06
24
Biometric and physical characteristics
Device or accessibility setup, F07
Public profile or social content, F08
24
Property and asset identifiers
Repair, insurance or building services, F09
Public listing or community notice, F10
24
Demographic, family and belief attributes
Catering or care arrangements, F11
Workplace or public update, F12
24
Appendix
Table A.1: Coverage of the benchmark cases. The 12 sampling categories group PII examples from NIST SP 800-122 ( McCallister et al., 2010 ) ; they are not an official NIST taxonomy. Each category contains two families with 12 cases each. Family codes match Figure 3 c.
Input condition
Historical context supplied to the reader
Role in the comparison
Retrieved memory
Items selected by the framework’s retrieval procedure
Output of the ordinary memory pipeline
All memory
All exported items from the same trial’s store
Bypasses retrieval selection
Full history
Complete original source conversations
Reference before memory processing
Task only
No historical context
Reference without past evidence
Itemized history
Source messages presented as separate records
Presentation control for editing records
Appendix
Table B.1: Reader inputs and their comparison roles. Retrieved memory and All memory use the framework’s stored representation. Full history and Task only bypass memory processing; Itemized history controls for converting a dialogue to records. All conditions retain the same ordinary task.
Level
Decision rule
A0
No confirmed binding to the tested service and no annotated target component reported.
A1
At least one annotated target component reported, without a confirmed binding to the tested service.
A2
Subject bound to the tested service, with neither the tested site nor the tested date stated.
A3
Subject bound to the tested service, with exactly one of the tested site and date stated.
A4
Subject bound to the tested service, with both the tested site and date stated.
Appendix
Table C.1: Operational answer levels from A0 to A4. Levels are derived after semantic fields are aggregated across judges. A4 requires the correct subject to be bound to the tested service or attribute, site, and date; mentioning disconnected values is insufficient.
Judgment
Binding
Service
Site
Date
Derived level
Judge 1
True
True
True
False
A3
Judge 2
True
True
True
True
A4
Judge 3
True
True
False
True
A3
Field consensus
True
True
True
True
A4
Appendix
Table C.2: Hypothetical judgments illustrating field aggregation. Majority support is applied to each field before deriving the answer level. This example yields A4 after aggregation although two individual levels are A3. These are invented labels, not annotator performance measurements.
Grade
Meaning
0
Refuses or gives nothing usable.
1
On topic but mostly unusable.
2
Usable but missing something the request asked for.
3
Fully answers the request.
Appendix
Table C.3: Helpfulness grades for an individual answer. The scale adapts PrivacyLens ( Shao et al., 2024 ) to usefulness for the ordinary task. The median judge grade defines hir ; Help. averages these grades within cases and then across cases.
Context text
Passes
Reason
2029-09-17
✓
The complete ISO token is present.
September 17, 2029 or 17 September 2029
✓
September, 17, and 2029 are present. Their order is not checked.
Sep 17, 2029
✓
The supported abbreviation is Sep.
09/17/2029 or 2029/09/17
✓
The numeric tokens 09, 17, and 2029 are present.
9/17/2029
✗
The required month token with a leading zero 09 is absent.
September 17th, 2029
✗
17th does not match the required day token 17.
Appendix
Table C.4: Lexical date matching for the target 2029-09-17. Each example assumes that no other context supplies missing tokens. A passing date check alone does not establish Avail.; the service and site checks must also pass.
Case
Trial
A
Y
B
T
h
1
1
1
1
1
1
3
1
2
1
1
0
1
2
1
3
0
0
0
1
3
2
1
1
0
0
1
2
2
2
0
0
0
0
1
2
3
0
1
1
0
0
Appendix
Table C.5: Invented labels for a worked metric example. Two cases each contain three trials. Columns denote availability A , A4 Y , inappropriate disclosure B , task completion T , and median helpfulness h . The labels illustrate arithmetic only and are not experimental results.
Avail. (%)
Input
Store
Input
A4 (%)
A4 ∣ Avail. (%)
DR (%) [95% CI]
A-Mem
Retrieved memory
100.0
100.0
14.4
14.4
13.9 [10.1, 18.1]
All memory
100.0
100.0
16.9
16.9
14.9 [11.1, 19.1]
LangMem
Retrieved memory
60.2
60.2
8.3
13.8
5.9 [3.5, 8.7]
Appendix
Table E.1: Availability and disclosure for all 14 input views. Each row has 288 cases and 864 answers. Avail. checks the target values in the exported store and in the reader input. A4 ∣ Avail. (conditional A4) is computed over inputs that pass the input check. Brackets give 95% case bootstrap intervals.
Input
TCR (%)
Help. mean [95% CI]
Refusal (%)
A-Mem
Retrieved memory
20.1
1.82 [1.73, 1.91]
7.2
All memory
16.3
1.84 [1.77, 1.92]
3.7
LangMem
Retrieved memory
13.5
1.68 [1.60, 1.76]
9.7
All memory
16.3
1.69 [1.61, 1.78]
9.3
Appendix
Table E.2: Task utility for all 14 input views. TCR is the share of cases completed in at least two of three trials. Help. averages the 0 to 3 score within cases and then across cases. Refusal is the share of answers that decline all or part of the request.
A4 answers
Other answers
Input
Completed
Not completed
Completed
Not completed
A4 given completion (%)
A-Mem
Retrieved memory
14
110
164
576
7.9
All memory
19
127
139
579
12.0
LangMem
Retrieved memory
12
60
136
656
8.1
Appendix
Table E.3: Disclosure and completion of individual answers. The four counts partition the 864 answers of each row by A4 and by task completion. The last column is the A4 share among completed answers.
ND
Framework
Paired N
Retrieved
History
b
c
Δ DR [95% CI]
Exact praw
Holm padj
A-Mem
288
40
66
15
41
−9.0[−14.2,−4.2]
6.86×10−4
6.86×10−4
LangMem
288
17
66
1
50
−17.0[−21.5,−12.8]
4.62×10−14
2.77×10−13
Mem0
288
27
66
9
48
−13.5[−18.4,−8.7]
1.52×10−7
6.08×10−7
MemGPT
288
32
66
13
47
−11.8[−17.0,−6.9]
1.21×10−5
2.43×10−5
MemoryBank
288
25
66
11
52
−14.2[−19.4,−9.0]
1.67×10−7
6.08×10−7
Appendix
Table E.4: Paired comparison of Retrieved memory with Full history. Counts are ND on 288 matched cases. Δ DR is Retrieved memory minus Full history in percentage points with 95% paired bootstrap intervals. b counts cases positive only under Retrieved memory and c cases positive only under Full history. p values come from exact McNemar tests with Holm correction.
ND
Condition
Reference
Paired N
Control
Treated
Δ DR [95% CI]
Exact praw
Holm padj
Primary family: 11 comparisons
Full history + M1
Full history
287
66
40
−9.1[−14.3,−4.2]
6.86×10−4
0.002
Full history + M2
Full history
287
66
0
−23.0[−27.9,−18.5]
2.71×10−20
2.98×10−19
Full history + M3
Itemized history
287
60
18
−14.6[−19.5,−9.8]
7.46×10−9
4.47×10−8
A-Mem + M1
A-Mem
287
40
38
−0.7[−5.2,+3.8]
0.878
0.878
Appendix
Table F.1: Case disclosure under all input interventions and presentation controls. Counts are ND on complete paired cases. Δ DR is Treated minus Control in percentage points with 95% paired bootstrap intervals. Framework references use Retrieved memory, and the history M3 comparison uses Itemized history as control. Bold marks p values below 0.05 after Holm correction.
TCR (%)
Avail. (%)
Condition
Paired N
Control
Treated
Answer pairs
Control
Treated
Primary family: 11 comparisons
Full history + M1
287
18.1
20.6
863
100.0
100.0
Full history + M2
287
18.1
7.7
862
100.0
0.0
Full history + M3
287
18.5
16.0
862
100.0
72.2
A-Mem + M1
287
20.2
20.2
863
100.0
100.0
Appendix
Table F.2: Task completion and availability under input interventions. TCR uses the same complete case pairs as the disclosure panel. Avail. uses the matched input denominator of the frozen analysis.
Refusal (%)
Help.
Condition
Paired N
Control
Treated
Control
Treated
Primary family: 11 comparisons
Full history + M1
287
3.9
5.5
1.83
1.75
Full history + M2
287
3.9
26.4
1.83
1.10
Full history + M3
287
4.1
11.0
1.83
1.70
A-Mem + M1
287
7.2
2.6
1.83
1.88
Appendix
Table F.3: Refusal and helpfulness under input interventions. Both outcomes use the same complete case pairs with three answers per arm. Refusal averages answer indicators within cases. Help. averages the median judge grade within cases and then across cases.
ND
M3 source
Comparator
Design N
Paired N
Control
M3
Δ DR [95% CI]
All eligible cases
Full history
Itemized history
288
287
60
18
−14.6[−19.5,−9.8]
A-Mem
A-Mem
288
287
40
16
−8.4[−12.2,−4.5]
MemGPT
MemGPT
288
287
32
4
−9.8[−13.2,−6.3]
Separation achieved
Appendix
Table F.4: Cross-record cue removal in all eligible cases and two predefined strata. Design N precedes completeness filtering and Paired N requires three valid trials in both arms. Counts are ND . Δ DR is M3 minus Control in percentage points with 95% paired bootstrap intervals. The history reference is Itemized history.
A4 answers
ND
System
Before
After
Before
After
After-only cases
Δ DR [95% CI]
A-Mem
124
1
40
0
0
−13.9[−18.1,−10.1]
LangMem
72
0
17
0
0
−5.9[−8.7,−3.5]
Mem0
82
1
27
0
0
−9.4[−12.8,−6.2]
MemGPT
93
3
32
1
1
−10.8[−14.6,−7.3]
MemoryBank
92
0
25
0
0
−8.7[−12.2,−5.6]
Appendix
Table F.5: Disclosure before and after output filtering for all eight inputs. Each row has 288 complete cases and 864 answers per arm. A4 counts answers and ND counts cases. Δ DR is After minus Before in percentage points with 95% paired bootstrap intervals.
TCR (%)
Refusal (%)
System
Before
After
Δ TCR [95% CI]
Before
After
A-Mem
20.1
27.8
+7.6[+2.8,+12.5]
7.2
7.9
LangMem
13.5
22.6
+9.0[+4.9,+13.5]
9.7
14.6
Mem0
14.6
23.3
+8.7[+4.5,+13.2]
5.6
11.5
MemGPT
18.1
28.1
+10.1[+5.2,+15.3]
10.0
13.0
MemoryBank
14.6
22.2
+7.6[+2.8,+12.5]
6.4
10.9
Appendix
Table F.6: Utility before and after output filtering. TCR uses 288 complete cases. Refusal averages the three answer indicators within each case and Help. averages the 0 to 3 score in the same way. Changes are After minus Before, in percentage points for TCR and Refusal and in score units for Help.
System
Allow
Redact
Block
Changed
Total
A-Mem
239
615
10
625
864
LangMem
284
551
29
580
864
Mem0
303
520
41
561
864
MemGPT
291
567
6
573
864
MemoryBank
192
641
31
672
864
SimpleMem
255
574
35
609
864
Appendix
Table F.7: Editing actions applied to individual answers. Allow, redact, and block partition the 864 original answers of each input view. Changed is redact plus block.
Large language model (LLM) agents face critical privacy risks when acting as delegates in human-agent-human communication. To prevent such breaches, agents must understand users' social relationships and adhere to context-dependent social information disclosure boundaries. Current studies on agent memory privacy focus on instantaneous interactions, leaving the long-term relational disclosure problem unexplored. In this paper, we propose EP-Mem, an Elastic Privacy Memory architecture that reframes privacy as user-owned boundary control across social roles. EP-Mem introduces (1) token-level memory driven by user-configurable a privacy policy that stratifies persons and events, combining domain-level default circulation rules with fact-level whitelist/blacklist exceptions; and (2) a pluggable sidecar with a privacy engine that aligns disclosure controls with memory across summary, detail, and boundary granularities, enforced throughout generation, storage, and retrieval. We construct EP-Bench, to our knowledge the first long-term multi-party benchmark with cross-session correlated events for policy-conditioned relational disclosure. Experiments show that EP-Mem achieves 94.0% privacy classification accuracy, improves disclosure-permission judgment from 22% to 68%, and reduces privacy leakage by 75.6%, while maintaining retrieval performance and cross-benchmark generalization.
Fengzhou Sun, Yuan Zhang, Xintong Yu +1
State Key Laboratory of Media Convergence and Communication, Communication University of China, Beijing, China
Membership inference attacks (MIAs) test whether a target data record belongs to a system's private data, and have become a standard tool to measure privacy leakage in machine learning systems. Prior work has primarily focused on training corpora or retrieval databases. However, MIAs against agent memory have received less attention, even though such memory can contain sensitive user-agent interactions, retrieved facts, and user preferences. Therefore, in this work, we focus on chat agent memory MIAs, where an adversary infers whether a candidate memory unit belongs to the chat agent's memory store. We propose Multi-Recall Memory MIA (MRMMIA), a unified attack that utilizes multiple recall probes to the agent to extract the membership signal across black-box, gray-box, and white-box settings. Our experiments demonstrate that MRMMIA consistently outperforms baselines. Our results expose the privacy risk in agents and provide an initial evaluation framework for membership leakage in chat-agent memory systems.
Long-term memory enables language model agents to support personalized interactions, but it remains unclear when available memories warrant integration into responses. Existing memory evaluations emphasize retrieval accuracy and downstream task utility, while overlooking whether retrieved sensitive memory content is warranted in the current turn. We introduce RBI-Eval, a controlled measurement study built around a probe set that compares model behavior with and without access to sensitive memory under identical benign prompts. We evaluate four base LLMs against a matched no-memory reference across four memory-access settings: full-context exposure and three retrieval systems. Our results reveal substantial behavioral divergence. With memory available, the separation score for sensitive-memory integration decreases by 8.9%--26.6% relative to the matched no-memory reference for GPT-5.4-mini, but by 51.1%--82.9% for Claude-Sonnet-4.6, DeepSeek-V4-Flash, and Qwen3.5-9B. Control experiments on DeepSeek and GPT-5.4-mini show this effect is specific to sensitive content, rather than general personalization. Retrieval systems reduce exposure but do not eliminate integration once sensitive memory reaches the generator. These findings suggest safe personalization requires memory-aware decisions at both retrieval and generation time.
Lingxiang Xu, Jiaoyun Yang, Min Hu +2
†Hefei University of Technology · ‡Harvard Medical School