Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs

Yian Wang; Agam Goyal; Eshwar Chandrasekharan; Hari Sundaram

cs.AIAug 29, 2026

Facts Without Rules: Boundary Metadata Collapse in Multi-Agent LLM Handoffs

Authors: Yian Wang, Agam Goyal, Eshwar Chandrasekharan, Hari Sundaram

Organizations: Siebel School of Computing and Data Science University of Illinois Urbana-Champaign Urbana, IL, USA

Abstract

Multi-agent LLM systems often coordinate by compressing an upstream interaction into a handoff artifact that downstream agents treat as shared state. We show that this handoff step is a structural source of privacy leakage: summaries preferentially preserve operational facts while weakening the boundary metadata that governs how those facts may be used---a failure mode we call summary collapse. On a controlled multi-agent coordination testbed we measure marker survival with a human-validated judge ( $κ= 0.74$ ), where $σ_b = 1$ means every boundary marker survives verbatim and $σ_b = 0$ means all are lost. Boundary-marker and operational-fact survival are nearly uncorrelated at the handoff level on both GPT-5-mini and DeepSeek-R1-32B (Pearson $r$ near zero): uncompressed free-text handoffs preserve boundaries at $σ_b \approx 0.80$ , whereas a $25$ -word budget drops $σ_b$ to ${\approx}0.57$ while operational-fact survival stays near ceiling. Controlled downstream tests reveal that protection depends on boundary explicitness: vague languages leak in $73\%$ of GPT and $50\%$ of DeepSeek cases, while explicit constraints reduce leakage to under $15\%$ across all three tested models. A no-handoff single-agent control further shows the failure is not reducible to multi-agent topology as direct full-marker access still leaks more often than the operationalized handoff. Prompt-only mitigation and exact-string redaction only partially address the problem, while a gold-derived audience allowlist nearly eliminates leakage across models, showing that correctly identifying audience boundaries is the key factor.

Figures & tables

Figure 1: Summary collapse. A task-relevant operational fact

f

and the boundary rule

b

that licenses its use both enter the handoff; under compression,

f

survives while

b

is weakened or omitted, so the downstream agent treats

f

as unconstrained. We quantify rule survival as

\sigma_{b}

and fact survival as

\sigma_{\mathrm{op}}

throughout the paper. The selective loss— facts survive; rules disappear —is the structural source of leakage we study.

Sym.	Name	Definition (scale)
$\sigma_{b}$	boundary-marker	mean over upstream boundary markers of a four-level judge
	survival	score: preserved (1.0), paraphrased (0.75), weakened (0.35), absent (0.0).
$\sigma_{\mathrm{op}}$	operational-fact	same score averaged over task-relevant facts: times, entities,
	survival	decisions, owners, constraints.
$\rho$	compression	$\rho=1-\|h\|/\|u\|$ for upstream transcript $u$ and handoff artifact $h$ ;
	ratio	$\rho>0$ compression, $\rho<0$ expansion. token-level

Table 1: Notation.

\sigma=1.0

means every marker survives verbatim;

\sigma=0

means all are lost.

Model	Condition	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
GPT-5-mini	Baseline (E1)	0.954	0.994
GPT-5-mini	40-word	0.919	0.990
GPT-5-mini	25-word	0.690	0.997
DS-R1-32B	Baseline (E1)	0.880	0.980
DS-R1-32B	40-word	0.774	0.979
DS-R1-32B	25-word	0.580	0.972

Table 2: Compression stress: boundary-marker survival drops under hard compression while operational-fact survival remains near ceiling. Qwen3-32B reproduces the same baseline-to-compression pattern in the targeted third-model replication. Bootstrap 95% CIs and the full

\sigma_{b}

-vs-

\rho

trajectory for the two primary models are in Section C.2 . The flagship row is a fixed-snapshot model with no reasoning or schema augmentation, run under matched compression.

Model	Item	pres	para	weak	abs
GPT-5-mini	marker	0.56 [0.43, 0.68]	0.17 [0.08, 0.25]	0.03 [0.00, 0.07]	0.25 [0.15, 0.36]
	operational	0.97 [0.92, 1.00]	0.03 [0.00, 0.08]	0.00 [0.00, 0.00]	0.00 [0.00, 0.00]
DS-R1-32B	marker	0.39 [0.26, 0.51]	0.22 [0.14, 0.32]	0.07 [0.01, 0.12]	0.32 [0.21, 0.43]
	operational	0.92 [0.82, 1.00]	0.05 [0.00, 0.13]	0.03 [0.00, 0.08]	0.00 [0.00, 0.00]
Qwen3-32B	marker	0.42 [0.31, 0.56]	0.18 [0.10, 0.26]	0.01 [0.00, 0.04]	0.39 [0.26, 0.50]
	operational	0.92 [0.84, 1.00]	0.05 [0.00, 0.14]	0.03 [0.00, 0.08]	0.00 [0.00, 0.00]

Table 3: Ordinal label proportions under 25-word compression with 95% bootstrap confidence intervals in brackets. Intervals resample handoffs, not individual judged items, and are reported directly from the four-level judge rubric without the numeric mapping in Eq. 1 . The marker

\to

operational shift in mass toward the preserved bucket is the same selective-fragility signature that the

\sigma

\sigma_{\mathrm{op}}

gap reports in the body.

	GPT-5-mini		DS-R1		Qwen3
Marker level	exact	sem	exact	sem	exact	sem
L0 no marker	44	48	38	47	35	48
L1 vague hedge	35	44	24	36	18	31
L2 soft suggestion	6	10	5	6	2	1
L3 constraint	2	8	2	3	3	4
L4 imperative	2	2	2	1	1	2

Table 4: E8 marker-strength gradient: final leakage out of 48 per cell, under exact/alias matching and the calibrated semantic judge. The sharp L1

\to

L2 transition holds on both detectors and all three models; semantic rates run higher because paraphrase and category-level disclosures count. Endpoint counts in § C.7 and § C.6 .

	GPT-5-mini		DS-R1
Probe	leak	util	leak	util
Prompt-only mitigation	58/288	0.997	42/288	0.986
Exact-string redaction	0/288	0.998	0/288	0.997
Gold typed allowlist	0/48	1.000	0/48	1.000
5% label error	2/48	0.990	3/48	0.979
10% label error	4/48	0.948	4/48	0.979

Table 5: RQ3 enforcement probes on the two primary models, pairing exact leakage with downstream task success. Prompt-only mitigation and exact-string redaction leave leakage while task success stays near ceiling, so the failure is not driven by refusal; gold-derived typed boundaries suppress leakage at full utility, and graded label error degrades protection smoothly. Task success is not defined for the negation-flip stress test, which deliberately corrupts the boundary field rather than varying a deployment parameter. Qwen3-32B is run only for the corrupted-allowlist test ( Section C.14 ); the manual high-risk audit of E7 outputs is discussed in the text. Full curve with scenario-bootstrap intervals in Section C.13 .

Appendix figures & tables29 assets

Supplementary material from the paper’s appendix.

Appendix

Domain	N	Ops	Words	Markers (a / c / h / o)
customer support	4	4	27	8 (3 / 3 / 1 / 1)
enterprise IT	6	7	32	12 (3 / 5 / 3 / 1)
HR	6	6	29	12 (5 / 3 / 3 / 1)
legal compliance	4	4	28	8 (1 / 3 / 2 / 2)
medical admin	4	4	26	8 (3 / 3 / 1 / 1)
project management	6	6	26	12 (3 / 3 / 3 / 3)

Appendix

Table 6: Per-domain manifest of the 36-scenario Bound-Handoff benchmark. “Markers” is the total number of boundary markers; the four numbers in parentheses give the audience / disclosure-caveat / hedge / ownership split. “Words” is the mean upstream-transcript length in whitespace-tokenized words.

Experiment	Design	Prompts/model
E1 Baseline handoff	stratified subset of $36\times 5\times 4$	180
E2 Compression stress	$36\times 2$ hard-budget handoffs	72
E3 Context dump	$12\times 3\times 4$	144
E4 Weak markers	$12\times 4\times 4$	192
E5 No-handoff control	$12\times 3\times 4$	144
E6 Prompt-only mitigation	$72$ handoffs $\times 4$ pressures	288

Appendix

Table 7: Prompt counts per experiment per model. E1 samples a stratified subset from the full handoff design space. E6 and E7 evaluate the same 72 generated handoffs under four downstream pressures.

	GPT-5-mini		DS-R1
Condition	$\sigma_{b}$	$\sigma_{\mathrm{op}}$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
free_text	0.976	1.000	0.781	0.993
compressed_free_text	0.916	0.990	0.885	0.962
preserve_markers_instruction	0.959	0.993	0.949	0.993
sectioned_template	0.984	1.000	0.949	0.991
structured_schema	0.938	0.990	0.834	0.963

Appendix

Table 8: E1 baseline handoff survival by condition on two primary models. Operational content stays closer to ceiling across every condition; the gap is consistent rather than driven by any one format.

Surface	Marker items	$\sigma_{b}$	Op items	$\sigma_{\mathrm{op}}$
explicit_summary	110	0.944	55	0.995
forwarding_to_summary	80	0.961	40	0.988
memory_replay	80	0.979	40	1.000
report_writer	90	0.940	50	0.994

Appendix

Table 9: E1 baseline survival by handoff surface on GPT-5-mini.

Surface	Marker items	$\sigma_{b}$	Op items	$\sigma_{\mathrm{op}}$
explicit_summary	110	0.880	55	0.986
forwarding_to_summary	80	0.896	40	0.975
memory_replay	80	0.912	40	0.988
report_writer	90	0.836	50	0.972

Appendix

Table 10: E1 baseline survival by handoff surface on DeepSeek-R1-32B.

Marker category	Items	$\sigma_{b}$ GPT	$\sigma_{b}$ DeepSeek
audience	120	0.951	0.907
caveat	120	0.977	0.888
hedge	70	0.943	0.836
ownership	50	0.925	0.855

Appendix

Table 11: E1 baseline boundary survival by marker category on two primary models.

	GPT-5-mini		DeepSeek-R1-32B
Domain	$\sigma_{b}$	$\sigma_{\mathrm{op}}$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
customer support	0.953	1.000	0.831	0.988
enterprise IT	0.939	0.992	0.896	0.958
HR	0.983	1.000	0.939	1.000
legal compliance	0.953	0.988	0.824	1.000
medical admin	0.963	1.000	0.868	0.988

Appendix

Table 12: E1 baseline survival by domain on two primary models.

Condition	$\rho$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
free_text	-1.096	0.939	1.000
compressed_free_text	-0.029	0.894	0.997
preserve_markers_instruction	-2.105	0.956	0.993
sectioned_template	-1.627	0.941	0.983
structured_schema	-3.039	0.847	0.958
Overall	-1.579	0.915	0.986

Appendix

Table 13: E1 baseline handoff survival on Qwen3-32B.

Condition	$\rho$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
structured_schema	-7.536	0.938	0.990
sectioned_template	-3.103	0.984	1.000
free_text	-1.989	0.976	1.000
preserve_markers_instruction	-1.006	0.959	0.993
compressed_free_text	-0.362	0.916	0.990
hard_budget_compressed	0.077	0.919	0.990

Appendix

Table 14: Full

\sigma_{b}

-vs-

\rho

trajectory on GPT.

Condition	$\rho$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
structured_schema	-3.992	0.834	0.963
sectioned_template	-1.477	0.949	0.991
preserve_markers_instruction	-1.088	0.949	0.993
free_text	-0.631	0.781	0.993
compressed_free_text	-0.002	0.885	0.962
hard_budget_compressed	0.379	0.774	0.979

Appendix

Table 15: Full

\sigma_{b}

-vs-

\rho

trajectory on DeepSeek-R1-32B.

Surface	gpt-5-mini		deepseek-r1:32b
	$\sigma_{b}$	$\sigma_{\mathrm{op}}$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
explicit_summary	0.516	1.000	0.652	0.918
forwarding_to_summary	0.859	1.000	0.731	1.000
memory_replay	0.694	1.000	0.366	1.000
report_writer	0.750	0.986	0.547	0.986

Appendix

Table 16: Survival by handoff surface under 25-word compression on two primary models.

Marker category	gpt-5-mini	deepseek-r1:32b
audience	0.806	0.815
caveat	0.865	0.675
hedge	0.482	0.375
ownership	0.285	0.075

Appendix

Table 17: Marker-category survival under 25-word compression.

Domain	gpt-5-mini	deepseek-r1:32b
customer support	0.763	0.575
enterprise IT	0.667	0.583
HR	0.875	0.696
legal compliance	0.563	0.231
medical admin	0.844	0.638
project management	0.425	0.417

Appendix

Table 18: Boundary survival by domain under 25-word compression.

Contrast	Model	RR [95% CI]	Fisher $p$
E3 +mark vs. no-mark	GPT-5	$0.042$ $[0.011,0.162]$	$1.6\mathrm{e}{-}12$
E3 +mark vs. no-mark	DS-R1	$0.027$ $[0.004,0.189]$	$2.0\mathrm{e}{-}13$
E8 L1 vs. L2	GPT-5	$5.83$ $[2.71,12.58]$	$2.1\mathrm{e}{-}9$
E8 L1 vs. L2	DS-R1	$4.80$ $[2.00,11.53]$	$4.0\mathrm{e}{-}5$
E9 allowlist vs. E6	GPT-5	$0.050$ $[0.003,0.80]^{\dagger}$	$1.0\mathrm{e}{-}4$
E9 allowlist vs. E6	DS-R1	$0.069$ $[0.004,1.11]^{\dagger}$	$1.6\mathrm{e}{-}3$

Appendix

Table 19: Fisher exact tests with risk ratios for the main leakage contrasts. † Haldane corrected for the

0/48

E9 cells; the DS-R1-32B RR upper bound just exceeds

1

because of the zero-cell correction.

Scheme	Model	$\sigma_{b}$	$\sigma_{\mathrm{op}}$	Gap
original	GPT-5-mini	0.690	0.997	$-0.306$
original	DS-R1-32B	0.580	0.972	$-0.392$
binary strict	GPT-5-mini	0.722	1.000	$-0.278$
binary strict	DS-R1-32B	0.611	0.972	$-0.361$
binary lenient	GPT-5-mini	0.750	1.000	$-0.250$
binary lenient	DS-R1-32B	0.681	1.000	$-0.319$

Appendix

Table 20:

\sigma_{b}

weighting robustness under 25-word compression.

		markers ( $n=72$ )				operational ( $n=37$ )
Model	Condition	pres	para	weak	abs	pres	para	weak	abs
GPT-5-mini	free_text	0.90	0.10	0.00	0.00	1.00	0.00	0.00	0.00
GPT-5-mini	compressed_free_text	0.79	0.15	0.03	0.03	0.95	0.05	0.00	0.00
GPT-5-mini	preserve_markers_instruction	0.90	0.06	0.04	0.00	0.97	0.03	0.00	0.00
GPT-5-mini	sectioned_template	0.96	0.03	0.01	0.00	1.00	0.00	0.00	0.00
GPT-5-mini	structured_schema	0.79	0.19	0.00	0.01	0.95	0.05	0.00	0.00

Appendix

Table 21: E1 baseline ordinal label proportions by condition. Proportions are reported directly from the four-level judge rubric without collapsing labels into binary outcomes.

Model	Test	MH OR [95% CI]	$\chi^{2}$	df	$p$
GPT-5-mini	M vs. O (25w)	3.92 [1.39, 11.03]	10.80	1	$0.001$
GPT-5-mini	25w vs. free (M)	6.54 [2.40, 17.78]	20.25	1	$6.8\mathrm{e}{-06}$
DS-R1-32B	M vs. O (25w)	4.36 [1.68, 11.31]	14.69	1	$1.3\mathrm{e}{-04}$
DS-R1-32B	25w vs. free (M)	6.00 [2.40, 15.01]	11.07	1	$8.8\mathrm{e}{-04}$
Qwen3-32B	M vs. O (25w)	4.33 [1.83, 10.25]	17.82	1	$2.4\mathrm{e}{-05}$
Qwen3-32B	25w vs. free (M)	7.22 [3.14, 16.61]	32.67	1	$1.1\mathrm{e}{-08}$

Appendix

Table 22: Scenario-stratified Cochran–Mantel–Haenszel tests for ordinal-label degradation. Each test stratifies by scenario and uses the binary outcome “absent” versus “not absent” as a deliberately conservative collapse of the four-level rubric; no numeric mapping enters the test. The marker-vs.-operational rows compare item kinds under 25-word compression, and the marker-compression rows compare 25-word compression against free_text within marker items.

Model	Variant	neutral	explain reason	audit trace	compressed report
gpt-5-mini	no_dump_marker	0/12	0/12	0/12	0/12
	dump_with_marker	1/12	0/12	1/12	0/12
	dump_no_marker	12/12	12/12	12/12	12/12
deepseek-r1:32b	no_dump_marker	0/12	0/12	0/12	0/12
	dump_with_marker	1/12	0/12	0/12	0/12
	dump_no_marker	7/12	9/12	10/12	11/12

Appendix

Table 23: E3 final leakage by downstream pressure. Cells report leakage / total.

Pressure	gpt-5-mini	deepseek-r1:32b
neutral	10 / 12	8 / 12
explain_reason	11 / 12	12 / 12
audit_trace	12 / 12	9 / 12
compressed_report	11 / 12	9 / 12
Total	44 / 48	38 / 48

Appendix

Table 24: Manual leakage matrix for the dump_weak_marker condition.

Model	Variant	neutral	explain reason	audit trace	compressed report
gpt-5-mini	direct_full_marker	5/12	5/12	6/12	4/12
	direct_weak_marker	7/12	8/12	8/12	8/12
	direct_no_marker	11/12	10/12	11/12	11/12
deepseek-r1:32b	direct_full_marker	3/12	3/12	4/12	1/12
	direct_weak_marker	8/12	10/12	6/12	9/12
	direct_no_marker	9/12	8/12	10/12	9/12

Appendix

Table 25: E5 no-handoff single-agent control by pressure. Cells report exact-phrase leakage / total.

Model	Mitigation condition	neutral	explain reason	audit trace	compressed report
gpt-5-mini	free_text	2/12	2/12	2/12	2/12
	minimality_instruction	1/12	2/12	3/12	2/12
	preserve_markers_instruction	2/12	2/12	3/12	3/12
	sectioned_template	2/12	3/12	4/12	0/12
	structured_schema_safe	1/12	2/12	4/12	4/12
	structured_schema_unsafe	2/12	1/12	5/12	4/12

Appendix

Table 26: E6 final leakage by mitigation condition

\times

downstream pressure. Cells report leakage / 12.

Condition	Rows redacted	Replac.
free_text	6 / 12	6
minimality_instruction	4 / 12	4
preserve_markers_instruction	9 / 12	13
sectioned_template	7 / 12	13
structured_schema_safe	10 / 12	42
structured_schema_unsafe	10 / 12	67

Appendix

Table 27: E7 redaction counts before downstream evaluation.

Slice	GPT-5-mini	DS-R1-32B
Overall high-risk audit	5 / 40	14 / 48
structured_schema_safe	3 / 22	7 / 24
structured_schema_unsafe	2 / 18	7 / 24
audit_trace pressure	4 / 20	9 / 24
compressed_report pressure	1 / 20	5 / 24

Appendix

Table 28: E7 high-risk manual audit on two primary models.

Domain	Manual leaks	Rows
customer support	2	8
enterprise IT	7	8
HR	4	8
legal compliance	0	4
medical admin	0	4
project management	1	8

Appendix

Table 29: DeepSeek E7 high-risk manual audit by domain.

Experiment	GPT-5-mini	DS-R1-32B
E1 Baseline handoff	0.969	0.947
E6 Prompt-only mitigation	0.997	0.986
E7 Enforced redaction	0.998	0.997

Appendix

Table 30: Task success across experiments and models.

Pressure	GPT-5-mini	DS-R1-32B
neutral	0 / 12	0 / 12
explain_reason	0 / 12	0 / 12
audit_trace	0 / 12	0 / 12
compressed_report	0 / 12	0 / 12
Total	0 / 48	0 / 48

Appendix

Table 31: E9 operationalized boundary probe: final leakage by downstream pressure.

Figure 2: Privacy–utility frontier under graded audience-label error. Leak rate and task success at nominal 5/10/20/30% error against the gold-allowlist control, with 95% scenario-bootstrap intervals.

Variant	GPT-5-mini	DeepSeek-R1-32B	Qwen3-32B
clean allowlist	0 / 48 (0.0%)	1 / 48 (2.1%)	4 / 48 (8.3%)
dump-with-marker control	0 / 48 (0.0%)	1 / 48 (2.1%)	1 / 48 (2.1%)
corrupt: scope drift	10 / 48 (20.8%)	4 / 48 (8.3%)	15 / 48 (31.2%)
corrupt: typo allowlist key	23 / 48 (47.9%)	18 / 48 (37.5%)	14 / 48 (29.2%)
corrupt: format noise	40 / 48 (83.3%)	22 / 48 (45.8%)	33 / 48 (68.8%)
corrupt: negation flip	44 / 48 (91.7%)	38 / 48 (79.2%)	35 / 48 (72.9%)

Appendix

Table 32: Corrupted-allowlist stress test for the E9 operationalized-boundary probe. Each cell reports final leakage over 48 downstream prompts (12 scenarios

\times

4 downstream pressures). Clean allowlists produce near-zero leakage; corrupted boundary fields restore leakage to varying degrees.

Condition	$\rho$	$\sigma_{b}$	$\sigma_{\mathrm{op}}$
structured_schema	$-0.30$	0.73	0.98
preserve_markers_instruction	$0.13$	0.67	0.99
sectioned_template	$0.36$	0.79	0.98
free_text	$0.45$	0.47	0.99
compressed_free_text	$0.80$	0.28	0.95

Appendix

Table 33: Real-trace survival on 493 PrivacyLens scenarios (2,465 handoffs).

\sigma_{b}

falls by

0.45

across the compression range while

\sigma_{\mathrm{op}}

stays in a

0.94

–

0.99

band. Per-condition

95\%

bootstrap intervals and all per-handoff scores are released with the code.

Privacy Leakage in Language Models

Privacy leakage in language models concerns how models expose training data, infer sensitive traits, or transmit private context through outputs and internal channels.

See all 280 papers

Oct 12, 2026

When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems

Multi-agent LLM systems (MALS) introduce a privacy-leakage problem that conventional sensitivity-based defenses miss, since information disclosed to one authorized agent can propagate through routine coordination to an unauthorized one, without compromise or malicious behavior. This motivates topology-aware enforcement that blocks unauthorized propagation while preserving authorized disclosure. We evaluate 485 scenarios across coordinator-worker, peer-to-peer, and hierarchical topologies, instrumenting seven communication channels and measuring surface leakage and disclosure to topology-introduced recipients. We introduce RECONS, a Detect -> Decide -> Transform sanitizer conditioning intervention on recipient and execution context rather than sensitivity alone. Topology changes where protected information propagates, with disclosure to topology-introduced recipients rising from 0% to 52.2% while surface leakage stays nearly flat at 60.6-62.8%. This effect persists across four generator models and a second orchestration framework, though magnitude and relay paths differ; framework choice separately drives a 150x leakage spread, so topology alone is not sufficient. RECONS reduces residual disclosure from 96.7% without sanitization to 29.25%, preserving 94.1% of authorized disclosures versus 20.5% under blanket redaction. Detection coverage is the dominant bottleneck, accounting for 95.5% of residual disclosures. These results show that privacy enforcement in MALS must reason about the recipient and communication path realized at runtime, not sensitivity or topology assumptions alone.

Sunday Oyinlola Ogundoyin, Muhammad Ikram

Macquarie University Sydney, NSW, Australia

Multi-Agent LLM Systems AI Agent Security Multi-Agent System Security Information Flow Control

Ask PDF

Aug 3, 2026

MNC: Scope-Bound Semantic Declassification for Private LLM-Agent Communication

Multi-agent large language model (LLM) systems can expose protected state through internal messages, tool arguments, logs, and persistent memory even when their public outputs appear innocuous. Existing privacy prompts, redaction methods, and source-level access controls restrict surface content or data access, but do not specify what a legitimately informed agent should disclose or how that disclosure may be reused downstream. We introduce Minimum-Necessary Communication (MNC), a typed semantic-declassification protocol that selects a task-sufficient disclosure from an application-authored candidate family and binds it to explicit recipient, purpose, forwarding, lifetime, logging, and memory scopes. A reference monitor enforces these scopes across subsequent operations, while a history-aware extension accounts for inference risk accumulated over repeated disclosures. Controlled semantic-join, memory, probing, and longitudinal experiments show that conventional defenses can preserve protocol-level utility while exposing substantial additional inference signal. Under identical receipt text, MNC preserves authorized delivery while blocking unauthorized forwarding, logging, durable storage, and retrieval after expiration that a text-only semantic declassifier permits. Two-backbone MAGPIE executions further show that mediated disclosures propagate through subsequent planning, tool use, coordination, and memory retrieval. These results support scope-bound semantic declassification as a practical communication boundary for private LLM-agent systems.

Jinghan Xu, Longze Fan, Zeyuan Wang +2

1Nankai University, China · 2China University of Petroleum, China · 3Sun Yat-sen University, China +1

Multi-Agent LLM Systems LLM Agent Security Information Flow Control Privacy Protection for LLM Agents

Ask PDF

May 26, 2026

Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems

LLM safety evaluations predominantly test models in isolation, yet deployed AI agents increasingly operate within persistent social environments alongside other agents. We introduce a Moltbook-style simulation platform where thousands of LLM agents interact across communities over a simulated month, and use it to evaluate privacy as a downstream safety concern under varying degrees of social pressure. We find that shifting from single turn to multi turn social evaluation amplifies privacy violations (CIMemories 19.95% to Ours 45.30% across OpenAI models), that leakage is socially contagious, with agents 8 times more likely to disclose sensitive information after observing a peer do so, and that explicit privacy instructions reduce but do not eliminate this effect, leaving leakage rates above 37.8% even with safeguards. Our findings suggest that static chat based safety benchmarks systematically underestimate risks in agentic deployment, and that social context alone is sufficient to elicit sensitive disclosures that single turn evaluations would never surface.

Aman Priyanshu, Supriti Vijay, Esha Pahwa

Foundation AI · USA · Corvic AI

AI Agent Safety Benchmarks Multi-Agent LLM Systems LLM Agent Evaluation LLM Agent Security Privacy Leakage in Language Models Data Leakage+2

Ask PDF

Abstract

Privacy Leakage in Language Models

Explore similar work

When Agents Talk, Secrets Travel: Topology-Driven Leakage and Recipient-Conditioned Sanitization in Multi-Agent LLM Systems

MNC: Scope-Bound Semantic Declassification for Private LLM-Agent Communication

Got a Secret? LLM Agents Can't Keep It: Evaluating Privacy in Multi-Agent Systems