SNEAK: Evaluating Strategic Communication and Information Leakage in Large Language Models

Adar Avsian; Larry Heck

cs.CLMar 31, 2026

SNEAK: Evaluating Strategic Communication and Information Leakage in Large Language Models

Authors: Adar Avsian, Larry Heck

Organizations: Georgia Institute of Technology

Abstract

Large language models (LLMs) are increasingly deployed in multi-agent settings where communication must balance informativeness and secrecy. In such settings, an agent may need to signal information to collaborators while preventing an adversary from inferring sensitive details. However, existing LLM benchmarks primarily evaluate capabilities such as reasoning, factual knowledge, or instruction following, and do not directly measure strategic communication under asymmetric information. We introduce SNEAK (Secret-aware Natural language Evaluation for Adversarial Knowledge), a benchmark for evaluating selective information sharing in language models. In SNEAK, a model is given a semantic category, a candidate set of words, and a secret word, and must generate a message that indicates knowledge of the secret without revealing it too clearly. We evaluate generated messages using two simulated agents with different information states: an ally, who knows the secret and must identify the intended message, and a chameleon, who does not know the secret and attempts to infer it from the message. This yields two complementary metrics: utility, measuring how well the message communicates to collaborators, and leakage, measuring how much information it reveals to an adversary. Using this framework, we analyze the trade-off between informativeness and secrecy in modern language models and show that strategic communication under asymmetric information remains a challenging capability for current systems. Notably, human participants outperform all evaluated models by a large margin, achieving up to four times higher scores.

Figures & tables

Figure 1: Overview of the SNEAK benchmark. A message-generating language model (left) plays the role of an ally by observing the category, candidate set, and secret word and produces a message. A second ally (center), who knows the secret, hears the message and must determine whether it corresponds to the secret. The chameleon (right), who does not know the secret, hears the message and must infer the secret word from the candidate set.

Figure 2: Utility–leakage trade-off for a single instance (secret: hat). The message rabbit (as in a rabbit in a hat ) achieves high utility while maintaining low leakage.

Instances	Categories	Words per Category	Decoys
1,394	117	10–12	5

Table 1: Summary of the SNEAK dataset. Each instance corresponds to a (category, word set, secret) tuple with associated decoy messages.

Table 4

Figure 3: Utility–leakage trade-off across message-generating models. Each point corresponds to a baseline, with marker size proportional to SoftScore. The dashed curve shows the empirical Pareto frontier. The upper-left region corresponds to desirable behavior: high utility for the ally and low leakage to the chameleon.

Figure 4: Sensitivity of SNEAK performance for Gemma-3-27B to candidate set size ( $|W|$ ) and number of decoy messages ( $|M|$ )

Benchmark	Multi-Agent?	Asymmetric Info?	Strategic Signaling?	Utterance Eval?	Leakage Measured?
MMLU-Pro ( Wang et al., 2024b )	✗	✗	✗	✓	✗
MT-Bench ( Zheng et al., 2023 )	✗	✗	✗	✓	✗
SocialMaze ( Xu et al., 2025 )	✓	✓	✗	✗	✗
Mini-Mafia ( Costa and Vicente, 2025 )	✓	✓	✓	✗	✗
WhoIsSpy ( Chen et al., 2025b )	✓	✓	✓	✗	✗
SNEAK (ours)	✓	✓	✓	✓	✓

Table 4: Comparison of SNEAK with representative LLM benchmarks across key properties.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Source	Utility	Leakage	SoftScore	BinaryScore
Human Participants	91.26	33.81	59.22	54.70

Appendix

Table 5: Performance of human-generated messages under the SNEAK evaluation protocol. Each instance includes three human responses; we report the best-performing response per instance.

Task	Worker Acc.	Plurality Acc.
Ally	77.3%	88.9%
Chameleon	44.6%	68.4%

Appendix

Table 6: Human performance on SNEAK message evaluation tasks. Chance levels are 16.7% for ally and 8.3% for chameleon.

Task	Top-1 Agreement	Strict-Majority Agreement
Ally	88.9%	91.7%
Chameleon	66.7%	74.2%

Appendix

Table 7: Agreement between LLM evaluators and human plurality judgments. Strict-majority considers only items with a clear human majority.

Figure 5: Human annotation interface for message generation. Annotators are given the category, candidate set, and secret word, and are asked to produce a 1-5 word message that signals knowledge of the secret without revealing it directly.

Figure 6: Human annotation interface for the ally evaluation task. Annotators are given the category, candidate word set, secret word, and a set of messages (one gold and several decoys), and are asked to select the message corresponding to the secret.

Figure 7: Human annotation interface for the chameleon evaluation task. Annotators are given the category, candidate word set, and a message, and are asked to select the most likely secret word inferred from the message.

	Utility				Leakage				SoftScore				BinaryScore
Baseline	Mistral	Llama	Qwen	Avg	Mistral	Llama	Qwen	Avg	Mistral	Llama	Qwen	Avg	Mistral	Llama	Qwen	Avg
Random	2.28	6.11	11.61	6.67	8.67	7.04	7.94	7.88	1.86	4.77	9.94	5.52	1.94	4.81	11.98	6.24
Category Synonym	21.61	11.48	15.97	16.35	7.17	8.01	7.51	7.56	19.47	9.85	14.15	14.49	19.80	10.19	16.43	15.47
Secret Synonym	62.22	69.67	75.32	69.07	66.02	74.02	72.88	70.97	11.21	8.60	11.52	10.44	11.12	8.32	11.84	10.43
GPT-5.4	67.87	85.43	88.15	80.48	57.79	76.54	72.43	68.92	23.61	16.44	21.48	20.51	23.74	15.85	20.95	20.18
Llama-4 Scout	85.24	94.44	95.13	91.60	58.81	72.58	69.56	66.98	32.18	23.45	26.77	27.47	31.99	22.53	25.90	26.81

Appendix

Table 8: Decoy ablation results using LLM-generated decoys.

Category	Secret	Candidates	Message	Util.	Leak.	S.S.	B.S.
Academic Subject	Psychology	maths, history, biology, sociology, chemistry, geography, french, physics, english literature, spanish, english, psychology	Mind	100	97.5	2.5	0.0
Animal	Dog	cat, giraffe, elephant, lion, horse, rabbit, cow, pig, tiger, sheep, fish, dog	Fetch	100	99.9	0.1	0.0
Body of Water	Ocean	sea, lake, river, pond, stream, puddle, reservoir, canal, swimming pool, bath, atlantic ocean, ocean	Salty	66.4	33.3	33.1	33.3
Colour	Green	blue, yellow, purple, red, orange, black, pink, white, grey, turquoise, violet, green	Grass	66.5	100	0.0	0.0
Dairy Product	Ice Cream	milk, cheese, yoghurt, butter, cream, chocolate, eggs, sour cream, margarine, whipped cream, cadbury, ice cream	Cone	100	100	0.0	0.0
Emotion	Happy	sad, angry, fear, excited, depressed, anxious, love, frustration, nervous, jealous, upset, happy	Birthday	66.7	33.3	33.3	33.3

Appendix

Table 9: Example messages and their utility–leakage trade-offs. Utility, leakage, SoftScore, and BinaryScore are averaged across evaluator models.

	Utility				Leakage				SoftScore				BinaryScore
Baseline	Mistral	Llama	Qwen	Avg	Mistral	Llama	Qwen	Avg	Mistral	Llama	Qwen	Avg	Mistral	Llama	Qwen	Avg
Random	2.4	10.2	11.0	7.8	8.7	7.0	8.0	7.9	2.0	8.6	9.5	6.7	2.1	9.1	10.8	7.3
Category Synonym	31.4	30.1	33.1	31.5	7.2	8.0	7.5	7.6	28.7	26.5	30.1	28.4	29.3	27.1	33.9	30.1
Secret Synonym	66.6	80.2	80.4	75.7	66.1	74.1	72.9	71.0	12.3	11.6	13.0	12.3	12.1	11.5	12.8	12.1
Human Participants	84.5	97.0	92.2	91.3	28.8	38.5	34.1	33.8	60.0	58.3	59.3	59.2	58.1	51.3	54.7	54.7
Claude Opus 4.6	78.8	94.1	92.4	88.4	57.1	76.6	73.9	69.2	28.0	18.8	19.8	22.2	27.8	17.9	18.5	21.4

Appendix

Table 10: Per-evaluator SNEAK results using Mistral, Llama, and Qwen as evaluators. Metrics are averaged across evaluators.

Figure 8: Comparison of self-evaluation and cross-model evaluation scores for each evaluator model. Bars show the average score assigned when evaluating its own outputs (Self) versus outputs from other models (Others). Differences are small and inconsistent across evaluators and metrics, indicating no systematic same-model bias.

Figure 9: Sensitivity of SNEAK performance to candidate set size ( $|W|$ ) and number of decoy messages. Each row corresponds to a model and each column reports a different metric (utility, leakage, SoftScore, and BinaryScore). Increasing the candidate set size generally reduces leakage, while increasing the number of decoy messages reduces utility, resulting in opposing effects on overall performance. Despite changes in absolute difficulty, qualitative trends remain consistent across models.

Multi-Agent Communication

Multi-agent communication studies how autonomous agents exchange information to coordinate decisions, share knowledge, and act together under constraints.

See all 375 papers

Oct 25, 2026

InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants

Large language model (LLM)-based personal agents are increasingly deployed to handle everyday tasks on users' behalf. These agents have access to private user information, while their tasks involve communicating with third parties; this raises a critical privacy concern that agents may inappropriately reveal sensitive user information to these recipients. Existing benchmarks primarily assess direct privacy leakage, where an agent explicitly states a pre-specified secret or attribute. However, they do not capture inferential leakage, where the model leaves indirect cues in the reply that implicitly leaks sensitive attributes. We introduce InferPriv, a benchmark that measures whether an honest-but-curious recipient can infer a user's sensitive attributes from an LLM's reply on routine task. Evaluating five frontier models, we find that inferential leakage substantially exceeds direct leakage across models and prompts. Under a privacy-aware system prompt, GPT-6.1-Sol, Sonnet-5.5, and Gemini-3.8-Flash achieve near-zero direct leakage, yet the inferential leakage rate remains between 16.2--25.2%. These findings suggest that evaluating explicit leakage alone can substantially understate the privacy risks.

Pengrun Huang, Yiwei Fu, Luoxi Meng +1

UC San Diego · University of Illinois Urbana-Champaign

Privacy Leakage in Language Models Privacy Protection for LLM Agents

Ask PDF

Apr 7, 2026

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

LLM agents increasingly draft messages on behalf of users, yet users routinely overshare sensitive information and disagree on what counts as private. Existing systems support only suppression (omitting sensitive information) and generalization (replacing information with an abstraction), and are typically evaluated on single isolated messages, leaving both the strategy space and evaluation setting incomplete. We formalize privacy-preserving LLM communication as an Information Sufficiency (IS) task, introduce free-text pseudonymization as a third strategy that replaces sensitive attributes with functionally equivalent alternatives, and propose a conversational evaluation protocol that assesses strategies under realistic multi-turn follow-up pressure. Across 792 scenarios spanning three power-relation types (institutional, peer, intimate) and three sensitivity categories (discrimination risk, social cost, boundary), we evaluate seven frontier LLMs on privacy at two granularities, covertness, and utility. Pseudonymization yields the strongest privacy\textendash utility tradeoff overall, and single-message evaluation systematically underestimates leakage, with generalization losing up to 16.3 percentage points of privacy under follow-up.

Yunze Xiao, Wenkai Li, Xiaoyuan Wu +3

Carnegie Mellon University · Massachusetts Institute of Technology · 2077AI +1

Privacy-Preserving Language Models Privacy Protection for LLM Agents Privacy Leakage in Language Models

Ask PDF

May 11, 2026

Can You Keep a Secret? Involuntary Information Leakage in Language Model Writing

Language models are deployed in settings that require compartmentalization: system prompts should not be disclosed, chain-of-thought reasoning is hidden from users, and sensitive data passes through shared contexts. We test whether models can keep prompted information out of their writing. We give each model a secret word with instructions not to reveal it, then ask it to write a story. A second model tries to identify the secret from the story in a binary discrimination test. The secret word never appears literally in any output, but all five frontier models we test leak it thematically -- through topic choice, imagery, and setting--6hy-at rates significantly different from chance, up to 79%. When told to actively hide the secret, models write away from it, and this avoidance is itself detectable. The leakage is cross-model readable, scales sharply with model size within two model families, and disappears entirely for short-form writing like jokes. Giving the model a decoy concept to ``focus on instead'' partially redirects the leakage from the real secret to the decoy. Attending to a secret appears to open up an information channel that frontier LLMs cannot close, even when instructed to.

Ari Holtzman, Peter West

University of Chicago · University of British Columbia

Data Leakage Privacy Leakage in Language Models Language Model Auditing

Ask PDF

Abstract

Multi-Agent Communication

Explore similar work

InferPriv: Evaluating Inferential Privacy Leakage in LLM Assistants

Say Something Else: Rethinking Contextual Privacy as Information Sufficiency

Can You Keep a Secret? Involuntary Information Leakage in Language Model Writing