The Curse of Knowledge in Role-Playing Language Models

Xiao Zhang

cs.CLOct 12, 2026

The Curse of Knowledge in Role-Playing Language Models

Authors: Xiao Zhang

Organizations: University of Groningen

Abstract

Role-playing language models speak as characters in games and agent simulations, where the system records what each character can hear, see and understand. We test whether characters respect these boundaries in 480 scenes, each with a counterfactual twin that changes only the hidden fact, so mentions of the twin's fact measure guessing. Across 11 open-weight models, characters use content from speech they cannot understand or hear at 67 and 72 percentage points (pp) above this baseline, although most models correctly state what the character perceives. This is a curse of knowledge. The leak extends to hidden actions and private messages and changes where characters search. Five of six proprietary models rarely leak, but compliance in strong models depends on input format. Changing only the scene layout or character description raises the rate in two otherwise compliant open-weight models from under 10 to over 60 pp, and combined changes raise two proprietary models from near zero to 13 and 21~pp. Writing the line as the scene's author leaks nearly as much as role-play, and thinking rarely helps. Giving models only what the character observes stops the leak without hiding what it may report.

Figures & tables

Figure 1: Gemma-4-31B speaks as a character who does not understand Spanish (replies abridged, names changed). When version B changes the hidden object and place, the reply changes with them, so the character uses content it cannot access. Adding a flag to the same state saying that the character did not understand the speech stops the leak. Each reply is scored for its own version’s facts and for the other version’s, which gives a baseline for guessing.

Figure 2: Design. (a) Study 1 states the boundary in five ways. The sentence is added to the repertoire for comprehension and to the flag for hearing. (b) Study 2’s four channels, with what the character view keeps (green). (c) Study 3 uses Study 1’s core cells with the model writing as the character, the scene’s author or a narrator, or under a perspective rule, and runs role-play with thinking on.

Figure 3: Study 1, 11 open-weight models. (a) Net use under the five ways of stating the boundary (96 skeletons in three languages). (b) Each model’s state-probe accuracy against its net use. (c) Net use for each of the companion’s questions, with models as dots and the mean as a bar.

Test	Estimate [95% CI]	$p$
Leak, cannot understand	66.7 [64.9, 68.5]	0.004
Leak, cannot hear	72.2 [69.0, 75.5]	0.004
Sentence lowers leak	32.7 [30.8, 34.6]	0.004
Narration lowers leak	25.8 [23.7, 27.8]	0.004
Flag vs. repertoire	21.6 [20.2, 23.0]	0.004
Hear minus understand	27.1 [24.9, 29.2]	0.004

Table 1: The eight tests of Study 1 (pp unless noted). Thresholds are 10 pp for the three net-use tests, zero for the four contrasts, and a 50% reduction with a loss of at most 5 pp in permitted use for the list-first prompt. Values of

p

come from one-sided bootstrap tests with Holm correction over the eight tests, and 0.004 is the smallest value 2,000 draws resolve. The last row is the permitted-use condition of the list-first test.

Figure 4: Leaks across models. (a, b) Net use by model size in two families. (c) Proprietary models on the Study 1 core cells with the content question (288 scenes per cell).

Figure 5: Study 2. (a) Net use of inaccessible content under full context in the confirmation set (96 scenes with invented characters), per model and channel. (b) Change in net use when the access flag is replaced by a membership list or supplemented by a prose rule (24 scenes). (c) Net use after zero, two and four neutral exchanges (four models, 32 scenes). (d) Change in choosing place A when the hidden fact switches from B to A (four models). Bars show 95% bootstrap intervals.

Figure 6: Compliance depends on the input format. (a, b) Adding Study 2’s features to Study 1’s prompts one at a time (four models, 96 scenes). (c) Swapping Study 1’s system prompt, character sheet, state layout and question into Study 2’s hearing prompts. (d) Swapping one feature at a time, on the discovery scenes and on 96 new scenes.

Figure 7: Study 3, framings and thinking (Study 1 core cells). (a) Pooled net use as narrator, as author and with a perspective rule, against role-play, with permitted use at right. (b) Thinking off (circle) and on (dot).

Figure 8: Remedies. (a) The list-first prompt in Study 1. Each point is one model and boundary. (b, c) Five methods in Study 2 (six models, 48 scenes), with one marker per channel.

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Repertoire	Audibility	Compr.	Prose	Permitted	State	Transl.	Bridge
	flag	flag	flag	sentence	use	probe	probe	base $\to$ all
GPT-6 Sol	0.2	0.0	0.9	0.2	100.0	100.0	97.9	0.0 $\to$ 0.0
Claude Opus 5.5	2.1	2.1	5.4	1.2	99.3	100.0	92.7	0.0 $\to$ 1.0
Claude Sonnet 5.5	26.0	8.0	35.8	21.7	99.0	96.5	95.1	0.5 $\to$ 13.0
Gemini 3.1 Pro	1.6	9.0	0.0	0.0	99.7	100.0	97.2	0.0 $\to$ 21.4
Gemini 3.8 Flash	0.2	-0.2	0.0	0.0	99.0	100.0	93.8	0.0 $\to$ 0.0

Appendix

Table 2: Proprietary models and DeepSeek-V4-Pro on Study 1 core conditions (content question, 288 scenes per cell) and on both ends of the bridge. Net use is in pp, and permitted use and probes are percentages.

Model	Repertoire	Audibility flag	Compr. flag	Prose sentence
GPT-6 Sol	0.2 [0.0, 0.5]	0.0 [0.0, 0.0]	0.9 [0.2, 1.9]	0.2 [0.0, 0.5]
Claude Opus 5.5	2.1 [0.5, 4.0]	2.1 [0.9, 3.5]	5.4 [3.5, 7.6]	1.2 [0.0, 3.0]
Claude Sonnet 5.5	26.0 [20.5, 31.6]	8.0 [5.6, 10.6]	35.8 [30.7, 41.1]	21.7 [17.4, 26.4]
Gemini 3.1 Pro	1.6 [0.5, 3.0]	9.0 [5.2, 13.0]	0.0 [0.0, 0.0]	0.0 [0.0, 0.0]
Gemini 3.8 Flash	0.2 [-0.3, 0.9]	-0.2 [-0.5, 0.0]	0.0 [0.0, 0.0]	0.0 [-0.5, 0.5]
Grok 4.7	1.0 [0.3, 1.9]	0.0 [0.0, 0.0]	1.0 [0.3, 1.9]	0.3 [0.0, 0.9]

Appendix

Table 3: Proprietary models and DeepSeek-V4-Pro on the Study 1 core cells, with net use (pp) and 95% intervals from 2,000 skeleton-bootstrap draws (288 scenes, 576 replies per cell). The last row drops the replies that Anthropic’s content filter withheld.

Condition	$n$	Human $D$	Scorer $D$	Precision	Recall
Study 1, list-first answer	14	25.1	25.1	100.0	100.0
Study 1, control, fluent	14	75.5	75.5	100.0	100.0
Study 1, language, prose sentence	10	33.9	33.9	100.0	100.0
Study 1, language, repertoire only	30	66.7	66.7	100.0	100.0
Study 1, not audible, flag only	20	75.0	72.2	100.0	96.3
Study 2, character view, inaccessible	12	0.4	0.5	83.3	100.0

Appendix

Table 4: Human audit, first sample (150 replies from both studies, one annotator who is not an author).

D

is net use (pp) reweighted to each condition. Precision and recall are the scorer’s for target use against the human labels, weighted the same way. Neither the annotator nor the scorer found a foil use.

Channel	Access	$n$	Author $D$	Scorer $D$	Target agreement
Hear	no	80	28.8	30.0	79/80
Hear	yes	80	91.2	91.2	80/80
Understand	no	77	10.4	9.1	76/77
Understand	yes	78	91.0	92.3	77/78
See	no	77	19.5	19.5	77/77
See	yes	80	86.2	86.2	80/80

Appendix

Table 5: Human audit, second sample (640 Study 2 replies, labeled by one of the authors).

D

is net use (pp) over the replies without an unresolved label, access marks whether the character can access the event, and target agreement counts replies where author and scorer agree on target use.

Figure 9: The list-first prompt per model in Study 1 (content and planning questions). (a, b) Net use with the base prompt (gray) and with the list-first prompt. (c) Change in the use of speech made accessible by a working translator, with 95% skeleton-bootstrap intervals and the noninferiority margin of

-5

pp.

Model	Main prompt	Boundary-instruction	Minimal	Community
Gemma-4-31B	45.5 [38.9, 52.2]	23.1 [18.3, 28.4]	50.6 [44.3, 57.2]	51.0 [44.7, 57.4]
Qwen3.8-27B	49.6 [43.7, 54.9]	23.0 [18.2, 27.6]	46.9 [41.5, 52.0]	51.8 [46.9, 56.4]
Qwen3.5-9B	83.9 [80.5, 87.0]	84.7 [81.4, 87.8]	93.1 [90.7, 95.1]	95.6 [94.3, 96.9]

Appendix

Table 6: Net use in Study 1 under the language boundary (repertoire only, pp) under three alternative system prompts. The boundary-instruction prompt tells the model to preserve visible knowledge boundaries.

Subset	Reduction (%)	Transl. (pp)	Holm $p$
Chinese	66.6	-5.1	0.523
Japanese	66.0	-2.8	0.055
Spanish	61.5	2.0	0.004
Without Kerry	62.9	-0.5	0.004
Without Panam	64.1	-0.5	0.004
Without Rogue	67.0	-4.9	0.464

Appendix

Table 7: The list-first test of Study 1 by language, leaving one character out, and with a two-way (skeleton

\times

character) bootstrap. The joint criterion fails in several subsets.

Variant	Not underst.	Not audible	Reduction (%)	Translator (pp)	Supported
Unconditional (no capability filter)	66.5	72.2	64.7	-2.2	8/8
Soft hedge credit	66.7	72.2	64.7	-1.9	8/8
Exclude truncated	66.7	72.2	64.7	-2.0	8/8
Exclude language/echo flags	67.0	72.2	64.8	-2.3	8/8
Complete responses only	66.7	72.2	64.7	-1.9	8/8
Worst-case failed list-first reply	66.7	72.2	64.6	-2.0	8/8

Appendix

Table 8: Study 1 scoring and conditioning sensitivities.

Exploratory contrast	Estimate (pp)	95% CI	BH $q$
Denial while using the content, prose sentence	4.9	[4.3, 5.5]	0.002
Unattributed use, prose sentence minus no language state	7.6	[5.6, 9.6]	0.002
Offline $-$ absent translator	0.1	[-0.5, 0.7]	0.767
Repertoire minus narrated inaudibility (unmatched)	20.3	[19.1, 21.5]	0.002
Few-shot reduction (4 models)	40.4	[37.6, 43.3]	0.002
Direct address $-$ overheard	8.3	[4.9, 11.6]	0.002

Appendix

Table 9: Study 1 exploratory family (Benjamini–Hochberg,

q=.10

). No language state means the state omits the character’s languages. The world comparison uses different scenario sets.

Table 19

Figure 10: Bridge experiment for all 11 open-weight models. Net use of unheard speech (pp) under Study 2’s hearing prompt (Base), with one Study 1 feature swapped in, and with all four. Outlined cells differ from Base with a paired 95% skeleton-bootstrap interval that excludes zero. The last column gives the range of target use for the same speech when audible (%). (a) 96 discovery scenes. (b) 96 confirmation scenes with each character renamed to a Study 1 persona.

Model	Base	Sys.	Sheet	State	Quest.	All
Claude Sonnet 5.5	0.5	9.9	15.1	8.9	9.4	13.0
Gemini 3.1 Pro	0.0	–	0.0	0.0	–	21.4

Appendix

Table 24: Single-feature swaps for two proprietary models on the discovery hearing prompts (192 unheard prompts per cell, net use in pp). Bold marks a change from Base whose paired 95% interval excludes zero. Gemini 3.1 Pro was run on the two swaps that moved the open-weight models most.

Not-audible flag
Model	Study 1 prompt	+ channel	+ question	+ minimal	All three
Gemma-4-31B	52.1	52.1	51.0	54.7	50.0
Qwen3.8-27B	71.4	74.0	80.2	51.6	32.8
Seed-OSS-36B	90.1	88.0	100.0	87.0	83.3
Phi-4	99.0	100.0	94.3	100.0	94.3
Not-understood flag

Appendix

Table 25: Format ablation from Study 1 toward Study 2 (96 scenes, one language per skeleton, content question). Net use (pp) when one Study 2 feature is added (a field naming the channel, the direct question, a minimal context) and with all three.

Large Language Model-Based Role-Play Simulation

Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.

See all 111 papers

CHARM: Character Hallucination for Multicultural Role Play Benchmark
Sep 1, 2026Sunkyung Han, Nahyeon Park, Gaeun Seo +2LLM Evaluation Large Language Model-Based Role-Play Simulation
Sungkyunkwan University, Suwon, South Korea · Adobe Research, CA, USA
When Role-playing, Do Models Believe What They Say?
Jun 9, 2026Benjamin Sturgeon, David Africa, Sid BlackLarge Language Model-Based Role-Play Simulation Personality Modeling in Language Models
MATS
Deception by Omission: Language Models Knowingly Hide Their Mistakes
Oct 8, 2026Lucas Florin, Amelie Knecht, Ulysse Schaller +1Deception in Language Models Language Model Error Detection
AI Safety Research Group, University of Stuttgart, Germany

Sep 1, 2026J/K move · Enter open · S save

CHARM: Character Hallucination for Multicultural Role Play Benchmark

Sunkyung Han, Nahyeon Park, Gaeun Seo, Seunghyun Yoon+1

Sungkyunkwan University, Suwon, South Korea · Adobe Research, CA, USA

LLM Evaluation Large Language Model-Based Role-Play Simulation Hallucination in Language Models Cultural Bias in Language Models LLM Abstention

Open paper AskPDF arXiv

Role-playing large language models (LLMs) are expected to adopt a character's style while also respecting that character's knowledge boundaries. Prior evaluations detect character hallucination but rarely distinguish whether errors arise from failure to recognize a boundary or from failure to comply despite recognition. We introduce CHARM, a multicultural benchmark of 40 real and fictional characters drawn from five cultural-linguistic regions, and validated by native reviewers. It probes two boundary types, Temporal (historical vs. modern) and Cross-Universe (entities outside a character's narrative or historical universe), using abstention-enabled multiple-choice questions. We propose a two-stage evaluation that separates Boundary-Awareness (explicit recognition that a query is out of scope) from Boundary-Compliance (abstention when answering concrete questions). Evaluations across six LLMs show that hallucination is driven predominantly by compliance failures. Models frequently acknowledge that a query lies outside the character's knowledge yet still provide factual, out-of-character answers. By re-posing the same questions to the target character, we confirm that a large fraction of these cases are verified parametric overrides; the model stores the relevant fact but fails to suppress it. We also observe systematic cultural variation in these failures, consistent with imbalances in how characters from different regions are represented in model knowledge.

Abstract

Large Language Model-Based Role-Play Simulation

Explore similar work

CHARM: Character Hallucination for Multicultural Role Play Benchmark

When Role-playing, Do Models Believe What They Say?

Deception by Omission: Language Models Knowingly Hide Their Mistakes