ANIMASK: What the Model Contributes to Role Play in Simulated Story Worlds

Xiucheng Zhang; Zhuoning Xu; Hanjun Luo; Yankai Chen; Hanan Salam; Xue Liu

cs.AISep 15, 2026

ANIMASK: What the Model Contributes to Role Play in Simulated Story Worlds

Authors: Xiucheng Zhang, Zhuoning Xu, Hanjun Luo, Yankai Chen, Hanan Salam, Xue Liu

Organizations: New York University · New York University Abu Dhabi · McGill University · MBZUAI

Abstract

When a language model plays a character, the observed behavior reflects both the assigned persona and the default dispositions of the actor model itself. Existing evaluations test persona fidelity or model defaults in isolation, but neither says, at a specific choice with consequences, what the persona changed and what the model's default kept. We introduce ANIMASK, a simulation framework that freezes books and scripts into story worlds whose characters act on their own motivations and replays each story from its freeze point. We hold out the author's continuation as a human reference, verify through in-story interviews that each persona remains present, and at every decision point compare the character's action with what the model produces when the persona is removed. Across 40 stories, 6 actor models, and 3,846 decision points, the replays converge away from their canons in one shared direction, toward flatter, cooler stories that leave their tensions open. The personas stay present and obeyed throughout. On three choices in four the model's default already falls inside what the persona accepts, and where the two diverge the model is the cautious one, holding where the persona would press. The persona guarantees who the character is, and the model sets how far the character will go.

Figures & tables

Figure 1: Overview of ANIMASK. A source frozen mid-story becomes a world, the engine runs it with personas on a shared actor model, and the replay is the text they produce.

	$y_{d}\in I_{d}$	$y_{d}\notin I_{d}$
$y^{0}_{d}=y_{d}$	adhere-but-empty	overridden
$y^{0}_{d}\neq y_{d}$	adhere-and-effective	neither
adherence = left column, efficacy = lower-left cell

Table 1: Decision-point classes by persona adherence and no-persona agreement.

Figure 2: Three examples, one per result. (a) One script under all six actors. (b) A decision point of Sela in the Kimi replay. (c) A decision point of Isolde in the Claude replay.

Figure 3: Story-level convergence. (a) Dispersion of each score across stories at the five scoring checkpoints, the canons against the six actors. (b) Mean per-story deviation of each actor from canon, with the persona-free replays of GPT as a dashed row.

Figure 4: Persona presence over the replay, one line per actor for each presence curve. Dashed, the persona-free reference of GPT, graded against the removed card.

Actor	Adherence	Efficacy	Inside (%)	Persona (%)	Default (%)	Wins (%)	Shift (scale units)	Cross calls (%)	Cross kept (%)	High risk	$\rho$
GPT	0.86	0.17	74	17	6	65	+0.13	11	64	0.79	$-0.22$
Claude	0.83	0.16	73	16	9	59	+0.12	15	79	0.77	$-0.20$
Gemini	0.83	0.07	81	7	11	36	+0.03	25	80	0.81	$+0.09$
DeepSeek	0.86	0.18	74	18	6	68	+0.14	16	84	0.83	$+0.00$
Kimi	0.80	0.16	72	16	11	54	+0.14	19	79	0.78	$-0.15$
Qwen	0.84	0.16	74	16	8	60	+0.23	18	87	0.83	$+0.02$

Table 2: Decision points by actor, on characters that pass the presence criterion. Inside, the share of points whose default action lies in the implied set. Persona and default, the shares of adhere-and-effective and overridden points. Wins, the persona’s share of the disagreements. Shift, the mean scale position of the action minus the default. Cross calls, the share of points whose call is to cross a line, and cross kept, the share of those kept. High risk, adherence on high-risk points.

\rho

, the rank correlation between a character’s traits score and its efficacy.

Call	Share (%)	Kept (%)	Default kept (%)	Lowered (%)
yield	4.5	67	59	0
hold	6.8	69	66	3
press	52.9	86	80	3
oppose	18.3	85	70	8
cross	17.5	80	66	20

Table 3: Decision points by the label the persona calls for, pooled over the six actors.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Field type	Card sections	Content
traits and speech style	speech style, verbal tics, core values, constant traits, self-concept	how the character talks, what it values, how it sees itself
conditional dispositions	behavioral axioms	when this happens the character does that, one domain tag each
relationships	relationships	one entry per cast member, relation labels and a sentence of detail
private knowledge	private knowledge, you do not know	facts only this character holds, and facts withheld from it
ability limits	embodiment, competences you lack, capabilities and limits	the physical envelope and hard skill limits
plot memory, pre-freeze part	turning points so far, memories	carried in the same block, read with the next subsection

Appendix

Table 4: Sections of the persona card. The field types are those of Section 4.1 .

Field	Kept	Rationale
name	yes	addresses the speaker
location	yes	a fact of the world state, not of the character
factual memories	yes	events the character lived through
profile (traits, tendencies)	placeholder	the trait carrier
relation map	no	trait-laden readings of the other characters
motivation, goal seed	no	re-derived at run time from the placeholder

Appendix

Table 5: Fields of the persona card under persona removal. The profile is replaced by the single sentence “You are a character in this story.” Everything downstream reads these fields, so the removal acts at one choke point, and the single-step ablation of the no-persona replays uses the same placeholder.

				Replays and operations
Medium (lang.)	Stories	Cast (med.)	DPs / replay	With personas	Persona-free	Interviewed	No-persona
Novels (en)	20	2	12	120	20	306	1,681
Scripts (zh)	20	2.5	18	120	20	379	2,165
Total	40	–	14	240	40	685	3,846

Appendix

Table 6: Corpus and experimental scale. Cast is the median simulated cast per story and DPs the median labeled decision points per replay. With personas counts one replay per story and actor, persona-free the replays of GPT with the personas removed, interviewed the character-runs interviewed at five checkpoints each, and no-persona the labeled decision points, each of which carries a no-persona replay. Model versions are in Appendix J .

(a) English short stories
ID	Length	Cast	Freeze	DPs
en01	4,891	3	0.58	8
en02	2,960	2	0.58	14
en03	2,631	1	0.67	6
en04	4,170	3	0.56	13
en05	4,022	4	0.57	13.5

Appendix

Table 7: The forty stories. Freeze is the freeze point as a fraction of the text, and DPs the median number of labeled decision points per replay over the six actors. Length counts words for the English stories and characters, whitespace excluded, for the Chinese scripts. Titles and venues are not given.

Label	Definition	Checklist question, in order
cross	force or violence, a credible threat of harm, coercion, theft, deception or concealment of key facts, breaking a rule or the law	1. does it cross a line?
oppose	openly opposes another party, challenges, accuses, refuses, makes a demand, stands its ground, without threatening harm	2. does it openly oppose someone?
yield	gives way, backs down, drops a claim or plan, complies under pressure, accepts the other side’s terms	3. does it give way?
press	presses on by ordinary means, asks, goes, does, proposes, cooperates, discloses, negotiates	4. does it move the goal forward?
hold	waits, watches, keeps to routine, avoids or postpones acting, keeps the current state	5. otherwise

Appendix

Table 8: The five labels of the action-strength scale and the checklist that places an action. The judge asks the questions in order and stops at the first yes.

Judgment	$n$	Human–human	Judge–human
Story questionnaire	600	.58	.76
Interview grade	300	.48	.64
Violation present	300	.44	.72
Knowledge audit	100	.87	.76
Called label	300	.51	.54
Implied set	300	.59	.63

Appendix

Table 9: Human calibration. Graded rows report Krippendorff’s

\alpha

, categorical rows Fleiss’

\kappa

among the annotators and Cohen’s

\kappa

of the judge against the human majority. The listing row reports the mean pairwise overlap of the annotators’ lists and the scanner’s recall of their union, over the 36 sweeps with a nonempty union. The ending row drops three items without a human majority.

Judgment	Human–human	GPT	Claude	Gemini	DeepSeek	Kimi	Qwen
Called label	.51	.59	.49	.53	.55	.54	.51
Implied set	.59	.66	.61	.63	.67	.58	.60
Risk level	.31	.48	.45	.50	.56	.32	.48
Actual label	.51	.72	.72	.68	.65	.72	.74
No-persona label	.43	.68	.91	.47	.61	.73	.61
$n$ per actor	300	50	50	50	50	50	50

Appendix

Table 10: Judge agreement with the human majority by actor model on the decision-point rows of Table 9 . The first column repeats the pooled agreement among the annotators.

	Deviation at last checkpoint		Correlation with canon		Net movement
Score	English (20)	Chinese (20)	first	last	canon	replays
Mood	$-0.8$ to $-1.1$ (10–14)	$-2.7$ to $-3.4$ (17–18)	0.20 to 0.39	0.27 to 0.48	$+1.52$	$-0.28$ to $+0.12$
Plot intensity	$-0.8$ to $-2.1$ (11–16)	$+0.4$ to $+2.4$ (1–6)	0.20 to 0.35	$-0.03$ to 0.14	$-0.47$	$-0.38$ to $+0.40$
Tension progress	$-1.6$ to $-2.7$ (14–19)	$-3.1$ to $-3.7$ (19–20)	0.10 to 0.34	$-0.35$ to 0.05	$+3.05$	$+0.38$ to $+0.93$
Relationship warmth	$-0.7$ to $-1.4$ (9–12)	$-3.3$ to $-4.2$ (18)	0.45 to 0.52	$-0.08$ to 0.10	$+1.98$	$-0.10$ to $+0.25$

Appendix

Table 11: Story-level detail, ranges over the six actors. Deviation is the last-checkpoint replay score minus canon, with the number of stories of twenty in which it is negative. Correlation is the Spearman rank correlation between replay and canon across the forty stories at the first and the last checkpoint. Net movement is the last-checkpoint score minus the first, averaged over stories.

Actor	Standstill	Goal achieved	Other resolution	Open
GPT	24	0	2	14
Claude	24	1	3	12
Gemini	17	3	2	18
DeepSeek	22	1	2	15
Kimi	16	0	1	23
Qwen	15	2	3	20

Appendix

Table 12: Replay endings by actor, out of forty. Standstill is the terminator’s final verdict. Other resolution counts goals failed, transformed, or settled. Open counts replays that reach their round budget with the story still moving, four of them under Kimi cut by the wall-clock limit.

	GPT	Claude	Gemini	DeepSeek	Kimi	Qwen	persona- free
Knowledge	91.2	87.4	89.4	87.3	87.1	86.7	78.9
Relations	94.9	89.3	92.2	92.2	91.1	88.9	79.1
Traits	96.4	92.9	96.3	94.3	94.7	93.3	79.3
Present (%)	83	69	80	75	73	66	–

Appendix

Table 13: Persona presence by actor. Facet means sit on the hundred-point scale over all interviewed character-runs, and Present is the share of character-runs passing the presence criterion of Section 5.2 . The persona-free column grades the same interviews against the removed card under GPT.

Actor	Adherence	Efficacy	Shift (scale units)	Cross calls (%)	Wins (%)
GPT	0.86 [0.82, 0.89]	0.17 [0.14, 0.21]	$+0.13$ [ $+0.04$ , $+0.21$ ]	11 [6, 17]	65 [59, 73]
Claude	0.83 [0.79, 0.88]	0.16 [0.12, 0.20]	$+0.12$ [ $+0.03$ , $+0.22$ ]	15 [8, 22]	59 [49, 69]
Gemini	0.83 [0.78, 0.87]	0.07 [0.05, 0.09]	$+0.03$ [ $-0.02$ , $+0.08$ ]	25 [16, 36]	36 [27, 48]
DeepSeek	0.86 [0.82, 0.89]	0.18 [0.14, 0.22]	$+0.14$ [ $+0.06$ , $+0.24$ ]	16 [8, 25]	68 [59, 76]
Kimi	0.80 [0.77, 0.84]	0.16 [0.13, 0.20]	$+0.14$ [ $+0.09$ , $+0.20$ ]	19 [12, 27]	54 [47, 61]
Qwen	0.84 [0.81, 0.88]	0.16 [0.12, 0.20]	$+0.23$ [ $+0.15$ , $+0.34$ ]	18 [11, 25]	60 [52, 68]

Appendix

Table 14: Decision-point statistics by actor with 95% intervals from a clustered bootstrap over stories. Shift is the mean scale position of the action minus that of the no-persona action, cross calls the share of points whose persona calls for crossing a line, and wins the persona’s share of the disagreements with the default.

Figure 5: Label matrices by actor. Each cell is the share of decision points, in percent, whose persona calls for the row label and whose action lands on the column label, with the action taken on the left and the no-persona action on the right of each pair. Rows sum to one hundred, the outlined diagonal is the share that keeps the call, and

n

counts the points of each row.

Actor	Character-runs pass / fail	Rank correlation	Adherence pass / fail	Efficacy pass / fail
GPT	66 / 14	$-0.22$	0.855 / 0.767	0.175 / 0.221
Claude	57 / 28	$-0.20$	0.818 / 0.832	0.159 / 0.187
DeepSeek	65 / 25	$+0.00$	0.855 / 0.804	0.181 / 0.182
Gemini	76 / 17	$+0.09$	0.827 / 0.873	0.073 / 0.099
Qwen	64 / 34	$+0.02$	0.845 / 0.854	0.153 / 0.158
Kimi	75 / 25	$-0.15$	0.805 / 0.805	0.163 / 0.168

Appendix

Table 15: Interview scores against decision behavior, by actor, over characters with at least three labeled decision points, counted once per actor. Rank correlation is Spearman’s between the trait facet mean and efficacy. Pass and fail refer to the presence criterion.

Condition	Calls	Prompt (M)	Completion (M)	Total (M)
GPT	13,823	87.3	4.2	91.5
Claude	14,735	63.7	5.5	69.1
Gemini	16,226	57.4	7.8	65.2
DeepSeek	14,112	51.0	4.7	55.7
Kimi	23,273	123.3	10.1	133.4
Qwen	15,189	54.8	13.8	68.6

Appendix

Table 16: Engine-side calls and tokens by condition, one replay per story and forty per row. Tokens are what the APIs report, in millions.

Large Language Model-Based Role-Play Simulation

Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.

See all 107 papers

Aug 8, 2026

"Many Are My Names": The Anatomy of the Assistant and Its Personas via Sparse Autoencoders

How a language model internally represents who is speaking, the Assistant, an assigned roleplay persona, or a narrated story character, remains underexplored. We study speaker representations using a dataset of user-expressed emotional text and corresponding model responses. We decompose three generation settings (Assistant, Roleplay, and Story) into sparse autoencoder features extracted at turn-boundary and pronoun-token positions and selected through a filtering pipeline for different depths. We characterize each surviving feature through its steering effects and activation distribution. Our main finding is that the Assistant and roleplay personas are not independent alternatives: personas retain the Assistant-associated feature core while progressively differentiating from it across layers, starting from operational machinery towards behavioral and stylistic features. Meanwhile, generated story characters lack the Assistant-associated core. Both Story and Roleplay can be distinguished from the Assistant with Immersive Simulation Mode. However, the Assistant can sometimes enter or slowly drift into it even in the default setting.

Adelaide Danilov, Aria Nourbakhsh, Oleksandr Marchenko Breneur +1

Department of Computer Science, Faculty of Science, Technology and Medicine, University of Luxembourg

Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models Language Model Introspection Persona-Conditioned Generation Language Model Probing

Ask PDF

Mar 3, 2026

Eval4Sim: An Evaluation Framework for Persona Simulation

Large Language Model personas, explicit profiles specifying a user's attributes, preferences, and behavioural tendencies, are increasingly used to simulate human conversations for user modelling, social reasoning, and behavioural analysis. Evaluating whether such simulations faithfully reflect human conversational behaviour is critical, yet current practice often relies on LLM-as-a-judge approaches that provide limited grounding in observable behaviour and produce opaque scalar scores. We present Eval4Sim, an evaluation framework that measures alignment between simulated and human conversations across three dimensions: adherence, whether persona traits are recoverable from dialogue via dense retrieval; consistency, whether a persona maintains a distinguishable stylistic identity via authorship verification; and naturalness, whether conversations exhibit human-like turn-to-turn flow via dialogue NLI. Unlike optimization-oriented metrics, each dimension takes a human corpus as a reference baseline and penalizes deviations in both directions, distinguishing insufficient persona encoding from over-optimized, unnatural behaviour. The framework is corpus-agnostic: any persona-annotated conversational dataset can serve as the reference. Evaluated over ten simulation corpora, Eval4Sim surfaces systematic trade-offs invisible to single-score methods.

Eliseo Bao, Anxo Perez, Javier Parapar +1

IRLab, CITIC, Universidade da Coruña A Coruña, Spain · University of Sheffield Sheffield, United Kingdom

Large Language Model-Based Role-Play Simulation Large Language Model-Based Social Simulation Multi-Turn Dialogue Evaluation

Ask PDF

Sep 9, 2026

Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble

Language models are trained to implement a helpful AI Assistant character (e.g., Claude). We explore how finetuning on synthetic stories affects this character. Does it change the Assistant's behavior in multi-turn conversations with users, a format quite different from the stories? And does the Assistant adopt the behaviors and preferences of human characters? We refer to this adoption as story imprinting. We finetune GPT-4.1 and Kimi-K2.6 on stories in which generally helpful human characters give subtly harmful advice after being insulted. The Assistant adopts the same conditional behavior while otherwise remaining helpful. This occurs even when fewer than 2% of stories depict the behavior. In a separate experiment, the Assistant adopts preferences that are only implicit in the narration. A human character's body language suggests they dislike working on spreadsheets, yet they never say so and continue giving good advice on spreadsheets. After finetuning, the Assistant becomes less likely to choose spreadsheet tasks. Next we ask which characters most influence the Assistant. We find the Assistant adopts behaviors more often from characters that resemble it (e.g., helpful rather than dismissive). We call this the affinity effect. The effect extends to other personas elicited with system prompts: unhelpful personas adopt behaviors from unhelpful characters. We also observe it in finetuned base models. We use the affinity effect to learn how models represent the Assistant. We find the Assistant adopts behaviors more from characters affiliated with elite universities (e.g., Yale) than non-elite ones. This implies the model's internal representation of the Assistant is more similar to humans from elite universities. Overall, the Assistant can be influenced by stories that depict only human characters (no AIs), which may conflict with the Persona Selection Model for the Assistant.

Jorio Cocola, Lev McKinney, Harry Mayne +2

Truthful AI · Harvard University · METR +1

Fine-Tuning Personality Modeling in Language Models

Ask PDF

Abstract

Large Language Model-Based Role-Play Simulation

Explore similar work

"Many Are My Names": The Anatomy of the Assistant and Its Personas via Sparse Autoencoders

Eval4Sim: An Evaluation Framework for Persona Simulation

Story Imprinting: AI Assistants Absorb Traits from Human Characters They Resemble