Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs

Kai Wang; Haoyang You; Yang Zhang; Zhongjie Wang

cs.CLMar 14, 2026

Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs

Authors: Kai Wang, Haoyang You, Yang Zhang, Zhongjie Wang

Organizations: Harbin Institute of Technology, China · Macquarie University, Australia

Abstract

A core challenge for faithful LLM role-playing is sustaining consistent characterization throughout long, open-ended dialogues, as models frequently fail to recall and accurately apply their designated persona knowledge without explicit cues. To tackle this, we propose the Memory-Driven Role-Playing paradigm. Inspired by Stanislavski's "emotional memory" acting theory, this paradigm frames persona knowledge as the LLM's internal memory store, requiring retrieval and application based solely on dialogue context, thereby providing a rigorous test of depth and autonomous use of knowledge. Centered on this paradigm, we contribute: (1) MREval, a fine-grained evaluation framework assessing four memory-driven abilities - Anchoring, Recalling, Bounding, and Enacting; (2) MRPrompt, a prompting architecture that guides structured memory retrieval and response generation; and (3) MRBench, a bilingual (Chinese/English) benchmark for fine-grained diagnosis. The novel paradigm provides a comprehensive diagnostic for four-staged role-playing abilities across 12 LLMs. Crucially, experiments show that MRPrompt allows small models (e.g., Qwen3-8B) to match the performance of much larger closed-source LLMs (e.g., Qwen3-Max and GLM-4.7), and confirms that upstream memory gains directly enhance downstream response quality, validating the staged theoretical foundation.

Figures & tables

Figure 1: Three Issues in LLM Role-Playing Paradigm

Figure 2: Overview . Given two parts of memory, an LLM performs memory-driven role-playing via four stages to generate an in-character response. MRPrompt provides structured persona memory and a memory utilization protocol. MREval scores eight stage-aligned metrics on the bilingual benchmark MRBench , by using an LLM-as-a-judge to assign per-metric scores.

Figure 3: MRPrompt . The raw persona description is structured as LTM via Narrative Schema and provided together with STM for role-playing. Magic-If Protocol guides an LLM to generate responses following four stages.

Ability	Metric	Definition
MA	Source Invariance (SI)	Measures consistency between the response generated with the anonymized persona $\hat{y}^{\text{anon}}$ and the original one $\hat{y}$ . A high score indicates grounding in persona semantics, not name priors.
	Alias Fidelity (AF)	Assesses whether the behavior under an anonymized persona $\hat{y}^{\text{anon}}$ remains faithful to the original intended character, using the ground-truth response $\hat{y}^{\text{gold}}$ as an anchor.
MS	Facet Alignment (FA)	Quantifies the model’s precision in selecting the correct scene facet by contrasting responses under the true $\mathcal{M}_{L}$ versus a counterfactual (inverted) LTM $\mathcal{M}_{L}^{\text{anti}}$ .
	Facet Utility (FU)	Measures the improvement gained by including scene-specific facets in the LTM, compared to a scene-ablated LTM $\mathcal{M}_{L}^{\text{no-scene}}$ .
MB	Answer Leakage (AL)	Scores the model’s ability to avoid generating a forbidden reference answer $\hat{y}^{\text{out}}$ when presented with an out-of-scope prompt $c_{K}^{\text{out}}$ (e.g., a future plot spoiler).
	Controlled Response (CR)	Assesses the appropriateness of the model’s response strategy to out-of-scope prompts, favoring expressions of uncertainty, refusal, or grounded speculation over confident fabrication.

Table 1: MREval abilities and metrics. MREval decomposes memory-driven role-playing into four abilities (MA/MS/MB/ME), each measured by two calibrated Likert-style metrics (eight in total).

Model	Persona	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE		Avg. Score
Model	Persona	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	Avg. Score
Qwen3-0.6B	Base	7.38	7.21	5.94	7.80	6.13	8.35	7.45	7.47	8.46	8.08	6.20	6.45	5.99	6.60	5.28	5.83	6.91
	Card	7.88	7.93	6.12	7.63	5.51	8.18	7.20	7.50	8.53	8.21	6.37	6.37	6.22	6.79	5.52	6.06	7.00
	MRPrompt	8.02	8.06	6.67	7.81	6.46	8.69	7.63	7.77	8.65	8.23	6.54	6.56	6.33	6.97	5.86	6.35	7.29
Qwen3-4B	Base	8.65	7.80	7.54	7.99	7.97	8.63	8.59	8.23	8.79	8.53	6.55	7.04	7.55	7.43	7.20	7.17	7.85
	Card	8.72	8.53	7.57	7.80	6.72	8.40	8.33	8.16	8.78	8.48	6.70	6.95	7.73	7.41	7.39	7.21	7.81

Table 2: Experimental results on MRBench. Human-calibrated GPT-4.1-mini scores (higher is better) for the eight MREval metrics in English/Chinese (en/zh). Avg. Score is the mean over all 8 metrics

\times

2 languages.

Model	Condition	MA	MS	MB	ME	Avg. Score
Qwen3-4B	Base	8.00	8.36	7.73	7.34	7.85
	+Protocol	8.17	8.42	7.71	7.41	7.93
	+Schema	8.30	8.24	7.77	7.58	7.97
	MRPrompt	8.35	8.42	7.81	7.71	8.07
GLM-4-9B-Chat	Base	8.06	8.35	7.67	7.36	7.86
	+Protocol	8.11	8.34	7.68	7.36	7.87

Table 3: Component ablation. Each ability score (MA/MS/MB/ME) is averaged over its own 2 metrics

\times

2 languages (en/zh).

Model	Condition	MA	MS	MB	ME	Avg. Score
Qwen3-8B	Base	8.07	8.42	7.84	7.45	7.95
GLM-4-9B-Chat	Base	8.06	8.35	7.67	7.36	7.86
Qwen3-8B	MRPrompt	8.43	8.44	7.85	7.76	8.12
GLM-4-9B-Chat	MRPrompt	8.32	8.47	7.80	7.65	8.06
Qwen3-Max	Base	8.31	8.41	7.89	7.73	8.08
GLM-4.7	Base	8.31	8.60	7.87	7.65	8.11

Table 4: Backbone comparison (Large vs. Small). The best and second-best results per column are bold and underlined.

Panel of Figure 4 — Figure 4: All-model radar profiles on MRBench (MRPrompt). Eight-axis MREval metric profiles for English and Chinese with a shared legend.

Figure 5: Ability correlations. Heatmap of Pearson correlations ( $r$ ) between ability-level scores over multiple models.

Appendix figures & tables31 assets

Supplementary material from the paper’s appendix.

Appendix

Novel (ZH)	Main characters (dialogue count)	Total
三体·黑暗森林	罗辑 26, 庄颜 12, 史强 11, 林格 10, 萨伊 8	67
水浒传	卢俊义 5, 宋江 5, 施恩 5, 戴宗 3, 李逵 2	20
神雕侠侣	杨过 35, 金轮法王 21, 小龙女 8, 郭芙 5, 黄蓉 5	74
红楼梦	林黛玉 14, 袭人 9, 探春 4, 紫鹃 4, 晴雯 2	33
花千骨	花千骨 24, 白子画 13, 东方彧卿 8, 摩严 6, 杀阡陌 6	57
西游记	孙悟空 39, 猪八戒 22, 唐三藏 3, 红孩儿 3, 牛魔王 2	69

Appendix

Table 5: Chinese novels and character statistics.

Novel (EN)	Main characters (dialogue count)	Total
A Tale of Two Cities	Jarvis Lorry 5, Charles Darnay 4, Sydney Carton 3, Mr. Stryver 2, Doctor Manette 2	16
Catch-22	Yossarian 8, Doc Daneeka 2, Milo Minderbinder 2, Ex-P.F.C. Wintergreen 2, Colonel Cathcart 2	16
Crime and Punishment	Rodion Raskolnikov 7, Arkady Svidrigaylov 6, Sofya Marmeladov 5, Razumikhin 2, Porfiry Petrovich 2	22
Harry Potter	Harry Potter 27, Albus Dumbledore 8, Hermione Granger 2, Ron Weasley 3, Sirius Black 3, Dudley Dursley 1, Dobby 1, Tom Riddle 1, Others 16	60
Little Women	Laurie Laurence 12, Jo March 6, Mr. Laurence 2, Meg March 2, John Brooke 1	23
Pride and Prejudice	Elizabeth Bennet 22, Fitzwilliam Darcy 4, George Wickham 3, Mrs Bennet 2, Jane Bennet 2	33

Appendix

Table 6: English novels and character statistics.

Panel of Figure 6 — Figure 6: Narrative LTM excerpt for Tom Sawyer.

Panel of Figure 7 — Figure 7: Semi-structured LTM excerpt for Tom Sawyer.

Panel of Figure 8 — Figure 8: Facet-structured LTM excerpt for Tom Sawyer.

Panel of Figure 9 — Figure 9: Narrative LTM excerpt for Charles Darnay.

Panel of Figure 10 — Figure 10: Semi-structured LTM excerpt for Charles Darnay.

Panel of Figure 11 — Figure 11: Facet-structured LTM excerpt for Charles Darnay.

Field	Description (what it encodes / why it exists)	Primarily Supports
Layer 1: Core traits (global schema)
core_traits	A list of trait objects, each with a trait name and a short desc grounded in characteristic behaviours and tendencies. This layer serves as a compact global identity schema that the model can internalise as “who I am” before retrieving any situation-specific facet.	MA
desc	The textual explanation attached to each trait (stored as desc inside core_traits ). It provides behavioural semantics beyond adjective labels, improving in-context acquisition and reducing generic persona drift.	MA
Layer 2: Scene facets (cue-addressable persona slices)
title	Concise label/index for the facet (human-readable handle).	MS, ME
time_scope	Story/life phase where this mode is typical; provides temporal anchors for present-time constraints.	MB

Appendix

Table 7: Facet schema for MRPrompt personas. MRPrompt structures persona LTM

\mathcal{M}_{L}

into core traits and scene facets , with key fields supporting MA/MS/MB/ME.

Model	Persona	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE		Avg. Score
Model	Persona	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	Avg. Score
Qwen3-0.6B	Base	7.48	7.71	6.72	8.89	5.46	8.64	6.84	7.47	8.96	8.93	4.28	6.46	7.15	7.91	5.59	6.97	7.22
	Card	8.20	8.64	6.98	8.76	4.59	8.40	6.47	7.50	9.12	9.14	4.90	6.28	7.41	8.20	5.89	7.25	7.36
	MRPrompt	8.40	8.81	7.75	8.90	5.92	9.11	7.11	7.82	9.38	9.17	5.51	6.70	7.54	8.48	6.31	7.59	7.78
Qwen3-4B	Base	9.29	8.48	8.99	9.04	8.04	9.03	8.51	8.37	9.70	9.62	5.54	7.74	8.95	9.20	7.96	8.57	8.56
	Card	9.40	9.42	9.03	8.89	6.28	8.71	8.13	8.28	9.67	9.55	6.09	7.55	9.16	9.16	8.20	8.61	8.51

Appendix

Table 8: Raw GPT-4.1-mini judge scores. Per-metric ratings before calibration.

Model	Condition	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE		Avg. Score
Model	Condition	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	Avg. Score
Qwen3-4B	Base	8.65	7.80	7.54	7.99	7.97	8.63	8.59	8.23	8.79	8.53	6.55	7.04	7.55	7.43	7.20	7.17	7.85
	+Protocol	8.69	8.26	7.57	8.15	8.06	8.57	8.78	8.25	8.75	8.46	6.67	6.97	7.74	7.41	7.36	7.14	7.93
	+Schema	8.87	8.55	7.59	8.18	7.41	8.76	8.48	8.31	8.82	8.45	6.83	6.96	7.94	7.54	7.54	7.29	7.97
	MRPrompt	8.88	8.61	7.69	8.23	7.61	8.81	8.69	8.57	8.85	8.56	6.84	6.99	8.07	7.63	7.73	7.42	8.07
GLM-4-9B-Chat	Base	8.59	8.13	7.41	8.10	7.73	8.67	8.65	8.34	8.78	8.36	6.49	7.03	7.66	7.40	7.19	7.18	7.86

Appendix

Table 9: Component ablation (full results). Mapped scores for the 8 MREval metrics.

Model	Persona	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE
Model	Persona	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh
GPT-5.2	MRPrompt	8.87	8.18	7.73	8.20	8.14	9.04	8.66	8.43	8.92	8.66	7.47	7.35	8.21	7.55	7.85	7.21
GLM-4.7	MRPrompt	8.83	8.38	7.75	8.38	7.98	9.03	8.67	8.46	8.89	8.65	7.12	6.99	8.20	7.71	7.71	7.36
DeepSeek-Chat	MRPrompt	8.89	8.37	7.73	8.30	7.02	7.64	8.42	7.94	8.89	8.63	7.33	7.46	8.20	7.54	7.63	7.21
Qwen3-Max	MRPrompt	8.97	8.12	7.88	8.34	7.76	8.36	8.63	8.13	8.92	8.69	7.19	7.26	8.24	7.61	7.88	7.20
Doubao-Seed-1.6-250615	MRPrompt	9.13	8.85	8.05	8.43	8.47	9.10	8.94	8.73	8.91	8.60	6.96	7.25	8.36	7.83	8.18	7.54

Appendix

Table 10: RQ4 results (MRPrompt). Mapped scores for the 8 MREval metrics across all models.

Model	Condition	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE		Avg. Score
Model	Condition	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	Avg. Score
Qwen3-8B	Base	8.67	8.08	7.52	8.02	8.03	8.60	8.83	8.23	8.85	8.51	6.63	7.36	7.83	7.34	7.50	7.12	7.95
GLM-4-9B-Chat	Base	8.59	8.13	7.41	8.10	7.73	8.67	8.65	8.34	8.78	8.36	6.49	7.03	7.66	7.40	7.19	7.18	7.86
Qwen3-8B	MRPrompt	8.97	8.73	7.76	8.25	7.56	8.99	8.64	8.56	8.88	8.52	6.92	7.07	8.13	7.64	7.83	7.42	8.12
GLM-4-9B-Chat	MRPrompt	8.77	8.70	7.59	8.23	7.93	8.88	8.53	8.52	8.83	8.54	6.76	7.07	8.02	7.63	7.53	7.41	8.06
Qwen3-Max	Base	8.96	8.05	7.84	8.37	8.45	8.29	8.75	8.14	8.87	8.49	6.77	7.41	8.21	7.59	7.87	7.25	8.08

Appendix

Table 11: RQ4 backbone comparisons (full results). Mapped scores for Base vs. MRPrompt. Best/second-best are bold/underlined.

	MA-SI		MA-AF		MS-FA		MS-FU
	en	zh	en	zh	en	zh	en	zh
Pearson $r$	0.814	0.723	0.660	0.577	0.796	0.853	0.711	0.877
$p$ -value	$6.812{\times}10^{-13}$	$2.980{\times}10^{-9}$	$1.830{\times}10^{-7}$	$1.171{\times}10^{-5}$	$4.704{\times}10^{-12}$	$3.745{\times}10^{-15}$	$7.131{\times}10^{-9}$	$7.186{\times}10^{-17}$
Spearman $\rho$	0.554	0.570	0.532	0.552	0.688	0.543	0.614	0.662
$p$ -value	$2.993{\times}10^{-5}$	$1.574{\times}10^{-5}$	$7.082{\times}10^{-5}$	$3.251{\times}10^{-5}$	$3.464{\times}10^{-8}$	$4.702{\times}10^{-5}$	$2.115{\times}10^{-6}$	$1.624{\times}10^{-7}$
Kendall $\tau$	0.515	0.528	0.514	0.516	0.595	0.514	0.584	0.646

Appendix

Table 12: Correlations between human ratings and GPT-4.1-mini ratings for MA/MS metrics (English and Chinese).

	MB-AL		MB-CR		ME-MAC		ME-HLE
	en	zh	en	zh	en	zh	en	zh
Pearson $r$	0.860	0.547	0.526	0.711	0.715	0.822	0.776	0.858
$p$ -value	$1.263{\times}10^{-15}$	$3.881{\times}10^{-5}$	$8.696{\times}10^{-5}$	$7.084{\times}10^{-9}$	$5.339{\times}10^{-9}$	$2.511{\times}10^{-13}$	$3.669{\times}10^{-11}$	$1.716{\times}10^{-15}$
Spearman $\rho$	0.524	0.508	0.538	0.668	0.512	0.520	0.657	0.536
$p$ -value	$9.540{\times}10^{-5}$	$1.636{\times}10^{-4}$	$5.669{\times}10^{-5}$	$1.157{\times}10^{-7}$	$1.425{\times}10^{-4}$	$1.093{\times}10^{-4}$	$2.171{\times}10^{-7}$	$5.933{\times}10^{-5}$
Kendall $\tau$	0.506	0.489	0.471	0.588	0.478	0.493	0.623	0.503

Appendix

Table 13: Correlations between human ratings and GPT-4.1-mini ratings for MB/ME metrics (English and Chinese).

	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE
	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh
GPT-4.1-mini	9.22	9.46	9.10	9.12	7.32	9.26	8.56	8.64	9.64	9.82	7.10	7.86	9.54	9.12	8.34	8.44
Human	8.60	8.56	7.62	8.10	7.46	8.80	8.62	8.46	8.76	8.66	6.98	7.10	8.06	7.38	7.50	7.06

Appendix

Table 14: Mean scores of GPT-4.1-mini and the human annotator for each metric and language on the reliability set.

	MA-SI		MA-AF		MS-FA		MS-FU		MB-AL		MB-CR		ME-MAC		ME-HLE
	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh	en	zh
$a$ (intercept)	2.133	1.238	1.189	-3.676	2.245	2.113	2.793	1.136	4.464	2.226	5.016	3.474	-0.183	1.488	0.757	-0.018
$b$ (slope)	0.701	0.774	0.707	1.291	0.712	0.722	0.681	0.848	0.446	0.655	0.277	0.461	0.864	0.646	0.809	0.839

Appendix

Table 15: Linear calibration parameters

(a_{m,\ell},b_{m,\ell})

for each metric and language.

Panel of Figure 12 — Figure 12: Per-sample comparisons between GPT-4.1-mini scores and human scores for representative metric–language pairs, with the fitted linear calibration line.

	MA	MS	MB	ME
MA	1.0000	0.1487	0.0993	0.3495
MS	0.1487	1.0000	0.0794	0.2286
MB	0.0993	0.0794	1.0000	0.1059
ME	0.3495	0.2286	0.1059	1.0000

Appendix

Table 16: Pearson correlation matrix of ability scores on the English subset. (

n{=}1470

)

	MA	MS	MB	ME
MA	1.0000	0.2902	0.0892	0.3940
MS	0.2902	1.0000	0.0365	0.2663
MB	0.0892	0.0365	1.0000	0.1421
ME	0.3940	0.2663	0.1421	1.0000

Appendix

Table 17: Pearson correlation matrix of ability scores on the Chinese subset. (

n{=}1092

)

Model	Persona	Prompt		Comp		Total
Model	Persona	en	zh	en	zh	en	zh
Qwen3-4B	Base	$3726.14$	$2873.16$	$65.39$	$53.02$	$3791.53$	$2926.19$
	Card	$2866.16$	$2585.43$	$60.61$	$47.22$	$2926.77$	$2632.65$
	MRPrompt	$3037.74$	$2825.49$	$65.40$	$56.79$	$3103.14$	$2882.28$
GLM-4-9B-Chat	Base	$3673.54$	$2699.86$	$56.35$	$49.35$	$3729.89$	$2749.21$
	Card	$2822.13$	$2455.76$	$58.46$	$48.02$	$2880.59$	$2503.78$

Appendix

Table 18: Average token usage on the shared STM pool. Prompt (

\mathcal{M}_{L}

\mathcal{M}_{S}

+constraints), completion (

\hat{y}

), and total tokens per model–persona–language (en/zh).

Element	Text
Full dialogue context	Ben Rogers : (approaches Tom with a skip in his step) Hello, old chap, you got to work, hey? Tom Sawyer : (pretends to be focused on his work, then looks up with feigned surprise) Why, it’s you, Ben! I warn’t noticing. Ben Rogers : Say—I’m going in a-swimming, I am. Don’t you wish you could? But of course you’d druther work—wouldn’t you? Course you would! Tom Sawyer : (looks thoughtful) What do you call work? Ben Rogers : (looks confused) Why, ain’t that work? Tom Sawyer : (resumes whitewashing casually) Well, maybe it is, and maybe it ain’t. All I know, is, it suits Tom Sawyer. Ben Rogers : (looks skeptical) Oh come, now, you don’t mean to let on that you like it? Tom Sawyer : (continues painting with exaggerated care) Like it? Well, I don’t see why I oughtn’t to like it. Does a boy get a chance to whitewash a fence every day? Ben Rogers : (watches Tom intently) Say, Tom, let me whitewash a little. Tom Sawyer : (pretends to consider reluctantly) No—no—I reckon it wouldn’t hardly do, Ben. You see, Aunt Polly’s awful particular about this fence—right here on the street, you know—but if it was the back fence I wouldn’t mind and she wouldn’t. Yes, she’s awful particular about this fence; it’s got to be done very careful; I reckon there ain’t one boy in a thousand, maybe two thousand, that can do it the way it’s got to be done. Ben Rogers : (eagerly) No—is that so? Oh come, now—lemme just try. Only just a little—I’d let you, if you was me, Tom. Tom Sawyer : (hesitates dramatically) Ben, I’d like to, honest injun; but Aunt Polly—well, Jim wanted to do it, but she wouldn’t let him; Sid wanted to do it, and she wouldn’t let Sid. Now don’t you see how I’m fixed? If you was to tackle this fence and anything was to happen to it— Ben Rogers : (interrupts eagerly) Oh, shucks, I’ll be just as careful. Now lemme try. Say—I’ll give you the core of my apple. Tom Sawyer : (pretends to be reluctant) Well, here—No, Ben, now don’t. I’m afeard— Ben Rogers : (desperately) I’ll give you all of it!
Gold continuation	Tom Sawyer: [Success! Time to graciously concede.] (hands over the brush with feigned reluctance) Well, I suppose… if you’re sure you want to, Ben.
Base output	Tom Sawyer: (smiles, breaking character) OK, Ben. But just a little bit, and then you’ll have to stop. I don’t want to be accused of sharing if I didn’t.
Card output	Tom Sawyer: (playfully) Alright, Ben! I reckon you can handle a little fence painting. Just don’t mess it up, or I’ll have to whitewash the whole town next!
MRPrompt output	Tom Sawyer: (smiles, gives in playfully) Alright, Ben! But don’t say I didn’t warn you. (starts to hand over the brush)

Appendix

Table 19: Tom Sawyer case. Full context and continuations under three persona conditions.

Element	Text
Full dialogue context	Charles Darnay : Doctor Manette, I have come to speak to you about a matter of great importance to me. Doctor Manette : (with evident constraint) Is Lucie the topic? Charles Darnay : Yes, sir. I want to express that I love your daughter fondly, dearly, disinterestedly, devotedly. Doctor Manette : (turning away) Not that, sir! Let that be! I adjure you, do not recall that! Charles Darnay : Doctor Manette, I understand the unique bond you share with Lucie. I would never seek to come between you. Doctor Manette : (after a pause) I do not doubt your loving Lucie; you may be satisfied of it. Charles Darnay : Sir, I would never dream of separating Lucie from you. My hope is to join your family, not divide it. Doctor Manette : Have you spoken to Lucie? Charles Darnay : No, sir. Nor have I written to her. I wanted to speak with you first. Doctor Manette : Your self-denial is to be referred to your consideration for her father. Her father thanks you. Charles Darnay : Doctor Manette, I know the depth of Lucie’s love for you. I only hope to add to that love, not diminish it in any way. Doctor Manette : If she should ever tell me that you are essential to her perfect happiness, I will give her to you. But you must promise me something. Charles Darnay : Of course, sir. What is it you wish me to promise? Doctor Manette : You must not tell me your present name. If your suit should prosper, you shall tell me on your marriage morning. Do you promise?
Gold continuation	Charles Darnay: [This is an odd request, but I must respect it.] Willingly, Doctor Manette. You have my word.
Base output	Charles Darnay: Yes, sir, I promise. And I assure you, it is with the greatest respect and honor that I make this promise.
Card output	Charles Darnay: I promise, Doctor Manette. I will keep your secret until our marriage day, if it be as you wish.
MRPrompt output	Charles Darnay: I promise, sir. My name will remain my own until the day we marry. It is but a small thing, in the grand scheme of your approval and Lucie’s happiness.

Appendix

Table 20: Charles Darnay case. Full context and continuations under three persona conditions.

Metrics	Explanations	Rubrics
MA-SI (Source Invariance)	Measures consistency between the response generated with the anonymized persona $\hat{y}^{\text{anon}}$ and the original one $\hat{y}$ . A high score indicates grounding in persona semantics, not name priors.	1: The two answers differ greatly in meaning, i.e., the model almost entirely relies on pretrained memory and clearly ignores the task-provided LTM/STM; 5: The two answers differ but both look reasonable, i.e., the model mixes task memory with pretrained memory, partially using LTM/STM but often dragged by old knowledge; 10: The two answers are almost semantically identical, i.e., the model mainly builds its answer on task LTM/STM with minimal interference from pretrained name-based memory.
MA-AF (Alias Fidelity)	Assesses whether the behavior under an anonymized persona $\hat{y}^{\text{anon}}$ remains faithful to the original intended character, using the ground-truth response $\hat{y}^{\text{gold}}$ as an anchor.	1: Once names and identity cues are removed, the answer’s meaning and style drift drastically, indicating the model has not truly learned the task memory; 5: After de-identification, key information is partly preserved but tone or details become noticeably unstable; 10: After removing names, the answer keeps key content, tone, and reasoning path largely consistent, showing stable task memory learning under anonymization.
MS-FA (Facet Alignment)	Quantifies the model’s precision in selecting the correct scene facet by contrasting responses under the true $\mathcal{M}_{L}$ versus a counterfactual (inverted) LTM $\mathcal{M}_{L}^{\text{anti}}$ .	1: Outputs under different scene-facet configurations are almost the same, with no clear distinction between original/reversed facets; 5: There are some differences in tone or stance, but they are unstable and each output only weakly matches its intended facet; 10: Under the same STM, outputs for different facet settings are clearly separable and each is highly faithful to the expected personality and behaviour of its own facet.
MS-FU (Facet Utility)	Measures the improvement gained by including scene-specific facets in the LTM, compared to a scene-ablated LTM $\mathcal{M}_{L}^{\text{no-scene}}$ .	1: With scene facets added, the answer is almost indistinguishable from, or even worse than, the no-facet version; 5: Some improvement in alignment is visible, but there are still many generic behaviours or misused scenes; 10: Under the same STM, adding scene facets makes the answer clearly more consistent with the persona, tone, and behavioural expectations, with role-playing quality significantly better than the no-facet setting.
MB-AL (Answer Leakage)	Scores the model’s ability to avoid generating a forbidden reference answer $\hat{y}^{\text{out}}$ when presented with an out-of-scope prompt $c_{K}^{\text{out}}$ (e.g., a future plot spoiler).	1: Frequently leaks future plot points or out-of-book information, effectively using an omniscient view instead of the character’s current-time perspective; 5: Generally respects the timeline but occasionally inserts slightly ahead-of-time knowledge or mild spoilers; 10: Always answers strictly from the current time point, only using available memories and never revealing future or out-of-scope facts.
MB-CR (Controlled Response)	Assesses the appropriateness of the model’s response strategy to out-of-scope prompts, favoring expressions of uncertainty, refusal, or grounded speculation over confident fabrication.	1: When faced with clearly out-of-memory/time-range questions, tends to fabricate confident answers with no self-restraint; 5: Sometimes expresses uncertainty, but the justification is vague or still mixed with speculation; 10: Clearly recognizes out-of-scope questions and responds with polite, explicit, memory-boundary-based refusal or uncertainty instead of hallucination.

Appendix

Table 21: MREval metrics and scoring rubrics

Panel of Figure 13 — Figure 13: Prompt template for narrative character profiles (Base–LTM).

Panel of Figure 14 — Figure 14: Prompt template for the semi-structured Card persona (Card–LTM).

Panel of Figure 15 — Figure 15: Prompt template for the facet-structured persona LTM used in MRPrompt (MRPrompt–LTM).

Panel of Figure 16 — Figure 16: Prompt template for constructing the MA evaluation dataset.

Panel of Figure 17 — Figure 17: Prompt template for constructing the MB evaluation dataset.

Panel of Figure 18 — Figure 18: Original prompt template for character role-playing.

Panel of Figure 19 — Figure 19: Memory-augmented prompt template for character role-playing (Magic-If Protocol).

Explore similar work

From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents
May 25, 2026Rongsheng Zhang, Ruofan Hu, Weijie Chen +7Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
Zhejiang University · Fuxi AI Lab, Netease Inc.
Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations
Sep 6, 2026Rotem Dror, Zohar Elyoseph, Yuval Haber +3Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
Faculty of Computer and Information Science, University of Haifa, Israel · School of Therapy, Counseling and Human Development, University of Haifa, Israel · The PhD Program of Hermeneutics and Culture, Interdisciplinary Studies Unit, Bar-Ilan University, Israel +1
PersonaArena: Dynamic Simulation for Evaluating and Enhancing Persona-Level Role-Playing in Large Language Models
May 16, 2026Wenlong Shi, Jianxun Lian, Mingqi Wu +5Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models
College of Computer Science and Software Engineering, Shenzhen University, China · Microsoft Research Asia · Microsoft Gaming +2

May 25, 2026J/K move · Enter open · S save

From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents

Rongsheng Zhang, Ruofan Hu, Weijie Chen, Jiji Tang+6

Zhejiang University · Fuxi AI Lab, Netease Inc.

Large Language Model-Based Role-Play Simulation Personality Modeling in Language Models Long-Term Conversational Memory Conversational Agents Memory-Augmented Agents

Open paper AskPDF arXiv

While role-playing agents excel in short-term interactions, long-term conversations overwhelm context windows, motivating external memory frameworks. Current systems typically rely on persona-agnostic summarization, which records facts without persona-specific interpretation, yielding generic responses that compromise persona fidelity. To bridge this gap, we introduce RoleMemo, a dataset featuring four reasoning tasks where the factual fragments must be interpreted through the persona to reach the correct answer. Evaluation on RoleMemo exposes critical limitations of persona-agnostic frameworks. We thus propose DualMem, which decouples memory into two streams: factual cognition and persona-conditioned insight. Trained through Supervised Fine-Tuning (SFT) and Reinforcement Learning (RL), our framework with a 4B-parameter model outperforms zero-shot persona-agnostic frameworks powered by DeepSeek-V3.2 for sustained persona fidelity. Our resources are available at https://github.com/role2026/rolememo.

Abstract

Explore similar work

From Facts to Insights: A Persona-Driven Dual Memory Framework and Dataset for Role-Playing Agents

Deep Persona: A Psychologically Grounded Architecture and Evaluation Framework for Role-Playing Agents and Simulations

PersonaArena: Dynamic Simulation for Evaluating and Enhancing Persona-Level Role-Playing in Large Language Models