Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models

Xingjie Zhuang; Jialong Tang; Chulun Zhou; Buchao Zhan; Zhirui Li; Junhui Li; Yazheng Yang; Jinsong Su

cs.CLSep 30, 2026

Cognitive Enhancement: Rethinking the Necessity of Role-Playing for Large Language Models

Authors: Xingjie Zhuang, Jialong Tang, Chulun Zhou, Buchao Zhan, Zhirui Li, Junhui Li, Yazheng Yang, Jinsong Su

Organizations: School of Informatics, Xiamen University · Tongyi Lab · The Chinese University of Hong Kong · Soochow University

Abstract

Role-playing prompting has become a popular yet simple technique for improving LLM reasoning and output quality. However, whether it consistently boosts performance across diverse domains remains unclear, as systematic validation is lacking. To fill this gap, we run multi-model, cross-domain, and multilingual experiments on MMLU and MMLU-Redux. We find that gains from role-play prompting depend heavily on model capacity, knowledge domain, and prompt language. Drawing on metacognition theory, we propose the persona-related cognitive alignment hypothesis: role-play works only when the LLM correctly grasps the designated persona and its associated knowledge domain. We test this hypothesis through persona information richness ablation, layer-wise entropy divergence analysis, and latent thought-space deflection observation. To reduce persona cognitive bias and stabilize role-play performance, we propose Mixed-Language Concatenate Prediction (MLCP), a simple, training-free, and efficient multilingual prompt concatenation strategy. It aggregates semantically equivalent role prompts to enrich complementary representational cues. Extensive experiments show that MLCP consistently outperforms vanilla role-play prompting across all tested LLMs.

Figures & tables

Figure 1: Performance comparison of domain experts with and without role-playing prompts.

Models	Benchmark	Base	Role-playing
Models	Benchmark	Base	en	zh	es	fr	ja	de	km	ms	avg (non-en)
Qwen3-4B	MMLU-Redux	0.756	0.754	0.730	0.746	0.746	0.757	0.746	0.746	0.757	0.747
Qwen3-4B	MMLU	0.756	0.754	0.724	0.762	0.763	0.757	0.763	0.757	0.757	0.755
Qwen3-8B	MMLU-Redux	0.764	0.767	0.749	0.769	0.769	0.772	0.769	0.769	0.774	0.767
Qwen3-8B	MMLU	0.782	0.782	0.756	0.796	0.793	0.792	0.792	0.788	0.790	0.787
Qwen3-14B	MMLU-Redux	0.798	0.810	0.787	0.810	0.810	0.820	0.810	0.810	0.806	0.808

Table 1: Performance under base setting and multilingual role-playing setting on MMLU and MMLU-Redux benchmarks. For each row, the best and worst language results are highlighted.

Model	Benchmark	Improved $\uparrow$	Degraded $\downarrow$	Unchanged
Qwen3-4B	MMLU-Redux	14	12	4
Qwen3-4B	MMLU	27	26	4
Qwen3-8B	MMLU-Redux	16	12	2
Qwen3-8B	MMLU	28	22	7
Qwen3-14B	MMLU-Redux	19	6	5
Qwen3-14B	MMLU	31	17	9

Table 2: Count statistics of benchmark-internal subjects, comparing model performance under the Role-playing prompt setting and the Base setting. Improved

\uparrow

, Degraded

\downarrow

, and Unchanged respectively represent the number of subjects where the role-playing setting outperforms, underperforms, and yields identical results to the base setting.

Subject	Simple	Moderate	Rich
anatomy	$-9\%$	$-4\%$	$-4\%$
high_school_geography	$-6\%$	$-3\%$	$-1\%$
college_physics	$-7\%$	$-4\%$	$0\%$
business_ethics	$-5\%$	$-1\%$	$+2\%$
human_aging	$-4\%$	$0\%$	$-3\%$
philosophy	$-4\%$	$0\%$	$0\%$

Table 3: Performance deviation of Qwen3-4B under incremental persona information richness settings, where values denote relative performance changes of role-playing prompts compared to the base prompt.

Figure 2: Layer-wise entropy difference across transformer layers for Qwen3-4B/8B/14B on the MMLU-Redux benchmark. Entropy is computed from the final-token hidden state at each layer after projection through the language model head, measuring the model’s predictive uncertainty for the next token. Positive values indicate that role-playing prompts increase uncertainty compared to base setting.

Figure 3: Comparison of hidden representation space deflection derived from layer-averaged states between base and role-playing prompting on multiple MMLU-Redux subjects.

Figure 4: Spatial geometric analysis of hidden representations elicited by Qwen3-14B. (a) Pairwise angular distances between mean representation centroids induced by role-playing prompts in eight languages. (b) 2D PCA visualization of last-layer hidden states.

Figure 5: Illustration of multilingual role-playing prompt composition. Different languages induce diverse representation patterns, and their composition provides a broader inference context.

Model	MMLU-Redux						MMLU
Model	Base	Role-playing	MLVP	SLVP-best	MLCP	SLCP-best	Base	Role-playing	MLVP	SLVP-best	MLCP	SLCP-best
Qwen3-4B	0.756	0.754	0.769	0.764	0.779	0.782	0.756	0.754	0.798	0.773	0.807	0.779
Qwen3-8B	0.764	0.767	0.790	0.779	0.822	0.819	0.782	0.782	0.821	0.801	0.847	0.815
Qwen3-14B	0.798	0.810	0.821	0.825	0.843	0.841	0.822	0.824	0.854	0.836	0.866	0.844
Qwen3-32B	0.834	0.852	0.850	0.849	0.868	0.865	0.841	0.856	0.872	0.856	0.884	0.862
Llama3-8B	0.539	0.524	0.525	0.529	0.545	0.522	0.611	0.618	0.619	0.614	0.621	0.594

Table 4: Performance comparison of different models and methods on MMLU-Redux and MMLU. SLVP-best and SLCP-best represent the best-performing language in Single-Language Voting Prediction and Single-Language Concatenate Prediction , respectively. Bold indicates the best-performing setting per model and benchmark.

Model	Benchmark	Improved $\uparrow$	Degraded $\downarrow$	Unchanged
Qwen3-4B	MMLU-Redux	14 $\boldsymbol{\rightarrow}$ 19	12 $\boldsymbol{\rightarrow}$ 9	4 $\boldsymbol{\rightarrow}$ 2
Qwen3-4B	MMLU	27 $\boldsymbol{\rightarrow}$ 42	26 $\boldsymbol{\rightarrow}$ 10	4 $\boldsymbol{\rightarrow}$ 5
Qwen3-8B	MMLU-Redux	16 $\boldsymbol{\rightarrow}$ 26	12 $\boldsymbol{\rightarrow}$ 2	2 $\boldsymbol{\rightarrow}$ 2
Qwen3-8B	MMLU	28 $\boldsymbol{\rightarrow}$ 49	22 $\boldsymbol{\rightarrow}$ 5	7 $\boldsymbol{\rightarrow}$ 3
Qwen3-14B	MMLU-Redux	19 $\boldsymbol{\rightarrow}$ 22	6 $\boldsymbol{\rightarrow}$ 4	5 $\boldsymbol{\rightarrow}$ 4
Qwen3-14B	MMLU	31 $\boldsymbol{\rightarrow}$ 39	17 $\boldsymbol{\rightarrow}$ 13	9 $\boldsymbol{\rightarrow}$ 5

Table 5: Statistics on the number of benchmark subjects with performance differences. Each entry follows the format

A\rightarrow B

A

counts subjects where role-playing outperforms/degrades/equals the base setting, while

B

counts subjects where MLCP outperforms/degrades/equals the base setting.

Figure 6: Performance comparison of Qwen3-8B on MLCP variant across 2–8 languages. The N denotes the number of mixed languages.

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: Multilingual role-playing prompts.

Panel of Table 6 — Table 6: Prompt template used for role-playing question answering throughout our experiments.

Panel of Table 7 — Table 7: Comparison of role-playing prompts with differently detailed character information (1/3).

Panel of Table 8 — Table 8: Comparison of role-playing prompts with differently detailed character information (2/3).

Panel of Table 9 — Table 9: Comparison of role-playing prompts with differently detailed character information (3/3).

Subject	Setting	Models
Subject	Setting	Mistral-7B	Llama3-8B	Qwen3-4B	Qwen3-8B	Qwen3-14B	Qwen3-32B
anatomy	Base	0.460	0.620	0.750	0.780	0.800	0.830
anatomy	Role-playing	0.490 (+3%)	0.650 (+3%)	0.660 (-9%)	0.720 (-6%)	0.810 (+1%)	0.850 (+2%)
astronomy	Base	0.610	0.680	0.860	0.910	0.940	0.950
astronomy	Role-playing	0.640 (+3%)	0.650 (-3%)	0.880 (+2%)	0.900 (-1%)	0.920 (-2%)	0.960 (+1%)
business_ethics	Base	0.590	0.600	0.820	0.790	0.840	0.860

Appendix

Table 10: Performance comparison of base vs. role-playing modes across different models and subjects on MMLU-Redux (Part 1/2).

Subject	Setting	Models
Subject	Setting	Mistral-7B	Llama3-8B	Qwen3-4B	Qwen3-8B	Qwen3-14B	Qwen3-32B
high_school_geography	Base	0.600	0.770	0.880	0.870	0.870	0.900
high_school_geography	Role-playing	0.610 (+1%)	0.720 (-5%)	0.820 (-6%)	0.870 (+0%)	0.900 (+3%)	0.910 (+1%)
high_school_macroeconomics	Base	0.380	0.460	0.770	0.810	0.890	0.900
high_school_macroeconomics	Role-playing	0.390 (+1%)	0.450 (-1%)	0.770 (+0%)	0.850 (+4%)	0.880 (-1%)	0.910 (+1%)
high_school_mathematics	Base	0.320	0.380	0.750	0.740	0.740	0.780

Appendix

Table 11: Performance comparison of base vs. role-playing modes across different models and subjects on MMLU-Redux (Part 2/2).

Subject	Setting	Models
Subject	Setting	Mistral-7B	Llama3-8B	Qwen3-4B	Qwen3-8B	Qwen3-14B	Qwen3-32B
abstract_algebra	Base	0.270	0.350	0.780	0.790	0.840	0.873
abstract_algebra	Role-playing	0.280 (+1.0%)	0.370 (+2.0%)	0.770 (-1.0%)	0.800 (+1.0%)	0.830 (-1.0%)	0.886 (+1.3%)
anatomy	Base	0.533	0.630	0.696	0.748	0.837	0.866
anatomy	Role-playing	0.570 (+3.7%)	0.659 (+3.0%)	0.667 (-3.0%)	0.726 (-2.2%)	0.800 (-3.7%)	0.879 (+1.3%)
astronomy	Base	0.651	0.658	0.842	0.928	0.901	0.934

Appendix

Table 12: Performance comparison of base vs. role-playing modes across different models and subjects on MMLU (Part 1/3).

Subject	Setting	Models
Subject	Setting	Mistral-7B	Llama3-8B	Qwen3-4B	Qwen3-8B	Qwen3-14B	Qwen3-32B
high_school_statistics	Base	0.431	0.421	0.810	0.843	0.884	0.903
high_school_statistics	Role-playing	0.454 (+2.3%)	0.417 (-0.5%)	0.838 (+2.8%)	0.843 (+0.0%)	0.898 (+1.4%)	0.914 (+1.1%)
high_school_us_history	Base	0.740	0.750	0.873	0.892	0.917	0.934
high_school_us_history	Role-playing	0.775 (+3.4%)	0.750 (+0.0%)	0.843 (-2.9%)	0.892 (+0.0%)	0.907 (-1.0%)	0.946 (+1.2%)
high_school_world_history	Base	0.772	0.806	0.823	0.861	0.886	0.904

Appendix

Table 13: Performance comparison of base vs. role-playing modes across different models and subjects on MMLU (Part 2/3).

Subject	Setting	Models
Subject	Setting	Mistral-7B	Llama3-8B	Qwen3-4B	Qwen3-8B	Qwen3-14B	Qwen3-32B
nutrition	Base	0.650	0.725	0.778	0.840	0.876	0.893
nutrition	Role-playing	0.621 (-2.9%)	0.755 (+2.9%)	0.784 (+0.7%)	0.827 (-1.3%)	0.866 (-1.0%)	0.905 (+1.2%)
philosophy	Base	0.637	0.617	0.701	0.714	0.775	0.803
philosophy	Role-playing	0.666 (+2.9%)	0.672 (+5.5%)	0.707 (+0.6%)	0.736 (+2.3%)	0.752 (-2.3%)	0.814 (+1.1%)
prehistory	Base	0.685	0.701	0.772	0.843	0.861	0.883

Appendix

Table 14: Performance comparison of base vs. role-playing modes across different models and subjects on MMLU (Part 3/3).

Model	Benchmark	Variant	en	zh	ja	km	ms	de	es	fr	avg
Qwen3-4B	MMLU-Redux	SLVP	0.741	0.730	0.751	0.729	0.751	0.758	0.764	0.760	0.748
	MMLU-Redux	SLCP	0.774	0.752	0.773	0.762	0.775	0.782	0.778	0.782	0.772
	MMLU	SLVP	0.760	0.738	0.769	0.765	0.770	0.772	0.773	0.773	0.765
	MMLU	SLCP	0.776	0.751	0.779	0.771	0.776	0.779	0.776	0.778	0.773
Qwen3-8B	MMLU-Redux	SLVP	0.755	0.751	0.769	0.754	0.773	0.774	0.779	0.776	0.766
	MMLU-Redux	SLCP	0.819	0.781	0.811	0.805	0.814	0.815	0.819	0.819	0.810

Appendix

Table 15: Performance of SLVP and SLCP on MMLU-Redux and MMLU benchmarks across 8 languages. For each row, the best result is highlighted.

LLM Prompting

Large language model prompting studies how instructions, examples, context, and formats steer model behavior at inference time.

See all 431 papers

Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs
Mar 14, 2026Kai Wang, Haoyang You, Yang Zhang +1Large Language Model-Based Role-Play Simulation LLM Prompting
Harbin Institute of Technology, China · Macquarie University, Australia
A Large-Scale Evaluation of Role-Enhanced Zero-Shot Chain-of-Thought Prompting
Oct 21, 2026Ibrahim Khan, Thai Van Nguyen, Yi Xia +5LLM Prompting Adaptive Reasoning in Language Models
College of Information Science and Technology, Ritsumeikan University, Japan · Graduate School of Information Science and Technology, Ritsumeikan University, Japan · Independent Researcher, and an alumnus of the Graduate School of Information Science and Technology, Ritsumeikan University, Japan +1
When Does Persona Prompting Actually Help? A Retrieval and Metric Analysis of Expert Role Injection in LLMs
May 28, 2026Shuai Xiao, Su Liu, Weikai Zhou +4LLM Evaluation LLM Prompting
Independent Researchers

Mar 14, 2026J/K move · Enter open · S save

Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs

Kai Wang, Haoyang You, Yang Zhang, Zhongjie Wang

Harbin Institute of Technology, China · Macquarie University, Australia

Large Language Model-Based Role-Play Simulation LLM Prompting Memory-Augmented Language Model Agents Multi-Turn Dialogue Evaluation

Open paper AskPDF arXiv

A core challenge for faithful LLM role-playing is sustaining consistent characterization throughout long, open-ended dialogues, as models frequently fail to recall and accurately apply their designated persona knowledge without explicit cues. To tackle this, we propose the Memory-Driven Role-Playing paradigm. Inspired by Stanislavski's "emotional memory" acting theory, this paradigm frames persona knowledge as the LLM's internal memory store, requiring retrieval and application based solely on dialogue context, thereby providing a rigorous test of depth and autonomous use of knowledge. Centered on this paradigm, we contribute: (1) MREval, a fine-grained evaluation framework assessing four memory-driven abilities - Anchoring, Recalling, Bounding, and Enacting; (2) MRPrompt, a prompting architecture that guides structured memory retrieval and response generation; and (3) MRBench, a bilingual (Chinese/English) benchmark for fine-grained diagnosis. The novel paradigm provides a comprehensive diagnostic for four-staged role-playing abilities across 12 LLMs. Crucially, experiments show that MRPrompt allows small models (e.g., Qwen3-8B) to match the performance of much larger closed-source LLMs (e.g., Qwen3-Max and GLM-4.7), and confirms that upstream memory gains directly enhance downstream response quality, validating the staged theoretical foundation.

Abstract

LLM Prompting

Explore similar work

Memory-Driven Role-Playing: Evaluation and Enhancement of Persona Knowledge Utilization in LLMs

A Large-Scale Evaluation of Role-Enhanced Zero-Shot Chain-of-Thought Prompting

When Does Persona Prompting Actually Help? A Retrieval and Metric Analysis of Expert Role Injection in LLMs