Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data

Minghao Guo; Ziyi Ye; Wujiang Xu; Xi Zhu; Wenyue Hua; Dimitris N. Metaxas

cs.CLMar 1, 2026

Individual Turing Test: A Case Study of LLM-based Simulation Using Longitudinal Personal Data

Authors: Minghao Guo, Ziyi Ye, Wujiang Xu, Xi Zhu, Wenyue Hua, Dimitris N. Metaxas

Organizations: Rutgers University, New Brunswick, NJ, USA · Fudan University, Shanghai, China · Microsoft, New York, NY, USA

Abstract

Large Language Models (LLMs) have demonstrated remarkable human-like capabilities, yet their ability to replicate a specific individual remains under-explored. This paper presents a case study to investigate LLM-based individual simulation with a volunteer-contributed archive of private messaging history spanning over ten years. Based on the messaging data, we propose the "Individual Turing Test" to evaluate whether acquaintances of the volunteer can correctly identify which response in a multi-candidate pool most plausibly comes from the volunteer. We investigate prevalent LLM-based individual simulation approaches including: fine-tuning, retrieval-augmented generation (RAG), memory-based approach, and hybrid methods that integrate fine-tuning and RAG or memory. Empirical results show that current LLM-based simulation methods do not pass the Individual Turing Test, but they perform substantially better when the same test is conducted on strangers to the target individual. Additionally, while fine-tuning improves the simulation in daily chats representing the language style of the individual, retrieval-augmented and memory-based approaches demonstrate stronger performance on questions involving personal opinions and preferences. These findings reveal a fundamental trade-off between parametric and non-parametric approaches to individual simulation with LLMs when given a longitudinal context.

Figures & tables

Method	Response Content	Avg. Rank
Ground Truth	Big cities. You get to meet all kinds of people and be exposed to a lot more.	2.00
A-Mem+LoRA	Small cities are too slow for me. I can’t really live like that. I’d rather go out and work elsewhere.	2.33
RAG+LoRA	Probably the opportunities. Big cities just offer more possibilities.	3.17
LoRA	Ah, that’s a tough one. Both have their own appeal! What about you?	3.33
RAG+Base	I do envy how many opportunities big cities have. Do you like the quiet of smaller towns? Of course, competition in big cities is intense too. What about you?	3.50
A-Mem+Base	I lean toward the opportunities in big cities—there’s simply more. But sometimes I also want to slow down and enjoy the quiet of a small town.	3.67

Table 1. Case Study: Qualitative Comparison of Responses to “Are you more drawn to the opportunities in big cities, or the slower pace of life in smaller cities?” . Avg. Rank denotes the mean ranking position assigned by human judges (lower is better).

Panel of Figure 1 — Figure 1. Comparison between General and Individual Turing Tests. We report selection rates for each method under two evaluation settings: (a) General Turing Test , where responses are judged by strangers, and (b) Individual Turing Test , where responses are judged by acquaintances of the target individual. Results are stacked by prompt type, distinguishing daily conversations and personal opinion prompts.

Method	BLEU-1 $\uparrow$	BLEU-2 $\uparrow$	ROUGE-L $\uparrow$	Prec. $\uparrow$	Rec. $\uparrow$	Dist-2 $\uparrow$
LoRA-only	0.1578	0.0501	0.1496	0.1565	0.2128	0.7171
RAG+Base	0.1011	0.0264	0.1089	0.1005	0.2949	0.6407
A-Mem+Base	0.1113	0.0309	0.1228	0.1109	0.3613	0.6433
RAG+LoRA	0.1837	0.0735	0.1456	0.1776	0.1714	0.7680
A-Mem+LoRA	0.1983	0.1175	0.1457	0.1649	0.1586	0.7881

Table 2. Selected automated metrics (diagnostic). Hybrid methods (LoRA + retrieval/memory) consistently outperform LoRA-only and retrieval/memory-only baselines.

Panel of Figure 2 — Figure 2. Effect of expanding recent memory on hybrid simulation (A-Mem + LoRA). $Y_{i}$ denotes using the most recent $i$ years of dialogue history (e.g., $Y_{1}$ = most recent year only; $Y_{10}$ = most recent ten years).

Large Language Model-Based Role-Play Simulation

Large language model-based role-play simulation creates interactive agents that portray people, characters, or users for training, testing, and research.

See all 107 papers

PrivacySIM: Evaluating LLM Simulation of User Privacy Behavior
May 12, 2026James Flemings, Murali AnnavaramLLM Evaluation User Modeling
University of Southern California
Mind the Gaps: Mixture-of-Minds for Human Simulation
Aug 6, 2026Pranav DahiyaMixture-of-Experts Language Models Human Behavior Simulation
Semilattice
Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation
Oct 19, 2026Jiajie He, Yuanhong Jiang, Xintong Chen +2Human Behavior Simulation User Simulation
Hithink Research · McMaster University · Nanyang Technological University

May 12, 2026J/K move · Enter open · S save

PrivacySIM: Evaluating LLM Simulation of User Privacy Behavior

James Flemings, Murali Annavaram

University of Southern California

LLM Evaluation User Modeling User Simulation

Open paper AskPDF arXiv

Large language models (LLMs) are increasingly used to simulate human behavior, but their ability to simulate

individual

privacy decisions is not well understood. In this paper, we address the problem of evaluating whether a core set of user persona attributes can drive LLMs to simulate individual-level privacy behavior. We introduce PrivacySIM, an evaluation suite that benchmarks LLM simulation of user privacy behavior against the ground-truth responses of 1,000 users. These users are drawn from five published user studies on privacy spanning LLM healthcare consultations, conversational agents, and chatbots. Drawing on these user studies, we hypothesize three persona facets as plausible predictors of privacy decision-making: demographics, previous experiences, and stated privacy attitudes. We condition nine frontier LLMs on subsets of these three facets and measure how often each model's response to a data-sharing scenario matches the user's actual response. Our findings show that (1) privacy persona conditioning consistently improves simulation quality over no-persona conditioning, but even the strongest model (40.4% accuracy) remains far from faithfully simulating individual privacy decisions. (2) A user's stated privacy attitudes alone may not be the best predictor because they often diverge from the user's actual privacy behavior. (3) Users with high AI/chatbot experience but low stated privacy attitudes are the most challenging to simulate. PrivacySIM is a first step toward understanding and improving the capabilities of LLMs to simulate user privacy decisions. We release PrivacySIM to enable further evaluation of LLM privacy simulation.

Abstract

Large Language Model-Based Role-Play Simulation

Explore similar work

PrivacySIM: Evaluating LLM Simulation of User Privacy Behavior

Mind the Gaps: Mixture-of-Minds for Human Simulation

Beyond Behavioral Accuracy: Evaluating Individual-Level Fidelity in LLM Investor Simulation