cs.CLNov 5, 2026

Structuring Survey Histories for LLM Digital Twins: A Controlled Study of Background, Decision Procedure, and Evaluation Profiles

Authors: Iris Ye, Tianze Deng, Ozan Candogan

Organizations: Booth School of Business, University of Chicago.

Abstract

LLM-based digital twins aim to predict how individuals would respond in new settings from representations of their prior responses. Existing evidence that compression of survey histories need not reduce predictive accuracy suggests that representation structure, rather than information volume alone, may be an important design choice. We study BDE, a fixed representation that organizes each respondent's survey history into Background, Decision procedure, and Evaluation. We evaluate it on Twin-2K-500 (Toubia et al., 2025) by predicting each respondent's held-out answers on 17 tasks. In a paired evaluation with 100 respondents, a fully inferred BDE profile improves overall accuracy over the raw survey transcript by 1.27 percentage points (pp) on gpt-5.4-nano (p < 0.01) and 1.19 pp on gpt-5.4-mini (p < 0.001). Three analyses then examine the schema, extraction depth, and component composition. Against a length-matched free-form profile written from the same transcript by the same extractor, BDE improves pricing accuracy on both simulators (2.15 pp on nano, p < 0.01; 1.38 pp on mini, p < 0.05), while its overall advantage is significant only on nano (1.05 pp, p < 0.05). Allowing deeper inference during extraction yields no statistically significant gain on either simulator. By contrast, adding Evaluation to a Background--Decision procedure profile improves overall accuracy by 1.44 pp on nano (p < 0.001) and 1.22 pp on mini (p < 0.001). These results indicate that the advantage over raw transcripts is not explained by compression or length alone, and attribute it primarily to profile composition rather than deeper extraction.

Figures & tables

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Jun 13, 2026

From Survey Histories to Interpretable Digital Twins: Auto-Discovering Task-Specific Personas and Behavioral Mechanisms

LLM-based "digital twins" aim to simulate how an individual would behave in new environments or respond to novel questions, given some representation of that individual's prior responses. We ask whether long survey histories can be transformed into compact, task-specific personas that remain predictive for unseen respondents and products within the same task while making simulated decisions more interpretable. We introduce an iterative auto-discovery pipeline that constructs persona representations and learns how persona evidence influences simulated responses. We evaluate the procedure on the 40-item Product Preferences--Pricing task of Twin-2K-500 (Toubia et al., 2025), learning from 20 respondents and 20 products and evaluating the frozen representation and mechanism on 30 unseen respondents and 20 unseen products. The holdout answers come from two survey waves, and every prediction is scored against each wave separately. For gpt-5.4-nano, replacing the full survey transcript with the learned channel representation and mechanism improves exact accuracy from 0.573 to 0.623 and from 0.558 to 0.626 in the two waves. Larger and cross-provider simulators (gpt-5.4, gpt-5-mini, qwen3-8b) using the full transcript do not achieve statistically significant improvements over the baseline in both waves, and none significantly outperforms the learned representation in either wave. This pattern shows that the learned representation produces consistent gains against human responses collected in both waves while retaining an explicit mapping from respondent evidence to predicted decisions.
Jun 3, 2026

Synthetic Personalities: How Well Can LLMs Mimic Individual Respondents Using Socio-Economic Microdata?

LLM-based digital twins promise to scale and accelerate market research, but most published twins are either coarse persona bots conditioned on a few demographic questions or detailed individual-level twins built on purpose-collected surveys and interview transcripts. Neither setup speaks to the operationally most relevant case for marketing practice: building detailed individual twins from the pre-existing heterogeneous panel data that firms already accumulate through CRM systems, loyalty programs, and repeat surveys. We construct detailed individual-level twins from the German Socio-Economic Panel (SOEP) and evaluate them across a 3×5×2×23 \times 5 \times 2 \times 2 construction-method grid that covers three open-weights LLMs, five cumulative information depths ranked by normalized Shannon entropy, two embedding methods, and two reasoning modes, scoring over 2.1 million twin responses on 500 participants and 183 held-out questions. Twin quality rises with information depth but with diminishing returns past the 75 percent entropy quartile, which acts as a cost-efficient Pareto point relative to the best-performing 100 percent cells. Switching the embedding from a narrative persona summary to a raw dialog history of past responses raises hold-out accuracy in every model-by-reasoning cell at the 100 percent depth, while an explicit thinking mode raises rank-order correlation without moving accuracy. Best-cell accuracy reaches 78.8 percent and Fisher-zz correlation reaches r=0.590r = 0.590 on the SOEP held-out evaluation set. The findings suggest that twin-based market research is no longer gated by data design, but by item volume, model selection, and a small set of construction-level decisions that this paper now maps.
Dec 22, 2025

Psychometric Comparability of LLM-Based Digital Twins

Large language models (LLMs) act as digital twins for human respondents, yet their psychometric comparability remains uncertain. We propose a construct validity framework spanning construct representation and the nomothetic span, benchmarking models against human gold standards. Across studies, digital twins achieved high aggregate-level accuracy and profile correlations, but showed attenuated item-level correlations. In word association tests, LLM networks exhibited humanlike small-world structure and theory-consistent communities, yet diverged lexically and in local structure. In decision-making and contextualized tasks, they under-reproduced heuristic biases, demonstrating normative rationality, compressed variance, and limited temporal sensitivity. Feature-rich and trait relevant conditioning improved Big Five personality prediction and nomothetic-span alignment, but network invariance remained limited, with partial configural solutions and persistent loading differences. In cross-language free-text tasks in English and Chinese, feature-rich digital twins better approximated construct-level narrative content, but linguistic and idiographic differences persisted. These findings clarify that digital twins are most useful within validated boundaries, where the construct, task and level of inference align with evidence from human data.