cs.AIOct 31, 2026

Designing Behavioral Profiles for Language Model Simulation of Out-of-Distribution Human Behavior

Authors: Robin Na, Kehang Zhu, Logan Cross, Michiel A. Bakker

Organizations: Massachusetts Institute of Technology · Digital Economy Lab, Stanford University · Google DeepMind

Abstract

Large language models (LLMs) are increasingly used to simulate human behavior, but which descriptions of people improve prediction when experimental designs change remains unclear. We develop a framework that draws candidate behavioral features from the literature and constructs profiles from earlier human experiments. An LLM-based profile generator assigns feature values from observed actions, messages, and surveys; a separate LLM simulates behavior using these profiles. By comparing simulated and observed outcomes in the training data, we select profile contents through forward feature selection, adding features while they improve prediction. We then evaluate the selected contents without modification against outcomes from new participants under new designs. Across 616 public-goods games, 144 multi-party bargaining games, and 1,480 generalized 11--20 money-request designs, leading one-feature profiles predict human outcome distributions better than no profile or general-purpose personas. Profiles with more features provide no clear additional benefit, and features' relative predictive value largely persists across learning and validation. Using profiles from one game family to simulate another produces uneven gains and can make predictions less accurate than simulations without profiles. The framework makes the choice of behavioral information a testable part of simulation design, helping researchers assess which descriptions remain useful as experimental conditions change.

Figures & tables

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Predicting Effects, Missing Distributions: Evaluating LLMs as Human Behavior Simulators in Operations Management

    Sep 30, 2025Runze Zhang, Xiaowei Zhang, Mingyang ZhaoLLM EvaluationHuman Behavior Simulation

  2. How Well Do Large Language Models Capture Human Personality?

    May 12, 2026Aanisha Bhattacharyya, Yaman Kumar Singla, Rajiv Ratn Shah +2Persona-Conditioned GenerationRepresentation Collapse

  3. Large language models replicate and predict human cooperation across experiments in game theory

    Nov 6, 2025Andrea Cera Palatsi, Samuel Martin-Gutierrez, Ana S. Cardenal +1LLM EvaluationGame Theory