Designing Behavioral Profiles for Language Model Simulation of Out-of-Distribution Human Behavior
Organizations: Massachusetts Institute of Technology · Digital Economy Lab, Stanford University · Google DeepMind
Abstract
Large language models (LLMs) are increasingly used to simulate human behavior, but which descriptions of people improve prediction when experimental designs change remains unclear. We develop a framework that draws candidate behavioral features from the literature and constructs profiles from earlier human experiments. An LLM-based profile generator assigns feature values from observed actions, messages, and surveys; a separate LLM simulates behavior using these profiles. By comparing simulated and observed outcomes in the training data, we select profile contents through forward feature selection, adding features while they improve prediction. We then evaluate the selected contents without modification against outcomes from new participants under new designs. Across 616 public-goods games, 144 multi-party bargaining games, and 1,480 generalized 11--20 money-request designs, leading one-feature profiles predict human outcome distributions better than no profile or general-purpose personas. Profiles with more features provide no clear additional benefit, and features' relative predictive value largely persists across learning and validation. Using profiles from one game family to simulate another produces uneven gains and can make predictions less accurate than simulations without profiles. The framework makes the choice of behavioral information a testable part of simulation design, helping researchers assess which descriptions remain useful as experimental conditions change.
Figures & tables
| Game family | Profile represents | Designs for profile learning | Validation designs | Outcome |
|---|---|---|---|---|
| Public goods | Participant in one game | 366 game designs | 40 nonoverlapping designs | Normalized efficiency |
| Multi-party bargaining | Participant across two games and survey | Designs with two or three chip types | Designs with four chip types | Final surplus ratio |
| Generalized 11–20 | Weighted payoff-design–choice type | 596 payoff designs | 884 nonoverlapping designs | Normalized request |
| Game family | Leading feature | Second selected feature | Lowest-ranked feature |
|---|---|---|---|
| Public-goods games | Withdrawal after adverse peer behavior ( Fischbacher et al. 2001 ; Keser and van Winden 2000 ) Reduces later contributions after peer free riding, conflict, or another adverse social event. | Rule comprehension ( Houser and Kurzban 2002 ) Applies the game’s payoff and mechanism rules accurately. | Commitment making ( Kerr and Kaufman-Gilliland 1994 ) Makes or elicits explicit, nonbinding promises about future contributions. |
| Multi-party bargaining | Package breadth ( Qian et al. 2026 ; Chakraborty and Harbaugh 2003 ) Negotiates larger multi-chip packages rather than minimum-size exchanges. | Integrative trade search ( Chakraborty and Harbaugh 2003 ) Searches for packages that can create gains for both sides. | Profitable-offer uptake ( Güth et al. 1982 ) Accepts offers that clearly increase the participant’s induced value. |
| Generalized 11–20 money-request games | Payoff and incentive sensitivity ( McKelvey and Palfrey 1995 ; Manning and Horton 2026 ) Responds to the payoff schedule and its material tradeoffs. | Instruction comprehension and adherence ( Manning and Horton 2026 ) Chooses consistently with the stated rules and instruction checks. | Opponent-belief sophistication and hedging ( Arad and Rubinstein 2012 ; Goeree et al. 2018 ) Adjusts the request to uncertainty about the other player’s reasoning. |
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| ID | Feature and definition | Literature |
|---|---|---|
| P01 | Baseline cooperation propensity . Typical contribution level before accounting for responses to other participants’ later behavior. | Ledyard (1995) ; Fischbacher et al. (2001) |
| P02 | Costly contribution orientation . Willingness to sacrifice own immediate material payoff through contribution when contribution is privately costly. This is a behavioral tradeoff, not a claim that altruism, efficiency, warm glow, or conformity is the cause. | Ledyard (1995) ; Fischbacher et al. (2001) ; Andreoni (1990) ; Charness and Rabin (2002) |
| P03 | Opportunistic withholding . Withholding contribution while benefiting from substantial peer contribution, including late or situational exploitation. | Ledyard (1995) ; Fischbacher et al. (2001) ; Andreoni (1988) |
| P04 | Peer-responsive cooperation . Degree to which contribution changes in the same direction as observed peer cooperation or explicit cooperative coordination. | Fischbacher et al. (2001) ; Keser and van Winden (2000) |
| P05 | Threshold cooperation . Sharply changing contribution levels when peer cooperation crosses a salient level. | Fischbacher et al. (2001) |
| P06 | Contribution volatility . Within-game variability in contribution after accounting for clear peer-contingent responses. | Ledyard (1995) ; Chaudhuri (2011) |
| ID | Feature and definition | Literature |
|---|---|---|
| C01 | Own-surplus protection . Consistency with which choices protect or increase the participant’s payoff, calculated using their assigned private chip values. | Nash (1950) ; Murphy et al. (2011) |
| C02 | Integrative trade search . Tendency to construct or support packages that plausibly create gains for both sides using participant-visible evidence. | Nash (1950) ; Charness and Rabin (2002) ; Chakraborty and Harbaugh (2003) |
| C03 | Preference for equal exchange quantities . Preference for equal or nearly equal quantities of chips on both sides of an exchange, regardless of private chip values. | Güth et al. (1982) ; Kagel et al. (1996) ; Fehr and Schmidt (1999) |
| C04 | Relative-outcome concern . Concern with avoiding disadvantage or seeking an advantage relative to others, beyond absolute own surplus. | Fehr and Schmidt (1999) ; Bolton and Ockenfels (2000) ; Straub and Murnighan (1995) |
| C05 | Proposal claim level . How much the participant seeks to gain from proposed trades, given the available chips. | Nash (1950) ; Güth et al. (1982) ; Roth et al. (1991) |
| C06 | Acceptance-calibrated offer adaptation . Degree to which proposals become more likely to be accepted after observed responder feedback. | Rubinstein (1982) ; Güth et al. (1982) ; Straub and Murnighan (1995) |
| ID | Feature and definition | Literature |
|---|---|---|
| B01 | Strategic iteration depth . How strongly an observed request supports successive reasoning about the opponent’s response. | Arad and Rubinstein (2012) ; Nagel (1995) ; Camerer et al. (2004) |
| B02 | Opponent-belief sophistication and hedging . How strongly an observed request supports a response to uncertainty about the opponent’s choice. | Goeree et al. (2018) ; Arad and Rubinstein (2012) |
| B03 | Payoff and incentive sensitivity . The extent to which an observed request reflects the material payoff consequences of the stated rule. | McKelvey and Palfrey (1995) ; Manning and Horton (2026) |
| B04 | Upper-endpoint salience . The extent to which an observed request suggests attraction to the upper endpoint beyond what payoff incentives explain. | Arad and Rubinstein (2012) |
| B05 | Payoff-computation precision . How precisely an observed request follows a payoff-relevant best response or rule-specific target. | Goeree et al. (2018) ; McKelvey and Palfrey (1995) |
| B06 | Bonus pursuit versus security . The extent to which an observed request sacrifices secure base payoff to pursue the contingent bonus. | Li and Rong (2018) ; Manning and Horton (2026) |
| Family | Feature | Learning reduction | Validation reduction | Learning rank | Validation rank |
|---|---|---|---|---|---|
| Public goods | P08 ∗ | 0.1488 | 0.0604 | 1 | 5 |
| Public goods | P06 | 0.1432 | 0.0643 | 2 | 3 |
| Public goods | P01 | 0.1415 | 0.0642 | 3 | 4 |
| Public goods | P03 | 0.1391 | 0.0667 | 4 | 1 |
| Public goods | P12 | 0.1377 | 0.0439 | 5 | 9 |
| Public goods | P02 | 0.1345 | 0.0455 | 6 | 8 |
| Family | Current profile | Added feature | Comparison A | Comparison B | Combined / panel | Selected |
|---|---|---|---|---|---|---|
| Public goods | P08 | P13 | — | — | -0.0196 | Yes |
| Public goods | P08 | P19 | — | — | -0.0061 | No |
| Public goods | P08 | P15 | — | — | -0.0020 | No |
| Public goods | P08 | P28 | — | — | +0.0051 | No |
| Public goods | P08 | P23 | — | — | +0.0099 | No |
| Public goods | P08 | P10 | — | — | +0.0126 | No |
| Game family | Comparison | Units | Difference | 95% bootstrap interval |
|---|---|---|---|---|
| Public goods | No profile | 250 | ||
| Public goods | Leading two-feature profile | 250 | ||
| Public goods | Complete profile | 250 | ||
| Public goods | Nemotron persona | 250 | ||
| Public goods | Twin-2K-500 persona | 250 | ||
| Public goods | ANES persona | 250 |
| Family | Condition | Alignment [95% interval] | Usable outputs | Independent units |
|---|---|---|---|---|
| Public goods | Leading one-feature profile | 65.5 [37.6, 68.1] | 250 | 250 games |
| Public goods | Leading two-feature profile | 59.0 [34.5, 65.5] | 250 | 250 games |
| Public goods | Complete profile | 62.5 [37.9, 65.2] | 250 | 250 games |
| Public goods | Nemotron persona | -1.5 [-24.7, 7.4] | 250 | 250 games |
| Public goods | Twin-2K-500 persona | 18.6 [-5.7, 23.2] | 250 | 250 games |
| Public goods | ANES persona | -8.7 [-29.4, 0.8] | 250 | 250 games |
| Game family | Uninformed reference | Human sampling benchmark |
|---|---|---|
| Public goods | 0.258469438 | 0.090731881 |
| Bargaining | 0.545100531 | 0.057550213 |
| Generalized 11–20 | 0.188932147 | 0.022499968 |
| Family | Persona library | Equal-weight loss | Calibrated loss | Change [95% interval] |
|---|---|---|---|---|
| Public goods | Nemotron | 0.2609 | 0.2571 | |
| Twin-2K-500 | 0.2273 | 0.2273 | ||
| ANES | 0.2731 | 0.2652 | ||
| Bargaining | Nemotron | 0.3078 | 0.2692 | |
| Twin-2K-500 | 0.3333 | 0.3211 | ||
| ANES | 0.3162 | 0.2997 |
| Family used to construct profiles | Profile size | Usable games | Paired cohorts vs. no profile | |
|---|---|---|---|---|
| Public goods | One feature | 47 | 23 | 0.057963316 |
| Public goods | Two features | 46 | 22 | 0.058861741 |
| Public goods | Complete | 45 | 21 | 0.059357288 |
| Generalized 11–20 | One feature | 48 | 24 | 0.057550213 |
| Generalized 11–20 | Two features | 46 | 22 | 0.058666046 |
| Generalized 11–20 | Complete | 48 | 24 | 0.057550213 |
| Family | Generator / simulator | Features | Alignment scores | Difference vs. no profile | Difference vs. complete |
|---|---|---|---|---|---|
| Public goods | Mini/Mini | P08 | 65.5 / -1.0 / 62.5 | [ ] | [ ] |
| Public goods | 5.1/Mini | P06+P13 | 59.7 / -10.4 / 65.3 | [ ] | [ ] |
| Public goods | Mini/5.1 | P01 | 58.6 / -9.3 / 56.2 | [ ] | [ ] |
| Bargaining | 5.1/Mini | C07 | 77.5 / 40.5 / 53.1 | [ ] | [ ] |
| Bargaining | Mini/Mini | C02+C15 | 53.4 / 40.3 / 52.4 | [ ] | [ ] |
| Bargaining | Mini/5.1 | C08 | 95.2 / 91.6 / 103.8 | [ ] | [ ] |
| Family | Generic profile | Generic no profile (95% bootstrap interval) | Generic leading profile (95% bootstrap interval) |
|---|---|---|---|
| Public goods | Demographic | [ ] | [ ] |
| Public goods | Length-matched | [ ] | [ ] |
| Bargaining | Demographic | [ ] | [ ] |
| Bargaining | Length-matched | [ ] | [ ] |
| Generalized 11–20 | Demographic | [ ] | [ ] |
| Generalized 11–20 | Length-matched | [ ] | [ ] |
| Profile condition | Design-level Wasserstein distance | Log loss based on the simulated request |
|---|---|---|
| No profile | 0.2711 | 4.9486 |
| Leading one-feature profile | 0.3194 | 5.4579 |
| Leading two-feature profile | 0.3239 | 5.3828 |
| Complete profile | 0.3203 | 5.4259 |
Human Behavior Simulation
Human behavior simulation models how individuals and groups decide, interact, move, and adapt in social, economic, educational, and physical settings.
See all 249 papers