THEATRON: A Benchmark for Evaluating Persona Persistence in Long-Horizon LLM Decision-Making
Organizations: The University of Manchester · The Fin AI · ELLIS Manchester
Abstract
Existing evaluations often test persona expression rather than whether persona information persists in extended decision-making. We introduce THEATRON, a controlled benchmark that uses financial replay to evaluate persona persistence in long-horizon sequential decision-making. In these environments, models conditioned on fixed behavioral-finance personas observe shared daily market evidence and trajectory-specific account-state feedback and make sequential decisions over multi-month trajectories. We evaluate Zero-Shot, One-Shot, and Chain-of-Thought persona injection across four LLMs, using cumulative-return and action trajectories to diagnose within-persona stability, cross-persona separability, and action collapse. Persona conditioning produces systematic behavioral shifts beyond a no-persona baseline, and persona-level separation generally exceeds the variation induced by semantically equivalent descriptions. Nevertheless, these shifts do not reliably form coherent persona-specific policies: apparently stable separation can arise from stereotyped action collapse, while diverse policies can exhibit noisy trajectory switching. These findings expose a gap between observable persona-induced behavior and persistent, context-sensitive persona-conditioned decision policies. Code and benchmark data are publicly available at: https://github.com/yuyan12138/theatron
Figures & tables
| Model | Mtd. | Stock | Overconfidence | Loss Aversion | Herding | B/W (R/A) |
| GPT-5.4-mini | Zero-Shot | AAPL | ||||
| TSLA | ||||||
| One-Shot | AAPL | |||||
| TSLA | ||||||
| CoT | AAPL | |||||
| TSLA |
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | Mode | Maximum tokens | Seed semantics |
| GPT-5.4-mini | no explicit reasoning | 512 | best effort |
| Gemini-2.5- Flash | thinking disabled | 2,048 | best effort |
| Qwen3-8B | non- thinking | 512 | explicit |
| Qwen3-14B | thinking | 8,192 | explicit |
| Positive-result component | Operational evidence | Reported in |
| Beyond no-persona behavior | Exact persona–baseline action and return contrasts | Appendix C , F |
| Within-persona stability | , final-return SD, and intervals | Appendix D – H |
| Cross-persona separation | Return/action and B/W | Appendix E – H |
| Non-collapse and state sensitivity | Entropy, dominant actions, switches, and responses to price direction | Appendix D , J |
| Observable intended direction | Overconfidence activity, Herding alignment, and descriptive Loss-Aversion feedback | Appendix C , F |
| Model | Stock | Final CR (mean SD) | Entropy | BUY | HOLD | SELL |
| Qwen3-8B | AAPL | .173 | 1.0 | 96.5 | 2.5 | |
| Qwen3-8B | TSLA | .252 | 1.4 | 93.9 | 4.7 | |
| Qwen3-14B | AAPL | .690 | 6.5 | 75.9 | 17.6 | |
| Qwen3-14B | TSLA | .816 | 4.5 | 58.4 | 37.1 | |
| GPT-5.4-mini | AAPL | .393 | 4.9 | 90.0 | 5.1 | |
| GPT-5.4-mini | TSLA | .658 | 3.3 | 75.1 | 21.6 |
| Model | Stock | Task non-HOLD | Overconf. non-HOLD | Uplift | Task align. | Herding align. | Uplift |
| Qwen3-8B | AAPL | 3.5 | 100.0 | 96.5 | 3.6 | 64.4 | 60.8 |
| Qwen3-8B | TSLA | 6.1 | 100.0 | 93.9 | 5.9 | 72.7 | 66.7 |
| Qwen3-14B | AAPL | 24.1 | 99.8 | 75.7 | 16.6 | 72.3 | 55.6 |
| Qwen3-14B | TSLA | 41.6 | 100.0 | 58.4 | 32.7 | 85.5 | 52.9 |
| GPT-5.4-mini | AAPL | 10.0 | 61.4 | 51.4 | 9.1 | 63.8 | 54.7 |
| GPT-5.4-mini | TSLA | 24.9 | 74.9 | 50.0 | 22.2 | 78.0 | 55.8 |
| Model | Stock | Paired | Sign-flip | BH | |
| Qwen3-8B | AAPL | 5 | -.619 | .0625 | .100 |
| Qwen3-8B | TSLA | 4 | .331 | .3750 | .429 |
| Qwen3-14B | AAPL | 5 | -.223 | .0625 | .100 |
| Qwen3-14B | TSLA | 5 | -.277 | .0625 | .100 |
| GPT-5.4-mini | AAPL | 5 | -.000 | 1.0000 | 1.000 |
| GPT-5.4-mini | TSLA | 5 | .273 | .1250 | .167 |
| Model | Method | Overconfidence | Loss Aversion | Herding |
| GPT-5.4-mini | Zero-Shot | 1.03 | .59 | 1.03 |
| One-Shot | .81 | .56 | .89 | |
| CoT | .85 | .54 | .94 | |
| Gemini-2.5-Flash | Zero-Shot | .01 | .71 | .78 |
| One-Shot | .00 | .73 | .86 | |
| CoT | .01 | .73 | .77 |
| Model | Stock | Return B/W | Return | Action B/W | Action |
| GPT-5.4-mini | AAPL | 3.401 | .000114 | 6.111 | .000114 |
| GPT-5.4-mini | TSLA | 1.476 | .000200 | 4.098 | .000114 |
| Gemini-2.5-Flash | AAPL | 3.760 | .000114 | 11.147 | .000114 |
| Gemini-2.5-Flash | TSLA | 6.745 | .000114 | 6.422 | .000114 |
| Qwen3-8B | AAPL | 4.026 | .000114 | 26.479 | .000114 |
| Qwen3-8B | TSLA | 2.703 | .000114 | 14.881 | .000114 |
| Model | Scenario | Stock | Return B/W | Return | Action B/W | Action |
| GPT-5.4-mini | Positive | AAPL | 2.264 | .000133 | 3.989 | .000133 |
| Positive | TSLA | 2.055 | .000133 | 3.515 | .000133 | |
| Negative | AAPL | 1.212 | .035410 | 4.242 | .000133 | |
| Negative | TSLA | 2.099 | .000686 | 2.916 | .000133 | |
| Gemini-2.5-Flash | Positive | AAPL | 2.623 | .000474 | 5.155 | .000133 |
| Positive | TSLA | 4.413 | .000133 | 6.715 | .000133 |
| Scenario | Overconfidence | Loss Aversion | Herding |
| Mixed evidence | 65.5–100.0 | 50.9–88.5 | 42.0–55.1 |
| Positive evidence | 67.4–92.7 | 0.4–5.2 | 28.5–40.8 |
| Negative evidence | 15.2–41.3 | 63.5–89.3 | 33.5–50.0 |
| Model | Condition | Final CR (mean SD) | BUY | HOLD | SELL | Entropy |
| Qwen3-8B | No-persona | 1.8 | 98.2 | 0.0 | .089 | |
| Overconfidence | 99.4 | 0.6 | 0.0 | .031 | ||
| Loss Aversion | 0.0 | 95.8 | 4.2 | .170 | ||
| Herding | 33.9 | 63.0 | 3.2 | .730 | ||
| Qwen3-14B | No-persona | 5.3 | 94.5 | 0.2 | .217 | |
| Overconfidence | 93.9 | 5.9 | 0.2 | .235 |
| Overconfidence non-HOLD uplift | Herding alignment uplift | |||||
| Model | AAPL | TSLA | BMRN | AAPL | TSLA | BMRN |
| Qwen3-8B | .964 | .939 | .976 | .608 | .667 | .267 |
| Qwen3-14B | .754 | .586 | .885 | .556 | .529 | .935 |
| GPT-5.4-mini | .515 | .499 | .503 | .547 | .558 | .709 |
| Gemini-2.5-Flash | .414 | .226 | .911 | .442 | .323 | .913 |
| Model | Persona | Return | Return | Action | Action |
| Qwen3-8B | Overconfidence | .0974 | .00794 | .9901 | .00794 |
| Loss Aversion | .0254 | .00794 | .0327 | .00794 | |
| Herding | .0354 | .00794 | .8139 | .00794 | |
| Qwen3-14B | Overconfidence | .0946 | .00794 | .7817 | .00794 |
| Loss Aversion | .0386 | .00794 | .6209 | .00794 | |
| Herding | .0141 | .00794 | .8492 | .00794 |
| Model | Stock | Metric | 95% CI for | ||||
| GPT-5.4-mini | AAPL | Return | .0652 | .0687 | .0887 | 1.29 | [.0079,.0270] |
| AAPL | Action | .1631 | .2313 | .4771 | 2.06 | [.2261,.2628] | |
| TSLA | Return | .1055 | .1223 | .1488 | 1.22 | [.0075,.0432] | |
| TSLA | Action | .1793 | .2400 | .4278 | 1.78 | [.1750,.1978] | |
| Gemini-2.5-Flash | AAPL | Return | .0436 | .0757 | .1811 | 2.39 | [.0910,.1162] |
| AAPL | Action | .1122 | .2163 | .6673 | 3.09 | [.4439,.4602] |
| Return B/W | Entropy | ||||
| Method | AAPL | TSLA | Overconf. | Loss Av. | Herding |
| Zero-Shot | 2.61 | 12.93 | .26 | .45 | .77 |
| One-Shot | 4.03 | 2.70 | .11 | .36 | .67 |
| CoT | 2.25 | 2.60 | .32 | .49 | .79 |
| SFT | 4.44 | 3.40 | .58 | .00 | .86 |
| Date | Price | Action | Cumulative return |
| 2025-01-02 | 379.28 | buy | 0.00% |
| 2025-01-03 | 410.44 | buy | 7.90% |
| 2025-01-06 | 411.05 | buy | 8.04% |
| 2025-01-07 | 394.36 | buy | 3.90% |
| 2025-01-08 | 394.94 | buy | 4.05% |
| 2025-01-10 | 394.74 | buy | 4.00% |
LLM Decision-Making
LLM decision-making studies how language models choose, act, seek information, and allocate resources under goals, uncertainty, constraints, and social pressures.
See all 279 papers