Comparative reversal learning reveals rigid adaptation in LLMs under non-stationary uncertainty
Organizations: Insight Research Ireland Centre for Data Analytics, Ireland · School of Computing, Dublin City University, Ireland
Abstract
Non-stationary environments require agents to revise previously learned action values when contingencies change. We treat large language models (LLMs) as sequential decision policies in a two-option probabilistic reversal-learning task with three latent states and switch events triggered by either a performance criterion or timeout. We compare a deterministic fixed transition cycle to a stochastic random schedule that increases volatility, and evaluate DeepSeek-V3.2, Gemini-3, and GPT-5.2, with human data as a behavioural reference. Across models, win-stay was near ceiling while lose-shift was markedly attenuated, revealing asymmetric use of positive versus negative evidence. DeepSeek-V3.2 showed extreme perseveration after reversals and weak acquisition, whereas Gemini-3 and GPT-5.2 adapted more rapidly but still remained less loss-sensitive than humans. Random transitions amplified reversal-specific persistence across LLMs yet did not uniformly reduce total wins, demonstrating that high aggregate payoff can coexist with rigid adaptation. Hierarchical reinforcement-learning (RL) fits indicate dissociable mechanisms: rigidity can arise from weak loss learning, inflated policy determinism, or value polarisation via counterfactual suppression. These results motivate reversal-sensitive diagnostics and volatility-aware models for evaluating LLMs under non-stationary uncertainty.
Figures & tables
| Provider | Model | API model ID | Temperature | top- |
|---|---|---|---|---|
| DeepSeek | DeepSeek-V3.2 | deepseek-chat | 1.0 | 1.0 |
| Gemini-3-Flash | gemini-3-flash-preview | 1.0 | 1.0 | |
| OpenAI | GPT-5.2 | gpt-5.2 | 1.0 | 1.0 |
| Agent | Schedule | Lose-shift | Perseveration | Post-rev. regret | Total wins |
|---|---|---|---|---|---|
| DeepSeek-V3.2 | Fixed | 0.035 | 19.63 | 811.73 | 84.06 |
| DeepSeek-V3.2 | Random | 0.038 | 27.39 | 459.69 | 117.58 |
| Gemini-3 | Fixed | 0.235 | 7.24 | 403.43 | 126.11 |
| Gemini-3 | Random | 0.210 | 10.05 | 316.82 | 136.10 |
| GPT-5.2 | Fixed | 0.241 | 7.34 | 459.27 | 119.98 |
| GPT-5.2 | Random | 0.209 | 10.90 | 369.23 | 131.13 |
| Agent | Schedule | ||||
|---|---|---|---|---|---|
| Dual RL | |||||
| DeepSeek-V3.2 | Fixed | 0.421 (0.097) | 0.026 (0.030) | 3.342 (0.769) | – |
| DeepSeek-V3.2 | Random | 0.143 (0.092) | 0.030 (0.022) | 7.519 (0.696) | – |
| Gemini-3 | Fixed | 0.250 (0.016) | 0.260 (0.020) | 7.421 (0.070) | – |
| Gemini-3 | Random | 0.252 (0.017) | 0.259 (0.022) | 8.456 (0.041) | – |
| GPT-5.2 | Fixed | 0.202 (0.014) | 0.117 (0.012) | 5.876 (0.201) | – |
Non-Stationary RL
Non-stationary reinforcement learning studies how agents learn and act as rewards, dynamics, or task distributions change over time.
See all 63 papers