More Than Just Style: Benchmarking Personality Influence in Large Language Models
Organizations: University of Exeter · University of Edinburgh · Nanchang University
Abstract
In addition to the effects of explicit input prompts and instructions, large language model outputs are influenced by underlying behavioral preferences similar to human personality tendencies. These tendencies can affect how models interpret situations and make decisions. However, their influence is difficult to disentangle from the effects of model capabilities and instructions. Therefore, we introduce SPACE, a benchmark framework combining a controlled evaluation protocol with complementary datasets for personality verification, capability assessment, and hierarchical situational tasks. Controlled comparisons across guidance conditions help distinguish personality influence from measured capability changes and immediate instruction effects. Experimental results demonstrate that although prompting can shift generation style, models' internal personality tendencies are persistent and not readily overridden by opposing instructions. Furthermore, these underlying tendencies have an observable influence on LLM task performance, highlighting the need for greater attention to these characteristics and potential applications. Training code and SPACE benchmark specifications are available at https://github.com/Niklauseik/SPACE-training.
Figures & tables
| Level | Task focus | Dataset / task | Description |
| Situational interpretation | Information interpretation; situational judgments | Circa ( Louis et al., 2020 ) | Indirect answers |
| CommitmentBank ( de Marneffe et al., 2019 ) | Speaker commitment | ||
| FOMC ( Shah et al., 2023 ) | Monetary policy stance | ||
| Individual decision making | Action selection; rewards, costs, time, and uncertainty | Choices13k ( Peterson et al., 2021 ) | Lottery choices |
| Transportation | Time and cost | ||
| Machine selection | Limited reward observations |
| Model | Base | F guided | T guided |
|---|---|---|---|
| Llama 3.2 3B | 63.0 | 82.6 / 93.5 | 39.1 / 8.7 |
| Qwen2.5 3B | 39.1 | 54.3 / 87.0 | 30.4 / 6.5 |
| Qwen2.5 7B | 47.8 | 56.5 / 95.7 | 41.3 / 2.2 |
| Llama 3.1 8B | 43.5 | 58.7 / NA | 39.1 / NA |
| Model | ARC-Easy | BoolQ | GSM8K | |
|---|---|---|---|---|
| Llama 3.2 3B | T | −0.55 / −3.70 | −2.25 / −0.80 | −0.69 / −1.54 |
| F | −0.15 / −5.65 | +1.65 / +0.20 | −0.85 / −1.85 | |
| Qwen2.5 3B | T | −0.20 / −1.95 | +0.30 / −0.35 | −0.54 / −0.31 |
| F | −0.10 / −2.00 | −0.85 / −0.25 | +0.77 / −0.23 | |
| Qwen2.5 7B | T | −0.05 / −0.85 | +0.20 / −4.05 | +0.46 / −0.23 |
| F | −0.15 / −1.35 | −0.20 / −3.85 | −0.62 / −0.23 |
| Situational level | Retained | Reversed |
|---|---|---|
| Situational interpretation | 78.6% | 21.4% |
| Individual decision making | 68.8% | 31.3% |
| Interactive decision making | 78.1% | 21.9% |
| Overall | 73.9% | 26.1% |
Appendix figures & tables58 assets
Supplementary material from the paper’s appendix.
Appendix
| Behavioral level | Task version | Inputs per condition |
| Interpretation | Circa | 32,602 |
| CommitmentBank | 1,009 | |
| FOMC | 2,421 | |
| FiQA-SA / SST-2 / News | 1,173 / 10,000 / 11,931 | |
| Individual decisions | Choices13k (unequal payoff variances) | 1,764 |
| Transportation | 210 |
| Item | Value |
|---|---|
| GPU model | NVIDIA GH200 |
| GPUs per job | 1 |
| Computing platform | Isambard-AI HPC |
| Training GPU-hours | |
| Evaluation GPU-hours | |
| Total GPU-hours |
| Model | Condition | ARC-Easy | BoolQ | GSM8K |
|---|---|---|---|---|
| Llama 3B | Base | 82.30 | 73.50 | 79.23 |
| T | 81.75 | 71.25 | 78.54 | |
| F | 82.15 | 75.15 | 78.38 | |
| Qwen 3B | Base | 89.00 | 79.60 | 11.54 |
| T | 88.80 | 79.90 | 11.00 | |
| F | 88.90 | 78.75 | 12.31 |
| Model | Condition | ARC-Easy | BoolQ | GSM8K |
|---|---|---|---|---|
| Llama 3B | Base P | 82.25 | 73.50 | 79.23 |
| Prompt T | 78.55 | 72.70 | 77.69 | |
| Prompt F | 76.60 | 73.70 | 77.38 | |
| Qwen 3B | Base P | 89.00 | 79.60 | 11.77 |
| Prompt T | 87.05 | 79.25 | 11.46 | |
| Prompt F | 87.00 | 79.35 | 11.54 |
| Model | Condition | F | T |
|---|---|---|---|
| Llama 3B | T | 39.1 | 60.9 |
| F | 82.6 | 17.4 | |
| T+T | 6.5 | 93.5 | |
| T+F | 91.3 | 8.7 | |
| F+T | 10.9 | 89.1 | |
| F+F | 93.5 | 6.5 |
| Model | Condition | ARC-Easy | BoolQ | GSM8K |
|---|---|---|---|---|
| Llama 3B | T | 82.05 | 68.85 | 35.00 |
| F | 81.70 | 76.05 | 13.08 | |
| T+T | 78.20 | 67.45 | 54.92 | |
| T+F | 77.60 | 69.05 | 54.31 | |
| F+T | 80.10 | 75.20 | 55.62 | |
| F+F | 75.85 | 75.90 | 52.77 |
| Dataset | Model | Accuracy (%) Base / T / F | Macro F1 Base / T / F |
|---|---|---|---|
| Circa | Llama 3.2 3B | 58.54 / 70.52 / 23.60 | 0.4914 / 0.5617 / 0.2630 |
| Circa | Qwen2.5 3B | 12.10 / 13.17 / 8.62 | 0.1581 / 0.1690 / 0.1062 |
| Circa | Qwen2.5 7B | 27.27 / 29.30 / 23.95 | 0.3270 / 0.3498 / 0.2953 |
| Circa | Llama 3.1 8B | 27.54 / 41.77 / 13.13 | 0.2597 / 0.3743 / 0.1514 |
| CommitmentBank | Llama 3.2 3B | 48.02 / 45.57 / 55.01 | 0.3131 / 0.2654 / 0.4033 |
| CommitmentBank | Qwen2.5 3B | 36.01 / 40.79 / 25.99 | 0.3265 / 0.3561 / 0.2445 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | UNCLEAR | 23.90 | 2.66 | 72.40 |
| Llama 3B | NO | 19.76 | 45.06 | 0.44 |
| Llama 3B | YES | 41.60 | 37.06 | 17.59 |
| Llama 3B | CONDITIONAL | 14.74 | 15.22 | 9.57 |
| Llama 3B | None | 0.00 | 0.00 | 0.00 |
| Llama 8B | UNCLEAR | 27.85 | 10.72 | 68.35 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | FALSE | 84.24 | 92.86 | 55.90 |
| Llama 3B | TRUE | 15.76 | 7.14 | 43.11 |
| Llama 3B | UNCERTAIN | 0.00 | 0.00 | 0.99 |
| Llama 8B | FALSE | 12.39 | 30.82 | 1.19 |
| Llama 8B | TRUE | 8.72 | 20.22 | 2.68 |
| Llama 8B | UNCERTAIN | 78.89 | 48.96 | 96.13 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | NEUTRAL | 53.61 | 46.22 | 64.23 |
| Llama 3B | DOVISH | 45.39 | 52.29 | 35.11 |
| Llama 3B | HAWKISH | 0.99 | 1.49 | 0.66 |
| Llama 8B | NEUTRAL | 83.97 | 76.04 | 90.13 |
| Llama 8B | DOVISH | 6.86 | 11.11 | 2.60 |
| Llama 8B | HAWKISH | 9.17 | 12.85 | 7.27 |
| Task version | Focal label | Retained | Reversed |
|---|---|---|---|
| CommitmentBank | FALSE | 87.5% | 12.5% |
| Circa | UNCLEAR | 75.0% | 25.0% |
| FOMC | HAWKISH | 62.5% | 37.5% |
| FiQA-SA | positive | 100.0% | 0.0% |
| SST-2 | positive | 83.3% | 16.7% |
| News | bullish | 66.7% | 33.3% |
| Task | Model | Variant | Base | Trained | Prompt | Opposed | Outcome |
|---|---|---|---|---|---|---|---|
| CommitmentBank | Llama 3B | T | 84.24 | 92.86 | 66.20 | 89.40 | Retained |
| CommitmentBank | Llama 3B | F | 84.24 | 55.90 | 83.55 | 50.74 | Retained |
| CommitmentBank | Llama 8B | T | 12.39 | 30.82 | 25.17 | 54.01 | Retained |
| CommitmentBank | Llama 8B | F | 12.39 | 1.19 | 45.39 | 2.87 | Retained |
| CommitmentBank | Qwen 3B | T | 25.57 | 31.81 | 26.16 | 32.61 | Retained |
| CommitmentBank | Qwen 3B | F | 25.57 | 13.97 | 29.34 | 20.02 | Retained |
| Task | Model | Variant | Base | Trained | Prompt | Opposed | Outcome |
|---|---|---|---|---|---|---|---|
| Risk choice | Llama 3B | T | 54.48 | 50.68 | 22.96 | 22.96 | Retained |
| Risk choice | Llama 3B | F | 54.48 | 69.73 | 22.90 | 24.38 | Reversed |
| Risk choice | Llama 8B | T | 48.53 | 47.22 | 41.50 | 39.34 | Retained |
| Risk choice | Llama 8B | F | 48.53 | 69.78 | 29.65 | 46.03 | Reversed |
| Risk choice | Qwen 3B | T | 71.15 | 67.74 | 75.28 | 74.09 | Reversed |
| Risk choice | Qwen 3B | F | 71.15 | 75.45 | 72.79 | 74.32 | Retained |
| Task | Model | Variant | Base | Trained | Prompt | Opposed | Outcome |
|---|---|---|---|---|---|---|---|
| INSPIRED | Llama 3B | T | 21.01 | 5.88 | 5.88 | 0.84 | Retained |
| INSPIRED | Llama 3B | F | 21.01 | 74.79 | 10.08 | 56.30 | Retained |
| INSPIRED | Llama 8B | T | 36.13 | 20.17 | 78.15 | 63.03 | Reversed |
| INSPIRED | Llama 8B | F | 36.13 | 84.03 | 64.71 | 91.60 | Retained |
| INSPIRED | Qwen 3B | T | 86.55 | 81.51 | 90.76 | 87.39 | Reversed |
| INSPIRED | Qwen 3B | F | 86.55 | 91.60 | 73.95 | 87.39 | Retained |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | A | 41.56 | 61.66 | 8.55 |
| Llama 3B | B | 58.44 | 38.34 | 91.45 |
| Qwen 3B | B | 73.95 | 66.59 | 80.41 |
| Qwen 3B | A | 26.05 | 33.41 | 19.59 |
| Qwen 7B | B | 86.35 | 82.90 | 88.90 |
| Qwen 7B | A | 13.65 | 17.10 | 11.10 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | Faster, costlier | 31.90 | 32.38 | 0.48 |
| Llama 3B | Slower, cheaper | 68.10 | 67.62 | 99.52 |
| Qwen 3B | Faster, costlier | 95.71 | 100.00 | 71.43 |
| Qwen 3B | Slower, cheaper | 4.29 | 0.00 | 28.57 |
| Qwen 7B | Faster, costlier | 27.14 | 68.57 | 10.48 |
| Qwen 7B | Slower, cheaper | 72.86 | 31.43 | 89.52 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | Fewer observations | 74.33 | 84.18 | 31.30 |
| Llama 3B | More observations | 25.67 | 15.82 | 68.70 |
| Qwen 3B | More observations | 50.96 | 42.50 | 60.00 |
| Qwen 3B | Fewer observations | 49.04 | 57.50 | 40.00 |
| Qwen 7B | More observations | 65.96 | 62.55 | 69.42 |
| Qwen 7B | Fewer observations | 34.04 | 37.45 | 30.58 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | SELL | 38.81 | 26.76 | 8.49 |
| Llama 3B | HOLD | 47.06 | 36.26 | 91.51 |
| Llama 3B | BUY | 14.13 | 36.98 | 0.00 |
| Qwen 3B | SELL | 99.90 | 99.28 | 100.00 |
| Qwen 3B | BUY | 0.10 | 0.72 | 0.00 |
| Qwen 7B | SELL | 19.77 | 19.38 | 19.96 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | COUNTER | 98.11 | 98.54 | 96.32 |
| Llama 3B | ACCEPT | 1.71 | 1.46 | 2.53 |
| Llama 3B | EXIT | 0.17 | 0.00 | 1.16 |
| Qwen 3B | COUNTER | 97.09 | 98.59 | 83.93 |
| Qwen 3B | ACCEPT | 0.94 | 0.94 | 0.34 |
| Qwen 3B | EXIT | 1.97 | 0.47 | 15.72 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | SOONER | 100.00 | 100.00 | 100.00 |
| Qwen 3B | SOONER | 44.10 | 58.33 | 44.44 |
| Qwen 3B | LATER | 55.90 | 41.67 | 55.56 |
| Qwen 7B | LATER | 99.65 | 95.14 | 100.00 |
| Qwen 7B | SOONER | 0.35 | 4.86 | 0.00 |
| Llama 8B | SOONER | 100.00 | 100.00 | 100.00 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | COUNTER | 100.00 | 100.00 | 99.89 |
| Llama 3B | EXIT | 0.00 | 0.00 | 0.11 |
| Qwen 3B | EXIT | 8.89 | 0.89 | 28.00 |
| Qwen 3B | COUNTER | 91.11 | 99.11 | 72.00 |
| Qwen 7B | COUNTER | 100.00 | 100.00 | 100.00 |
| Llama 8B | COUNTER | 100.00 | 100.00 | 100.00 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | COUNTER | 100.00 | 100.00 | 100.00 |
| Qwen 3B | EXIT | 85.19 | 27.35 | 99.72 |
| Qwen 3B | COUNTER | 14.81 | 72.65 | 0.28 |
| Qwen 7B | COUNTER | 100.00 | 100.00 | 100.00 |
| Llama 8B | COUNTER | 100.00 | 100.00 | 100.00 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | SEARCH_OTHER | 45.04 | 36.52 | 46.81 |
| Llama 3B | ACCEPT | 29.43 | 39.01 | 18.79 |
| Llama 3B | ASK_DETAILS | 25.53 | 24.47 | 34.40 |
| Qwen 3B | SEARCH_OTHER | 90.43 | 80.50 | 91.49 |
| Qwen 3B | ASK_DETAILS | 8.16 | 16.67 | 7.09 |
| Qwen 3B | ACCEPT | 1.42 | 2.84 | 1.42 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | ACCEPT | 78.15 | 93.28 | 25.21 |
| Llama 3B | ASK_MORE | 21.01 | 5.88 | 74.79 |
| Llama 3B | REJECT | 0.84 | 0.84 | 0.00 |
| Qwen 3B | ASK_MORE | 86.55 | 81.51 | 91.60 |
| Qwen 3B | ACCEPT | 12.61 | 14.29 | 8.40 |
| Qwen 3B | REJECT | 0.84 | 4.20 | 0.00 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | SELL | 37.21 | 29.75 | 28.16 |
| Llama 3B | HOLD | 20.90 | 27.06 | 58.61 |
| Llama 3B | BUY | 41.89 | 43.18 | 13.23 |
| Qwen 3B | SELL | 99.00 | 94.43 | 99.80 |
| Qwen 3B | HOLD | 1.00 | 5.57 | 0.20 |
| Qwen 7B | SELL | 39.90 | 39.60 | 41.29 |
| Model | Label | Base | T | F |
|---|---|---|---|---|
| Llama 3B | COUNTER | 99.82 | 100.00 | 98.55 |
| Llama 3B | EXIT | 0.18 | 0.00 | 1.23 |
| Llama 3B | ACCEPT | 0.00 | 0.00 | 0.22 |
| Qwen 3B | COUNTER | 98.46 | 99.96 | 85.04 |
| Qwen 3B | EXIT | 1.54 | 0.04 | 14.96 |
| Qwen 7B | COUNTER | 99.74 | 99.82 | 99.74 |
| Model | Base | T | F |
|---|---|---|---|
| Llama 3B | 0.18 | 0.00 | 1.23 |
| Llama 8B | 0.04 | 0.00 | 0.04 |
| Qwen 3B | 7.35 | 0.26 | 48.24 |
| Qwen 7B | 0.26 | 0.18 | 0.26 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Circa / UNCLEAR | T-BASE | 32602/3375 | -21.24 | [-21.84, -20.63] | 1.00e-05 | 3.50e-04 |
| Llama 3B | Circa / UNCLEAR | F-BASE | 32602/3375 | +48.50 | [+47.74, +49.24] | 1.00e-05 | 3.50e-04 |
| Llama 3B | Circa / UNCLEAR | F-T | 32602/3375 | +69.74 | [+69.01, +70.45] | 1.00e-05 | 3.50e-04 |
| Llama 8B | Circa / UNCLEAR | T-BASE | 32602/3375 | -17.13 | [-17.64, -16.62] | 1.00e-05 | 3.50e-04 |
| Llama 8B | Circa / UNCLEAR | F-BASE | 32602/3375 | +40.50 | [+39.80, +41.21] | 1.00e-05 | 3.50e-04 |
| Llama 8B | Circa / UNCLEAR | F-T | 32602/3375 | +57.63 | [+56.88, +58.37] | 1.00e-05 | 3.50e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | CB / FALSE | T-BASE | 1009/1006 | +8.62 | [+6.94, +10.41] | 1.00e-05 | 3.50e-04 |
| Llama 3B | CB / FALSE | F-BASE | 1009/1006 | -28.34 | [-31.09, -25.54] | 1.00e-05 | 3.50e-04 |
| Llama 3B | CB / FALSE | F-T | 1009/1006 | -36.97 | [-39.96, -33.99] | 1.00e-05 | 3.50e-04 |
| Llama 8B | CB / FALSE | T-BASE | 1009/1006 | +18.43 | [+16.09, +20.85] | 1.00e-05 | 3.50e-04 |
| Llama 8B | CB / FALSE | F-BASE | 1009/1006 | -11.20 | [-13.18, -9.24] | 1.00e-05 | 3.50e-04 |
| Llama 8B | CB / FALSE | F-T | 1009/1006 | -29.63 | [-32.44, -26.81] | 1.00e-05 | 3.50e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Circa / accuracy | T-BASE | 32602/3375 | +11.98 | [+11.36, +12.62] | 1.00e-05 | 7.20e-04 |
| Llama 3B | Circa / macro F1 | T-BASE | 32602/3375 | +7.03 | [+6.26, +7.80] | 1.00e-05 | 7.20e-04 |
| Llama 3B | Circa / accuracy | F-BASE | 32602/3375 | -34.94 | [-35.63, -34.24] | 1.00e-05 | 7.20e-04 |
| Llama 3B | Circa / macro F1 | F-BASE | 32602/3375 | -22.84 | [-23.44, -22.24] | 1.00e-05 | 7.20e-04 |
| Llama 3B | Circa / accuracy | F-T | 32602/3375 | -46.93 | [-47.70, -46.15] | 1.00e-05 | 7.20e-04 |
| Llama 3B | Circa / macro F1 | F-T | 32602/3375 | -29.87 | [-30.78, -28.96] | 1.00e-05 | 7.20e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 7B | Circa / accuracy | T-BASE | 32602/3375 | +2.03 | [+1.86, +2.21] | 1.00e-05 | 7.20e-04 |
| Qwen 7B | Circa / macro F1 | T-BASE | 32602/3375 | +2.27 | [+2.10, +2.45] | 1.00e-05 | 7.20e-04 |
| Qwen 7B | Circa / accuracy | F-BASE | 32602/3375 | -3.32 | [-3.54, -3.11] | 1.00e-05 | 7.20e-04 |
| Qwen 7B | Circa / macro F1 | F-BASE | 32602/3375 | -3.17 | [-3.37, -2.96] | 1.00e-05 | 7.20e-04 |
| Qwen 7B | Circa / accuracy | F-T | 32602/3375 | -5.35 | [-5.64, -5.07] | 1.00e-05 | 7.20e-04 |
| Qwen 7B | Circa / macro F1 | F-T | 32602/3375 | -5.44 | [-5.71, -5.18] | 1.00e-05 | 7.20e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 3B | CB / accuracy | T-BASE | 858/856 | +4.78 | [+3.38, +6.29] | 1.00e-05 | 7.20e-04 |
| Qwen 3B | CB / macro F1 | T-BASE | 858/856 | +2.95 | [+1.95, +4.01] | 1.00e-05 | 7.20e-04 |
| Qwen 3B | CB / accuracy | F-BASE | 858/856 | -10.02 | [-12.22, -7.92] | 1.00e-05 | 7.20e-04 |
| Qwen 3B | CB / macro F1 | F-BASE | 858/856 | -8.20 | [-10.06, -6.39] | 1.00e-05 | 7.20e-04 |
| Qwen 3B | CB / accuracy | F-T | 858/856 | -14.80 | [-17.37, -12.33] | 1.00e-05 | 7.20e-04 |
| Qwen 3B | CB / macro F1 | F-T | 858/856 | -11.16 | [-13.22, -9.16] | 1.00e-05 | 7.20e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 8B | fomc / accuracy | T-BASE | 2421/27 | +1.98 | [+0.82, +3.11] | 0.0054 | 0.0763 |
| Llama 8B | fomc / macro F1 | T-BASE | 2421/27 | +5.27 | [+3.78, +6.69] | 1.00e-05 | 7.20e-04 |
| Llama 8B | fomc / accuracy | F-BASE | 2421/27 | -2.85 | [-3.85, -1.95] | 1.00e-05 | 7.20e-04 |
| Llama 8B | fomc / macro F1 | F-BASE | 2421/27 | -7.28 | [-8.95, -5.69] | 1.00e-05 | 7.20e-04 |
| Llama 8B | fomc / accuracy | F-T | 2421/27 | -4.83 | [-6.26, -3.40] | 1.00e-05 | 7.20e-04 |
| Llama 8B | fomc / macro F1 | F-T | 2421/27 | -12.54 | [-14.40, -10.66] | 1.00e-05 | 7.20e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Circa / UNCLEAR | T-BASE | 32602/10 | -21.24 | [-25.12, -18.10] | 0.0020 | 0.0234 |
| Llama 3B | Circa / UNCLEAR | F-BASE | 32602/10 | +48.50 | [+44.10, +53.39] | 0.0020 | 0.0234 |
| Llama 3B | Circa / UNCLEAR | F-T | 32602/10 | +69.74 | [+64.66, +74.73] | 0.0020 | 0.0234 |
| Llama 8B | Circa / UNCLEAR | T-BASE | 32602/10 | -17.13 | [-18.62, -15.37] | 0.0020 | 0.0234 |
| Llama 8B | Circa / UNCLEAR | F-BASE | 32602/10 | +40.50 | [+35.99, +44.72] | 0.0020 | 0.0234 |
| Llama 8B | Circa / UNCLEAR | F-T | 32602/10 | +57.63 | [+52.91, +62.16] | 0.0020 | 0.0234 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Circa / accuracy | T-BASE | 32602/10 | +11.98 | [+9.70, +14.99] | 0.0020 | 0.0469 |
| Llama 3B | Circa / macro F1 | T-BASE | 32602/10 | +7.03 | [+5.26, +9.11] | 0.0020 | 0.0469 |
| Llama 3B | Circa / accuracy | F-BASE | 32602/10 | -34.94 | [-39.17, -31.20] | 0.0020 | 0.0469 |
| Llama 3B | Circa / macro F1 | F-BASE | 32602/10 | -22.84 | [-25.59, -19.97] | 0.0020 | 0.0469 |
| Llama 3B | Circa / accuracy | F-T | 32602/10 | -46.93 | [-50.76, -43.00] | 0.0020 | 0.0469 |
| Llama 3B | Circa / macro F1 | F-T | 32602/10 | -29.87 | [-31.97, -27.64] | 0.0020 | 0.0469 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 7B | Circa / accuracy | T-BASE | 32602/10 | +2.03 | [+1.67, +2.46] | 0.0020 | 0.0469 |
| Qwen 7B | Circa / macro F1 | T-BASE | 32602/10 | +2.27 | [+1.96, +2.62] | 0.0020 | 0.0469 |
| Qwen 7B | Circa / accuracy | F-BASE | 32602/10 | -3.32 | [-3.79, -2.84] | 0.0020 | 0.0469 |
| Qwen 7B | Circa / macro F1 | F-BASE | 32602/10 | -3.17 | [-3.49, -2.83] | 0.0020 | 0.0469 |
| Qwen 7B | Circa / accuracy | F-T | 32602/10 | -5.35 | [-6.23, -4.54] | 0.0020 | 0.0469 |
| Qwen 7B | Circa / macro F1 | F-T | 32602/10 | -5.44 | [-6.08, -4.81] | 0.0020 | 0.0469 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | risk / high variance | T-BASE | 1764/1764 | -3.80 | [-5.90, -1.70] | 4.60e-04 | 0.0092 |
| Llama 3B | risk / high variance | F-BASE | 1764/1764 | +15.25 | [+12.64, +17.91] | 1.00e-05 | 4.60e-04 |
| Llama 3B | risk / high variance | F-T | 1764/1764 | +19.05 | [+15.82, +22.34] | 1.00e-05 | 4.60e-04 |
| Llama 3B | Machine / fewer observations | T-BASE | 4160/26 | +9.86 | [+8.85, +11.01] | 1.00e-05 | 4.60e-04 |
| Llama 3B | Machine / fewer observations | F-BASE | 4160/26 | -43.03 | [-44.52, -41.44] | 1.00e-05 | 4.60e-04 |
| Llama 3B | Machine / fewer observations | F-T | 4160/26 | -52.88 | [-54.81, -50.91] | 1.00e-05 | 4.60e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 8B | INSPIRED / ASK | T-BASE | 119/119 | -15.97 | [-22.69, -10.08] | 3.81e-06 | 1.83e-04 |
| Llama 8B | INSPIRED / ASK | F-BASE | 119/119 | +47.90 | [+38.66, +57.14] | 1.00e-05 | 4.60e-04 |
| Llama 8B | INSPIRED / ASK | F-T | 119/119 | +63.87 | [+55.46, +72.27] | 1.00e-05 | 4.60e-04 |
| Llama 8B | Frames / ACCEPT | T-BASE | 282/282 | +0.71 | [+0.00, +1.77] | 0.5000 | 1.000 |
| Llama 8B | Frames / ACCEPT | F-BASE | 282/282 | -2.13 | [-3.90, -0.71] | 0.0312 | 0.5000 |
| Llama 8B | Frames / ACCEPT | F-T | 282/282 | -2.84 | [-4.96, -1.06] | 0.0078 | 0.1406 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 7B | risk / high variance | T-BASE | 1764/1764 | +1.19 | [+0.34, +2.04] | 0.0091 | 0.1542 |
| Qwen 7B | risk / high variance | F-BASE | 1764/1764 | +0.28 | [-0.45, +1.02] | 0.5537 | 1.000 |
| Qwen 7B | risk / high variance | F-T | 1764/1764 | -0.91 | [-2.04, +0.23] | 0.1446 | 1.000 |
| Qwen 7B | Machine / fewer observations | T-BASE | 4160/26 | +3.41 | [+2.69, +4.13] | 1.00e-05 | 4.60e-04 |
| Qwen 7B | Machine / fewer observations | F-BASE | 4160/26 | -3.46 | [-4.04, -2.88] | 1.00e-05 | 4.60e-04 |
| Qwen 7B | Machine / fewer observations | F-T | 4160/26 | -6.88 | [-7.79, -5.87] | 1.00e-05 | 4.60e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Risk summary / switch rate | T-BASE | 24/6 | +25.00 | [-4.17, +58.33] | 0.3750 | 1.000 |
| Llama 3B | Risk summary / switch rate | F-BASE | 24/6 | -16.67 | [-25.00, -8.33] | 0.1250 | 1.000 |
| Llama 3B | Risk summary / switch rate | F-T | 24/6 | -41.67 | [-66.67, -20.83] | 0.0625 | 0.6250 |
| Llama 8B | Risk summary / switch rate | T-BASE | 24/6 | -8.33 | [-16.67, +0.00] | 0.5000 | 1.000 |
| Llama 8B | Risk summary / switch rate | F-BASE | 24/6 | -45.83 | [-50.00, -37.50] | 0.0312 | 0.3750 |
| Llama 8B | Risk summary / switch rate | F-T | 24/6 | -37.50 | [-45.83, -29.17] | 0.0312 | 0.3750 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Risk summary / switch rate | T-BASE | 80/20 | +0.00 | [-10.00, +10.00] | 1.000 | 1.000 |
| Llama 3B | Risk summary / switch rate | F-BASE | 80/20 | -13.75 | [-20.00, -6.25] | 0.0074 | 0.0517 |
| Llama 3B | Risk summary / switch rate | F-T | 80/20 | -13.75 | [-21.25, -5.00] | 0.0114 | 0.0681 |
| Llama 8B | Risk summary / switch rate | T-BASE | 80/20 | -17.50 | [-22.50, -12.50] | 1.22e-04 | 0.0012 |
| Llama 8B | Risk summary / switch rate | F-BASE | 80/20 | -50.00 | [-50.00, -50.00] | 1.91e-06 | 2.29e-05 |
| Llama 8B | Risk summary / switch rate | F-T | 80/20 | -32.50 | [-37.50, -27.50] | 1.91e-06 | 2.29e-05 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | bargaining / EXIT | T-BASE | 315/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT | F-BASE | 315/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT | F-T | 315/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 8B | bargaining / EXIT | T-BASE | 315/15 | -24.44 | [-31.11, -17.78] | 1.22e-04 | 0.0011 |
| Llama 8B | bargaining / EXIT | F-BASE | 315/15 | +1.90 | [-1.27, +4.76] | 0.3350 | 1.000 |
| Llama 8B | bargaining / EXIT | F-T | 315/15 | +26.35 | [+19.68, +32.70] | 6.10e-05 | 7.32e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | bargaining / EXIT response | BASE:high-low | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT response | T:high-low | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT response | F:high-low | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT response difference | T-BASE | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT response difference | F-BASE | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Llama 3B | bargaining / EXIT response difference | F-T | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 7B | bargaining / EXIT response | BASE:high-low | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Qwen 7B | bargaining / EXIT response | T:high-low | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Qwen 7B | bargaining / EXIT response | F:high-low | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Qwen 7B | bargaining / EXIT response difference | T-BASE | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Qwen 7B | bargaining / EXIT response difference | F-BASE | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Qwen 7B | bargaining / EXIT response difference | F-T | 90/15 | +0.00 | [+0.00, +0.00] | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | Movie / fits:ACCEPT | T-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Llama 3B | Movie / fits:ACCEPT | F-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Llama 3B | Movie / fits:ACCEPT | F-T | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Llama 3B | Movie / mismatch:OTHER | T-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Llama 3B | Movie / mismatch:OTHER | F-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Llama 3B | Movie / mismatch:OTHER | F-T | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 8B | Movie / fits:ACCEPT | T-BASE | 10/2 | +60.00 | NA | 0.5000 | 1.000 |
| Llama 8B | Movie / fits:ACCEPT | F-BASE | 10/2 | -20.00 | NA | 1.000 | 1.000 |
| Llama 8B | Movie / fits:ACCEPT | F-T | 10/2 | -80.00 | NA | 0.5000 | 1.000 |
| Llama 8B | Movie / mismatch:OTHER | T-BASE | 10/2 | -50.00 | NA | 0.5000 | 1.000 |
| Llama 8B | Movie / mismatch:OTHER | F-BASE | 10/2 | +20.00 | NA | 0.5000 | 1.000 |
| Llama 8B | Movie / mismatch:OTHER | F-T | 10/2 | +70.00 | NA | 0.5000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 3B | Movie / fits:ACCEPT | T-BASE | 12/2 | -16.67 | NA | 0.5000 | 1.000 |
| Qwen 3B | Movie / fits:ACCEPT | F-BASE | 12/2 | -16.67 | NA | 1.000 | 1.000 |
| Qwen 3B | Movie / fits:ACCEPT | F-T | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Qwen 3B | Movie / mismatch:OTHER | T-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Qwen 3B | Movie / mismatch:OTHER | F-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Qwen 3B | Movie / mismatch:OTHER | F-T | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Qwen 7B | Movie / fits:ACCEPT | T-BASE | 12/2 | -16.67 | NA | 1.000 | 1.000 |
| Qwen 7B | Movie / fits:ACCEPT | F-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Qwen 7B | Movie / fits:ACCEPT | F-T | 12/2 | +16.67 | NA | 1.000 | 1.000 |
| Qwen 7B | Movie / mismatch:OTHER | T-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Qwen 7B | Movie / mismatch:OTHER | F-BASE | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Qwen 7B | Movie / mismatch:OTHER | F-T | 12/2 | +0.00 | NA | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|
| Llama 3B | ARC-Easy | T:Base P | 2000/2000 | -3.70 | [-4.90, -2.50] | 2.56e-09 | 3.84e-08 | 1.92e-08 |
| Llama 3B | BoolQ | T:Base P | 2000/2000 | -0.80 | [-1.95, +0.35] | 0.1915 | 1.000 | 0.7660 |
| Llama 3B | ARC-Easy | F:Base P | 2000/2000 | -5.65 | [-6.95, -4.35] | 6.02e-18 | 1.08e-16 | 5.42e-17 |
| Llama 3B | BoolQ | F:Base P | 2000/2000 | +0.20 | [-0.95, +1.35] | 0.7956 | 1.000 | 1.000 |
| Qwen 3B | GSM8K | T:Base P | 1300/1300 | -0.31 | [-1.85, +1.23] | 0.7750 | 1.000 | 1.000 |
| Qwen 3B | ARC-Easy | T:Base P | 2000/2000 | -1.95 | [-2.80, -1.05] | 2.18e-05 | 3.05e-04 | 1.53e-04 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | BoolQ | T:Base P | 2000/1897 | -0.80 | [-1.95, +0.35] | 0.1975 | 0.7901 |
| Llama 3B | BoolQ | F:Base P | 2000/1897 | +0.20 | [-0.95, +1.36] | 0.7993 | 1.000 |
| Qwen 3B | BoolQ | T:Base P | 2000/1897 | -0.35 | [-1.16, +0.45] | 0.4759 | 1.000 |
| Qwen 3B | BoolQ | F:Base P | 2000/1897 | -0.25 | [-1.11, +0.60] | 0.6483 | 1.000 |
| Qwen 7B | BoolQ | T:Base P | 2000/1897 | -4.05 | [-5.27, -2.83] | 1.00e-05 | 6.00e-05 |
| Qwen 7B | BoolQ | F:Base P | 2000/1897 | -3.85 | [-5.06, -2.67] | 1.00e-05 | 6.00e-05 |
| Model | Task / outcome | Contrast | N/G | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|
| Llama 3B | GSM8K | T+T:T | 1300/1300 | +19.92 | [+16.92, +22.92] | 3.01e-37 | 1.02e-35 | 1.000 |
| Llama 3B | ARC-Easy | T+T:T | 2000/2000 | -3.85 | [-5.05, -2.65] | 5.95e-10 | 1.55e-08 | 8.92e-09 |
| Llama 3B | BoolQ | T+T:T | 2000/2000 | -1.40 | [-2.55, -0.25] | 0.0212 | 0.2332 | 0.2438 |
| Llama 3B | GSM8K | T+F:T | 1300/1300 | +19.31 | [+16.31, +22.38] | 1.34e-33 | 4.41e-32 | 1.000 |
| Llama 3B | ARC-Easy | T+F:T | 2000/2000 | -4.45 | [-5.75, -3.20] | 1.37e-11 | 3.83e-10 | 2.19e-10 |
| Llama 3B | BoolQ | T+F:T | 2000/2000 | +0.20 | [-0.95, +1.35] | 0.7985 | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|
| Qwen 3B | GSM8K | F+T:F | 1300/1300 | +2.62 | [+0.92, +4.38] | 0.0042 | 0.0670 | 1.000 |
| Qwen 3B | ARC-Easy | F+T:F | 2000/2000 | -0.90 | [-1.90, +0.05] | 0.0854 | 0.5979 | 0.8114 |
| Qwen 3B | BoolQ | F+T:F | 2000/2000 | -1.15 | [-2.05, -0.30] | 0.0140 | 0.1820 | 0.1750 |
| Qwen 3B | GSM8K | F+F:F | 1300/1300 | +3.00 | [+1.23, +4.77] | 0.0011 | 0.0185 | 1.000 |
| Qwen 3B | ARC-Easy | F+F:F | 2000/2000 | -1.00 | [-2.00, +0.00] | 0.0619 | 0.5226 | 0.6193 |
| Qwen 3B | BoolQ | F+F:F | 2000/2000 | -1.15 | [-2.10, -0.20] | 0.0235 | 0.2349 | 0.2584 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | BoolQ | T+T:T | 2000/1897 | -1.40 | [-2.57, -0.20] | 0.0253 | 0.1703 |
| Llama 3B | BoolQ | T+F:T | 2000/1897 | +0.20 | [-0.99, +1.36] | 0.8033 | 1.000 |
| Llama 3B | BoolQ | F+T:F | 2000/1897 | -0.85 | [-2.06, +0.35] | 0.1893 | 0.9466 |
| Llama 3B | BoolQ | F+F:F | 2000/1897 | -0.15 | [-1.31, +1.00] | 0.8655 | 1.000 |
| Qwen 3B | BoolQ | T+T:T | 2000/1897 | +0.20 | [-0.75, +1.15] | 0.7564 | 1.000 |
| Qwen 3B | BoolQ | T+F:T | 2000/1897 | -0.05 | [-1.01, +0.91] | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|
| Llama 3B | ARC-Easy | F:Base | 2000/2000 | -0.15 | [-0.85, +0.55] | 0.7877 | 1.000 | 1.000 |
| Llama 3B | BoolQ | F:Base | 2000/2000 | +1.65 | [+0.55, +2.75] | 0.0050 | 0.1046 | 1.000 |
| Llama 3B | ARC-Easy | T:Base | 2000/2000 | -0.55 | [-1.15, +0.05] | 0.0989 | 1.000 | 1.000 |
| Llama 3B | BoolQ | T:Base | 2000/2000 | -2.25 | [-3.05, -1.50] | 1.18e-08 | 2.82e-07 | 1.41e-07 |
| Qwen 3B | GSM8K | F:Base | 1300/1300 | +0.77 | [-0.31, +1.85] | 0.2026 | 1.000 | 1.000 |
| Qwen 3B | ARC-Easy | F:Base | 2000/2000 | -0.10 | [-0.60, +0.40] | 0.8388 | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | ||||
|---|---|---|---|---|---|---|---|---|
| Llama 8B | GSM8K | T:Base | 1300/1300 | -0.08 | [-1.62, +1.46] | 1.000 | 1.000 | 1.000 |
| Llama 8B | GSM8K | F:Base | 1300/1300 | -1.15 | [-2.69, +0.38] | 0.1548 | 1.000 | 1.000 |
| Llama 8B | ARC-Easy | T:Base | 2000/2000 | -0.05 | [-0.45, +0.35] | 1.000 | 1.000 | 1.000 |
| Llama 8B | ARC-Easy | F:Base | 2000/2000 | -0.85 | [-1.40, -0.30] | 0.0033 | 0.0732 | 0.0366 |
| Llama 8B | BoolQ | T:Base | 2000/2000 | -0.05 | [-0.55, +0.45] | 1.000 | 1.000 | 1.000 |
| Llama 8B | BoolQ | F:Base | 2000/2000 | -0.40 | [-0.95, +0.15] | 0.2153 | 1.000 | 1.000 |
| Model | Task / outcome | Contrast | N/G | 95% CI | |||
|---|---|---|---|---|---|---|---|
| Llama 3B | BoolQ | F:Base | 2000/1897 | +1.65 | [+0.55, +2.79] | 0.0051 | 0.0305 |
| Llama 3B | BoolQ | T:Base | 2000/1897 | -2.25 | [-3.05, -1.45] | 1.00e-05 | 8.00e-05 |
| Qwen 3B | BoolQ | F:Base | 2000/1897 | -0.85 | [-1.40, -0.30] | 0.0030 | 0.0211 |
| Qwen 3B | BoolQ | T:Base | 2000/1897 | +0.30 | [-0.15, +0.75] | 0.2632 | 1.000 |
| Qwen 7B | BoolQ | T:Base | 2000/1897 | +0.20 | [-0.20, +0.60] | 0.4545 | 1.000 |
| Qwen 7B | BoolQ | F:Base | 2000/1897 | -0.20 | [-0.65, +0.25] | 0.5034 | 1.000 |
| Axis | Model | Measure | A guided | B guided | A+A | A+B | B+A | B+B |
|---|---|---|---|---|---|---|---|---|
| E/I | Llama 3B | Circa unclear | 5.6 / 0.6 | 78.5 / 3.4 | 0.6 | 1.9 | 10.7 | 44.5 |
| E/I | Llama 3B | Circa unparseable | 0.5 / 0.0 | 0.0 / 0.0 | 0.0 | 0.0 | 0.0 | 0.0 |
| E/I | Llama 3B | CB false | 2.1 / 75.5 | 60.6 / 66.9 | 55.2 | 37.2 | 97.7 | 91.0 |
| E/I | Llama 3B | CB unparseable | 94.3 / 0.1 | 38.9 / 3.6 | 4.3 | 22.9 | 0.0 | 3.9 |
| E/I | Qwen 3B | Circa unclear | 75.9 / 78.9 | 94.3 / 83.4 | 70.0 | 75.2 | 87.0 | 90.5 |
| E/I | Qwen 3B | Circa unparseable | 4.9 / 0.0 | 0.9 / 0.0 | 0.2 | 0.2 | 0.0 | 0.0 |
| Axis | Model | Measure | A guided | B guided | A+A | A+B | B+A | B+B |
|---|---|---|---|---|---|---|---|---|
| E/I | Llama 3B | Stock sell | 48.5 / 29.4 | 9.2 / 0.0 | 29.9 | 0.7 | 12.9 | 0.0 |
| E/I | Llama 3B | Bargaining exit | 0.0 / 0.0 | 0.0 / 84.0 | 0.0 | 11.7 | 1.2 | 100.0 |
| E/I | Llama 3B | INSPIRED ask | 17.6 / 0.0 | 89.9 / 10.9 | 0.0 | 3.4 | 32.8 | 86.6 |
| E/I | Llama 3B | Frames search | 28.7 / 51.1 | 49.3 / 69.1 | 18.4 | 37.9 | 84.4 | 70.6 |
| E/I | Qwen 3B | Stock sell | 99.7 / 81.9 | 92.9 / 0.1 | 98.4 | 78.3 | 0.2 | 0.0 |
| E/I | Qwen 3B | Bargaining exit | 9.1 / 0.0 | 10.7 / 1.8 | 0.0 | 0.3 | 0.4 | 20.6 |
Personality Modeling in Language Models
Personality modeling in language models studies how traits are measured, represented, simulated, and controlled in model behavior.
See all 185 papers