LLMs increasingly excel on AI benchmarks, but doing so does not guarantee validity for downstream tasks. This study contrasts LLM alignment on benchmarks, downstream tasks, and, importantly the intended impact of those tasks. We evaluate the performance of leading LLMs (i.e., generative pre-trained base models) on difficult-to-verify tasks of the teaching and learning of schoolchildren. Across all LLMs, inter-model behaviors on disparate tasks correlate higher than they do with expert human behaviors on target tasks. These biases shared across LLMs are poorly aligned with downstream measures of teaching quality and often negatively aligned with the intended impact of student learning outcomes. Further, we find multi-model ensembles, both unanimous model voting and expert-weighting by benchmark performance, further exacerbate misalignment with learning. We measure that selection of LLM and/or prompting strategy only reliably accounts for 15% of all measured misalignment error and that variation in misalignment error is shared across LLMs, suggesting that common pretraining accounts for much of the misalignment in these tasks. We demonstrate methods for robustly measuring alignment of complex tasks and provide unique insights into practical applications of LLMs in high-noise contexts.
Figures & tables
Figure 1: Cascading levels of inference found in LLM development and evaluation and a parallel adaption of Ackoff’s progression ( Ackoff, 1989 ; Rowley, 2007 )
Figure 2: Task Data and Experimental Design . Each LLM is provided classroom transcripts. Using several prompting techniques for each model, LLMs place an ordinal rating on the quality on an aspect of teaching and learning. This is done across seven distinct tasks. We evaluate for alignment of LLM values, and not for accuracy, when comparing the relative ranking provided by each LLM on each task with human experts and with student learning gains for the class.
Figure 3: Mean Inter-task Bias Corrected Squared Distance Correlations dCorn2 : between LLMs and human raters across different evaluation tasks. Top row: Same-task Correlation Mean inter-rater distance correlations across transcripts for the same task and ( bottom row: different task correlation ) for different tasks using the same transcript. ( left: correlations with humans ) Mean inter-rater distance correlations with expert human raters, ( center: correlations with other LLMs ) with other LLMs, and ( right: intramodel intertask correlations ) each LLM with itself. The top right is omitted for redundancy. Lines are standard errors for each estimated mean. Means and SEs were computed under Fisher’s z transformation and back transformed to preserve variance. The complete correlation matrix for each task and model are found in Figs. 10 and 11
Figure 4: (Mis)alignment with Downstream Task (Teaching) and Intended Impact (Learning) : The x-axes measure the alignment of scores ( Sf ) from each LLM, ensemble, and baseline f with expert human ratings on downstream tasks ( X ) on the quality of teaching skills in a given lesson: τSfX . Similarly, the y-axes measure alignment with the value-added to learning via student achievement gains ( Y ): τSfY . Each color-shape combination represents a different rater family or baseline. Each column represents a specific instructional rating task, from ( left to right ): CLBM, CLINSTD, LANGIMP , and REMED . Each row represents a distinct implementation scenario: Top row (individual rating models) The 95% CI are shown for individual models. Middle row (Pedagogy Expertise Weighted Ensembles) and Bottom row (Unanimous Vote Ensembles) : The alignment correlations of all possible model-prompt ensembles are represented by the density contours. The innermost contours represent the alignment regions’ greatest ensemble performance density. For the bottom row ensembles, we amplify the shared signal by only measuring observations where all three models are in agreement (See Appendix D.3 ). Baselines : the gray line and shaded regions on CLBM and CLINSTD represent the estimate and confidence intervals from the human expert rating alignment with student learning gains on the study’s joint test set. The green Encoder models (data from Hardy (2024) and our own replications) are LMs but not LLMs, and they merely serve as a deep learning baseline for possible alignment based on transcripts. The baseline of Teacher Experience represent pairwise comparisons that always rank more experienced teachers higher than less experiened ones. For better plotting details, the “oracle” VAM baseline which puts a higher value to teachers with higher prior year VAMs, is only displayed on the bottom row. Random baselines (both uniform and stratified) predictably clustered around the origin and, with no real-world example of this as a valid baseline, we exclude them from the plots.
Facet of Error Variation
median
MAD
MAP
95% HDI
ITEM
0.08
0.08
0.03
[0.01,0.62]
LLM
0.07
0.04
0.06
[0.02,0.18]
OBS
0.02
0.01
0.02
[0,0.03]
PROMPT
0.02
0.03
0.00
[0,0.58]
ITEM:OBS
0.04
0.01
0.05
[0.01,0.06]
LLM:ITEM
0.03
0.01
0.03
[0.01,0.06]
Table 1: Misalignment Shares : Proportion of Variation in Squared Error by Source. Row partitions separate main, second-order, and higher-order effect posterior distributions.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Proportions of human rater scores by item.
Figure 6: Proportions of rater scores by MQI item.
Item
Item Name
[Factor] Item Description
MQI
EXPL
Teacher Explanations
[Mathematical Instruction] Teacher explanations that give meaning to ideas, procedures, steps, or solution methods.
LANGIMP †
Imprecision in Language or Notation
[Mathematical Errors] Imprecision in language or notation, with regard to mathematical symbols and technical or general mathematical language.
REMED
Remediation of Student Errors and Difficulties
[Mathematical Instruction] Remediation of student errors and difficulties addressed in a substantive manner.
SMQR
Student Mathematical Questioning and Reasoning
[Mathematical Instruction] Student mathematical questioning and reasoning, such as posing mathematically motivated questions, offering mathematical claims or counterclaims.
CLASS
Appendix
Table 2: CLASS and MQI item descriptions and corresponding abbreviations from the test set of Wang and Demszky . †denotes items that are reverse coded due to being negatively worded with respect to the construct of teacher ability. Bolded items are the focus items of the present alignment study. Bracketed item descriptions are the names of the factors with highest identified loadings by Blazar et al. per category in Appendix 2.b of the original study Kane et al. (2015) . While in the present impact alignment study we only evaluated four items, we performed a full replication of all seven items, the responses and outputs of which can be found in the online data. Variance decompositions for all seven items on the test set can be found in Appendix C .
Task
τ>0
Low. 90% CI >0
Thiel Sen >0
RME >0
T> Rand
T≥T . Exp
Q4>Q1
Robust. Pass Rate
CLBM
0.07, Y
0.06***, Y
0.013, Y
0.008, Y
−0.04 , Y
0.0004, Y ( −0.04 , 0.04)
0.02**, Y
100 (7/7)
CLINSTD
0.02, Y
0.003*, Y
0.003, Y
0.0002, Y
0.004, Y
0.0004, Y ( −0.04 , 0.04)
0.02**, Y
100 (7/7)
LANGIMP
0.12, Y
0.10***, Y
0.06, Y
0.05, Y
0.03, Y
0.04, Y (0.000, 0.08)
0.03**, Y
100 (7/7)
REMED
0.02, Y
0.001*, Y
0.008, Y
0.004, Y
−0.01 , Y
0.04, Y (0.000, 0.08)
0.01, N
71 (5/7)
Appendix
Table 3: Robustness Test Results Across Different Tasks
Figure 7: Bayesian Error Variance Decomposition : ( left ) fully crossed facet diagram for sources of variance. ( right ) corresponding brms code listing.
variable
median
MAD
MAP
HDI
mean
SD
CI
R^
ITEM
0.08
0.08
0.03
[0.01,0.62]
0.14
0.17
[0.01,0.54]
0.99
LLM
0.07
0.04
0.06
[0.02,0.18]
0.08
0.05
[0.02,0.16]
1.00
OBS
0.02
0.01
0.02
[0,0.03]
0.02
0.01
[0.01,0.03]
1.01
PROMPT
0.02
0.03
0.00
[0,0.58]
0.08
0.15
[0,0.46]
1.00
ITEM:OBS
0.04
0.01
0.05
[0.01,0.06]
0.04
0.01
[0.02,0.06]
1.00
LLM:ITEM
0.03
0.01
0.03
[0.01,0.06]
0.03
0.01
[0.01,0.05]
1.00
Appendix
Table 4: Bayesian Squared Error Variance Decomposition (e^) Parameter Estimates and Fit Statistics
variable
median
MAD
MAP
HDI
mean
SD
CI
R^
ITEM
0.03
0.04
0.01
[0,0.34]
0.07
0.09
[0,0.22]
1.00
LLM
0.09
0.04
0.07
[0.03,0.24]
0.10
0.05
[0.03,0.19]
1.01
OBS
0.03
0.01
0.03
[0.01,0.05]
0.03
0.01
[0.01,0.04]
1.01
PROMPT
0.00
0.01
0.00
[0,0.5]
0.06
0.13
[0,0.47]
1.05
ITEM:OBS
0.12
0.02
0.12
[0.06,0.15]
0.11
0.02
[0.07,0.14]
1.05
LLM:ITEM
0.06
0.02
0.06
[0.03,0.1]
0.06
0.02
[0.03,0.1]
1.02
Appendix
Table 5: Bayesian Signed Squared Error Variance Decomposition (e^±) Parameter Estimates and Fit Statistics
variable
median
MAD
MAP
HDI
mean
SD
CI
R^
ITEM
0.29
0.13
0.27
[0.14,0.63]
0.33
0.14
[0.15,0.59]
0.99
LLM
0.04
0.03
0.04
[0.01,0.16]
0.05
0.04
[0.01,0.12]
0.99
OBS
0.07
0.02
0.07
[0.03,0.09]
0.07
0.02
[0.04,0.09]
1.00
PROMPT
0.00
0.00
0.00
[0,0.23]
0.02
0.06
[0,0.13]
1.01
ITEM:OBS
0.09
0.02
0.09
[0.04,0.12]
0.08
0.02
[0.05,0.11]
1.01
LLM:ITEM
0.04
0.01
0.04
[0.02,0.07]
0.04
0.02
[0.02,0.07]
1.00
Appendix
Table 6: Bayesian Signed Squared Error Extended Variance Decomposition (e^(+)±) Parameter Estimates and Fit Statistics for all replication study items. Note the large difference in Item variation when other items are included and the decrease in Prompting volatility.
Metric
Unit of Measure
e^
e^±
e^(+)±
Eρ^2
C×I
0.657
0.837
0.819
Φ
C×I
0.551
0.790
0.781
Eρ^2
{C×I,C,I}
0.819
0.839
0.946
Φ
{C×I,C,I}
0.732
0.809
0.931
Appendix
Table 7: Decision Study : performing a similar set of estimations to the LLM variance decomposition, except using the human experts on same test sets. We estimate the coefficients for generalizability and dependability for each of the same decompositions, e^ , e^± , and e^(+)± for two units of measurement
Figure 8: Reliability of Relative Shared Error Signal for Item × Obs : Bayesian Decision Study for item-transcript scores as the object of study across varying numbers of LLMs and Prompting Techniques required to remove LLM and prompt-specific idiosyncrasies from the error signal. Each value is calculated directly from sampling the posterior and extracting the median. For this study, where NLLM=16 and NPrompt Strat.=3 , the reliability Eρic2 is 0.66±0.02 ( R^=1.00 ). This suggests that the estimated item-transcript scores in this study achieve approximately the target level of consistency expected for these types of data ( 0.65 , see Ho and Kane 2013 ).
Figure 9: Reliability of Absolute Shared Error Signal : Bayesian Decision Study for any of item, transcript, or their interaction as the objects of measurement ( σα2=σi2+σc2+σci2 ) across varying numbers of LLMs and Prompting Techniques required to remove LLM and prompt-specific idiosyncrasies from the error signal. Each value is calculated directly from sampling the posterior and extracting the median. For this study, where NLLM=16 and NPrompt Strat.=3 , the reliability Φ , Equation 9 , of the mean rating across models and prompts shows is 0.73 ( R^=1.00 ).
Facet
Pct( e^ )
Pct (e^±)
Pct (e^(+)±)
ITEM
33.76
63.75
69.27
OBS
0.00
0.00
1.10
RATERID
0.00
0.00
0.00
ITEM:OBS
25.22
13.73
11.84
RATERID:OBS
12.85
8.01
3.69
RATERID:ITEM
4.42
3.27
3.35
Appendix
Table 8: Human Error Variance Decomposition : performing a similar set of estimations to the LLM variance decomposition, except using the human experts on same test sets. We estimate the percentage of human expert-VAM error for each of the same decompositions, e^ , e^± , and e^(+)± using Restricted Maximum Likelihood with lmer ( Bates et al., 2015 ) .
Figure 10: Between and Within LLM Distance Correlations: MQI , distance correlations nonparametric measure of dependence between and within rater families across MQI items. Correlation are conducted at the item-transcript level using pairwise-complete observations. Nonsignificant relationships (at α<0.05 ) are shown as blank after adjusting for family-wise error rate using the Bonferroni correction. Hierarchical clustering is done using complete linkages.
Figure 11: Between and Within LLM Distance Correlations: CLASS , distance correlations nonparametric measure of dependence between and within rater families across CLASS items. Correlation are conducted at the item-transcript level using pairwise-complete observations. Nonsignificant relationships (at α<0.05 ) are shown as blank after adjusting for family-wise error rate using the Bonferroni correction. Hierarchical clustering is done using complete linkages.
Parameter
Group
Effect Type
Coefficient
CI Low
CI High
(Intercept)
—
Fixed
0.46
0.33
0.58
Segment
wk
Random ( σk2 )
0.03
–
–
RATERID:OBSID:ITEM
Random
0.09
–
–
RATERID:ITEM:NCTETID
Random
0.03
–
–
RATERID:OBS_CHAPS
Random
0.03
–
–
RATERID:ITEM
Random
0.05
–
–
Appendix
Table 9: Mixed Effect Estimates for Teacher Skill Model
Parameter
Fit
R2 (conditional)
0.26
R2 (marginal)
0.14
Sigma
0.12
Appendix
Table 10: Measures of fit for Residualization
Item
Estimate (CI)
p.value
All
-0.156 (-0.205, -0.107)
0
CLBM
-0.356 (-0.426, -0.287)
0
CLINSTD
-0.139 (-0.21, -0.069)
0
LANGIMP
-0.181 (-0.23, -0.131)
0
REMED
-0.233 (-0.282, -0.183)
0
CLPC
-0.141 (-0.212, -0.071)
0
Appendix
Table 11: Mean VAM-alignment estimates across models after noise control, including three additional items from replication study.
Figure 12: see the caption of Figure 4 for full description of details. The difference is that the axes have undergone the noise controlling transformations of this appendix prior to plotting.
Are large language models (LLMs) bad at capturing human judgment? Two commonly stated limitations are that LLMs fail to capture full distributions of responses, and that their judgments are unstable across wording variations. We demonstrate simple prompting strategies that mitigate these limitations. Across two datasets--a U.S.-representative set of 144 moral scenarios and 38 moral beliefs from the International Social Survey Programme's Family and Changing Gender Roles module covering 32 countries--we show how simple elicitation techniques help improve AI-human alignment. First, prompting models to report standard deviations and response proportions recovers the full range of human responses better than common strategies. Second, ensuring scenarios are clear to human participants--as reflected in human confusion ratings--boosts model alignment, and LLMs can track human confusion ratings. At the same time, we find that LLMs' estimates of their own error are poorly calibrated, though they can predict human variability relatively well. These results suggest that asking better questions to LLMs can yield better answers.
Danica Dillion, Chen Cecilia Liu, Baihui Wang +5
Complexity Science Hub, Vienna · The Ohio State University · University of Cambridge +2
How biased is a language model? The answer depends on how you ask. A model that refuses to choose between castes for a leadership role will, in a fill-in-the-blank task, reliably associate upper castes with purity and lower castes with lack of hygiene. Single-task benchmarks miss this because they capture only one slice of a model's bias profile. We introduce a hierarchical taxonomy covering 9 bias types, including under-studied axes like caste, linguistic, and geographic bias, operationalized through 7 evaluation tasks that span explicit decision-making to implicit association. Auditing 7 commercial and open-weight LLMs with \textasciitilde45K prompts, we find three systematic patterns. First, bias is task-dependent: models counter stereotypes on explicit probes but reproduce them on implicit ones, with Stereotype Score divergences up to 0.43 between task types for the same model and identity groups. Second, safety alignment is asymmetric: models refuse to assign negative traits to marginalized groups, but freely associate positive traits with privileged ones. Third, under-studied bias axes show the strongest stereotyping across all models, suggesting alignment effort tracks benchmark coverage rather than harm severity. These results demonstrate that single-benchmark audits systematically mischaracterize LLM bias and that current alignment practices mask representational harm rather than mitigating it.
Large Language Model (LLM) alignment is intended to ensure that models remain helpful and safe, but its stability under input distributional shift is not yet fully understood. Prior work shows that aligned models can fail under jailbreak prompts, alternative encodings, and cross-lingual transfer, yet these failures are usually studied as attacks rather than controlled probes of alignment generalization. Moreover, existing evidence is largely grounded in natural language variation already represented during pretraining, leaving unresolved whether alignment generalizes with semantic content or remains tied to superficial surface patterns. In this paper, we study this question using synthetic semantic-preserving transformations that are rule-based and invertible, preserving task-relevant meaning while shifting inputs beyond standard linguistic variation. Across four open-weight and four commercial models, under both fine-tuning and in-context learning, we use these transformations as a probe of alignment generalization and identify an empirical pattern we term Alignment--utility asymmetry: once models can operate effectively on transformed inputs, task utility is often substantially retained while alignment failure increases more sharply. For example, adapted GPT-4.1 mini shows only limited utility degradation under transformation while its harmful rate rises from 13.3 to 74.3; Gemini 3 Flash similarly retains near-original utility while its harmful rate increases from 2.3 to 43.0. Taken together, these results suggest that semantic-preserving distribution shifts can expose a recurring gap in how utility and alignment generalize in current LLMs.
Mohan Li, Chengyu Yu, Francesco Sovrano +2
Università della Svizzera italiana Lugano, Switzerland