Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact

Michael Hardy; Yunsung Kim

cs.LGMar 1, 2026

Knowledge without Wisdom: Measuring Misalignment between LLMs and Intended Impact

Authors: Michael Hardy, Yunsung Kim

Organizations: Stanford University

Abstract

LLMs increasingly excel on AI benchmarks, but doing so does not guarantee validity for downstream tasks. This study contrasts LLM alignment on benchmarks, downstream tasks, and, importantly the intended impact of those tasks. We evaluate the performance of leading LLMs (i.e., generative pre-trained base models) on difficult-to-verify tasks of the teaching and learning of schoolchildren. Across all LLMs, inter-model behaviors on disparate tasks correlate higher than they do with expert human behaviors on target tasks. These biases shared across LLMs are poorly aligned with downstream measures of teaching quality and often negatively aligned with the intended impact of student learning outcomes. Further, we find multi-model ensembles, both unanimous model voting and expert-weighting by benchmark performance, further exacerbate misalignment with learning. We measure that selection of LLM and/or prompting strategy only reliably accounts for $15\%$ of all measured misalignment error and that variation in misalignment error is shared across LLMs, suggesting that common pretraining accounts for much of the misalignment in these tasks. We demonstrate methods for robustly measuring alignment of complex tasks and provide unique insights into practical applications of LLMs in high-noise contexts.

Figures & tables

Figure 1: Cascading levels of inference found in LLM development and evaluation and a parallel adaption of Ackoff’s progression ( Ackoff, 1989 ; Rowley, 2007 )

Figure 2: Task Data and Experimental Design . Each LLM is provided classroom transcripts. Using several prompting techniques for each model, LLMs place an ordinal rating on the quality on an aspect of teaching and learning. This is done across seven distinct tasks. We evaluate for alignment of LLM values, and not for accuracy, when comparing the relative ranking provided by each LLM on each task with human experts and with student learning gains for the class.

Figure 3: Mean Inter-task Bias Corrected Squared Distance Correlations

\operatorname{dCor}^{2}_{n}

: between LLMs and human raters across different evaluation tasks. Top row: Same-task Correlation Mean inter-rater distance correlations across transcripts for the same task and ( bottom row: different task correlation ) for different tasks using the same transcript. ( left: correlations with humans ) Mean inter-rater distance correlations with expert human raters, ( center: correlations with other LLMs ) with other LLMs, and ( right: intramodel intertask correlations ) each LLM with itself. The top right is omitted for redundancy. Lines are standard errors for each estimated mean. Means and SEs were computed under Fisher’s

z

transformation and back transformed to preserve variance. The complete correlation matrix for each task and model are found in Figs. 10 and 11

Figure 4: (Mis)alignment with Downstream Task (Teaching) and Intended Impact (Learning) : The x-axes measure the alignment of scores ( $S_{f}$ ) from each LLM, ensemble, and baseline $f$ with expert human ratings on downstream tasks ( $X$ ) on the quality of teaching skills in a given lesson: $\tau_{S_{f}X}$ . Similarly, the y-axes measure alignment with the value-added to learning via student achievement gains ( $Y$ ): $\tau_{S_{f}Y}$ . Each color-shape combination represents a different rater family or baseline. Each column represents a specific instructional rating task, from ( left to right ): CLBM, CLINSTD, LANGIMP , and REMED . Each row represents a distinct implementation scenario: Top row (individual rating models) The 95% CI are shown for individual models. Middle row (Pedagogy Expertise Weighted Ensembles) and Bottom row (Unanimous Vote Ensembles) : The alignment correlations of all possible model-prompt ensembles are represented by the density contours. The innermost contours represent the alignment regions’ greatest ensemble performance density. For the bottom row ensembles, we amplify the shared signal by only measuring observations where all three models are in agreement (See Appendix D.3 ). Baselines : the gray line and shaded regions on CLBM and CLINSTD represent the estimate and confidence intervals from the human expert rating alignment with student learning gains on the study’s joint test set. The green Encoder models (data from Hardy (2024) and our own replications) are LMs but not LLMs, and they merely serve as a deep learning baseline for possible alignment based on transcripts. The baseline of Teacher Experience represent pairwise comparisons that always rank more experienced teachers higher than less experiened ones. For better plotting details, the “oracle” VAM baseline which puts a higher value to teachers with higher prior year VAMs, is only displayed on the bottom row. Random baselines (both uniform and stratified) predictably clustered around the origin and, with no real-world example of this as a valid baseline, we exclude them from the plots.

Facet of Error Variation	median	MAD	MAP	95% HDI
ITEM	0.08	0.08	0.03	[0.01,0.62]
LLM	0.07	0.04	0.06	[0.02,0.18]
OBS	0.02	0.01	0.02	[0,0.03]
PROMPT	0.02	0.03	0.00	[0,0.58]
ITEM:OBS	0.04	0.01	0.05	[0.01,0.06]
LLM:ITEM	0.03	0.01	0.03	[0.01,0.06]

Table 1: Misalignment Shares : Proportion of Variation in Squared Error by Source. Row partitions separate main, second-order, and higher-order effect posterior distributions.

Appendix figures & tables19 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 5: Proportions of human rater scores by item.

Figure 6: Proportions of rater scores by MQI item.

Item	Item Name	[Factor] Item Description
MQI
EXPL	Teacher Explanations	[Mathematical Instruction] Teacher explanations that give meaning to ideas, procedures, steps, or solution methods.
LANGIMP †	Imprecision in Language or Notation	[Mathematical Errors] Imprecision in language or notation, with regard to mathematical symbols and technical or general mathematical language.
REMED	Remediation of Student Errors and Difficulties	[Mathematical Instruction] Remediation of student errors and difficulties addressed in a substantive manner.
SMQR	Student Mathematical Questioning and Reasoning	[Mathematical Instruction] Student mathematical questioning and reasoning, such as posing mathematically motivated questions, offering mathematical claims or counterclaims.
CLASS

Appendix

Table 2: CLASS and MQI item descriptions and corresponding abbreviations from the test set of Wang and Demszky . †denotes items that are reverse coded due to being negatively worded with respect to the construct of teacher ability. Bolded items are the focus items of the present alignment study. Bracketed item descriptions are the names of the factors with highest identified loadings by Blazar et al. per category in Appendix 2.b of the original study Kane et al. (2015) . While in the present impact alignment study we only evaluated four items, we performed a full replication of all seven items, the responses and outputs of which can be found in the online data. Variance decompositions for all seven items on the test set can be found in Appendix C .

Task	$\tau>0$	Low. 90% CI $>0$	Thiel Sen $>0$	RME $>0$	$T>$ Rand	$T\geq T$ . Exp	$Q_{4}>Q_{1}$	Robust. Pass Rate
CLBM	0.07, Y	0.06***, Y	0.013, Y	0.008, Y	$-0.04$ , Y	0.0004, Y ( $-0.04$ , 0.04)	0.02**, Y	100 (7/7)
CLINSTD	0.02, Y	0.003*, Y	0.003, Y	0.0002, Y	0.004, Y	0.0004, Y ( $-0.04$ , 0.04)	0.02**, Y	100 (7/7)
LANGIMP	0.12, Y	0.10***, Y	0.06, Y	0.05, Y	0.03, Y	0.04, Y (0.000, 0.08)	0.03**, Y	100 (7/7)
REMED	0.02, Y	0.001*, Y	0.008, Y	0.004, Y	$-0.01$ , Y	0.04, Y (0.000, 0.08)	0.01, N	71 (5/7)

Appendix

Table 3: Robustness Test Results Across Different Tasks

Figure 7: Bayesian Error Variance Decomposition : ( left ) fully crossed facet diagram for sources of variance. ( right ) corresponding brms code listing.

variable	median	MAD	MAP	HDI	mean	SD	CI	$\hat{R}$
ITEM	0.08	0.08	0.03	[0.01,0.62]	0.14	0.17	[0.01,0.54]	0.99
LLM	0.07	0.04	0.06	[0.02,0.18]	0.08	0.05	[0.02,0.16]	1.00
OBS	0.02	0.01	0.02	[0,0.03]	0.02	0.01	[0.01,0.03]	1.01
PROMPT	0.02	0.03	0.00	[0,0.58]	0.08	0.15	[0,0.46]	1.00
ITEM:OBS	0.04	0.01	0.05	[0.01,0.06]	0.04	0.01	[0.02,0.06]	1.00
LLM:ITEM	0.03	0.01	0.03	[0.01,0.06]	0.03	0.01	[0.01,0.05]	1.00

Appendix

Table 4: Bayesian Squared Error Variance Decomposition

(\hat{e})

Parameter Estimates and Fit Statistics

variable	median	MAD	MAP	HDI	mean	SD	CI	$\hat{R}$
ITEM	0.03	0.04	0.01	[0,0.34]	0.07	0.09	[0,0.22]	1.00
LLM	0.09	0.04	0.07	[0.03,0.24]	0.10	0.05	[0.03,0.19]	1.01
OBS	0.03	0.01	0.03	[0.01,0.05]	0.03	0.01	[0.01,0.04]	1.01
PROMPT	0.00	0.01	0.00	[0,0.5]	0.06	0.13	[0,0.47]	1.05
ITEM:OBS	0.12	0.02	0.12	[0.06,0.15]	0.11	0.02	[0.07,0.14]	1.05
LLM:ITEM	0.06	0.02	0.06	[0.03,0.1]	0.06	0.02	[0.03,0.1]	1.02

Appendix

Table 5: Bayesian Signed Squared Error Variance Decomposition

(\hat{e}^{\pm})

Parameter Estimates and Fit Statistics

variable	median	MAD	MAP	HDI	mean	SD	CI	$\hat{R}$
ITEM	0.29	0.13	0.27	[0.14,0.63]	0.33	0.14	[0.15,0.59]	0.99
LLM	0.04	0.03	0.04	[0.01,0.16]	0.05	0.04	[0.01,0.12]	0.99
OBS	0.07	0.02	0.07	[0.03,0.09]	0.07	0.02	[0.04,0.09]	1.00
PROMPT	0.00	0.00	0.00	[0,0.23]	0.02	0.06	[0,0.13]	1.01
ITEM:OBS	0.09	0.02	0.09	[0.04,0.12]	0.08	0.02	[0.05,0.11]	1.01
LLM:ITEM	0.04	0.01	0.04	[0.02,0.07]	0.04	0.02	[0.02,0.07]	1.00

Appendix

Table 6: Bayesian Signed Squared Error Extended Variance Decomposition

(\hat{e}^{\pm}_{(+)})

Parameter Estimates and Fit Statistics for all replication study items. Note the large difference in Item variation when other items are included and the decrease in Prompting volatility.

Metric	Unit of Measure	$\hat{e}$	$\hat{e}^{\pm}$	$\hat{e}^{\pm}_{(+)}$
$E\hat{\rho}^{2}$	$\mathcal{C}\times\mathcal{I}$	0.657	0.837	0.819
$\widehat{\Phi}$	$\mathcal{C}\times\mathcal{I}$	0.551	0.790	0.781
$E\hat{\rho}^{2}$	$\{\mathcal{C}\times\mathcal{I},\mathcal{C},\mathcal{I}\}$	0.819	0.839	0.946
$\widehat{\Phi}$	$\{\mathcal{C}\times\mathcal{I},\mathcal{C},\mathcal{I}\}$	0.732	0.809	0.931

Appendix

Table 7: Decision Study : performing a similar set of estimations to the LLM variance decomposition, except using the human experts on same test sets. We estimate the coefficients for generalizability and dependability for each of the same decompositions,

\hat{e}

\hat{e}^{\pm}

, and

\hat{e}^{\pm}_{(+)}

for two units of measurement

Figure 8: Reliability of Relative Shared Error Signal for Item

\times

Obs : Bayesian Decision Study for item-transcript scores as the object of study across varying numbers of LLMs and Prompting Techniques required to remove LLM and prompt-specific idiosyncrasies from the error signal. Each value is calculated directly from sampling the posterior and extracting the median. For this study, where

N_{\text{LLM}}=16

and

N_{\text{Prompt Strat.}}=3

, the reliability

E\widehat{\rho}^{2}_{ic}

0.66\pm 0.02

(

\hat{R}=1.00

). This suggests that the estimated item-transcript scores in this study achieve approximately the target level of consistency expected for these types of data (

0.65

, see Ho and Kane 2013 ).

Figure 9: Reliability of Absolute Shared Error Signal : Bayesian Decision Study for any of item, transcript, or their interaction as the objects of measurement (

\sigma^{2}_{\alpha}=\sigma^{2}_{i}+\sigma^{2}_{c}+\sigma^{2}_{ci}

) across varying numbers of LLMs and Prompting Techniques required to remove LLM and prompt-specific idiosyncrasies from the error signal. Each value is calculated directly from sampling the posterior and extracting the median. For this study, where

N_{\text{LLM}}=16

and

N_{\text{Prompt Strat.}}=3

, the reliability

\widehat{\Phi}

, Equation 9 , of the mean rating across models and prompts shows is 0.73 (

\hat{R}=1.00

Facet	Pct( $\hat{e}$ )	Pct $(\hat{e}^{\pm})$	Pct $(\hat{e}^{\pm}_{(+)})$
ITEM	33.76	63.75	69.27
OBS	0.00	0.00	1.10
RATERID	0.00	0.00	0.00
ITEM:OBS	25.22	13.73	11.84
RATERID:OBS	12.85	8.01	3.69
RATERID:ITEM	4.42	3.27	3.35

Appendix

Table 8: Human Error Variance Decomposition : performing a similar set of estimations to the LLM variance decomposition, except using the human experts on same test sets. We estimate the percentage of human expert-VAM error for each of the same decompositions,

\hat{e}

\hat{e}^{\pm}

, and

\hat{e}^{\pm}_{(+)}

using Restricted Maximum Likelihood with lmer ( Bates et al., 2015 ) .

Figure 10: Between and Within LLM Distance Correlations: MQI , distance correlations nonparametric measure of dependence between and within rater families across MQI items. Correlation are conducted at the item-transcript level using pairwise-complete observations. Nonsignificant relationships (at $\alpha<0.05$ ) are shown as blank after adjusting for family-wise error rate using the Bonferroni correction. Hierarchical clustering is done using complete linkages.

Figure 11: Between and Within LLM Distance Correlations: CLASS , distance correlations nonparametric measure of dependence between and within rater families across CLASS items. Correlation are conducted at the item-transcript level using pairwise-complete observations. Nonsignificant relationships (at $\alpha<0.05$ ) are shown as blank after adjusting for family-wise error rate using the Bonferroni correction. Hierarchical clustering is done using complete linkages.

Parameter	Group	Effect Type	Coefficient	CI Low	CI High
(Intercept)	—	Fixed	0.46	0.33	0.58
Segment	$w_{k}$	Random ( $\sigma^{2}_{k}$ )	0.03	–	–
	RATERID:OBSID:ITEM	Random	0.09	–	–
	RATERID:ITEM:NCTETID	Random	0.03	–	–
	RATERID:OBS_CHAPS	Random	0.03	–	–
	RATERID:ITEM	Random	0.05	–	–

Appendix

Table 9: Mixed Effect Estimates for Teacher Skill Model

Parameter	Fit
R2 (conditional)	0.26
R2 (marginal)	0.14
Sigma	0.12

Appendix

Table 10: Measures of fit for Residualization

Item	Estimate (CI)	p.value
All	-0.156 (-0.205, -0.107)	0
CLBM	-0.356 (-0.426, -0.287)	0
CLINSTD	-0.139 (-0.21, -0.069)	0
LANGIMP	-0.181 (-0.23, -0.131)	0
REMED	-0.233 (-0.282, -0.183)	0
CLPC	-0.141 (-0.212, -0.071)	0

Appendix

Table 11: Mean VAM-alignment estimates across models after noise control, including three additional items from replication study.

Figure 12: see the caption of Figure 4 for full description of details. The difference is that the axes have undergone the noise controlling transformations of this appendix prior to plotting.

Parameter	Coefficient	CI	t	p
DISTRICT [11]	0.41	[0.15, 0.67]	3.09	0.00
DISTRICT [12]	0.20	[-0.06, 0.46]	1.54	0.12
DISTRICT [13]	0.60	[0.34, 0.86]	4.52	0.00
DISTRICT [14]	0.14	[-0.12, 0.4]	1.07	0.29
GRADE	-0.10	[-0.16, -0.04]	-3.47	0.00
V CS ALT IRT M TM1	0.14	[0.14, 0.14]	319.98	0.00

Appendix

Table 24

LLM Alignment

Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.

See all 611 papers

LLMs Can Better Capture Human Judgments--With the Right Prompts
Jun 10, 2026Danica Dillion, Chen Cecilia Liu, Baihui Wang +5Human Preference Evaluation LLM Alignment
Complexity Science Hub, Vienna · The Ohio State University · University of Cambridge +2
Redirected, Not Removed: Task-Dependent Stereotyping Reveals the Limits of LLM Alignments
Apr 3, 2026Divyanshu Kumar, Ishita Gupta, Nitin Aravind Birur +3Social Bias in Language Models Language Model Bias Evaluation
Enkrypt AI
LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations
Sep 26, 2026Mohan Li, Chengyu Yu, Francesco Sovrano +2LLM Alignment Language Model Safety Evaluation
Università della Svizzera italiana Lugano, Switzerland

Jun 10, 2026J/K move · Enter open · S save

LLMs Can Better Capture Human Judgments--With the Right Prompts

Danica Dillion, Chen Cecilia Liu, Baihui Wang, Daniele Barolo+4

Complexity Science Hub, Vienna · The Ohio State University · University of Cambridge+2

Human Preference Evaluation LLM Alignment Moral Reasoning in Language Models Human Preference Modeling LLM Prompting LLM Uncertainty Estimation

Open paper AskPDF arXiv

Are large language models (LLMs) bad at capturing human judgment? Two commonly stated limitations are that LLMs fail to capture full distributions of responses, and that their judgments are unstable across wording variations. We demonstrate simple prompting strategies that mitigate these limitations. Across two datasets--a U.S.-representative set of 144 moral scenarios and 38 moral beliefs from the International Social Survey Programme's Family and Changing Gender Roles module covering 32 countries--we show how simple elicitation techniques help improve AI-human alignment. First, prompting models to report standard deviations and response proportions recovers the full range of human responses better than common strategies. Second, ensuring scenarios are clear to human participants--as reflected in human confusion ratings--boosts model alignment, and LLMs can track human confusion ratings. At the same time, we find that LLMs' estimates of their own error are poorly calibrated, though they can predict human variability relatively well. These results suggest that asking better questions to LLMs can yield better answers.

Abstract

LLM Alignment

Explore similar work

LLMs Can Better Capture Human Judgments--With the Right Prompts

Redirected, Not Removed: Task-Dependent Stereotyping Reveals the Limits of LLM Alignments

LLM Alignment--Utility Asymmetry under Semantic-Preserving Transformations