Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality

Xiaoyuan Zhu; Kimberly Le Truong; Riccardo Fogliato; Gokul Swamy; Weijian Zhang; Minglai Yang; Longtian Ye; Bangya Liu

cs.HCApr 6, 2026

Justified or Just Convincing? Error Verifiability as a Dimension of LLM Quality

Authors: Xiaoyuan Zhu, Kimberly Le Truong, Riccardo Fogliato, Gokul Swamy, Weijian Zhang, Minglai Yang, Longtian Ye, Bangya Liu,

Organizations: University of Southern California · Carnegie Mellon University · Microsoft Core AI · 2077AI

Abstract

As LLMs are deployed in high-stakes settings, users must judge the correctness of individual responses, often relying on model-generated justifications such as reasoning chains or explanations. Yet, no standard measure exists for whether these justifications help users distinguish correct answers from incorrect ones. We formalize this idea as error verifiability and propose $v_{\text{bal}}$ , a balanced metric that measures whether justifications enable raters to accurately assess answer correctness, validated against human raters who show high agreement. We find that neither common approaches, such as post-training and model scaling, nor more targeted interventions recommended improve verifiability. We introduce two methods that succeed at improving verifiability: reflect-and-rephrase (RR) for mathematical reasoning and oracle-rephrase (OR) for factual QA, both of which improve verifiability by incorporating domain-appropriate external information. Together, our results establish error verifiability as a distinct dimension of response quality that does not emerge from accuracy improvements alone and requires dedicated, domain-aware methods to address.

Figures & tables

	AO	AO-CoT	AJ	AJ-CoT
$\kappa$	0.065	0.481	0.501	0.488
Acc (LLMs)	0.550	0.850	0.848	0.908
Acc (Human)	0.836	—	0.809	—

Table 1: LLM–human

\kappa

and accuracy.

Model		MATH500	GSM8K	MMLU	TruthfulQA
Llama3.1-8B	Base	0.763	0.773	0.625	0.636
	Prof. $\Delta$	$+$ 0.028	$-$ 0.040	$+$ 0.029	$+$ 0.009
	Struct. $\Delta$	$+$ 0.012	$-$ 0.047	$+$ 0.037	$+$ 0.022
	Simpl. $\Delta$	$+$ 0.002	$-$ 0.080	$+$ 0.043	$+$ 0.026
	RR $\Delta$	$\mathbf{+0.040}$	$\mathbf{+0.027}$	$+$ 0.036	$+$ 0.007
	OR $\Delta$	—	—	$\mathbf{+0.086}$	$\mathbf{+0.069}$

Table 2:

v_{\text{bal}}

across all benchmarks (

\Delta

vs. Base). Per-cell results in Appendix F .

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	Stage	Acc	$v_{\text{bal}}$	FP	TN	FN	TP
GSM8K	Base	0.324	0.737	0.703	0.897	0.633	0.716
	SFT	0.659	0.854	0.717	0.872	0.872	0.955
	DPO	0.822	0.823	0.602	0.824	0.889	0.977
	Instruct	0.882	0.839	0.656	0.830	0.897	0.974
MATH500	Base	0.157	0.691	0.724	0.937	0.398	0.706
	SFT	0.306	0.755	0.626	0.899	0.626	0.869

Appendix

Table 3: Post-training checkpoint results for Tulu3.1-8B across four benchmarks. All values averaged over three LLM raters.

Dataset	Stage	Acc	$v_{\text{bal}}$	FP	TN	FN	TP
GSM8K	Base	0.308	0.672	0.442	0.909	0.569	0.766
	SFT	0.614	0.799	0.612	0.857	0.821	0.905
	DPO	0.749	0.782	0.574	0.808	0.844	0.904
	Instruct	0.754	0.781	0.541	0.820	0.870	0.892
MATH500	Base	0.143	0.653	0.639	0.945	0.285	0.742
	SFT	0.222	0.719	0.713	0.924	0.439	0.800

Appendix

Table 4: Post-training checkpoint results for OLMo2-7B across four benchmarks. All values averaged over three LLM raters.

Dataset	Model	Acc	$v_{\text{bal}}$	FP	TN	FN	TP
GSM8K	Llama3.1-8B	0.779	0.804	0.595	0.764	0.893	0.963
	Tulu3.1-8B	0.882	0.788	0.506	0.768	0.898	0.982
	OLMo2-7B	0.754	0.760	0.469	0.723	0.886	0.962
	Qwen3	0.978	0.653	0.157	0.355	0.927	0.967
	DeepSeek-V3	0.959	0.569	0.073	0.263	0.950	0.991
	Llama4-Maverick	0.972	0.640	0.223	0.429	0.916	0.991

Appendix

Table 5: Full results for seven models on mathematical reasoning benchmarks. All values averaged over three LLM raters.

Dataset	Model	Acc	$v_{\text{bal}}$	FP	TN	FN	TP
MMLU	Llama3.1-8B	0.624	0.629	0.329	0.657	0.583	0.949
	Tulu3.1-8B	0.628	0.648	0.320	0.835	0.494	0.941
	OLMo2-7B	0.563	0.654	0.479	0.844	0.405	0.890
	Qwen3	0.712	0.639	0.426	0.568	0.635	0.925
	DeepSeek-V3	0.833	0.563	0.052	0.275	0.932	0.995
	Llama4-Maverick	0.771	0.590	0.166	0.353	0.853	0.987

Appendix

Table 6: Full results for seven models on factual knowledge QA benchmarks. All values averaged over three LLM raters.

MATH500
Model		$v_{\text{bal}}$	FP	TN	FN	TP
Llama3.1-8B	Base	0.763	0.652	0.762	0.729	0.907
	Prof. $\Delta$	$+$ 0.028	$+$ 0.057	$+$ 0.077	$-$ 0.019	$-$ 0.004
	Struct. $\Delta$	$+$ 0.012	$+$ 0.034	$+$ 0.090	$-$ 0.033	$-$ 0.041
	Simpl. $\Delta$	$+$ 0.002	$+$ 0.031	$+$ 0.082	$-$ 0.072	$-$ 0.036
	RR $\Delta$	$+$ 0.040	$+$ 0.114	$+$ 0.113	$-$ 0.050	$-$ 0.019

Appendix

Table 7: Full per-cell results for rephrase methods on mathematical benchmarks (

\Delta

relative to Base).

MMLU
Model		$v_{\text{bal}}$	FP	TN	FN	TP
Llama3.1-8B	Base	0.625	0.293	0.679	0.585	0.944
	Prof. $\Delta$	$+$ 0.029	$+$ 0.058	$+$ 0.092	$-$ 0.031	$-$ 0.004
	Struct. $\Delta$	$+$ 0.037	$+$ 0.072	$+$ 0.062	$+$ 0.013	$+$ 0.001
	Simpl. $\Delta$	$+$ 0.043	$+$ 0.090	$+$ 0.115	$-$ 0.031	$-$ 0.002
	RR $\Delta$	$+$ 0.036	$+$ 0.069	$+$ 0.049	$+$ 0.004	$+$ 0.021

Appendix

Table 8: Full per-cell results for rephrase methods on factual knowledge QA benchmarks (

\Delta

relative to Base).

MATH500
	Llama3.1-8B		Tulu3.1-8B		OLMo2-7B
Selection	Acc	$v_{\text{bal}}$	Acc	$v_{\text{bal}}$	Acc	$v_{\text{bal}}$
Random	0.430	0.762	0.446	0.797	0.310	0.717
Min Len. $\Delta$	$-$ 0.004	$+$ 0.010	$-$ 0.012	$-$ 0.104	$+$ 0.020	$+$ 0.051
Max Len. $\Delta$	$-$ 0.022	$+$ 0.001	$+$ 0.020	$+$ 0.043	$+$ 0.004	$+$ 0.012
Min Steps $\Delta$	$-$ 0.010	$-$ 0.029	$-$ 0.010	$-$ 0.043	$+$ 0.004	$+$ 0.070

Appendix

Table 9: Best-of-5 selection results on MATH500 and MMLU-Pro (

\Delta

vs. Random ). Values are means over 10 experiments of 200 questions each. Model Sel. was not evaluated on MMLU-Pro.

Paper Name	API Identifier	Role
GPT-4.1-mini	gpt-4.1-mini-2025-04-14	Rater
Claude-Haiku-4.5	claude-haiku-4-5-20251001	Rater
Gemini-2.5-Flash-Lite	gemini-2.5-flash-lite	Rater
GPT-5.2	gpt-5.2-2025-12-11	Grading
Claude-Sonnet-4.5	claude-sonnet-4-5	Fact checker
Llama3.1-8B	meta-llama/Llama-3.1-8B-Instruct	Response model

Appendix

Table 10: Mapping between model names used in this paper and their API identifiers.

Event type	Description	Severity
tab_switch	Switch to another browser tab	Severe
visibility_hidden	Window minimized/hidden	Severe
screenshot_attempt	Screenshot attempt detected	Severe
window_blur	Window loses focus	Moderate
copy_attempt	Copy action detected	Moderate
paste_attempt	Paste action detected	Moderate

Appendix

Table 11: Violation events monitored by the platform.

Field	Meaning
participantId	Unique participant identifier
itemId	Item ID (e.g., TP01, FN05, GSM-CHECK)
category	TP/TN/FP/FN/GSM-CHECK
condition	AO or AJ
questionIndex	Item position in the sequence (0-indexed)
responseCorrect	Stage 1 judgment (true/false/null)

Appendix

Table 12: Core fields recorded for each item response.

	Cohen’s $\kappa$				Accuracy
Rater	AO	AO-CoT	AJ	AJ-CoT	AO	AO-CoT	AJ	AJ-CoT
claude-haiku-4-5	$-0.060$	$0.515$	$0.529$	$0.484$	$0.475$	$0.900$	$0.795$	$0.925$
gemini-2.5-flash-lite	$0.217$	$0.436$	$0.535$	$0.479$	$0.650$	$0.775$	$0.875$	$0.875$
gpt-4.1-mini	$0.039$	$0.491$	$0.438$	$0.501$	$0.525$	$0.875$	$0.875$	$0.925$
LLM (avg.)	$0.065$	$0.481$	$0.501$	$0.488$	$0.550$	$0.850$	$0.848$	$0.908$
Human	—	—	—	—	$0.836$	—	$0.809$	—

Appendix

Table 13: Per-rater Cohen’s

\kappa

(vs. human judgments) and ground-truth accuracy across all four evaluation settings on the 40-item human study benchmark. Human judgments are collected only under the AO and AJ settings;

\kappa

is not applicable for the human row.

Metric	Value
Completed participants	19
Mean completion time (minutes)	27.2
Completion time range (minutes)	16.4 – 40.4
Attention check pass rate	100% (19/19)
Participants with 0 violations	12 (63%)
Participants with $\geq$ 1 violation	7 (37%)

Appendix

Table 14: Data collection snapshot.

LLM Evaluation

Large language model evaluation measures what models can do, how reliably they do it, and whether benchmark scores predict real-world performance.

See all 2,408 papers

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
Jul 16, 2026Jungseob Lee, Seungyoon Lee, Suhyune Son +4CoT Distillation Language Model Distillation
1Korea University · 2Zoom Communications · 3Yonsei University
Measuring Reasoning Quality in LLMs: A Multi-Dimensional Behavioral Framework
May 23, 2026Ali Şenol, Garima Agrawal, Huan LiuLLM Evaluation Language Model Robustness
Department of Computer Engineering, Tarsus University, Takba¸s M. Kartaltepe S., Tarsus 33400, Mersin, Türkiye · School of Computing and Augmented Intelligence (SCAI), Arizona State University (ASU), Tempe, AZ 85281, USA · HumaConn AI Consulting, Queen Creek, AZ 85140, USA
Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Traces
Apr 13, 2026Manas Pathak, Xingyao Chen, Shuozhe Li +2LLM Evaluation Language Model Generation Evaluation
University of Texas at Austin

Jul 16, 2026J/K move · Enter open · S save

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

Jungseob Lee, Seungyoon Lee, Suhyune Son, Dongyub Jude Lee+3

1Korea University · 2Zoom Communications · 3Yonsei University

CoT Distillation Language Model Distillation LLM Reasoning Reasoning Verification Structured Reasoning in Language Models

Open paper AskPDF arXiv

A standard recipe for distilling the reasoning ability of large language models (LLMs) is to sample chains of thought from the model, keep those that reach the correct final answer, and fine-tune on the survivors. When sampling fails, a common fix shows the generator the gold answer and asks it to write a chain that reaches that answer. We show that this second step degrades the training data in a way that correctness filtering cannot catch. We run a controlled experiment that fixes the generator, the problem set, and the correctness filter, and varies only whether the chain is generated under answer-conditioning, the gold answer shown with a request to reach it. Training a strong instruction-tuned reasoning model on its own answer-conditioned chains sharply lowers its verifiable-reasoning accuracy. The loss grows with difficulty, reaching as much as about 27 points on the hardest competition problems. The mechanism is legible in the chains themselves, which rationalize backward from the shown answer instead of deriving it, with the early final-answer statement as the measurable symptom. The harm is a property of the data rather than the generator, read off unlabeled generations before any fine-tuning, ordering the penalty across eight thinking models from four families, and transferring across teacher families. A prompt ablation localizes it to the rationalize-toward instruction rather than the answer's bare visibility. The practical takeaway is to generate answer-blind, because no correctness filter can see this damage in the data.

Abstract

LLM Evaluation

Explore similar work

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models

Measuring Reasoning Quality in LLMs: A Multi-Dimensional Behavioral Framework

Filtered Reasoning Score: Evaluating Reasoning Quality on a Model's Most-Confident Traces