Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens

Somin Kim; Robert Feldt; Shin Yoo

cs.SEOct 20, 2026

Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens

Authors: Somin Kim, Robert Feldt, Shin Yoo

Organizations: KAIST Republic of Korea · Chalmers University of Technology Sweden

Abstract

Testing LLM-based software is expensive: each input needs several sampled responses, each judged by a costly oracle. Further, every prompt or model change calls for re-testing. Pre-generation adequacy measures cut this cost by ranking inputs by failure risk based on the model's internal state. However, the state of the art, Clotho, models only passing reference inputs, and its hidden-state representation says nothing about why inputs fail. We present Foreword, a framework built on two ideas: learn from both passing and failing references, and represent inputs through the Jacobian lens (J-lens), an interpretability technique that reads intermediate activations out as scores over vocabulary tokens. Foreword pairs these representations with two estimators: a sparse logistic-regression probe or dual Gaussian Mixture Models (dual GMMs), which also yield a reference density (to flag inputs the references do not cover) and failing-class components (to group similar failures). On ten tasks and three open-weight LLMs, all four combinations of representations and estimators outperform Clotho on average, both for prediction of failure (ROC-AUC improves by 0.14 on average and by up to 0.31) and test prioritisation (15% more failures among the 100 top-ranked inputs on average, up to 33%). Prioritisation obtained using small models also transfers better to proprietary models (on average 22% and up to 41% more failures in the first quarter of prioritised inputs). Importantly, our qualitative case study shows that Foreword supports semantic characterisation of failures by pointing to ambiguities in labels or task specifications. Combined, Foreword can support testers for prioritising inputs, checking support for a prediction, and characterising specific failures, all without generating output.

Figures & tables

Task	Inputs	Description	Llama 3.1 8B	Gemma 2 9B	Qwen 3 8B
ODD-ADD	6,000	Odd-number summation [ 31 ]	0.60	0.60	0.69
GH-TYPO	10,000	Github Typo correction [ 17 ]	0.51	0.64	0.58
JSON-FIX	6,563	Repair invalid JSON [ 10 ]	0.75	0.83	0.94
MODEL-EX	9,810	Extract ML model names [ 9 ]	0.55	0.67	0.75
POS-TAG	15,359	Detect Part-of-speech tags [ 27 ]	0.77	0.81	0.79
SPELL-CHK	10,000	Fix misspelt words [ 14 , 21 ]	0.82	0.88	0.86

Table 1: Target tasks, datasets, and pass rates (

p=0.5

) of the studied open-weight LLMs.

	Gemma-2-9B					Llama-3.1-8B					Qwen3-8B
Task ID	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR
ODD-ADD	0.682	0.799	0.838	0.809	0.836	0.811	0.880	0.901	0.858	0.896	0.726	0.792	0.828	0.818	0.837
GH-TYPO	0.676	0.811	0.838	0.790	0.851	0.723	0.807	0.848	0.801	0.848	0.621	0.723	0.813	0.779	0.839
JSON-FIX	0.726	0.818	0.866	0.752	0.859	0.721	0.758	0.823	0.764	0.812	0.595	0.709	0.800	0.751	0.828
MODEL-EX	0.624	0.848	0.890	0.846	0.873	0.673	0.820	0.858	0.782	0.859	0.471	0.720	0.773	0.723	0.785
POS-TAG	0.631	0.628	0.708	0.712	0.740	0.561	0.622	0.697	0.683	0.731	0.596	0.674	0.745	0.668	0.752

Table 2: ROC-AUC for pre-generation pass/fail prediction ( Best , second best )

Task ID	Gemma-2-9B				Llama-3.1-8B				Qwen3-8B
	LOHS var	TOK prob	TOK ent	LIHS LR	LOHS var	TOK prob	TOK ent	LIHS LR	LOHS var	TOK prob	TOK ent	LIHS LR
ODD-ADD	0.742	0.434	0.755	0.838	0.705	0.764	0.714	0.901	0.723	0.825	0.831	0.828
GH-TYPO	0.868	0.657	0.800	0.838	0.868	0.710	0.755	0.848	0.870	0.829	0.857	0.813
JSON-FIX	0.736	0.545	0.604	0.866	0.827	0.628	0.645	0.823	0.617	0.531	0.539	0.800
MODEL-EX	0.808	0.391	0.581	0.890	0.846	0.599	0.611	0.858	0.776	0.547	0.558	0.773
POS-TAG	0.772	0.526	0.627	0.708	0.803	0.628	0.651	0.697	0.711	0.596	0.623	0.745

Table 3: ROC-AUC for post-generation pass/fail prediction, with LIHS LR as the pre-generation reference.

		Gemma-2-9B						Llama-3.1-8B						Qwen3-8B
Task ID	$N$	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Rand.	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Rand.	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Rand.
ODD-ADD	100	100.0	100.0	100.0	100.0	100.0	38.4	99.2	93.7	99.0	90.4	99.3	44.3	83.7	94.5	93.4	73.1	95.4	36.8
	300	299.9	299.9	299.7	300.0	298.2	115.5	292.9	278.5	296.0	272.0	296.1	132.2	228.3	263.4	265.2	234.0	270.5	99.4
	500	463.9	472.1	479.0	484.8	481.6	193.3	476.5	461.9	487.9	446.8	488.4	221.7	360.2	413.6	424.9	394.7	431.9	165.2
GH-TYPO	100	83.3	89.9	89.9	90.2	94.4	35.1	91.9	96.1	98.3	91.1	95.1	52.3	73.2	86.2	94.1	87.0	96.1	42.3
	300	223.6	265.5	267.4	268.5	278.5	106.0	272.4	291.7	295.1	279.0	288.3	151.8	211.4	246.1	273.2	260.1	280.6	128.5

Table 4: Mean Failure@

N

(

N\in\{100,300,500\}

) for pre-generation methods and baselines.

Dual GMM						LR
Target	JSON-FIX		Target	TOPIC-CLS		Target	JSON-FIX		Target	TOPIC-CLS
OOD Source	FPR95	ROC-AUC	OOD Source	FPR95	ROC-AUC	OOD Source	FPR95	ROC-AUC	OOD Source	FPR95	ROC-AUC
TOPIC-CLS	0.342 $\pm$ 0.28	0.940 $\pm$ 0.05	JSON-FIX	0.004 $\pm$ 0.00	0.999 $\pm$ 0.00	TOPIC-CLS	0.812 $\pm$ 0.25	0.629 $\pm$ 0.23	JSON-FIX	0.372 $\pm$ 0.26	0.861 $\pm$ 0.13
SPELL-CHK	0.172 $\pm$ 0.11	0.973 $\pm$ 0.02	SPELL-CHK	0.196 $\pm$ 0.07	0.965 $\pm$ 0.01	SPELL-CHK	0.961 $\pm$ 0.05	0.441 $\pm$ 0.20	SPELL-CHK	0.353 $\pm$ 0.13	0.871 $\pm$ 0.06
SYN-BUG	0.222 $\pm$ 0.15	0.952 $\pm$ 0.03	SYN-BUG	0.001 $\pm$ 0.00	0.999 $\pm$ 0.00	SYN-BUG	0.782 $\pm$ 0.30	0.581 $\pm$ 0.26	SYN-BUG	0.874 $\pm$ 0.11	0.411 $\pm$ 0.23
MODEL-EX	0.005 $\pm$ 0.00	0.999 $\pm$ 0.00	MODEL-EX	0.455 $\pm$ 0.11	0.892 $\pm$ 0.03	MODEL-EX	0.959 $\pm$ 0.07	0.227 $\pm$ 0.24	MODEL-EX	0.958 $\pm$ 0.04	0.551 $\pm$ 0.23

Table 5: Per-source and average out-of-distribution detection results.

Figure 1: True-topic (left) and predicted-topic (right) distributions among selected failures in each failing-class component of Foreword on TOPIC-CLS with Llama3-8B (seed 0).

	$\|F_{j}\|$	Top-20 tokens
$F_{0}$	194	law, legal, legality, laws, legally, legislation, legislative, lawyers, juris, lawyer, lawsuits, legis, jur, judicial, legislature, government, courtroom, justice, legisl, regulatory
$F_{1}$	111	global, international, globally, worldwide, geopol, globalization, geopolitical, internationally, internacional, globe, diplomatic, world, geography, glob, ernational, diplomacy, diplom, unesco, globals, europe
$F_{2}$	269	companies, company, tech, business, businesses, technology, corporations, corporate, software, microsoft, industry, computer, industries, empresas, corporation, empresa, ibm, marketing, computers, manufacturers
$F_{3}$	69	entertainment, celebrity, celebrities, personality, cultural, persona, hobby, fandom, person, personalities, recreational, personas, personen, musician, individuals, science, history, human, humanity, americans
LR +	–	mvp, wereld, cultural, etsy, climate, locals, entertainment, administration, countryside, vendor, lawsuits, globals, china, lobbyist, healthcare, venture, travel, incidents, salesforce, nokia
LR -	–	nationally, wrestling, juventus, uruguay, ski, asia, yemen, 科学[science], surname, african, terms, ge, 산업[industry], products, automotive, otechnology, retail, monarch, brokerage, corp

Table 6: Failing-class component tokens on TOPIC-CLS and the top 20 tokens with positive (LR + , failure) or negative (LR - , passing) LR probe weights.

Panel of Figure 2 — Figure 2: Representative failures of the failing-class components

Target	Source	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Target	Source	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR
Llama-3.3-70B	Llama-3.1-8B	0.546	0.562	0.614	0.545	0.610	GPT-4o-mini	Llama-3.1-8B	0.515	0.547	0.597	0.564	0.607
	Gemma-2-9B	0.483	0.574	0.634	0.586	0.626		Gemma-2-9B	0.477	0.590	0.644	0.591	0.648
	Qwen3-8B	0.488	0.574	0.621	0.577	0.633		Qwen3-8B	0.481	0.590	0.662	0.622	0.676
Gemma-3-27B	Llama-3.1-8B	0.522	0.573	0.617	0.581	0.611	Claude Haiku 4.5	Llama-3.1-8B	0.479	0.498	0.537	0.475	0.528
	Gemma-2-9B	0.520	0.606	0.664	0.619	0.673		Gemma-2-9B	0.415	0.492	0.553	0.502	0.556
	Qwen3-8B	0.482	0.580	0.652	0.613	0.656		Qwen3-8B	0.435	0.513	0.582	0.533	0.587

Table 7: Normalised failure@

25\%

on each target model, grouped by the open-weight source model used for scoring, averaged over ten tasks and ten seeds.

Panel of Figure 3 — Figure 3: Computational cost of representation extraction and scorer fitting for each task–model combination.

Language Model Performance Prediction

See all 30 papers

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs
Sep 22, 2025Juyeon Yoon, Somin Kim, Robert Feldt +1LLM Evaluation LLM Reliability
Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · Chalmers University of Technology, Gothenburg, Sweden
Robust LLM Performance Certification via Constrained Maximum Likelihood Estimation
Mar 11, 2026Minghe Shen, Ananth Balashankar, Adam Fisch +2LLM Reliability LLM Evaluation
University College London · Google DeepMind
Auditing Failure Attribution: Label Predictability and Judge Behavior
Oct 13, 2026Tianyang Xu, Jing GaoFailure Attribution for LLM Agents AI Agent Evaluation
Purdue University

Sep 22, 2025J/K move · Enter open · S save

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

Juyeon Yoon, Somin Kim, Robert Feldt, Shin Yoo

Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · Chalmers University of Technology, Gothenburg, Sweden

LLM Evaluation LLM Reliability

Open paper AskPDF arXiv

Software increasingly relies on the emergent capabilities of Large Language Models (LLMs), from natural language understanding to program analysis and generation. Yet testing them on specific tasks remains difficult and costly: many prompts lack ground truths, forcing reliance on human judgments, while existing test adequacy measures typically rely on output uncertainty and thus are only available after full inference. A key challenge is to assess how useful a test input is in a way that reflects the demands of the task, ideally before even generating any output. We introduce Clotho, a task-specific, pre-generation test adequacy measure that estimates input difficulty directly from LLM hidden states. Given a large pool of unlabelled inputs for a specific task, Clotho uses a Gaussian Mixture Model (GMM) to adaptively sample the most informative cases for human labelling. Based on this reference set the GMM can then rank unseen inputs by their likelihood of failure. In our empirical evaluation across eight benchmark tasks and three open-weight LLMs, Clotho can predict failures with a ROC-AUC of 0.716, after labelling reference sets that are on average only 5.4% of inputs. It does so without generating any outputs, thereby significantly reducing LLM execution costs compared to output-based uncertainty or confidence measures. Comparison of Clotho and these post-generation adequacy measures shows that the two approaches complement each other. Crucially, we show that adequacy scores learnt from open-weight LLMs transfer effectively to proprietary models, extending the applicability of the approach. When prioritising test inputs for proprietary models, Clotho increases the average number of failing inputs from 18.7 to 42.5 out of 100, compared to random prioritisation.

Abstract

Language Model Performance Prediction

Explore similar work

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

Robust LLM Performance Certification via Constrained Maximum Likelihood Estimation

Auditing Failure Attribution: Label Predictability and Judge Behavior