Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens

Somin Kim; Robert Feldt; Shin Yoo

cs.SEOct 20, 2026

Foreword: Pre-Generation Prediction and Characterisation of LLM Test Failures using the Jacobian Lens

Authors: Somin Kim, Robert Feldt, Shin Yoo

Organizations: KAIST Republic of Korea · Chalmers University of Technology Sweden

Abstract

Testing LLM-based software is expensive: each input needs several sampled responses, each judged by a costly oracle. Further, every prompt or model change calls for re-testing. Pre-generation adequacy measures cut this cost by ranking inputs by failure risk based on the model's internal state. However, the state of the art, Clotho, models only passing reference inputs, and its hidden-state representation says nothing about why inputs fail. We present Foreword, a framework built on two ideas: learn from both passing and failing references, and represent inputs through the Jacobian lens (J-lens), an interpretability technique that reads intermediate activations out as scores over vocabulary tokens. Foreword pairs these representations with two estimators: a sparse logistic-regression probe or dual Gaussian Mixture Models (dual GMMs), which also yield a reference density (to flag inputs the references do not cover) and failing-class components (to group similar failures). On ten tasks and three open-weight LLMs, all four combinations of representations and estimators outperform Clotho on average, both for prediction of failure (ROC-AUC improves by 0.14 on average and by up to 0.31) and test prioritisation (15% more failures among the 100 top-ranked inputs on average, up to 33%). Prioritisation obtained using small models also transfers better to proprietary models (on average 22% and up to 41% more failures in the first quarter of prioritised inputs). Importantly, our qualitative case study shows that Foreword supports semantic characterisation of failures by pointing to ambiguities in labels or task specifications. Combined, Foreword can support testers for prioritising inputs, checking support for a prediction, and characterising specific failures, all without generating output.

Figures & tables

Task	Inputs	Description	Llama 3.1 8B	Gemma 2 9B	Qwen 3 8B
ODD-ADD	6,000	Odd-number summation [ 31 ]	0.60	0.60	0.69
GH-TYPO	10,000	Github Typo correction [ 17 ]	0.51	0.64	0.58
JSON-FIX	6,563	Repair invalid JSON [ 10 ]	0.75	0.83	0.94
MODEL-EX	9,810	Extract ML model names [ 9 ]	0.55	0.67	0.75
POS-TAG	15,359	Detect Part-of-speech tags [ 27 ]	0.77	0.81	0.79
SPELL-CHK	10,000	Fix misspelt words [ 14 , 21 ]	0.82	0.88	0.86

Table 1: Target tasks, datasets, and pass rates (

p=0.5

) of the studied open-weight LLMs.

	Gemma-2-9B					Llama-3.1-8B					Qwen3-8B
Task ID	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR
ODD-ADD	0.682	0.799	0.838	0.809	0.836	0.811	0.880	0.901	0.858	0.896	0.726	0.792	0.828	0.818	0.837
GH-TYPO	0.676	0.811	0.838	0.790	0.851	0.723	0.807	0.848	0.801	0.848	0.621	0.723	0.813	0.779	0.839
JSON-FIX	0.726	0.818	0.866	0.752	0.859	0.721	0.758	0.823	0.764	0.812	0.595	0.709	0.800	0.751	0.828
MODEL-EX	0.624	0.848	0.890	0.846	0.873	0.673	0.820	0.858	0.782	0.859	0.471	0.720	0.773	0.723	0.785
POS-TAG	0.631	0.628	0.708	0.712	0.740	0.561	0.622	0.697	0.683	0.731	0.596	0.674	0.745	0.668	0.752

Table 2: ROC-AUC for pre-generation pass/fail prediction ( Best , second best )

Task ID	Gemma-2-9B				Llama-3.1-8B				Qwen3-8B
	LOHS var	TOK prob	TOK ent	LIHS LR	LOHS var	TOK prob	TOK ent	LIHS LR	LOHS var	TOK prob	TOK ent	LIHS LR
ODD-ADD	0.742	0.434	0.755	0.838	0.705	0.764	0.714	0.901	0.723	0.825	0.831	0.828
GH-TYPO	0.868	0.657	0.800	0.838	0.868	0.710	0.755	0.848	0.870	0.829	0.857	0.813
JSON-FIX	0.736	0.545	0.604	0.866	0.827	0.628	0.645	0.823	0.617	0.531	0.539	0.800
MODEL-EX	0.808	0.391	0.581	0.890	0.846	0.599	0.611	0.858	0.776	0.547	0.558	0.773
POS-TAG	0.772	0.526	0.627	0.708	0.803	0.628	0.651	0.697	0.711	0.596	0.623	0.745

Table 3: ROC-AUC for post-generation pass/fail prediction, with LIHS LR as the pre-generation reference.

		Gemma-2-9B						Llama-3.1-8B						Qwen3-8B
Task ID	$N$	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Rand.	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Rand.	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Rand.
ODD-ADD	100	100.0	100.0	100.0	100.0	100.0	38.4	99.2	93.7	99.0	90.4	99.3	44.3	83.7	94.5	93.4	73.1	95.4	36.8
	300	299.9	299.9	299.7	300.0	298.2	115.5	292.9	278.5	296.0	272.0	296.1	132.2	228.3	263.4	265.2	234.0	270.5	99.4
	500	463.9	472.1	479.0	484.8	481.6	193.3	476.5	461.9	487.9	446.8	488.4	221.7	360.2	413.6	424.9	394.7	431.9	165.2
GH-TYPO	100	83.3	89.9	89.9	90.2	94.4	35.1	91.9	96.1	98.3	91.1	95.1	52.3	73.2	86.2	94.1	87.0	96.1	42.3
	300	223.6	265.5	267.4	268.5	278.5	106.0	272.4	291.7	295.1	279.0	288.3	151.8	211.4	246.1	273.2	260.1	280.6	128.5

Table 4: Mean Failure@

N

(

N\in\{100,300,500\}

) for pre-generation methods and baselines.

Dual GMM						LR
Target	JSON-FIX		Target	TOPIC-CLS		Target	JSON-FIX		Target	TOPIC-CLS
OOD Source	FPR95	ROC-AUC	OOD Source	FPR95	ROC-AUC	OOD Source	FPR95	ROC-AUC	OOD Source	FPR95	ROC-AUC
TOPIC-CLS	0.342 $\pm$ 0.28	0.940 $\pm$ 0.05	JSON-FIX	0.004 $\pm$ 0.00	0.999 $\pm$ 0.00	TOPIC-CLS	0.812 $\pm$ 0.25	0.629 $\pm$ 0.23	JSON-FIX	0.372 $\pm$ 0.26	0.861 $\pm$ 0.13
SPELL-CHK	0.172 $\pm$ 0.11	0.973 $\pm$ 0.02	SPELL-CHK	0.196 $\pm$ 0.07	0.965 $\pm$ 0.01	SPELL-CHK	0.961 $\pm$ 0.05	0.441 $\pm$ 0.20	SPELL-CHK	0.353 $\pm$ 0.13	0.871 $\pm$ 0.06
SYN-BUG	0.222 $\pm$ 0.15	0.952 $\pm$ 0.03	SYN-BUG	0.001 $\pm$ 0.00	0.999 $\pm$ 0.00	SYN-BUG	0.782 $\pm$ 0.30	0.581 $\pm$ 0.26	SYN-BUG	0.874 $\pm$ 0.11	0.411 $\pm$ 0.23
MODEL-EX	0.005 $\pm$ 0.00	0.999 $\pm$ 0.00	MODEL-EX	0.455 $\pm$ 0.11	0.892 $\pm$ 0.03	MODEL-EX	0.959 $\pm$ 0.07	0.227 $\pm$ 0.24	MODEL-EX	0.958 $\pm$ 0.04	0.551 $\pm$ 0.23

Table 5: Per-source and average out-of-distribution detection results.

Figure 1: True-topic (left) and predicted-topic (right) distributions among selected failures in each failing-class component of Foreword on TOPIC-CLS with Llama3-8B (seed 0).

	$\|F_{j}\|$	Top-20 tokens
$F_{0}$	194	law, legal, legality, laws, legally, legislation, legislative, lawyers, juris, lawyer, lawsuits, legis, jur, judicial, legislature, government, courtroom, justice, legisl, regulatory
$F_{1}$	111	global, international, globally, worldwide, geopol, globalization, geopolitical, internationally, internacional, globe, diplomatic, world, geography, glob, ernational, diplomacy, diplom, unesco, globals, europe
$F_{2}$	269	companies, company, tech, business, businesses, technology, corporations, corporate, software, microsoft, industry, computer, industries, empresas, corporation, empresa, ibm, marketing, computers, manufacturers
$F_{3}$	69	entertainment, celebrity, celebrities, personality, cultural, persona, hobby, fandom, person, personalities, recreational, personas, personen, musician, individuals, science, history, human, humanity, americans
LR +	–	mvp, wereld, cultural, etsy, climate, locals, entertainment, administration, countryside, vendor, lawsuits, globals, china, lobbyist, healthcare, venture, travel, incidents, salesforce, nokia
LR -	–	nationally, wrestling, juventus, uruguay, ski, asia, yemen, 科学[science], surname, african, terms, ge, 산업[industry], products, automotive, otechnology, retail, monarch, brokerage, corp

Table 6: Failing-class component tokens on TOPIC-CLS and the top 20 tokens with positive (LR + , failure) or negative (LR - , passing) LR probe weights.

Panel of Figure 2 — Figure 2: Representative failures of the failing-class components

Target	Source	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR	Target	Source	Clotho	LIHS DG	LIHS LR	JRS DG	JRS LR
Llama-3.3-70B	Llama-3.1-8B	0.546	0.562	0.614	0.545	0.610	GPT-4o-mini	Llama-3.1-8B	0.515	0.547	0.597	0.564	0.607
	Gemma-2-9B	0.483	0.574	0.634	0.586	0.626		Gemma-2-9B	0.477	0.590	0.644	0.591	0.648
	Qwen3-8B	0.488	0.574	0.621	0.577	0.633		Qwen3-8B	0.481	0.590	0.662	0.622	0.676
Gemma-3-27B	Llama-3.1-8B	0.522	0.573	0.617	0.581	0.611	Claude Haiku 4.5	Llama-3.1-8B	0.479	0.498	0.537	0.475	0.528
	Gemma-2-9B	0.520	0.606	0.664	0.619	0.673		Gemma-2-9B	0.415	0.492	0.553	0.502	0.556
	Qwen3-8B	0.482	0.580	0.652	0.613	0.656		Qwen3-8B	0.435	0.513	0.582	0.533	0.587

Table 7: Normalised failure@

25\%

on each target model, grouped by the open-weight source model used for scoring, averaged over ten tasks and ten seeds.

Panel of Figure 3 — Figure 3: Computational cost of representation extraction and scorer fitting for each task–model combination.

Language Model Performance Prediction

See all 30 papers

Sep 22, 2025

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

Software increasingly relies on the emergent capabilities of Large Language Models (LLMs), from natural language understanding to program analysis and generation. Yet testing them on specific tasks remains difficult and costly: many prompts lack ground truths, forcing reliance on human judgments, while existing test adequacy measures typically rely on output uncertainty and thus are only available after full inference. A key challenge is to assess how useful a test input is in a way that reflects the demands of the task, ideally before even generating any output. We introduce Clotho, a task-specific, pre-generation test adequacy measure that estimates input difficulty directly from LLM hidden states. Given a large pool of unlabelled inputs for a specific task, Clotho uses a Gaussian Mixture Model (GMM) to adaptively sample the most informative cases for human labelling. Based on this reference set the GMM can then rank unseen inputs by their likelihood of failure. In our empirical evaluation across eight benchmark tasks and three open-weight LLMs, Clotho can predict failures with a ROC-AUC of 0.716, after labelling reference sets that are on average only 5.4% of inputs. It does so without generating any outputs, thereby significantly reducing LLM execution costs compared to output-based uncertainty or confidence measures. Comparison of Clotho and these post-generation adequacy measures shows that the two approaches complement each other. Crucially, we show that adequacy scores learnt from open-weight LLMs transfer effectively to proprietary models, extending the applicability of the approach. When prioritising test inputs for proprietary models, Clotho increases the average number of failing inputs from 18.7 to 42.5 out of 100, compared to random prioritisation.

Juyeon Yoon, Somin Kim, Robert Feldt +1

Korea Advanced Institute of Science and Technology, Daejeon, Republic of Korea · Chalmers University of Technology, Gothenburg, Sweden

LLM Evaluation LLM Reliability

Ask PDF

Mar 11, 2026

Robust LLM Performance Certification via Constrained Maximum Likelihood Estimation

The ability to rigorously estimate the failure rates of large language models (LLMs) is a prerequisite for their safe deployment. Currently, however, practitioners often face a tradeoff between expensive human gold standards and potentially severely-biased automatic annotation schemes such as "LLM-as-a-Judge" labeling. In this paper, we propose a new, practical, and efficient approach to LLM failure rate estimation based on constrained maximum-likelihood estimation (MLE). Our method integrates three distinct signal sources: (i) a small, high-quality human-labeled calibration set, (ii) a large corpus of LLM-judge annotations, and, most importantly, (iii) additional side information via domain-specific constraints derived from known bounds on judge performance statistics. We validate our approach through a comprehensive empirical study, benchmarking it against state-of-the-art baselines like Prediction-Powered Inference (PPI). Across diverse experimental regimes -- spanning varying judge accuracies, calibration set sizes, and LLM failure rates -- our constrained MLE consistently delivers more accurate and lower-variance estimates than existing methods. By moving beyond the "black-box" use of automated judges to a flexible framework, we provide a principled, interpretable, and scalable pathway towards LLM failure-rate certification.

Minghe Shen, Ananth Balashankar, Adam Fisch +2

University College London · Google DeepMind

LLM Reliability LLM Evaluation

Ask PDF

Oct 13, 2026

Auditing Failure Attribution: Label Predictability and Judge Behavior

Failure-attribution benchmarks ask an LLM judge which step or agent caused a failure. A high score should mean the judge checked the faulty step against the run (a cross-step check). However, spurious signals (surface cues) that predict the annotation can produce a high score. In this paper, we propose four tests to separate checking from cue use. Input restriction asks whether partial input predicts the annotation. Cue following checks whether moved cues change answers. Residual accuracy scores judges where cue-only predictors fail. Aggregation asks whether the failure mix or metric changes rankings. Real observations motivate these tests: on three benchmarks, simple predictors match or beat GPT-4.1. In two Who&When Pro systems, a classifier reading one step's reasoning finds 95-100% of faulty steps, motivating input restriction. In one of them, error injection leaves no reasoning after the faulty step; judges fine-tuned on its labels beat GPT-4.1 by 41 points. Adding task-related reasoning after that step moves this cue: fine-tuned judges lose 31-99 points, but zero-shot judges lose at most 9, motivating cue following. On Aegis, blaming every agent matches GPT-4.1 on micro-F1 but trails by 19 points in exact-set accuracy, motivating aggregation. We recommend reporting these tests when developing failure-attribution benchmarks.

Tianyang Xu, Jing Gao

Purdue University

Failure Attribution for LLM Agents AI Agent Evaluation LLM-as-a-Judge AI Agent Benchmarks Shortcut Learning

Ask PDF

Abstract

Language Model Performance Prediction

Explore similar work

Clotho: Measuring Task-Specific Pre-Generation Test Adequacy for LLM Inputs

Robust LLM Performance Certification via Constrained Maximum Likelihood Estimation

Auditing Failure Attribution: Label Predictability and Judge Behavior