Mean Accuracy Is the Wrong Estimand for a Release Decision

Jaewon Choi; Jihye Kang; Namhyuk Ahn

cs.LGOct 12, 2026

Mean Accuracy Is the Wrong Estimand for a Release Decision

Authors: Jaewon Choi, Jihye Kang, Namhyuk Ahn

Organizations: Department of Transdisciplinary Medicine, Seoul National University Hospital · Department of Electrical and Computer Engineering, Inha University

Abstract

A release decision about a fine-tuned model is an operational commitment about the next rerun of its training recipe. Yet standard benchmark reporting gives only the mean accuracy of past seeds. These target fundamentally different questions: release decisions depend on catastrophic tail risk, whereas standard evaluation reports only central tendency. Because fine-tuning induces bimodal mode collapse, high average accuracy frequently conceals severe failure rates. Auditing $2\,124$ fine-tuned models across eleven language models, we show that shipping models based on mean accuracy violates a $10\%$ deployment failure budget $48.0\%$ of the time. Common heuristics like $\bar b - 2σ$ prove equally dangerous: by underestimating variance on small samples, they become most permissive when evidence is weakest. We introduce recipe cards, an evaluation framework that directly certifies rerun tail failure rates using exact finite-sample binomial bounds. Recipe cards reduce held-out budget violations to $0.6\%$ , enforce a mathematical sample floor ( $N \ge 22$ ) that prevents certification on thin evidence, and explicitly refuse unverified claims under domain and task shifts.

Figures & tables

Figure 1: Empirical bimodality of fine-tuning reruns (RoBERTa-large,

n{=}2000

N{=}59

independent seeds). The true rerun distribution splits into a high-accuracy working mode (median

0.968

) and a collapsed constant-predictor mode (

\leq 0.52

). The reported mean (

\bar{b}=0.884

) lands in the empty valley between modes, representing a phantom model that the training recipe practically never produces. A Gaussian unimodal assumption

\mathcal{N}(\bar{b},\sigma^{2})

fails to capture the true failure mass below the release bar

\tau=0.70

Figure 2: Held-out risk control and evidence responsiveness across decision rules (

36

cells,

59

reruns each, HaluEval-QA,

\tau{=}0.70

\varepsilon_{\mathrm{ship}}{=}0.10

). (a) Held-out failure rate distribution (§ 4.2 ): empirical failure rates on held-out reruns across

36\,000

audit splits whenever a rule approves shipping (diamonds indicate means; red shaded zone marks budget violation

>\varepsilon_{\mathrm{ship}}

). Central-tendency rules exhibit heavy violation tails, whereas the Recipe Card stays pinned strictly below the budget. (b) Evidence responsiveness (§ 4.3 ): cells shipped as a function of audit size

N^{\prime}

. Mean and median are sample-size blind, while

\bar{b}-2\sigma

is anti-responsive. The card certifies nothing below

N^{\prime}{=}22

by ( 4 ).

Release rule	Decision criterion	Ships ( $/36$ )	Ships $\widehat{\beta}>\varepsilon_{\mathrm{ship}}$	$\Pr[\text{over budget}\mid\text{ship}]$
Mean	$\bar{b}\geq\tau$	27	15	0.480
Median	$\mathrm{median}_{b}\geq\tau$	27	15	0.483
Bootstrap LCL	$\mathrm{Percentile}_{10}(\bar{b})\geq\tau$	27	15	0.457
$\bar{b}-2\sigma$	$\bar{b}-2\sigma_{b}\geq\tau$	11	0	0.043
Recipe card (CP)	$\mathrm{CP}_{\text{upper}}(k/N,N,\delta_{\mathrm{stat}})\leq\varepsilon_{\mathrm{ship}}$	9	0	0.006

Table 1: Auditing release decision rules on identical rerun data. Evaluated across

36

headline (model, budget) cells on HaluEval-QA (

N{=}59

\tau{=}0.70

\delta_{\mathrm{stat}}{=}\varepsilon_{\mathrm{ship}}{=}0.10

). Ships

\widehat{\beta}>\varepsilon_{\mathrm{ship}}

: count of approved recipes whose empirical failure fraction strictly exceeds the

10\%

budget (

k\geq 8

collapsed seeds out of

59

\Pr[\text{over budget}\mid\text{ship}]

: held-out violation probability across

36\,000

splits (§ 4.2 ).

Model ( $\widehat{\beta}\,/\,\mathrm{CP}_{\text{upper}}$ )	$\tau{=}0.50$	$\tau{=}0.55$	$\tau{=}0.60$	$\tau{=}0.70$
BERT-base	$0/59\,/\,0.038$ r	$0/59\,/\,0.038$ r	$0/59\,/\,0.038$ r	$0/59\,/\,0.038$ r
DeBERTa-v3-base	$0/59\,/\,0.038$ r	$1/59\,/\,0.064$ r	$2/59\,/\,0.088$ r	$3/59\,/\,0.110$ u
RoBERTa-base	$0/59\,/\,0.038$ r	$24/59\,/\,0.499$ u	$30/59\,/\,0.599$ u	$46/59\,/\,0.849$ u

Table 2: Same task, same recipe, three distinct verdicts.

N{=}59

reruns on HaluEval-QA at

n{=}500

, shared encoder recipe,

\delta_{\mathrm{stat}}{=}\varepsilon_{\mathrm{ship}}{=}0.10

: BERT-base clears every bar, DeBERTa-v3-base flips with

\tau

(release-supp. at

\tau{\leq}0.60

, unsupp. at

\tau{=}0.70

), RoBERTa-base is strong-unsupp. from

\tau{=}0.55

. r: release-supported, u: unsupported.

		calibration audit $D_{P}$		target audit $D_{Q}$
Recipe	$D_{P}\!\to\!D_{Q}$	$\widehat{\beta}$	$U_{D_{P}}$	$\widehat{\beta}$	$U_{D_{Q}}$
Mistral-7B	SST-2 $\to$ IMDB	$0/29$	$0.076$ r	$29/29$	$1.000$ u
Pythia-1.4B	SST-2 $\to$ IMDB	$0/29$	$0.076$ r	$29/29$	$1.000$ u
Mistral-7B	RTE $\to$ ANLI	$0/29$	$0.076$ r	$6/29$	$0.335$ u
Qwen3-14B	RTE $\to$ ANLI	$0/29$	$0.076$ r	$6/29$	$0.335$ u
Pythia-160M	SST-2 $\to$ IMDB	$0/29$	$0.076$ r	$0/29$	$0.076$ r

Table 3: A card earned on the calibration audit does not carry to a different deployment audit. The six recipes that earn a release-supported card at

\tau{=}0.50

N{=}29

on their calibration audit, re-measured on a labelled target audit. 4 of the 6 are refused there. r: release-supported, u : unsupported.

Artifact Family	Evaluation Unit	Output Quantity	Release Gating?
Model Cards	Checkpoint	Static metrics / text	No (Descriptive only)
Transferability	Model–task pair	Relative score / rank	No (Pairwise ranking)
Conformal / PAC-PS	Single query	Set coverage	No (Inference-time set)
Conformal Risk Control	Risk functional	Loss confidence interval	Different estimand (Mean loss)
Selective Classification	Single query	Abstention rule	No (Per-query rejection)
Recipe Cards (This Work)	Recipe cell	Rerun failure bound $U_{D_{Q}}(\tau)$	Yes (Tail risk control)

Table 4: Conceptual taxonomy of evaluation artifacts. Prior artifacts evaluate static checkpoints, rank model pairs, or provide guarantees on individual test inputs. Recipe cards fill a fundamental gap by bounding the tail failure probability of the stochastic training procedure itself.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Model	Family	Params	Type	Optimizer	Learning Rate
BERT-base	BERT [ Devlin et al., 2019 ]	110M	Encoder (Full)	AdamW	$2{\times}10^{-5}$
BERT-large	BERT [ Devlin et al., 2019 ]	340M	Encoder (Full)	AdamW	$2{\times}10^{-5}$
RoBERTa-base	RoBERTa [ Liu et al., 2019 ]	125M	Encoder (Full)	AdamW	$2{\times}10^{-5}$
RoBERTa-large	RoBERTa [ Liu et al., 2019 ]	355M	Encoder (Full)	AdamW	$2{\times}10^{-5}$
DeBERTa-v3-base	DeBERTa [ He et al., 2021 ]	86M	Encoder (Full)	AdamW	$2{\times}10^{-5}$
Pythia-160M	Pythia [ Biderman et al., 2023 ]	160M	Decoder (QLoRA)	AdamW	$2{\times}10^{-4}$

Appendix

Table 5: Evaluated model architectures and fine-tuning configurations. Encoders use full sequence-classification fine-tuning; decoders use 4-bit QLoRA with rank

r=8

and

\alpha=16

LLM Reliability

Large language model reliability studies whether outputs remain correct, calibrated, robust, and controllable across prompts, contexts, and deployment conditions.

See all 802 papers

Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims
May 11, 2026Phongsakon Mark Konrad, Toygar Tanyel, Serkan AyvazFine-Tuning Language Model Safety Evaluation
Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark · ProMake, Newark, DE, USA
Training ML Models with Predictable Failures
May 14, 2026Will Schwarzer, Scott NiekumLanguage Model Safety Evaluation LLM Fine-Tuning
University of Massachusetts Amherst
Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets
Jul 27, 2026Zongyou Yang, Yinghan HouLLM Evaluation LLM Answer Verification
Dyson School of Design Engineering Imperial College London London, United Kingdom · Department of Electrical and Electronic Engineering Imperial College London London, United Kingdom

May 11, 2026J/K move · Enter open · S save

Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims

Phongsakon Mark Konrad, Toygar Tanyel, Serkan Ayvaz

Centre for Industrial Software, University of Southern Denmark, Alsion 2, Sønderborg, 6400, Denmark · ProMake, Newark, DE, USA

Fine-Tuning Language Model Safety Evaluation LLM Auditing

Open paper AskPDF arXiv

Safe fine-tuning defenses are often endorsed on the basis of a held-out gap reduction, but the same reduction can come from sampling noise, subject artifacts, capability loss, or a mechanism that does not transfer. We introduce Acceptance Cards: an evaluation protocol, a documentation object, an executable audit package, and a claim-specific evidential standard for safe fine-tuning defense claims. The protocol checks statistical reliability, fresh semantic generalization, mechanism alignment, and cross-task transfer before treating a gap reduction as a full-card pass. Re-scored under this installed-gap protocol, SafeLoRA fails the full-card pass on Gemma-2-2B-it: under strict mechanism-class coding it fails all four diagnostics, and under a permissive shrinkage relabel it still fails three of four. This is a narrow installed-gap audit on one model family, not a global judgment of SafeLoRA's effectiveness. In a 46-cell audit, no cell satisfies the strict conjunction. The closest family is a near miss that passes reliability and mechanism checks where the required data are available, but fails the fresh-subject threshold, lacks a strict transfer pass, and carries a measurable deployment-accuracy cost.

Abstract

LLM Reliability

Explore similar work

Acceptance Cards:A Four-Diagnostic Standard for Safe Fine-Tuning Defense Claims

Training ML Models with Predictable Failures

Accuracy Hides How Language Models Fail: Measuring Failure States Under Matched Output Budgets