Auditing System-1 Models on Biosecurity-Relevant Benchmarks: Calibration, Selective Prediction, and Permutation Instability in a Non-Generative Model

Kimon Antonios Provatas; Ilias Georgakopoulos-Soares

cs.LGSep 24, 2026

Auditing System-1 Models on Biosecurity-Relevant Benchmarks: Calibration, Selective Prediction, and Permutation Instability in a Non-Generative Model

Authors: Kimon Antonios Provatas, Ilias Georgakopoulos-Soares

Organizations: Division of Pharmacology and Toxicology, College of Pharmacy The University of Texas at Austin, Dell Pediatric Research Institute Austin, TX, USA

Abstract

Non-generative "System-1" models return structured probabilistic decisions in a single forward pass, without autoregressive decoding, at a small fraction of the inference cost of a generative model. This makes them of interest as inexpensive components in larger pipelines, but their reliability on biosecurity-relevant tasks has not been systematically examined. We audit one commercial System-1 model on 6,020 multiple-choice items drawn from the Weapons of Mass Destruction Proxy (WMDP), a paraphrase-robust WMDP-Bio variant, and six LAB-Bench subtasks, measuring accuracy, calibration, error detection, selective prediction, and sensitivity to the order in which answer options are presented. Accuracy is strongly task-dependent. Once the vendor's uncertainty field is correctly interpreted, the model is reasonably well calibrated (pooled expected calibration error 0.034) and its top-1 probability separates correct from incorrect predictions (pooled AUROC 0.820), though both degrade substantially on the weaker tasks. Under four cyclic rotations of the answer options, 37.4% of WMDP-Cyber items receive different answers; a control using byte-identical repeated calls attributes most of this to option order rather than run-to-run variation. Averaging probabilities across rotations improves WMDP-Cyber accuracy by 3.8 percentage points, and applying it only to low-confidence items recovers most of that gain at well under the cost of averaging every item.

Figures & tables

Suite	Dataset	$N$	opts.	accuracy [95% CI]	lift	ECE	AUROC	$P(\text{err}\mid p_{\max}\!\geq\!0.9)$	retained
WMDP	WMDP-Bio	1273	4	0.852 [0.83, 0.87]	0.802	0.036	0.855	0.039 [0.03, 0.05]	0.971
	WMDP-Bio-Robust	811	4	0.801 [0.77, 0.83]	0.735	0.044	0.823	0.063 [0.04, 0.09]	0.938
	WMDP-Chem	408	4	0.730 [0.69, 0.77]	0.641	0.057	0.824	0.056 [0.03, 0.10]	0.898
	WMDP-Cyber	1987	4	0.634 [0.61, 0.66]	0.512	0.033	0.774	0.075 [0.06, 0.10]	0.789
LAB-Bench	LB-CloningScenarios	32	4–8	0.469 [0.31, 0.64]	0.314	0.126	0.671	0.500 [0.09, 0.91]	0.579
	LB-ProtocolQA	108	4–7	0.537 [0.44, 0.63]	0.397	0.124	0.620	0.077 [0.01, 0.33]	0.600

TABLE I : Per-dataset results. Lift is chance-adjusted accuracy,

(\mathrm{acc}-c)/(1-c)

, using each item’s own option count

c=1/n

rather than a single

25\%

line, since option counts range from 2 to 10. ECE and AUROC are computed on

p_{\max}

rather than on the vendor confidence field (Section IV-A ).

P(\text{err}\mid p_{\max}\!\geq\!0.9)

is the error rate given a confident answer, with Wilson intervals. Retained is accuracy on the items kept when the least confident 40% are withheld (Section IV-B ).

Fig. 1 : Uncertainty and selective prediction. (a) Pooled reliability curves for the two quantities the API reports: the confidence field (squares) is an affine rescaling of

p_{\max}

(circles), so calibration measured against it differs from calibration measured against the probability itself. Bins holding fewer than 25 items are omitted, because

p_{\max}\geq 1/n

leaves the lowest bins of the

p_{\max}

curve nearly empty; error bars are Wilson intervals. (b) Accuracy on retained items against coverage, withholding the least confident first, for the three WMDP suites and the largest LAB-Bench subtask (dashed); the remaining six subtasks are omitted for legibility and appear in Table I . The dashed vertical marker is the 40% deferral operating point quoted in Section IV-B and in Table I .

Fig. 2 : Answer-order instability and selective permutation averaging. (a) Items answered differently across four byte-identical calls, which isolates run-to-run variation, against four cyclic rotations, which adds option order; the annotation gives the difference. Wilson intervals. (b) Accuracy gain over a single evaluation against mean inference calls per item, plotted as a gain so that two datasets 22 pp apart in absolute accuracy share one scale. Items are selected for additional permutations least-confident first, and compared against random selection at the same budget (averaged over 400 draws) and against an oracle that selects, under a fixed budget, the items where averaging helps.

Language Model Calibration

Language model calibration studies whether models’ confidence estimates match their actual accuracy across tasks, prompts, and deployment settings.

See all 383 papers

TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models
Oct 8, 2026Rahul Sharma, Andrew B. Duncan, Gaétan Marceau Caron +1LLM Decision-Making Language Model Calibration
DFKI GmbH · RPTU Kaiserslautern · Imperial College London +1
The Value of System 1 Models in Medicine
Oct 22, 2026Xiufeng Yang, Konstantinos Vrettos, Zeineb Ben Chaaben +17Clinical Language Model Evaluation Multimodal Model Evaluation
Department of Diagnostic and Interventional Radiology, School of Medicine and Health, Technical University of Munich, Munich, Germany · Artificial Intelligence and Translational Imaging Lab, Department of Radiology, School of Medicine, University of Crete, Heraklion, Greece · Department of Medical Imaging, University Hospital of Heraklion, Heraklion, Crete, Greece +6
System1-Arena: Which Decisions Can Be Left to System 1?
Oct 15, 2026Tianyu Chen, Xinyang Liu, Ruyi An +2LLM Decision-Making LLM Evaluation
The University of Texas at Austin

Oct 8, 2026J/K move · Enter open · S save

TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models

Rahul Sharma, Andrew B. Duncan, Gaétan Marceau Caron, Sebastian J. Vollmer

DFKI GmbH · RPTU Kaiserslautern · Imperial College London+1

LLM Decision-Making Language Model Calibration Framing Effects in Language Models

Open paper AskPDF arXiv

System One models output calibrated probabilities over typed answers such as categorical choices, ordinal levels, or binary outcomes, via a non-generative interface. Software can act on these probabilities through thresholds, cost-weighted choices, and escalation rules. Consequently, if these probabilities are miscalibrated or wording-sensitive, the software ma take unintended actions leaving human operators with no textual rationale to inspect. Current evaluations largely report accuracy and calibration on public classification datasets without a clear reference. We present TypedBench, a benchmark for typed decision models like Jev built from seven policy-labelled generators and nine evaluation suites. We report accuracy as median and range across paraphrases, and calibration error relative to the finite-sample noise floor of a matched, perfectly calibrated predictor. We assess probability quality through selective prediction, ordinal proper scoring rules, and realised cost under asymmetric cost matrices. We evaluate a hosted model, an open encoder, and a family of open decoders spanning 0.8B-9B parameters on identical items. The hosted model follows the stated policy but is wording-sensitive and systematically underconfident; under asymmetric costs, using its probabilities can be worse than taking its top answer. Decoders route exactly and are slow as options or questions are added. The decoder is least accurate on policy questions and degrades with more options. Overall, typed decision models must be evaluated jointly on policy adherence, wording robustness, probability quality, and induced decision outcomes.

Abstract

Language Model Calibration

Explore similar work

TypedBench: A Benchmark for Calibration, Framing Sensitivity, and Cost in System One Decision Models

The Value of System 1 Models in Medicine

System1-Arena: Which Decisions Can Be Left to System 1?