Auditing Conformal Prediction for Language Models Under Inference-Time Model Shift

Achref Doula; Max Mühlhäuser

cs.LGOct 20, 2026

Auditing Conformal Prediction for Language Models Under Inference-Time Model Shift

Authors: Achref Doula, Max Mühlhäuser

Organizations: Tongji University, Shanghai, China · Technical University of Darmstadt, Darmstadt, Germany

Abstract

Conformal prediction (CP) for language models is typically evaluated under a matched protocol, in which calibration and deployment use the same post-training inference configuration, such as quantization mode and inference temperature. In deployment, these choices often differ from those used at calibration to meet memory or throughput constraints or serving policies. We argue that this exposes a limitation in current CP evaluation protocols for language models. We formalize the underlying phenomenon as inference-time model shift: a change in the conformal score distribution induced by inference configuration rather than by the input--label distribution, and theoretically characterize how it breaks split-conformal exchangeability even when the task distribution and trained model are unchanged. We then evaluate its consequences across 8 language models, 4 finite-label benchmarks, and 12 inference configurations. We find that matched evaluation can substantially overstate deployment-time coverage, and that the effect is heterogeneous across models and datasets. We argue that inference configuration should be reported as a first-class experimental axis in CP evaluations for language-model systems.

Figures & tables

Figure 1: MMLU-Pro with deployment fixed to

\texttt{nf4}@T{=}0.5

and nominal target

0.9

. Matched calibration tracks target across eight model families; mismatched calibration, differing only in inference configuration between calibration and deployment, produces substantial coverage degradation.

Figure 2: Coverage with deployment fixed to

\texttt{nf4}@T=0.5

. Coverage is shown for 6 CP approaches across eight models and four datasets.

Figure 3: Coverage with deployment temperature fixed to

T=0.5

, averaged over deployment quantization modes. Coverage is shown for all six procedures across eight models and four datasets.

Figure 4: Coverage with deployment quantization fixed to nf4 , averaged over deployment temperatures. Coverage is shown for all six procedures across eight models and four datasets.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Work	Primary focus	Relation to our setting
Kumar et al. [ 23 ]	CP for multiple-choice QA with LLMs	Finite-label LLM setting, but not calibration–deployment mismatch across inference configurations.
Quach et al. [ 31 ]	Conformal language modeling for generated candidate sets	CP for generation under a fixed score-generating setup.
Mohri and Hashimoto [ 27 ]	Conformal factuality guarantees	Correctness/factuality guarantees, not inference-configuration-induced score shift.
Cherian et al. [ 5 ]	Enhanced conformal validity for LLM outputs	Stronger validity notions for LLM outputs, but not quantization/temperature mismatch.
Su et al. [ 35 ]	API-only CP without logit access	Black-box uncertainty scores with the inference interface treated as fixed.
Wang et al. [ 42 ]	Correctness coverage for open-ended generation	Open-ended correctness coverage, not coverage transport across inference configurations.

Appendix

Table 1: Representative prior work and its relation to calibration–deployment inference-configuration mismatch.

Dataset	$n_{\mathrm{cal}}$	$n_{\mathrm{test}}$	$n_{\mathrm{total}}$
CommonsenseQA	6000	6000	12000
DBPedia-14	8000	50000	58000
MMLU-Pro	2500	8000	10500
MobileActions	2500	5000	7500

Appendix

Table 2: Dataset sizes used throughout the benchmark.

Panel of Figure 5 — Figure 5: Pairwise vanilla calibration $\rightarrow$ deployment structure for Qwen2.5-1.5B under LAC with $\alpha=0.1$ . Each panel corresponds to one dataset. White cell annotations report the exact values used to construct the heatmap.

Panel of Figure 6 — Figure 6: Pairwise vanilla calibration $\rightarrow$ deployment structure for Qwen2.5-3B under LAC with $\alpha=0.1$ .

Panel of Figure 7 — Figure 7: Pairwise vanilla calibration $\rightarrow$ deployment structure for Qwen2.5-7B under LAC with $\alpha=0.1$ .

Panel of Figure 8 — Figure 8: Pairwise vanilla calibration $\rightarrow$ deployment structure for DeepSeek-R1-Distill-Qwen-7B under LAC with $\alpha=0.1$ .

Panel of Figure 9 — Figure 9: Pairwise vanilla calibration $\rightarrow$ deployment structure for Gemma-2-2B under LAC with $\alpha=0.1$ .

Panel of Figure 10 — Figure 10: Pairwise vanilla calibration $\rightarrow$ deployment structure for Llama-3.2-3B under LAC with $\alpha=0.1$ .

Panel of Figure 11 — Figure 11: Pairwise vanilla calibration $\rightarrow$ deployment structure for Phi-3-mini under LAC with $\alpha=0.1$ .

Panel of Figure 12 — Figure 12: Pairwise vanilla calibration $\rightarrow$ deployment structure for Mistral-7B under LAC with $\alpha=0.1$ .

Model	Coverage	Set size	$\Gamma$ coverage	Avg. $\|\Gamma\|$
Qwen2.5-1.5B	0.962 $\pm$ 0.023	2.652 $\pm$ 0.402	0.896 $\pm$ 0.062	4.582 $\pm$ 0.605
Qwen2.5-3B	0.998 $\pm$ 0.002	4.957 $\pm$ 0.012	0.945 $\pm$ 0.046	5.120 $\pm$ 1.795
Qwen2.5-7B	0.954 $\pm$ 0.015	2.007 $\pm$ 0.198	0.946 $\pm$ 0.027	3.762 $\pm$ 0.635
DeepSeek-R1-7B	0.952 $\pm$ 0.013	3.404 $\pm$ 0.198	0.917 $\pm$ 0.018	5.082 $\pm$ 0.376
Gemma-2-2B	0.973 $\pm$ 0.008	3.573 $\pm$ 0.139	0.878 $\pm$ 0.038	4.728 $\pm$ 0.135
Llama-3.2-3B	0.967 $\pm$ 0.013	2.826 $\pm$ 0.223	0.903 $\pm$ 0.067	6.965 $\pm$ 0.382

Appendix

Table 23: Diagnostics for Latent-Robust on CommonsenseQA in the main-paper setting: LAC,

\alpha=0.1

, deployment

\mathrm{nf4}@T=0.5

\alpha_{\mathrm{stage1}}=\beta_{\mathrm{stage2}}=0.05

, and

d_{1}=0.5

. Each entry reports mean

\pm

standard deviation across runs.

Model	Coverage	Set size	$\Gamma$ coverage	Avg. $\|\Gamma\|$
Qwen2.5-1.5B	0.947 $\pm$ 0.008	2.505 $\pm$ 0.319	0.967 $\pm$ 0.011	5.595 $\pm$ 0.595
Qwen2.5-3B	0.946 $\pm$ 0.002	2.342 $\pm$ 0.034	0.975 $\pm$ 0.004	6.142 $\pm$ 0.183
Qwen2.5-7B	0.966 $\pm$ 0.000	0.993 $\pm$ 0.000	0.955 $\pm$ 0.008	4.703 $\pm$ 0.548
DeepSeek-R1-7B	0.956 $\pm$ 0.003	3.026 $\pm$ 0.200	0.993 $\pm$ 0.002	3.032 $\pm$ 0.022
Gemma-2-2B	0.987 $\pm$ 0.005	10.787 $\pm$ 0.672	0.972 $\pm$ 0.001	3.042 $\pm$ 0.030
Llama-3.2-3B	0.947 $\pm$ 0.004	3.576 $\pm$ 0.266	0.984 $\pm$ 0.000	3.076 $\pm$ 0.028

Appendix

Table 24: Diagnostics for Latent-Robust on DBPedia-14 in the main-paper setting: LAC,

\alpha=0.1

, deployment

\mathrm{nf4}@T=0.5

\alpha_{\mathrm{stage1}}=\beta_{\mathrm{stage2}}=0.05

, and

d_{1}=0.5

. Each entry reports mean

\pm

standard deviation across runs.

Model	Coverage	Set size	$\Gamma$ coverage	Avg. $\|\Gamma\|$
Qwen2.5-1.5B	0.954 $\pm$ 0.003	8.533 $\pm$ 0.022	0.963 $\pm$ 0.007	6.411 $\pm$ 0.306
Qwen2.5-3B	1.000 $\pm$ 0.000	9.464 $\pm$ 0.017	0.926 $\pm$ 0.013	6.281 $\pm$ 0.342
Qwen2.5-7B	1.000 $\pm$ 0.001	9.461 $\pm$ 0.018	0.942 $\pm$ 0.008	6.511 $\pm$ 0.041
DeepSeek-R1-7B	0.958 $\pm$ 0.007	8.491 $\pm$ 0.095	0.969 $\pm$ 0.003	5.068 $\pm$ 0.033
Gemma-2-2B	0.972 $\pm$ 0.001	8.656 $\pm$ 0.033	0.953 $\pm$ 0.026	8.149 $\pm$ 0.143
Llama-3.2-3B	0.957 $\pm$ 0.004	8.152 $\pm$ 0.075	0.950 $\pm$ 0.028	8.069 $\pm$ 0.190

Appendix

Table 25: Diagnostics for Latent-Robust on MMLU-Pro in the main-paper setting: LAC,

\alpha=0.1

, deployment

\mathrm{nf4}@T=0.5

\alpha_{\mathrm{stage1}}=\beta_{\mathrm{stage2}}=0.05

, and

d_{1}=0.5

. Each entry reports mean

\pm

standard deviation across runs.

Model	Coverage	Set size	$\Gamma$ coverage	Avg. $\|\Gamma\|$
Qwen2.5-1.5B	0.940 $\pm$ 0.010	1.049 $\pm$ 0.029	0.947 $\pm$ 0.008	11.652 $\pm$ 0.050
Qwen2.5-3B	0.961 $\pm$ 0.001	1.265 $\pm$ 0.008	0.969 $\pm$ 0.004	6.037 $\pm$ 0.033
Qwen2.5-7B	0.955 $\pm$ 0.001	0.998 $\pm$ 0.000	0.975 $\pm$ 0.004	11.808 $\pm$ 0.019
DeepSeek-R1-7B	0.946 $\pm$ 0.006	1.277 $\pm$ 0.035	0.952 $\pm$ 0.011	8.161 $\pm$ 1.169
Gemma-2-2B	0.942 $\pm$ 0.003	1.674 $\pm$ 0.014	0.968 $\pm$ 0.004	3.115 $\pm$ 0.012
Llama-3.2-3B	0.943 $\pm$ 0.007	1.451 $\pm$ 0.084	0.978 $\pm$ 0.008	4.453 $\pm$ 0.624

Appendix

Table 26: Diagnostics for Latent-Robust on MobileActions in the main-paper setting: LAC,

\alpha=0.1

, deployment

\mathrm{nf4}@T=0.5

\alpha_{\mathrm{stage1}}=\beta_{\mathrm{stage2}}=0.05

, and

d_{1}=0.5

. Each entry reports mean

\pm

standard deviation across runs.

Conformal Prediction for Language Models

See all 15 papers

Conformalized Large Language Models under Configuration Shift
Aug 2, 2026Yuqicheng Zhu, Jialin Yu, Lin Li +7Conformal Prediction Conformal Prediction for Language Models
University of Stuttgart · Robert Bosch GmbH · University of Oxford +4
Audited Conformal Prediction for Classification under Unknown Distribution Shift
Jun 12, 2026Yanfei Zhou, Rizal Fathony, Nam H. Nguyen +1Distribution Shift Robustness Uncertainty Quantification
Department of Data Sciences and Operations University of Southern California Los Angeles, CA 90089 · AI Foundations, Capital One New York, NY 10017 · Department of Data Sciences and Operations Thomas Lord Department of Computer Science University of Southern California Los Angeles, CA 90089
ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers
Oct 9, 2026Bishnu Bista, Dhanraj Haibatrao Jadhav, Vibha Choudhary +4Language Model Calibration Conformal Prediction
Independent Researcher · University of Massachusetts Amherst

Aug 2, 2026J/K move · Enter open · S save

Conformalized Large Language Models under Configuration Shift

Yuqicheng Zhu, Jialin Yu, Lin Li, Gengyuan Zhang+6

University of Stuttgart · Robert Bosch GmbH · University of Oxford+4

Conformal Prediction Conformal Prediction for Language Models Language Model Calibration LLM Uncertainty Estimation

Open paper AskPDF arXiv

Conformal prediction (CP) is a distribution-free framework for uncertainty quantification that has recently been adapted to large language models (LLMs), providing prediction sets with finite-sample coverage guarantees under exchangeability. Yet for LLMs, nonconformity scores are often induced by an inference pipeline, not just a fixed model, making them depend not only on the data distribution but also on configurable factors such as the prompt template, decoding parameters, and deployment setting. Since such configurations are routinely modified in practice but rarely treated as a source of shift, their impact on CP validity remains poorly understood. We call this configuration shift and study it systematically along three axes: prompt template, decoding temperature, and weight quantization. In a broad empirical study spanning

9

LLMs,

4

datasets, and

4

nonconformity scores, we find that configuration shift consistently erodes CP validity, often driving empirical coverage below the target. By contrast, efficiency is largely preserved: valid prediction sets remain close in size to the i.i.d. baseline. We derive coverage lower bounds that attribute this loss to a discrepancy between calibration and test score distributions, and use their finite-sample plug-in versions as empirical diagnostics of shift severity. We further show that these findings lead to practical mitigations: bound-inspired recalibration is effective with limited test examples, while fragility-aware calibration ensembling recovers much of the lost coverage without test data.

Abstract

Conformal Prediction for Language Models

Explore similar work

Conformalized Large Language Models under Configuration Shift

Audited Conformal Prediction for Classification under Unknown Distribution Shift

ConformalGuard: False-Positive-Controlled Action Gating for LM Classifiers