DANCE: Doubly Adaptive Neighborhood Conformal Estimation

Brandon R. Feng; Brian J. Reich; Daniel Beaglehole; Xihaier Luo; David Keetae Park; Shinjae Yoo; Zhechao Huang; Xueyu Mao

stat.MLFeb 24, 2026

DANCE: Doubly Adaptive Neighborhood Conformal Estimation

Authors: Brandon R. Feng, Brian J. Reich, Daniel Beaglehole, Xihaier Luo, David Keetae Park, Shinjae Yoo, Zhechao Huang, Xueyu Mao, +2 more

Organizations: Department of Statistics, North Carolina State University, Raleigh, NC, USA · Amazon.com, Seattle, WA, USA · Department of Computer Science, University of California San Diego, San Diego, CA, USA · Computational Science Initiative, Brookhaven National Laboratory, Upton, NY, USA · Amazon.com, San Diego, WA, USA

Abstract

The recent developments of complex deep learning models have led to unprecedented ability to accurately predict across multiple data representation types. Conformal prediction for uncertainty quantification of these models has risen in popularity, providing adaptive, statistically-valid prediction sets. For classification tasks, conformal methods have typically focused on utilizing logit scores. For pre-trained models, however, this can result in inefficient, overly conservative set sizes when not calibrated towards the target task. We propose DANCE, a doubly locally adaptive nearest-neighbor based conformal algorithm combining two novel nonconformity scores directly using the data's embedded representation. DANCE first fits a task-adaptive kernel regression model from the embedding layer before using the learned kernel space to produce the final prediction sets for uncertainty quantification. We test against state-of-the-art local, task-adapted and zero-shot conformal baselines, demonstrating DANCE's superior blend of set size efficiency and robustness across various datasets.

Figures & tables

Figure 1: The DANCE pipeline. (a) The calibration set is used to compute nonconformity scores for a given test input. Our architectural contribution is the introduction of a task-adapted Recursive Feature Machines (RFM) kernel together with two complementary nonconformity scores. (b) The Neighbor Nonconformity module derives a rank cutoff $q_{\mathrm{knn}}$ in the kernel space from the calibration set. This cutoff is used to identify nearest labels and to construct the candidate label set $\hat{C}_{\mathrm{knn}}$ for the test input. (c) The Contrastive Nonconformity module defines a density cutoff $q_{\mathrm{clr}}$ , below which a second candidate label set $\hat{C}_{\mathrm{clr}}$ is formed. The intersection of $\hat{C}_{\mathrm{knn}}$ and $\hat{C}_{\mathrm{clr}}$ yields the final conformal prediction set $\hat{C}_{\mathrm{dance}}$ , yielding $1-\alpha$ coverage.

Panel of Figure 2 — Figure 2: Comparison of embedding projections for original and RFM kernel-transformed space for the top 10 most frequent classes (represented by colors) of the Imagenet-R dataset. Note the per-class clustering of the transformed kernel space. (a) Original embeddings; (b) Kernel-transformed embeddings.

		$\alpha=0.1$				$\alpha=0.05$
Method	Acc	Size	Cov	Cov Range	CCV	Size	Cov	Cov Range	CCV
DANCE (Ours)	0.856	2.49	0.930	(0.903, 0.964)	8.442	4.96	0.963	(0.949, 0.986)	5.364
$k$ -NN Set (Ours)	0.856	2.33	0.921	(0.902, 0.956)	8.877	4.77	0.958	(0.949, 0.971)	5.785
CLR Set (Ours)	0.856	5.82	0.936	(0.913, 0.974)	8.128	8.20	0.965	(0.955, 0.989)	5.346
Deep $k$ -NN	0.856	4.26	0.936	(0.905, 0.987)	8.799	7.50	0.967	(0.951, 0.998)	5.265
RFM Adapter (APS)	0.856	2.23	0.904	(0.897, 0.917)	8.817	5.28	0.952	(0.945, 0.957)	5.948

Table 1: CLIP ViT-B/16: Average accuracy and UQ quality metrics (Set Size, Coverage, Coverage Range, Class-conditional Coverage Violation) across 11 datasets. Best and second best set size and CCV are bolded and underlined , respectively.

		$\alpha=0.1$				$\alpha=0.05$
Method	Acc	Size	Cov	Cov Range	CCV	Size	Cov	Cov Range	CCV
DANCE (Ours)	0.820	4.11	0.926	(0.900, 0.957)	8.746	8.07	0.962	(0.946, 0.976)	5.627
$k$ -NN Set (Ours)	0.820	4.03	0.920	(0.900, 0.961)	9.006	8.47	0.957	(0.950, 0.973)	5.963
CLR Set (Ours)	0.820	7.49	0.903	(0.903, 0.970)	8.626	9.96	0.965	(0.945, 0.989)	5.430
Deep $k$ -NN	0.820	5.66	0.935	(0.904, 0.989)	8.855	11.58	0.972	(0.957, 1.000)	5.018
RFM Adapter (APS)	0.820	3.13	0.904	(0.890, 0.915)	8.761	8.96	0.954	(0.946, 0.963)	5.982

Table 2: CLIP ResNet-101: Average accuracy and UQ quality metrics (Set Size, Coverage, Coverage Range, Class-conditional Coverage Violation) across 11 datasets. Best and second best set size and CCV are bolded and underlined , respectively.

Panel of Figure 3 — Figure 3: Ablation study of UQ based on $\lambda$ . We observe the mix of $S_{knn}$ and $S_{clr}$ generally provides a trade-off between CCV and set size.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Dataset	$\alpha$	$\mathcal{D}_{\rm ref}$	Set Size	Coverage	CCV
ImageNet	0.05	Reuse	10.75	0.953	6.450
	0.05	Disjoint	10.09	0.953	6.430
	0.10	Reuse	5.01	0.913	8.770
	0.10	Disjoint	4.55	0.914	8.760
ImageNet-R	0.05	Reuse	6.30	0.951	4.627
	0.05	Disjoint	6.15	0.955	4.525

Appendix

Table 3: Comparison of reused (Main Paper) vs. disjoint reference set for UQ quality metrics( Set Size, Coverage and Class-conditional Coverage Violation) for the 3 ImageNet variants.

Panel of Figure 4 — Figure 4: Additional ablation studies. Top Row: Impact of neighbor number for $m_{knn}$ . Bottom Row: Impact of neighbor number for $m_{clr}$

Figure 5: RFM Training Set Size vs Training Time (Both axes are log-scaled.)

Dataset	Classes	Test size	Method	Accuracy	Set size	Coverage	CCV
Imagenet	1000	10000	DANCE (Ours)	0.729	10.75	0.953	6.450
			$k$ -NN Set (Ours)	0.729	10.75	0.953	6.450
			CLR Set (Ours)	0.729	16.18	0.957	6.390
			Deep $k$ -NN	0.729	11.52	0.953	6.650
			RFM Adapter (APS)	0.729	8.04	0.951	6.500
			RFM Adapter (RAPS)	0.729	8.09	0.950	6.540

Appendix

Table 4: Conformal prediction results (part 1).

Dataset	Classes	Test size	Method	Accuracy	Set size	Coverage	CCV
Pets	37	734	DANCE (Ours)	0.955	1.09	0.963	3.514
			$k$ -NN Set (Ours)	0.955	1.09	0.966	3.784
			CLR Set (Ours)	0.955	2.46	0.969	3.814
			Deep $k$ -NN	0.955	1.37	0.970	4.219
			RFM Adapter (APS)	0.955	1.20	0.950	4.369
			RFM Adapter (RAPS)	0.955	1.20	0.952	4.354

Appendix

Table 5: Conformal prediction results (part 2).

Explore similar work

Softmax is not Enough (for Adaptive Conformal Classification)
Feb 23, 2026Navid Akhavan Attar, Hesam Asadollahzadeh, Ling Luo +1Conformal Prediction Uncertainty Quantification
School of Computing and Information Systems, Faculty of Engineering and IT The University of Melbourne, Victoria, Australia
SPACR: Single-Pass Adaptive Training of Uncertainty-Aware Conformal Regressors
Jun 9, 2026Soundouss Messoudi, Sylvain Rousseau, Sébastien DesterckeConformal Prediction Adaptive Conformal Inference
Heudiasyc - UMR CNRS 7253, Université de Technologie de Compiègne
Adaptive Conformal Prediction for Image Regression Models with Application to an Inertial Confinement Fusion Emulator
Sep 30, 2026Carrie J. Lei-Cramer, Michael S. Jones, Laura J. WendelbergerUncertainty Quantification Conformal Prediction
Department of Statistics, Texas A&M University, College Station, TX, USA. · Lawrence Livermore National Laboratory, Livermore, CA, USA.

Feb 23, 2026J/K move · Enter open · S save

Softmax is not Enough (for Adaptive Conformal Classification)

Navid Akhavan Attar, Hesam Asadollahzadeh, Ling Luo, Uwe Aickelin

School of Computing and Information Systems, Faculty of Engineering and IT The University of Melbourne, Victoria, Australia

Conformal Prediction Uncertainty Quantification

Open paper AskPDF arXiv

The merit of Conformal Prediction (CP), as a distribution-free framework for uncertainty quantification, depends on generating prediction sets that are efficient, reflected in small average set sizes, while adaptive, meaning they signal uncertainty by varying in size according to input difficulty. A central limitation for deep conformal classifiers is that the nonconformity scores are derived from softmax outputs, which can be unreliable indicators of how certain the model truly is about a given input, sometimes leading to overconfident misclassifications or undue hesitation. In this work, we argue that this unreliability can be inherited by the prediction sets generated by CP, limiting their capacity for adaptiveness. We propose a new approach that leverages information from the pre-softmax logit space, using the Helmholtz Free Energy as a measure of model uncertainty and sample difficulty. By reweighting nonconformity scores with a monotonic transformation of the energy score of each sample, we improve their sensitivity to input difficulty. Our experiments with four state-of-the-art score functions on multiple datasets and deep architectures show that this energy-based enhancement improves the adaptiveness of the prediction sets, leading to a notable increase in both efficiency and adaptiveness compared to baseline nonconformity scores, without introducing any post-hoc complexity.

Abstract

Explore similar work

Softmax is not Enough (for Adaptive Conformal Classification)

SPACR: Single-Pass Adaptive Training of Uncertainty-Aware Conformal Regressors

Adaptive Conformal Prediction for Image Regression Models with Application to an Inertial Confinement Fusion Emulator