Organizations: Department of Medical Physics, Memorial Sloan Kettering Cancer Center, New York, NY, USA · Department of Radiation Oncology, Memorial Sloan Kettering Cancer Center, New York, NY, USA
Scientific imaging data involving medical (e.g., 3D radiographic, microscopic images) and astronomical sensors pose different challenges from natural images. One key challenge is that these datasets are often orders of magnitude larger than natural images. Efficient training and inference on such data is challenging due to the increased hardware requirements, often needing bespoke solutions. Moreover, the relevant objects such as tumors and astronomical phenomena are sparsely distributed with respect to the background information. Standard efficiency techniques like quantization and pruning apply uniform scaling of all regions, potentially compromising the precision/fidelity of predictions. Thus, efficient methods that reduce computational requirements without losing resolution for relevant parts of the data and with uncertainty estimates on predictions are essential. We present Conformal Adaptation with risk guarantees for uncertainty-aware sparse computations. Our formalism is general, spanning dense pixel-level classification and regression tasks where hierarchical models predict at low resolution, certify the signal using conformal risk control, and refine the certified predictions. We benchmark our method on multiple tasks such as 3D medical image segmentation, 2D galaxy flux prediction, and 1D calcium spike rate prediction, achieving 2.5--72× wall-clock speedup across the various tasks, including on commodity CPUs.
Figures & tables
Figure 1: The two-stage process for Conformal Adaptation consists of (1) dense predictions at a spatial scale reduced by a factor r and (2) sparse refinement on the selected set S of coarse cells. The selection is based on thresholds calibrated using CRC, and the operating regime we target is ∣S∣≪∣Ωcoarse∣ .
Model
Full
Coarse
Coarse + Refine ( α=0.1 )
Cell Miss Rate
FLOPs Speedup
U-Net
0.63±0.03
0.51±0.00
0.68±0.01
0.10±0.01
2.70 ×
Swin-UNETR
0.74±0.01
0.61±0.01
0.70±0.01
0.09±0.00
54.03 ×
VISTA3D
0.74±0.00
0.62±0.01
0.71±0.00
0.08±0.01
82.10 ×
Table 1: Performance comparison on MSD Task01. Values are reported as mean ± standard deviation over three seeds. Columns 2-4 are the DSC for each model arrangement. Capacity-constrained models like U-Net see an improvement in performance due to additional weights. Larger scans require sliding window inference, increasing the FLOPs required for a single sample.
Figure 2: Left: Empirical verification of the CRC calibration quality. Middle: mean DSC of the Full baseline against the Coarse + Refine model at α=0.1 . Right: Corresponding measured wall-clock speedup over the Full baseline on different devices.
Figure 3: Predicted tumor boundaries on a held-out test scan (Swin-UNETR, α=0.1 ). The Coarse + Refine model tracks the boundary as closely as the Full baseline at a fraction of the FLOPs (Table 1 ).
Figure 4: (Left) : False negative rates (FNR) can rapidly increase depending on chosen threshold, requiring a principled methods like CRC to choose appropriate λ (See Appendix C.3 . (Middle) Tumor cells span a small fraction of the total scanned volume in most scans in the dataset. (Right) Training the coarse model on the aggregated samples and a separate refiner to convergence requires 15× fewer FLOPs. All models are trained until validation loss stops decreasing with a set patience level.
Figure 5: (Left) Visualizations of model segmentation with ground truth showing Coarse + Refine models with comparable contours. (Right) A coarse inference reduces per-scan compute cost while a sparse refinement improves prediction quality with little additional expense.
Figure 6: (a) Input signal. (b) Example test window: ground-truth spike rate overlaid with the Full prediction and the Coarse + Refine reconstruction, at a retained fraction of 8.4% .
Figure 7: (Left) Achieved real-world wall-clock speedups on GPUs and CPU thread counts. (Right) Sample selection and comparison between the Full and Coarse + Refine models. The CRC-certified Coarse + Refine predictions have little discrepancy while costing a fraction of the compute and time.
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 8: Selected (refined) region against the calibrated threshold for two held-out test scans (Swin-UNETR, seed 0). ”Full refinement” is the reference at foreground-probability threshold 0 , i.e. λ=1 in Eq. 6 (every foreground-probable cell refined); at α=1% and 2% , CRC’s calibrated threshold is itself 0 ( λ^=1 ) for this architecture/seed. As α relaxes, the selected region shrinks toward the tumor boundary and the measured cell miss rate rises toward its target.
System
GTVp
GTVn
Mean
Speedup
nnU-Net (250 ep)
0.7938
0.7271
0.7604
–
Full
0.7755
0.6957
0.7356
1.0 ×
Coarse
0.7487 ± 0.0117
0.5660 ± 0.0307
0.6573 ± 0.0145
68.3 ×
Coarse + Refine ( α=0.1 )
0.7810 ± 0.0005
0.6646 ± 0.0068
0.7228 ± 0.0034
68.1 ×
Appendix
Table 2: DSC and FLOPs speedup across models. All rows except nnU-Net are Swin-UNETR. Mean DSC is the unweighted average of GTVp and GTVn; speedup is the ratio of Full-model FLOPs to pipeline FLOPs.
GTVp
GTVn
System
Recall
Prec.
DSC
Recall
Prec.
DSC
nnU-Net (250 ep)
0.766
0.572
0.819
0.615
0.378
0.773
Full
0.685
0.625
0.822
0.553
0.526
0.772
Coarse
0.599
0.628
0.799
0.339
0.471
0.745
Coarse + Refine ( α=0.1 )
0.680
0.638
0.814
0.465
0.475
0.732
Appendix
Table 3: Lesion-specific results for our HNC models. All rows except nnU-Net are Swin-UNETR. Unlike the volumetric DSC, we perform post-processing to detect lesions from predicted volumes to calculate lesion detection metrics.
Figure 9: Grid of HECKTOR predictions. From top to bottom: nnU-Net, Swin-UNETR native resolution, Swin-UNETR Coarse, and Swin-UNETR Coarse + Refine.
FNR at fixed λ
FNR with CRC ( α=0.1 )
Dataset
Model
λ=0.5
λ=0.1
λ=0.01
Calib.
Test
Brain Tumor
UNet
0.17 ± 0.01
0.072 ± 0.008
0.021 ± 0.003
0.085 ±< 0.001
0.097 ± 0.006
Swin-UNETR
0.11 ± 0.02
0.07 ± 0.01
0.031 ± 0.006
0.086 ±< 0.001
0.091 ± 0.001
VISTA3D
0.11 ± 0.01
0.07 ± 0.01
0.04 ± 0.01
0.086 ± 0.001
0.084 ± 0.004
HECKTOR
Swin-UNETR
0.86 ± 0.02
0.25 ± 0.02
0.13 ± 0.01
0.092 ± 0.001
0.072 ± 0.009
Appendix
Table 4: A naively chosen like threshold can result in high FNR as the predicted scores are non-linear compared to the selection. CRC
Model
Role
Params
FLOPs (window = 1024)
FineUNet1D
Full baseline ( FΩfine )
10,475,204
2,668,494,848
FineOnLowRes
Coarse ( FΩcoarse )
10,475,204 (same)
333,561,856 (8.0 × fewer)
LocalRefinerUNet1D
Refiner (count head)
158,721
–
Appendix
Table 5: CASCADE model architectures, parameter counts, and analytic FLOPs. The total FLOPs for the refiner depends on the number of selected windows and is therefore omitted.
α
Retained fraction
FLOPs speedup
Reconstructed MAE ( Δ )
0.01
1.00
0.89 ×
0.30 ( +0.05 )
0.05
0.48
1.65 ×
0.34 ( +0.09 )
0.10
0.08
4.77 ×
0.32 ( +0.06 )
0.20
0.07
5.21 ×
0.35 ( +0.10 )
Appendix
Table 6: Reconstructed MAE is reported in Hz, with the gap to the full-resolution baseline ( 0.250 Hz) in parentheses.
Model
Role
Params
FLOPs
2D U-Net, base width 64
Full baseline
31,389,764
1.78×1012
Same U-Net, 8× -downsampled input
Coarse
31,389,764 (same)
2.78×1010 ( 64× fewer)
Local refinement operator
Refiner
17,809
3.05×106 per cell
Appendix
Table 7: Galaxy flux field model sizes and analytic floating-point operation counts, per 10242 field for the full-resolution baseline and the coarse model, and per certified cell for the refiner.
Conformal prediction provides set-valued predictions with distribution-free coverage guarantees, making it attractive for high-stakes image classification. However, split conformal prediction is data-inefficient, while full conformal prediction (FCP), despite its stronger statistical efficiency, is computationally prohibitive at scale because it requires candidate-specific model refits at test time. We address this limitation by leveraging zero-shot vision-language models (VLMs) to guide scalable FCP in large label spaces. We introduce Targeted Full Conformal Prediction (T-FCP), which uses a lightweight inductive conformal predictor to prune unlikely labels and applies FCP only to the remaining candidates, reducing computation while retaining the formal guarantee of the combined conformal procedure. We further propose Stabilized Online LDA (SO-LDA), an efficient VLM adaptation solver based on rank-one inverse-covariance updates. Across multiple benchmarks, including ImageNet, T-FCP enables practical full-conformal image classification with modest test-time overhead, yielding efficient prediction sets and more stable empirical coverage than split conformal alternatives.
Uncertainty quantification is critical in scientific machine learning, where black-box, image-based models are increasingly deployed in high-stakes settings. In many such applications, model outputs inform costly decisions, yet most methods provide only point estimates without quantifying predictive uncertainty. This challenge is compounded by the limited accessibility and interpretability of model internals, making it difficult to assess reliability across different regions of the input space. As a result, there is a growing need for methods that can provide input-dependent uncertainty estimates to guide both model development and downstream experimentation. To address this need, we propose Adaptive Conformal Prediction using Nearest Neighbors (ACPNN), an input-adaptive conformal framework for image regression. ACPNN leverages information from neighboring samples to produce locally adaptive uncertainty estimates while maintaining low computational cost. The neighborhood structure is defined using a scaled distance metric learned via a Gaussian Process with an automatic relevance determination (ARD) kernel. We demonstrate the effectiveness of ACPNN on a diffusion model for emulating inertial confinement fusion (ICF) simulations, showing that it achieves reliable and adaptive uncertainty quantification.
Carrie J. Lei-Cramer, Michael S. Jones, Laura J. Wendelberger
Department of Statistics, Texas A&M University, College Station, TX, USA. · Lawrence Livermore National Laboratory, Livermore, CA, USA.
The recent developments of complex deep learning models have led to unprecedented ability to accurately predict across multiple data representation types. Conformal prediction for uncertainty quantification of these models has risen in popularity, providing adaptive, statistically-valid prediction sets. For classification tasks, conformal methods have typically focused on utilizing logit scores. For pre-trained models, however, this can result in inefficient, overly conservative set sizes when not calibrated towards the target task. We propose DANCE, a doubly locally adaptive nearest-neighbor based conformal algorithm combining two novel nonconformity scores directly using the data's embedded representation. DANCE first fits a task-adaptive kernel regression model from the embedding layer before using the learned kernel space to produce the final prediction sets for uncertainty quantification. We test against state-of-the-art local, task-adapted and zero-shot conformal baselines, demonstrating DANCE's superior blend of set size efficiency and robustness across various datasets.
Brandon R. Feng, Brian J. Reich, Daniel Beaglehole +7
Department of Statistics, North Carolina State University, Raleigh, NC, USA · Amazon.com, Seattle, WA, USA · Department of Computer Science, University of California San Diego, San Diego, CA, USA +2