ReCIRC: Rectified Conformal Risk Control

Bruno Marcondes e Resende; Helton Graziadei; Thiago Rodrigo Ramos; Rafael Izbicki

stat.MLSep 29, 2026

ReCIRC: Rectified Conformal Risk Control

Authors: Bruno Marcondes e Resende, Helton Graziadei, Thiago Rodrigo Ramos, Rafael Izbicki

Organizations: Department of Statistics, Federal University of São Carlos

Abstract

Many applications of black-box predictive models require controlling task-relevant error rates, such as missed lesion pixels in segmentation or missed labels in multilabel classification. Conformal risk control (CRC; Angelopoulos et al., arXiv:2208.02814) gives distribution-free guarantees for such losses, but it calibrates a single threshold shared by all inputs. Because conditional risk varies with the input, this marginal guarantee often overprotects easy cases and underprotects hard ones. We propose ReCIRC (Rectified Conformal Risk Control), which inverts each input's estimated local risk curve to reparameterize the calibrated threshold as a risk budget $a$ representing a common target conditional risk, and then applies CRC unchanged to the resulting family. ReCIRC retains CRC's finite-sample marginal guarantee regardless of the accuracy of the estimated curves, while accurate curves yield approximate conditional risk control and, under additional conditions, asymptotically exact conditional risk control; they also support a risk-calibration diagnostic. Across three synthetic and five real-data settings spanning segmentation, multilabel and multiclass classification, and regression, ReCIRC attained the lowest average worst-group risk and mean positive group excess in every setting, while maintaining marginal risk close to the target, whereas changes in prediction size were application-dependent.

Figures & tables

Figure 1: Flattening the risk curve. Conditional false-negative rate (FNR) by difficulty in the polyp-segmentation experiment (Section 5 ; target $\alpha=0.10$ ). Global CRC’s FNR rises sharply on difficult images, whereas ReCIRC keeps it near the target. Curves show means and 95% confidence intervals over 20 repetitions; images illustrate the easiest and hardest bins.

Panel of Figure 2 — Figure 2: Risk rectification as a reparameterization. (a) The same raw threshold $\lambda_{0}$ gives different conditional risks at an easy and a hard input. (b) A common risk budget $a$ instead gives each input its own threshold $R^{-1}(a\mid x)$ , obtained by inverting its local risk curve. (c) Conditional risk across inputs, with the two inputs of (a) marked. A global threshold whose average risk is $\alpha$ (here $\lambda_{0}$ ) leaves easy inputs below $\alpha$ and hard inputs above it; oracle rectification with $a=\alpha$ brings every input to $\alpha$ . In practice the curves are estimated and CRC calibrates $a$ (Section 3.2 ). The plots are normalized to $B=1$ .

Panel of Figure 3 — Figure 3: The ReCIRC pipeline. The risk-training set $\mathcal{D}$ is used only to train and tune the conditional-risk estimator; its estimated local risk curves are inverted to define the rectified family indexed by the risk budget $a$ ; the calibration set $\mathcal{C}$ is used only by the final CRC step, which selects the budget $\widehat{a}$ . Because calibration sees only observed losses of a monotone one-parameter family, marginal validity holds regardless of the quality of the estimated curves.

Figure 4: Held-out risk-calibration diagnostics for ReCIRC in the polyp-segmentation experiment. Each panel plots empirical average loss $\widehat{r}_{G}(a)$ against the risk budget $a$ , overall or within one of five groups defined by a label-free uncertainty score. Loss is the fraction of positive pixels omitted, averaged across images. Thick blue curves show means over 20 repetitions; faint curves show individual repetitions, with groups fixed within each repetition. The dashed black diagonal marks equality between risk and budget, the red dash-dotted vertical line marks $\alpha=0.10$ , and the shaded band highlights $a\in[0.05,0.20]$ .

Synthetic settings
Setting	Global CRC	AA-CRC	ReCIRC –TabICLv2
Heteroscedastic	0.2340 (0.0061)	0.1312 (0.0047)	0.1130 (0.0039)
Latent difficulty	0.2329 (0.0044)	0.1303 (0.0038)	0.1179 (0.0022)
XOR interaction	0.1338 (0.0026)	0.1166 (0.0021)	0.1130 (0.0016)
Real benchmarks
Polyp segmentation	0.3941 (0.0135)	0.1148 (0.0042)	0.1092 (0.0027)

Table 1: Worst-group risk at

\alpha=0.10

. Entries report mean values across repetitions, with standard errors in parentheses. Bold indicates the lowest mean within each setting.

Synthetic settings
Setting	Global CRC	AA-CRC	ReCIRC –TabICLv2
Heteroscedastic	0.0371 (0.0019)	0.0083 (0.0010)	0.0043 (0.0010)
Latent difficulty	0.0331 (0.0012)	0.0093 (0.0013)	0.0058 (0.0011)
XOR interaction	0.0145 (0.0012)	0.0066 (0.0008)	0.0041 (0.0006)
Real benchmarks
Polyp segmentation	0.0588 (0.0027)	0.0032 (0.0008)	0.0027 (0.0006)

Table 2: Mean positive group excess at

\alpha=0.10

. Entries report mean values across repetitions, with standard errors in parentheses. Bold indicates the lowest mean within each setting.

Synthetic settings
Setting	Global CRC	AA-CRC	ReCIRC –TabICLv2
Heteroscedastic	1.764 (0.018)	1.615 (0.021)	1.660 (0.022)
Latent difficulty	5.380 (0.030)	6.528 (0.055)	6.813 (0.044)
XOR interaction	3.948 (0.027)	4.000 (0.023)	4.065 (0.016)
Real benchmarks
Polyp segmentation	15,538 (187)	20,258 (195)	22,455 (335)

Table 3: Mean prediction size or interval width at

\alpha=0.10

. Entries report mean values across repetitions, with standard errors in parentheses. Bold indicates the lowest mean within each setting.

Synthetic settings
Setting	Global CRC	AA-CRC	ReCIRC –TabICLv2
Heteroscedastic	0.0963 (0.0022)	0.0975 (0.0017)	0.0908 (0.0021)
Latent difficulty	0.0924 (0.0017)	0.0993 (0.0023)	0.0941 (0.0021)
XOR interaction	0.0937 (0.0014)	0.1018 (0.0009)	0.0975 (0.0008)
Real benchmarks
Polyp segmentation	0.0917 (0.0027)	0.0848 (0.0010)	0.0956 (0.0014)

Table 4: Marginal risk at

\alpha=0.10

. Entries report mean values across repetitions, with standard errors in parentheses.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Setting	Base model	AA-CRC feature map	Splits (risk train / cal / test)
Synthetic settings
Heteroscedastic	GAM mean with absolute-residual intervals (regression)	RF leaf indicators; RF target: absolute residual	1,000 / 500 / 1,000
Latent difficulty	Synthetic label scores, 50 labels (multilabel)	Linear: intercept, difficulty covariate and score summaries (standardized on $D$ )	1,000 / 500 / 1,000
XOR interaction	Quantile random forest (regression)	RF leaf indicators; RF target: minimum covering multiplier	6,400 / 4,800 / 4,800
Real benchmarks
Polyp segmentation	PraNet pixelwise scores (segmentation)	90 probability quantiles (label-free)	700 / 700 / 398

Appendix

Table 5: Overview of the experimental settings. All experiments are repeated 20 times. The AA-CRC feature map column summarizes how the input-dependent score threshold is parameterized. In the RF-based settings, a random forest is fitted on

D

using a task-specific scalar measure of prediction difficulty, and

\Phi(x)

is the concatenation of one-hot indicators of the leaves reached by

x

. Split sizes refer to the risk-training partition

\mathcal{D}

, calibration partition

\mathcal{C}

, and test partition

T

of ReCIRC . Base-model fitting, ReCIRC grids and representations, AA-CRC regularization, and evaluation-group definitions are given in the subsections below.

Figure 5: Held-out risk-calibration diagnostics for ReCIRC in the heteroscedastic experiment, overall and within five equal-mass bins of the known noise scale $\sigma(X)$ . Thick curves show mean empirical loss over 20 repetitions; faint curves show individual repetitions, with groups fixed within each repetition.

Figure 6: Held-out risk-calibration diagnostics for ReCIRC in the latent-difficulty experiment, overall and within five equal-mass bins of the known latent difficulty. Thick curves show mean empirical loss over 20 repetitions; faint curves show individual repetitions, with groups fixed within each repetition.

Figure 7: Held-out risk-calibration diagnostics for ReCIRC in RCV1. Results are shown over 20 repetitions, using fixed groups within each repetition; faint curves correspond to individual repetitions.

Figure 8: Held-out risk-calibration diagnostics for ReCIRC in Letter Recognition. Results are shown over 20 repetitions, using fixed groups within each repetition; faint curves correspond to individual repetitions.

Figure 9: Held-out risk-calibration diagnostics for ReCIRC in Medical Insurance. Results are shown over 20 repetitions, using fixed groups within each repetition; faint curves correspond to individual repetitions.

Figure 10: Held-out risk-calibration diagnostics for ReCIRC in Superconductor. Results are shown over 20 repetitions, using fixed groups within each repetition; faint curves correspond to individual repetitions.

Figure 11: Held-out risk-calibration diagnostics for ReCIRC in the XOR interaction experiment. Results are shown over 20 repetitions, using fixed groups within each repetition; faint curves correspond to individual repetitions.

Explore similar work

Conformal Risk Control for Non-Monotonic Losses
Feb 23, 2026Anastasios N. AngelopoulosConformal Risk Control
Arena
Conformal Risk Control under Non-Monotone Losses: Theory and Finite-Sample Guarantees
Apr 2, 2026Tareq Aldirawi, Yun Li, Wenge GuoConformal Risk Control Statistical Learning Theory
Department of Mathematical Sciences, New Jersey Institute of Technology, Newark, NJ, USA
Hierarchical Group-Conditional Conformal Risk Control for Selective Prediction in Language Models
Jul 27, 2026Murilo Salem, Luísa Böhm, Daniel Pontes +1Selective Prediction Group-Conditional Conformal Prediction
CDTec (Centro de Desenvolvimento Tecnológico) Universidade Federal de Pelotas Pelotas, RS, Brasil

Feb 23, 2026J/K move · Enter open · S save

Conformal Risk Control for Non-Monotonic Losses

Anastasios N. Angelopoulos

Arena

Conformal Risk Control

Open paper AskPDF arXiv

Conformal risk control is an extension of conformal prediction for controlling risk functions beyond miscoverage. The original algorithm controls the expected value of a loss that is monotonic in a one-dimensional parameter. Here, we present risk control guarantees for generic algorithms applied to possibly non-monotonic losses with multidimensional parameters. The guarantees depend on the stability of the algorithm -- unstable algorithms have looser guarantees. We give applications of this technique to selective image classification, FDR and IOU control of tumor segmentations, and multigroup debiasing of recidivism predictions across overlapping race and sex groups using empirical risk minimization.

Abstract

Explore similar work

Conformal Risk Control for Non-Monotonic Losses

Conformal Risk Control under Non-Monotone Losses: Theory and Finite-Sample Guarantees

Hierarchical Group-Conditional Conformal Risk Control for Selective Prediction in Language Models