QUTCC: Quantile Uncertainty Training and Conformal Calibration for Imaging Inverse Problems
Abstract
While deep learning offers tremendous promise for scientific and medical imaging, any failures and hallucinations (predictions that do not coincide with reality) are hard to pinpoint and can have serious downstream consequences. Uncertainty estimation techniques, such as conformal prediction, can help by predicting statistically valid error bars for a model's prediction. However, popular conformal prediction methods were not designed for high-dimensional image-valued problems and do not take into account spatial correlations within an image during conformal calibration, resulting in larger-than-necessary uncertainty intervals. We propose a practical simultaneous quantile regression method that enables non-linear, spatially-adaptive scaling during conformal calibration. Our method, QUTCC uses a U-Net architecture with a quantile embedding to learn a full conditional quantile distribution during training, and then leverages this non-linear, learned function for spatially-adaptive conformal calibration. At test time, our method can efficiently estimate uncertainty intervals with pixel-marginal coverage guarantees. In addition, QUTCC can also predict pixel-wise conditional probability density estimates without any built-in distributional assumptions. We evaluate our method on several denoising problems, accelerated magnetic resonance imaging, and quantitative phase microscopy. Our method consistently produces tighter uncertainty intervals than prior conformal methods at the same coverage level, can predict plausible conditional distributions for different tasks, and in some cases, high-uncertainty regions can help us locate hallucinations in a model's prediction.
Figures & tables
| Task | Interval | Posterior-Based | Quantile-Based | ||||
| MC-Drop. | Deep Ens. | Im2Im | Im2Im-Asym. | K-RCPS | QUTCC | ||
| Poisson | 0.00–0.20 | ||||||
| 0.20–0.40 | |||||||
| 0.40–0.60 | |||||||
| 0.60–0.80 | |||||||
| 0.80–1.00 | |||||||
| Im2Im | 0.1121 | 0.1371 | 0.1530 | 0.1690 |
| QUTCC | 0.1116 | 0.1348 | 0.1511 | 0.1643 |
| QUTCC gain (%) | 0.45 | 1.68 | 1.24 | 2.78 |
| KL Divergence | Wasserstein | |
| MC-Dropout | ||
| Deep Ensemble | ||
| QUTCC |
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
| Metric | Model | MRI | QPI | Gaussian | Poisson | Real Noise |
| MSE ( ) | MC-Drop. | 0.002 0.002 | 0.001 0.001 | 0.001 0.001 | 0.0002 0.0002 | 0.0002 0.0001 |
| Deep Ens. | 0.001 0.002 | 0.0002 0.0001 | 0.001 0.001 | 0.0003 0.0003 | 0.0001 0.0001 | |
| Im2Im-UQ | 0.003 0.002 | 0.0006 0.0005 | 0.0007 0.0006 | 0.0003 0.0003 | 0.0030 0.0004 | |
| Im2Im-Deep | 0.001 0.002 | 0.0004 0.0003 | 0.0006 0.0006 | 0.0003 0.0002 | 0.0004 0.0002 | |
| Im2Im-Median | 0.001 0.002 | 0.0003 0.0002 | 0.0006 0.0005 | 0.0003 0.0002 | 0.0006 0.0004 | |
| QUTCC | 0.001 0.002 | 0.0004 0.0003 | 0.0006 0.0005 | 0.0003 0.0003 | 0.0002 0.0001 |
| Task | Normal Calibration | Monotonic Post-Processing | ||
| Lower | Upper | Lower | Upper | |
| Gaussian | 0.0078125 | 0.9794922 | 0.0078125 | 0.9794922 |
| Poisson | 0.9999995 | 0.9999995 | ||
| Real-Noise | 0.2177734 | 0.9580078 | 0.2177734 | 0.9580078 |
| MRI | 0.0380859 | 0.9570313 | 0.0380859 | 0.9570313 |
| QPI | 0.0205078 | 0.9921875 | 0.0205078 | 0.9921875 |
| Model | Interval Length | Total Risk |
| Embedding in all blocks (ours) | 0.0592 | 0.0908 |
| Embedding in first block only | 0.0619 | 0.1243 |
| Embedding in encoder blocks only | 0.0645 | 0.1199 |
| Embedding in middle block only | 0.0695 | 0.0829 |
| Embedding in decoder blocks only | 0.0679 | 0.0895 |
| Embedding in final block only | 0.0648 | 0.0987 |
| Metric | Method | MRI | QPI | Gaussian | Poisson | Real Noise |
| Interval Length | MC-Dropout | |||||
| Deep Ensemble | 0.1055 0.059 | 0.041 0.012 | 0.055 0.046 | 0.044 0.070 | 0.025 0.026 | |
| Im2Im-Deep | 0.1096 0.056 | 0.065 0.007 | 0.063 0.049 | 0.047 0.038 | 0.038 0.045 | |
| Im2Im-Asym. | 0.1096 0.048 | 0.064 0.007 | 0.062 0.028 | 0.046 0.016 | 0.037 0.007 | |
| K-RCPS (k = 2) | 0.1084 0.047 | 0.067 0.006 | 0.060 0.026 | 0.047 0.014 | 0.038 0.006 | |
| QUTCC | 0.1083 0.057 | 0.063 0.008 | 0.059 0.048 | 0.040 0.029 | 0.036 0.035 |
| Task | MC-Drop. | Deep Ens. | Im2Im-UQ | Im2Im-Deep | QUTCC |
| Poisson | |||||
| Gaussian | |||||
| Real-Noise | |||||
| MRI | |||||
| QPI |
| Task | MC-Drop. | Deep Ens. | Im2Im-UQ | Im2Im-Deep | QUTCC |
| Poisson | |||||
| Gaussian | |||||
| Real-Noise | |||||
| MRI | |||||
| QPI |
Conformal Prediction
Conformal prediction wraps predictive models with calibrated sets or intervals that provide statistical coverage guarantees.
See all 397 papers