Neural Processes (NPs) are flexible meta-learning models capable of uncertainty quantification across diverse applications. However, the standard NP framework assumes access to noise-free inputs. This is unrealistic in many practical settings, where sensor noise or measurement errors are unavoidable. This discrepancy between model assumptions and real-world data can significantly degrade prediction quality and uncertainty estimates. We introduce Denoising Neural Processes (DnNPs), a principled extension of the NP framework that explicitly models input uncertainty. We train DnNPs via variational inference. DnNPs jointly learn to denoise corrupted inputs and perform meta-learning, making them robust to noisy training data. We demonstrate DnNPs' effectiveness on both synthetic and real-world datasets, showing consistent improvements in prediction and uncertainty quantification compared to standard NPs. On PM2.5 spatial interpolation from real-world air quality sensors, DnNP reduces RMSE by up to 30% and NLL by up to 73% relative to standard NPs. We also provide theoretical grounding: we show that the Bayes-optimal predictor under noisy inputs belongs to the DnNP model class, and that plug-in models with fixed observation variance incur an irreducible KL divergence gap that DnNP provably avoids.
Figures & tables
Figure 1: Comparison of Neural Processes (NP) and DnNP in terms of graphical model (top) and architecture (bottom). In the graphical model, shaded nodes represent observed variables. Standard NPs assume that inputs are observed noise-free, an assumption violated in many real-world settings where measurements are inherently corrupted by noise. DnNP addresses this by treating noise-free inputs as latent variables x∗ that must be inferred from noisy observations x , while preserving the standard NP structure of context and target sets with a global latent z that captures function-level uncertainty. In the architecture, DnNP extends the standard NP encoder-decoder pipeline by prepending a denoising component (shown in green) that recovers noise-free input estimates from noisy inputs before passing them to the encoder.
Figure 2: Comparison of Clean NP, Noisy NP, and DnNP on 1D GP regression across noise levels σx∈{0.1,0.2,0.3,0.4,0.5} . Shaded regions show the predictive mean ±σ and ±2σ uncertainty bands. Black circles indicate true context inputs and orange triangles show the corrupted observed input. As input noise increases, Noisy NP produces increasingly biased mean predictions, whereas DnNP maintains closer agreement with the true function. The NLL sweep (right) confirms this quantitatively, showing that DnNP achieves consistently lower NLL than Noisy NP across all noise levels.
Method
NLL ↓
RMSE ↓
Acc ↑
NP
-0.0631
0.1413
0.706
CNP
-0.00571
0.232
0.319
ANP
-0.00702
0.190
0.733
ConvCNP
-0.00454
0.213
0.576
Table 1: MNIST results. Left: clean-input baselines ( σx=0 ). Right: performance under increasing input noise σx for Noisy NP vs. DnNP; bold marks the better of the two at each noise level.
Figure 3: MNIST image completion with noisy pixel coordinates ( σx=0.1 ) at varying context sizes. Each row shows context, prediction, and per-pixel uncertainty for Clean NP ( σx=0 , reference), Noisy NP, and DnNP. Noisy NP fails to recover the correct digit at all context sizes, while DnNP recovers digits comparable to Clean NP, with uncertainty concentrated along edges.
Dataset
Feature Set
Mean
NP
DnNP (ours)
RMSE ↓
NLL ↓
RMSE ↓
NLL ↓
RMSE ↓
PM2.5 (Air Quality)
Lat, Lon, RH
0.776
1.098±0.273
0.457±0.036
0.296±0.078
0.320±0.023
Lat, Lon, Temp
0.776
1.038±0.187
0.439±0.030
0.363±0.089
0.337±0.055
Lat, Lon, RH, Temp
0.776
0.925±0.161
0.430±0.032
0.351±0.130
0.348±0.062
Pressure (NOAA CA 2026)
Lat, Lon, Temp
0.108
−1.240±0.032
0.052±0.008
−1.239±0.021
0.036±0.006
Lat, Lon, RH
0.108
−1.091±0.064
0.076±0.008
−1.229±0.036
0.039±0.010
Table 2: Spatial interpolation results on two real-world datasets, reported as mean ± standard deviation over 5 seeds. Mean RMSE is the constant-prediction baseline (predicting the dataset mean). Bold marks the better mean of NP vs. DnNP per metric within each row.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Dataset
K
d
Task
Noise
1D GP
10,000
1
Regression
No
MNIST
70,000
2
Image Compl.
No
Air Quality
2,856
4
Spatial Interp.
Yes
NOAA CA 2026
90
4
Spatial Interp.
Yes
Appendix
Table 3: Overview of datasets used in experiments. d denotes the input dimensionality, K the number of training tasks (episodes), Task specifies the type of problem (regression, image completion, or spatial interpolation), and Noise indicates whether inputs are naturally corrupted. MNIST and 1D GP datasets do not inherently contain input noise, so we add noise synthetically to simulate noisy measurements.
We propose a methodology based on the standard ReLU Deep Neural Networks (DNN) to make predictions and quantify their uncertainty. Classically, people rely on linear, non-linear, or non-parametric kernel methods to fit and then predict the time series. As the universal approximation ability was revealed for DNN, its application has become more and more popular for prediction tasks in various scientific areas. However, the corresponding uncertainty quantification has not been studied thoroughly. Particularly, the uncertainty in prediction will consist of two parts: (1) the future variability; (2) the estimation variability within training data. To capture both variabilities, we build the so-called pertinent prediction interval (PPI) with the DNN model estimator. We first explore the consistency property of the DNN estimator with beta-mixing dependent data. Subsequently, we show that the implied forward bootstrap series is still beta-mixing and possesses the same stationary distribution as the original time series in probability, which is a key condition to enable the PPI. Lastly, the desired PPI is built after imposing minimal conditions on the limiting distribution of predictive roots. Simulations and real-data analysis are deployed to challenge our approach with standard non-parametric methods.
Kejin Wu
Department of Mathematics and Statistics, Loyola University Chicago
Neural Processes (NPs) are model-based meta-learners that implicitly learn a stochastic process and adapt to a new task from a small context set. Most extensions of NPs focus on improving the neural network architecture. We instead develop an extension motivated by the shared hierarchical interpretation of meta-learning and mixed-effects models. Specifically, we introduce Neural Process Boosting (NPBoost), which decomposes structured response variability into tree-boosted fixed effects shared across tasks and NP random effects that capture stochastic task-to-task variation. We propose to train the two components jointly using a boosting algorithm in which an NP learns residual task-specific structure and a tree ensemble estimates common patterns across tasks. Across synthetic and real-world tabular meta-learning problems, this decomposition improves over a standard NP when the shared structure contains discontinuities or other irregular patterns that boosted trees can represent effectively.
Andrea Nava, Ken Rölli, Armin Begic +1
Seminar for Statistics, ETH Zurich, Switzerland · Lucerne University of Applied Sciences and Arts, Switzerland
Deep neural networks achieve impressive results across diverse applications, yet their overconfidence on unseen inputs necessitates reliable epistemic uncertainty modeling. Existing methods for uncertainty modeling face a fundamental dilemma: Bayesian approaches provide principled estimates but remain computationally prohibitive, while efficient second-order predictors lack rigorous connections between their specific objectives and epistemic uncertainty quantification. To resolve this dilemma, we introduce Dirichlet-approximated possibilistic posterior predictions (DAPPr), a principled framework grounded in possibility theory. We define a possibilistic posterior over parameters, project it to the prediction space via supremum operators, and approximate the projected posterior using learnable Dirichlet possibility functions. This projection-and-approximation strategy yields a simple training objective with closed-form solutions. Despite its simplicity, extensive experiments across diverse benchmarks show that DAPPr achieves competitive or superior uncertainty quantification performance over state-of-the-art second-order predictors while maintaining both principled derivation and computational efficiency. Code is available at https://github.com/MaxwellYaoNi/DAPPr.
Yao Ni, Jeremie Houssineau, Yew-Soon Ong +1
Nanyang Technological University · A*STAR · The University of New South Wales +1