Conditional Distribution Estimation

Follow this topic Ask about this topic

The field estimates how an outcome varies with observed covariates, not just its average, so predictions can represent uncertainty, skew, and multiple plausible outcomes. Approaches include quantile regression and tree ensembles, mixture and Gaussian-process models, and neural methods such as normalizing flows, energy models, and conditional diffusion. Researchers also study how to estimate these distributions efficiently for high-dimensional, mixed-type, censored, or sequential data, and how to evaluate them with proper scores and calibration. A central tension is balancing flexible modeling of complex distributions with reliable uncertainty estimates, computational scalability, and finite-sample guarantees.

Latest papers

78 papers23 papers in the last four weeks, up 130% on the four weeks before. 0.1% of all new papers.

Apr 9, 2026

CERBERUS: A Three-Headed Decoder for Vertical Cloud Profiles

Atmospheric clouds exhibit complex three-dimensional structure and microphysical details that are poorly constrained by the predominantly two-dimensional satellite observations available at global scales. This mismatch complicates data-driven learning and evaluation of cloud processes in weather and climate models, contributing to ongoing uncertainty in atmospheric physics. We introduce CERBERUS, a probabilistic inference framework for generating vertical radar reflectivity profiles from geostationary satellite brightness temperatures, near-surface meteorological variables, and temporal context. CERBERUS employs a three-headed encoder-decoder architecture to predict a zero-inflated (ZI) vertically-resolved distribution of radar reflectivity. Trained and evaluated using ground-based Ka-band radar observations at the ARM Southern Great Plains site, CERBERUS recovers coherent structures across cloud regimes, generalizes to withheld test periods, and provides uncertainty estimates that reflect physical ambiguity, particularly in multilayer and dynamically complex clouds. These results demonstrate the value of distribution-based learning targets for bridging observational scales, introducing a path toward model-relevant synthetic observations of clouds.

Emily K. deJong, Nipun Gunawardena, Kevin Smalley +2

Lawrence Livermore National Laboratory, Livermore, CA 94550 USA

Remote Sensing Conditional Distribution Estimation

Apr 9, 2026

Accurate and Reliable Uncertainty Estimates for Deterministic Predictions Extensions to Under and Overpredictions

Computational models support high-stakes decisions across engineering and science, and practitioners increasingly seek probabilistic predictions to quantify uncertainty in such models. Existing approaches generate predictions either by sampling input parameter distributions or by augmenting deterministic outputs with uncertainty representations, including distribution-free and distributional methods. However, sampling-based methods are often computationally prohibitive for real-time applications, and many existing uncertainty representations either ignore input dependence or rely on restrictive Gaussian assumptions that fail to capture asymmetry and heavy-tailed behavior. Therefore, we extend the ACCurate and Reliable Uncertainty Estimate (ACCRUE) framework to learn input-dependent, non-Gaussian uncertainty distributions, specifically two-piece Gaussian and asymmetric Laplace forms, using a neural network trained with a loss function that balances predictive accuracy and reliability. Through synthetic and real-world experiments, we show that the proposed approach captures an input-dependent uncertainty structure and improves probabilistic forecasts relative to existing methods, while maintaining flexibility to model skewed and non-Gaussian errors.

Rileigh Bandy, Enrico Camporeale, Andong Hu +2

Optimization and Uncertainty Quantification Department, Sandia National Laboratories, Albuquerque, NM · School of Physical and Chemical Sciences, Queen Mary University of London, London, UK and Space Weather Technology, Research and Education Center, University of Colorado, Boulder, CO · Space Weather Technology, Research and Education Center, University of Colorado, Boulder, CO +2

Distributional Regression Uncertainty Quantification Probabilistic Forecasting Uncertainty Calibration Conditional Distribution Estimation

Apr 8, 2026

Learning Markov Processes as Sum-of-Square Forms for Analytical Belief Propagation

Harnessing the predictive capability of Markov process models requires propagating probability density functions (beliefs) through the model. For many existing models however, belief propagation is analytically infeasible, requiring approximation or sampling to generate predictions. This paper proposes a functional modeling framework leveraging sparse Sum-of-Squares (SoS) forms for valid (conditional) density estimation. We study the theoretical restrictions of modeling conditional densities using the SoS form, and propose a novel functional form for addressing such limitations. The proposed architecture enables generalized simultaneous learning of basis functions and coefficients, while preserving analytical belief propagation. In addition, we propose a training method that allows for exact adherence to the normalization and non-negativity constraints. Our results show that the proposed method achieves accuracy comparable to state-of-the-art approaches while requiring significantly less memory in low-dimensional spaces, and it further scales to 12D systems when existing methods fail beyond 2D.

Peter Amorese, Morteza Lahijanian

University of Colorado Boulder

Belief Propagation Markov Models Conditional Distribution Estimation Density Estimation

Apr 1, 2026

Reconsidering Dependency Networks from an Information Geometry Perspective

Dependency networks (Heckerman et al., 2000) provide a flexible framework for modeling complex systems with many variables by combining independently learned local conditional distributions through pseudo-Gibbs sampling. Despite their computational advantages over Bayesian and Markov networks, the theoretical foundations of dependency networks remain incomplete, primarily because their model distributions -- defined as stationary distributions of pseudo-Gibbs sampling -- lack closed-form expressions. This paper develops an information-geometric analysis of pseudo-Gibbs sampling, interpreting each sampling step as an m-projection onto a full conditional manifold. Building on this interpretation, we introduce the full conditional divergence and derive an upper bound that characterizes the location of the stationary distribution in the space of probability distributions. We then reformulate both structure and parameter learning as optimization problems that decompose into independent subproblems for each node, and prove that the learned model distribution converges to the true underlying distribution as the number of training samples grows to infinity. Experiments confirm that the proposed upper bound is tight in practice.

Kazuya Takabatake, Shotaro Akaho

Independent Researcher, Izumisano, Osaka, Japan · The Institute of Statistical Mathematics, Tachikawa, Tokyo, 190-8565, Japan

Graph Structure Learning Markov Chain Monte Carlo Conditional Distribution Estimation Information Geometry

Mar 31, 2026

DiSGMM: A Method for Time-varying Microscopic Weight Completion on Road Networks

Microscopic road-network weights represent fine-grained, time-varying traffic conditions obtained from individual vehicles. An example is travel speeds associated with road segments as vehicles traverse them. These weights support tasks including traffic microsimulation and vehicle routing with reliability guarantees. We study the problem of time-varying microscopic weight completion. During a time slot, the available weights typically cover only some road segments. Weight completion recovers distributions for the weights of every road segment at the current time slot. This problem involves two challenges: (i) contending with two layers of sparsity, where weights are missing at both the network layer (many road segments lack weights) and the segment layer (a segment may have insufficient weights to enable accurate distribution estimation); and (ii) achieving a weight distribution representation that is closed-form and can capture complex conditions flexibly, including heavy tails and multiple clusters. To address these challenges, we propose DiSGMM that combines sparsity-aware embeddings with spatiotemporal modeling to leverage sparse known weights alongside learned segment properties and long-range correlations for distribution estimation. DiSGMM represents distributions of microscopic weights as learnable Gaussian mixture models, providing closed-form distributions capable of capturing complex conditions flexibly. Experiments on two real-world datasets show that DiSGMM can outperform state-of-the-art methods.

Yan Lin, Jilin Hu, Shengnan Guo +3

Aalborg University · East China Normal University · Beijing Jiaotong University

Learning with Missing Data Conditional Distribution Estimation Traffic Flow Estimation

Mar 27, 2026

Benchmarking Tabular Foundation Models for Conditional Density Estimation in Regression

Conditional density estimation (CDE) - recovering the full conditional distribution of a response given tabular covariates - is essential in settings with heteroscedasticity, multimodality, or asymmetric uncertainty. Recent tabular foundation models, such as TabPFN and TabICL, naturally produce predictive distributions, but their effectiveness as general-purpose CDE methods has not been systematically evaluated, unlike their performance for point prediction, which is well studied. We benchmark three tabular foundation model variants against a diverse set of parametric, tree-based, and neural CDE baselines on 39 real-world datasets, across training sizes from 50 to 20,000, using six metrics covering density accuracy, calibration, and computation time. Across all sample sizes, foundation models achieve the best CDE loss, log-likelihood, and CRPS on the large majority of datasets tested. Calibration is competitive at small sample sizes but, for some metrics and datasets, lags behind task-specific neural baselines at larger sample sizes, suggesting that post-hoc recalibration may be a valuable complement. In a photometric redshift case study using SDSS DR18, TabPFN exposed to 50,000 training galaxies outperforms all baselines trained on the full 500,000-galaxy dataset. Taken together, these results establish tabular foundation models as strong off-the-shelf conditional density estimators.

Rafael Izbicki, Pedro L. C. Rodrigues

Department of Statistics, Federal University of São Carlos (UFSCar), Brazil. · Univ. Grenoble Alpes, Inria, CNRS, Grenoble INP, LJK, France.

Conditional Distribution Estimation Tabular Foundation Models

Mar 26, 2026

A CDF-First Framework for Free-Form Density Estimation

Conditional density estimation (CDE) is a fundamental task in machine learning that aims to model the full conditional law

\mathbb{P}(\mathbf{y} \mid \mathbf{x})

, beyond mere point prediction (e.g., mean, mode). A core challenge is free-form density estimation, capturing distributions that exhibit multimodality, asymmetry, or topological complexity without restrictive assumptions. However, prevailing methods typically estimate the probability density function (PDF) directly, which is mathematically ill-posed: differentiating the empirical distribution amplifies random fluctuations inherent in finite datasets, necessitating strong inductive biases that limit expressivity and fail when violated. We propose a CDF-first framework that circumvents this issue by estimating the cumulative distribution function (CDF), a stable and well-posed target, and then recovering the PDF via differentiation of the learned smooth CDF. Parameterizing the CDF with a Smooth Min-Max (SMM) network, our framework guarantees valid PDFs by construction, enables tractable approximate likelihood training, and preserves complex distributional shapes. For multivariate outputs, we use an autoregressive decomposition with SMM factors. Experiments demonstrate our approach outperforms state-of-the-art density estimators on a range of univariate and multivariate tasks.

Chenglong Song, Mazharul Islam, Lin Wang +2

Shandong Key Laboratory of Ubiquitous Intelligent Computing, University of Jinan · Shandong Provincial Key Laboratory of Green and Intelligent Building Materials, University of Jinan, Jinan, Shandong 250022, China · Quancheng Shandong Laboratory, Jinan 250100, China +1

Conditional Distribution Estimation Density Estimation

Mar 23, 2026

CRPS-Optimal Binning for Univariate Conformal Regression

We propose a method for non-parametric conditional distribution estimation based on partitioning covariate-sorted observations into contiguous bins and using the within-bin empirical CDF as the predictive distribution. Bin boundaries are chosen to minimise the total leave-one-out Continuous Ranked Probability Score (LOO-CRPS), which admits a closed-form cost function with

O(n^2 \log n)

precomputation and

O(n^2)

storage; the globally optimal

K

-partition is recovered by a dynamic programme in

O(n^2 K)

time. Minimisation of within-sample LOO-CRPS turns out to be inappropriate for selecting

K

as it results in in-sample optimism. We instead select

K

by

K

-fold cross-validation of test CRPS, which yields a U-shaped criterion with a well-defined minimum. Having selected

K^*

and fitted the full-data partition, we form two complementary predictive objects: the Venn prediction band and a conformal prediction set based on CRPS as the nonconformity score, which carries a finite-sample marginal coverage guarantee at any prescribed level

\varepsilon

. The conformal prediction is transductive and data-efficient, as all observations are used for both partitioning and p-value calculation, with no need to reserve a hold-out set. On real benchmarks against split-conformal competitors (Gaussian split conformal, CQR, CQR-QRF, and conformalized isotonic distributional regression), the method produces substantially narrower prediction intervals while maintaining near-nominal coverage.

Paolo Toccaceli

Centre for Reliable Machine Learning, Royal Holloway, University of London, Egham, Surrey TW20 0EX, UK

Conditional Distribution Estimation Regression Prediction Intervals Conformal Prediction

Mar 22, 2026

Closed-form conditional diffusion models for data assimilation

We propose closed-form conditional diffusion models for data assimilation. Diffusion models use data to learn the score function (defined as the gradient of the log-probability density of a data distribution), allowing them to generate new samples from the data distribution by reversing a noise injection process. While it is common to train neural networks to approximate the score function, we leverage the analytical tractability of the score function to assimilate the states of a system with measurements. To enable the efficient evaluation of the score function, we use kernel density estimation to model the joint distribution of the states and their corresponding measurements. The proposed approach also inherits the capability of conditional diffusion models of operating in black-box settings, i.e., the proposed data assimilation approach can accommodate systems and measurement processes without their explicit knowledge. The ability to accommodate black-box systems combined with the superior capabilities of diffusion models in approximating complex, non-Gaussian probability distributions means that the proposed approach offers advantages over many widely used filtering methods. We evaluate the proposed method on nonlinear data assimilation problems based on the Lorenz-63 and Lorenz-96 systems of moderate dimensionality and nonlinear measurement models. Results show the proposed approach outperforms the widely used ensemble Kalman and particle filters when small to moderate ensemble sizes are used.

Brianna Binder, Agnimitra Dasgupta, Assad Oberai

Department of Aerospace & Mechanical Engineering, University of Southern California, 3650 McClintock Ave, Los Angeles, 90089, California, USA. · Optimization & Uncertainty Quantification, Sandia National Laboratories, 1450 Innovation Pkwy SE, Albuquerque, 87123, New Mexico, USA.

Data Assimilation Conditional Diffusion Models Conditional Distribution Estimation

Mar 20, 2026

Scalable Learning of Multivariate Distributions via Coresets

Efficient and scalable non-parametric or semi-parametric regression analysis and density estimation are of crucial importance to the fields of statistics and machine learning. However, available methods are limited in their ability to handle large-scale data. We address this issue by developing a novel coreset construction for multivariate conditional transformation models (MCTMs) to enhance their scalability and training efficiency. To the best of our knowledge, these are the first coresets for semi-parametric distributional models. Our approach yields substantial data reduction via importance sampling. It ensures with high probability that the log-likelihood remains within multiplicative error bounds of

(1\pm\varepsilon)

and thereby maintains statistical model accuracy. Compared to conventional full-parametric models, where coresets have been incorporated before, our semi-parametric approach exhibits enhanced adaptability, particularly in scenarios where complex distributions and non-linear relationships are present, but not fully understood. To address numerical problems associated with normalizing logarithmic terms, we follow a geometric approximation based on the convex hull of input data. This ensures feasible, stable, and accurate inference in scenarios involving large amounts of data. Numerical experiments demonstrate substantially improved computational efficiency when handling large and complex datasets, thus laying the foundation for a broad range of applications within the statistics and machine learning communities.

Zeyu Ding, Katja Ickstadt, Nadja Klein +2

Coreset Selection Conditional Distribution Estimation Density Estimation

Mar 17, 2026

Contextual Preference Distribution Learning

Decision-making problems often feature uncertainty stemming from heterogeneous and context-dependent human preferences. To address this, we propose a sequential learning-and-optimization pipeline to learn preference distributions and leverage them to solve downstream problems, for example risk-averse formulations. We focus on human choice settings that can be formulated as (integer) linear programs. In such settings, existing inverse optimization and choice modelling methods infer preferences from observed choices but typically produce point estimates or fail to capture contextual shifts, making them unsuitable for risk-averse decision-making. Using a bounded-variance score function gradient estimator, we train a predictive model mapping contextual features to a rich class of parameterizable distributions. This approach yields a maximum likelihood estimate. The model generates scenarios for unseen contexts in the subsequent optimization phase. In a synthetic ridesharing environment, our approach reduces average post-decision surprise by up to 114

\times

compared to a risk-neutral approach with perfect predictions and up to 25

\times

compared to leading risk-averse baselines.

Benjamin Hudson, Laurent Charlin, Emma Frejinger

Mila – Quebec Artificial Intelligence Institute, Montreal, Quebec, Canada · Département d’informatique et de recherche opérationnelle (DIRO), Université de Montréal, Montreal, Quebec, Canada · HEC Montréal, Montreal, Quebec, Canada

Preference Learning Decision-Making under Uncertainty Conditional Distribution Estimation Discrete Choice Modeling

Mar 11, 2026

GGMPs: Generalized Gaussian Mixture Processes

Conditional density estimation is complicated by multimodality, heteroscedasticity, and strong non-Gaussianity. Gaussian processes (GPs) provide a principled nonparametric framework with calibrated uncertainty, but standard GP regression is limited by its unimodal Gaussian predictive form. We introduce the Generalized Gaussian Mixture Process (GGMP), a GP-based method for multimodal conditional density estimation in settings where each input may be associated with a complex output distribution rather than a single scalar response. GGMP combines local Gaussian mixture fitting, cross-input component alignment and per-component heteroscedastic GP training to produce a closed-form Gaussian mixture predictive density. The method is tractable, compatible with standard GP solvers and scalable methods, and avoids the exponentially large latent-assignment structure of naive multimodal GP formulations. Empirically, GGMPs improve distributional approximation on synthetic and real-world datasets with pronounced non-Gaussianity and multimodality.

Vardaan Tekriwal, Mark D. Risser, Hengrui Luo +1

Applied Math & Computational Research Division, Lawrence Berkeley National Laboratory, Berkeley, CA · Department of Electrical Engineering & Computer Sciences, University of California, Berkeley, CA · Engineering Mathematics & Statistics Division, University of California, Berkeley, CA +3

Conditional Distribution Estimation Gaussian Processes Gaussian Mixture Models

Mar 10, 2026

Marginals Before Conditionals

We construct a minimal task that isolates conditional learning in neural networks: a surjective map with K-fold ambiguity, resolved by a selector token z, so H(A | B) = log K while H(A | B, z) = 0. The model learns the marginal P(A | B) first, producing a plateau at exactly log K, before acquiring the full conditional in a sharp, collective transition. The plateau has a clean decomposition: height = log K (set by ambiguity), duration = f(D) (set by dataset size D, not K). Gradient noise stabilizes the marginal solution: higher learning rates monotonically slow the transition (3.6* across a 7* η range at fixed throughput), and batch-size reduction delays escape, consistent with an entropic force opposing departure from the low-gradient marginal. Internally, a selector-routing head assembles during the plateau, leading the loss transition by ~50% of the waiting time. This is the Type 2 directional asymmetry of Papadopoulos et al. [2024], measured dynamically: we track the excess risk from log K to zero and characterize what stabilizes it, what triggers its collapse, and how long it takes.

Mihir Sahasrabudhe

School of Information Sciences, University of Illinois Urbana-Champaign

Neural Network Training Dynamics Phase Transitions in Neural Networks Conditional Distribution Estimation Statistical Physics of Learning

Mar 7, 2026

Conditional Rank-Rank Regression via Deep Conditional Transformation Models

Intergenerational mobility quantifies the transmission of socio-economic outcomes from parents to children. While rank-rank regression (RRR) is standard, adding covariates directly (RRRX) often yields parameters with unclear interpretation. Conditional rank-rank regression (CRRR) resolves this by using covariate-adjusted (conditional) ranks to measure within-group mobility. We improve and extend CRRR by estimating conditional ranks with a deep conditional transformation model (DCTM) and cross-fitting, enabling end-to-end conditional distribution learning with structural constraints and strong performance under nonlinearity, high-order interactions, and discrete ordered outcomes where the distributional regression used in traditional CRRR may be cumbersome or prone to misconfiguration. We further extend CRRR to discrete outcomes via an

ω

-indexed conditional-rank definition and study sensitivity to

ω

. For continuous outcomes, we establish an asymptotic theory for the proposed estimators and verify the validity of exchangeable bootstrap inference. Simulations across simple/complex continuous and discrete ordered designs show clear accuracy gains in challenging settings. Finally, we apply our method to two empirical studies, revealing substantial within-group persistence in U.S. income and pronounced gender differences in educational mobility in India.

Xiaoyi Wang, Long Feng, Zhaojun Wang

Nankai University

Conditional Distribution Estimation Semiparametric Inference

Feb 23, 2026

Discrete Diffusion with Sample-Efficient Estimators for Conditionals

We study a discrete denoising diffusion framework that integrates a sample-efficient estimator of single-site conditionals with round-robin noising and denoising dynamics for generative modeling over discrete state spaces. Rather than approximating a discrete analog of a score function, our formulation treats single-site conditional probabilities as the fundamental objects that parameterize the reverse diffusion process. We employ a sample-efficient method known as Neural Interaction Screening Estimator (NeurISE) to estimate these conditionals in the diffusion dynamics. Controlled experiments on synthetic Ising models, MNIST, and scientific data sets produced by a D-Wave quantum annealer, synthetic Potts model and one dimensional quantum systems demonstrate the proposed approach. On the binary data sets, these experiments demonstrate that the proposed approach outperforms popular existing methods including ratio-based approaches, achieving improved performance in total variation, cross-correlations, and kernel density estimation metrics.

Karthik Elamvazhuthi, Abhijith Jayakumar, Andrey Y. Lokhov

Diffusion Models Conditional Distribution Estimation

May 26, 2025

Representation Learning for Equivariant Inference with Guarantees

In many real-world applications of regression, conditional probability estimation, and uncertainty quantification, exploiting symmetries rooted in physics or geometry can dramatically improve generalization and sample efficiency. While geometric deep learning has made empirical advances by incorporating symmetry and geometry priors, less attention has been given to statistical learning guarantees. In this paper, we introduce an equivariant representation learning framework that simultaneously addresses regression, conditional probability estimation, and uncertainty quantification while providing first-of-its-kind non-asymptotic statistical learning guarantees. Grounded in operator and group representation theory, our framework approximates the spectral decomposition of the conditional expectation operator, building representations that are both equivariant and disentangled along independent symmetry quotient groups. Empirical evaluations on synthetic datasets and real-world robotics applications confirm the potential of our approach, matching or outperforming existing equivariant baselines in regression while providing well-calibrated uncertainty estimates.

Daniel Ordoñez-Apraez, Vladimir Kostić, Alek Fröhlich +3

CSML, Italian Institute of Technology, Genova, Italy · Universit`a di Genova, Genova, Italy · University of Novi Sad, Novi Sad, Serbia +2

Disentangled Representation Learning Representation Learning Uncertainty Quantification Conditional Distribution Estimation Equivariant Representation Learning Statistical Learning Theory

Oct 17, 2024

Discrete distributions are learnable from metastable samples

Physically motivated stochastic dynamics are widely used to sample from high-dimensional distributions. However, such samplers often get trapped in metastable states, approximately sampling from a distribution that differs significantly from the desired stationary state. We rigorously show that for multivariable discrete distributions, the true stationary model can nevertheless be recovered from these metastable samples. This relies on a fundamental observation: for distributions satisfying a strong metastability condition, their single-variable conditional probabilities are on average extremely close to those of the true stationary distribution. This remains true even when the two distributions are far apart under global metrics such as Kullback-Leibler divergence. Consequently, we can effectively learn the true model using a conditional-likelihood estimator even when the samples are drawn from a restricted state space. Extending these general results to Ising models, we prove rigorous parameter and structure learning guarantees. Finally, we demonstrate this phenomenon numerically on higher-alphabet spin glass models.

Abhijith Jayakumar, Andrey Y. Lokhov, Sidhant Misra +1

Theoretical Division, Los Alamos National Laboratory, Los Alamos, NM 87545, USA

Conditional Distribution Estimation Statistical Learning Theory

Feb 2, 2024

Deep conditional distribution learning via conditional Föllmer flow

We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional Föllmer Flow. Starting from a standard Gaussian distribution, the proposed flow could approximate the target conditional distribution very well when the time is close to 1. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network. Furthermore, we also establish the convergence result for the Wasserstein-2 distance between the distribution of the learned samples and the target conditional distribution, providing the first comprehensive end-to-end error analysis for conditional distribution learning via ODE flow. Our numerical experiments showcase its effectiveness across a range of scenarios, from standard nonparametric conditional density estimation problems to more intricate challenges involving image data, illustrating its superiority over various existing conditional density estimation methods.

Jinyuan Chang, Zhao Ding, Yuling Jiao +2

Joint Laboratory of Data Science and Business Intelligence, Southwestern University of Finance and Economics, Chengdu, Sichuan 611130, China · State Key Laboratory of Mathematical Sciences, Academy of Mathematics and Systems Science, Chinese Academy of Sciences, Beijing 100190, China · School of Mathematics and Statistics, Wuhan University, Wuhan, Hubei 430072, China +1

Conditional Distribution Estimation Flow-Based Generative Modeling Neural ODEs