Conditional Distribution Estimation

Overview

The field estimates how an outcome varies with observed covariates, not just its average, so predictions can represent uncertainty, skew, and multiple plausible outcomes. Approaches include quantile regression and tree ensembles, mixture and Gaussian-process models, and neural methods such as normalizing flows, energy models, and conditional diffusion. Researchers also study how to estimate these distributions efficiently for high-dimensional, mixed-type, censored, or sequential data, and how to evaluate them with proper scores and calibration. A central tension is balancing flexible modeling of complex distributions with reliable uncertainty estimates, computational scalability, and finite-sample guarantees.

Latest papers

92 papers18 papers in the last four weeks, down 22% on the four weeks before. 0.1% of all new papers.

Oct 17, 2024

Discrete distributions are learnable from metastable samples

Physically motivated stochastic dynamics are widely used to sample from high-dimensional distributions. However, such samplers often get trapped in metastable states, approximately sampling from a distribution that differs significantly from the desired stationary state. We rigorously show that for multivariable discrete distributions, the true stationary model can nevertheless be recovered from these metastable samples. This relies on a fundamental observation: for distributions satisfying a strong metastability condition, their single-variable conditional probabilities are on average extremely close to those of the true stationary distribution. This remains true even when the two distributions are far apart under global metrics such as Kullback-Leibler divergence. Consequently, we can effectively learn the true model using a conditional-likelihood estimator even when the samples are drawn from a restricted state space. Extending these general results to Ising models, we prove rigorous parameter and structure learning guarantees. Finally, we demonstrate this phenomenon numerically on higher-alphabet spin glass models.
Feb 2, 2024

Deep conditional distribution learning via conditional Föllmer flow

We introduce an ordinary differential equation (ODE) based deep generative method for learning conditional distributions, named Conditional Föllmer Flow. Starting from a standard Gaussian distribution, the proposed flow could approximate the target conditional distribution very well when the time is close to 1. For effective implementation, we discretize the flow with Euler's method where we estimate the velocity field nonparametrically using a deep neural network. Furthermore, we also establish the convergence result for the Wasserstein-2 distance between the distribution of the learned samples and the target conditional distribution, providing the first comprehensive end-to-end error analysis for conditional distribution learning via ODE flow. Our numerical experiments showcase its effectiveness across a range of scenarios, from standard nonparametric conditional density estimation problems to more intricate challenges involving image data, illustrating its superiority over various existing conditional density estimation methods.