Conditional Distribution Estimation

Follow this topic Ask about this topic

Momentum

12 papers in the last four weeks, against 1 the four weeks before. 0.1% of all new papers.

Jul 13Week of Sep 28

Latest papers 32

Oct 7, 2026stat.AP

Learning joint probabilistic weather forecasts from station observations alone

Assessing compound weather risks requires forecasts representing dependence between variables. CLARA (Calibrated Advection-Routing Attention) learns joint Gaussian predictive distributions of five surface variables from station observations alone, without numerical weather prediction or reanalysis; the approximately 28,000-parameter model supports CPU training and prediction. Across six multi-year folds on 96 stations, its lead-mean energy score is 4.9% lower than that of a learned comparator with matched temporal inputs (4.7% with a similar parameter count) and 11-65% lower than those of statistical baselines. Holding marginal variances fixed, removing learned correlations worsens joint negative log-likelihood by 1.0-2.8 nats per station. A covariance-scale estimator, proved consistent under stated assumptions, improves short-lead calibration but over-corrects at long leads. Synthetic interventions show an attention-bias coefficient alone does not measure forecast influence. Retrained in ten regions on six continents, CLARA outperforms persistence in all 60 multi-year region-lead comparisons and a similarly sized learned model in 57 of 60.

Chaeyeon Yi, Yun Am Seo

Research Center for Atmospheric Environment, Hankuk University of Foreign Studies, Yongin 17035, Republic of Korea · NAVI Hyper-Tropicalization Research Institute, Jeju National University, Jeju 63243, Republic of Korea · Department of Data Science, Jeju National University, Jeju 63243, Republic of Korea

Weather Forecasting Probabilistic Forecasting Conditional Distribution Estimation

Oct 7, 2026stat.ML

Sparsifying Stochasticity, Not Capacity: Partial Stochasticity via Deep Weight Factorization of Prior Scales

Bayesian neural networks need not be fully stochastic to be universal conditional density approximators, but it remains open which parameters should be stochastic. We learn this split by applying deep weight factorization to the prior scales, which are the standard deviations of the parameter priors, while fitting the functional prior to a Gaussian process with a maximum mean discrepancy objective. A parameter whose prior scale falls below a cutoff becomes deterministic and is optimized during inference, so the regularizer sparsifies stochasticity rather than capacity. We give a certificate for universal conditional density approximation that is checkable in linear time, together with a minimal repair when it fails. We further show that the common hybrid scheme of sampling some parameters and optimizing the others is stochastic approximation for a type-II maximum a posteriori objective, and that coupled step sizes can leave a tracking error that does not vanish as the step size shrinks. On a bimodal target, the learned split stays close to an unconstrained reference across all budgets and is insensitive to the cutoff, while random masks that distribute the same prior scales across layers are worse by up to two orders of magnitude. On UCI benchmarks, our method performs on par with a fully stochastic network while keeping about half of its parameters deterministic.

Marius P Linhard, Maurizio Filippone

Université Grenoble Alpes, France · KAUST, Saudi Arabia

Bayesian Neural Networks Conditional Distribution Estimation

Oct 7, 2026stat.ML

Scalable Logistic Gaussian Process Density Regression with Kinetic Langevin Sampling

Conditional density estimation targets the full distribution of a response given covariates, as required, for example, for per-galaxy photometric redshifts. We develop a scalable Bayesian estimator based on the logistic Gaussian process. The log conditional density has a separable covariance: a Matérn kernel along the response, represented in a truncated Fourier basis on a circle, and a covariate kernel represented by Nyström features, which accommodate non-stationary kernels with input-dependent amplitudes and length scales. Instead of a Laplace or variational approximation, we sample the latent field of this finite-feature model. Given the hyperparameters, its posterior is strongly log-concave with a uniformly bounded Hessian, and we draw from it by simulating kinetic Langevin dynamics with symmetric minibatch splitting in Kronecker-whitened coordinates. Marginal-likelihood gradients follow from Fisher's identity as posterior expectations. Under the conditions of our analysis their bias is controlled by the sampler's step size and run length, and the predictive averages over the non-Gaussian latent posterior instead of a Gaussian around its mode. On photometric-redshift benchmarks with up to 3.9 million training observations, trained on a single GPU, the estimator is competitive with state-of-the-art tabular foundation models on density and calibration metrics.

Daniel Paulin, Ádám Jung, András A. Benczúr

Nanyang Technological University · Eötvös Lor´and University (ELTE) HUN-REN Institute for Computer Science and Control (SZTAKI)

Conditional Distribution Estimation Density Estimation Gaussian Processes

Oct 6, 2026stat.ML

Learning Transition Kernels of Jump-Diffusion Processes with Conditional Diffusion Models

We study the problem of learning transition kernels for time-homogeneous jump-diffusion processes using conditional diffusion models, with the goal of generating new sample paths from training data consisting of N independent trajectories observed on a high-frequency discrete time grid. On the theoretical side, we establish non-asymptotic bounds for the conditional score estimation error and for the KL divergence between the laws of the true and generated discretely observed paths. On the numerical side, we first evaluate our method on synthetic data to assess the theoretical findings and benchmark its performance against the approach of Gao et al. (2025). We then apply our method to real-world data and investigate its performance on a probabilistic forecasting task.

Yuzhen Zhao, Yating Liu, Quentin Guibert

CEREMADE, CNRS, Universite Paris-Dauphine, PSL, 75016 Paris, France

Stochastic Differential Equations Conditional Distribution Estimation Conditional Diffusion Models Markov Models

Oct 4, 2026cs.LG

Delay-coordinate reconstruction and conditional-moment causal diagnostics in stochastic systems

Partial observation and delay-coordinate reconstruction are rooted in deterministic dynamical-systems theory, whereas there are many systems with intrinsic stochasticity. We propose a conditional-moment interpretation of delay-coordinate reconstruction for stochastic systems, in which a delay vector is used to reconstruct conditional moments of a future distribution rather than a unique future sample path. Two complementary arguments motivate this viewpoint. First, the probability density of a stochastic differential equation obeys a deterministic Fokker-Planck equation and, under certain assumptions, is represented by an infinite deterministic hierarchy of moments. Hence, a finite-moment closure suggests a Takens-like finite-dimensional approximation. Second, a discussion based on the Koopman operator theory clarifies that the time evolution of an observable in the Mori-Zwanzig formalism yields a conditional expectation in stochastic systems. Then, the orthogonal "noise" term in the coefficient-space Mori-Zwanzig equation vanishes in the stochastic cases; this result is consistent with the moment-based argument. As an application of this stochastic delay-reconstruction viewpoint, we revisit convergent cross mapping (CCM) for diagnosing certain causal relationships. Although CCM based on the embedding theorem cannot generally be applied to stochastic systems, it is possible to examine certain types of causal relationships by using conditional moments. Using coupled logistic systems with additive and multiplicative coupling mechanisms, we discuss how causal relationships are embedded in stochastic systems.

Jun Ohkubo

Graduate School of Science and Engineering, Saitama University, Sakura, Saitama, 338–8570 Japan

Dynamical Systems Stochastic Linear Dynamical Systems Time-Series Causal Discovery Conditional Distribution Estimation

Oct 1, 2026stat.ME

Generalized Engression Models

We consider estimating the conditional distribution of a multivariate outcome given covariates when its coordinates may be continuous, binary, categorical, ordinal or rankings, and are conditionally dependent on one another. Different statistical methods have been developed for each outcome type, and most of them target a summary of the conditional distribution, such as the mean of each coordinate, rather than the joint distribution of the outcome vector. We develop generalized engression models, a unified nonparametric distributional regression framework for outcomes of any type. The proposed method builds upon engression, a scoring-rule-based deep generative model, and introduces a data-type-specific link function and a stochastic perturbation that smooths the loss, enabling gradient-based training even with discontinuous links. We establish universal representation results for continuous, discrete and mixed outcomes. In simulations and in two applications, 242 species in a community ecology benchmark and a 17-dimensional mixed-type health outcome, the method matches type-specific models on marginal scores, improves on them on the joint distribution, and matches or exceeds purpose-built state-of-the-art joint species distribution models. Software is available in Python.

Xinwei Shen, Zijian Guo, Francis Bach

Department of Statistics, University of Washington · Center for Data Science, Zhejiang University · Inria, ´Ecole Normale Sup´erieure, PSL Research University

Conditional Distribution Estimation Conditional Generative Modeling

Sep 29, 2026stat.ML

Generative sequence modeling for infinite memory processes via predictive states

We consider estimating the one-step-ahead conditional distribution of a multivariate stochastic process. Many existing approaches rely on assumptions such as finite-range memory, sparsity, or additivity, which can be poorly suited to processes with long-range nonlinear interactions. However, without such structural assumptions, nonparametric estimation is challenging due to the curse of dimensionality. To address this challenge, we introduce a new estimation approach based on the predictive states of a process, possibly with infinite-range memory. We show that our estimator achieves fast convergence rates when the past history can be compressed into a low-dimensional statistic that is sufficient for predicting the future. Specifically, we show that the statistical complexity of the estimation problem is determined by the intrinsic dimension of the predictive state space. We establish guarantees for an instantiation of our method based on deep neural network estimators, and we support these theoretical results with experiments.

Michael Wieck-Sosa, Cosma Rohilla Shalizi

Department of Statistics & Data Science Carnegie Mellon University

Conditional Distribution Estimation Sequence Modeling

Sep 28, 2026stat.ML

Multi-Task Learning of Conditional Mean Operators: applications to dynamical systems and uncertainty quantification

Estimating conditional statistics and learning representations of a population of conditional distributions are central problems in many data-driven applications, including uncertainty quantification and dynamical systems analysis. Conditional mean operators (CMOs), a class of linear operators between function spaces, resolve these objectives by providing access to a broad class of conditional statistics. However, existing methods typically estimate each CMO independently or constrain it to prespecified function spaces, thereby preventing the exploitation of shared structure across related distributions. In this work, we posit that related CMOs share finite-dimensional input and output function spaces, and are specialized for each task with a linear operator mapping these spaces. Based on this hypothesis, we introduce MTL-CMO, a multi-task framework that jointly learns shared function spaces and task-specific operators across multiple datasets. We further introduce T-CMO, a transfer learning method that reuses the shared spaces to estimate, in closed form, the operator of a new conditional distribution. We establish statistical guarantees quantifying the benefits of jointly learning the shared function spaces. Our experiments demonstrate that learning shared function spaces improves uncertainty quantification across a broad range of conditional distributions and, when applied to Langevin and plasma dynamics, yields compact representations of complex dynamics that retain physically meaningful information and enable parameter identification.

Sami Chemlal, Thibaut Germain, Rémi Flamary +2

CMAP, Ecole Polytechnique · University of Novi Sad

Dynamical Systems Uncertainty Quantification Multi-Task Learning Conditional Distribution Estimation Transfer Learning

Sep 27, 2026stat.ML

Domain-Adapted Diffusion Models for Conditional Independence Testing

Conditional independence (CI) is a fundamental concept in statistics and machine learning. Recent advances in conditional generative modeling provide flexible tools for generative-model-based CI tests, which rely on an estimated conditional distribution to generate randomized samples. However, errors in estimating this distribution accumulate in existing Type I error bounds, and consistency of the generative estimator alone does not guarantee asymptotic Type I error control. To address this limitation, we formulate conditional generative modeling as a domain adaptation problem and leverage auxiliary data from multiple source domains to improve estimation in the target CI testing domain. We propose Domain-Adapted Diffusion (DA-Diff), a multi-source domain adaptation framework for conditional diffusion models based on weighted empirical risk minimization over both target and source domains. We establish the convergence rate of DA-Diff and show how transferable source data can improve target-domain estimation through an increased effective sample size while controlling transfer bias. Building on DA-Diff, we further propose Domain-Adapted Conditional Independence Testing (DA-CIT) and show that its Type I error satisfies

P(p \leq α) \leq α+ o(1)

. Experiments demonstrate that DA-Diff improved conditional generation quality compared with transfer-learning diffusion baselines, while DA-CIT provides strong Type I error control and competitive power.

Yanfeng Yang, Junda Zhao, Yijie Gao +8

The Graduate University for Advanced Studies, SOKENDAI, Tokyo, Japan · The Institute of Statistical Mathematics, Tokyo, Japan · East China Normal University, Shanghai, China +4

Multi-Source Domain Adaptation Domain Adaptation Conditional Distribution Estimation Conditional Independence Testing Conditional Diffusion Models

Sep 24, 2026stat.ME

Sufficiently Reduced Distributional Regression

We propose Sufficiently Reduced Distributional Regression (SRDR), a generative method that combines conditional distribution estimation with nonlinear sufficient dimension reduction (SDR). It builds on a characterization of sufficiency through strictly proper scoring rules: a dimension reduction is sufficient if and only if predicting the response from the reduced covariates incurs no loss in expected score relative to the full covariates. Sufficient dimension reduction thus becomes a risk minimization problem. SRDR jointly trains a dimension reduction map and a generative prediction model by minimizing the energy score, which can be estimated by sampling without density evaluation or adversarial training. The framework extends to multi-environment data and to classification. We prove that the estimated conditional distributions converge in energy distance to the true ones, which implies that the learned representation is asymptotically sufficient. In simulations and applications to CT slice localization, superconductivity, and digit classification, SRDR recovers low-dimensional sufficient structure and matches or outperforms state-of-the-art nonlinear SDR methods in representation quality and predictive performance.

Alexander Henzi, Tiange Liu, Xinwei Shen

Tsinghua University, Department of Statistics and Data Science · University of Washington

Conditional Distribution Estimation Inverse Dimensionality Reduction Dimensionality Reduction Conditional Generative Modeling

Sep 22, 2026stat.ML

Conditional Tensor Diffusion: Distributional Counterfactual Learning and Inference

Causal inference guides operational and managerial decisions but remains challenging in high-dimensional panel or tensor settings, where decisions may depend on the joint conditional distribution of missing control outcomes. We develop \emph{Counterfactual Tucker Diffusion} (\CFTDiff), which integrates the treatment mask and latent Tucker structure into conditional diffusion to recover this distribution given observed control outcomes through efficient nonlinear score learning in a low-dimensional core. The masked Tucker score preserves dependence across tensor modes while reducing the dimension of nonlinear score learning from the product of mode dimensions to the much smaller product of Tucker ranks. We establish high-probability error bounds for conditional score estimation that depend on the Tucker ranks, largest mode dimension, and the factor-strength-adjusted number of missing outcomes, and show how these bounds translate into recovery guaranties for the conditional distribution of the missing control outcomes. Across missing rates, simulations show more accurate point recovery than common causal panel and matrix/tensor completion methods; comparisons with nested diffusion specifications further demonstrate the gains from masked conditioning and Tucker dimension reduction. In Norway's iFlex experiment, \CFTDiff recovers missing outcomes more accurately than competing methods; when applied to causal analysis, its estimated conditional distributions yield counterfactual prediction intervals and target-attainment probabilities, allowing pricing interventions to be evaluated by demand-reduction magnitude and reliability.

Xinbing Kong, Zeyu Li, Junfan Mao +1

Southeast University · University of Science and Technology of China

Causal Counterfactual Generation Low-Rank Matrix Decomposition Learning with Missing Data Conditional Distribution Estimation Conditional Diffusion Models

Sep 22, 2026cs.LG

Marginal Log-Likelihood Increments under Dirichlet-Smoothed Markov Estimation

For a Dirichlet-smoothed transition model, the effect of adding one workflow trace to the training archive is an exact change in reference-weighted log likelihood. We derive that change and show that it is a weighted reduction of Kullback--Leibler divergence between the reference conditionals and the model. From this form we obtain an upper bound on the gain available to any acquisition, which expresses a millinat difference as a share of what is attainable, an exact covariance identity for the effect of the reference weighting, and a sign criterion for the interaction between two candidates, from which the batch objective is neither submodular nor supermodular. A case study on the BPI Challenge 2012 loan-application log measures all three and finds a positive selection result in one of the four combinations of reference weighting and budget unit. There, of two regressors fitted to identical descriptors and identical labels, the one that predicts individual increments far more accurately, median

R^2

0.87 against 0.62, realizes the smaller share of the attainable gain, 61 against 69 per cent, so ranking accuracy for individual traces is neither necessary nor sufficient for batch quality.

Levin David Schwab

Markov Models Process Mining Training Data Selection Conditional Distribution Estimation

Sep 20, 2026cs.LG

Multivariate quantile regression via Kolmogorov-Arnold Networks

This paper introduces a novel algorithm for predicting conditional joint distributions of vector-valued targets in stochastic systems whose randomness is intrinsic rather than arising from observation errors or additive noise. Multivariate quantile regression also involves modeling conditional joint distributions but represents a less challenging task. It predicts the probability that vector-valued targets fall within predefined regions, identifies regions corresponding to predefined probability levels, or performs both tasks simultaneously. The proposed identification technique employs ensembles of Kolmogorov--Arnold networks (KANs) as flexible function approximators. Although the suggested technique is not theoretically restricted to KANs, KANs are particularly well suited to the proposed construction and are therefore used throughout this study. In addition to the training procedure, this work introduces a new discrepancy measure for joint distributions and a goodness-of-fit (GoF) test based on it. This GoF test was initially developed to validate and calibrate the proposed identification technique and is used here in an ad hoc manner. Although the test could be tabulated for broader use, such a tabulation is not pursued in this work. The test is also applicable more generally.

Andrew Polar, Michael Poluektov

Independent software consultant, Duluth, GA, USA · School of Computing and Mathematical Sciences, University of Greenwich, Park Row, London SE10 9LS, UK

Quantile Regression Conditional Distribution Estimation

Sep 20, 2026cs.LG

Belted Engression: Sufficient Dimension Reduction for Generative Distributional Regression

Modern conditional generative models face significant challenges when learning complex covariate dependencies. While sufficient dimension reduction (SDR) provides a principled approach to compress these dependencies, traditional SDR frameworks were not formulated for conditional generation. To bridge this gap, we propose Belted Engression, a unified and architecturally parameter-efficient framework for generative distributional regression. Our approach establishes an end-to-end compress-then-generate paradigm driven by sufficient representation learning, embedding a structural bottleneck into the generative architecture. Theoretically, we prove that the standard SDR condition is equivalent to a law-preserving generative factorization, which is achieved at the global optimum of the population Belted Engression objective. Furthermore, by uncovering a localized Bernstein-type control for the energy-score loss, we establish finite-sample convergence rates that are sharper than those of existing results. We also prove that this belted architecture is strictly smaller, operating with an asymptotically vanishing parameter count relative to the unstructured baseline. Extensive simulations and real-world applications demonstrate that Belted Engression achieves superior distributional prediction and SDR recovery with fewer trainable parameters.

Wenxi Tan, Bing Li, Lingzhou Xue

Department of Statistics, The Pennsylvania State University

Conditional Distribution Estimation Inverse Dimensionality Reduction Dimensionality Reduction Conditional Generative Modeling

Sep 20, 2026stat.ML

Bayesian Filtering in Physical Systems via Test-time Trained Flow Matching

Bayesian filtering provides a principled framework for online state estimation under uncertainty, yet its application to systems with high-dimensional states and complicated posterior distributions remains challenging. Recent generative models, such as flow matching, have shown potential in Bayesian filtering. However, they still rely on particle-based representations of the posterior, which lose the rich information of the full distribution, or tackle a trajectory-level inverse problem that conflicts with the recursive structure of Bayesian filtering. To address this, we propose a new perspective of directly encoding the evolving distribution into flow matching model weights, namely, the Belief Flow Filter (BFF). It is a generative filtering framework that updates model weights via gradient descent at test time to track the posterior evolution. Thereby, BFF bypasses the scalability issue of particle representations or the flexibility limitation of Gaussian assumptions in conventional filters. We theoretically justify that the BFF design is structurally aligned with Bayesian filtering, and its training objective targets the recursive filtering operator. BFF is empirically verified across 5 different physical systems, including ones with chaotic dynamics and highly sparse, non-linear observations. The results show that BFF attains the best score in 8 of 9 metric-benchmark cells across the three standard 1D and 2D PDE benchmarks, and similarly leads on the extreme single-moving-sensor setting and on a real-world-grounded tokamak plasma estimation task, demonstrating its potential to accurately approximate the Bayesian filtering operator in high-dimensional probability space.

Ruiqi Feng, Chongyi Wang, Tao Zhang +1

Westlake University & Zhejiang University · Westlake University

Flow Matching Bayesian Filtering Conditional Distribution Estimation

Sep 14, 2026cs.LG

Cross-Block Conditioning in Deep Boltzmann Machines for Statistical Data Fusion

Statistical data fusion combines two panels that share a block of covariates but observe disjoint outcome blocks, and in its traditional form no row observes both outcomes at once. That rules out the discriminative criterion one would rather train a Deep Boltzmann Machine with, since multi-prediction training needs ground truth for whatever it holds out. We propose observed-block multi-prediction, which restricts the multi-prediction objective to targets drawn from what each row actually observes. It is well defined for any missingness pattern and reduces to the original criterion when rows are complete. Having a discriminative criterion that survives the setting lets us ask whether the joint model is needed at all, by separating what it contributes into a representation part and an inference part. On two datasets of different kinds, a consumer purchase panel and public-domain census microdata, over grids in sample size and covariate width spanning 40 cells and 200 runs per method, almost none of the fine-tuned DBM's advantage comes from generative pre-training, which is confined to the smallest sample size on one dataset and absent on the other. It comes from conditioning on one outcome block when predicting the other. This term amounts to +0.19 and +0.36 percentage points, is positive in all 40 cells, never decays as the panels grow (it is flat on one dataset and grows on the other), and requires neither a second hidden layer nor more inference. Against baselines tuned on validation and given the same conditioning, the fine-tuned DBM is the best method in 37 of the 40 cells. The imputers that can also condition on the other outcome block mostly lose accuracy when they do, whereas the DBM gains in every cell; since fusion data cannot validate that choice, this is the property that matters.

Junichiro Niimi

Meijo University

Learning with Missing Data Conditional Distribution Estimation

Aug 12, 2026cs.LG

A Local Sinkhorn Framework for Conditional Distribution Reconstruction of Multidimensional Random Fields

In this paper, we propose a local Sinkhorn divergence framework for conditional distribution reconstruction of multidimensional random fields. By utilizing the debiased Sinkhorn divergence, our proposed approach develops a differentiable and computationally efficient local distribution matching objective to train stochastic neural networks (SNNs). Furthermore, we establish theoretical generalization error estimates for our local Sinkhorn divergence framework, which explicitly characterizes the trade-off between approximation bias and statistical efficiency controlled by the regularization parameter and reveals how our proposed local Sinkhorn divergence loss function can be efficiently applied to learning multidimensional random field models. The proposed framework provides a scalable alternative to exact local optimal transport for conditional distribution reconstruction, offering a practical compromise between geometric fidelity, statistical efficiency, and computational scalability for uncertainty quantification and probabilistic scientific machine learning. Through various numerical examples, we compare our proposed local Sinkhorn divergence framework with other loss functions to train SNNs and with other machine-learning-based uncertainty quantification frameworks, demonstrating that the proposed local Sinkhorn divergence framework achieves an effective balance between reconstruction accuracy and computational efficiency while maintaining good scalability for multidimensional stochastic systems.

Mingtao Xia, Qijing Shen

School of Mathematics, University of Birmingham, Watson Building, Edgbaston, Birmingham, B15 2TT, United Kingdom · Department of Mathematics, University of Houston, 3551 Cullen Blvd, Houston, 77204, Texas, United States · Nuffield Department of Medicine, University of Oxford, United Kingdom, Old Road Campus, Oxford, OX3 7BN, United Kingdom

Uncertainty Quantification Conditional Distribution Estimation Wasserstein Distance

Jul 30, 2026stat.ML

Error Analysis of Neural-Network-Based Engression

Engression (Shen and Meinshausen, 2024) learns a conditional distribution by fitting a generative model

Y = f(X,\varepsilon)

under the energy score, a strictly proper scoring rule. We provide a theoretical error analysis of engression implemented with deep neural networks. We decompose the excess risk into three components: the approximation error, the stochastic error, and the Monte Carlo error. Based on this decomposition, we establish convergence rates under the assumption that the target conditional generator admits a compositional smoothness structure.

Juntong Chen, Zijian Guo, Xinwei Shen

School of Mathematical Sciences, Xiamen University · Center for Data Science, Zhejiang University · Department of Statistics, University of Washington

Conditional Distribution Estimation Neural Network Approximation Theory Statistical Learning Theory

Jul 15, 2026stat.ML

Parallel gradient boosting for flexible estimation of conditional distributions

Boosting is one of the most successful learning techniques for standard classification and regression tasks. Its extension to multi-output prediction problems has found an increasing number of applications in recent years. Among them is the prediction of entire conditional distributions rather than single functionals, which can often be framed as a multi-output regression problem, for example multiple quantile regression. Addressing such problems with classical implementations of boosting is computationally challenging, because usually one base model is trained for each target at every iteration. More efficient variants of boosting have been proposed to speed up training, but they tend to be tied to specific loss functions and classes of base learners, usually decision trees. In this work, we study a modification of the gradient boosting algorithm, which we call parallel gradient boosting, designed to circumvent all these limitations. The core idea is to use a common descent direction for all training observations. By doing so, only one base model is needed at each iteration, regardless of the number of targets, which allows for considerable performance gains. We establish sufficient conditions for the convergence of the algorithm, whose practical use is introduced via the multiple quantile regression setting. We show that in such a setting, it provides predictions of similar quality to state-of-the-art boosting libraries such as XGBoost, while being faster by several orders of magnitude. Then, we evaluate the properties of the resulting conditional distribution estimator, which is shown empirically to outperform other nonparametric and semiparametric estimators, especially in high-dimensional settings and in the presence of mixed and/or missing covariates.

Rémy Chapelle, Nicolas Vayatis, Bruno Falissard +1

CESP, CB, EVDG · Université Paris-Saclay, UVSQ, Inserm, CESP, 94807, Villejuif, France · CB +2

Ensemble Learning Conditional Distribution Estimation

Jun 29, 2026cs.LG

Physics-informed Conditional Normalizing Flows for Angles-only Cislunar Orbit Determination

Generative Astrodynamics is advanced in this work by extending generative modelling to an orbit determination problem in the cislunar environment. The task is formulated as conditional density estimation, aiming to infer the probability distribution of the initial state from angles-only measurements over short observation arcs. A normalising flow is trained on perturbed topocentric observations from Near Rectilinear Halo Orbits, enabling a flexible and potentially multimodal posterior representation. Given new measurements, the learned density is sampled to generate statistically consistent and physics-informed state hypotheses. These estimates are refined via nonlinear least-squares minimisation, providing a competitive warm start for classical algorithms.

Walther Litteri, Massimiliano Vasile

Aerospace Centre of Excellence, University of Strathclyde, G11XQ Glasgow, UK · Aerospace Centre of Excellence, University of Strathclyde, G1 1XQ Glasgow, UK

Normalizing Flows Conditional Distribution Estimation Conditional Generative Modeling

Jun 29, 2026stat.ML

Factorizable Normalizing Flows for parameter-dependent density morphing

Normalizing Flows excel at modeling a single fixed density, yet many problems across the sciences, such as high energy physics, instead require modeling how that density deforms as a function of continuous parameters: the strength of a physical effect, a calibration constant, or a source of systematic uncertainty. Learning a separate flow for every parameter configuration quickly becomes intractable, since the number of joint settings grows exponentially with the number of parameters. We introduce Factorizable Normalizing Flows (FNFs), which represent the parameter-dependent density as a fixed, high-fidelity flow for a reference configuration composed with a learnable transformation that is polynomial in the parameters and factorized over them. This structure has a practical consequence: each parameter's effect is learned in isolation, from samples in which that parameter alone is varied. The combined response of many parameters is then recovered by summation at inference, without ever sampling their combinatorially large joint space. On a controlled problem with two interpretable deformations applied jointly to the data, the learned transformation reproduces the true deformations and matches the optimal likelihood, while optional interaction terms capture residual correlations when several parameters vary strongly at once. The resulting model is interpretable, scales linearly with the number of parameters, and keeps the likelihood tractable. This provides a general tool for any inference workflow requiring continuous density morphing, and directly enables the next generation of unbinned likelihood fits in high energy physics.

Davide Valsecchi, Mauro Donegà, Rainer Wallny

Institute for Particle Physics and Astrophysics, D-PHYS, ETH Zurich, Zurich, Switzerland

Normalizing Flows Conditional Distribution Estimation

Jun 8, 2026stat.ML

Convergence Rates for Neural-Network Estimation with Current-Status Data

Current-status data arise when an event time is observed only through an indicator of whether it occurred before an examination time. This paper studies a nonparametric neural-network sieve maximum likelihood estimator of the conditional cumulative distribution function of the event time. Under Hölder smoothness assumptions, we establish an explicit convergence rate by combining approximation theory for rectified linear unit neural networks with empirical-process arguments. This result provides theoretical support for neural-network estimation and subsequent inference under current-status observation.

Yuan Wu, Tianhui Zhou

Department of Biostatistics and Bioinformatics, Duke University, Durham, North Carolina, USA

Survival Analysis Conditional Distribution Estimation

May 31, 2026stat.ME

Theoretical Analysis of Engression and Reverse Markov Engression

Engression is a recently proposed and effective framework for conditional distribution learning. Its multi-step Reverse Markov extension further improves generative flexibility by decomposing complex conditional sampling into sequential reverse transitions. Despite their strong empirical performance, rigorous finite-sample statistical guarantees for these methods remain unavailable. In this paper, under deep neural network parameterizations, we establish nonasymptotic convergence bounds for Engression by directly controlling the Energy Distance between the learned and target conditional distributions. For the Reverse Markov framework, we further develop an Energy-Distance-based chain rule that enables a rigorous analysis of error propagation across reverse steps. Our analysis yields corresponding excess-risk bounds that are near-optimal up to logarithmic factors relative to the classical minimax rate over a general Hölder class.

Jiaqi Huang, Gongjun Xu, Ji Zhu

Department of Statistics, University of Michigan · Ann Arbor, MI 48109, U.S.A.

Conditional Distribution Estimation Statistical Learning Theory Conditional Generative Modeling

May 22, 2026cs.CL

When Is Next-Token Prediction Useful? Marginalization, Ergodicity, Mixture Identifiability, Local Sufficiency, RAG, Tools, and Programming

Language models trained on observed sequences are often described as learning the conditional distribution of the next token given previous tokens. This description is only conditionally correct. A model trained on realized token trajectories does not observe full conditional laws; it receives sampled continuations. Moreover, real language generation is conditioned not only on previous words but also on non-textual circumstances: facts, events, intentions, goals, beliefs, social context, and task-specific constraints. This paper distinguishes three objects that are often conflated: the full conditional language process conditioned on latent circumstances, the marginal text-only process obtained by integrating those circumstances out, and the model-induced distribution learned from finite observed corpora. The paper argues that interpreting model training as estimating the marginal text-only law requires strong assumptions of stationarity, representativeness, and ergodicity, assumptions that are standard in statistical estimation but problematic when applied to heterogeneous language corpora. Even if these assumptions hold, the marginal text-only law is useful only when the observed prefix is an approximately sufficient statistic for the latent circumstances relevant to continuation. In information-theoretic terms, usefulness requires that the residual conditional mutual information between the next token and the omitted circumstances, given the observed text, be small. The paper then extends this argument to heterogeneous training corpora. Finally, the paper interprets Retrieval Augmented Generation (RAG) and tool use as conditional sufficiency devices.

Francesco Corielli

Autoregressive Language Modeling Next-Token Prediction Conditional Distribution Estimation Sequence Modeling

May 20, 2026cs.CL

PromptNCE: Conditional Probabilities and PMI Using Only LLMs and Contrastive Estimation Prompts

Estimating mutual information from text usually requires training a task-specific critic, which limits its use in low-data settings. We ask whether large language models can instead estimate pointwise mutual information zero-shot, using only prompts and elicited probabilities. We construct a benchmark from three publicly available human-annotated datasets with ground-truth PMI, and evaluate five information-theoretic prompting-based estimators. Our main method, PromptNCE, frames conditional probability estimation as a contrastive task and augments the candidate set with an explicit OTHER category. The OTHER category allows the model to assign probability mass outside the candidate set, avoiding the closed-set normalization of standard contrastive prompts. PromptNCE gives the best conditional probability estimates on all three datasets. For full PMI, we find that estimating label base rates is the primary bottleneck on two of the three datasets, with the best methods reaching Spearman correlation up to 0.78. We also present a case study in computer science education showing how these estimators can be used to score student knowledge summaries in a low-data setting. We release our code and prompts.

Juliette Woodrow, Chris Piech

Department of Computer Science Stanford University

LLM Prompting Conditional Distribution Estimation Mutual Information Estimation

May 20, 2026stat.ML

Conditioning Gaussian Processes on Almost Anything

Gaussian processes (GPs) offer a principled probabilistic model over functions, but exact inference is restricted to the linear-Gaussian regime. We establish an explicit equivalence between GPs and a class of linear diffusion models, recasting predictive sampling as an ODE with closed-form Gaussian dynamics and a likelihood-dependent guidance term that admits a simple Monte Carlo approximation. In the linear-Gaussian setting, we recover standard GP conditioning exactly; beyond conjugacy, the same machinery handles any conditioning statement admitting point-wise likelihood evaluation -- including non-linear physics, and, for the first time, natural language via large language models. Whitening isolates the irreducible non-Gaussian dynamics, minimising Wasserstein-2 transport cost and eliminating numerical stiffness. The result is a general-purpose GP inference scheme requiring no bespoke derivations. Together, these results provide a general mechanism for incorporating the full richness of real-world knowledge as conditioning information, opening a new frontier for the probabilistic modelling of real-world problems.

Henry Moss, Lachlan Astfalck, Thomas Cowperthwaite +5

Lancaster University · University of New South Wales · University of Cambridge +1

Gaussian Processes Conditional Distribution Estimation

May 11, 2026cs.LG

Free Energy Manifold: Score-Based Inference for Hybrid Bayesian Networks

We introduce the Free Energy Manifold (FEM), a score-trained conditional energy model specialized for inference in hybrid Bayesian networks with discrete and continuous variables. FEM represents each conditional factor as an energy landscape over learned discrete-parent embeddings and continuous observations, enabling posterior evaluation, generative sampling, and compositional inference across multiple continuous leaves by energy addition under conditional independence. A central finding is the mode-bridge artifact: standard conditional energy models can create low-energy ridges between separated modes of the same class, producing overconfident posteriors at off-data interior points. We analyze this failure and propose valley regularization, an off-data calibration term that restores near-uniform posteriors in such regions while preserving in-data fit. Across synthetic multimodal hybrid-BN benchmarks, FEM substantially reduces KL divergence relative to classical baselines and a vanilla conditional EBM, including large gains at mode-bridge midpoint queries and in multi-leaf evidence composition. We also evaluate high-cardinality discrete-parent settings and a UCI Breast Cancer sanity check, showing that FEM is most useful when multimodal or compositional Bayesian-network inference is required, while discriminative classifiers remain preferable for closed-world classification tasks.

Cheol Young Park, Shou Matsumoto

ATOS Co., Ltd., 273, Digital-ro, Guro-gu, Seoul, 08381, Republic of Korea · C5I Center, George Mason University, Fairfax, VA 22030, USA

Bayesian Networks Energy-Based Models Conditional Distribution Estimation Score-Based Generative Modeling Uncertainty Calibration

May 8, 2026stat.ML

CONTRA: Conformal Prediction Region via Normalizing Flow Transformation

Density estimation and reliable prediction regions for outputs are crucial in supervised and unsupervised learning. While conformal prediction effectively generates coverage-guaranteed regions, it struggles with multi-dimensional outputs due to reliance on one-dimensional nonconformity scores. To address this, we introduce CONTRA: CONformal prediction region via normalizing flow TRAnsformation. CONTRA utilizes the latent spaces of normalizing flows to define nonconformity scores based on distances from the center. This allows for the mapping of high-density regions in latent space to sharp prediction regions in the output space, surpassing traditional hyperrectangular or elliptical conformal regions. Further, for scenarios where other predictive models are favored over flow-based models, we extend CONTRA to enhance any such model with a reliable prediction region by training a simple normalizing flow on the residuals. We demonstrate that both CONTRA and its extension maintain guaranteed coverage probability and outperform existing methods in generating accurate prediction regions across various datasets. We conclude that CONTRA is an effective tool for (conditional) density estimation, addressing the under-explored challenge of delivering multi-dimensional prediction regions.

Zhenhan Fang, Aixin Tan, Jian Huang

Department of Statistics and Actuarial Science The University of Iowa Iowa City, IA 52242, USA · Department of Applied Mathematics The Hong Kong Polytechnic University Hongkong, China

Conformal Prediction Conditional Distribution Estimation

May 7, 2026stat.ML

One Operator for Many Densities: Amortized Approximation of Conditioning by Neural Operators

Probabilistic conditioning is concerned with the identification of a distribution of a random variable

X

given a random variable

Y

. It is a cornerstone of scientific and engineering applications where modeling uncertainty is key. This problem has traditionally been addressed in machine learning by directly learning the conditional distribution of a fixed joint distribution. This paper introduces a novel perspective: we propose to solve the conditioning problem by identifying a single operator that maps any joint density to its conditional, thus amortizing over joint-conditional pairs. We establish that the conditioning operator can be approximated to arbitrary accuracy by neural operators. Our proof relies on new results establishing continuity of the conditioning operator over suitable classes of densities. Finally, we learn the conditioning map for a class of Gaussian mixtures using neural operators, illustrating the promise of our framework. This work provides the theoretical underpinnings for general-purpose, amortized methods for probabilistic conditioning, such as foundation models for Bayesian inference.

Panos Tsimpos, Edoardo Calvello, Ayoub Belhadji +1

Operations Research Center MASSachusetts Institute of Technology Cambridge, MA 02139 · Department of Computing and Mathematical Sciences California Institute of Technology Pasadena, CA 91125 · Laboratory for Information and Decision Systems Center for Computational Science and Engineering MASSachusetts Institute of Technology Cambridge, MA 02139 +2

Amortized Inference Bayesian Inference Conditional Distribution Estimation Neural Operators

Apr 22, 2026cs.LG

Interpretable Quantile Regression by Optimal Decision Trees

The field of machine learning is subject to an increasing interest in models that are not only accurate but also interpretable and robust, thus allowing their end users to understand and trust AI systems. This paper presents a novel method for learning a set of optimal quantile regression trees. The advantages of this method are that (1) it provides predictions about the complete conditional distribution of a target variable without prior assumptions on this distribution; (2) it provides predictions that are interpretable; (3) it learns a set of optimal quantile regression trees without compromising algorithmic efficiency compared to learning a single tree.

Valentin Lemaire, Gaël Aglin, Siegfried Nijssen

Euranova, Rue Emile Francqui, Mont-Saint-Guibert, Belgium · Université Catholique de Louvain, Louvain-la-Neuve, Belgium

Quantile Regression Interpretable ML Conditional Distribution Estimation Decision Tree Learning