Dirichlet Process Mixtures within Semiparametric Bayesian Networks: A Novel Conditional Density Estimation Approach

Victor Alejandre; Pedro Larrañaga; Concha Bielza

cs.LGOct 26, 2026

Dirichlet Process Mixtures within Semiparametric Bayesian Networks: A Novel Conditional Density Estimation Approach

Authors: Victor Alejandre, Pedro Larrañaga, Concha Bielza

Organizations: Computational Intelligence Group Departamento de Inteligencia Artificial Universidad Politécnica de Madrid Campus de Montegancedo, Boadilla del Monte 28660 Madrid, Spain

Abstract

Semiparametric Bayesian networks (SPBNs) are flexible probabilistic graphical models for representing joint distributions of continuous variables through lower-dimensional conditional probability distributions (CPDs). Existing SPBN models use kernel density estimators (KDEs) for non-parametric CPD modeling, but their one-component-per-observation mixture representation can lead to increased inference costs and density estimation errors, which can be limiting. In this work, we define a new CPD modeling approach for non-parametric components based on Bayesian non-parametric Dirichlet Process Mixtures (DPMs) within SPBNs. DPMs provide a data-adaptive clustering of observations, yielding a more compact representation while retaining the flexibility of non-parametric density estimation, unlike KDE. This results in lower inference costs while improving density estimation performance. We implement the proposed DPM-based SPBN (DPM-SPBN) by extending the open-source PyBNesian package and evaluate it on synthetic and real-world data. For a given network structure, DPM-based CPDs significantly outperform state-of-the-art Bayesian network models for continuous variables in density estimation, achieving lower absolute log-likelihood errors on synthetic data. We further investigate the impact of the proposed CPD modeling on structure learning methods relying on CPD-based scores. DPM-based SPBNs achieve lower structure learning and density estimation errors than KDE-based SPBNs. Experiments on real-world data corroborate these findings, identifying DPM-based CPD modeling as the most effective alternative among the evaluated non-parametric approaches for continuous variables.

Figures & tables

Figure 1: Comparison of density estimation using a DPM and a KDE with a $200$ -sample from a true underlying three-component Gaussian mixture.

Figure 2: Schematic overview of parametric and non-parametric approaches for modeling continuous CPDs in BNs. Each approach branches into different models and underlying assumptions, with the resulting BNs shown in rectangles. The novel proposal introduced in this work is highlighted in green.

Figure 3: Example of a SPBN for JPD estimation. The figure illustrates a sample dataset $\mathcal{D}$ , the network structure distinguishing non-parametric ( $X_{1},X_{3}$ ) and parametric ( $X_{2},X_{4}$ ) nodes, the estimated CPDs derived from $\mathcal{D}$ and encoded conditional independencies.

Figure 4: Bayesian networks defined for the experimental setup of the work with sample sizes of 5, 10 and 15 nodes.

Smooth	$\displaystyle\bullet\;f(x_{1})=\mathcal{N}(0,1),$ $\displaystyle\bullet\;f(x_{2})=0.5\mathcal{N}(-2,2^{2})+0.5\mathcal{N}(2,2^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=\mathcal{N}(x_{1}x_{2},1),$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=\mathcal{N}(10+0.8x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=\mathcal{N}\!\left(\frac{1}{1+e^{-x_{4}}},0.5^{2}\right)$
Medium	$\displaystyle\bullet\;f(x_{1})=\mathcal{N}(0,3^{2}),$ $\displaystyle\bullet\;f(x_{2})=0.25\mathcal{N}(-8,1^{2})+0.25\mathcal{N}(8,1^{2})+0.25\mathcal{N}(-4,0.5^{2})+0.25\mathcal{N}(4,0.5^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=w_{1}\mathcal{N}(-0.5x_{1},1)+w_{2}\mathcal{N}(0.5x_{2},1)+w_{3}\mathcal{N}(0,1),$ $\displaystyle w_{1}=\frac{x_{1}^{2}}{x_{1}^{2}+x_{2}^{2}+2\|x_{1}x_{2}\|},\quad w_{2}=\frac{x_{2}^{2}}{x_{1}^{2}+x_{2}^{2}+2\|x_{1}x_{2}\|},\quad w_{3}=\frac{2\|x_{1}x_{2}\|}{x_{1}^{2}+x_{2}^{2}+2\|x_{1}x_{2}\|},$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=\mathcal{N}(0.5x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=w\mathcal{N}(-2,0.5^{2})+(1-w)\mathcal{N}(2,0.5^{2}),\quad w=\frac{1}{1+\exp(-x_{4}/3)}$
Rough	$\displaystyle\bullet\;f(x_{1})=\mathcal{N}(0,3^{2}),$ $\displaystyle\bullet\;f(x_{2})=0.2\mathcal{N}(-8,1^{2})+0.2\mathcal{N}(8,1^{2})+0.2\mathcal{N}(-4,0.5^{2})+0.2\mathcal{N}(0,0.5^{2})+0.2\mathcal{N}(4,0.5^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=w_{1}\mathcal{N}(-0.5x_{1}^{2},1)+w_{2}\mathcal{N}(0.5x_{2}^{2},1)+w_{3}\mathcal{N}(0.5x_{1},1)+w_{4}\mathcal{N}(-0.5x_{2},1),$ $\displaystyle w_{1}=\frac{x_{1}^{2}}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},\quad w_{2}=\frac{x_{2}^{2}}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},$ $\displaystyle w_{3}=\frac{\|x_{1}\|}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},\quad w_{4}=\frac{\|x_{2}\|}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=\mathcal{N}(0.5x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=w_{1}\mathcal{N}(-2,0.5^{2})+w_{2}\mathcal{N}(0,0.5^{2})+w_{3}\mathcal{N}(2,0.5^{2}),$ $\displaystyle w_{1}=\frac{1}{(1+\exp(-x_{4}/3))2},\quad w_{2}=\frac{1}{(1+\exp(-x_{4}/3))2},\quad w_{3}=1-w_{1}-w_{2}$
Nonparametric	$\displaystyle\bullet\;f(x_{1})=0.25\mathcal{N}(-4,0.5^{2})+0.25\mathcal{N}(-2,0.5^{2})+0.25\mathcal{N}(2,0.5^{2})+0.25\mathcal{N}(4,0.5^{2}),$ $\displaystyle\bullet\;f(x_{2})=0.5\mathcal{N}(-2,2^{2})+0.5\mathcal{N}(2,2^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=\mathcal{N}(x_{1}x_{2},1),$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=0.5\mathcal{N}(10+0.8x_{3},0.5^{2})+0.5\mathcal{N}(10-0.8x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=\mathcal{N}\!\left(\frac{1}{1+e^{-x_{4}}},0.5^{2}\right)$

Table 1: Example of data generation mechanisms for the Smooth , Medium , Rough , and Nonparametric densities in the 5 node network.

Nodes	Density	Sample Size	SPBN-CAVI IQI		SPBN-GPCAVI IQI		SPBN-MEMOIZED IQI		Best Alternative IQI
5	Smooth	200	$\mathbf{3548}$ .	$\mathbf{44}~[3388.59,3740.85]$	$3647$ .	$55~[3416.09,3888.17]$	$4285$ .	$07~[3862.44,4566.23]$	$6085$ .	$68~[5910.16,6344.54]$ (NR)
		2000	$\mathbf{896}$ .	$\mathbf{37}~[847.70,939.15]$	$950$ .	$71~[902.42,1007.55]$	$1047$ .	$26~[984.11,1147.39]$	$2550$ .	$46~[2160.48,2941.39]$ (UCV)
		10000	$383$ .	$94~[363.52,409.47]$	$351$ .	$49~[321.34,372.01]$	$\mathbf{346}$ .	$\mathbf{07}~[325.27,374.63]$	$1066$ .	$21~[920.25,1249.90]$ (UCV)
		50000	$251$ .	$95~[234.60,280.55]$	$200$ .	$69~[185.11,208.88]$	$\mathbf{117}$ .	$\mathbf{36}~[101.12,124.94]$	$434$ .	$73~[383.04,521.34]$ (UCV)
		100000	$241$ .	$00~[223.69,258.95]$	$190$ .	$76~[171.64,199.12]$	$\mathbf{75}$ .	$\mathbf{14}~[65.70,87.74]$	$472$ .	$16~[421.11,568.19]$ (UCV)
	Medium	200	$6431$ .	$66~[6238.14,6623.57]$	$6558$ .	$40~[6260.33,6882.96]$	$7276$ .	$29~[7101.96,7376.25]$	$\mathbf{3622}$ .	$\mathbf{71}~[3375.82,4680.40]$ (UCV)

Table 2: Synthetic data experiments Absolute error summary for the LL across different node structures, densities and sample sizes in parameter learning experiments. Each case displays median and [IQI] for each method with the best median highlighted in bold. Best alternative method for each case is display between parenthesis.

Proposed Method	Total Wins	Sig. vs CAVI	Sig. vs GPCAVI	Sig. vs MEMOIZED	Sig. vs Best Alt.
SPBN-CAVI	19	-	14	17	19
SPBN-GPCAVI	0	0	-	0	0
SPBN-MEMOIZED	40	38	39	-	40

Table 3: Statistical significance of the proposed methods when achieving the lowest median absolute error. For each method, the number of cases in which it achieves the lowest median absolute error across all methods (total wins) and, among these, the number of wins with statistically significant pairwise differences are reported.

Figure 5: Pairwise comparison of state-of-the-art BNs for continous variables modeling and DPM-SPBN methods in parameter learning experiments. Each cell shows the number of cases in which the row method achieves a better median performance than the column method, with statistically significant differences reported in parentheses. Last column summarizes the overall and significant wins.

Figure 6: Parameter learning experiments distribution of relative computation times across network structures and sample sizes, with times averaged over density levels. The reference value corresponds to NR for training and SPBN-CAVI for prediction. Relative times are shown on a logarithmic scale.

Figure 7: Pairwise comparison of DPM-SPBN and standalone DPM methods in parameter learning experiments. Each cell shows the number of cases in which the row method achieves a better median performance than the column method, with statistically significant differences cases reported in parentheses. Last column summarizes the overall wins and significant wins out of those against all competitors.

Nodes	Density	Sample Size	SPBN-CAVI		SPBN-GPCAVI		NR		UCV
Nodes	Density	Sample Size	SHD IQI	LL Err.	SHD IQI	LL Err.	SHD IQI	LL Err.	SHD IQI	LL Err.
5	Smooth	200	$1.5~[1.0,5.8]$	$\times$	$\mathbf{1.0}~[1.0,4.8]$	✓	$4.5~[1.2,5.8]$	$\times$	$4.5~[4.0,5.8]$	$\times$
		2000	$5.0~[2.0,5.0]$	✓	$\mathbf{1.0}~[1.0,1.8]$	$\times$	$1.5~[1.0,5.0]$	$\times$	$2.5~[1.0,5.0]$	$\times$
		10000	$3.5~[1.0,6.0]$	$\times$	$\mathbf{1.0}~[1.0,4.2]$	✓	$4.0~[1.2,6.8]$	$\times$	$2.5~[1.0,5.0]$	$\times$
		50000	$\mathbf{4.0}~[3.2,6.0]$	$\times$	$\mathbf{4.0}~[3.2,6.0]$	✓	$5.0~[3.5,5.8]$	$\times$	-	-
		100000	$\mathbf{2.0}~[2.0,2.8]$	$\times$	$3.0~[3.0,6.0]$	✓	$5.5~[2.8,6.0]$	$\times$	-	-

Table 5: Structural Hamming distance (SHD) and log-likelihood error (LL Err.) across network sizes, densities, and sample sizes in the structure learning experiments. SHD is reported as median [IQR], while LL Err. indicates whether the method achieves (✓) the lowest absolute log-likelihood error or not (

\times

Dataset	$N$	# Variables
Glass	$214$	$9$
Ionosphere	$351$	$31$
QSAR Aquatic	$546$	$9$
Breast Cancer	$683$	$9$
QSAR Fish Toxicity	$908$	$7$
Yeast	$1484$	$8$

Table 7: Datasets from the Machine Learning UCI repository.

Figure 8: Performance comparison on real-world datasets between the proposed DPM-SPBN with inference methods CAVI and GPCAVI and state-of-the-art SPBNs using NR and UCV bandwidth selection. Distributions show test-set log-likelihoods obtained from $10$ -fold cross-validation.

Bayesian Networks

Bayesian networks represent joint probability distributions as graphs of variables and conditional dependencies, enabling inference under uncertainty.

See all 47 papers

Oct 22, 2026

Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines

Bayesian networks for continuous variables often rely on restrictive assumptions: linear Gaussian models lack flexibility to capture nonlinear dependencies, while non-parametric approaches provide greater flexibility at a high computational cost. This creates a gap for modeling scenarios where variables are approximately Gaussian but exhibit nonlinear relationships. We introduce Semi-Linear Gaussian Bayesian Networks (SLGBN), a novel class of continuous Bayesian networks designed to bridge this gap, preserving Gaussian conditional distributions while introducing nonlinear conditional mean functions. The proposed framework provides a unified probabilistic framework combining linear Gaussian and nonlinear Gaussian conditional models within the same Bayesian network Nonlinear conditional mean functions are estimated using generalized additive models (GAMs) with penalized B-splines. We present parameter estimation procedures for both types of nodes, as well as a greedy hill-climbing algorithm for learning the network structure under the SLGBN framework. To assess its performance, we compare SLGBN with other continuous Bayesian network models, including semiparametric Bayesian networks (SPBN) and Linear Gaussian Bayesian networks (LGBN). Experimental results evaluate both goodness of fit and the ability to recover the underlying network structure. The results show that, in the presence of nonlinear Gaussian relationships, SLGBN achieves better fit and improved structure recovery compared to alternative models, particularly when combined with the Extended Bayesian Information Criterion (EBIC*) score. Furthermore, by retaining Gaussian conditional distributions, SLGBN offers computational advantages over fully semiparametric approaches.

Aitor Oviedo-Madrid, Mabel Morales-Otero, Jose Gonzalez-Gomariz +1

Institute of Data Science and Artiﬁcial Intelligence (DATAI), University of Navarra, Pamplona, Spain · TECNUN School of Engineering, University of Navarra, San Sebastián, Spain

Bayesian Network Structure Learning Bayesian Networks Semiparametric Inference

Ask PDF

Aug 22, 2026

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

We propose a Bayesian nonparametric mixture of regression trees with a Dirichlet process prior over tree-parameter pairs, enabling data-driven selection of ensemble size and unifying CART, BART, random forests, and boosting. A novel splitting rule driven by the posterior predictive of a Gaussian process within each terminal node generates flexible, smooth decision boundaries; remarkably, the GP density cancels exactly in the Metropolis--Hastings ratio for GROW/PRUNE moves, ensuring computational feasibility. An exact Gibbs sampler for posterior predictive inference propagates uncertainty through random tree traversal. A parallel MPI implementation distributes independent tree updates across processors, achieving adequate speedups. We prove posterior consistency at rate

n^{-1/4}

in Hellinger distance under only continuity of the true regression function, allowing misspecification, via the identity

h(Θ)=0

. Simulations on Friedman benchmark show near-nominal coverage (0.94 Gaussian, 0.92 Cauchy), robust to high-dimensional noise and heavy tails, outperforming BART and bagged CART. Applications to QSAR toxicity, crime, riboflavin, wheat genomics, and air quality confirm reliable credible intervals and automatic sparsity. The DP mixture offers a principled, robust, theoretically justified alternative for challenging regression with honest uncertainty quantification.

Subhasish Basak, Anik Roy, Sourabh Bhattacharya

Indian Statistical Institute

Dirichlet Process Mixture Models Posterior Contraction Bayesian Inference Regression Prediction Intervals

Ask PDF

Sep 30, 2026

BayesNDE: Bayesian Generative Modeling for Neural Density Estimation

Density estimation is a fundamental problem in statistics and machine learning. In this work, we introduce BayesNDE, a neural density estimator based on Bayesian generative modeling. BayesNDE learns a Bayesian generative model and evaluates its density without requiring invertible networks or Jacobian-determinant computation. For each observation, it infers a sample-specific latent posterior to construct an adaptive proposal that focuses computation on regions contributing most to its density. Bridge sampling then combines samples from this proposal with separate posterior samples to estimate the density. Experiments on nonlinear and multimodal synthetic datasets show improved estimation of density values and better recovery of the density structure compared to the state-of-the-art neural density estimators. Applications to real-world datasets further demonstrate improved anomaly detection. Together, these results highlight BayesNDE as a flexible and effective neural density estimator, demonstrating how posterior inference can turn generative models into tools for density estimation. The code and tutorials are available at https://github.com/liuq-lab/BayesNDE.

Chenglin Li, Qiao Liu

Department of Biostatistics, Yale University New Haven, Connecticut, USA

Generative Modeling Density Estimation

Ask PDF

Abstract

Bayesian Networks

Explore similar work

Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

BayesNDE: Bayesian Generative Modeling for Neural Density Estimation