Dirichlet Process Mixtures within Semiparametric Bayesian Networks: A Novel Conditional Density Estimation Approach

Victor Alejandre; Pedro Larrañaga; Concha Bielza

cs.LGOct 26, 2026

Dirichlet Process Mixtures within Semiparametric Bayesian Networks: A Novel Conditional Density Estimation Approach

Authors: Victor Alejandre, Pedro Larrañaga, Concha Bielza

Organizations: Computational Intelligence Group Departamento de Inteligencia Artificial Universidad Politécnica de Madrid Campus de Montegancedo, Boadilla del Monte 28660 Madrid, Spain

Abstract

Semiparametric Bayesian networks (SPBNs) are flexible probabilistic graphical models for representing joint distributions of continuous variables through lower-dimensional conditional probability distributions (CPDs). Existing SPBN models use kernel density estimators (KDEs) for non-parametric CPD modeling, but their one-component-per-observation mixture representation can lead to increased inference costs and density estimation errors, which can be limiting. In this work, we define a new CPD modeling approach for non-parametric components based on Bayesian non-parametric Dirichlet Process Mixtures (DPMs) within SPBNs. DPMs provide a data-adaptive clustering of observations, yielding a more compact representation while retaining the flexibility of non-parametric density estimation, unlike KDE. This results in lower inference costs while improving density estimation performance. We implement the proposed DPM-based SPBN (DPM-SPBN) by extending the open-source PyBNesian package and evaluate it on synthetic and real-world data. For a given network structure, DPM-based CPDs significantly outperform state-of-the-art Bayesian network models for continuous variables in density estimation, achieving lower absolute log-likelihood errors on synthetic data. We further investigate the impact of the proposed CPD modeling on structure learning methods relying on CPD-based scores. DPM-based SPBNs achieve lower structure learning and density estimation errors than KDE-based SPBNs. Experiments on real-world data corroborate these findings, identifying DPM-based CPD modeling as the most effective alternative among the evaluated non-parametric approaches for continuous variables.

Figures & tables

Figure 1: Comparison of density estimation using a DPM and a KDE with a $200$ -sample from a true underlying three-component Gaussian mixture.

Figure 2: Schematic overview of parametric and non-parametric approaches for modeling continuous CPDs in BNs. Each approach branches into different models and underlying assumptions, with the resulting BNs shown in rectangles. The novel proposal introduced in this work is highlighted in green.

Figure 3: Example of a SPBN for JPD estimation. The figure illustrates a sample dataset $\mathcal{D}$ , the network structure distinguishing non-parametric ( $X_{1},X_{3}$ ) and parametric ( $X_{2},X_{4}$ ) nodes, the estimated CPDs derived from $\mathcal{D}$ and encoded conditional independencies.

Figure 4: Bayesian networks defined for the experimental setup of the work with sample sizes of 5, 10 and 15 nodes.

Smooth	$\displaystyle\bullet\;f(x_{1})=\mathcal{N}(0,1),$ $\displaystyle\bullet\;f(x_{2})=0.5\mathcal{N}(-2,2^{2})+0.5\mathcal{N}(2,2^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=\mathcal{N}(x_{1}x_{2},1),$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=\mathcal{N}(10+0.8x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=\mathcal{N}\!\left(\frac{1}{1+e^{-x_{4}}},0.5^{2}\right)$
Medium	$\displaystyle\bullet\;f(x_{1})=\mathcal{N}(0,3^{2}),$ $\displaystyle\bullet\;f(x_{2})=0.25\mathcal{N}(-8,1^{2})+0.25\mathcal{N}(8,1^{2})+0.25\mathcal{N}(-4,0.5^{2})+0.25\mathcal{N}(4,0.5^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=w_{1}\mathcal{N}(-0.5x_{1},1)+w_{2}\mathcal{N}(0.5x_{2},1)+w_{3}\mathcal{N}(0,1),$ $\displaystyle w_{1}=\frac{x_{1}^{2}}{x_{1}^{2}+x_{2}^{2}+2\|x_{1}x_{2}\|},\quad w_{2}=\frac{x_{2}^{2}}{x_{1}^{2}+x_{2}^{2}+2\|x_{1}x_{2}\|},\quad w_{3}=\frac{2\|x_{1}x_{2}\|}{x_{1}^{2}+x_{2}^{2}+2\|x_{1}x_{2}\|},$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=\mathcal{N}(0.5x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=w\mathcal{N}(-2,0.5^{2})+(1-w)\mathcal{N}(2,0.5^{2}),\quad w=\frac{1}{1+\exp(-x_{4}/3)}$
Rough	$\displaystyle\bullet\;f(x_{1})=\mathcal{N}(0,3^{2}),$ $\displaystyle\bullet\;f(x_{2})=0.2\mathcal{N}(-8,1^{2})+0.2\mathcal{N}(8,1^{2})+0.2\mathcal{N}(-4,0.5^{2})+0.2\mathcal{N}(0,0.5^{2})+0.2\mathcal{N}(4,0.5^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=w_{1}\mathcal{N}(-0.5x_{1}^{2},1)+w_{2}\mathcal{N}(0.5x_{2}^{2},1)+w_{3}\mathcal{N}(0.5x_{1},1)+w_{4}\mathcal{N}(-0.5x_{2},1),$ $\displaystyle w_{1}=\frac{x_{1}^{2}}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},\quad w_{2}=\frac{x_{2}^{2}}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},$ $\displaystyle w_{3}=\frac{\|x_{1}\|}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},\quad w_{4}=\frac{\|x_{2}\|}{x_{1}^{2}+x_{2}^{2}+\|x_{1}\|+\|x_{2}\|},$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=\mathcal{N}(0.5x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=w_{1}\mathcal{N}(-2,0.5^{2})+w_{2}\mathcal{N}(0,0.5^{2})+w_{3}\mathcal{N}(2,0.5^{2}),$ $\displaystyle w_{1}=\frac{1}{(1+\exp(-x_{4}/3))2},\quad w_{2}=\frac{1}{(1+\exp(-x_{4}/3))2},\quad w_{3}=1-w_{1}-w_{2}$
Nonparametric	$\displaystyle\bullet\;f(x_{1})=0.25\mathcal{N}(-4,0.5^{2})+0.25\mathcal{N}(-2,0.5^{2})+0.25\mathcal{N}(2,0.5^{2})+0.25\mathcal{N}(4,0.5^{2}),$ $\displaystyle\bullet\;f(x_{2})=0.5\mathcal{N}(-2,2^{2})+0.5\mathcal{N}(2,2^{2}),$ $\displaystyle\bullet\;f(x_{3}\mid x_{1},x_{2})=\mathcal{N}(x_{1}x_{2},1),$ $\displaystyle\bullet\;f(x_{4}\mid x_{3})=0.5\mathcal{N}(10+0.8x_{3},0.5^{2})+0.5\mathcal{N}(10-0.8x_{3},0.5^{2}),$ $\displaystyle\bullet\;f(x_{5}\mid x_{4})=\mathcal{N}\!\left(\frac{1}{1+e^{-x_{4}}},0.5^{2}\right)$

Table 1: Example of data generation mechanisms for the Smooth , Medium , Rough , and Nonparametric densities in the 5 node network.

Nodes	Density	Sample Size	SPBN-CAVI IQI		SPBN-GPCAVI IQI		SPBN-MEMOIZED IQI		Best Alternative IQI
5	Smooth	200	$\mathbf{3548}$ .	$\mathbf{44}~[3388.59,3740.85]$	$3647$ .	$55~[3416.09,3888.17]$	$4285$ .	$07~[3862.44,4566.23]$	$6085$ .	$68~[5910.16,6344.54]$ (NR)
		2000	$\mathbf{896}$ .	$\mathbf{37}~[847.70,939.15]$	$950$ .	$71~[902.42,1007.55]$	$1047$ .	$26~[984.11,1147.39]$	$2550$ .	$46~[2160.48,2941.39]$ (UCV)
		10000	$383$ .	$94~[363.52,409.47]$	$351$ .	$49~[321.34,372.01]$	$\mathbf{346}$ .	$\mathbf{07}~[325.27,374.63]$	$1066$ .	$21~[920.25,1249.90]$ (UCV)
		50000	$251$ .	$95~[234.60,280.55]$	$200$ .	$69~[185.11,208.88]$	$\mathbf{117}$ .	$\mathbf{36}~[101.12,124.94]$	$434$ .	$73~[383.04,521.34]$ (UCV)
		100000	$241$ .	$00~[223.69,258.95]$	$190$ .	$76~[171.64,199.12]$	$\mathbf{75}$ .	$\mathbf{14}~[65.70,87.74]$	$472$ .	$16~[421.11,568.19]$ (UCV)
	Medium	200	$6431$ .	$66~[6238.14,6623.57]$	$6558$ .	$40~[6260.33,6882.96]$	$7276$ .	$29~[7101.96,7376.25]$	$\mathbf{3622}$ .	$\mathbf{71}~[3375.82,4680.40]$ (UCV)

Table 2: Synthetic data experiments Absolute error summary for the LL across different node structures, densities and sample sizes in parameter learning experiments. Each case displays median and [IQI] for each method with the best median highlighted in bold. Best alternative method for each case is display between parenthesis.

Proposed Method	Total Wins	Sig. vs CAVI	Sig. vs GPCAVI	Sig. vs MEMOIZED	Sig. vs Best Alt.
SPBN-CAVI	19	-	14	17	19
SPBN-GPCAVI	0	0	-	0	0
SPBN-MEMOIZED	40	38	39	-	40

Table 3: Statistical significance of the proposed methods when achieving the lowest median absolute error. For each method, the number of cases in which it achieves the lowest median absolute error across all methods (total wins) and, among these, the number of wins with statistically significant pairwise differences are reported.

Figure 5: Pairwise comparison of state-of-the-art BNs for continous variables modeling and DPM-SPBN methods in parameter learning experiments. Each cell shows the number of cases in which the row method achieves a better median performance than the column method, with statistically significant differences reported in parentheses. Last column summarizes the overall and significant wins.

Figure 6: Parameter learning experiments distribution of relative computation times across network structures and sample sizes, with times averaged over density levels. The reference value corresponds to NR for training and SPBN-CAVI for prediction. Relative times are shown on a logarithmic scale.

Figure 7: Pairwise comparison of DPM-SPBN and standalone DPM methods in parameter learning experiments. Each cell shows the number of cases in which the row method achieves a better median performance than the column method, with statistically significant differences cases reported in parentheses. Last column summarizes the overall wins and significant wins out of those against all competitors.

Nodes	Density	Sample Size	SPBN-CAVI		SPBN-GPCAVI		NR		UCV
Nodes	Density	Sample Size	SHD IQI	LL Err.	SHD IQI	LL Err.	SHD IQI	LL Err.	SHD IQI	LL Err.
5	Smooth	200	$1.5~[1.0,5.8]$	$\times$	$\mathbf{1.0}~[1.0,4.8]$	✓	$4.5~[1.2,5.8]$	$\times$	$4.5~[4.0,5.8]$	$\times$
		2000	$5.0~[2.0,5.0]$	✓	$\mathbf{1.0}~[1.0,1.8]$	$\times$	$1.5~[1.0,5.0]$	$\times$	$2.5~[1.0,5.0]$	$\times$
		10000	$3.5~[1.0,6.0]$	$\times$	$\mathbf{1.0}~[1.0,4.2]$	✓	$4.0~[1.2,6.8]$	$\times$	$2.5~[1.0,5.0]$	$\times$
		50000	$\mathbf{4.0}~[3.2,6.0]$	$\times$	$\mathbf{4.0}~[3.2,6.0]$	✓	$5.0~[3.5,5.8]$	$\times$	-	-
		100000	$\mathbf{2.0}~[2.0,2.8]$	$\times$	$3.0~[3.0,6.0]$	✓	$5.5~[2.8,6.0]$	$\times$	-	-

Table 5: Structural Hamming distance (SHD) and log-likelihood error (LL Err.) across network sizes, densities, and sample sizes in the structure learning experiments. SHD is reported as median [IQR], while LL Err. indicates whether the method achieves (✓) the lowest absolute log-likelihood error or not (

\times

Dataset	$N$	# Variables
Glass	$214$	$9$
Ionosphere	$351$	$31$
QSAR Aquatic	$546$	$9$
Breast Cancer	$683$	$9$
QSAR Fish Toxicity	$908$	$7$
Yeast	$1484$	$8$

Table 7: Datasets from the Machine Learning UCI repository.

Figure 8: Performance comparison on real-world datasets between the proposed DPM-SPBN with inference methods CAVI and GPCAVI and state-of-the-art SPBNs using NR and UCV bandwidth selection. Distributions show test-set log-likelihoods obtained from $10$ -fold cross-validation.

Bayesian Networks

Bayesian networks represent joint probability distributions as graphs of variables and conditional dependencies, enabling inference under uncertainty.

See all 47 papers

Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines
Oct 22, 2026Aitor Oviedo-Madrid, Mabel Morales-Otero, Jose Gonzalez-Gomariz +1Bayesian Network Structure Learning Bayesian Networks
Institute of Data Science and Artiﬁcial Intelligence (DATAI), University of Navarra, Pamplona, Spain · TECNUN School of Engineering, University of Navarra, San Sebastián, Spain
Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate
Aug 22, 2026Subhasish Basak, Anik Roy, Sourabh BhattacharyaDirichlet Process Mixture Models Posterior Contraction
Indian Statistical Institute
BayesNDE: Bayesian Generative Modeling for Neural Density Estimation
Sep 30, 2026Chenglin Li, Qiao LiuGenerative Modeling Density Estimation
Department of Biostatistics, Yale University New Haven, Connecticut, USA

Oct 22, 2026J/K move · Enter open · S save

Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines

Aitor Oviedo-Madrid, Mabel Morales-Otero, Jose Gonzalez-Gomariz, Ruben Armañanzas

Institute of Data Science and Artiﬁcial Intelligence (DATAI), University of Navarra, Pamplona, Spain · TECNUN School of Engineering, University of Navarra, San Sebastián, Spain

Bayesian Network Structure Learning Bayesian Networks Semiparametric Inference

Open paper AskPDF arXiv

Bayesian networks for continuous variables often rely on restrictive assumptions: linear Gaussian models lack flexibility to capture nonlinear dependencies, while non-parametric approaches provide greater flexibility at a high computational cost. This creates a gap for modeling scenarios where variables are approximately Gaussian but exhibit nonlinear relationships. We introduce Semi-Linear Gaussian Bayesian Networks (SLGBN), a novel class of continuous Bayesian networks designed to bridge this gap, preserving Gaussian conditional distributions while introducing nonlinear conditional mean functions. The proposed framework provides a unified probabilistic framework combining linear Gaussian and nonlinear Gaussian conditional models within the same Bayesian network Nonlinear conditional mean functions are estimated using generalized additive models (GAMs) with penalized B-splines. We present parameter estimation procedures for both types of nodes, as well as a greedy hill-climbing algorithm for learning the network structure under the SLGBN framework. To assess its performance, we compare SLGBN with other continuous Bayesian network models, including semiparametric Bayesian networks (SPBN) and Linear Gaussian Bayesian networks (LGBN). Experimental results evaluate both goodness of fit and the ability to recover the underlying network structure. The results show that, in the presence of nonlinear Gaussian relationships, SLGBN achieves better fit and improved structure recovery compared to alternative models, particularly when combined with the Extended Bayesian Information Criterion (EBIC*) score. Furthermore, by retaining Gaussian conditional distributions, SLGBN offers computational advantages over fully semiparametric approaches.

Abstract

Bayesian Networks

Explore similar work

Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines

Dirichlet Process Mixtures of Trees with Gaussian Process Splits: A Bayesian Nonparametric Framework with Posterior Contraction Rate

BayesNDE: Bayesian Generative Modeling for Neural Density Estimation