Semi-Linear Gaussian Bayesian Networks: Enhancing Flexibility in Node Dependencies via penalized B-Splines
Authors: Aitor Oviedo-Madrid, Mabel Morales-Otero, Jose Gonzalez-Gomariz, Ruben Armañanzas
Organizations: Institute of Data Science and Artificial Intelligence (DATAI), University of Navarra, Pamplona, Spain · TECNUN School of Engineering, University of Navarra, San Sebastián, Spain
Bayesian networks for continuous variables often rely on restrictive assumptions: linear Gaussian models lack flexibility to capture nonlinear dependencies, while non-parametric approaches provide greater flexibility at a high computational cost. This creates a gap for modeling scenarios where variables are approximately Gaussian but exhibit nonlinear relationships. We introduce Semi-Linear Gaussian Bayesian Networks (SLGBN), a novel class of continuous Bayesian networks designed to bridge this gap, preserving Gaussian conditional distributions while introducing nonlinear conditional mean functions. The proposed framework provides a unified probabilistic framework combining linear Gaussian and nonlinear Gaussian conditional models within the same Bayesian network Nonlinear conditional mean functions are estimated using generalized additive models (GAMs) with penalized B-splines. We present parameter estimation procedures for both types of nodes, as well as a greedy hill-climbing algorithm for learning the network structure under the SLGBN framework. To assess its performance, we compare SLGBN with other continuous Bayesian network models, including semiparametric Bayesian networks (SPBN) and Linear Gaussian Bayesian networks (LGBN). Experimental results evaluate both goodness of fit and the ability to recover the underlying network structure. The results show that, in the presence of nonlinear Gaussian relationships, SLGBN achieves better fit and improved structure recovery compared to alternative models, particularly when combined with the Extended Bayesian Information Criterion (EBIC*) score. Furthermore, by retaining Gaussian conditional distributions, SLGBN offers computational advantages over fully semiparametric approaches.
Figures & tables
Figure 1: Greedy Hill-Climbing: The figure shows four steps of the Greedy Hill-Climbing Algorithm and changes in the estimation of X2 . Yellow nodes represent nonlinear nodes and gray nodes represent linear nodes. Changes in arcs are represented with red arrows and changes in types are represented by red contours. Step 1 starts with a linear X2 node depending on X1 . In step 2 node type is changed from linear to nonlinear and X2 is estimated using B-splines. In step 3 arc ( X1 , X2 ) is reversed and X2 loses its only parent. Its density is at this point estimated with sample mean but the node type is still nonlinear. Note that at this point type_change operator is not available for X2 as it has no parents. In step 4 X2 gains a new parent after adding ( X3 , X2 ) edge and is again nonlinearly estimated with B-splines, but now depending on X3 .
Network
Number of nodes
Number of edges
Source
B5
5
5
Own elaboration
B8
8
12
Own elaboration
B_Insu
27
52
bnlearn
Table 1: Synthetic Networks.
Figure 2: B5 network. First synthetic network structure and node probability distributions.
Figure 3: B8 network. Second synthetic network structure and node probability distribution.
Figure 4: Fixed metrics. Evolution of the metrics in structure learning with fixed structure for different model-score configurations. Values represent the mean across all random seeds for each sample size. Rows correspond to metrics: a) Log-likelihood, b) THD (Lower is better). Columns correspond to networks: B5 (left), B8 (center), and B_insu (right). X axis shows sample sizes. Y axis represents metric values, being common for THD.
Figure 5: Full metrics. Evolution of metrics in full structure learning for different model-score configurations. Values represent the mean across all random seeds for each sample size. Rows correspond to metrics: a) Log-likelihood, b) THD (Lower is better), c) SHD (Lower is better), d) Precision, and e) Recall. Columns correspond to networks: B5 (left), B8 (center), and B_insu (right). X axis shows sample sizes. Y axis represents metric values, with common axis by network for all metrics except log-likelihood.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: BN_insu network. The structure of the insurance network from bnlean . Third synthetic network.
Semiparametric Bayesian networks (SPBNs) are flexible probabilistic graphical models for representing joint distributions of continuous variables through lower-dimensional conditional probability distributions (CPDs). Existing SPBN models use kernel density estimators (KDEs) for non-parametric CPD modeling, but their one-component-per-observation mixture representation can lead to increased inference costs and density estimation errors, which can be limiting. In this work, we define a new CPD modeling approach for non-parametric components based on Bayesian non-parametric Dirichlet Process Mixtures (DPMs) within SPBNs. DPMs provide a data-adaptive clustering of observations, yielding a more compact representation while retaining the flexibility of non-parametric density estimation, unlike KDE. This results in lower inference costs while improving density estimation performance. We implement the proposed DPM-based SPBN (DPM-SPBN) by extending the open-source PyBNesian package and evaluate it on synthetic and real-world data. For a given network structure, DPM-based CPDs significantly outperform state-of-the-art Bayesian network models for continuous variables in density estimation, achieving lower absolute log-likelihood errors on synthetic data. We further investigate the impact of the proposed CPD modeling on structure learning methods relying on CPD-based scores. DPM-based SPBNs achieve lower structure learning and density estimation errors than KDE-based SPBNs. Experiments on real-world data corroborate these findings, identifying DPM-based CPD modeling as the most effective alternative among the evaluated non-parametric approaches for continuous variables.
Victor Alejandre, Pedro Larrañaga, Concha Bielza
Computational Intelligence Group Departamento de Inteligencia Artificial Universidad Politécnica de Madrid Campus de Montegancedo, Boadilla del Monte 28660 Madrid, Spain
The Generalized Naive Bayes (GNB) model was introduced for discrete and categorical random variables as an extension of classic Naive Bayes. We now accommodate the GNB framework to continuous explanatory variables. A central result of the paper is that structure learning of the GNB depends only on the pair copulas of the bi-variate marginals. We proved that the GNB structure can be assigned to the basis of a matroid, therefore we give greedy algorithms for finding the optimal GNB structure on the training data, in sense of minimizing Kullback-Leibler divergence. Three cases are considered: joint Gaussian distribution, then a more flexible model where we suppose the dependence structure to be described by a Gaussian copula with arbitrary marginals, and an even more flexible case where the joint continuous probability distribution is arbitrary, i.e. copula and marginal distributions are arbitrary. A method for model reduction, based on the newly introduced concept of GNB forest is given. We close the paper by comparing the newly introduced GNB classification results to other classical "glass-box" algorithms on real datasets.
Ábrahám Papp, Botond Szilágyi, Edith Alice Kovács
Department of Analysis and Operations Research, Institute of Mathematics, Budapest University of Technology and Economics · Department of Chemical and Environmental Process Engineering, Faculty of Chemical Technology and Biotechnology, Budapest University of Technology and Economics
Bayesian Networks (BNs) are established tools for letting agents reason under uncertainty in sequential decision-making. They split a joint distribution into local conditional mechanisms arranged on a directed acyclic graph, which makes them modular, interpretable, and ready to perform probabilistic inference. Their classical formulation, however, has limitations in both expressiveness and scale: each conditional is usually stored as a table or a linear-Gaussian form, and inference evaluates the graph one node and one query at a time. In this paper we introduce Neural Bayesian Networks (NBNs), in which each conditional is a differentiable, tensor-valued mechanism, while the graph and its factorization are unchanged. NBNs retain the modularity and local mechanisms of BNs, can represent any learnable conditional distribution, and run natively on GPUs. We also release NBN, a PyTorch-based library for learning and inference in NBNs, together with a public benchmark. Our evaluation shows gains in both expressiveness and scale: NBNs fit distributions that classical BN libraries struggle to represent, and on large networks and large batches of queries they run two orders of magnitude faster thanks to GPU parallelization. NBN is available at https://github.com/Giovannibriglia/NeuralBayesianNetworks.
Giovanni Briglia, Marco Lippi, Stefano Mariani +1
University of Modena and Reggio Emilia, Reggio Emilia, Italy · University of Florence, Florence, Italy