Transfer learning for nonparametric Bayesian networks

Rafael Sojo; Pedro Larrañaga; Concha Bielza

cs.LGApr 1, 2026

Transfer learning for nonparametric Bayesian networks

Authors: Rafael Sojo, Pedro Larrañaga, Concha Bielza

Organizations: Aingura IIoT, Paseo Mikeletegui 43, 20009 Donostia-San Sebastián, Gipuzkoa, Spain · Universidad Politécnica de Madrid, Departamento de Inteligencia Artificial, 28660 Boadilla del Monte, Madrid, Spain

Abstract

This paper introduces two transfer learning methodologies for estimating nonparametric Bayesian networks under scarce data. We propose two algorithms, a constraint-based structure learning method, called PC-stable-transfer learning (PCS-TL), and a score-based method, called hill climbing transfer learning (HC-TL). We also define particular metrics to tackle the negative transfer problem in each of them, a situation in which transfer learning has a negative impact on the model's performance. Then, for the parameters, we propose a log-linear pooling approach. For the evaluation, we learn kernel density estimation Bayesian networks, a type of nonparametric Bayesian network, and compare their transfer learning performance with the models alone. To do so, we sample data from small, medium and large-sized synthetic networks and datasets from the UCI Machine Learning repository. Then, we add noise and modifications to these datasets to test their ability to avoid negative transfer. To conclude, we perform a Friedman test with a Bergmann-Hommel post-hoc analysis to show statistical proof of the enhanced experimental behavior of our methods. Thus, PCS-TL and HC-TL demonstrate to be reliable algorithms for improving the learning performance of a nonparametric Bayesian network with scarce data, which in real industrial environments implies a reduction in the required time to deploy the network.

Figures & tables

Data	Comments	Structure		Parameters	Negative	Reference
		Constraint	Score		transfer
Discrete	PC-TL with DBLP and LoLP	$\times$		$\times$		[ 21 ]
Discrete	Temporal node Bayesian networks	$\times$		$\times$		[ 23 ]
Discrete	PC-TL with majority vote	$\times$		$\times$		[ 24 ]
Discrete	Genetic algorithm for the structures		$\times$	$\times$		[ 25 ]
Discrete	Parameter transfer learning			$\times$		[ 26 ]

Table 1: Classification of Bayesian network transfer learning related works

Data	Comments	Negative transfer	Reference
Gaussian	Bayesian modeling		[ 28 ]
Gaussian	Bayesian modeling	$\times$	[ 30 ]
Discrete	Nonparametric modeling	$\times$	[ 30 ]
Nonparametric	Gaussian process regression		[ 32 ]
Nonparametric	Custering		[ 34 ]
Nonparametric	Distribution estimation		[ 35 ]

Table 2: Classification non-Bayesian network transfer learning related works

Figure 1: Flowchart of PCS-TL (in orange) and HC-TL (in blue). In green, functions that are shared by both processes.

Panel of Figure 2 — Figure 2: Structures of the synthetic SPBNs ( 56 ) .

Model	Nodes	Arcs	Max indegree
Synthetic SPBN 1	7	10	3
Synthetic SPBN 2	13	21	5
Synthetic SPBN 3	8	7	1
Synthetic SPBN 4	15	14	1

Table 3: Characteristics of the synthetic SPBNs.

Model	Nodes	Arcs	Max indegree
Magic-niab	44	66	9
Magic-irri	64	102	11

Table 4: Characteristics of the bnlearn GBNs

Panel of Figure 3 — Figure 3: Results for the synthetic SPBNs with two auxiliary source domains. 0% and 10% of arc modification.

Panel of Figure 4 — Figure 4: Results for the synthetic SPBNs with three auxiliary source domains. 5%, 10% and 20% of arc modification.

Panel of Figure 5 — Figure 5: Results for the bnlearn networks with two auxiliary source domains. 0% and 10% of arc modification.

Panel of Figure 6 — Figure 6: Results for the bnlearn networks with three auxiliary source domains. 5%, 10% and 20% of arc modification.

Dataset	Name	$N$	$n$
1	Single elder home monitoring: gas and position ( 58 )	416153	9
2	HTRU2 ( 59 )	17898	8
3	Individual household electric power consumption ( 60 )	2049280	7
4	MAGIC gamma telescope ( 61 )	19020	10
5	Appliances energy prediction ( 62 )	19735	24

Table 5: Datasets from the UCI Machine Learning repository.

Panel of Figure 7 — Figure 7: Results for the UCI datasets with two auxiliary source domains. 0% and 10% of arc modification.

Panel of Figure 8 — Figure 8: Results for the UCI datasets with three auxiliary source domains. 5%, 10 and 20% of arc modification.

Panel of Figure 9 — Figure 9: Average UCI structures for the HC and HC-TL algorithms with 25 target instances. Results for 3 auxiliary sources with 5%, 10% and 20% of arc modification.

Panel of Figure 10 — Figure 10: Average UCI structures for the PC-stable and PCS-TL algorithms with 25 target instances. Results for 3 auxiliary sources with 5%, 10% and 20% of arc modification.

Panel of Figure 11 — Figure 11: Critical difference diagram for the network’s DHD results. Evaluation for less than $525$ target instances.

Panel of Figure 12 — Figure 12: Significance heatmap with $p$ -values for the network’s DHD results. Evaluation for less than $525$ target instances.

Panel of Figure 13 — Figure 13: Critical difference diagram for the network’s log-likelihood results. Evaluation for less than $525$ target instances.

Panel of Figure 14 — Figure 14: Significance heatmap with $p$ -values for the network’s log-likelihood results. Evaluation for less than $525$ target instances.

Bayesian Network Structure Learning

See all 13 papers

Formal Bayesian Transfer Learning via the Total Risk Prior
Jul 31, 2025Nathan Wycoff, Ali Arab, Lisa O. SinghBayesian Inference Transfer Learning
Department of Mathematics and Statistics, University of Massachusetts Amherst · Department of Computer Science, Georgetown University
Transfer Learning in Nonparametric Regression with Deep ReLU Networks
Aug 20, 2026Junpeng Ren, Carlos Misael Madrid Padilla, Yanzhen Chen +1Transfer Learning Nonparametric Regression
Department of Statistics and Data Science, University of California, Los Angeles, California, USA · Department of Statistics and Data Science, Washington University in St. Louis, Missouri, USA · Department of Information Systems, Business Statistics and Operations Management, Hong Kong University of Science and Technology, Hong Kong, China.
$σ$ Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$
Oct 8, 2026Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni +2Uncertainty Quantification Bayesian Inference
University of Cambridge · Spotify

Jul 31, 2025J/K move · Enter open · S save

Formal Bayesian Transfer Learning via the Total Risk Prior

Nathan Wycoff, Ali Arab, Lisa O. Singh

Department of Mathematics and Statistics, University of Massachusetts Amherst · Department of Computer Science, Georgetown University

Bayesian Inference Transfer Learning

Open paper AskPDF arXiv

Existing methods for transfer learning struggle to deal with situations where the source datasets are limited and not guaranteed to be well-aligned with the target dataset. A typical strategy is to use the empirical loss minimizer on the source data as a prior mean for the target parameters. We propose using a risk minimizer conditional on source parameters instead. This allows us to construct a joint posterior for all parameters from the source and target datasets. As a consequence, we benefit from full Bayesian uncertainty quantification and can perform model averaging via Gibbs sampling over indicator variables governing the inclusion of each source dataset. We show how a particular instantiation of our prior leads to a Bayesian Lasso in a transformed coordinate system and describe connections between the Maximum a Posteriori estimate associated with our approach and the Trans-Lasso method. We find that the MAP estimator MSE-dominates the Trans-Lasso in the normal means setting when there is no regularization on the source datasets. Finally, we perform numerical experiments finding that our approach provides superior predictive performance relative to frequentist and Bayesian baselines on a genetics application, particularly when the source data are limited.

Abstract

Bayesian Network Structure Learning

Explore similar work

Formal Bayesian Transfer Learning via the Total Risk Prior

Transfer Learning in Nonparametric Regression with Deep ReLU Networks

σσσTransfer: Uncertainty Transfer from Small to Large Networks under μPμ\mathrm{P}μP

$σ$ Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$