Transfer learning for nonparametric Bayesian networks

Rafael Sojo; Pedro Larrañaga; Concha Bielza

cs.LGApr 1, 2026

Transfer learning for nonparametric Bayesian networks

Authors: Rafael Sojo, Pedro Larrañaga, Concha Bielza

Organizations: Aingura IIoT, Paseo Mikeletegui 43, 20009 Donostia-San Sebastián, Gipuzkoa, Spain · Universidad Politécnica de Madrid, Departamento de Inteligencia Artificial, 28660 Boadilla del Monte, Madrid, Spain

Abstract

This paper introduces two transfer learning methodologies for estimating nonparametric Bayesian networks under scarce data. We propose two algorithms, a constraint-based structure learning method, called PC-stable-transfer learning (PCS-TL), and a score-based method, called hill climbing transfer learning (HC-TL). We also define particular metrics to tackle the negative transfer problem in each of them, a situation in which transfer learning has a negative impact on the model's performance. Then, for the parameters, we propose a log-linear pooling approach. For the evaluation, we learn kernel density estimation Bayesian networks, a type of nonparametric Bayesian network, and compare their transfer learning performance with the models alone. To do so, we sample data from small, medium and large-sized synthetic networks and datasets from the UCI Machine Learning repository. Then, we add noise and modifications to these datasets to test their ability to avoid negative transfer. To conclude, we perform a Friedman test with a Bergmann-Hommel post-hoc analysis to show statistical proof of the enhanced experimental behavior of our methods. Thus, PCS-TL and HC-TL demonstrate to be reliable algorithms for improving the learning performance of a nonparametric Bayesian network with scarce data, which in real industrial environments implies a reduction in the required time to deploy the network.

Figures & tables

Data	Comments	Structure		Parameters	Negative	Reference
		Constraint	Score		transfer
Discrete	PC-TL with DBLP and LoLP	$\times$		$\times$		[ 21 ]
Discrete	Temporal node Bayesian networks	$\times$		$\times$		[ 23 ]
Discrete	PC-TL with majority vote	$\times$		$\times$		[ 24 ]
Discrete	Genetic algorithm for the structures		$\times$	$\times$		[ 25 ]
Discrete	Parameter transfer learning			$\times$		[ 26 ]

Table 1: Classification of Bayesian network transfer learning related works

Data	Comments	Negative transfer	Reference
Gaussian	Bayesian modeling		[ 28 ]
Gaussian	Bayesian modeling	$\times$	[ 30 ]
Discrete	Nonparametric modeling	$\times$	[ 30 ]
Nonparametric	Gaussian process regression		[ 32 ]
Nonparametric	Custering		[ 34 ]
Nonparametric	Distribution estimation		[ 35 ]

Table 2: Classification non-Bayesian network transfer learning related works

Figure 1: Flowchart of PCS-TL (in orange) and HC-TL (in blue). In green, functions that are shared by both processes.

Panel of Figure 2 — Figure 2: Structures of the synthetic SPBNs ( 56 ) .

Model	Nodes	Arcs	Max indegree
Synthetic SPBN 1	7	10	3
Synthetic SPBN 2	13	21	5
Synthetic SPBN 3	8	7	1
Synthetic SPBN 4	15	14	1

Table 3: Characteristics of the synthetic SPBNs.

Model	Nodes	Arcs	Max indegree
Magic-niab	44	66	9
Magic-irri	64	102	11

Table 4: Characteristics of the bnlearn GBNs

Panel of Figure 3 — Figure 3: Results for the synthetic SPBNs with two auxiliary source domains. 0% and 10% of arc modification.

Panel of Figure 4 — Figure 4: Results for the synthetic SPBNs with three auxiliary source domains. 5%, 10% and 20% of arc modification.

Panel of Figure 5 — Figure 5: Results for the bnlearn networks with two auxiliary source domains. 0% and 10% of arc modification.

Panel of Figure 6 — Figure 6: Results for the bnlearn networks with three auxiliary source domains. 5%, 10% and 20% of arc modification.

Dataset	Name	$N$	$n$
1	Single elder home monitoring: gas and position ( 58 )	416153	9
2	HTRU2 ( 59 )	17898	8
3	Individual household electric power consumption ( 60 )	2049280	7
4	MAGIC gamma telescope ( 61 )	19020	10
5	Appliances energy prediction ( 62 )	19735	24

Table 5: Datasets from the UCI Machine Learning repository.

Panel of Figure 7 — Figure 7: Results for the UCI datasets with two auxiliary source domains. 0% and 10% of arc modification.

Panel of Figure 8 — Figure 8: Results for the UCI datasets with three auxiliary source domains. 5%, 10 and 20% of arc modification.

Panel of Figure 9 — Figure 9: Average UCI structures for the HC and HC-TL algorithms with 25 target instances. Results for 3 auxiliary sources with 5%, 10% and 20% of arc modification.

Panel of Figure 10 — Figure 10: Average UCI structures for the PC-stable and PCS-TL algorithms with 25 target instances. Results for 3 auxiliary sources with 5%, 10% and 20% of arc modification.

Panel of Figure 11 — Figure 11: Critical difference diagram for the network’s DHD results. Evaluation for less than $525$ target instances.

Panel of Figure 12 — Figure 12: Significance heatmap with $p$ -values for the network’s DHD results. Evaluation for less than $525$ target instances.

Panel of Figure 13 — Figure 13: Critical difference diagram for the network’s log-likelihood results. Evaluation for less than $525$ target instances.

Panel of Figure 14 — Figure 14: Significance heatmap with $p$ -values for the network’s log-likelihood results. Evaluation for less than $525$ target instances.

Bayesian Network Structure Learning

See all 13 papers

Jul 31, 2025

Formal Bayesian Transfer Learning via the Total Risk Prior

Existing methods for transfer learning struggle to deal with situations where the source datasets are limited and not guaranteed to be well-aligned with the target dataset. A typical strategy is to use the empirical loss minimizer on the source data as a prior mean for the target parameters. We propose using a risk minimizer conditional on source parameters instead. This allows us to construct a joint posterior for all parameters from the source and target datasets. As a consequence, we benefit from full Bayesian uncertainty quantification and can perform model averaging via Gibbs sampling over indicator variables governing the inclusion of each source dataset. We show how a particular instantiation of our prior leads to a Bayesian Lasso in a transformed coordinate system and describe connections between the Maximum a Posteriori estimate associated with our approach and the Trans-Lasso method. We find that the MAP estimator MSE-dominates the Trans-Lasso in the normal means setting when there is no regularization on the source datasets. Finally, we perform numerical experiments finding that our approach provides superior predictive performance relative to frequentist and Bayesian baselines on a genetics application, particularly when the source data are limited.

Nathan Wycoff, Ali Arab, Lisa O. Singh

Department of Mathematics and Statistics, University of Massachusetts Amherst · Department of Computer Science, Georgetown University

Bayesian Inference Transfer Learning

Ask PDF

Aug 20, 2026

Transfer Learning in Nonparametric Regression with Deep ReLU Networks

This paper develops a general transfer learning framework for nonparametric regression with data consisting of multiple groups. Under the assumption that groups share a common structure along with group-specific deviations in additive form, the proposed method employs a two-stage offset learning procedure: the first stage pools data from all groups to estimate an overall mean function, and the second stage estimates offsets for each group, yielding final group-level estimators through additive combination. Upper bounds on the

\mathcal L_2

error are established for the proposed framework, covering a broad class of nonparametric estimators under mild complexity and noise conditions. When instantiated with deep ReLU networks, explicit convergence rates are derived under hierarchical composition models, demonstrating the ability to overcome the curse of dimensionality. Conditions that enable positive transfer with faster rates are considered, including learning with simpler functions and data augmentation through pooling samples across groups. Various simulations and real-data experiments further validate the effectiveness of the proposed method.

Junpeng Ren, Carlos Misael Madrid Padilla, Yanzhen Chen +1

Department of Statistics and Data Science, University of California, Los Angeles, California, USA · Department of Statistics and Data Science, Washington University in St. Louis, Missouri, USA · Department of Information Systems, Business Statistics and Operations Management, Hong Kong University of Science and Technology, Hong Kong, China.

Transfer Learning Nonparametric Regression Statistical Learning Theory Multi-Task Learning

Ask PDF

Oct 8, 2026

$σ$ Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$

Reliable predictive uncertainty in Laplace approximations depends critically on the prior precision, yet selecting it requires a posterior sweep that is prohibitively expensive for neural networks with billions of parameters. Under the Maximal Update Parametrization (

μ\mathrm{P}

), we derive a rescaling of the prior covariance that makes the selected precision stable as model width grows. This leads to

σ\mathrm{Transfer}

: we select the precision on a smaller model and zero-shot transfer it to the much larger model, i.e., without searching for the precision on the larger model at all. We show convergence of the prior kernel, posterior covariance, selected precision, and posterior-derived decisions under explicit conditions, and verify

σ\mathrm{Transfer}

across regression, image classification, and Transformer readouts. For example, measured precision-sweep speedups reach

\sim 5000\times

when transferring from width 128 to 4096 on MNIST, at a target-NLL degradation of

0.002

; transferring from a public 1B to 7B model gives a median search speedup of

\sim 2.3\times

(up to

\sim 330\times

), with a mean measured target-NLL increase below

10^{-4}

across ten tasks. The same posterior stability also enables transfer of acquisition, OOD-detection, and abstention decisions without constructing a target posterior.

Richard Bergna, Fernando Ruiz Mazo, Nicolò Felicioni +2

University of Cambridge · Spotify

Uncertainty Quantification Bayesian Inference Hyperparameter Transfer

Ask PDF

Abstract

Bayesian Network Structure Learning

Explore similar work

Formal Bayesian Transfer Learning via the Total Risk Prior

Transfer Learning in Nonparametric Regression with Deep ReLU Networks

σσσTransfer: Uncertainty Transfer from Small to Large Networks under μPμ\mathrm{P}μP

$σ$ Transfer: Uncertainty Transfer from Small to Large Networks under $μ\mathrm{P}$