Bayesian Tensor Train Kernel Machines with Automatic Model Complexity Inference

Afra Kilic; Kim Batselier

stat.MLNov 4, 2026

Bayesian Tensor Train Kernel Machines with Automatic Model Complexity Inference

Authors: Afra Kilic, Kim Batselier

Organizations: Delft Center for Systems and Control, Delft University of Technology, Delft, 2628 CN, The Netherlands

Abstract

Tensor Network (TN) kernel machines accelerate supervised learning by using low-rank TNs to reduce computation and memory. However, most TN-based methods are deterministic with manually tuned ranks and feature dimensions, while the existing Bayesian TN kernel machine is restricted to the CANDECOMP/PARAFAC decomposition (CPD), whose single rank is shared by all modes. We extend this framework to the tensor train (TT) and propose Bayesian Tensor Train Kernel Machines (BTT-KM), whose hierarchical ARD-type priors on the TT cores infer the TT ranks and feature dimensions from data. Since each TT core is coupled to the others only through its neighbors, its closed-form posterior update requires propagating the contributions of all other cores along the chain. Mean-field variational inference yields a Bayesian alternating least squares algorithm with the same computational complexity as its deterministic counterpart and a closed-form predictive distribution. A synthetic experiment illustrates that BTT-KM recovers the true TT ranks and feature dimensions from an over-parameterized initialization.

Figures & tables

Panel of Figure 1 — Figure 1: Horizontal $\mathbf{W}^{(d)}_{r_{d}::}$ , lateral $\mathbf{W}^{(d)}_{:m_{d}:}$ , and frontal $\mathbf{W}^{(d)}_{::r_{d+1}}$ slices of a TT core $\bm{\mathcal{W}}^{(d)}$ .

Panel of Figure 2 — Figure 2: Representation of BTT-KM with the TT-decomposed weight vector $\mathbf{w}$ as a probabilistic graphical model showing the ARD-type hierarchical priors over the TT cores $\{\bm{\mathcal{W}}^{(d)}\}_{d=1}^{D}$ by the relevance parameters $\{\bm{\lambda}_{d}\}_{d=2}^{D}$ and $\{\bm{\delta}_{d}\}_{d=1}^{D}$ . The dashed node denotes the observed data $\mathbf{y}$ , while the solid nodes represent random variables. Shape and rate hyperparameters of the Gamma priors placed on $\{\bm{\lambda}_{d}\}_{d=2}^{D}$ , $\{\bm{\delta}_{d}\}_{d=1}^{D}$ and $\tau$ are shown as unbounded nodes.

Figure 3: Variance explained (%) per rank component of each TT rank

R_{d}

, together with the posterior mean log-precisions

\log\tilde{\bm{\lambda}}_{d}

Figure 4: Variance explained (%) per feature component of each core, together with the posterior mean log-precisions

\log\tilde{\bm{\delta}}_{d}

Kernel Methods

Kernel methods use similarity functions and reproducing kernel spaces to model nonlinear relationships, quantify uncertainty, and compare data.

See all 248 papers

Gradient Boosting for Efficient Rank Discovery in Tensor Network Kernel Machines
Nov 5, 2026Albert Saiapin, Kim BatselierTensor Networks Kernel Methods
Delft Center for Systems and Control Delft University of Technology
Laplace Approximation for Bayesian Tensor Network Kernel Machines
Apr 29, 2026Albert Saiapin, Kim BatselierUncertainty Quantification Bayesian Inference
1Delft Center for Systems and Control, Delft University of Technology
Joint Bayesian Parameter and Model Order Estimation for Low-Rank Probability Mass Tensors
Oct 8, 2024Joseph K. Chege, Arie Yeredor, Martin HaardtBayesian Inference Tensor Decomposition
Communications Research Laboratory, Ilmenau University of Technology, Ilmenau, Germany · School of Electrical and Computer Engineering, Tel Aviv University, Tel Aviv, Israel

Nov 5, 2026J/K move · Enter open · S save

Gradient Boosting for Efficient Rank Discovery in Tensor Network Kernel Machines

Albert Saiapin, Kim Batselier

Delft Center for Systems and Control Delft University of Technology

Tensor Networks Kernel Methods

Open paper AskPDF arXiv

Tensor network kernel machines (TNKMs) combine nonlinear features with low-rank tensor representations to model complex relationships efficiently. However, selecting an appropriate tensor rank remains challenging, as conventional rank-search methods require training multiple candidate models, increasing computational costs. In this work, we focus on a canonical polyadic (CP) tensor network parameterization and propose GB-CPKM, a gradient boosting framework that builds a higher-capacity predictor by sequentially adding fixed-rank learners. This approach provides a new perspective on rank selection by incrementally increasing model capacity, with ensemble size selected based on validation performance rather than through explicit rank search. Experiments on real-world regression benchmarks show that GB-CPKM trains up to 6x faster and uses up to 2x less peak memory than a conventional rank-selection method based on cross-validation, while achieving comparable predictive performance. These results highlight the potential of gradient boosting to improve the computational efficiency and scalability of tensor-network learning.

Abstract

Kernel Methods

Explore similar work

Gradient Boosting for Efficient Rank Discovery in Tensor Network Kernel Machines

Laplace Approximation for Bayesian Tensor Network Kernel Machines

Joint Bayesian Parameter and Model Order Estimation for Low-Rank Probability Mass Tensors