Gaussian Universality and Its Breakdown in Tensor-Network Machine Learning

Shi-Tuan Wang; Zidu Liu; Li-Wei Yu

quant-phOct 5, 2026

Gaussian Universality and Its Breakdown in Tensor-Network Machine Learning

Authors: Shi-Tuan Wang, Zidu Liu, Li-Wei Yu

Organizations: Theoretical Physics Division, Chern Institute of Mathematics and LPMC, Nankai University, Tianjin 300071, China · Max Planck Institute of Quantum Optics, 85748 Garching, Germany

Abstract

Gaussian-process limits are powerful in describing overparameterized machine learning models, yet their validity in structured tensor-network architectures remains unclear. Here we analytically present a moment-based approach that identifies precise conditions for the emergence and breakdown of Gaussian universality in tensor-network learning models, with a focus on matrix product states. We prove that in the large bond dimension limit, the learning models with both local and global observables converge to Gaussian processes, with explicit finite-size bounds on higher-order moment deviations. Whereas in the large physical dimension limit, the Gaussian universality no longer persists: while the models with local observables retain Gaussian-process behavior, those global cases exhibit persistent non-Gaussian corrections. Our results reveal that Gaussian-process behavior in tensor-network learning is controlled not only by parameter number, but also by architectural scaling, observable locality, and the spectral properties.

Figures & tables

Fig. 1: Schematic summary of statistical behaviors arising in different tensor-network scaling regimes. For the local observable outcome $C(\hat{O}_{l})=\langle\psi|\hat{O}_{l}|\psi\rangle$ , as either $D$ or $d$ increases, the outcome distribution approaches a Gaussian distribution. For the global observable outcome $C(\hat{O}_{g})=\langle\psi|\hat{O}_{g}|\psi\rangle$ , as $D$ increases, the outcome distribution gradually approaches a Gaussian distribution, whereas for increasing $d$ , the outcome distribution is no longer Gaussian. For a finite sequence $\{\hat{O}_{ai}\}_{i=1}^{p}$ , one has $\mathcal{P}(C(\hat{O}_{a1}),\cdots,C(\hat{O}_{ap}))\sim\mathcal{N}\left(0,\varSigma\right)$ , $a\in\{l,g\}$ .

Observable Type	Asymptotic Regime	Gaussian?	Deviation Term
	$(D,n)\to\infty,\,D^{2}\ll d^{n}$	$\checkmark$	$\mathcal{O}(k^{2}/D^{2}d)+\mathcal{O}(\mathrm{poly}\left(k\right)D^{2}/d^{n})$
Local $\hat{O}_{l}$	$(D,n)\to\infty,\,D^{2}\gg d^{n}$	$\checkmark$	$\mathcal{O}(k^{2}d^{n/2}/D)+\mathcal{O}(k^{2}/d^{n})$
Local $\hat{O}_{l}$	$d\to\infty$	$\checkmark$	$\mathcal{O}(k^{2}D^{2}/d)$
Global $\hat{O}_{g}$	$(D,n)\to\infty$	$\checkmark$	$\mathcal{O}(nk!m^{k/2}/D)+\mathcal{O}(k^{2}/d^{n})$
Global $\hat{O}_{g}$	$d\to\infty$	$\times$	Non-vanishing * * * For explicit analytical expressions, see Sec. IV D of the Supplemental Material [ 1 ]

Table 1: Summary of the statistical regimes and asymptotic behaviors of output vector

\mathscr{C}

Fig. 2: Two-dimensional joint probability densities and their scaled marginal distributions obtained numerically with $n=15$ physical sites, bond dimension $D=128$ , and $4\times 10^{6}$ samples. The left panel shows the joint distribution of $\langle X\rangle$ and $\langle Z\rangle$ , while the right panel presents the distribution for $\langle\frac{1}{\sqrt{2}}\left(X+Z\right)\rangle$ and $\langle Z\rangle$ .

Fig. 3: Numerical distributions and theoretical predictions for MPS observables, using approximately $10^{7}$ samples for each parameter set. Upper panel: local outputs $C(\hat{O}_{l})$ for different system sizes $n$ at $D=32$ , compared with the predicted Gaussian distribution. The inset shows the excess kurtosis $\kappa$ as a function of $D$ . Lower panel: global outputs $C(\hat{O}_{g})$ at $d=64$ and $D=2$ , compared with a Gaussian distribution of equal variance. The inset shows $\kappa$ as a function of $d$ at $D=2$ .

Fig. S2: Subscript connection modes in the second-order case

Fig. S3: Subscript connection modes in the third-order case

Fig. S4: Output distribution with local operator basis ( $C(\hat{Q}_{l})$ )

Fig. S5: Output distribution with global operator basis ( $C(\hat{Q}_{g})$ )

Fig. S6: Representation diagram of $X_{l}^{k}$

Fig. S7: Representation diagram of the observation site

Fig. S8: Representation diagram of the non-observation site

Fig. S9: Representation diagram of $X_{g}^{k}$

Fig. S10: Representation diagram of a local observation site in $X_{g}^{k}$

Explore similar work

Jul 7, 2026cs.LG

Quantitative Gaussian-Process limits of Tensor Programs

We study the infinite-width Gaussian-process limit of random neural networks through the lens of tensor programs, and we provide a quantitative convergence theory in Wasserstein distance. Our main result gives explicit finite-width error bounds, of order inverse square-root of the widths between finite-network executions and their Gaussian-process limits. The framework is architecture-agnostic and covers feed-forward models together with weight-sharing schemes relevant for recurrent and transformer-type architectures.

Andrea Agazzi, Eloy Mosig García, Dario Trevisan

Two-Layer Neural Networks Gaussian Process Wasserstein Distance Quantifying

Ask PDF

Apr 29, 2026stat.ML

Laplace Approximation for Bayesian Tensor Network Kernel Machines

Uncertainty estimation is essential for robust decision-making in the presence of ambiguous or out-of-distribution inputs. Gaussian Processes (GPs) are classical kernel-based models that offer principled uncertainty quantification and perform well on small- to medium-scale datasets. Alternatively, formulating the weight space learning problem under tensor network assumptions yields scalable tensor network kernel machines. However, these assumptions break Gaussianity, complicating standard probabilistic inference. This raises a fundamental question: how can tensor network kernel machines provide principled uncertainty estimates? We propose a novel Bayesian Tensor Network Kernel Machine (LA-TNKM) that employs a (linearized) Laplace approximation for Bayesian inference. A comprehensive set of numerical experiments shows that the proposed method consistently matches or surpasses Gaussian Processes and Bayesian Neural Networks (BNNs) across diverse UCI regression benchmarks, highlighting both its effectiveness and practical relevance.

Albert Saiapin, Kim Batselier

1Delft Center for Systems and Control, Delft University of Technology

Bayesian Neural Networks Gaussian Process Tensor Networks Uncertainty-Aware Classification Probabilistic Inference Approximation

Ask PDF

Feb 17, 2022math.ST

Universality of empirical risk minimization

We study a general class of optimization problems with decision variable

\boldsymbolΘ \in \mathbb{R}^{p \times k}

and cost function which is the sum of

n

terms, each dependent on

\boldsymbolΘ

through the

k

-dimensional projection

\boldsymbolΘ^\top \boldsymbol{x}_i

, where

\boldsymbol{x}_i

i \leq n

are i.i.d. random vectors. This setting is general enough to include examples of current interest in statistical physics, high-dimensional statistics, and statistical learning theory. We consider the proportional asymptotics

n, p \to \infty

, with

n/p = Θ(1)

, and prove that, whenever there exists a minimizer satisfying a suitable generalization of a "delocalization" condition, the minimum value is universal. Namely, (for subgaussian

\boldsymbol{x}_i

) it depends on the distribution of

\boldsymbol{x}_i

only through its asymptotic mean and covariance. This delocalization condition is essentially necessary. Earlier universality results for such problems were limited to strongly convex loss functions. We derive applications of our theory to statistical learning and prove general universality results both for train and (under additional conditions) test error. In particular, we establish universality for vectors

\boldsymbol{x}_i

generated by random 1-layer neural networks (random features models) and first-order Taylor approximations of 2-layer networks (neural tangent models). Finally, we establish that the delocalization property holds for a class of statistical learning problems under a condition that is easy to verify.

Andrea Montanari, Basil Saeed

Department of Statistics and Department of Mathematics, Stanford University · Department of Electrical Engineering, Stanford University

Empirical Risk Minimization Neural Tangent Kernel Loss Function Risk

Ask PDF