Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Models

Grégoire Sergeant-Perthuis; Elias Tsigaridas; Jules Tsukahara

cs.LGAug 20, 2026

Exact Algebraic Computation of Learning Coefficients for Two-Dimensional Singular Models

Authors: Grégoire Sergeant-Perthuis, Elias Tsigaridas, Jules Tsukahara

Organizations: CQSB, Sorbonne Université · Ouragan Team, INRIA · Ouragan team, Inria Paris

Abstract

Classical information criteria such as the Bayesian Information Criterion (BIC) rely on regularity assumptions that break down for singular models, leading to incorrect model selection in settings such as deep learning. The Widely Applicable Bayesian Information Criterion (WBIC) relies on local learning coefficients $λ$ , which in the analytic case coincides with local Real Log Canonical Thresholds (RLCT) of the Kullback-Leibler divergence of the model, to capture correct marginal likelihood asymptotics. Exact computation of the learning coefficients has been limited to special cases, and only sampling-based estimation methods are generally applicable. We present the first deterministic algorithm that computes local RLCTs exactly for any two-dimensional model whose Kullback-Leibler distance is contact equivalent to a polynomial, derive a bound on its complexity, and demonstrate its effectiveness for a broad class of models, with applications including polynomial neural networks. Beyond providing ground truth to calibrate sampling-based estimators, exact computation reveals algebraic structure in learning coefficients that sampling cannot and out-speeds it in the shallow regime.

Figures & tables

		$L=1$	$L=2$	$L=3$	$L=4$	$L=5$
$r=2$	$\lambda_{L,r}$	$3/4$	$1/4$	$3/28$	$1/20$	$3/124$
	Exp	0.001s	0.004s	0.031s	2.09s	153.7s
	Alg. 1	0.045s	0.059s	0.271s	3.17s	66.4s
	Total	0.047s	0.063s	0.302s	5.26s	220.2s
	$\hat{\lambda}^{\text{SGLD}}$	.677 (.072)	.154 (.095)	.038 (.068)	.014 (.035)	.006 (.017)
	Time (SGLD)	461s	430s	421s	473s	581s

Table 1: Local RLCT at the origin of

H_{L,r}(\theta)

: exact rationals

\lambda_{L,r}

from Algorithm 1 ( SageMath , Intel Core Ultra 7) versus SGLD estimates

\hat{\lambda}^{\text{SGLD}}

(absolute error in parentheses;

\gamma\!=\!1

\epsilon\!=\!10^{-5}

C\!=\!5

chains,

T\!=\!10{,}000

; Google Colab, 2 vCPUs). Exp is the time to expand

H_{L,r}(\theta)

; Alg. 1 is Algorithm 1 alone. NaN marks numerical instability;

>

1 hr marks runs that did not finish for Algorithm 1 .

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 1: Newton polygon $\mathcal{N}(f)$ of $f(x,y)=y^{5}+y^{2}x+yx^{3}+x^{6}+y^{2}x^{3}+y^{3}x^{2}$ , with $K=3$ compact faces.

Figure 2: Newton polygon $\mathcal{N}(f)$ of $f(x,y)=y^{5}+y^{2}x+yx^{3}+x^{6}+y^{2}x^{3}+y^{3}x^{2}$ , with $K=3$ compact faces. The main face of $\mathcal{N}(f)$ is $\Delta_{2}$ , and the Newton distance of $f$ is $\delta_{f}=\frac{5}{3}$ .

Singular Learning Theory

Singular Learning Theory studies how non-identifiability and geometric singularities shape learning, Bayesian inference, and generalization.

See all 16 papers

Upper Bounds for Local Learning Coefficients of Three-Layer Neural Networks
Mar 13, 2026Yuki KurumadaniTwo-Layer Neural Networks Singular Learning Theory
The University of Osaka, 1 Chome-3 Machikaneyamacho, Toyonaka, Osaka 560-8531, Japan
Symmetries and Singularities
Sep 13, 2026Vishnu Varadarajan, Mihir More, Aritra Das +1Singular Learning Theory Graph Attention Networks
Truth Audit Labs
Thermodynamic Response Functions in Singular Bayesian Models
Mar 5, 2026Sean PlummerSingular Learning Theory Statistical Physics of Learning

Mar 13, 2026J/K move · Enter open · S save

Upper Bounds for Local Learning Coefficients of Three-Layer Neural Networks

Yuki Kurumadani

The University of Osaka, 1 Chome-3 Machikaneyamacho, Toyonaka, Osaka 560-8531, Japan

Two-Layer Neural Networks Singular Learning Theory Feedforward Neural Networks

Open paper AskPDF arXiv

Three-layer neural networks are known to form singular learning models, and their Bayesian asymptotic behavior is governed by the learning coefficient, or real log canonical threshold. Although this quantity has been clarified for regular models and for some special singular models, broadly applicable methods for evaluating it in neural networks remain limited. Recently, a formula for the local learning coefficient of semiregular models was proposed, yielding an upper bound on the learning coefficient. However, this formula applies only to nonsingular points in the set of realization parameters and cannot be used at singular points. In particular, for three-layer neural networks, the resulting upper bound has been shown to differ substantially from learning coefficient values already known in some cases. In this paper, we derive a formula for an upper bound on local learning coefficients at a class of singular realization parameters in three-layer neural networks. This formula can be interpreted as a counting rule under budget, demand, and supply constraints. In the non-polynomial real-analytic case, the formula applies in general settings, whereas in the polynomial case it applies under the restriction that the true distribution has no hidden units. In particular, our result covers activation functions such as the swish function and also includes polynomial activation functions under the above restriction, thereby extending previous results to a broader class of activation functions. We further show that, when the input dimension is one, the numerical value given by the right-hand side of our upper-bound formula agrees with the previously known learning coefficient, thereby providing a useful comparison with known exact results. Our result also provides a systematic perspective on how the weight parameters of three-layer neural networks affect the learning coefficient.

Abstract

Singular Learning Theory

Explore similar work

Upper Bounds for Local Learning Coefficients of Three-Layer Neural Networks

Symmetries and Singularities

Thermodynamic Response Functions in Singular Bayesian Models