Variation Brownian Kernel Ladders

Mahdi Mohammadigohari

cs.LGAug 14, 2026

Variation Brownian Kernel Ladders

Authors: Mahdi Mohammadigohari

Abstract

Claims about the benefit of depth depend on the complexity assigned to a representation. We introduce the \emph{Variation Brownian Kernel Ladder} (VBKL), a path-atomic function-space framework that separates nonlinear recursive dictionary construction from linear variation superposition. Starting from linear projections, each atom recursively composes unit-ball profiles from the Brownian reproducing kernel Hilbert space; the full VBKL space is then the signed-measure variation hull of the completed dictionary. We identify each recursive dictionary as a union of Brownian pullback RKHS balls and establish variation-controlled Hölder regularity, compactness and attainment, and strict growth with depth under a local non-degeneracy condition whose trace lies in the support of the input measure. For associated finite lower-support architectures, we derive Rademacher and generalization bounds through Brownian quadratic chaos, signed threshold traces, and VC entropy. We also construct two-stage approximants by discretizing the outer measure and the selected outer Brownian profiles, obtaining an $M^{-1/2}+m^{-1/2}$ error bound, a sharp interpolation constant $\sqrt{A/2}$ , and at most $2M$ active outer-profile basis contributions per evaluation. Controlled experiments illustrate the approximation mechanisms and indicate a favorable limited-data accuracy--complexity trade-off.

Figures & tables

Result	Content	Page
Theorem 4	Analytical properties of the VBKL spaces	page 4
Theorem 6	Architecture-dependent Rademacher bound	page 6
Corollary 8	Finite-architecture generalization guarantee	page 8
Theorem 9	Finite-atomic approximation of the full VBKL space	page 9
Theorem 10	Two-stage finite-atomic and outer-profile approximation	page 10
Proposition 1	Brownian profile interpolation	page 1

Table 1: Overview of the main theoretical results. Their logical dependencies are illustrated in Fig. 5 ; supporting results are summarized in Table 9 .

Figure 1: Brownian profile interpolation. (a) Interpolation errors for three representative unit-norm Brownian RKHS profiles. (b) Interpolation error of the normalized tent profile together with the theoretical interpolation bound. The coincidence of the curves demonstrates sharpness of the interpolation estimate.

Figure 2: Two-stage constructive approximation. (a) Finite signed-measure discretization with exact profiles. (b) Brownian-profile discretization with a fixed finite measure representation, together with the worst-case upper bound. (c) Balanced refinement

M=m=N

. The reference slopes correspond to the rates in Theorems 9 and 12 .

Panel of Figure 3 — Figure 3: Statistical learning and representation efficiency. Test mean-squared error is averaged over five independent random seeds. VBKL complexity is selected independently by validation for each training-set size and seed. Panel (c) uses logarithmic axes; small markers represent individual seeds and outlined markers represent means.

$n$	VBKL wins vs. DNVS	Parameter ratio
100	4/5	4.6 $\times$
250	2/5	13.7 $\times$
500	2/5	18.3 $\times$

Table 2: Seed-wise and parameter-efficiency comparison between VBKL and DNVS. The parameter ratio is DNVS/VBKL.

Figure 4: Optimization validation of finite VBKL realizations. (a) Relative error between automatic-differentiation directional derivatives and the averaged directional estimator across interaction scales. (b) Estimator standard deviation under Monte Carlo directional averaging. Increasing the number of sampled directions reduces the variability of the estimate.

Figure 5: Proof-dependency structure of the theoretical results. The displayed graph is the exact supplied dependency graph; every panel title, result node, and experiment node is clickable and jumps to the corresponding statement, section, or protocol.

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

$n_{\mathrm{train}}$	Paths by seed	Median paths	Epochs by seed	Median epochs
100	32, 8, 2, 8, 16	8	100, 190, 600, 430, 30	190
250	2, 64, 64, 32, 8	32	450, 70, 80, 190, 110	110
500	64, 16, 16, 64, 32	32	140, 280, 90, 60, 110	110
1,000	16, 64, 16, 4, 2	16	170, 70, 150, 380, 380	170
2,000	4, 8, 8, 64, 32	8	1,990, 990, 440, 90, 110	440
5,000	32, 4, 64, 16, 64	32	170, 510, 90, 570, 80	170

Appendix

Table 3: Validation-selected VBKL configurations. Path counts and training epochs are listed in seed order. The architecture and optimization horizon are selected independently for each training size and random seed using the validation set.

$n$	VBKL	DNVS	KRR	RBF
100	0.051656 (0.025140)	0.092731 (0.013174)	0.059793 (0.031508)	0.050172 (0.022180)
250	0.007220 (0.004147)	0.005756 (0.001281)	0.005158 (0.001125)	0.006908 (0.001264)
500	0.004793 (0.003983)	0.002442 (0.000232)	0.002627 (0.000309)	0.002636 (0.000276)

Appendix

Table 4: Test MSE on the Energy Efficiency benchmark. Values are means, with standard deviations in parentheses, over five random seeds.

Model	Training time (s)	Inference time (ms)	Parameters
VBKL	2.797 $\pm$ 2.189	13.104 $\pm$ 11.204	3,248
DNVS	1.338 $\pm$ 0.232	0.882 $\pm$ 0.523	8,960
KRR	0.103 $\pm$ 0.180	48.811 $\pm$ 60.318	750
RBF	0.007 $\pm$ 0.009	8.413 $\pm$ 4.517	0

Appendix

Table 5: Computational characteristics on the recursive-learning benchmark. Training and inference times are reported as mean

\pm

standard deviation over all training sizes and seeds. Parameter counts are reported by their median because the validation-selected VBKL architecture may vary across runs.

Model	Training samples
	$100$	$250$	$500$	$1000$	$2000$	$5000$
VBKL	1.086 (0.0698)	1.912 (0.0308)	2.551 (0.0543)	1.471 (0.0092)	3.878 (0.0442)	5.882 (0.0064)
DNVS	1.123 (0.0057)	1.181 (0.0096)	1.353 (0.0142)	1.413 (0.0271)	1.435 (0.0364)	1.523 (0.0003)
KRR	0.015 (0.0023)	0.005 (0)	0.007 (0.0001)	0.018 (0.0005)	0.078 (0.0034)	0.492 (0.0002)
RBF	0.002 (0.0001)	0.002 (0)	0.003 (0.0001)	0.005 (0.0005)	0.009 (0.0007)	0.020 (0.0013)

Appendix

Table 6: Training time in seconds on the recursive stochastic teacher benchmark. Lower values are better.

Model	Training samples
	$100$	$250$	$500$	$1000$	$2000$	$5000$
VBKL	0.0069 (0.0062)	0.0161 (0.0137)	0.0183 (0.0111)	0.0096 (0.0117)	0.0110 (0.0117)	0.0167 (0.0125)
DNVS	0.0010 (0.0004)	0.0011 (0.0004)	0.0013 (0.0005)	0.0011 (0.0005)	0.0005 (0.0004)	0.0003 (0)
KRR	0.0028 (0.0003)	0.0062 (0.0002)	0.0126 (0.0003)	0.0314 (0.0068)	0.0688 (0.0104)	0.1711 (0.0101)
RBF	0.0073 (0.0035)	0.0108 (0.0038)	0.0070 (0.0044)	0.0072 (0.0048)	0.0085 (0.0054)	0.0097 (0.0059)

Appendix

Table 7: Inference time in seconds for the complete held-out test set. Lower values are better.

Model	Training samples
	$100$	$250$	$500$	$1000$	$2000$	$5000$
VBKL	2,680 (2,360)	6,902 (6,008)	7,795 (4,926)	4,141 (5,123)	4,710 (5,148)	7,308 (5,567)
DNVS	12,723 (12,391)	14,029 (11,334)	24,166 (13,881)	21,555 (17,457)	8,806 (14,254)	2,432
KRR	100	250	500	1,000	2,000	5,000
RBF	0	0	0	0	0	0

Appendix

Table 8: Number of trainable or fitted parameters used by each method. Lower values indicate a more compact representation.

Result	Content	Page
Lemma 15	Well-definedness of the VBKL space	page 15
Lemma 16	Non-degeneracy of the variation complexity	page 16
Lemma 17	Finite-profile characterization and finite-architecture bounds	page 17
Lemma 18	Recursive union-of-RKHS characterization	page 18
Lemma 19	Exact reduction from finite variation hulls to outer dictionaries	page 19
Lemma 20	Rademacher bound for unions of Brownian pullback RKHS balls	page 20

Appendix

Table 9: Overview of the supporting theoretical results. Their logical dependencies are illustrated in Fig. 5 ; the main theoretical results are summarized in Table 1 .

Explore similar work

Brownian Kernel Ladders
Jun 14, 2026Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia +1Hierarchical Representation Learning Reproducing Kernel Hilbert Spaces
Faculty of Engineering, Free University of Bozen-Bolzano, Via Bruno Buozzi 1, Bolzano, 39100, Italy · Department of Biomedical and Biotechnological Sciences, University of Catania, Catania, 95123, Santa Sofia, 97, Italy · Center for Applied Optimization, Department of Industrial and Systems Engineering, University of Florida, FL 32611, Gainesville, USA
Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression
Aug 18, 2026Tao Jiang, Minbo Gao, Shaowei CaiReLU Neural Networks Minimax Estimation
Brownian Heads for Deep ReLU Representations: Activation Mass and the Cost of Same-Sample Selection
Sep 18, 2026Mahdi Mohammadigohari, Nicole MückeReLU Neural Networks Statistical Learning Theory
Faculty of Engineering Free University of Bozen–Bolzano 39100 Bolzano, Italy · Institute for Statistics and Foundations of Machine Learning Technische Universität Braunschweig Universitätsplatz 2 38106 Braunschweig, Germany

Jun 14, 2026, cs.LGJ/K move · Enter open · S save

Brownian Kernel Ladders

Mahdi Mohammadigohari, Giuseppe Di Fatta, Giuseppe Nicosia, Panos M Pardalos

Faculty of Engineering, Free University of Bozen-Bolzano, Via Bruno Buozzi 1, Bolzano, 39100, Italy · Department of Biomedical and Biotechnological Sciences, University of Catania, Catania, 95123, Santa Sofia, 97, Italy · Center for Applied Optimization, Department of Industrial and Systems Engineering, University of Florida, FL 32611, Gainesville, USA

Hierarchical Representation Learning Reproducing Kernel Hilbert Spaces Kernel Methods Statistical Learning Theory Generalization Bounds

Open paper AskPDF arXiv

Constructing mathematically tractable function spaces that capture hierarchical compositional representations remains a central challenge in statistical learning theory. We introduce Brownian kernel ladders (BKLs), a recursively defined hierarchy of integral reproducing kernel Hilbert spaces generated through Brownian-kernel integral constructions. Starting from linear functionals, each layer is obtained by integrating Brownian kernels over probability measures supported on subsets of the previous layer, yielding a recursive function-space model in which depth is encoded directly through the hierarchy. Based on this framework, we define canonical BKL spaces together with an associated complexity functional. We establish several analytical and statistical properties of these spaces. In particular, we show that BKL spaces form quasi-Banach spaces, satisfy depth-dependent Hölder regularity estimates, and exhibit strict monotonicity with respect to depth. We further prove existence results for regularized empirical risk minimization and derive Gaussian complexity bounds that remain uniformly controlled with respect to both the ambient dimension and the hierarchy depth. A key ingredient of the analysis is a combinatorial proof technique based on recursive subset decompositions and Brownian-kernel threshold representations. These estimates yield excess-risk guarantees of near-parametric order for regularized empirical risk minimization over BKL spaces. Our results provide a mathematically tractable hierarchical function-space framework for studying compositional representations in deep learning.

Abstract

Explore similar work

Brownian Kernel Ladders

Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression

Brownian Heads for Deep ReLU Representations: Activation Mass and the Cost of Same-Sample Selection