Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning

Ziran Liu

cs.LGMar 18, 2026

Variational Kernel Design for Internal Noise: Gaussian Chaos Noise, Representation Compatibility, and Reliable Deep Learning

Authors: Ziran Liu

Organizations: Shanghai Institute for Mathematics and Interdisciplinary Sciences (SIMIS), Shanghai 200433, China · Research Institute of Intelligent Complex Systems, Fudan University, Shanghai 200433, China

Abstract

Internal noise in deep networks is usually inherited from heuristics such as dropout, hard masking, or additive perturbation. We ask two questions: what correlation geometry should internal noise have, and is the implemented perturbation compatible with the representations it acts on? We answer these questions through Variational Kernel Design (VKD), a framework in which a noise mechanism is specified by a law family, a correlation kernel, and an injection operator, and is derived from learning desiderata. In a solved spatial subfamily, a quadratic maximum-entropy principle over latent log-fields yields a Gaussian optimizer with precision given by the Dirichlet Laplacian, so the induced geometry is the Dirichlet Green kernel. Wick normalization then gives a canonical positive mean-one gate, Gaussian Chaos Noise (GCh). For the sample-wise gate used in practice, we prove exact Gaussian control of pairwise log-ratio deformation, margin-sensitive ranking stability, and an exact expected intrinsic roughness budget; hard binary masks instead induce singular or coherence-amplified distortions on positive coherent representations. On ImageNet and ImageNet-C, GCh consistently improves calibration and under shift also improves NLL at competitive accuracy.

Figures & tables

Method	$g$	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
None	0	0.765 $\pm$ 0.001	0.931 $\pm$ 0.004	0.030 $\pm$ 0.001
Dropout	0.1	0.764 $\pm$ 0.001	0.942 $\pm$ 0.005	0.033 $\pm$ 0.001
DropBlock	0.1	0.765 $\pm$ 0.000	0.930 $\pm$ 0.002	0.032 $\pm$ 0.000
IID Gauss.	0.1	0.765 $\pm$ 0.001	0.930 $\pm$ 0.005	0.032 $\pm$ 0.002
Cor. Gauss.	0.1	0.765 $\pm$ 0.000	0.944 $\pm$ 0.002	0.037 $\pm$ 0.001
GCh (ours)	0.1	0.764 $\pm$ 0.001	0.934 $\pm$ 0.004	0.020 $\pm$ 0.001

Table 1: ImageNet val (uncorrupted) under late-stage injection (layer4). Mean

\pm

std over 3 seeds. Here

g

denotes the method-specific strength knob:

g=\gamma

for GCh,

g=\sigma

for Gaussian baselines, and

g=p

for Dropout/DropBlock.

Method	Top-1 Acc. $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
Baseline (None)	80.03%	0.9213	0.0762
GCh (ours)	80.11%	0.9131	0.0738

Table 2: Swin-T (best checkpoint). Full-recipe training, single run.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Method	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
None	76.41	0.96	0.082
DropBlock	75.86	0.99	0.085
GCh (ours)	76.23	0.95	0.076

Appendix

Table 3: ImageNet (clean), best checkpoint, L4 injection.

Method	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
None	76.35	0.97	0.084
DropBlock	75.21	1.04	0.091
GCh (ours)	76.18	0.96	0.078

Appendix

Table 4: ImageNet (clean), latest checkpoint / final epoch, L4 injection.

Stage	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
L2-early	0.767 $\pm$ 0.001	0.918 $\pm$ 0.003	0.031 $\pm$ 0.001
L3-mid	0.765 $\pm$ 0.001	0.925 $\pm$ 0.006	0.029 $\pm$ 0.002
L4-late	0.764 $\pm$ 0.001	0.934 $\pm$ 0.004	0.020 $\pm$ 0.001

Appendix

Table 5: Injection depth ablation for our method at fixed strength

\gamma=0.1

(3 seeds). Mean

\pm

std.

$\gamma$	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
0.03	0.766 $\pm$ 0.002	0.926 $\pm$ 0.006	0.027 $\pm$ 0.001
0.07	0.765 $\pm$ 0.002	0.928 $\pm$ 0.009	0.021 $\pm$ 0.001
0.1	0.764 $\pm$ 0.001	0.934 $\pm$ 0.004	0.020 $\pm$ 0.001
0.18	0.759 $\pm$ 0.001	1.005 $\pm$ 0.006	0.076 $\pm$ 0.002
0.27	0.667 $\pm$ 0.034	1.880 $\pm$ 0.201	0.316 $\pm$ 0.017
0.35	0.164 $\pm$ 0.017	5.204 $\pm$ 0.119	0.149 $\pm$ 0.017

Appendix

Table 6:

\gamma

sweep at late-stage injection (each

\gamma

retrained). Mean

\pm

std over completed seeds (

n=3

for all shown).

Method	$g$	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
None	0	0.382 $\pm$ 0.003	3.400 $\pm$ 0.030	0.105 $\pm$ 0.002
Dropout	0.1	0.384 $\pm$ 0.003	3.317 $\pm$ 0.020	0.084 $\pm$ 0.001
DropBlock	0.1	0.390 $\pm$ 0.009	3.300 $\pm$ 0.100	0.093 $\pm$ 0.004
IID Gaussian	0.1	0.388 $\pm$ 0.003	3.316 $\pm$ 0.044	0.096 $\pm$ 0.006
Corr. Gaussian	0.1	0.386 $\pm$ 0.002	3.340 $\pm$ 0.028	0.103 $\pm$ 0.010
GCh (ours)	0.1	0.383 $\pm$ 0.005	3.287 $\pm$ 0.064	0.056 $\pm$ 0.005

Appendix

Table 7: ImageNet-C overall (mean over 7 corruptions

\times

5 severities) for late-stage injection. Mean

\pm

std over 3 seeds.

Stage	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
early	0.390 $\pm$ 0.002	3.314 $\pm$ 0.018	0.096 $\pm$ 0.003
mid	0.393 $\pm$ 0.003	3.230 $\pm$ 0.037	0.088 $\pm$ 0.004
late	0.383 $\pm$ 0.005	3.287 $\pm$ 0.064	0.056 $\pm$ 0.005

Appendix

Table 8: Stage-wise ablation on ImageNet-C for GCh (ours) with

g=0.1

. Mean

\pm

std over 3 seeds.

$g$	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
0.03	0.388 $\pm$ 0.001	3.317 $\pm$ 0.045	0.091 $\pm$ 0.007
0.07	0.388 $\pm$ 0.007	3.304 $\pm$ 0.032	0.075 $\pm$ 0.006
0.1	0.383 $\pm$ 0.005	3.287 $\pm$ 0.064	0.056 $\pm$ 0.005
0.18	0.385 $\pm$ 0.004	3.277 $\pm$ 0.048	0.073 $\pm$ 0.001
0.27	0.276 $\pm$ 0.038	4.266 $\pm$ 0.228	0.169 $\pm$ 0.018
0.35	0.050 $\pm$ 0.003	6.187 $\pm$ 0.030	0.043 $\pm$ 0.004

Appendix

Table 9: Strength sweep on ImageNet-C for GCh (late-stage injection). Mean

\pm

std over 3 seeds.

Corruption	Acc (None)	Acc (Ours)	ECE (None)	ECE (Ours)
defocus_blur	0.402 $\pm$ 0.003	0.398 $\pm$ 0.003	0.038 $\pm$ 0.002	0.039 $\pm$ 0.002
gaussian_noise	0.308 $\pm$ 0.004	0.310 $\pm$ 0.012	0.156 $\pm$ 0.011	0.076 $\pm$ 0.011
glass_blur	0.273 $\pm$ 0.002	0.263 $\pm$ 0.004	0.122 $\pm$ 0.003	0.075 $\pm$ 0.002
jpeg_compression	0.547 $\pm$ 0.002	0.550 $\pm$ 0.008	0.059 $\pm$ 0.004	0.026 $\pm$ 0.001
motion_blur	0.396 $\pm$ 0.006	0.400 $\pm$ 0.004	0.089 $\pm$ 0.006	0.049 $\pm$ 0.004
pixelate	0.462 $\pm$ 0.011	0.467 $\pm$ 0.006	0.096 $\pm$ 0.004	0.047 $\pm$ 0.008

Appendix

Table 10: ImageNet-C corruption-wise breakdown (severity-averaged) comparing None vs GCh (ours) at late-stage

g=0.1

. Mean

\pm

std over 3 seeds.

Method	$g$	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
None	0	0.9009 $\pm$ 0.0044	0.3669 $\pm$ 0.0016	0.0325 $\pm$ 0.0044
Dropout ( $p{=}0.1$ )	0.1	0.8957 $\pm$ 0.0007	0.4246 $\pm$ 0.0131	0.0503 $\pm$ 0.0007
DropBlock ( $p{=}0.1$ )	0.1	0.9002 $\pm$ 0.0027	0.3669 $\pm$ 0.0007	0.0317 $\pm$ 0.0053
GCh (ours)	0.1	0.9010 $\pm$ 0.0023	0.3627 $\pm$ 0.0039	0.0302 $\pm$ 0.0037

Appendix

Table 11: Oxford-IIIT Pets test performance (ResNet-18, 224

\times

224, late-stage injection; mean

\pm

std over 3 seeds). The strength parameter

g

is shared across rows for compactness; for Dropout/DropBlock it corresponds to the drop probability

p

(see text).

$g$	Top-1 $\uparrow$	NLL $\downarrow$	ECE $\downarrow$
0.1	0.9010 $\pm$ 0.0023	0.3627 $\pm$ 0.0039	0.0302 $\pm$ 0.0037
0.5	0.8989 $\pm$ 0.0031	0.3660 $\pm$ 0.0038	0.0314 $\pm$ 0.0053
1.0	0.8978 $\pm$ 0.0030	0.3661 $\pm$ 0.0024	0.0323 $\pm$ 0.0037

Appendix

Table 12: GCh strength sweep on Oxford-IIIT Pets (test; mean

\pm

std over 3 seeds). As in ImageNet/ImageNet-C, moderate strengths are best; larger strengths do not yield further gains.

Kernel Methods

Kernel methods use similarity functions and reproducing kernel spaces to model nonlinear relationships, quantify uncertainty, and compare data.

See all 223 papers

Variation Brownian Kernel Ladders
Aug 14, 2026Mahdi MohammadigohariKernel Methods Neural Network Approximation Theory
Faculty of Engineering, Free University of Bozen-Bolzano Bruno Buozzi 1, Bolzano, 39100, Italy
Permutation-preserving Functions and Neural Vecchia Covariance Kernels
May 6, 2026Jian Cao, Nian Liu, Ying LinGaussian Processes Kernel Methods
Department of Mathematics University of Houston Houston, TX 77047 · Department of Industrial and Systems Engineering University of Houston Houston, TX 77047
Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression
Aug 18, 2026Tao Jiang, Minbo Gao, Shaowei CaiReLU Neural Networks Minimax Estimation
Key Laboratory of System Software (Chinese Academy of Sciences) · State Key Laboratory of Computer Science · Institute of Software, Chinese Academy of Sciences +1

Aug 14, 2026J/K move · Enter open · S save

Variation Brownian Kernel Ladders

Mahdi Mohammadigohari

Faculty of Engineering, Free University of Bozen-Bolzano Bruno Buozzi 1, Bolzano, 39100, Italy

Kernel Methods Neural Network Approximation Theory Reproducing Kernel Hilbert Spaces Statistical Learning Theory

Open paper AskPDF arXiv

Claims about the benefit of depth depend on the complexity assigned to a representation. We introduce the Variation Brownian Kernel Ladder (VBKL), a path-atomic function-space framework that separates nonlinear recursive dictionary construction from linear variation superposition. Starting from linear projections, each atom recursively composes unit-ball profiles from the Brownian reproducing kernel Hilbert space; the full VBKL space is then the signed-measure variation hull of the completed dictionary. We identify each recursive dictionary as a union of Brownian pullback RKHS balls and establish variation-controlled Hölder regularity, compactness and attainment, and strict growth with depth under a local non-degeneracy condition whose trace lies in the support of the input measure. For associated finite lower-support architectures, we derive Rademacher and generalization bounds through Brownian quadratic chaos, signed threshold traces, and VC entropy. We also construct two-stage approximants by discretizing the outer measure and the selected outer Brownian profiles, obtaining an

M^{-1/2}+m^{-1/2}

error bound, a sharp interpolation constant

\sqrt{A/2}

, and at most

2M

active outer-profile basis contributions per evaluation. Controlled experiments illustrate the approximation mechanisms and indicate a favorable limited-data accuracy--complexity trade-off.

Abstract

Kernel Methods

Explore similar work

Variation Brownian Kernel Ladders

Permutation-preserving Functions and Neural Vecchia Covariance Kernels

Depth Enables Local Entropy: Quadratic Depth Dependence in Deep Variation-Norm ReLU Regression