cs.LGApr 17, 2026

Collective Kernel EFT for Pre-activation ResNets

Authors: Hidetoshi KawaseToshihiro Ota

Organizations: CyberAgent, Inc., Shibuya, Tokyo 150–6121 Japan

Abstract

In finite-width deep neural networks, the empirical kernel GG evolves stochastically across layers. We develop a collective kernel effective field theory (EFT) for pre-activation ResNets based on a GG-only closure hierarchy and diagnose its finite validity window. Exploiting the exact conditional Gaussianity of residual increments, we derive an exact stochastic recursion for GG. Applying Gaussian approximations systematically yields a continuous-depth ODE system for the mean kernel K0K_0, the kernel covariance V4V_4, and the 1/n1/n mean correction K1,EFTK_{1,\mathrm{EFT}}, which emerges diagrammatically as a one-loop tadpole correction. Numerically, K0K_0 remains accurate at all depths. However, the V4V_4 equation residual accumulates to an O(1)O(1) error at finite time, primarily driven by approximation errors in the GG-only transport term. Furthermore, K1,EFTK_{1,\mathrm{EFT}} fails due to the breakdown of the source closure, which exhibits a systematic mismatch even at initialization. These findings highlight the limitations of GG-only state-space reduction and suggest extending the state space to incorporate the sigma-kernel.

Explore similar work

CardsList