cs.LGAug 20, 2026

Kähler landscapes for complex neural network descents and guarantees including a search and destroy of the Calabi-Yau manifold

Authors: Andrew Gracyk

Abstract

We study landscapes for complex-parameterized networks. Our approach is motivated with an information-theoretic manifold perspective of the parameter and via classical optimization guarantees although of complex geometric variety such as through Dolbeault asymptotics. The descent path admits a Kähler information metric under a cross-entropy via the Wirtinger Hessian on the log-likelihood potential. We restrict attention to a descent update rule with natural gradient descent via a differentiated loss scaled by the inverse metric, so the descent path remains in the holomorphic tangent bundle. We emphasize Calabi-Yau information manifolds which profane theoretical guarantees via an ill-curvature-conditioned landscape. Under a Calabi-Yau metric, specifically in a non-compact setting with a global potential so defined geometrically rather than invoking the topological requirements of the Calabi conjecture, a wedged nowhere-vanishing holomorphic form is the top exterior product of the Kähler form up to constants, yielding a constant determinant condition with respect to a background metric and ill-conditioned eigenvalues under nonuniform and almost low-rank assumptions. Moreover, it has been discovered that negative curvature subverts the loss landscape, specifically sectional curvature, so we expand on this and draw interconnections to negative-definite Ricci curvature. Our arguments primarily exist in a geometric analytic modality, although we establish roots in deep learning theory such as through asymptotics at initialization and connections through failure modes of neural network guarantees under vanishing and negative Ricci curvature.

Explore similar work

CardsList
Apr 20, 2026math.DG

Complex normalizing flows can almost be information Kähler-Ricci flows

We develop interconnections between the complex normalizing flow for data drawn from Borel probability measures on the twofold realification of the complex manifold and a nonlinear flow nearly Kähler-Ricci. The complex normalizing flow relates the initial and target realified densities under the complex change of variables, necessitating the log determinant of the ensemble of Wirtinger Jacobians. The Ricci curvature of a Kähler manifold is the second order mixed Wirtinger partial derivative of the log of the local density of the volume form. Therefore, we reconcile these two facts by drawing forth the connection that the log determinant used in the complex normalizing flow matches a Ricci curvature term under differentiation and conditions. The log density under the normalizing flow is kindred to a spatial information metric under an augmented Jacobian and a Bayesian perspective to the parameter, thus under the continuum limit the log likelihood matches a Fisher metric, or more closely a Kähler cross-entropy Hessian. This recovers a Kähler-Ricci flow variation up to a time derivative and expectation, or an average-valued Kähler-Einstein flow. Using this framework, we establish other relevant results, attempting to bridge the statistical and ordinary behaviors of the complex normalizing flow to the geometric features of our derived Kähler flow.
Andrew Gracyk
Normalizing FlowsRiemannian ManifoldsCondition Number
AskPDF
Jun 27, 2026cs.LG

Closed-Form Steepest Descent Direction toward Flat Minima: Reducing Upper Bounds on the Loss Hessian Eigenspectrum in Neural Networks

The flatness hypothesis suggests that flatness of the loss landscape, as measured by the eigenvalues of the loss Hessian, correlates with better neural network generalization. While various algorithms reduce these eigenvalues, most focus on procedural design, leaving it unclear how data distributions and NN parameters structurally determine directions toward flat minima. Characterizing these directions analytically is generally intractable. To overcome this mathematical difficulty, recent studies derived the Wolkowicz-Styan (WS) upper bound on the maximum eigenvalue of the cross-entropy loss Hessian in three-layer NNs. Although this upper bound is differentiable, its gradient was not derived. Therefore, we analytically derive the gradient of the WS upper bound to characterize directions leading to flat minima. Based on this, we propose Hessian Spectral Range (HSR) Regularization, which updates parameters along the steepest descent direction of the WS bound. Experiments demonstrate that HSR Regularization narrows the Hessian eigenvalue spectrum, avoids sharp minima and saddle points, and promotes convergence to flat minima. Although the applicability of this method is currently limited to cross-entropy loss and three-layer architectures, to the best of the authors' knowledge, this is the first study to report a closed-form gradient that promotes convergence to flat minima without numerical approximations. Therefore, the theoretical analysis of this gradient is expected to contribute to the further development of NNs.
Yuto Omae, Kazuki Sakai, Yohei Kakimoto +3
Condition NumberGradient DescentTwo-Layer Neural NetworksFlat MinimaNeural NetworkLargest Eigenvalue
AskPDF
Apr 16, 2026cs.LG

Curvature-Aligned Probing for Local Loss-Landscape Stabilization

Local loss-landscape stabilization under sample growth is typically measured either pointwise or through isotropic averaging in the full parameter space. Despite practical value, both choices probe directions that contribute little to the dominant local deformation of strongly anisotropic neural landscapes. We recast stabilization as an observational problem and introduce a unified family of criteria parameterized by an aggregation order and a probing distribution; within this family we propose a curvature-aligned criterion Δ2(D)Δ_2^{(D)} that probes the loss increment field in the top-DD eigenspace of the empirical Hessian near a trained solution. Solely from a local quadratic model, we prove that Δ2(D)Δ_2^{(D)} preserves the O(k−2)O(k^{-2}) mean-squared rate of the full-space criterion while replacing ambient-dimension curvature dependence with dependence on the subspace dimension DD; a corollary gives a closed-form spectral expression and a proposition identifies the top-DD eigenspace as extremal within the eigenspace-aligned family. We also derive scalable estimators based on Hessian-vector products, subspace Monte Carlo, and a closed-form Gaussian-moment proxy. On a decoder-only transformer, a curvature-aligned probe occupying a tiny fraction of parameter space already reproduces the full-space mean-squared signal to within numerical noise throughout the validated local regime, and the closed-form estimator is orders of magnitude faster than direct Monte Carlo after subspace construction.
Nikita Kiselev, Andrey Grabovoy
Curvature-Aware Quadratic BanksLoss LandscapeMean Squared ErrorFederated Averaging
AskPDF