Loss-Parameterized Fisher Width Along Learning Trajectories

Vu Khac Ky

cs.LGAug 21, 2026

Loss-Parameterized Fisher Width Along Learning Trajectories

Authors: Vu Khac Ky

Organizations: Department of Mathematics, FPT University, Vietnam

Abstract

Fisher width measures the Gaussian width of a probe set after deformation by the local Fisher geometry. We study its evolution along learning trajectories and ask when training loss can serve as an effective coordinate for this quantity. We first derive an exact trace--shape factorization and a deterministic stability bound for fixed compact probes. In a population Gaussian-teacher logistic model, the teacher-aligned state is extremal on every loss level below $\log 2$ : it has minimal parameter norm and maximizes both Fisher trace and Euclidean-ball Fisher width. We then show that population gradient flow asymptotically selects this branch, with explicit rates for the aligned and orthogonal coordinates. This yields, for $d\geq2$ ,

\frac{w_F(B_2^d;θ(t))} {\sqrt{L(θ(t))}} \longrightarrow \frac{\sqrt6}π\mathbb E[χ_{d-1}].

Controlled full-Fisher experiments support the matched-loss branch and the population predictions. In a nonlinear MLP with a diagonal model-Fisher approximation, GD and SGD remain close at matched loss, whereas Adam follows a substantially displaced branch; the fixed probes tested retain highly similar temporal shapes. These results support a branchwise, rather than universal, loss parametrization of Fisher width.

Figures & tables

Optimizer	$\overline{\rho}$	Observation
GD	$1.000$	reference
NGD	$1.009\pm 0.002$	close agreement
Noisy GD	$0.998\pm 0.001$	close agreement
Adam	$0.995\pm 0.003$	close agreement

Table 1: Matched-loss full-Fisher-width ratios in the controlled logistic experiment. Each optimizer is compared with GD only over their shared observed loss range. Values are mean

\pm

sample standard deviation across six initializations.

Figure 1: Controlled logistic experiment for one predetermined initialization. Left: full model-Fisher width along training. Right: the same trajectories reparametrized by training loss. The matched-loss curves are much more closely aligned than their iteration-time trajectories. Numerical summaries over all six initializations are reported in Table 1 .

Probe	$\widehat{a}_{T}$	$\widehat{\chi}_{T}$	$\max_{t}\widehat{\Delta}_{T}(t)$	$\max_{t}\widehat{\chi}_{T}\widehat{\varepsilon}_{t}$
$B_{2}^{d}$	$0.992$	$5.476$	$0.0007$	$0.294$
Random $8$ -dimensional subspace	$0.493$	$11.028$	$0.0032$	$0.593$
$5$ -sparse unit probe	$0.748$	$7.264$	$0.0017$	$0.391$
Normalized ellipsoid	$0.476$	$11.401$	$0.0132$	$0.613$

Table 2: Trace–shape diagnostics for one GD trajectory in the controlled logistic experiment. The quantities

\widehat{a}_{T}

and

\widehat{\Delta}_{T}

are estimated with

1200

shared Gaussian draws. The last column is the corresponding empirical benchmark from the probe-stability bound.

Diagnostic	Asymptotic prediction	Value at $t=5000$
$R_{\alpha}(t)$	$\displaystyle\frac{\pi^{2}}{2\sqrt{2\pi}}\approx 1.9687$	$1.9273$
$R_{\beta}(t)$	$\displaystyle\frac{3}{\pi^{2}}\approx 0.30396$	$0.31072$
$R_{\mathrm{EoS}}(t)$	$1$	$0.999587$

Table 3: Population diagnostics for dynamical selection and the late-loss Fisher-width law. The predictions are fixed by Theorem 4.10 and Corollary 4.13 ; the final column reports the numerical integration at

t=5000

System	Optimizer	$\overline{\rho}$	Observation
Logistic	Adam	$0.995\pm 0.003$	close agreement
MLP	SGD	$1.017\pm 0.009$	close agreement
MLP	Adam	$0.847\pm 0.012$	distinct observed branch

Table 4: Matched-loss comparisons with the GD reference. The logistic quantity uses the full model-Fisher width, whereas the MLP quantity uses the diagonal model-Fisher ball width of ( 91 ). Values are mean

\pm

sample standard deviation over six initializations.

Probe	Shape correlation with $B_{2}^{p}$	Matched-loss CV
$B_{2}^{p}$	—	$0.0367$
Random $8$ -dimensional subspace	$0.99987\pm 0.00010$	$0.0325$
$256$ -sparse unit probe	$0.99385\pm 0.00641$	$0.0261$
Normalized ellipsoid	$0.99991\pm 0.00006$	$0.0470$

Table 5: Fixed-probe behaviour under the diagonal model-Fisher geometry. Shape correlation measures temporal agreement with the Euclidean-ball trajectory and is reported as mean

\pm

sample standard deviation over six initializations. Matched-loss CV measures variability across the same six initializations.

Figure 2: Nonlinear MLP diagnostics under the diagonal model-Fisher approximation. Left: GD and SGD remain close at matched loss, whereas Adam follows a visibly displaced branch. Right: standardized Fisher widths for four fixed probes along the GD trajectory retain closely similar temporal shapes. Numerical summaries over all six initializations are reported in Tables 4 and 5 .

Fisher Information

Fisher information measures how strongly data or model predictions respond to changes in parameters, linking statistical uncertainty to neural-network sensitivity.

See all 22 papers

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds
Jun 16, 2026Vu Khac KyInformation Geometry Riemannian Geometry
Department of Mathematics, FPT University, Vietnam
Fisher Widths: Local Learning Geometry and Anisotropic Recovery
Jul 22, 2026Vu Khac KyInformation Geometry Sparse Recovery
Department of Mathematics, FPT University, Vietnam
Reducing the Adaptation Gap Through Reachable Fisher Geometry
Sep 28, 2026Wasif Jalal, Sachin Deb, Asif SalekinParameter-Efficient Fine-Tuning Group Fairness
Arizona State University Tempe, Arizona, USA

Jun 16, 2026J/K move · Enter open · S save

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

Vu Khac Ky

Department of Mathematics, FPT University, Vietnam

Information Geometry Riemannian Geometry

Open paper AskPDF arXiv

Gaussian width is a central geometric complexity measure in high-dimensional probability, compressed sensing, convex optimization, and learning theory. It quantifies the average extent of a set along random directions, thereby capturing the effective dimension of constraint sets, hypothesis classes, and descent cones. However, this notion is intrinsically Euclidean. Statistical models instead carry a natural Riemannian geometry induced by the Fisher information metric, where directions are scaled according to statistical distinguishability rather than ambient Euclidean length. We introduce Fisher width, a Fisher-geometric analogue of Gaussian width for statistical manifolds. At a parameter point

θ

, Fisher width replaces the Euclidean identity by the local metric tensor

G(θ)^{1/2}

, measuring the Gaussian width of the Fisher-rescaled set. This makes the resulting quantity sensitive to local statistical curvature and invariant under smooth reparameterizations. We develop the basic theory of Fisher width, showing that it retains key structural features of Gaussian width, including concentration, metric perturbation stability, and spectral comparison bounds with the Euclidean baseline, while also capturing anisotropic geometric effects invisible to Euclidean measures. As an application, we prove a generalization bound for Fisher-Lipschitz hypothesis classes and propose computable estimators, which we evaluate empirically on MNIST across three model classes. Fisher width is to statistical manifolds what Gaussian width is to Euclidean convex bodies. This work lays the foundation for studying complexity and learning on curved statistical manifolds.

Abstract

Fisher Information

Explore similar work

Fisher Width: A Geometric Measure of Complexity on Statistical Manifolds

Fisher Widths: Local Learning Geometry and Anisotropic Recovery

Reducing the Adaptation Gap Through Reachable Fisher Geometry