Spectral Indistinguishability and Learning-Curve Identifiability in Quadratic Networks

Dekun Yang

cs.LGOct 14, 2026

Spectral Indistinguishability and Learning-Curve Identifiability in Quadratic Networks

Authors: Dekun Yang

Organizations: Zhejiang University, China

Abstract

Motivated by information-organization sculpting, we study spectral summaries and scalar learning curves in centered quadratic networks. We construct analytic positive input densities with Gaussian one- and two-coordinate marginals. Their complete fourth-moment operators are orthogonally intertwined by a map fixing the entire diagonal teacher algebra, giving equal anchored spectral measures. For selected common teachers with a nonzero cycle invariant, their factorized loss curves differ because the map does not preserve matrix multiplication. Cycle length $r$ , which can be arbitrarily large, gives $O(\varepsilon^r)$ contrast uniformly in training time for each fixed construction. For a supplied pair, noisy scalar loss queries require order $\varepsilon^{-2r}$ observations, while bounded initialization interventions with scalar gradient readouts require order $\varepsilon^{-2}$ . A shared-input score, equal to a paired-gradient difference, attains the optimal input-sample exponent without knowing the teacher. An explicit family of diverging auxiliary teacher scales instead admits a nonzero finite limiting contrast at coupling $1/\log L$ . Conversely, for Gaussian inputs and a rationally independent teacher spectrum, one exact scalar loss curve identifies the initial predictor up to sign conjugation. Within this rationally independent class, with nonzero diagonal inverse-state error in every coordinate and calibrated time and loss scales, it also identifies the unknown teacher spectrum and active dimension, modulo simultaneous rotations. For fixed dimension and distinct unordered teacher sums, dimension-minimal exact observations give generic joint local recovery. Finite exact global bounds hold for a known teacher and, on the active class, under a blind dimension cap. Exact certificates and full-weight replay verify the construction; numerical inverse checks reveal severe ill-conditioning.

Figures & tables

Ingredient	Established result	Dynamic statement proved here
Quadratic predictor dynamics	Matrix closure and isotropic Riccati solutions ( Liu Ziyin et al., 2026 )	A fixed initial factor develops different effective states under matched anchored spectra.
Signed-cycle invariants	Switching classes, cycle products and principal-minor recovery ( Urschel et al., 2017 ; Brunel and Urschel, 2024 )	One exact scalar loss trajectory determines those invariants for a rationally independent teacher; diagonal activity also permits blind spectrum recovery within the active class.
Finite observations	Nonlinear observability and quadratic principal-minor query counts ( Sontag, 2002 ; Brunel and Urschel, 2024 )	For fixed dimension and distinct unordered teacher sums, any dimension-minimal set of exact observations at distinct times gives generic joint local identification.
Testing difficulty	Gaussian divergence bounds and cycle-dependent DPP bounds ( Urschel et al., 2017 )	A loss-signal bound uniform over adaptive observation times.

Table 1: The contribution is in the learning dynamics and observation channel. The listed algebraic and statistical tools are prior results.

Symbol	Meaning and context
$A(x)$	Centered quadratic feature matrix.
$A,C$	Auxiliary and interaction coordinate sets in the paired-law construction.
$B_{\pm}$	Signed interaction adjacency matrices defining the two input laws.
$C=M_{0}^{-1}-T^{-1}$	Inverse-state error matrix, used in the Gaussian inverse problem.
$A_{\pm}$	Cycle matrices defining a supplied initial-state pair in the noisy inverse test.
$t,s$	Training time in the forward construction and Gaussian inverse problem, respectively.

Table 2: Notation across the construction and inverse problem. The coordinate set

C

and inverse-state matrix

C

occur in separate sections. Arguments and subscripts distinguish

A(x)

and

A_{\pm}

from the coordinate set

A

Figure 1: Spectral equivalence, finite training contrast and its scale dependence. a , Paired common-loss differences from complete weight flow and actual weight updates at the three fixed observation times. Lines joining the flow observations only guide the eye. b , Absolute errors between the common-loss contrasts of the original coupled dynamics and the derived limiting contrasts at four fixed times, as the auxiliary teacher scale grows. Some errors are nonmonotone. All four declared scales are shown. Both panels are deterministic checks without sampling error bars. The teacher in panel a was selected by the disclosed 108-cell exploration; panel b uses diverging teacher norms.

Figure 2: Finite identifiability does not ensure stable reconstruction. a , All six singular values of the analytic known-teacher observation Jacobian in the specified three-dimensional example. b , Relative parameter errors for every declared local inverse attempt, including the noise-free control. Each noise level uses the same seeded Gaussian direction; these are single deterministic realizations, not averaged performance estimates. Initialization is within

2\%

of the true parameter in every case.

Parameter Identifiability

Parameter identifiability studies when observations and model assumptions uniquely determine a system’s unknown parameters.

See all 103 papers

Quotient Dynamics, Effective Curvature, and Implicit Bias in Positive Quadratic Networks
Jul 28, 2026Pengcheng ChengRiemannian Optimization Implicit Regularization
School of Mathematics, Jilin University Changchun, 130012, China
Algebraic Expressivity Certificates for Shallow Polynomial Neural Networks
Sep 11, 2026Sepehr Akbari, Shahrzad JamshidiNeural Network Expressivity Shallow Neural Networks
Dept. of Math & Computer Science Lake Forest College
Teacher Geometry Shapes Learnability in Teacher-Student Networks
Sep 9, 2026Kai J. Sandbrink, Flavio Martinelli, Alexander van Meegen +2Neural Network Optimization Neural Network Training Dynamics
Department of Experimental Psychology, University of Oxford · Department of Physics, RWTH Aachen

Jul 28, 2026J/K move · Enter open · S save

Quotient Dynamics, Effective Curvature, and Implicit Bias in Positive Quadratic Networks

Pengcheng Cheng

School of Mathematics, Jilin University Changchun, 130012, China

Riemannian Optimization Implicit Regularization Gradient Descent Dynamics

Open paper AskPDF arXiv

Positive quadratic networks admit the low-rank representation f_U(x)=x^top UU^top x, where Uinmathbb{R}^{dtimes r} is identifiable only up to right orthogonal multiplication, representing a rank-r PSD matrix Q=UU^top. We study how this quotient structure governs training dynamics, curvature, recovery, and interpolation bias. On the full-column-rank stratum, we identify mathbb{R}^{dtimes r}_*/O(r) with the rank-r PSD manifold. For smooth objectives L(U)=ell(UU^top), the Euclidean factor gradient is horizontal. Thus, factor gradient flow projects exactly to quotient Riemannian gradient flow, while finite-step gradient descent induces an exact congruence recursion for the predictor. For quadratic regression, we derive the effective Hessian at interpolators as the empirical measurement Gram form restricted to the tangent space relative to the quotient metric. Under Gaussian rank-one measurements, we compute population curvature, prove uniform deviation bounds for the empirical normal operator, construct a spectral initializer, and establish local exponential convergence for gradient flow and linear convergence for small-step descent. Recovery guarantees are explicit but conservative due to reliance on full-space second-moment control. In underdetermined commuting regimes, factor gradient flow becomes an exact entropy mirror flow in joint spectral coordinates. Strictly positive initializations converge to Bregman projections onto the interpolation set. With isotropic initialization q(0)=varepsilon^2mathbf{1}, predictors approach the minimum-trace solution set as varepsilondownarrow0, resolving nonuniqueness via weighted entropy within the invariant joint spectral algebra. Finite-step descent selects interpolants differing from continuous-time Bregman projections by O(eta). Numerical experiments verify these quotient identities, curvature predictions, recovery behaviors, and selection laws.

Abstract

Parameter Identifiability

Explore similar work

Quotient Dynamics, Effective Curvature, and Implicit Bias in Positive Quadratic Networks

Algebraic Expressivity Certificates for Shallow Polynomial Neural Networks

Teacher Geometry Shapes Learnability in Teacher-Student Networks