Frame Theoretical Derivation of Three Factor Learning Rule for Oja's Subspace Rule

Taiki Yamada

cs.NEApr 3, 2026

Frame Theoretical Derivation of Three Factor Learning Rule for Oja's Subspace Rule

Authors: Taiki Yamada

Abstract

We show that the error-gated Hebbian rule for PCA (EGHR-PCA), a three-factor learning rule equivalent to Oja's subspace rule under Gaussian inputs, can be systematically derived from Oja's subspace rule using frame theory. The global third factor in EGHR-PCA arises exactly as a frame coefficient when the learning rule is expanded with respect to a natural frame on the space of symmetric matrices. This provides a principled, non-heuristic derivation of a biologically plausible learning rule from its mathematically canonical counterpart.

Hebbian Learning

Hebbian learning studies how synapses adapt through correlations between neural activity, enabling local learning, memory, and adaptation.

See all 22 papers

Information-Geometric Decomposition of Generalization Error in Unsupervised Learning
Apr 14, 2026Gilhan KimUnsupervised Learning Information Geometry
Department of Physics and Astronomy, Seoul National University, Seoul 08826, Republic of Korea, where this work was performed. · Present address: Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea.
Supervised Training Rapidly Degrades Early Visual Cortex Alignment Across Biologically Plausible Learning Rules
May 28, 2026Nils LeuteneggerNeural Representation Geometry Predictive Coding
Independent Researcher, Switzerland
Untrained CNNs Match Backpropagation at V1: A Systematic RSA Comparison of Four Learning Rules Against Human fMRI
Apr 18, 2026Nils LeuteneggerBackpropagation Visual Neuroscience
Independent Researcher, Switzerland

Apr 14, 2026J/K move · Enter open · S save

Information-Geometric Decomposition of Generalization Error in Unsupervised Learning

Gilhan Kim

Department of Physics and Astronomy, Seoul National University, Seoul 08826, Republic of Korea, where this work was performed. · Present address: Department of Statistics and Data Science, Yonsei University, Seoul 03722, Republic of Korea.

Unsupervised Learning Information Geometry

Open paper AskPDF arXiv

We decompose the Kullback--Leibler generalization error (GE) -- the expected KL divergence from the data distribution to the trained model -- of unsupervised learning into three non-negative components: model error, data bias, and variance. The decomposition is exact for any e-flat model class and follows from two identities of information geometry: the generalized Pythagorean theorem and a dual e-mixture variance identity. As an analytically tractable demonstration, we apply the framework to

ε

-PCA, a regularized principal component analysis in which the empirical covariance is truncated at rank

N_K

and discarded directions are pinned at a fixed noise floor

ε

. Although rank-constrained

ε

-PCA is not itself e-flat, it admits a technical reformulation with the same total GE on isotropic Gaussian data, under which each component of the decomposition takes closed form. The optimal rank emerges as the cutoff

λ_{\mathrm{cut}}^{*} = ε

-- the model retains exactly those empirical eigenvalues exceeding the noise floor -- with the cutoff reflecting a marginal-rate balance between model-error gain and data-bias cost. A boundary comparison further yields a three-regime phase diagram -- retain-all, interior, and collapse -- separated by the lower Marchenko--Pastur edge and an analytically computable collapse threshold

ε_{*}(α)

, where

α

is the dimension-to-sample-size ratio. All claims are verified numerically.

Abstract

Hebbian Learning

Explore similar work

Information-Geometric Decomposition of Generalization Error in Unsupervised Learning

Supervised Training Rapidly Degrades Early Visual Cortex Alignment Across Biologically Plausible Learning Rules

Untrained CNNs Match Backpropagation at V1: A Systematic RSA Comparison of Four Learning Rules Against Human fMRI