Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity

Zavier Li

cs.LGSep 14, 2026

Information-Induced Training Geometry: Exact Reduction, Canonical Completion, and Structured Expressivity

Authors: Zavier Li

Abstract

Training data constrains optimizer geometry through the covectors visible to a declared information channel. We study how such partial information determines a full positive cometric relative to a reference and which degrees of freedom remain unidentified. Our central result resolves full-column-rank positive-definite compression under affine-invariant Riemannian geometry. The compression map is a split-Hadamard metric submetry and admits an explicit unique completion that is the affine-invariant nearest full geometry realizing a visible target and yields exact full-to-visible variational reduction. When the channel moves, the completions form a gauge-invariant rank stratification of the positive-definite cone. Its closed-form pullback pair metric separates visible-metric motion from subspace rotation through a reference-mismatch weight, yields an explicit positive-semidefinite multi-direction Gram matrix, and exposes the precise singularity of reference-valued modes. The mechanism is explained by a metric theorem equating ball submetry, attained fiber distance, and lossless reduction of every monotone radial visible decision problem. A smooth split-Hadamard theorem supplies coherent information sheets, proximal commutation, and solution-wise gradient-flow lifting. The positive-definite realization also gives closed-form prior-data shrinkage. Diagonal and block optimizer families reduce to relative-interior conic image tests with valid facial certificates, while deterministic and finite-sample bounds quantify recovery of the visible geometry and its subspace. Together these results characterize exact reduction, reference-dependent completion, and structured expressivity for the stated finite-dimensional affine-invariant model.

Explore similar work

May 28, 2026cs.LG

Open Problem: Separating Geometric and Algorithmic Compression via Cayley-Table Completion

Modern statistical learning theory and deep learning characterize generalization primarily in terms of continuous capacity control (e.g., norm-based regularization, margin maximization, low-rank bias). While highly successful in continuous domains, deep learning consistently fails to extrapolate exact algorithmic or discrete algebraic rules, reflecting a missing inductive bias toward algorithmic complexity minimization. We propose the Cayley-table completion as the canonical testbed for this missing bias, serving as the discrete algebraic counterpart to matrix completion. Just as matrix factorization combined with weight decay yields an implicit geometric bias toward low linear rank, recent results demonstrate that operator-valued tensor factorizations paired with a flatness prior yield an implicit algorithmic bias toward exact discrete associativity. We pose the open problem of establishing formal exact recovery bounds for Cayley-table completion, and challenge the community to generalize continuous flatness priors to autonomously discover broader discrete algorithmic axioms without combinatorial search.

Dongsung Huh

Tensor Completion Optimal Sample Complexity Exact Context Compression Prior Knowledge Tabular Data

Ask PDF

Nov 4, 2025cs.LG

Geometry as a Missing Axis of Representation Quality: The Variational Geometric Information Bottleneck under Data Scarcity

We study latent geometry as an explicit component of representation quality in data-scarce learning. For an encoder (φ), we define (Q_{β,γ}(φ)=I(φ(X);Y)-β\mathcal C(φ)-γd_{\mathrm{int}}(φ)), combining task-relevant information with penalties for curvature and intrinsic latent dimension. Thus geometry becomes part of the bottleneck criterion, not only a post hoc diagnostic. Under smooth-manifold, loss-transfer, and estimator-concentration assumptions, we derive non-asymptotic low-label generalization bounds where intrinsic dimension and covering complexity enter explicitly. We characterize the information--geometry frontier and prove empirical-surrogate consistency. The analysis links encoder geometry to learning through latent covering numbers, loss-class entropy, and uniform deviation. We instantiate the theory as \texttt{V-GIB}, adding curvature and dimension penalties to variational bottleneck training. Real low-label benchmarks compare \texttt{V-GIB} with ERM, VIB, and ablations across (1%)--(20%) label fractions. Results show improved performance and reduced geometric complexity in several regimes, especially FashionMNIST and CIFAR-10, while confirming that no fixed regularizer is universally dominant.

Ronald Katende

Information Bottleneck Generalization Bounds Data Scarcity Representation Quality 3D Encoder Missing Data+1

Ask PDF

Feb 2, 2026cs.LG

Discovering Data Manifold Geometry through Geometric Properties

A prevailing paradigm in modern representation learning is the map-first approach, in which a representation map is learned from reconstruction, embedding, or task objectives. At the optimum, when the learned map accurately recovers a global coordinate chart, it should exhibit three structural properties whose geometric meaning can be illustrated through a face-editing example: Commutativity requires that changing pose and then expression gives the same result as applying them in the reverse order; Time Coherence requires that the same variation along one coordinate induces the same expression change across faces; Common-Reference requires that all faces are organized relative to a common reference face. However, small approximation errors in the learned map need not translate into small errors in these structural properties, and can therefore disrupt the global organization of the representation. Based on this observation, we consider the converse of the map-first formulation and ask whether a global representation can instead emerge by directly learning these properties. We represent variations along individual coordinates through vector fields defined in the ambient space and introduce a non-contraction condition preventing one transformation from destroying directions associated with the others. We derive an unsupervised objective that learns these structural properties and establish theoretical results connecting its minimization to tangent-space recovery. Experiments on controlled manifolds validate the predicted tangent-space recovery and global structure, while an autoencoder baseline shows that small map-first errors can still produce substantial violations of the targeted properties.

David Vigouroux, Lucas Drumetz, Ronan Fablet +1

Topic pending

Ask PDF