cs.LGNov 9, 2026

Identifiable but Hard to Estimate: Rotation in Equally Spaced Overcomplete ICA

Authors: Wenjun Xiao, Yinghao Fang, Hanxin Chen, Sihan Jia, Vince D Calhoun, Yuda BI

Organizations: The George Washington University · University of Health and Rehabilitation Sciences · University of California, San Diego · Georgia State University · 

Abstract

How many observations are needed to estimate a common rotation in two-dimensional overcomplete independent component analysis? We study equally spaced mixing columns with known source count, common source law, and column norm. For a fixed non-Gaussian fourth-order Gram-Charlier source law, the rotation is identifiable modulo the column spacing. Yet the sample size for fixed relative angular accuracy grows exponentially with the source count. We characterize this sample size, up to absolute constants, by the reciprocal of the squared Hellinger distance between two rotations separated by half the column spacing. An explicit radially weighted phase estimator attains this scale uniformly over the rotation family, using the observation radius to prevent cancellation in the mean of an angular harmonic. We also determine the exact exponential decay rate of single-observation total variation for the same pair. This gives bounds on the sample size exponent; its exact value remains open. Finally, a noisy binary-source example shows how an exact zero line of the observed characteristic function can have an exponentially small normal derivative.

Figures & tables

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

Aug 13, 2026

Foundations of Independent Component Analysis

We present the mathematical foundations of linear independent component analysis (ICA) models based on standard literature in a self-contained note. It is aimed at readers with a background in measure-theoretic probability theory. We first develop the theory of the characteristic functions of probability measures on Rd\mathbb{R}^d, including their analyticity and the way in which they determine and characterise the distributions. We then focus on several identifiability results of ICA models with successively strengthened assumptions on the sources: from merely non-constant, to non-Gaussian, to Gaussian-free independent sources. Under the strictest assumptions, we show that the independent sources are identifiable up to translation, permutation, scales and signs, and this even in the presence of additive Gaussian noise. Furthermore, we present the online equivariant gradient descent ICA algorithm for recovering the independent sources from data, in the standard complete noiseless non-Gaussian ICA setting.
Nov 4, 2026

Separation Is a Riesz Energy: How Many Environments Does ICA Need?

How many environments and samples does it take to separate dd Gaussian sources whose variances change across environments? Each source has a signature, its vector of variances. The Riesz 22-energy EjE_j of source jj sums the inverse Fisher informations for rotating it towards other sources. With known whitening, we prove a worst-case squared-angle risk lower bound of order Ej/(n+Ej)E_j/(n+E_j) for orthogonal estimators at every nn and dd. A quadratic pencil given the signatures attains Ej/nE_j/n up to a constant above a separation-dependent sample threshold, even with estimated whitening. For well-spread signatures at fixed KK, the energy grows as d2d^2, dlog⁡dd\log d and dd for K=2K=2, 33 and K≥4K\ge4, respectively. A design with K≍log⁡dK\asymp\log d and pairwise Fisher information bounded below by a constant makes dlog⁡dd\log d samples sufficient. The eigenvectors of a single fixed linear combination have an average first-order error coefficient of at least order d2d^2. Simulations up to d=1000d=1000 support the first-order predictions, and on task fMRI the energy helps rank split-half instability.
Apr 10, 2026

Finite-Sample Analysis of Nonlinear Independent Component Analysis:Sample Complexity and Identifiability Bounds

Independent Component Analysis (ICA) is a fundamental unsupervised learning technique foruncovering latent structure in data by separating mixed signals into their independent sources. While substantial progress has been made in establishing asymptotic identifiability guarantees for nonlinear ICA, the finite-sample statistical properties of learning algorithms remain poorly understood. This gap poses significant challenges for practitioners who must determine appropriate sample sizes for reliable source recovery. This paper presents a comprehensive finite-sample analysis of nonlinear ICA with neural network encoders, providing the first complete characterization with matching upper and lower bounds. Our theoretical development introduces three key technical contributions. First, we establish a direct relationship between excess risk and identification error that bypasses parameter-space arguments, thereby avoiding the rate degradation that would otherwise yield suboptimal scaling. Second, we prove matching information-theoretic lower bounds that confirm the optimality of our sample complexity results. Third, we extend our analysis to practical SGD optimization, showing that the same sample efficiency can be achieved with finite-iteration gradient descent under standard landscape assumptions. We validate our theoretical predictions through carefully designed simulation experiments. This gap points toward valuable future research on finite-sample behavior of neural network training and highlights the importance of our validated scaling laws for dimension and diversity.