stat.MLNov 29, 2025

Self-sufficient Independent Component Analysis for Demixing Flows

Authors: Song Liu

Organizations: University of Bristol

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

We study the problem of learning disentangled signals from data using non-linear Independent Component Analysis (ICA). Motivated by advances in self-supervised learning, we propose to learn self-sufficient signals: Given the remaining values of a recovered signal, observing other signals should not change the conditional distribution of its missing value. We formulate this problem as the minimization of a conditional KL divergence. Our algorithm is prior-free and likelihood-free in the sense that it prescribes neither parametric source densities nor an observation likelihood. To tackle the KL divergence minimization problem, we propose a sequential algorithm that learns a de-mixing flow model at each iteration, and prove local descent of the total correlation for its idealized Wasserstein-gradient-flow variant with exact velocities and a population projection condition. This approach completely avoids the unstable adversarial training, a common issue in minimizing the KL divergence. Experiments on toy and real-world datasets show the effectiveness of our method.

Figures & tables

Figure 1: Two dependent signals mixed by a linear mixing function. Both a linear ICA (LICA, Suzuki and Sugiyama,, 2011 ) and a non-linear ICA (iVAE, Khemakhem et al.,, 2020 ) fail to de-mix the signals (LICA returns a tilted heart, and iVAE loses its shape), whereas the proposed method recovers the heart (SICA-WGF; SICA-RF with some distortion in this run). The MCC with the true signals is given in brackets; see Appendix B.5 for details, results over 20 runs and further baselines.
Figure 2: Graphical models of ICA, auxiliary-variable ICA and SICA. In SICA, each signal si{\textnormal{s}}_{i} has its own auxiliary variable ui{\mathbf{u}}_{i} , and the ui{\mathbf{u}}_{i} may depend on each other arbitrarily.
Figure 3: MCC under non-linear mixing f{\bm{f}} over the number of mixing steps MM (higher is better), over 20 independent runs; error bars represent the standard error. Results under linear mixing and illustrative examples are in Appendix F .
Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

MethodMCC
Mixed signals0.875±0.0000.875\pm 0.000
FastICA0.775±0.0010.775\pm 0.001
LICA0.714±0.0010.714\pm 0.001
PCL0.687±0.0270.687\pm 0.027
iVAE0.386±0.0170.386\pm 0.017
AMUSE0.999±0.000\mathbf{0.999\pm 0.000}
Appendix
Table 1: MCC on the heart example over 20 independent runs (mean ±\pm standard error).
Figure 4: The SICA assumption equation 5 on sequence data S∈Rd×T{\mathbf{S}}\in\mathbb{R}^{d\times T} : given the context ui,t{\mathbf{u}}_{i,t} , i.e., the ii -th row without its tt -th entry (green), the entry si,t{\textnormal{s}}_{i,t} (orange) is independent of the other rows S−i,:{\mathbf{S}}_{-i,:} (blue).
Figure 5: AR(7) dataset. Left: Illustrative Example. Middle and Right: MCC over various mixing steps. The higher the better. MCC is measured over 20 independent runs, and error bars represent the standard error.
Figure 6: MNIST dataset. Left: Illustrative Example. Middle and Right: MCC over various mixing steps. MCC is measured over 20 independent runs, and error bars represent the standard error.
Figure 7: Speech mixtures ( d=3d=3 ) of LibriSpeech (a, b) and CMU ARCTIC (c, d) under linear and non-linear mixing f{\bm{f}} . MCC over various mixing steps, measured over 20 independent runs; error bars represent the standard error.

Explore similar work

CardsList

  1. Linear Independent Component Analysis via Optimal Transport

    Jul 15, 2026Ashutosh Jha, Michel Besserve, Simon BuchholzIndependent Component Analysis

    Methods Center, University of Tübingen · Max Planck Institute for Intelligent Systems, Tübingen · Institute of Artificial Intelligence, TU Braunschweig

  2. Foundations of Independent Component Analysis

    Aug 13, 2026Patrick ForréParameter IdentifiabilityIndependent Component Analysis

    AI4Science Lab Korteweg-de Vries Institute for Mathematics University of Amsterdam

  3. Kinks vs. Smoothness: Identifiability of Real Analytic nICA for Laplace-like Sources

    Sep 18, 2026Isaac Manring, Kejun HuangIndependent Component AnalysisRepresentation Identifiability

    Department of Computer Science University of Florida Gainesville, FL 32611, USA