Identifying Dependent Components from Multi-domain Linear Mixtures

Danru Xu; Lauri Parkkonen; Sara Magliacane; Aapo Hyvärinen

cs.LGOct 20, 2026

Identifying Dependent Components from Multi-domain Linear Mixtures

Authors: Danru Xu, Lauri Parkkonen, Sara Magliacane, Aapo Hyvärinen

Organizations: University of Amsterdam, Netherlands · Aalto University, Finland · Saarland University, Germany · University of Helsinki, Finland

Abstract

We study a linear mixing model with dependent latent components, assuming multiple data domains. Most existing models assume that the components are independent or at least uncorrelated, in line with independent component analysis (ICA). Some recent work allows for dependent components, but then makes specific assumptions such as parametric forms of dependencies, multi-view settings, or interventions, or does not recover the individual components. In contrast, we consider a multi-domain setting in which domains differ through domain-specific scalings of the components, while the distribution of the underlying latent components is the same across domains. This approach can model data collected, for example, from different sensors measuring the same process, different laboratories conducting the same experiment, different experimental conditions, or different subjects that might differ in biological or physiological factors. We show that, under sufficient domain variability, latent variables and mixing functions can be identified from second-order statistics alone. We propose the Multi-Domain Covariance Matching (MuDo-CoM) algorithm that generalizes previous methods of joint diagonalization. MuDo-CoM is validated on simulated data and a real-world fMRI dataset.

Figures & tables

Figure 1: Motivating example based on fMRI data. (a): latent variables $\mathbf{Z}=(Z_{1},\dots,Z_{5})$ with nontrivial dependencies, shown in their correlation matrix. (b): different domains (e.g., subjects) with their domain-specific diagonal scaling matrices $D^{\tau}$ , which rescale the latent variables while preserving their dependency structure, as illustrated in the marginal distribution of $Z_{1}$ . (c): the scaled latents are mixed through a shared, domain-invariant mixing matrix $A$ to generate observed data $X^{\tau}$ . (d): from these observations $X^{\tau}$ , the goal is to recover the latent variables $\mathbf{Z}$ (up to permutation and scaling), capturing both inter-regional dependence and inter-domain variability.

Method	Dependence	Para. Assump. on $\mathbf{Z}$	Struc. Assump. on $A$
FastICA ( Hyvarinen, 1999 )	$\mathbf{Z}_{i}\perp\mathbf{Z}_{j}$	non-gaussian	● None
QNDIAG ( Ablin et al., 2019 )	cov $(\mathbf{Z}_{i},\mathbf{Z}_{j})=0$	● None	● None
PARAFAC-COV ( Harshman and others, 1970 )	cov $(\mathbf{Z}_{i},\mathbf{Z}_{j})=0$	● None	● None
iVAE ( Khemakhem et al., 2020 )	$\mathbf{Z}_{i}\perp\mathbf{Z}_{j}\|U$	Cond. exponential family	● None
MHA ( Monti and Hyvärinen, 2018 )	● None	● None	Orthonormal + nonnegative
MuDo-CoM (Our method)	● None	● None	● None

Table 1: Comparison of closely related approaches in terms of assumptions on latent dependence, parametric assumptions on the latent variables

\mathbf{Z}

, and structural assumptions on the mixing matrix

A

. Both QNDIAG and PARAFAC implicitly assume that latent components are uncorrelated within each domain, because QNDIAG performs joint diagonalization of a collection of covariance matrices, and PARAFAC does not model cross-interactions between latent components. For more details, see App. E.2 .

$n$	$d$	$T$	$N$	$k$	SCMs	MuDo-CoM	FastICA	QNDIAG	PARAFAC	iVAE	MHA
5	$n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .000	0.786 $\pm$ .056	0.814 $\pm$ .049	0.688 $\pm$ .151	0.808 $\pm$ .062	0.783 $\pm$ .033
10	$n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .001	0.783 $\pm$ .013	0.823 $\pm$ .019	0.477 $\pm$ .200	0.735 $\pm$ .035	0.726 $\pm$ .039
20	$n$	2 $n$	5000	2	Gauss	0.998 $\pm$ .001	0.791 $\pm$ .021	0.835 $\pm$ .018	0.319 $\pm$ .081	0.711 $\pm$ .075	0.618 $\pm$ .018
10	1 $n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .001	0.783 $\pm$ .013	0.823 $\pm$ .019	0.477 $\pm$ .200	0.735 $\pm$ .035	0.726 $\pm$ .039
10	2 $n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .001	0.783 $\pm$ .013	0.823 $\pm$ .019	0.509 $\pm$ .167	0.734 $\pm$ .017	0.687 $\pm$ .027
10	4 $n$	2 $n$	5000	2	Gauss	0.997 $\pm$ .004	0.783 $\pm$ .013	0.823 $\pm$ .019	0.464 $\pm$ .204	0.792 $\pm$ .022	0.689 $\pm$ .031

Table 2: Results for the numerical experiments: Average d-MCC

\pm

Std over 5 random seeds. The bold font indicates which parameters are varying in each block of rows and the best results among all methods.

Figure 2: Visualization of the estimated components for the fMRI data for one random seed. Each column gives the five columns of the matrix

\hat{A}

for one method, after aligning permutation and signs. Red means positive activation in a parcel, while blue means negative activity; the brain is viewed from three different angles. The last row shows the sample correlation matrix of the recovered variables

\hat{\mathbf{Z}}

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$\tau$	Domain index
$\mathbf{Z}$	Latent variables
$\Lambda$	Covariance matrix of the noise term
$\mathbf{X}^{\tau}$	Observations from domain $\tau$
$A$	Linear mixing matrix
$D^{\tau}$	Domain-specific scaling matrix for domain $\tau$

Appendix

Table 5

	Numerical	CamCAN
Optimizer	Adam	Adam
Learning rate	5e-3	5e-3
# Seeds	5	5 or 20
# Iterations	50000	10000
# Initialization	3	20

Appendix

Table 3: Parameters for experiments results in Sec. 6 and App. E .

$n$	$d$	$T$	$N$	$k$	SCMs	MuDo-CoM	FastICA	QNDIAG	PARAFAC	iVAE	MHA
5	$n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .000	0.767 $\pm$ .053	0.800 $\pm$ .043	0.611 $\pm$ .193	0.715 $\pm$ .088	0.728 $\pm$ .046
10	$n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .001	0.740 $\pm$ .033	0.788 $\pm$ .029	0.433 $\pm$ .219	0.634 $\pm$ .072	0.687 $\pm$ .045
20	$n$	2 $n$	5000	2	Gauss	0.985 $\pm$ .021	0.732 $\pm$ .021	0.794 $\pm$ .016	0.274 $\pm$ .072	0.606 $\pm$ .083	0.581 $\pm$ .019
10	1 $n$	2 $n$	5000	2	Gauss	0.999 $\pm$ .001	0.740 $\pm$ .033	0.788 $\pm$ .029	0.433 $\pm$ .219	0.646 $\pm$ .042	0.687 $\pm$ .045
10	2 $n$	2 $n$	5000	2	Gauss	0.998 $\pm$ .001	0.741 $\pm$ .034	0.788 $\pm$ .029	0.443 $\pm$ .174	0.634 $\pm$ .072	0.653 $\pm$ .034
10	4 $n$	2 $n$	5000	2	Gauss	0.980 $\pm$ .037	0.741 $\pm$ .034	0.788 $\pm$ .029	0.382 $\pm$ .231	0.716 $\pm$ .034	0.632 $\pm$ .034

Appendix

Table 4: Results for the numerical experiments: Ave. MCC

\pm

Std over 5 random seeds. The bold font indicates which parameters vary across each block of rows, as well as the best performance method.

$n$	$d$	$T$	$N$	$k$	SCMs	Mudo-CoM	FastICA	QNDIAG	PARAFAC	MHA
5	$n$	2 $n$	5000	2	Gauss	0.07 $\pm$ 0.02	0.86 $\pm$ 0.10	0.80 $\pm$ 0.08	1.00 $\pm$ 0.18	1.43 $\pm$ 0.33
10	$n$	2 $n$	5000	2	Gauss	0.10 $\pm$ 0.02	1.04 $\pm$ 0.13	0.94 $\pm$ 0.11	1.80 $\pm$ 0.25	3.28 $\pm$ 0.26
20	$n$	2 $n$	5000	2	Gauss	0.19 $\pm$ 0.03	1.09 $\pm$ 0.11	0.94 $\pm$ 0.10	2.79 $\pm$ 0.63	6.31 $\pm$ 0.16
10	1 $n$	2 $n$	5000	2	Gauss	0.10 $\pm$ 0.02	1.04 $\pm$ 0.13	0.94 $\pm$ 0.11	1.80 $\pm$ 0.25	3.28 $\pm$ 0.26
10	2 $n$	2 $n$	5000	2	Gauss	0.11 $\pm$ 0.01	1.04 $\pm$ 0.13	0.94 $\pm$ 0.14	1.77 $\pm$ 0.51	3.36 $\pm$ 0.40
10	4 $n$	2 $n$	5000	2	Gauss	0.12 $\pm$ 0.04	1.04 $\pm$ 0.13	0.94 $\pm$ 0.12	1.69 $\pm$ 0.30	3.41 $\pm$ 0.31

Appendix

Table 5: Results for the numerical experiments: Ave. Amari

\pm

Std over 5 random seeds. The bold font indicates which parameters vary across each block of rows, as well as the best performance method.

Panel of Figure 3 — Figure 3: Effect of increasing the number of domains $T$ . ( a ) Mean $\pm$ standard deviation of MCC and d-MCC over $T\in\{n,2n,3n,4n\}$ across 5 random seeds. ( b ) Corresponding Amari distance, showing improved common mixing matrix recovery as more domains are observed.

$\sigma$	0	0.1	0.5	1
$\hat{\sigma}$	0.01	0.11	0.51	1.01
$Amari$	0.10	0.10	0.11	0.12

Appendix

Table 6: Average Amari destance and

\hat{\sigma}

over 5 random seeds

$m$	0	1	2	3
$MCC$	1.00	0.93	0.81	0.73

Appendix

Table 7: Average

MCC

over 5 random seeds

$\delta_{A}$	0	0.05	0.1	0.2
$MCC$	1.00	0.95	0.85	0.68
$\delta_{Corr}$	0	0.05	0.1	0.2
$MCC$	1.00	0.98	0.90	0.52

Appendix

Table 8: Average

MCC

over 5 random seeds

Latent Variable Models

Latent variable models infer hidden factors, classes, or states from observed data to explain structure, uncertainty, and variation.

See all 153 papers

Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence
Jun 6, 2026Takafumi Kanamori, Yushi Hirose, Shohei YamamotoLatent Variable Models Parameter Identifiability
Department of Mathematical and Computing Science, Institute of Science Tokyo · RIKEN Center for Advanced Intelligence Project
Learnability and Competition in High-Dimensional Multi-Component ICA
May 8, 2026Eser Ilke Genc, Samet Demir, Zafer DoganIndependent Component Analysis Unsupervised Representation Learning
MLIP Research Group, KUIS AI Center, Koç University · Department of EEE, Koç University
Explaining Data Mixing Scaling Laws
Jun 6, 2026Rui Dai, Shuran ZhengData Mixture Optimization Neural Scaling Laws
Beijing Institute of Technology · IIIS, Tsinghua University

Jun 6, 2026J/K move · Enter open · S save

Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence

Takafumi Kanamori, Yushi Hirose, Shohei Yamamoto

Department of Mathematical and Computing Science, Institute of Science Tokyo · RIKEN Center for Advanced Intelligence Project

Latent Variable Models Parameter Identifiability

Open paper AskPDF arXiv

We study component recovery and mixing-matrix estimation from unlabeled finite mixtures whose observable distributions share the same latent components but have unknown mixing weights. The main identifying signal is marginal independence: each component is assumed to be independent on at least one coordinate pair, but no labels, clean component samples, or mixing weights are observed. We first prove a structural result for product components: under linear independence of the univariate marginals, any independent affine combination of the components must coincide with a single component. We then extend this principle to observable mixtures and show that, under full-rank and no-cancellation conditions, marginally independent affine combinations recover the corresponding latent components. When every component is independent on some coordinate pair, all components are identifiable, and the mixing matrix is recoverable under the stated completion conditions. Finally, we propose a Product-Marginal Maximum Mean Discrepancy (PM-MMD) estimator over affine combinations of the observable mixtures and prove uniform convergence and stability under approximate marginal independence. This framework also separates the empirical roles of the assumptions: irreducibility is, in general, not directly testable from the unlabeled mixtures alone, whereas marginal independence yields a candidate-level diagnostic through held-out PM-MMD. Controlled and flow-cytometry experiments show when marginal independence provides a useful recovery signal. In the reported multi-component comparisons, condition-aware representative selection stabilizes PM-MMD and improves recovery relative to clustering, factorization, and pairwise mixture-proportion baselines using the same unlabeled mixtures.

Abstract

Latent Variable Models

Explore similar work

Identifiability and Estimation for Unlabeled Finite Mixtures under Marginal Independence

Learnability and Competition in High-Dimensional Multi-Component ICA

Explaining Data Mixing Scaling Laws