Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing

Danru Xu; Sébastien Lachapelle; Sara Magliacane

stat.MLApr 14, 2026

Identifiability of Potentially Degenerate Gaussian Mixture Models With Piecewise Affine Mixing

Authors: Danru Xu, Sébastien Lachapelle, Sara Magliacane

Organizations: University of Amsterdam · Samsung AI Lab, Montreal · Saarland University & University of Amsterdam

Abstract

Causal representation learning (CRL) aims to identify the underlying latent variables from high-dimensional observations, even when variables are dependent with each other. We study this problem for latent variables that follow a potentially degenerate Gaussian mixture distribution and that are only observed through the transformation via a piecewise affine mixing function. We provide a series of progressively stronger identifiability results for this challenging setting in which the probability density functions are ill-defined because of the potential degeneracy. For identifiability up to permutation and scaling, we leverage a sparsity regularization on the learned representation. Based on our theoretical results, we propose a two-stage method to estimate the latent variables by enforcing sparsity and Gaussianity in the learned representations. Experiments on synthetic and image data highlight our method's effectiveness in recovering the ground-truth latent variables.

Figures & tables

Figure 1: Example of a 3-dimensional pdGMM with 2 components: a rank-2 component with support represented by the yellow plane, and a rank-1 component with support represented by the red line. Knowing the distribution on the open set

E

(in brown), intersecting both components, is sufficient to identify the pdGMM.

$n$	$k$	$m$	$\rho$	$\delta$	$\theta$	$\mathbf{R}^{2}$ $\uparrow$ Stage1	MCC Stage1	MCC $\uparrow$ Stage2	MCC $\uparrow$ VaDE
5	1	10	50%	0	0	0.93	0.75	0.96	0.45
10	1	10	50%	0	0	0.94	0.56	0.97	0.32
20	1	10	50%	0	0	0.93	0.46	0.92	0.23
40	1	10	50%	0	0	0.94	0.37	0.94	0.25
10	0	10	50%	0	0	0.94	0.51	0.97	0.33
10	1	10	50%	0	0	0.94	0.56	0.97	0.32

Table 1: Results for the numerical experiments. The bold font highlights the parameters that vary in each block.

\mathbf{R}^{2}

(Stage 1) shows identifiability up to AT after Stage 1 MCC (Stage 1) reflects the necessity of sparsity to achieve identifiability up to PS; MCC (Stage 2) presents the identifiability up to PS after Stage 2; MCC (VaDE) reports the results of Kivva et al. (2022) . Full results with Std. Dev. are in Tab. 3 .

Figure 2: $R^{2}$ of the linear regression between ball positions $b_{1},\dots,b_{4}$ and learned representations $\hat{z}_{1},\dots,\hat{z}_{8}$ . We use two variables for each $(x,y)$ position.

Appendix figures & tables30 assets

Supplementary material from the paper’s appendix.

Appendix

Symbol	Description
$\mathbf{Z}$	Causal latent variables
$\mathbf{X}$	Observations
$\mathbf{f}$	Mixing function
$n$	Latent space dimension
$d$	Observation space dimension
$J$	Number of components in mixture models

Appendix

Table 4

Figure 3: Visualization of Example B.13 :

\mathbf{f}

switches half axes of

\mathbf{z}_{1}

and

\mathbf{z}_{2}

. The orange area refers to the support of component 1, and the green area refers to the support of component 2 before transformation.

	Numerical	Numerical	Balls	Balls	Balls
	Stage 1	Stage 2	CNN	Stage 1	Stage 2
Sparsity level $\epsilon$		0.01			0.01
Optimizer	Adam	ExtraAdam	Adam	Adam	ExtraAdam
Primal optimizer learning rate	1e-4	1e-4	1e-3	1e-3	1e-4
Dual optimizer learning rate		5e-5			5e-5
Batch size	6144	6144	256	256	256

Appendix

Table 2: Parameters for experiments results in Sec. 5 and App. D .

$n$	$k$	$m$	$\rho$	$\delta$	$\theta$	$\mathbf{R}^{2}$ $\uparrow$ (Stage1)	MCC (Stage1)	MCC (Stage1)	MCC $\uparrow$ (VaDE)
5	1	10	50%	0	0	0.93 $\pm$ 0.02	0.75 $\pm$ 0.03	0.96 $\pm$ 0.01 ‘	0.45 $\pm$ 0.05
10	1	10	50%	0	0	0.94 $\pm$ 0.02	0.56 $\pm$ 0.03	0.97 $\pm$ 0.01	0.32 $\pm$ 0.03
20	1	10	50%	0	0	0.93 $\pm$ 0.02	0.46 $\pm$ 0.01	0.92 $\pm$ 0.05	0.23 $\pm$ 0.15
40	1	10	50%	0	0	0.94 $\pm$ 0.01	0.37 $\pm$ 0.01	0.94 $\pm$ 0.03	0.25 $\pm$ 0.11
10	0	10	50%	0	0	0.94 $\pm$ 0.02	0.51 $\pm$ 0.11	0.97 $\pm$ 0.01	0.33 $\pm$ 0.05
10	1	10	50%	0	0	0.94 $\pm$ 0.02	0.56 $\pm$ 0.03	0.97 $\pm$ 0.01	0.32 $\pm$ 0.03

Appendix

Table 3: Results for the numerical experiments (Average

\pm

Standard Deviation). The bold font indicates which parameters are varying in each block of rows.

\mathbf{R}^{2}

(Stage 1) shows the identifiability up to AT after Stage 1 (Thm. 3.7 ); MCC (Stage 1) reflects the neccesarity of sparsity to achieve PTS identifiability; MCC (Stage 2) presents the identifiability up to PTS after Stage 2 (Thm. 3.9 ).

Panel of Figure 4 — Figure 4: Ablation study on increasing the latent dimension $n$ from 5 to 40 and fixing $k=1$ , $m=10$ , $\rho=50\%$ , $\delta=0.0\sigma$ , $\theta=0$ . The case with a higher $n$ is more complicated to learn the latent variables.

Panel of Figure 5 — Figure 5: Ablation study on increasing the density of causal graphs $k$ from 0 to 3 and fixing $n=10$ , $m=10$ , $\rho=50\%$ , $\delta=0.0\sigma$ , $\theta=0$ . In the case with a denser graph, it is more complicated to learn the latent variables.

Panel of Figure 6 — Figure 6: Ablation study on increasing the number of Leaky-ReLU layers $(m-1)$ from 3 to 20 and fixing $n=10$ , $k=1$ , $\rho=50\%$ , $\delta=0.0\sigma$ , $\theta=0$ . The case with a higher $n$ is more complicated to learn the latent variables. The case with larger $m$ is more complicated to learn the latent variables due to a greater extent of nonlinearity.

Panel of Figure 7 — Figure 7: Piecewise linear mixing function with linear causal relation and Gaussian noise: ablation study on increasing the ratio of active (unmasked) variables $\rho$ from 1 variable only to 75% and fixing $n=10$ , $k=1$ , $m=10$ , $\delta=0.0\sigma$ , $\theta=0$ . Learning the latent variables is more complicated in the case with a larger portion of active variables.

Panel of Figure 8 — Figure 8: Ablation study on increasing the distance between masked value and mean of latents from $0.0\sigma$ to $3.0\sigma$ , where $\sigma$ is the standard deviation of latents, and fixing $n=10$ , $k=1$ , $m=10$ , $\rho=50\%$ , $\theta=0$ . Disentangling the latent variables is more complicated in the case of a larger deviation from zero translation.

Panel of Figure 9 — Figure 9: Ablation study on increasing the rotation of common basis from $0.0^{\circ}$ to $45.0^{\circ}$ , and fixing $n=10$ , $k=1$ , $m=10$ , $\rho=50\%$ , $\delta=0.0\sigma$ . Disentangling the latent variables is more complicated in the case of a larger deviation from the standard basis.

$n$	$k$	$m$	$\rho$	$\delta$	$\theta$	MCC Independent	MCC in Tab. 1
5	1	10	50 %	0	0	0.96 $\pm$ 0.01	0.96 $\pm$ 0.01
10	1	10	50 %	0	0	0.97 $\pm$ 0.01	0.97 $\pm$ 0.01
20	1	10	50 %	0	0	0.93 $\pm$ 0.05	0.92 $\pm$ 0.05
40	1	10	50 %	0	0	0.94 $\pm$ 0.03	0.94 $\pm$ 0.03
10	0	10	50 %	0	0	0.97 $\pm$ 0.01	0.97 $\pm$ 0.01
10	1	10	50 %	0	0	0.97 $\pm$ 0.01	0.97 $\pm$ 0.01

Appendix

Table 4: Results for the numerical experiments (Stage 2) tested on independent data. The bold font indicates which parameters are varying in each block of rows.

$n$	5	10	20	40	50
Epoch	2,500	20,000	125,000	300,000	800,000
Execution time per 1000 epoch in Stage2 (s)	13.4	20.2	45.6	145.4	219.7

Appendix

Table 5: Number of epochs for convergence and training time

$\epsilon$	1000	100	10	1	1e-1
MCC	0.50 $\pm$ 0.01	0.50 $\pm$ 0.01	0.50 $\pm$ 0.01	0.50 $\pm$ 0.01	0.90 $\pm$ 0.08
$\epsilon$	1e-2	1e-3	1e-4	1e-5
MCC	0.97 $\pm$ 0.01	0.94 $\pm$ 0.03	0.90 $\pm$ 0.07	0.88 $\pm$ 0.07

Appendix

Table 6: Sensitivity analysis of sparsity level

\epsilon

in Eq. (5): Average MCCs over 5 random seeds (Setup:

n=10

k=1

m=10

\rho=50

\delta=0

\theta=0

)

Learning rate	1e-2	1e-3	1e-4	1e-5
MCC	0.25 $\pm$ 0.03	0.53 $\pm$ 0.08	0.97 $\pm$ 0.01	0.92 $\pm$ 0.02

Appendix

Table 7: Sensitivity analysis of learning rate: Average MCCs over 5 random seeds

Table 8: Ablation study without sparsity constraints and baselines: Average MCCs over 5 random seeds. Ours = proposed method, WO = without sparsity constraints, Oracle ( Xu et al., 2024 ) , VaDE ( Kivva et al., 2022 )

$n$	$k$	$m$	$\rho$	$\delta$	$\theta$	MCC of VaDE	MCC of Our Method
5	1	10	50 %	0	0	0.66 $\pm$ 0.09	0.94 $\pm$ 0.01
10	1	10	50 %	0	0	0.78 $\pm$ 0.05	0.94 $\pm$ 0.01
20	1	10	50 %	0	0	0.86 $\pm$ 0.04	0.93 $\pm$ 0.01
40	1	10	50 %	0	0	0.84 $\pm$ 0.07	0.93 $\pm$ 0.01
10	0	10	50 %	0	0	0.86 $\pm$ 0.05	0.95 $\pm$ 0.01
10	1	10	50 %	0	0	0.78 $\pm$ 0.05	0.94 $\pm$ 0.01

Appendix

Table 9: Results for the numerical experiments (Stage 1) on nondegenerate GMM data. The bold font indicates which parameters are varying in each block of rows.

$\hat{n}$	12(n+2)	14(n+4)	16(n+6)	18(n+8)	20(n+10)
R2	0.91	0.89	0.88	0.88	0.83
MCC	0.96	0.96	0.93	0.96	0.96

Appendix

Table 10: Average R2 and MCC over 5 random seeds

$n$	$k$	$m$	$\rho$	$\delta$	$\theta$	SHD ( $\mathbf{z}$ )	SHD ( $\hat{\mathbf{z}}$ )	$\Delta$ SHD
5	1	10	50 %	0	0	1.4 $\pm$ 1.67	3.8 $\pm$ 1.79	2.4
10	1	10	50 %	0	0	1.8 $\pm$ 1.79	4.2 $\pm$ 1.10	2.4
20	1	10	50 %	0	0	7.8 $\pm$ 2.86	22 $\pm$ 12.90	14.2
40	1	10	50 %	0	0	13.6 $\pm$ 4.04	39.8 $\pm$ 17.75	26.2
10	1	10	50 %	0	0	1.8 $\pm$ 1.79	4.2 $\pm$ 1.10	2.4
10	2	10	50 %	0	0	13.2 $\pm$ 2.17	16 $\pm$ 2.64	2.8

Appendix

Table 11: The average SHD on the CPDAG learned over our

\hat{\mathbf{z}}

and

\mathbf{z}

w.r.t. the oracle CPDAG.

Figure 10: Example images of multiple ball dataset, modified based on the rendering code provided by Ahuja et al. (2022) . Balls can move freely inside the frame, and each ball’s coordinates can be fixed randomly to a constant value, which causes their latent dimensions to be degenerate; Varying the number of balls from 2 to 6.

Layer Type	Configuration	Output Shape
Input	RGB image	(3, 64, 64)
Encoder
Conv2D + ReLU	3 $\rightarrow$ 32, kernel=3, stride=1, pad=1	(32, 64, 64)
MaxPool2D	2 $\times$ 2	(32, 32, 32)
Conv2D + ReLU	32 $\rightarrow$ 64, kernel=3, stride=1, pad=1	(64, 32, 32)
MaxPool2D	2 $\times$ 2	(64, 16, 16)

Appendix

Table 12: Architecture of the convolutional autoencoder used in Sec. 5 (input size:

3\times 64\times 64

	Ball 1	Ball 2
MCC	0.958 $\pm$ 0.052	0.914 $\pm$ 0.148

Appendix

Table 13: Results for Multiple Balls with

\mathbf{b}=2

: averaged

R^{2}

of linear regression between ground truth ball positions and learned representations.

	Ball 1	Ball 2	Ball 3	Ball 4
MCC	0.953 $\pm$ 0.033	0.853 $\pm$ 0.194	0.932 $\pm$ 0.041	0.880 $\pm$ 0.137

Appendix

Table 14: Results for Multiple Balls with

\mathbf{b}=4

: averaged

R^{2}

of linear regression between ground truth ball positions and learned representations.

	Ball 1	Ball 2	Ball 3
MCC	0.743 $\pm$ 0.106	0.105 $\pm$ 0.046	0.293 $\pm$ 0.229
	Ball 4	Ball 5	Ball 6
MCC	0.407 $\pm$ 0.375	0.474 $\pm$ 0.183	0.595 $\pm$ 0.241

Appendix

Table 15: Results for Multiple Balls with

\mathbf{b}=6

: averaged

R^{2}

of linear regression between ground truth ball positions and learned representations.

	$\mathbf{b}_{1}^{x}$	$\mathbf{b}_{1}^{y}$	$\mathbf{b}_{2}^{x}$	$\mathbf{b}_{2}^{y}$
$\hat{\mathbf{z}}_{1}$	0.03	0.00	0.98	0.00
$\hat{\mathbf{z}}_{2}$	0.95	0.01	0.01	0.00
$\hat{\mathbf{z}}_{3}$	0.00	0.00	0.00	0.97
$\hat{\mathbf{z}}_{4}$	0.02	0.99	0.00	0.01

Appendix

Table 16: Example results of

\hat{z}_{i}

assignments across blocks

$n$	$k$	$m$	$\rho$	MCC
				Exponential	Gumbel	Smooth LeakyReLU	Sigmoid
5	1	10	50 %	0.92 $\pm$ 0.06	0.95 $\pm$ 0.01	0.97 $\pm$ 0.01	0.36 $\pm$ 0.02
10	1	10	50 %	0.96 $\pm$ 0.01	0.97 $\pm$ 0.01	0.98 $\pm$ 0.01	0.32 $\pm$ 0.06
20	1	10	50 %	0.94 $\pm$ 0.02	0.95 $\pm$ 0.02	0.94 $\pm$ 0.09	0.29 $\pm$ 0.04
10	0	10	50 %	0.95 $\pm$ 0.01	0.97 $\pm$ 0.01	0.95 $\pm$ 0.08	0.37 $\pm$ 0.06
10	1	10	50 %	0.96 $\pm$ 0.01	0.97 $\pm$ 0.01	0.98 $\pm$ 0.01	0.32 $\pm$ 0.06

Appendix

Table 17: Results for the numerical experiments (Stage 2) on misspecified settings.

$n$	2	5	10
$R^{2}$	0.87	0.70	0.64
MCC	0.92	0.63	0.47

Appendix

Table 18: Average

R^{2}

and MCC over 5 random seeds

$k$	0	1	2	3
$R^{2}$	0.83	0.82	0.84	0.85

Appendix

Table 19: Average

R^{2}

over 5 random seeds

	$[\mathbf{z}_{0},\mathbf{z}_{1}]$	$[\mathbf{z}_{2},\mathbf{z}_{3}]$	$[\mathbf{z}_{4},\mathbf{z}_{5}]$	$[\mathbf{z}_{6},\mathbf{z}_{7}]$	$[\mathbf{z}_{8},\mathbf{z}_{9}]$
$\hat{\mathbf{z}}_{0}$	0.95	0.01	0.01	0.02	0.01
$\hat{\mathbf{z}}_{1}$	0.90	0.00	0.00	0.00	0.00
$\hat{\mathbf{z}}_{2}$	0.01	0.84	0.01	0.01	0.00
$\hat{\mathbf{z}}_{3}$	0.00	0.84	0.00	0.00	0.01
$\hat{\mathbf{z}}_{4}$	0.00	0.00	0.91	0.00	0.00
$\hat{\mathbf{z}}_{5}$	0.00	0.00	0.91	0.00	0.01

Appendix

Table 20: Example results of

\hat{\mathbf{z}}_{i}

assignments across blocks

	$[\mathbf{z}_{0}]$	$[\mathbf{z}_{1}]$	$[\mathbf{z}_{2}]$	$[\mathbf{z}_{3},\mathbf{z}_{4},\mathbf{z}_{5}]$	$[\mathbf{z}_{6},\mathbf{z}_{7},\mathbf{z}_{8},\mathbf{z}_{9}]$
$\hat{\mathbf{z}}_{0}$	0.95	0.00	0.00	0.01	0.03
$\hat{\mathbf{z}}_{1}$	0.00	0.89	0.00	0.00	0.00
$\hat{\mathbf{z}}_{2}$	0.01	0.00	0.92	0.01	0.02
$\hat{\mathbf{z}}_{3}$	0.00	0.00	0.00	0.91	0.02
$\hat{\mathbf{z}}_{4}$	0.01	0.00	0.00	0.94	0.01
$\hat{\mathbf{z}}_{5}$	0.00	0.00	0.00	0.92	0.00

Appendix

Table 21: Example results of

\hat{\mathbf{z}}_{i}

assignments across blocks

	$[\mathbf{z}_{0}]$	$[\mathbf{z}_{1}]$	$[\mathbf{z}_{2},\mathbf{z}_{3}]$	$[\mathbf{z}_{4},\mathbf{z}_{5},\mathbf{z}_{6},\mathbf{z}_{7},\mathbf{z}_{8},\mathbf{z}_{9}]$
$\hat{\mathbf{z}}_{0}$	0.92	0.00	0.01	0.05
$\hat{\mathbf{z}}_{1}$	0.00	0.89	0.00	0.01
$\hat{\mathbf{z}}_{2}$	0.01	0.00	0.91	0.04
$\hat{\mathbf{z}}_{3}$	0.00	0.00	0.88	0.01
$\hat{\mathbf{z}}_{4}$	0.01	0.00	0.00	0.94
$\hat{\mathbf{z}}_{5}$	0.01	0.00	0.01	0.91

Appendix

Table 22: Example results of

\hat{\mathbf{z}}_{i}

assignments across blocks

Causal Representation Learning

Causal representation learning seeks latent features that reveal underlying causal variables and mechanisms for robust prediction, intervention, and transfer.

See all 101 papers

Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing
May 18, 2026Manal Benhamza, Marianne Clausel, Myriam TamiLatent Variable Models Causal Representation Learning
Paris-Saclay University, CentraleSupélec, MICS Lab, France · Lorraine University, CRAN, France
Causal Representation Learning from General Environments under Nonparametric Mixing
Apr 26, 2026Ignavier Ng, Shaoan Xie, Xinshuai Dong +2Causal Representation Learning Causal Discovery
1Carnegie Mellon University · 2Mohamed bin Zayed University of Artificial Intelligence
Beyond identifiability: Learning causal representations with few environments and finite samples
Mar 26, 2026Inbeom Lee, Tongtong Jin, Bryon AragamCausal Representation Learning Causal Structure Learning

May 18, 2026J/K move · Enter open · S save

Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing

Manal Benhamza, Marianne Clausel, Myriam Tami

Paris-Saclay University, CentraleSupélec, MICS Lab, France · Lorraine University, CRAN, France

Latent Variable Models Causal Representation Learning Causal Identifiability Multimodal Learning

Open paper AskPDF arXiv

Causal representation learning (CRL) seeks to uncover meaningful latent variables and their corresponding causal structure from high-dimensional observational data. Although its significance, CRL identifiability remains a crucial property, as it ensures the recovery of the mechanisms behind the data generation process, and hence the interpretability and robustness of the representation. Proving identifiability in CRL is intrinsically difficult, and we address in this work an even more challenging setting: multimodality. We consider multimodal observed data with a latent partially shared structure. Each modality is generated, through non linear mixing functions, from a specific subset of causal latent variables. Under flexible assumptions and without imposing any parametric distribution on the latent variables, we establish component-wise identifiability guarantees for the causal latent representation. Our identifiability results, furthermore, apply to the undercomplete scenario where we have, for each modality, more observed than latent variables. To instantiate our theoretical analysis, we introduce a Wasserstein-based module to recover the partially shared latent structure. Due to its differentiability, the latter can be easily integrated into all types of architecture, only requiring minimal changes. Extensive experiments on synthetic and realistic datasets validate the superiority of our approach over SOTA methods.

Abstract

Causal Representation Learning

Explore similar work

Identifiable Multimodal Causal Representation Learning under Partial Latent Sharing

Causal Representation Learning from General Environments under Nonparametric Mixing

Beyond identifiability: Learning causal representations with few environments and finite samples