Structure-agnostic Causal Representation Learning

Arman Behnam; Binghui Wang

cs.LGOct 1, 2026

Structure-agnostic Causal Representation Learning

Authors: Arman Behnam, Binghui Wang

Organizations: Department of Computer Science Illinois Institute of Technology Chicago, IL, USA

Abstract

Causal representation learning aims to discover robust features by exploiting the causal structure underlying data generation. Existing methods require specifying the causal structure a priori, yet different structures demand fundamentally incompatible invariance constraints, and misspecification leads to representations that discard predictive information. We introduce SaCRL, a framework that jointly identifies the causal structure and learns the corresponding invariant representation without prior structural knowledge. Our approach formulates structure selection as a soft optimization over candidate invariances using HSIC-based violation metrics, with adaptive weights that automatically concentrate on the achievable structure. We provide theoretical guarantees for structure identification, including under random-feature approximation, invariance satisfaction, and out-of-distribution generalization. Empirically, SaCRL recovers the true structure on synthetic and semi-synthetic Bayesian-network benchmarks, outperforms fixed-invariance baselines on Colored MNIST, achieves state-of-the-art accuracy on three DomainBed benchmarks (PACS, VLCS, OfficeHome), and degrades gracefully under structural misspecification and limited environment diversity. Code is available at: https://github.com/ArmanBehnam/sacrl.

Figures & tables

Panel of Figure 1 — Figure 1 : Clinical toy example.

True Structure	$n=500$	$n=1000$	$n=2000$	$n=5000$
$\mathcal{G}_{1}$ (anti-causal)	73.5 (2.8)	85.0 (2.1)	92.5 (1.4)	96.5 (0.9)
$\mathcal{G}_{2}$ (conf-desc)	68.0 (3.2)	81.5 (2.5)	89.0 (1.8)	95.0 (1.1)
$\mathcal{G}_{3}$ (conf-out)	78.5 (2.4)	88.5 (1.9)	94.0 (1.2)	97.5 (0.7)

Table 1 : Structure identification accuracy (%) across sample sizes and true structures. Results averaged over 20 random seeds. Standard deviations in parentheses.

Benchmark	Network (nodes)	$Y$	True	Violation	ID acc. (%)	Margin
asia_g3	ASIA (8)	either	$\mathcal{G}_{3}$	56.3	95.0 (19/20)	0.45
asia_g2	ASIA (8)	either	$\mathcal{G}_{2}$	24.6	90.0 (18/20)	0.42
alarm_co	ALARM (37)	CO	$\mathcal{G}_{3}$	28.3	85.0 (17/20)	0.35
sachs_g2	SACHS (11)	Akt	$\mathcal{G}_{2}$	6.4	85.0 (17/20)	0.25
alarm_g2	ALARM (37)	EXPCO2	$\mathcal{G}_{2}$	4.2	75.0 (15/20)	0.18

Table 2 : Semi-synthetic Bayesian-network benchmarks. Violation: the competing invariance’s violation before training, measured in standard deviations of the permutation null. Margin:

\alpha_{(1)}-\alpha_{(2)}

Method	OOD Acc (%)	ID. Stru.
ERM	27.2 $\pm$ 11.3	—
IRM	21.3 $\pm$ 13.2	$\mathcal{G}_{2}$ (fixed)
VREx	24.3 $\pm$ 12.8	—
CIRCE	10.3 $\pm$ 0.5	$\mathcal{G}_{1}$ (fixed)
SaCRL	45.2 $\pm$ 10.0	$\mathcal{G}_{1}$ (92%)

Table 3: Colored MNIST OOD Acc. on anti-correlated test environment.

Held-out	$\alpha_{1}$	$\alpha_{2}$	$\alpha_{3}$	$\hat{k}$	Margin	$p$	Seeds	Cal. acc. (%)
Photo	0.21	0.68	0.11	$\mathcal{G}_{2}$	0.47	0.008	5/5	86.1
Art Painting	0.59	0.31	0.10	$\mathcal{G}_{1}$	0.28	0.021	5/5	65.0
Cartoon	0.35	0.52	0.13	$\mathcal{G}_{2}$	0.17	0.037	4/5	65.0
Sketch	0.63	0.25	0.12	$\mathcal{G}_{1}$	0.38	0.014	5/5	86.1

Table 4 : Structure selected on PACS; each row trains on the other three domains (5 seeds). Margin:

\alpha_{(1)}-\alpha_{(2)}

;

p

: permutation

p

-value of

\min_{k}V_{k}

; Seeds: agreement of

\hat{k}

across seeds; Cal. acc.: identification accuracy of synthetic runs in the same margin bin (Appendix H.5 ).

	PACS					VLCS					OfficeHome
Method	Photo	Art	Cartoon	Sketch	Avg	Caltech	LabelMe	SUN09	VOC	Avg	Art	Clipart	Product	Real	Avg
ERM	93.2	66.0	51.8	37.6	62.2	94.0	60.2	69.9	73.0	74.3	49.9	41.7	66.7	67.7	56.5
IRM	93.2	66.2	51.7	37.6	62.2	95.0	60.1	70.1	72.6	74.5	49.0	41.2	66.5	67.7	56.1
VREx	93.0	65.5	51.6	37.9	62.0	93.3	60.2	69.8	71.0	73.6	48.6	41.2	66.2	67.6	55.9
CIRCE	92.5	64.5	51.4	37.3	61.4	93.8	60.0	67.7	69.7	72.8	49.0	41.9	66.0	67.4	56.1
SaCRL	94.0	68.5	54.0	40.5	64.3	97.7	64.4	75.7	77.1	78.7	52.9	46.5	69.1	69.8	59.6

Table 5: Results on DomainBed benchmarks: average leave-one-domain-out OOD accuracy (%).

Misspecification ( $\delta_{\min}{=}1.0$ ):
Setting	ERM	IRM	VREx	CIRCE	SaCRL	ID. Stru.
$\nu{=}0.25$ (near-causal)	34.4 $\pm$ 2.7	—	—	—	50.2 $\pm$ 3.7	$\mathcal{G}_{2}$ (4/5)
$\nu{=}0.50$ (bidirectional)	61.2 $\pm$ 3.8	61.3 $\pm$ 4.2	61.4 $\pm$ 4.4	66.5 $\pm$ 2.6	73.9 $\pm$ 4.1	$\mathcal{G}_{2}$ (5/5)
$\nu{=}0.75$ (near-anticausal)	98.5 $\pm$ 0.6	—	—	—	99.6 $\pm$ 0.3	$\mathcal{G}_{1}$ (4/5)
Limited diversity ( $\nu{=}0.50$ ):
$\delta_{\min}{=}1.0$ (full)	61.2 $\pm$ 3.8	61.3 $\pm$ 4.2	61.4 $\pm$ 4.4	66.5 $\pm$ 2.6	73.9 $\pm$ 4.1	$\mathcal{G}_{2}$ (5/5)

Table 6 : Robustness under misspecification (

\nu

) and limited diversity (

\delta_{\min}

). OOD accuracy (%) over 5 seeds. ID. Stru.: majority structure (seeds agreeing).

Variant	Stru. ID (%)	OOD Acc (%)
Full method	89.0	84.2
Hard $\arg\min_{k}$	79.5	79.8
Fixed $\beta=10$	83.0	81.5
Fixed $\lambda=0.1$	72.5	78.3
Fixed $\lambda=5.0$	88.5	81.0
Uniform ( $\rho{=}1$ )	88.0	82.1

Table 7 : Ablation on

\mathcal{G}_{2}

(

n{=}2000

). Default: SoftMin,

\beta

annealing

1{\to}50

\lambda{=}1.0

\rho{=}0.5

Appendix figures & tables13 assets

Supplementary material from the paper’s appendix.

Appendix

Spaces & Sets		Random Variables & Data
$\mathcal{X},\mathcal{Y},\mathcal{Z}$	Input, output, representation spaces	$X,Y,E$	Features, target, environment
$\mathcal{E}$	Environment set $\{e_{1},\ldots,e_{m}\}$	$U,Z$	Latent confounder, mediator
$(\Omega,\mathscr{F},P)$	Probability space	$X_{Z},X_{Z}^{\perp}$	Mediated, orthogonal components
$\mathcal{H},\mathcal{H}_{\mathcal{X}}$	RKHS, RKHS on $\mathcal{X}$	$D_{e}$	Dataset $\{(x_{j}^{e},y_{j}^{e})\}_{j=1}^{n_{e}}$
$\Delta^{2}$	Probability simplex	$n,m,d$	Samples, environments, dimension
Causal Structure		Kernels & Independence

Appendix

Table 8 : Notation summary.

Parameter	Default	Range	Selection Strategy
$\lambda$	1.0	$[0.5,5]$	Per experiment, by OOD accuracy (oracle selection)
$\beta_{0}$	1.0	$[0.5,2]$	Fixed (soft selection initially)
$\beta_{\max}$	50	$[20,100]$	Higher for sharper selection
$\gamma$	$n^{-1/3}$	$[n^{-1/2},n^{-1/5}]$	Cross-validation for kernel regression
$\rho$	0.5	$[1/\|\mathcal{E}\|,1]$	Smaller $\rho$ when all environments are trusted
$D$	1000	$[500,2000]$	Increase if approximation error high

Appendix

Table 9: Recommended hyperparameter settings for Algorithm 1 .

Setting	$\lambda{=}0.5$	$\lambda{=}1.0$	$\lambda{=}2.0$	$\lambda{=}5.0$
$\nu{=}0.25$ (near-causal)	46.0	46.8	47.8	50.2
$\nu{=}0.50$ (bidirectional)	69.0	70.3	70.4	73.9
$\nu{=}0.75$ (near-anticausal)	99.7	99.5	99.5	99.6

Appendix

Table 10 : Sensitivity to invariance penalty weight

\lambda

(

\beta_{\max}=50

fixed). OOD accuracy (%).

Setting	$\beta_{\max}{=}20$	$\beta_{\max}{=}50$	$\beta_{\max}{=}100$
$\nu{=}0.25$ (near-causal)	50.3	50.2	51.3
$\nu{=}0.50$ (bidirectional)	71.4	73.9	72.9
$\nu{=}0.75$ (near-anticausal)	99.6	99.6	99.6

Appendix

Table 11 : Sensitivity to SoftMin temperature

\beta_{\max}

(

\lambda=5.0

fixed). OOD accuracy (%).

$\nu$	$\alpha_{1}$	$\alpha_{2}$	$\alpha_{3}$	SoftMin	Hard $\arg\min$	Perm. $p$
0.40	0.407 $\pm$ 0.041	0.489 $\pm$ 0.080	0.104 $\pm$ 0.058	59.5 $\pm$ 2.0	56.5 $\pm$ 2.7	0.013 $\pm$ 0.018
0.45	0.415 $\pm$ 0.073	0.478 $\pm$ 0.139	0.107 $\pm$ 0.075	66.2 $\pm$ 1.5	62.5 $\pm$ 2.9	0.011 $\pm$ 0.012
0.50	0.421 $\pm$ 0.089	0.422 $\pm$ 0.143	0.157 $\pm$ 0.071	72.4 $\pm$ 1.5	68.6 $\pm$ 3.6	0.181 $\pm$ 0.011
0.55	0.484 $\pm$ 0.079	0.417 $\pm$ 0.143	0.099 $\pm$ 0.077	79.8 $\pm$ 2.0	76.5 $\pm$ 2.2	0.033 $\pm$ 0.039
0.60	0.549 $\pm$ 0.069	0.346 $\pm$ 0.126	0.105 $\pm$ 0.077	87.4 $\pm$ 1.9	84.7 $\pm$ 2.2	0.016 $\pm$ 0.033

Appendix

Table 12 : Adaptive weights and OOD accuracy (%) across the

\mathcal{G}_{2}

–

\mathcal{G}_{1}

crossover.

Margin	Runs	Correct	ID accuracy (%)
$<0.1$	48	20	41.7
$0.1$ – $0.3$	60	39	65.0
$0.3$ – $0.5$	72	62	86.1
$>0.5$	60	57	95.0

Appendix

Table 13 : Identification accuracy by weight margin on synthetic data.

Variant	OOD Acc (%)
No splitting (default)	73.9 $\pm$ 2.7
Cross-fitted (2-fold)	74.0 $\pm$ 2.7
ERM (reference)	61.2 $\pm$ 4.5

Appendix

Table 14 : Cross-fitting comparison on the bidirectional DGP (

\nu=0.5

n{=}1000

per environment). Independent run of the default configuration.

Momentum $m$	Stru. ID (%)	OOD Acc (%)
0 (default)	89.0	84.2
0.99	89.4	84.3
0.999	89.8	84.5

Appendix

Table 15 : EMA conditioning target on synthetic

\mathcal{G}_{2}

(

n{=}2000

m{=}0

is the default online encoder.

$D$	100	250	500	1000	2000
Mean $\|\widehat{\mathrm{HSIC}}_{\mathrm{RFF}}-\widehat{\mathrm{HSIC}}\|$ ( $\times 10^{-4}$ )	14.2	12.9	8.4	7.7	4.4

Appendix

Table 16 : RFF approximation error versus exact HSIC (synthetic,

n{=}2000

$m$	Stru. ID. Acc (%)	OOD Acc (%)
2	71.5 $\pm$ 4.2	74.3 $\pm$ 2.8
3	89.0 $\pm$ 1.6	84.2 $\pm$ 1.3
4	96.5 $\pm$ 1.2	85.1 $\pm$ 1.1
5	97.0 $\pm$ 0.9	85.4 $\pm$ 0.9

Appendix

Table 17 : Effect of number of training environments

m

on synthetic

\mathcal{G}_{2}

data (

n{=}2000

Method	Time (min)	GPU Memory	Relative
ERM	8.3	2.1 GB	$1.0\times$
IRM	9.8	2.4 GB	$1.18\times$
SaCRL (RFF, $D{=}1000$ )	10.9	2.7 GB	$1.31\times$
SaCRL (exact HSIC)	33.2	11.8 GB	$4.0\times$

Appendix

Table 18 : Training time and memory for 100 epochs on Colored MNIST (10K samples).

Component	Time (s)	Fraction
Forward pass	4.59	42%
HSIC computation (RFF)	3.02	28%
Kernel regression	1.75	16%
Backward pass	1.19	11%
Other	0.34	3%
Total	10.9	100%

Appendix

Table 19 : Runtime breakdown per epoch on PACS.

Method	Frozen	Fine-tuned	$\Delta$
ERM	93.2	95.1	+1.9
IRM	93.2	95.2	+2.0
VREx	93.0	94.9	+1.9
CIRCE	92.5	94.1	+1.6
SaCRL	94.0	95.0	+1.0

Appendix

Table 20 : PACS (Photo held out): frozen vs. fine-tuned ResNet18. OOD accuracy (%), 3 seeds.

Explore similar work

A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation
May 20, 2026Yan Li, Yuewen Sun, Shaoan Xie +4Causal Representation Learning Representation Learning
1Mohamed bin Zayed University of Artificial Intelligence · 2Carnegie Mellon University
Sanity Checking Causal Representation Learning on a Simple Real-World System
Feb 27, 2025Juan L. Gamella, Simon Bing, Jakob RungeCausal Representation Learning Causal
Seminar for Statistics, ETH Zurich · Technische Universität Berlin · Department of Computer Science, University of Potsdam +1
Robust Representation Learning through Explicit Environment Modeling
Apr 28, 2026Yuli Slavutsky, David M. BleiCausal Representation Learning Representation Learning
Department of Statistics Columbia University · Departments of Statistics and Computer Science Columbia University

May 20, 2026, cs.LGJ/K move · Enter open · S save

A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation

Yan Li, Yuewen Sun, Shaoan Xie, Gongxu Luo+3

1Mohamed bin Zayed University of Artificial Intelligence · 2Carnegie Mellon University

Causal Representation Learning Representation Learning Causal Unified Formulation Choice

Open paper AskPDF arXiv

Causal representation learning (CRL) and traditional representation learning have largely developed along different trajectories. Traditional representation learning has been driven mainly by applications and empirical objectives, whereas CRL has focused more on theoretical questions, particularly identifiability. This difference in emphasis has created a gap between the two fields in terminology, problem formulation, and evaluation, limiting communication and sometimes leading to disconnected or redundant efforts. In this paper, we argue that these two fields should be brought into dialogue rather than treated as separate paradigms. To this end, we introduce a unified formulation in which the representation learning is characterized by two components: a task component, which specifies what information the learned representation is required to preserve, and a constraint component, which specifies what structure is imposed on the latent space. Under this formulation, the benefits run in both directions. CRL provides theoretical tools for understanding when structured latent constraints are useful or necessary, while traditional representation learning offers practical insights on task design and objective choice that can improve the development of CRL methods. To illustrate this interaction, we experimentally study how different task components affect the behavior of CRL methods under different structured constraints. Results on CausalVerse show that the effectiveness of causal constraints depends strongly on the tasks with which they are paired.

Abstract

Explore similar work

A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation

Sanity Checking Causal Representation Learning on a Simple Real-World System

Robust Representation Learning through Explicit Environment Modeling