Causal Representation Learning with Instantaneous and Lagged Relations via Nonstationarity

Tatsuya Yamada; Hiroshi Morioka; Yoshinobu Kawahara

cs.LGOct 2, 2026

Causal Representation Learning with Instantaneous and Lagged Relations via Nonstationarity

Authors: Tatsuya Yamada, Hiroshi Morioka, Yoshinobu Kawahara

Organizations: The University of Osaka · RIKEN · Shiga University

Abstract

Causal representation learning for time-series data aims to identify latent states and their causal relations from observations. In this setting, an important challenge is to model both lagged causal relations across observation intervals and faster causal effects that appear as instantaneous relations within an interval, while accounting for nonstationarity in time-series data. However, methods that jointly handle these causal relations and nonstationarity remain limited. To address this gap, we establish sufficient conditions for identifying latent states up to component permutation and component-wise invertible transformations, and their instantaneous and lagged causal structures up to the same permutation, using an observed auxiliary variable, such as time or a condition label, associated with changes in transition-noise distributions. Based on these results, we propose iCReN, a framework that uses contrastive learning with discrete or continuous auxiliary variables to learn latent representations and estimate their instantaneous and lagged causal structures. Experiments demonstrate accurate recovery of latent states and both instantaneous and lagged causal structures on synthetic data and the utility of the learned representations for downstream forecasting on real-world data.

Figures & tables

	Instantaneous causal structure identification	Nonstationarity	No intervention-target labels required	No variational inference required
LEAP ( Yao et al., 2022b )	$\times$	✓	✓	$\times$
iCITRIS ( Lippe et al., 2023 )	✓	✓*	$\times$	✓
G-CaRL ( Morioka and Hyvärinen, 2024 )	$\times$	$\times$	✓	✓
IDOL ( Li et al., 2025b )	✓	$\times$	✓	$\times$
iCReN	✓	✓	✓	✓

Table 1: Comparison of closely related CRL methods for time-series data by the causal relations they identify, the information they use for identification, and how they learn representations. For iCITRIS (*), the distribution changes are induced by interventions with known targets.

Figure 1: Generative model with

d_{z}=2

and

L=1

. Gray dash-dotted, solid black, red, blue, and dashed black arrows denote conditioning of the transition-noise distributions on the auxiliary variable, transition-noise inputs to the structural functions, instantaneous causal relations, lagged causal relations, and the observation mapping, respectively.

Figure 2: Learning framework. Gray nodes denote observed variables; white nodes denote learned representations, estimated transition noise, and history features. Dashed arrows denote auxiliary-variable inputs to TCL/GCL. The encoder

\phi

is shared across time, and blue modules are trained jointly using the orange objectives.

	Discrete variation			Continuous variation
Method	MCC	Inst. F1	Lagged F1	MCC	Inst. F1	Lagged F1
iCReN-T	$\mathbf{0.951\pm 0.011}$	$\mathbf{0.950\pm 0.158}$	$\mathbf{0.980\pm 0.063}$	$\mathbf{0.959\pm 0.010}$	$\mathbf{0.975\pm 0.079}$	$\mathbf{0.970\pm 0.095}$
iCReN-G	$0.940\pm 0.010$	$0.700\pm 0.158$	$\mathbf{0.980\pm 0.042}$	$0.945\pm 0.013$	$\mathbf{0.975\pm 0.079}$	$0.950\pm 0.071$
IDOL	$0.468\pm 0.046$	$0.225\pm 0.142$	$0.480\pm 0.079$	$0.534\pm 0.090$	$0.225\pm 0.142$	$0.520\pm 0.123$
G-CaRL	$0.764\pm 0.065$	--	$0.440\pm 0.117$	$0.762\pm 0.047$	--	$0.340\pm 0.084$
CtrlNS	$0.432\pm 0.071$	--	$0.600\pm 0.000$	$0.398\pm 0.102$	--	$0.600\pm 0.000$

Table 2: Comparison of the proposed methods with baselines on synthetic data. The left and right blocks report results for discrete-variation data and continuous-variation data, respectively. Each entry is the mean

\pm

standard deviation over 10 seeds; boldface indicates the best value in each column, and -- indicates a metric not applicable to the method.

	Treadmill: marker error [mm] $\downarrow$				Weather: standardized MSE $\downarrow$
Method	0.48 s	0.96 s	1.44 s	1.92 s	AU	CAN	CN	FR	GER	JP	KR	US
iCReN	$\mathbf{42.060}$	$\mathbf{58.650}$	$\mathbf{82.034}$	$\mathbf{107.309}$	$\mathbf{0.352}$	$0.481$	$\mathbf{0.546}$	$\mathbf{0.422}$	$0.474$	$\mathbf{0.414}$	$\mathbf{0.291}$	$0.427$
IDOL	$44.076$	$62.400$	$86.791$	$115.146$	$0.359$	$0.484$	$0.555$	$0.438$	$\mathbf{0.469}$	$0.440$	$0.304$	$0.440$
G-CaRL	$52.197$	$75.790$	$103.133$	$135.169$	$0.649$	$0.673$	$0.664$	$0.750$	$0.754$	$0.615$	$0.535$	$0.554$
CtrlNS	$43.593$	$62.337$	$86.040$	$117.059$	$0.385$	$0.522$	$0.618$	$0.459$	$0.504$	$0.501$	$0.382$	$0.462$
LEAP ∗	$44.220$	$63.431$	$87.338$	$113.210$	$0.396$	$0.524$	$0.591$	$0.486$	$0.566$	$0.482$	$0.327$	$0.460$

Table 3: Forecasting results on real-world data. Entries are means across runs; lower is better. Treadmill reports mean marker error in mm, and weather reports standardized MSE for three-hour-ahead relative-humidity forecasting. Weather columns are southeastern Australia (AU), Canada Great Lakes–St. Lawrence (CAN), China (CN), France (FR), Germany (GER), Japan (JP), South Korea (KR), and US Atlantic (US). Boldface marks the best result in each column.

Appendix figures & tables17 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Illustration of how cross-time relations resolve intimate-neighbor ambiguities. The left panel shows a four-component latent causal model, and the right panel shows the corresponding instantaneous and cross-time pairwise Markov networks. In the instantaneous network,

z_{t,2}\in\Psi_{\mathcal{M}_{z}}(z_{t,1})

and

z_{t,3}\in\Psi_{\mathcal{M}_{z}}(z_{t,4})

, as indicated by the red edges. The blue cross-time relations resolve the ambiguity of the first pair through

z_{t-1,1}

and that of the second through

z_{t+1,4}

, illustrating that either a past or a future relation can suffice for each intimate-neighbor pair. Dotted edges denote conditional-dependence relations that are not direct causal edges. All Markov-network relations are defined conditionally on the auxiliary variable.

Figure 4: Instantaneous and lagged causal graphs in the default five-component setting and their adjacency matrices. Red and blue indicate instantaneous and lagged causal edges, respectively; rows and columns index targets and sources.

Method	Optimization
IDOL	Adam, initial learning rate $10^{-3}$ ; multiplied by 0.1 at epoch 120
G-CaRL	SGD, momentum 0.9, initial learning rate $10^{-2}$ ; multiplied by 0.1 after 80% of updates
CtrlNS	AdamW, initial learning rate $10^{-3}$
iVAE / $\beta$ -VAE	AdamW, initial learning rate $10^{-2}$ ; multiplied by 0.1 at epoch 120
LEAP	AdamW (model/transition) and SGD (discriminator), initial learning rate $2\times 10^{-3}$

Appendix

Table 4: Main optimization settings for the baselines in the synthetic experiments.

Method	Discrete variation	Continuous variation
iCReN-T	Discrete environment $u_{t}$	256-bin pseudo-label $\widetilde{u}_{t}$
iCReN-G	Discrete environment $u_{t}$	Continuous time $t$
IDOL	None	None
G-CaRL	None	None
CtrlNS	None	None
LEAP	Discrete environment $u_{t}$	256-bin pseudo-label $\widetilde{u}_{t}$

Appendix

Table 5: Auxiliary variables supplied to each method in the synthetic experiments.

Panel of Figure 5 — Figure 5: Jacobian edge scores in the default setting with Gaussian transition-noise distributions that vary across discrete environments: (a) instantaneous and (b) lagged. In each panel, the left matrix is the true support, and the middle and right matrices show seeds 850 and 840 trained with the full objective. Both seeds recover the same support, although their score scales differ under component-wise invertible transformations. Rows and columns index targets and sources, respectively.

Instant. graph	Lagged graph	MCC	Inst. F1	Lagged F1
Default chain	Default	$0.951\pm 0.011$	$0.950\pm 0.158$	$0.980\pm 0.063$
Random DAGs	Default	$0.960\pm 0.008$	$1.000\pm 0.000$	$0.970\pm 0.067$
Default chain	Self-loops	$0.955\pm 0.009$	$0.925\pm 0.169$	$0.940\pm 0.135$
Random DAGs	Self-loops	$0.948\pm 0.032$	$0.975\pm 0.079$	$0.960\pm 0.126$

Appendix

Table 6: Latent-state and causal-graph recovery by iCReN-T under different instantaneous and lagged causal structures on synthetic data with Gaussian transition-noise distributions that vary across discrete environments. Random DAGs are distinct across seeds. Values are the mean

\pm

standard deviation over 10 seeds.

$d_{z}$	MCC	Inst. F1	Lagged F1
4	$0.958\pm 0.009$	$1.000\pm 0.000$	$1.000\pm 0.000$
5	$0.951\pm 0.011$	$0.950\pm 0.158$	$0.980\pm 0.063$
8	$0.860\pm 0.021$	$0.743\pm 0.284$	$0.869\pm 0.127$
12	$0.886\pm 0.017$	$0.527\pm 0.200$	$0.812\pm 0.118$
16	$0.792\pm 0.056$	$0.140\pm 0.058$	$0.569\pm 0.136$

Appendix

Table 7: Latent-state and causal-graph recovery by iCReN-T as the latent dimension varies on synthetic data with Gaussian transition-noise distributions that vary across discrete environments. Values are the mean

\pm

standard deviation over 10 seeds.

Figure 6: Latent-state and causal-graph recovery as total sample size varies on synthetic data with Gaussian transition-noise distributions. Orange and blue denote iCReN-T and iCReN-G , respectively; solid curves with circular markers and dashed curves with square markers denote discrete variation across environments and continuous variation over time, respectively. Shaded regions show mean

\pm

standard deviation over 10 seeds.

(a) Structural-function depth
Depth	MCC	Inst. F1	Lagged F1
1	$0.913\pm 0.012$	$0.275\pm 0.275$	$1.000\pm 0.000$
2	$0.953\pm 0.007$	$0.975\pm 0.079$	$1.000\pm 0.000$
3	$0.951\pm 0.011$	$0.950\pm 0.158$	$0.980\pm 0.063$
4	$0.952\pm 0.007$	$0.925\pm 0.237$	$0.950\pm 0.127$
5	$0.955\pm 0.006$	$1.000\pm 0.000$	$1.000\pm 0.000$

Appendix

Table 8: Latent-state and causal-graph recovery by iCReN-T as the complexity of the true data-generating functions varies on synthetic data with Gaussian transition-noise distributions that vary across discrete environments. We vary structural-function depth with a fixed three-layer observation function in (a), and observation-function depth with fixed three-layer structural functions in (b). Values are the mean

\pm

standard deviation over 10 seeds.

Method	One layer	Three layers
iCReN-T	$0.957\pm 0.008$	$0.951\pm 0.011$
iCReN-G	$0.947\pm 0.010$	$0.940\pm 0.010$
IDOL	$0.704\pm 0.061$	$0.468\pm 0.046$
G-CaRL	$0.775\pm 0.060$	$0.764\pm 0.065$
CtrlNS	$0.503\pm 0.050$	$0.432\pm 0.071$
LEAP	$0.536\pm 0.063$	$0.373\pm 0.107$

Appendix

Table 9: MCC of both proposed methods and the baselines with one- and three-layer observation functions on discrete-variation synthetic data. Each entry is the mean

\pm

standard deviation over 10 seeds.

	$d_{z}=5$			$d_{z}=8$
Objective	MCC	Inst. F1	Lagged F1	MCC	Inst. F1	Lagged F1
Full	$0.951\pm 0.011$	$0.950\pm 0.158$	$0.980\pm 0.063$	$0.860\pm 0.021$	$0.743\pm 0.284$	$0.869\pm 0.127$
No sparsity	$0.955\pm 0.007$	$1.000\pm 0.000$	$1.000\pm 0.000$	$0.867\pm 0.016$	$0.900\pm 0.191$	$0.950\pm 0.097$
No DAG	$0.957\pm 0.004$	$0.950\pm 0.105$	$0.940\pm 0.135$	$0.861\pm 0.027$	$0.786\pm 0.205$	$0.894\pm 0.135$
Neither	$0.955\pm 0.008$	$0.925\pm 0.121$	$0.890\pm 0.191$	$0.861\pm 0.017$	$0.800\pm 0.204$	$0.800\pm 0.164$

Appendix

Table 10: Ablation of structural regularization on synthetic data with Gaussian transition-noise distributions that vary across discrete environments, with

d_{z}=5

(left) and

d_{z}=8

(right). Values are the mean

\pm

standard deviation over 10 seeds.

Objective	MCC	Inst. F1	Lagged F1	Reconstruction loss
Full	$0.951\pm 0.011$	$0.950\pm 0.158$	$0.980\pm 0.063$	--
Full + reconstruction	$0.957\pm 0.009$	$1.000\pm 0.000$	$0.970\pm 0.048$	$0.0022\pm 0.0005$

Appendix

Table 11: Effect of adding a decoder and reconstruction loss to the default objective on synthetic data with Gaussian transition-noise distributions that vary across discrete environments. Values are the mean

\pm

standard deviation over 10 seeds.

Method	Pretraining objective	Auxiliary information
iCReN	Reconstruction, TCL, and sparsity regularization	12-class gait phase
IDOL	Reconstruction, KL divergence, and sparsity regularization	None
G-CaRL	Group-contrastive objective	None
CtrlNS	Variational transition objective	No observed label; at most five inferred regimes
LEAP	Variational transition, total correlation, and sparsity regularization	12-class gait phase
$\beta$ -VAE	Reconstruction and $\beta$ -weighted KL divergence	None

Appendix

Table 12: Main pretraining objectives and auxiliary information for the treadmill motion forecasting experiment. KL denotes Kullback–Leibler.

(a) Early prediction horizons
Prediction horizon [s]	0.24	0.48	0.72	0.96
iCReN	$\mathbf{35.518\pm 1.236}$	$\mathbf{42.060\pm 1.939}$	$\mathbf{47.863\pm 1.255}$	$\mathbf{58.650\pm 2.051}$
IDOL	$36.654\pm 2.106$	$44.076\pm 2.304$	$50.717\pm 1.786$	$62.400\pm 2.119$
G-CaRL	$45.942\pm 4.276$	$52.197\pm 4.917$	$60.568\pm 3.640$	$75.790\pm 7.785$
CtrlNS	$37.152\pm 1.859$	$43.593\pm 2.267$	$48.861\pm 1.610$	$62.337\pm 1.978$
LEAP	$37.281\pm 1.326$	$44.220\pm 1.825$	$50.902\pm 1.568$	$63.431\pm 2.825$

Appendix

Table 13: Mean marker error (mm) at every third predicted frame for Comfortable test sequences, which are excluded from training and model selection. Values are the mean

\pm

standard deviation across runs; boldface indicates the best value in each column. We report 10 runs for iCReN , IDOL, G-CaRL, CtrlNS, and

\beta

-VAE and the eight numerically stable runs for LEAP.

Region	Cities
Southeastern Australia	Brisbane, Sydney, Canberra, Melbourne, Hobart
Canada	Toronto, Kingston, Ottawa, Montreal, Quebec City
China	Harbin, Beijing, Shanghai, Wuhan, Guangzhou
France	Lille, Paris, Strasbourg, Bordeaux, Marseille
Germany	Hamburg, Berlin, Cologne, Frankfurt, Munich
Japan	Sapporo, Sendai, Tokyo, Osaka, Fukuoka

Appendix

Table 14: Cities used in the weather forecasting experiment. We use the ERA5

0.25^{\circ}

grid point nearest to each city center.

Method	Pretraining objective	Auxiliary information
iCReN	Reconstruction, TCL, and sparsity regularization	384-class calendar index
IDOL	Reconstruction, KL divergence, and sparsity regularization	None
G-CaRL	Group-contrastive objective	None
CtrlNS	Variational transition objective	No observed label; at most five inferred regimes
LEAP	Variational transition, total correlation, and sparsity regularization	384-class calendar index
$\beta$ -VAE	Reconstruction and $\beta$ -weighted KL divergence	None

Appendix

Table 15: Main pretraining objectives and auxiliary information for the weather forecasting experiment.

(a) Southeastern Australia, Canada, China, and France
Method	AU	CAN	CN	FR
iCReN	$\mathbf{0.352\pm 0.004}$	$0.481\pm 0.011$	$\mathbf{0.546\pm 0.010}$	$\mathbf{0.422\pm 0.015}$
IDOL	$0.359\pm 0.004$	$0.484\pm 0.010$	$0.555\pm 0.006$	$0.438\pm 0.007$
G-CaRL	$0.649\pm 0.017$	$0.673\pm 0.013$	$0.664\pm 0.009$	$0.750\pm 0.019$
CtrlNS	$0.385\pm 0.021$	$0.522\pm 0.084$	$0.618\pm 0.028$	$0.459\pm 0.061$
LEAP	$0.396\pm 0.006$	$0.524\pm 0.009$	$0.591\pm 0.012$	$0.486\pm 0.014$

Appendix

Table 16: Standardized MSE for three-hour-ahead relative-humidity forecasting in eight regions. Each region comprises five cities. Values for the learned methods are the mean

\pm

standard deviation over 10 runs; boldface indicates the best learned method in each column. The two panels partition the columns only for readability. Persistence is a deterministic forecasting reference and is shown without a standard deviation.

Explore similar work

Apr 26, 2026

Causal Representation Learning from General Environments under Nonparametric Mixing

Causal representation learning aims to recover the latent causal variables and their causal relations, typically represented by directed acyclic graphs (DAGs), from low-level observations such as image pixels. A prevailing line of research exploits multiple environments, which assume how data distributions change, including single-node interventions, coupled interventions, or hard interventions, or parametric constraints on the mixing function or the latent causal model, such as linearity. Despite the novelty and elegance of the results, they are often violated in real problems. Accordingly, we formalize a set of desiderata for causal representation learning that applies to a broader class of environments, referred to as general environments. Interestingly, we show that one can fully recover the latent DAG and identify the latent variables up to minor indeterminacies under a nonparametric mixing function and nonlinear latent causal models, such as additive (Gaussian) noise models or heteroscedastic noise models, by properly leveraging sufficient change conditions on the causal mechanisms up to third-order derivatives. These represent, to our knowledge, the first results to fully recover the latent DAG from general environments under nonparametric mixing. Notably, our results match or improve upon many existing works, but require less restrictive assumptions about changing environments.

Ignavier Ng, Shaoan Xie, Xinshuai Dong +2

1Carnegie Mellon University · 2Mohamed bin Zayed University of Artificial Intelligence

Causal Representation Learning Causal Discovery Causal Structure Learning Causal Identifiability

Ask PDF

Jan 29, 2026

TRACE: Trajectory Recovery for Continuous Mechanism Evolution in Causal Representation Learning

Temporal causal representation learning methods assume that causal mechanisms switch instantaneously between discrete domains, yet real-world systems often exhibit continuous mechanism transitions. For example, a vehicle's dynamics evolve gradually through a turning maneuver, and human gait shifts smoothly from walking to running. We formalize this setting by modeling transitional mechanisms as convex combinations of finitely many atomic mechanisms, governed by time-varying mixing coefficients. Our theoretical contributions establish that both the latent causal variables and the continuous mixing trajectory are jointly identifiable. We further propose TRACE, a Mixture-of-Experts framework where each expert learns one atomic mechanism during training, enabling recovery of mechanism trajectories at test time. This formulation generalizes to intermediate mechanism states never observed during training. Experiments on synthetic and real-world data demonstrate that TRACE recovers mixing trajectories with up to 0.99 correlation, substantially outperforming discrete-switching baselines.

Shicheng Fan, Kun Zhang, Lu Cheng

University of Illinois at Chicago · Carnegie Mellon University · Mohamed bin Zayed University of Artificial Intelligence

Causal Representation Learning Causal Identifiability

Ask PDF

May 20, 2026

A Dialogue between Causal and Traditional Representation Learning: Toward Mutual Benefits in a Unified Formulation

Causal representation learning (CRL) and traditional representation learning have largely developed along different trajectories. Traditional representation learning has been driven mainly by applications and empirical objectives, whereas CRL has focused more on theoretical questions, particularly identifiability. This difference in emphasis has created a gap between the two fields in terminology, problem formulation, and evaluation, limiting communication and sometimes leading to disconnected or redundant efforts. In this paper, we argue that these two fields should be brought into dialogue rather than treated as separate paradigms. To this end, we introduce a unified formulation in which the representation learning is characterized by two components: a task component, which specifies what information the learned representation is required to preserve, and a constraint component, which specifies what structure is imposed on the latent space. Under this formulation, the benefits run in both directions. CRL provides theoretical tools for understanding when structured latent constraints are useful or necessary, while traditional representation learning offers practical insights on task design and objective choice that can improve the development of CRL methods. To illustrate this interaction, we experimentally study how different task components affect the behavior of CRL methods under different structured constraints. Results on CausalVerse show that the effectiveness of causal constraints depends strongly on the tasks with which they are paired.

Yan Li, Yuewen Sun, Shaoan Xie +4

1Mohamed bin Zayed University of Artificial Intelligence · 2Carnegie Mellon University

Representation Learning Causal Representation Learning

Ask PDF