Large Causal Models for Temporal Causal Discovery

Nikolaos Kougioulis; Nikolaos Gkorgkolis; MingXue Wang; Bora Caglayan; Dario Simionato; Andrea Tonon; Ioannis Tsamardinos

cs.LGFeb 20, 2026

Large Causal Models for Temporal Causal Discovery

Authors: Nikolaos Kougioulis, Nikolaos Gkorgkolis, MingXue Wang, Bora Caglayan, Dario Simionato, Andrea Tonon, Ioannis Tsamardinos

Organizations: Institute of Applied & Computational Mathematics, FORTH · Computer Science Department, University of Crete · Huawei Ireland Research Centre, Dublin, Ireland

Abstract

Causal discovery for both cross-sectional and temporal data has traditionally followed a dataset-specific paradigm, where a new model is fitted for each individual dataset. Such an approach limits the potential of multi-dataset pretraining. The concept of large causal models (LCMs) envisions a class of pre-trained neural architectures specifically designed for temporal causal discovery. Prior approaches are constrained to small variable counts, degrade with larger inputs, and rely heavily on synthetic data, limiting generalization. We propose a principled framework for LCMs, combining diverse synthetic generators with realistic time-series datasets, allowing learning at scale. Extensive experiments on synthetic, semi-synthetic and realistic benchmarks show that LCMs scale effectively to higher variable counts and deeper architectures while maintaining strong performance. Trained models achieve competitive or superior accuracy compared to classical and neural baselines, particularly in out-of-distribution settings, while enabling fast, single-pass inference. Results demonstrate LCMs as a promising foundation-model paradigm for temporal causal discovery. Experiments and model weights are available at https://github.com/kougioulis/LCM/.

Figures & tables

Panel of Figure 1 — Figure 1: Temporal causal dependencies represented as a (a) lagged causal graph and (b) binary adjacency tensor. Each slice $\mathbb{A}^{(\ell-1)}$ encodes edges at a discrete lag $\ell\leq\ell_{\max}$ , where entry $\mathbb{A}^{(\ell-1)}_{j,i}=1$ denotes $V^{i}_{t-\ell}\to V^{j}_{t}$ .

Figure 2: Overview of the large causal model (LCM) pipeline. (1) Synthetic and realistic TSCM generators produce training pairs of multivariate time series and their lagged causal graphs. (2) The LCM is trained via supervised learning on these pairs to discover a lagged adjacency tensor $\hat{\mathbb{A}}$ for a time series $\mathbf{X}\in\mathbb{R}^{L\times V}$ , padded and normalized for stability. (3) At inference (CD phase), the pre-trained LCM predicts causal structure on unseen datasets in a zero-shot manner.

Figure 3: LCM architecture: A multivariate time series is embedded via Conv1D layers and positional encodings, processed through a Transformer encoder stack with distillation blocks, and augmented with lagged cross-correlations (training aids). A feedforward head outputs a lagged adjacency tensor representing the discovered temporal causal graph. Adapted from [ 30 ] .

	LCM	+CI	+CR ( $\lambda$ )
			.25	.5	.75	1.0
AUC	.868	$.914^{\textbf{*}}$	$.926^{\textbf{*}}$	$.918^{\textbf{*}}$	.926	$.925$

Table 1: Ablation of training aids on Synthetic_1 . Asterisks (*) indicate statistical significance over the preceding model (left column).

Synth/Realistic%	$f$ MRI_5	$f$ MRI	Power	Climate
100/0%	$.924^{\textbf{*}}$ $\pm$ .050	$.874^{\textbf{*}}$ $\pm$ .142	$.966^{\textbf{*}}$ $\pm$ .011	$.938^{\textbf{*}}$ $\pm$ .035
80/20%	.966 $\pm$ .028	.960 $\pm$ .028	.981 $\pm$ .006	.982 $\pm$ .011
50/50%	$.951$ $\pm$ .050	$.944^{\textbf{*}}$ $\pm$ .047	$.979$ $\pm$ .008	$.988$ $\pm$ .010
20/80%	$.968$ $\pm$ .004	$.966^{\textbf{*}}$ $\pm$ .036	$.975^{\textbf{*}}$ $\pm$ .010	$.986$ $\pm$ .015

Table 2: Out-of-distribution performance of LCMs trained on varying synthetic/realistic mixtures, evaluated on semi-synthetic

f

MRI benchmarks and the OOD Power & Climate benchmarks. Statistical significance versus the

80/20\%

reference ( bold ) is indicated by an asterisk ( * ).

Model	Synthetic_1	Synthetic_2	CDML
$\text{LCM}_{\texttt{2.5M}}$	.995 * $\pm$ .013	.903 * $\pm$ .107	.772 $\pm$ .177
$\text{LCM}_{\texttt{9.4M}}$	.995 * $\pm$ .013	.906 * $\pm$ .106	.768 $\pm$ .171
$\text{LCM}_{\texttt{12.2M}}$	. 996 $\pm$ .013	.909 $\pm$ .103	.773 $\pm$ .172
$\text{LCM}_{\texttt{24M}}$	.990 * $\pm$ .012	.865 * $\pm$ .133	.727 * $\pm$ .183
CP (Stein et al.)	.943 * $\pm$ .151	–	–
PCMCI	.671 * $\pm$ .444	.800 * $\pm$ .181	.521 * $\pm$ .108

Table 3: Causal discovery performance (AUC) of large-scale LCMs and baseline methods across synthetic benchmarks. Results are reported as mean

\pm

standard deviation across datasets. An asterisk indicates a statistically significant difference with respect to the best-performing model (LCM- 12.2M ), which achieves the highest mean AUC across all semi-synthetic collections.

Model	$f$ MRI_5	$f$ MRI	Kuramoto_5	Kuramoto_10
$\text{LCM}_{\texttt{2.5M}}$	.981 $\pm$ .017	.978 $\pm$ .017	.956 * $\pm$ .017	.919 * $\pm$ .018
$\text{LCM}_{\texttt{9.4M}}$	.983 $\pm$ .007	.979 $\pm$ .012	.972 $\pm$ .009	.924 * $\pm$ .008
$\text{LCM}_{\texttt{12.2M}}$	.986 $\pm$ .007	.982 $\pm$ .012	.953 * $\pm$ .016	.921 * $\pm$ .014
$\text{LCM}_{\texttt{24M}}$	.987 $\pm$ .006	.984 $\pm$ .010	.973 $\pm$ .007	.931 $\pm$ .010
CP (Stein et al.)	.883 * $\pm$ .078	–	.541 * $\pm$ .108	–
PCMCI	.746 * $\pm$ .111	.768 * $\pm$ .110	.476 * $\pm$ .103	.635 * $\pm$ .046

Table 4: Out-of-distribution (zero-shot) causal discovery performance (AUC) of large-scale LCMs and baseline methods across semi-synthetic benchmarks. Results are reported as mean

\pm

standard deviation across datasets.

Model	AQ	Clim	Gar	ETT	Gear	Pow
$\text{LCM}_{\texttt{2.5M}}$	$\mathbf{.985}_{\pm.014}$	$\mathbf{.989}_{\pm.007}$	$.995_{\pm.004}$	$\mathbf{.977}_{\pm.007}$	$.998_{\pm.007}$	$.980^{*}_{\pm.007}$
$\text{LCM}_{\texttt{9.4M}}$	$.982_{\pm.013}$	$.983_{\pm.012}$	$.996_{\pm.004}$	$.965_{\pm.024}$	$.998_{\pm.005}$	$\mathbf{.983}_{\pm.005}$
$\text{LCM}_{\texttt{12.2M}}$	$.977^{*}_{\pm.018}$	$.987_{\pm.010}$	$\mathbf{.996}_{\pm.003}$	$.975_{\pm.010}$	$\mathbf{.999}_{\pm.003}$	$.983^{*}_{\pm.005}$
$\text{LCM}_{\texttt{24M}}$	$.970^{*}_{\pm.020}$	$.982_{\pm.014}$	$.995_{\pm.006}$	$.972_{\pm.053}$	$.996_{\pm.010}$	$.979^{*}_{\pm.008}$
PCMCI	$.565^{*}_{\pm.209}$	$.927_{\pm.096}$	$.972^{*}_{\pm.053}$	$.898^{*}_{\pm.035}$	$.980_{\pm.040}$	$.966_{\pm.028}$
DYNO	$.706^{*}_{\pm.218}$	$.708^{*}_{\pm.118}$	$.536^{*}_{\pm.051}$	$.566^{*}_{\pm.037}$	$.693^{*}_{\pm.083}$	$.551^{*}_{\pm.053}$

Table 5: Out-of-distribution (zero-shot) causal discovery performance (AUC) of large-scale LCMs and baseline methods across realistic benchmarks. Results are mean

\pm

std. across datasets. AQ = AirQualityMS, Clim = Climate, Gar = Garments, ETT = ETTm2, Gear = Gearbox, Pow = Power.

Figure 4: Running times (in seconds) for LCMs and baseline algorithms on the Synthetic_2 holdout set, averaged over 10 runs. Traditional methods (e.g., PCMCI & DYNOTEARS) scale superlinearly with lag and variable count, while Transformer-based LCMs remain effectively independent of input dimensionality due to their constant-time forward pass.

Figure 5: Empirical convergence of LCMs with increasing training data. Test AUC for 500K, 1M, and 2M parameter models trained on subsampled datasets. Validation/test sets are fixed to isolate the effect of data scale.

Panel of Figure 6 — Figure 6: Hidden state representations with and without auxiliary training aids.

Panel of Figure 7 — Figure 7: Illustration of the padding strategies in LCMs. Left: Gaussian noise padding for time-series where $V<V_{\max}$ and $L<L_{\text{max}}$ . Right: Zero padding of lagged adjacency tensors when $V<V_{\max}$ or $\ell<\ell_{\max}$ . Red dashed boxes indicate padded regions. $\mathcal{N}$ denotes $\mathcal{N}(0,0.01)$ for brevity.

Collection	Pairs	Variables	Timesteps	$\ell_{\max}$	Func. Rels.
Synthetic_1	100k	$3\text{--}5$	500	3	L, NL
Synthetic_2	270k	$3\text{--}12$	500	3	L, NL
CDML	240	$4\text{--}11$	500	3	L, NL

Table 6: Synthetic time-series dataset collections. L = linear, NL = non-linear functional relationships.

Category	Example Function	Description
Linear	$f(x)=ax+b$	Additive linear effect
	$f(x_{1},x_{2})=a_{1}x_{1}+a_{2}x_{2}+b$	Linear combination of parents
Non-linear	$f(x)=\alpha_{n}x^{n}+\ldots\alpha_{0}x_{0}$	Polynomial
	$f(x)=e^{x}$	Exponential Transformation
	$f(x)=\sin(x)$	Sinusoidal
	$f(x_{1},x_{2})=x_{1}x_{2}$	Multiplicative interaction

Table 7: Functional dependencies used in synthetic SCM generation.

Collection	Datasets	Vars	Timesteps	Max Lag	Rels.
$f$ MRI5	21	5	1200–5000	1	Non-linear
$f$ MRI	26	5–10	1200–5000	1	Non-linear
$\text{Kuramoto}\_5$	1000	5	500	1	Non-linear
$\text{Kuramoto}\_10$	1000	10	500	1	Non-linear

Table 8: Overview of semi-synthetic time-series dataset collections.

Collection	Variables	Timesteps	Domain
WTH	12	35064	Weather
ETTh1	7	17420	Power
ETTm1	7	69680	Power
AirQualityUCI	12	9357	Weather
Bike-usage	5	52584	Transportation
Outdoors	3	1440	Environmental

Table 9: Overview of real time-series datasets used for generating realistic data pairs, employing the methodology by [ 20 ] .

Causal Discovery
Method	Description
PCMCI [ 47 ]	Constraint-Based Algorithm
DYNOTEARS [ 40 ]	Constraint Optimization Algorithm
CP [ 50 ]	Pre-trained Transformer
Functional Dependencies
Random Forest Regressor [ 5 ]	Ensemble Learning Regressor

Table 10: Implemented Methods for Temporal Functional Dependency Estimation, Noise Density Estimation in Temporal Causal Discovery.

Collection	Variables	$\mathbf{\ell_{\max}}$	Num. Datasets
WTH	3-12	3	20200
ETTh1	3-7	3	9000
ETTm1	3-7	3	9000
AirQualityUCI	3-12	3	3900
Bike-usage	3-5	3	1800
Outdoors	3	3	800

Table 11: Overview of realistically generated time-series collections for training and OOD evaluation of LCMs. Collections in bold notate inclusion in the SIM training corpus and the large scale, LS corpus.

Model	#Params	Enc. Layers	$d_{\text{model}}$	Attn. Heads	$d_{\text{ff}}$	Usage
-	905K-914K	1	256	2	128	Subsection 7.1
-	1M	1	256	2	256	Subsection 7.2
LCM- 2.5M	2.5M	4	256	4	256	Subsection 7.3
LCM- 9.4M	9.4M	4	512	4	512	Subsection 7.3
LCM- 12.2M	12.2M	4	512	4	512	Subsection 7.3
LCM- 24M	24M	8	1024	8	1024	Subsection 7.3

Table 12: Model configurations and parameter sizes for the LCM variants used in the ablations and additional results in Section 7 .

Figure 8: Heatmap visualization of the discovered lagged adjacency matrix (bottom row) compared to the ground truth (top row) in an example with $n_{\text{max}}=12$ and $\ell_{\text{max}}=3$ using the pretrained LCM-2.4M model. Brighter colors in the predicted adjacency indicate stronger confidence for edge existence.

Figure 9: Paired median AUC differences relative to the reference model across Synthetic collections. Positive values indicate improved performance. Statistical significance is determined via Wilcoxon signed-rank test with Bonferroni correction.

Figure 10: Paired median AUC differences across Semi-Synthetic collections. Positive values indicate improved performance. Statistical significance is determined via Wilcoxon signed-rank test with Bonferroni correction.

Figure 11: Paired median AUC differences across Realistic collections. Positive values indicate improved performance. Statistical significance is determined via Wilcoxon signed-rank test with Bonferroni correction.

Time-Series Causal Discovery

Time-series causal discovery seeks to infer which variables influence others over time and when those effects occur.

See all 65 papers

May 8, 2026

Mask2Cause: Causal Discovery via Adjacency Constrained Causal Attention

Leveraging deep learning for causal discovery in time series remains challenging because existing neural methods predominantly rely on component-wise architectures that fail to capture shared system dynamics or employ decoupled post-hoc graph extraction that risks overfitting to spurious correlations. We propose

\textbf{Mask2Cause}

, an end-to-end framework that recovers the underlying causal graph directly during the forecasting forward pass. Our approach introduces an Inverted Variable Embedding and an Adjacency-Constrained Masked Attention mechanism, trained with homoscedastic or heteroscedastic objectives to capture causal influences in both mean and variance. Empirical results on diverse benchmarks, from synthetic chaotic dynamics to realistic biological simulations, demonstrate state-of-the-art causal discovery with significantly reduced parameter complexity compared to standard baselines. We further show that inferred causal structures can be used to reduce parameter count of forecasting models by more than 70% on average while maintaining predictive accuracy.

Omar Muhammad, Pasupuleti Dhruv Shivkant, Deepak N. Subramani

1Indian Institute of Science, Bengaluru, India

Time-Series Causal Discovery Causal Structure Learning Causal Attention

Ask PDF

Feb 9, 2026

CauScale: Neural Causal Discovery at Scale

Causal discovery is essential for advancing data-driven fields such as scientific AI and data analysis, yet existing approaches face significant time- and space-efficiency bottlenecks when scaling to large graphs. To address this challenge, we present CauScale, a neural architecture designed for efficient causal discovery that scales inference to graphs with up to 1000 nodes. CauScale improves time efficiency via a reduction unit that compresses data embeddings and improves space efficiency by adopting tied attention weights to avoid maintaining axis-specific attention maps. To keep high causal discovery accuracy, CauScale adopts a two-stream design: a data stream extracts relational evidence from high-dimensional observations, while a graph stream integrates statistical graph priors and preserves key structural signals. CauScale successfully scales to 500-node graphs during training, where prior work fails due to space limitations. Across testing data with varying graph scales and causal mechanisms, CauScale achieves 99.6% mAP on in-distribution data and 84.4% on out-of-distribution data, while delivering 4-13,000 times inference speedups over prior methods. Our project page is at https://github.com/OpenCausaLab/CauScale.

Bo Peng, Sirui Chen, Jiaguo Tian +2

Shanghai Jiao Tong University · Shanghai Artificial Intelligence Laboratory · Shanghai Innovation Institute +1

Efficient Neural Network Inference Causal Discovery

Ask PDF

Jun 2, 2026

Learning Temporal Causal Structure via Smooth Differentiable Optimization

Causal discovery with instantaneous effects in multivariate time series is challenging, as the instantaneous structure must be acyclic. Prior methods enforce this by either separating instantaneous and lagged estimation into multi-stage pipelines or imposing algebraic acyclicity constraints via complex augmented Lagrangian optimization, both of which incur high computational cost. In this work, we propose a different approach: we learn a differentiable permutation of variables using the Gumbel--Sinkhorn operator and triangularize the instantaneous coefficient matrix of a Structural Vector Autoregressive (SVAR) model in the learned order. This converts acyclicity from a hard constraint into a parameterization and keeps it valid throughout optimization. In doing so, our method enables unified, continuous optimization with gradient-based learning, leading to improved efficiency in time--series causal discovery. Across three real-world benchmarks, our method achieves the best overall performance compared with 12 baselines in both discovery accuracy and efficiency. On the large-scale benchmark, it further demonstrates strong scalability, achieving more than a 6x speedup over competing methods.

Tong Zhao, Ce Guo, Wayne Luk +2

Time-Series Causal Discovery Causal Discovery

Ask PDF

Abstract

Time-Series Causal Discovery

Explore similar work

Mask2Cause: Causal Discovery via Adjacency Constrained Causal Attention

CauScale: Neural Causal Discovery at Scale

Learning Temporal Causal Structure via Smooth Differentiable Optimization