Causal Direction from Convergence Time: Faster Training in the True Causal Direction

Abdulrahman Tamim

cs.LGFeb 24, 2026

Causal Direction from Convergence Time: Faster Training in the True Causal Direction

Authors: Abdulrahman Tamim

Organizations: Independent Researcher

Abstract

We introduce Causal Computational Asymmetry (CCA), a principle for causal direction identification based on optimization dynamics in which one neural network is trained to predict $Y$ from $X$ and another to predict $X$ from $Y$ , and the direction that converges faster is inferred to be causal. Under the additive noise model $Y = f(X) + \varepsilon$ with $\varepsilon \perp X$ and $f$ nonlinear and injective, we establish a formal asymmetry: in the reverse direction, residuals remain statistically dependent on the input regardless of approximation quality, inducing a strictly higher irreducible loss floor and non-separable gradient noise in the optimization dynamics, so that the reverse model requires strictly more gradient steps in expectation to reach any fixed loss threshold; consequently, the forward (causal) direction converges in fewer expected optimization steps. CCA operates in optimization-time space, distinguishing it from methods such as RESIT, IGCI, and SkewScore that rely on statistical independence or distributional asymmetries, and proper z-scoring of both variables is required for valid comparison of convergence rates. On synthetic benchmarks, CCA achieves 26/30 correct causal identifications across six neural architectures, including 30/30 on sine and exponential data-generating processes. We further embed CCA into a broader framework termed Causal Compression Learning (CCL), which integrates graph structure learning, causal information compression, and policy optimization, with all theoretical guarantees formally proved and empirically validated on synthetic datasets.

Figures & tables

Tradition	Failure mode in isolation
Compression	Compresses spurious correlations; fails with confounders
PAC bounds	Bounds statistical (not causal) error; certifies confounded models as good
Causality	No unique solution without MDL; any connected graph satisfies identifiability
Causal IB	Undefined without a causal graph; $I_{c}$ requires a structural model

Table 1: Failure modes when each tradition operates without the others.

Symbol	Meaning
$f$	True structural mechanism: $Y=f(X)+\varepsilon$
$g_{\theta}$	Forward network: predicts $Y$ from $X$
$h_{\phi}$	Reverse network: predicts $X$ from $Y$
$\varepsilon$	Structural noise, independent of $X$ , zero-mean
$T_{\mathrm{fwd}}$	Steps for $g_{\theta}$ to reach MSE $<\tau$
$T_{\mathrm{rev}}$	Steps for $h_{\phi}$ to reach MSE $<\tau$

Table 2: Symbols used in the CCA framework.

DGP	Correct	Mechanism	Status
$Y=\sin(X)+\varepsilon$	30/30	Inj., mod. scale	✓
$Y=e^{0.5X}+\varepsilon$	30/30	Inj., mod. scale	✓
$Y=X^{3}+\varepsilon$	6/30	Injective, high scale	Scale BC †
$Y=X^{2}+\varepsilon$	30/30 ∗	Non-injective	Degen. BC ∗
$Y=2X+\varepsilon$	0/30	Linear Gaussian	Sym. (BC)
∗ Reverse target collapses to zero (wrong reasons).

Table 3: Experiment 1: CCA direction accuracy per DGP. 30 trials per DGP (6 architectures

\times

5 seeds). “Sym.” = symmetric convergence. “BC” = boundary condition predicted to fail by theory.

Architecture	$\sin$	$\exp$	$X^{3}$	$X^{2}$	Lin.
64-64-Tanh / Adam	5/5	5/5	1/5	5/5	0/5
128-128-Tanh / Adam	5/5	5/5	1/5	5/5	0/5
32-32-32-Tanh / Adam	5/5	5/5	0/5	5/5	0/5
64-64-ReLU / Adam	5/5	5/5	0/5	5/5	0/5
64-64-Tanh / SGD	5/5	5/5	3/5	5/5	0/5
64-64-Tanh / RMSProp	5/5	5/5	1/5	5/5	0/5

Table 4: Experiment 1: Per-architecture breakdown. Each cell shows correct/total (5 trials).

Figure 1: CCA accuracy per DGP and architecture (Experiment 1). Each group of bars represents one of the six network architectures tested. Bar colors correspond to the five DGPs. The dashed line at 0.5 marks chance. Injective DGPs ( $\sin$ , $\exp$ ) achieve perfect 1.0 accuracy across every architecture. The cubic DGP without normalization (blue, 6/30) reveals the scale boundary condition; with z-scoring it recovers to 26/30. Linear Gaussian (gray) and non-injective $X^{2}$ (red) boundary conditions behave exactly as predicted by theory.

	Raw (no z-score)	With z-scoring
$E[T_{\mathrm{fwd}}]$	$1152.5\pm 904.0$	$323\pm 531$
$E[T_{\mathrm{rev}}]$	$468.8\pm 860.4$	$717\pm 789$
Correct ID	6/30	26/30

Table 5: CCA score distribution for

Y=X^{3}+\varepsilon

without input normalization across 30 seeds. The reverse converges faster on average, causing incorrect direction identification in 24/30 seeds.

Figure 2: Forward vs. reverse convergence on $Y=X^{3}+\varepsilon$ (Seed 0, z-scored, MLP-64-64-Tanh/Adam). The forward network (solid blue) crosses the convergence threshold $\tau=0.05$ at step 161 and continues improving to below $10^{-3}$ MSE. The reverse network (dashed red) descends initially then plateaus just above $\tau$ , never crossing it within the 3000-step cap. The CCA score is $161-3000=-2839$ , strongly predicting $X\to Y$ . This 19-fold gap is substantially larger than the theoretical lower bound, because the reverse landscape also contains saddle points not captured by the PL approximation.

Figure 3: CCA score distribution across 30 seeds ( $Y=X^{3}+\varepsilon$ , z-scored). Left: Bar chart of CCA scores ( $T_{\mathrm{fwd}}-T_{\mathrm{rev}}$ ) per seed. Blue bars indicate correct identification (CCA $<0$ ); red bars incorrect. 26 of 30 seeds are correct. The four exceptions are seeds where initialization variance caused the forward network to take unusually many steps. Right: Scatter of $T_{\mathrm{fwd}}$ vs. $T_{\mathrm{rev}}$ . Points above the dashed diagonal ( $T_{\mathrm{fwd}}=T_{\mathrm{rev}}$ ) are correct identifications. The cluster in the top-left corner represents seeds where the reverse hit the 3000-step cap while forward converged early – the strongest possible CCA signal. Mean $T_{\mathrm{fwd}}=323\pm 531$ steps; mean $T_{\mathrm{rev}}=717\pm 789$ steps; reverse takes $2.2\times$ longer on average.

Figure 4: Boundary condition experiments, 30 seeds each. Left: Linear Gaussian ( $Y=2X+\varepsilon$ ). CCA $<0$ in only 1 of 30 seeds, indistinguishable from random. Scores are near zero (note the $y$ -axis scale is $[-1,+1]$ ): forward and reverse take nearly the same number of steps because Gaussian symmetry makes the two optimization problems identical. This is the correct predicted failure. Right: Non-injective ( $Y=X^{2}+\varepsilon$ ). CCA $<0$ in 29 of 30 seeds with scores down to $-3000$ . This is the degenerate collapse: the reverse network learns to predict zero (because $E[X\mid Y]=0$ by symmetry of $P(X)$ ) in under 25 steps, while the forward network needs hundreds of steps to learn $x\mapsto x^{2}$ . This is not CCA working – it is a structural identifiability failure that should not be used to benchmark the method.

$\lambda_{2}$	Mono.	$\mathcal{L}$ init	$\mathcal{L}$ final	Iter.	Spur.
0.010	YES	10.000	$-1.238$	3	1
0.050	YES	10.000	$-1.106$	3	1
0.080	YES	10.000	$-1.007$	3	1
0.100	YES	10.000	$-0.942$	3	1
0.150	YES	10.000	$-0.777$	3	1
0.200	YES	10.000	$-0.612$	3	1

Table 6: Experiment 2:

\mathcal{L}_{\mathrm{CCL+}}

over the

\lambda_{2}

sweep. All seven runs exhibit monotone decrease. The spurious edge at

\lambda_{2}\leq 0.2

is attributable to the non-injective mechanism

X_{2}=X_{1}^{2}

, the expected boundary condition. Zero spurious edges at

\lambda_{2}=0.5

confirms Theorem 5.8 .

Seed	$T_{\mathrm{fwd}}$ ( $X_{1}\to X_{2}$ )	$T_{\mathrm{rev}}$ ( $X_{2}\to X_{1}$ )	Result
0	1000	9	Wrong
1	1000	8	Wrong
2	1000	20	Wrong
3	1000	21	Wrong
4	1000	9	Wrong
5	1000	22	Wrong

Table 7: CCA on the non-injective edge

X_{2}=X_{1}^{2}

: all 10 seeds misidentify the direction, confirming the boundary condition. The injective edge

X_{1}\to X_{3}

is correctly identified in 10/10 seeds in the same SCM.

Figure 5: Tübingen Cause-Effect Pairs benchmark ( $T_{\max}=10{,}000$ , z-scored, 108 pairs). Left: CCA score distribution. Blue bars are correct predictions, red bars incorrect. The vast majority of pairs score near zero (mass concentrated at the decision boundary), which corresponds to low-confidence predictions. Incorrect predictions are sparse and concentrated near zero, consistent with the boundary conditions: pairs with near-linear mechanisms or near-symmetric marginals produce weak asymmetry signals. Right: Cumulative accuracy sorted by $|\mathrm{CCA}|$ (most confident pairs first). CCA overall accuracy of 0.96 (dashed gray) substantially exceeds ANM/RESIT at 0.63 (red dotted) and chance at 0.50 (black dotted). The volatile accuracy at low confidence (leftmost part of the curve, fewest pairs evaluated) stabilizes clearly above both baselines as more pairs are included.

DGP	Init. ratio	Mid-train ratio
$Y=X^{3}+\varepsilon$ (std.)	0.376	0.054
$Y=X^{3}+\varepsilon$ (raw)	0.003	0.000
$Y=\sin(X)+\varepsilon$	1.161	2.179
$Y=e^{0.5X}+\varepsilon$	0.684	0.276
$Y=X^{2}+\varepsilon$	0.730	0.608

Table 8: Gradient norm variance ratio (reverse / forward), standardized data, 10 seeds. The ratio is not claimed to be uniformly

>1

; see the interpretation below.

System	Rung 2	PAC Bound	Sample Eff.	OOD Robust	Direction ID	Scope
GPT-4	$\times$	$\times$	$\times$	$\times$	$\times$	Obs. corpus
DeepSeek R1	$\times$	$\times$	$\sim$	$\times$	$\times$	Obs. corpus
Bareinboim CRL	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$	Interv. data
ANM / IGCI	$\sim$	$\times$	$\sim$	$\sim$	$\checkmark$	Bivariate
SkewScore	$\times$	$\times$	$\sim$	$\sim$	$\checkmark$	Heteroscedastic
CCL+	$\checkmark$	$\checkmark^{\dagger}$	$\checkmark^{\dagger\ddagger}$	$\checkmark^{\dagger}$	$\checkmark$	Injective ANM

Table 9: CCL+ compared to existing approaches.

\checkmark

= supported.

\times

= not supported.

\sim

= partial.

\dagger

= theoretical bound only; empirical validation on IHDP, Twins, ACIC beyond Tübingen has not been done.

\ddagger

= against non-parametric full-joint learners only (Theorem S); does not extend to parametric systems.

Causal Discovery

Causal discovery infers cause-and-effect structures from observational and interventional data, distinguishing causal relationships from correlation.

See all 221 papers

Apr 29, 2026

Causal Learning with Neural Assemblies

Can Neural Assemblies -- groups of neurons that fire together and strengthen through co-activation -- learn the direction of causal influence between variables? While established as a computationally general substrate for classification, parsing, and planning, neural assemblies have not yet been shown to internalize causal directionality. We demonstrate that the inherent operations of neural assemblies -- projection, local plasticity control, and sparse winner selection -- are sufficient for directional learning. We introduce DIRECT (DIRectional Edge Coupling/Training), a mechanism that co-activates source and target assemblies under an adaptive gain schedule to internalize directed relations. Unlike backpropagation-based methods, DIRECT relies solely on local plasticity, making the resulting causal claims auditable at the mechanism level. Our findings are verified through a dual-readout validation strategy: (i) synaptic-strength asymmetry, measuring the emergent weight gap between forward and reverse links, and (ii) functional propagation overlap, quantifying the reliability of directional signal flow. Across multiple domains, the framework achieves perfect structural recovery under a supervised, known-structure setting. These results establish neural assemblies as an auditable bridge between biologically plausible dynamics and formal causal models, offering an "explainable by design" framework where causal claims are traceable to specific neural winners and synaptic asymmetries.

Evangelia Kopadi, Dimitris Kalles

Hellenic Open University Greece

Neural Processes Causal Discovery Causal Structure Learning Biologically Plausible Learning Causal Interpretability Local Learning

Ask PDF

Mar 27, 2026

Identification of Bivariate Causal Directionality Based on Anticipated Asymmetric Geometries

Identification of causal directionality in bivariate numerical data is a fundamental research problem with important practical implications. This paper presents two alternative methods to identify direction of causation by considering conditional distributions: (1) Anticipated Asymmetric Geometries (AAG) and (2) Monotonicity Index (MI). The AAG method compares the actual conditional distributions to anticipated ones along two variables. Different comparison metrics, such as Pearson correlation, cosine distance, Hellinger distance, Jaccard index, Jeffreys divergence, K-L divergence, K-S distance, MAE, MSE, mutual information, and Wasserstein distance have been evaluated. Anticipated distributions have been projected as normal based on dual response statistics: mean and standard deviation. The MI method compares the calculated monotonicity indexes of the gradients of conditional distributions along two axes and exhibits count of gradient sign changes. Both methods assume stochastic properties of the bivariate data and exploit anticipated unimodality of conditional distributions of the effect. The proposed methods are straightforward and include only a limited number of hyperparameters that affect the accuracy of the identification. For a given set of hyperparameters, both the AAG and MI methods provide a unique, deterministic solution. To address sensitivity to hyperparameters, tuning has been done by utilizing a full factorial Design of Experiment. It turns out that the AAG method outperforms MI, achieving top weighted accuracies of 82.7% with simple tuning and 84.4% with size-adaptive tuning, compared with 81.6% for GRCI or 82.0% for CAREFL-H on the 99 pairs of the Tubingen real-world cause-effect examples. To evaluate the decisiveness of the identification method, a decision tree was fitted on the input data's symmetrical bivariate statistics to detect misclassified cases.

Alex Glushkovsky

Causal Discovery

Ask PDF

Apr 2, 2026

Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives

Understanding causal dependencies in observational data is critical for informing decision-making. These relationships are often modeled as Bayesian Networks (BNs) and Directed Acyclic Graphs (DAGs). Existing methods, such as NOTEARS and DAG-GNN, often face issues with scalability and stability in high-dimensional data, especially when there is a feature-sample imbalance. Here, we show that the denoising score matching objective of diffusion models could smooth the gradients for faster, more stable convergence. We also propose an adaptive k-hop acyclicity constraint that improves runtime over existing solutions that require matrix inversion. We name this framework Denoising Diffusion Causal Discovery (DDCD). Unlike generative diffusion models, DDCD utilizes the reverse denoising process to infer a parameterized causal structure rather than to generate data. We demonstrate the competitive performance of DDCDs on synthetic benchmarking data. We also show that our methods are practically useful by conducting qualitative analyses on two real-world examples. Code is available at this url: https://github.com/haozhu233/ddcd.

Hao Zhu, Di Zhou, Donna Slonim

Beth Israel Deaconess Medical Center Harvard Medical School Boston, MA, USA · Department of Computer Science Tufts University Medford, MA, USA

Causal Discovery Causal Structure Learning Bayesian Network Structure Learning

Ask PDF

Abstract

Causal Discovery

Explore similar work

Causal Learning with Neural Assemblies

Identification of Bivariate Causal Directionality Based on Anticipated Asymmetric Geometries

Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives