Causal Direction from Convergence Time: Faster Training in the True Causal Direction

Abdulrahman Tamim

cs.LGFeb 24, 2026

Causal Direction from Convergence Time: Faster Training in the True Causal Direction

Authors: Abdulrahman Tamim

Organizations: Independent Researcher

Abstract

We introduce Causal Computational Asymmetry (CCA), a principle for causal direction identification based on optimization dynamics in which one neural network is trained to predict $Y$ from $X$ and another to predict $X$ from $Y$ , and the direction that converges faster is inferred to be causal. Under the additive noise model $Y = f(X) + \varepsilon$ with $\varepsilon \perp X$ and $f$ nonlinear and injective, we establish a formal asymmetry: in the reverse direction, residuals remain statistically dependent on the input regardless of approximation quality, inducing a strictly higher irreducible loss floor and non-separable gradient noise in the optimization dynamics, so that the reverse model requires strictly more gradient steps in expectation to reach any fixed loss threshold; consequently, the forward (causal) direction converges in fewer expected optimization steps. CCA operates in optimization-time space, distinguishing it from methods such as RESIT, IGCI, and SkewScore that rely on statistical independence or distributional asymmetries, and proper z-scoring of both variables is required for valid comparison of convergence rates. On synthetic benchmarks, CCA achieves 26/30 correct causal identifications across six neural architectures, including 30/30 on sine and exponential data-generating processes. We further embed CCA into a broader framework termed Causal Compression Learning (CCL), which integrates graph structure learning, causal information compression, and policy optimization, with all theoretical guarantees formally proved and empirically validated on synthetic datasets.

Figures & tables

Tradition	Failure mode in isolation
Compression	Compresses spurious correlations; fails with confounders
PAC bounds	Bounds statistical (not causal) error; certifies confounded models as good
Causality	No unique solution without MDL; any connected graph satisfies identifiability
Causal IB	Undefined without a causal graph; $I_{c}$ requires a structural model

Table 1: Failure modes when each tradition operates without the others.

Symbol	Meaning
$f$	True structural mechanism: $Y=f(X)+\varepsilon$
$g_{\theta}$	Forward network: predicts $Y$ from $X$
$h_{\phi}$	Reverse network: predicts $X$ from $Y$
$\varepsilon$	Structural noise, independent of $X$ , zero-mean
$T_{\mathrm{fwd}}$	Steps for $g_{\theta}$ to reach MSE $<\tau$
$T_{\mathrm{rev}}$	Steps for $h_{\phi}$ to reach MSE $<\tau$

Table 2: Symbols used in the CCA framework.

DGP	Correct	Mechanism	Status
$Y=\sin(X)+\varepsilon$	30/30	Inj., mod. scale	✓
$Y=e^{0.5X}+\varepsilon$	30/30	Inj., mod. scale	✓
$Y=X^{3}+\varepsilon$	6/30	Injective, high scale	Scale BC †
$Y=X^{2}+\varepsilon$	30/30 ∗	Non-injective	Degen. BC ∗
$Y=2X+\varepsilon$	0/30	Linear Gaussian	Sym. (BC)
∗ Reverse target collapses to zero (wrong reasons).

Table 3: Experiment 1: CCA direction accuracy per DGP. 30 trials per DGP (6 architectures

\times

5 seeds). “Sym.” = symmetric convergence. “BC” = boundary condition predicted to fail by theory.

Architecture	$\sin$	$\exp$	$X^{3}$	$X^{2}$	Lin.
64-64-Tanh / Adam	5/5	5/5	1/5	5/5	0/5
128-128-Tanh / Adam	5/5	5/5	1/5	5/5	0/5
32-32-32-Tanh / Adam	5/5	5/5	0/5	5/5	0/5
64-64-ReLU / Adam	5/5	5/5	0/5	5/5	0/5
64-64-Tanh / SGD	5/5	5/5	3/5	5/5	0/5
64-64-Tanh / RMSProp	5/5	5/5	1/5	5/5	0/5

Table 4: Experiment 1: Per-architecture breakdown. Each cell shows correct/total (5 trials).

Figure 1: CCA accuracy per DGP and architecture (Experiment 1). Each group of bars represents one of the six network architectures tested. Bar colors correspond to the five DGPs. The dashed line at 0.5 marks chance. Injective DGPs ( $\sin$ , $\exp$ ) achieve perfect 1.0 accuracy across every architecture. The cubic DGP without normalization (blue, 6/30) reveals the scale boundary condition; with z-scoring it recovers to 26/30. Linear Gaussian (gray) and non-injective $X^{2}$ (red) boundary conditions behave exactly as predicted by theory.

	Raw (no z-score)	With z-scoring
$E[T_{\mathrm{fwd}}]$	$1152.5\pm 904.0$	$323\pm 531$
$E[T_{\mathrm{rev}}]$	$468.8\pm 860.4$	$717\pm 789$
Correct ID	6/30	26/30

Table 5: CCA score distribution for

Y=X^{3}+\varepsilon

without input normalization across 30 seeds. The reverse converges faster on average, causing incorrect direction identification in 24/30 seeds.

Figure 2: Forward vs. reverse convergence on $Y=X^{3}+\varepsilon$ (Seed 0, z-scored, MLP-64-64-Tanh/Adam). The forward network (solid blue) crosses the convergence threshold $\tau=0.05$ at step 161 and continues improving to below $10^{-3}$ MSE. The reverse network (dashed red) descends initially then plateaus just above $\tau$ , never crossing it within the 3000-step cap. The CCA score is $161-3000=-2839$ , strongly predicting $X\to Y$ . This 19-fold gap is substantially larger than the theoretical lower bound, because the reverse landscape also contains saddle points not captured by the PL approximation.

Figure 3: CCA score distribution across 30 seeds ( $Y=X^{3}+\varepsilon$ , z-scored). Left: Bar chart of CCA scores ( $T_{\mathrm{fwd}}-T_{\mathrm{rev}}$ ) per seed. Blue bars indicate correct identification (CCA $<0$ ); red bars incorrect. 26 of 30 seeds are correct. The four exceptions are seeds where initialization variance caused the forward network to take unusually many steps. Right: Scatter of $T_{\mathrm{fwd}}$ vs. $T_{\mathrm{rev}}$ . Points above the dashed diagonal ( $T_{\mathrm{fwd}}=T_{\mathrm{rev}}$ ) are correct identifications. The cluster in the top-left corner represents seeds where the reverse hit the 3000-step cap while forward converged early – the strongest possible CCA signal. Mean $T_{\mathrm{fwd}}=323\pm 531$ steps; mean $T_{\mathrm{rev}}=717\pm 789$ steps; reverse takes $2.2\times$ longer on average.

Figure 4: Boundary condition experiments, 30 seeds each. Left: Linear Gaussian ( $Y=2X+\varepsilon$ ). CCA $<0$ in only 1 of 30 seeds, indistinguishable from random. Scores are near zero (note the $y$ -axis scale is $[-1,+1]$ ): forward and reverse take nearly the same number of steps because Gaussian symmetry makes the two optimization problems identical. This is the correct predicted failure. Right: Non-injective ( $Y=X^{2}+\varepsilon$ ). CCA $<0$ in 29 of 30 seeds with scores down to $-3000$ . This is the degenerate collapse: the reverse network learns to predict zero (because $E[X\mid Y]=0$ by symmetry of $P(X)$ ) in under 25 steps, while the forward network needs hundreds of steps to learn $x\mapsto x^{2}$ . This is not CCA working – it is a structural identifiability failure that should not be used to benchmark the method.

$\lambda_{2}$	Mono.	$\mathcal{L}$ init	$\mathcal{L}$ final	Iter.	Spur.
0.010	YES	10.000	$-1.238$	3	1
0.050	YES	10.000	$-1.106$	3	1
0.080	YES	10.000	$-1.007$	3	1
0.100	YES	10.000	$-0.942$	3	1
0.150	YES	10.000	$-0.777$	3	1
0.200	YES	10.000	$-0.612$	3	1

Table 6: Experiment 2:

\mathcal{L}_{\mathrm{CCL+}}

over the

\lambda_{2}

sweep. All seven runs exhibit monotone decrease. The spurious edge at

\lambda_{2}\leq 0.2

is attributable to the non-injective mechanism

X_{2}=X_{1}^{2}

, the expected boundary condition. Zero spurious edges at

\lambda_{2}=0.5

confirms Theorem 5.8 .

Seed	$T_{\mathrm{fwd}}$ ( $X_{1}\to X_{2}$ )	$T_{\mathrm{rev}}$ ( $X_{2}\to X_{1}$ )	Result
0	1000	9	Wrong
1	1000	8	Wrong
2	1000	20	Wrong
3	1000	21	Wrong
4	1000	9	Wrong
5	1000	22	Wrong

Table 7: CCA on the non-injective edge

X_{2}=X_{1}^{2}

: all 10 seeds misidentify the direction, confirming the boundary condition. The injective edge

X_{1}\to X_{3}

is correctly identified in 10/10 seeds in the same SCM.

Figure 5: Tübingen Cause-Effect Pairs benchmark ( $T_{\max}=10{,}000$ , z-scored, 108 pairs). Left: CCA score distribution. Blue bars are correct predictions, red bars incorrect. The vast majority of pairs score near zero (mass concentrated at the decision boundary), which corresponds to low-confidence predictions. Incorrect predictions are sparse and concentrated near zero, consistent with the boundary conditions: pairs with near-linear mechanisms or near-symmetric marginals produce weak asymmetry signals. Right: Cumulative accuracy sorted by $|\mathrm{CCA}|$ (most confident pairs first). CCA overall accuracy of 0.96 (dashed gray) substantially exceeds ANM/RESIT at 0.63 (red dotted) and chance at 0.50 (black dotted). The volatile accuracy at low confidence (leftmost part of the curve, fewest pairs evaluated) stabilizes clearly above both baselines as more pairs are included.

DGP	Init. ratio	Mid-train ratio
$Y=X^{3}+\varepsilon$ (std.)	0.376	0.054
$Y=X^{3}+\varepsilon$ (raw)	0.003	0.000
$Y=\sin(X)+\varepsilon$	1.161	2.179
$Y=e^{0.5X}+\varepsilon$	0.684	0.276
$Y=X^{2}+\varepsilon$	0.730	0.608

Table 8: Gradient norm variance ratio (reverse / forward), standardized data, 10 seeds. The ratio is not claimed to be uniformly

>1

; see the interpretation below.

System	Rung 2	PAC Bound	Sample Eff.	OOD Robust	Direction ID	Scope
GPT-4	$\times$	$\times$	$\times$	$\times$	$\times$	Obs. corpus
DeepSeek R1	$\times$	$\times$	$\sim$	$\times$	$\times$	Obs. corpus
Bareinboim CRL	$\checkmark$	$\times$	$\times$	$\checkmark$	$\times$	Interv. data
ANM / IGCI	$\sim$	$\times$	$\sim$	$\sim$	$\checkmark$	Bivariate
SkewScore	$\times$	$\times$	$\sim$	$\sim$	$\checkmark$	Heteroscedastic
CCL+	$\checkmark$	$\checkmark^{\dagger}$	$\checkmark^{\dagger\ddagger}$	$\checkmark^{\dagger}$	$\checkmark$	Injective ANM

Table 9: CCL+ compared to existing approaches.

\checkmark

= supported.

\times

= not supported.

\sim

= partial.

\dagger

= theoretical bound only; empirical validation on IHDP, Twins, ACIC beyond Tübingen has not been done.

\ddagger

= against non-parametric full-joint learners only (Theorem S); does not extend to parametric systems.

Causal Discovery

Causal discovery infers cause-and-effect structures from observational and interventional data, distinguishing causal relationships from correlation.

See all 221 papers

Causal Learning with Neural Assemblies
Apr 29, 2026Evangelia Kopadi, Dimitris KallesNeural Processes Causal Discovery
Hellenic Open University Greece
Identification of Bivariate Causal Directionality Based on Anticipated Asymmetric Geometries
Mar 27, 2026Alex GlushkovskyCausal Discovery
Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives
Apr 2, 2026Hao Zhu, Di Zhou, Donna SlonimCausal Discovery Causal Structure Learning
Beth Israel Deaconess Medical Center Harvard Medical School Boston, MA, USA · Department of Computer Science Tufts University Medford, MA, USA

Apr 29, 2026J/K move · Enter open · S save

Causal Learning with Neural Assemblies

Evangelia Kopadi, Dimitris Kalles

Hellenic Open University Greece

Neural Processes Causal Discovery Causal Structure Learning Biologically Plausible Learning Causal Interpretability Local Learning

Open paper AskPDF arXiv

Can Neural Assemblies -- groups of neurons that fire together and strengthen through co-activation -- learn the direction of causal influence between variables? While established as a computationally general substrate for classification, parsing, and planning, neural assemblies have not yet been shown to internalize causal directionality. We demonstrate that the inherent operations of neural assemblies -- projection, local plasticity control, and sparse winner selection -- are sufficient for directional learning. We introduce DIRECT (DIRectional Edge Coupling/Training), a mechanism that co-activates source and target assemblies under an adaptive gain schedule to internalize directed relations. Unlike backpropagation-based methods, DIRECT relies solely on local plasticity, making the resulting causal claims auditable at the mechanism level. Our findings are verified through a dual-readout validation strategy: (i) synaptic-strength asymmetry, measuring the emergent weight gap between forward and reverse links, and (ii) functional propagation overlap, quantifying the reliability of directional signal flow. Across multiple domains, the framework achieves perfect structural recovery under a supervised, known-structure setting. These results establish neural assemblies as an auditable bridge between biologically plausible dynamics and formal causal models, offering an "explainable by design" framework where causal claims are traceable to specific neural winners and synaptic asymmetries.

Abstract

Causal Discovery

Explore similar work

Causal Learning with Neural Assemblies

Identification of Bivariate Causal Directionality Based on Anticipated Asymmetric Geometries

Smoothing the Landscape: Causal Structure Learning via Diffusion Denoising Objectives