Dictionary Based Pattern Entropy for Causal Direction Discovery

Harikrishnan N B; Shubham Bhilare; Aditi Kathpalia; Nithin Nagaraj

stat.MLMar 4, 2026

Dictionary Based Pattern Entropy for Causal Direction Discovery

Authors: Harikrishnan N B, Shubham Bhilare, Aditi Kathpalia, Nithin Nagaraj

Organizations: Department of Computer Science and Information Systems, BITS Pilani K K Birla Goa Campus, NH 17B, Bypass, Road, Zuarinagar, 403726, Goa, India · Department of Applied Mechanics and Biomedical Engineering, Indian Institute of Technology Madras, IIT P.O., Chennai 600036, India · Complex Systems Programme, National Institute of Advanced Studies, IISc Campus, Bengaluru, 560012, Karnataka, India

Abstract

Discovering causal direction from temporal observational data is particularly challenging for symbolic sequences, where functional models and noise assumptions are often unavailable. We propose a novel \emph{Dictionary Based Pattern Entropy ( $DPE$ )} framework that infers both the direction of causation and the specific subpatterns driving changes in the effect variable. The framework integrates Algorithmic Information Theory (AIT) and Shannon Information Theory. Causation is interpreted as the emergence of compact, rule based patterns in the candidate cause that systematically constrain the effect. $DPE$ constructs direction-specific dictionaries and quantifies their influence using entropy-based measures, enabling a principled link between deterministic pattern structure and stochastic variability. Causal direction is inferred via a minimum-uncertainty criterion, selecting the direction exhibiting stronger and more consistent pattern-driven organization. As summarized in Table 7, $DPE$ consistently achieves reliable performance across diverse synthetic systems, including delayed bit-flip perturbations, AR(1) coupling, 1D skew-tent maps, and sparse processes, outperforming or matching competing AIT-based methods ( $ETC_E$ , $ETC_P$ , $LZ_P$ ). In biological and ecological datasets, performance is competitive, while alternative methods show advantages in specific genomic settings. Overall, the results demonstrate that minimizing pattern level uncertainty yields a robust, interpretable, and broadly applicable framework for causal discovery.

Figures & tables

Table 1: Illustration of sliding and XNOR comparison to extract common subsequences with consecutive matches highlighted in red.

Pattern	Change Count	No Change Count	Ratio
01	4	5	0.444
011	1	4	0.200
0110	0	2	0.000
011101	2	0	1.000
11	1	8	0.111
110	0	5	0.000

Table 2: Pattern statistics showing the number of changes, no-changes, and the ratio (From

X\rightarrow Y

Pattern	Change Count	No Change Count	Ratio
00	10	11	0.476
000	13	3	0.8125
10	3	1	0.75

Table 3: Pattern statistics showing the number of changes, no-changes, and the ratio (From

Y\rightarrow X

Pattern ( $p$ )	Ratio ( $r$ )	Weight ( $W_{p}$ )	Binary Entropy $H_{b}(r_{p})$ (bits)	Weighted Entropy $H_{w}(p)$ (bits)
01	0.444	0.310	0.991	0.307
011	0.200	0.179	0.722	0.129
0110	0.000	0.074	0.000	0.000
011101	1.000	0.080	0.000	0.000
11	0.111	0.310	0.503	0.156
110	0.000	0.179	0.000	0.000

Table 4: Entropy and Weights for Patterns (From

X\rightarrow Y

)

Pattern ( $p$ )	Ratio ( $r$ )	Weight ( $W_{p}$ )	Binary Entropy $H_{b}(r_{p})$ (bits)	Weighted Entropy $H_{w}(p)$ (bits)
00	0.476	0.724	0.999	0.723
000	0.8125	0.571	0.696	0.397
10	0.750	0.138	0.811	0.112
Average Weighted Entropy $\bar{H}_{Y\rightarrow X}$				0.411

Table 5: Entropy and Weights for Patterns (From

Y\rightarrow X

)

Figure 1: Directed network of symbolic patterns illustrating deterministic structure between $X$ and $Y$ . Nodes represent distinct substrings extracted from the driving sequence, and edges encode their weighted entropy with respect to the target sequence. A weighted entropy value close to $0$ indicates that the pattern induces highly deterministic transition behavior in the target, whereas larger values indicate increasing uncertainty in the induced transitions. The figure displays both directions, $X\rightarrow Y$ and $Y\rightarrow X$ , enabling comparison of directional determinism.

Figure 2: Effect of delayed bit-flip on causal direction detection: accuracy versus bit-flip delay for $DPE$ , $ETC_{P}$ , $ETC_{E}$ , and $LZ_{P}$ .

Figure 5: Sparse processes: comparison of accuracy for $DPE$ , $ETC_{E}$ , $ETC_{P}$ , and $LZ_{P}$ across varying sparsity levels ( $k$ ).

Figure 8: Genomic causal analysis of SARS-CoV-2: Comparative summary of $DPE$ , $ETC_{E}$ , $ETC_{P}$ , and $LZ_{P}$ sensitivity to global (RS) vs. local (CW) evolutionary drivers.

Figure 9: Population dynamics of the predator (Didinium nasutum) and prey (Paramecium aurelia).

Model	Direction	Strength	Pred $\rightarrow$ Prey	Prey $\rightarrow$ Pred
$DPE$	Pred $\rightarrow$ Prey	0.1125	0.1700	0.2825
$ETC_{P}$	Pred $\rightarrow$ Prey	1.0000	0.0000	1.0000
$ETC_{E}$	Pred $\rightarrow$ Prey	0.0778	0.8000	0.7222
$LZ_{P}$	Pred $\rightarrow$ Prey	5.0000	2.0000	7.0000

Table 6: Predator-Prey System: Comparative causal metrics for Predator-Prey interaction across different models.

Experiments	$DPE$	$ETC_{E}$	$ETC_{P}$	$LZ_{P}$
Delayed Bit-flip	$\checkmark$	$\times$	$\times$	$\checkmark$
AR(1) Coupling	$\checkmark$	$\times$	$\times$	$\checkmark$
1D Skew-tent Maps	$\checkmark$	$\times$	$\times$	$\times$
Sparse Processes	$\checkmark$	$\times$	$\times$	$\times$
SARS-CoV-2 (RS vs. CW)	$\times$	$\times$	$\checkmark$	$\checkmark$
Predator-Prey	$\checkmark$	$\checkmark$	$\checkmark$	$\checkmark$

Table 7: Summary of reliability across experiments. A check mark (

\checkmark

) indicates that the framework achieves an average accuracy of

\geq 80\%

across trials for the corresponding experiment, while (

\times

) indicates that it does not. The table reflects reliability, not exact performance values.

Time-Series Causal Discovery

Time-series causal discovery seeks to infer which variables influence others over time and when those effects occur.

See all 65 papers

Causal Direction from Convergence Time: Faster Training in the True Causal Direction
Feb 24, 2026Abdulrahman TamimCausal Discovery
Independent Researcher
Identification of Bivariate Causal Directionality Based on Anticipated Asymmetric Geometries
Mar 27, 2026Alex GlushkovskyCausal Discovery
Tags for DAGs: Graph Refinement with Meta-Informed Relations
Jun 24, 2025Florian Peter Busch, Moritz Willig, Florian Guldan +2Causal Discovery Causal Structure Learning
Computer Science Department, Technical University of Darmstadt · Hessian Center for AI (hessian.AI) · German Research Center for AI (DFKI) +2

Feb 24, 2026J/K move · Enter open · S save

Causal Direction from Convergence Time: Faster Training in the True Causal Direction

Abdulrahman Tamim

Independent Researcher

Causal Discovery

Open paper AskPDF arXiv

We introduce Causal Computational Asymmetry (CCA), a principle for causal direction identification based on optimization dynamics in which one neural network is trained to predict

Y

from

X

and another to predict

X

from

Y

, and the direction that converges faster is inferred to be causal. Under the additive noise model

Y = f(X) + \varepsilon

with

\varepsilon \perp X

and

f

nonlinear and injective, we establish a formal asymmetry: in the reverse direction, residuals remain statistically dependent on the input regardless of approximation quality, inducing a strictly higher irreducible loss floor and non-separable gradient noise in the optimization dynamics, so that the reverse model requires strictly more gradient steps in expectation to reach any fixed loss threshold; consequently, the forward (causal) direction converges in fewer expected optimization steps. CCA operates in optimization-time space, distinguishing it from methods such as RESIT, IGCI, and SkewScore that rely on statistical independence or distributional asymmetries, and proper z-scoring of both variables is required for valid comparison of convergence rates. On synthetic benchmarks, CCA achieves 26/30 correct causal identifications across six neural architectures, including 30/30 on sine and exponential data-generating processes. We further embed CCA into a broader framework termed Causal Compression Learning (CCL), which integrates graph structure learning, causal information compression, and policy optimization, with all theoretical guarantees formally proved and empirically validated on synthetic datasets.

Abstract

Time-Series Causal Discovery

Explore similar work

Causal Direction from Convergence Time: Faster Training in the True Causal Direction

Identification of Bivariate Causal Directionality Based on Anticipated Asymmetric Geometries

Tags for DAGs: Graph Refinement with Meta-Informed Relations