Causal discovery promises to find which recorded variables drive a process target such as product quality, yield or energy use. Process data with known causes are rare, and in existing benchmarks the difficulty of identifying the causes of a target is either fixed by the recorded data or set for the whole graph. We present a synthetic benchmark for target-centric causal discovery in industrial processes. Its generator is a two-level structural causal model inspired by batch production, assembled from components that each model one phenomenon of process data, such as a control loop or a delayed effect. The graph is built around the target: without sensor error, every non-cause can be distinguished from the direct causes by conditioning on them or by time order. Each component is set independently, so the difficulty can be varied one condition at a time. We evaluate eight methods, from association measures to effect estimation, with an error analysis by the role of the selected candidate; all degrade as the difficulties combine, each in a way consistent with its assumptions.
Figures & tables
Figure 1: How the generator produces data. (1) The causal graph of a plant and the role of each recorded variable for Y . (2) The profile of X3 in three batches, built from its profile parameters θ(X3) . (3) One batch: time-step links, control and the target.
Component
In the generator
In a batch plant
Once per batch
batch profile θ(Xi)
2–4 motifs per variable (step, ramp, spike, plateau); their onsets, durations, sizes and the level form θ(Xi)
a temperature or flow rises, holds and falls in every batch, at different times and by different amounts
batch conditions Uj
act on the profiles of many variables
conditions that differ between batches and affect many sensors, e.g. coolant or outside temperature
confounders Zk , Zk(t)
move a direct cause and its siblings together, also within the batch
an unmeasured property of a raw material, e.g. its purity, that moves several sensors together
batch-level links
between profile parameters, without lag; seen only across batches
one batch quantity sets another, e.g. a larger charge gives a higher temperature peak
Table 2: The released benchmark: 119 configurations of the generator. Candidates per plant: mean ± s.d. over all plants. Shares: each setting, with all else at the reference, and its value measured on the generated plants (median over 10).
Figure 2: The selected lists of the eight methods on the 10 plants of the reference configuration (top) and of the combined difficulties (bottom). Green beyond the black tick is better than random; grey lift: fewer than three non-empty lists.
selected list
ranking
Method
first pick
precision
lift
length
nothing selected
R-precision
mAP
MRR
nDCG@3
All configurations (1,190 plants)
random
0.13
0.13
1.0
n/a
n/a
0.13
0.24
0.31
0.15
Correlation
0.64 ±.09
0.15 ±.03
1.1 ±.1
16.9 ±.8
0.00 ±.00
0.47 ±.09
0.61 ±.09
0.76 ±.06
0.57 ±.11
Granger
0.79 ±.12
0.21 ±.06
1.7 ±.3
13.7 ±1.7
0.10 ±.04
0.59 ±.13
0.69 ±.11
0.88 ±.07
0.67 ±.12
VARLiNGAM
0.66 ±.05
0.30 ±.04
2.3 ±.7
7.6 ±1.6
0.05 ±.02
0.53 ±.08
0.63 ±.07
0.77 ±.04
0.60 ±.08
Table 3: Each method over all configurations (top) and over those with the combined difficulties (bottom). Subscripts: half-width of the 95% t -interval over the 10 plant structures, each averaged over its configurations (precision and lift: over the structures with a non-empty list).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Quantity
Value
Batch level
batch conditions Uj
m= 8–11 (log-uniform); random directed tree, built by adding the conditions one by one, each linked to a random earlier one or, with a small probability, to a neighbour of it, which creates a few hubs (growing network with redirection, Krapivsky and Redner, 2001 ); conditions without parents N(0,1) ; weights uniform in [−1,1] , noise s.d. σU=0.2
confounders Zk
one per direct cause, N(0,1) ; the p−∣D∣ recorded variables that are not direct causes, the actuator included, are assigned to the causes in blocks (19, 9, 5–6 or 4 per cause at p=20 for ∣D∣=1 to 4); Zk enters each profile parameter of a variable in its block with probability 0.7
profile parameters (Eq. 4 )
equal weights 1/∣Pa(θ)∣ except on routes with a share; noise s.d. σθ=0.2 ; onset 0–0.5 and duration 0.2–0.5 of the slot; size 0.5–2 (times a base size 1.5); level −0.5 – 0.5
batch-level links
expected in-degree 1; 2 upstream variables
lots and drift (Eq. 7 )
lot process: AR(1) over batches, persistence 0.9; drift: linear in the batch index
Appendix
Table 4: Parameter ranges of the generator in all experiments.
Share
Setting
Expected
Measured, median [IQR]
ssig
0.05
0.05
0.02 [0.01, 0.04]
0.1
0.1
0.07 [0.06, 0.10]
0.2
0.2
0.18 [0.16, 0.21]
0.4
0.4
0.39 [0.36, 0.42]
0.7
0.7
0.69 [0.68, 0.71]
0.9
0.9
0.88 [0.87, 0.89]
Appendix
Table 5: Configured and realised shares: each setting, with all else at the reference (in bold), against its value measured on the generated plants (median and interquartile range). For the confounder share, the measured quantity is the factor by which the within-batch odds of a cause or sibling change from the reference, designed to be g2 .
Figure 3: Two structures that break separability (left in each panel) and the corresponding structures of the generator (right). (a) An unrecorded variable U acting on the target directly: conditioning on the direct cause X does not separate the sibling L from Y . In the generator, unrecorded variables reach Y only through its direct causes. (b) X acting on Y and Y back on X within one time step, as a controller or an operator reacting at once would: X is both a cause and a consequence of Y and has no role. In the generator, every feedback loop acts with a delay of at least one step, so that, unrolled in time, X(t) is a cause of Y(t) and Y(t) a cause of X(t+1) .
Wrong candidates
number
not separable
by averaging
also at 4 steps
descendants
3,117
14%
12%
2%
disturbance of the controlled cause
1,150
23%
18%
5%
other ancestors
7,190
9%
7%
2%
siblings
12,010
7%
4%
3%
all
23,467
9%
7%
2%
Appendix
Table 6: Wrong candidates not separable from the direct causes at the evaluation resolution, by role, over all plants. The same test at a resolution of 4 steps, close to the full resolution, splits them into two groups: by averaging , candidates separable at 4 steps ( p≥0.05 ), and also at 4 steps , candidates not separable there either.
wrong selections not separable
as scored → without them
Method
all
by averaging
also at 4 steps
share of list
first pick
precision
lift
Correlation
11%
8%
3%
10%
0.64 → 0.69
0.15 → 0.16
1.1 → 1.2
Granger
13%
10%
4%
11%
0.79 → 0.83
0.21 → 0.24
1.7 → 1.7
VARLiNGAM
16%
12%
4%
12%
0.66 → 0.70
0.30 → 0.34
2.3 → 2.4
DYNOTEARS
2 wrong selections in all
0.28 → 0.28
1.00 → 1.00
7.3 → 6.6
PC
17%
12%
5%
12%
0.25 → 0.29
0.30 → 0.35
2.3 → 2.5
Appendix
Table 7: Wrong selections of each method that are not separable at the evaluation resolution. Left: their share of the method’s wrong selections, split as in Table 6 into by averaging and also at 4 steps , and their share of all its selections ( share of list ). Right: first pick, precision and lift as in Table 3 , then recomputed with the non-separable candidates removed from the candidates of each plant.
share of the wrong selections
Method
wrong per plant
descendants
disturbance of the controlled cause
other ancestors
siblings
nothing selected
random choice
13%
5%
31%
51%
Correlation
14.7
16%
5%
30%
48%
0%
Granger
11.6
12%
6%
33%
49%
10%
VARLiNGAM
5.7
23%
8%
28%
42%
5%
DYNOTEARS
0.0
2 wrong selections in all
72%
Appendix
Table 8: Errors of each method over the 1,190 plants of all 119 configurations: the share of each role among the method’s wrong selections, against the share among all wrong candidates ( random choice ). A cell is shaded when the method picks that role more often than a random choice would, darker the more often. Nothing selected : share of plants on which the method selects nothing. The disturbance of the controlled cause (Section 4.2 ) is an ordinary ancestor with the controller off, as in all configurations where it is scored.
Figure 4: The selected lists, as in Figure 2 , for two single difficulties: a faint signal (signal share 0.1, top) and causes that act only jointly (interaction share 1, bottom).
first pick
mAP
precision
Configuration
Correlation
Granger
VARLiNGAM
DYNOTEARS
PC
PCMCI+
SpaceTime
DoubleML
Correlation
Granger
VARLiNGAM
DYNOTEARS
PC
PCMCI+
SpaceTime
DoubleML
Correlation
Granger
VARLiNGAM
DYNOTEARS
PC
PCMCI+
SpaceTime
DoubleML
Reference (5)
sint 0 (reference)
10
10
10
6
3
10
8
10
0.79
0.78
0.84
0.94
0.31
0.96
0.77
0.94
0.14
0.15
0.33
1.00
0.35
0.79
0.85
0.70
100 batches
9
9
10
6
2
10
8
10
0.83
0.76
0.87
0.93
0.31
0.96
0.71
0.88
0.15
0.17
0.27
1.00
0.37
0.78
0.94
0.83
50 batches
9
8
10
8
3
9
8
9
0.82
0.75
0.83
0.95
0.44
0.89
0.63
0.87
0.16
0.25
0.46
1.00
0.50
0.88
0.94
0.82
25 batches
9
9
9
6
2
7
9
6
0.80
0.70
0.67
0.88
0.39
0.81
0.73
0.61
0.20
0.48
0.42
0.92
0.45
0.65
1.00
0.60
Appendix
Table 9: Every configuration and method, grouped by the share or component that the configuration changes: first pick , plants (of 10) whose first selected candidate is a direct cause; mAP , mean average precision of the ranking; precision , of the selected list over the plants where the method selects something (– when it never does). Each row departs from the reference configuration in the parameters named; combined difficulties : ssig 0.3, sconf=sup 0.6, sdesc 0.5 and sint 1 (Figure 2 , bottom). Darker shading means a higher value.
Method
Threshold
length
precision
lift
first pick
nothing selected
Correlation
BH, α=0.05 (used)
16.9
0.15
1.1
64%
0%
BH, α=10−3
13.2
0.17
1.3
64%
2%
BH, α=10−6
9.6
0.22
1.7
63%
6%
BH, α=10−10
7.1
0.26
2.0
61%
11%
Bonferroni, α=0.05
14.2
0.16
1.3
64%
0%
first 3 by score
3.0
0.42
3.3
64%
0%
Appendix
Table 10: Selected lists under other thresholds, over all 1,190 plants (columns as in Table 3 ). BH: Benjamini–Hochberg adjusted p -values; first 3 by score : the three highest-ranked candidates.
Figure 5: Difficulty measured on the data. Horizontal: test strength of the median direct cause (logarithmic scale). Vertical: share of wrong candidates that are look-alikes. (a) Our configurations and three public process datasets. (b) Share of plants whose first pick is a direct cause, averaged over the eight methods; the public datasets are not evaluated (hollow).
Many benchmarks for automated causal inference evaluate a system's performance based on a single numerical output, such as an Average Treatment Effect (ATE). This approach conflates two distinct steps in causal analysis: identification, formulating a valid research design under stated assumptions, and estimation, implementing that design numerically on finite data. We introduce CausalReasoningBenchmark, a benchmark of 174 queries across 139 real-world datasets, curated from 73 peer-reviewed research papers and three widely used causal-inference textbooks. For each query, a system must produce (i) a structured identification specification that names the strategy, the treatment, outcome, and control variables, and all design-specific elements, and (ii) a point estimate with a standard error. By scoring these two components separately, our benchmark enables granular diagnosis: it distinguishes failures in causal reasoning from errors in numerical execution. Baseline results with a state-of-the-art LLM show that, while the model correctly identifies the high-level strategy in 81% of cases, full identification-specification correctness drops to only 38%, revealing that the bottleneck lies in the nuanced details of research design rather than in computation. CausalReasoningBenchmark is publicly available on Hugging Face and is designed to foster the development of more robust automated causal-inference systems.
Causal discovery is challenging in general dynamical systems because, without strong structural assumptions, the underlying causal graph may not be identifiable even from interventional data. However, many real-world systems exhibit directional, cascade-like structure, in which components activate sequentially and upstream failures suppress downstream effects. We study causal discovery in such chain-reaction systems and show that the causal structure is uniquely identifiable from blocking interventions that prevent individual components from activating. We propose a minimal estimator with finite-sample guarantees, achieving exponential error decay and logarithmic sample complexity. Experiments on synthetic models and diverse chain-reaction environments demonstrate reliable recovery from a few interventions, while observational heuristics fail in regimes with delayed or overlapping causal effects.
Panayiotis Panayiotou, Özgür Şimşek
Department of Computer Science, University of Bath, UK
Causal discovery aims to uncover causal structures from data and is fundamental to scientific reasoning and intervention-based decision making. Its evaluation relies heavily on structural causal models (SCMs), which specify a causal graph together with the mechanisms that generate data, yet existing studies differ substantially in graph families, mechanisms, and evaluation protocols. The emergence of causal discovery foundation models (CDFMs) further complicates evaluation: performance may reflect not only causal discovery ability, but also overlap between pretraining environments and test SCMs, making results on fixed synthetic benchmarks difficult to interpret. We introduce CausalArena, a unified and evolvable benchmark for causal discovery under a common protocol. Synthetic SCMs supply controlled breadth over structures and mechanisms; semantic operational SCMs provide human-auditable, semantically grounded environments beyond standard synthetic generators; and formula-grounded SCMs test discovery under explicit scientific mechanisms. Public real-world datasets provide an additional external-validity check. Experiments across classical, neural, and pretrained methods reveal substantial ranking shifts across SCM families and protocols, showing that strong performance in one benchmark regime does not reliably transfer to others. These results highlight benchmark diversity and pretraining--evaluation overlap as central challenges for evaluating causal discovery in the foundation model era.
Zi-Rong Li, Si-Yang Liu, Tian-Zuo Wang +1
School of Artificial Intelligence, Nanjing University