The STAR (Student-Teacher Achievement Ratio) experiment (1985, Tennessee, USA) is a landmark hierarchical dataset designed to assess the impact of class size on student outcomes, with observations nested within classes. To encode class-level interventions in such hierarchical settings, we develop a complete, scalable, open-source pipeline for Hierarchical Structural Causal Models (HSCM) that bridges symbolic identification and practical estimation. Our approach integrates graph transformations, pyAgrum's do-calculus for automatic identification of causal effects, adaptation of symbolic expression into closed-form HSCM formulas, and numerical estimation from fitted local probability models. A key innovation is our adapted Abstract Syntax Tree (AST), which decomposes pyAgrum's identified formulas into independent density, expectation, and marginalization tasks, enabling parallel and scalable computation. We validate the pipeline on canonical HSCM motifs and benchmark scenarios with known ground truth, then apply it to STAR kindergarten mathematics outcomes. The results show that flat baselines (ignoring hierarchy) recover associations but fail to encode class-level interventions, and that symbolic identification alone is not enough for practical Hierarchical Structural Causal inference; scalable estimation and numerical stability checks are central parts of the scientific object.
Figures & tables
Figure 1: An example of hierarchical causal model and its transformation. (a) an HSCM with n units, each of which containing m subunits, (b) HCGM with explicit HCM’s latent Q variables, (c) HCM’s matching collapsed model and (d) its augmented/marginalized version. We apply do-calculus to this final model to perform identification in the original HCM.
Figure 2: Pipeline for causal identification in Hierarchical Structural Causal Models (HSCMs).
Figure 3: An example of hierarchical causal model and its transformation. (a) an HSCM with n units, each of which containing m subunits, (b) HCGM with explicit HCM’s latent Q variables, (c) HCM’s matching collapsed model and (d) its augmented/marginalized version. We apply do-calculus to this final model to perform identification in the original HCM.
Step
Sequential cost
Parallelization structure
Effective cost with P workers
Collapse/augmentation
O(e+v) graph operations
graph-level (negligible)
O(e+v)
Build pyAgrum model
O(e+v) (copy plus identification search)
symbolic backend
O(e+v)
AST translation
O(K) expression traversal
factor-level traversal
O(K)
Per-unit Q fitting
O(nmd) for simple parametric factors
independent accross units and factors
∼O(nmd/P) + scheduling overhead
Monte Carlo evaluation
O(BK) per intervention
Independent across samples, units, and branches
∼O(BK/P) when branches are balanced
Table 1: Algorithmic cost and parallelization structure of the pipeline. The effective cost assumes ideal parallelization across P workers. We denote n is the number of units, m is the average number of sub-units per unit, d is the maximum parent count, v and e are the numbers of transformed graph nodes and edges, K is the number of density factors in the identified formula, B is the number of Monte Carlo samples or quadrature points.
Figure 5
Figure 6: Class-level STAR diagnostics. Left: gender heterogeneity across classes, showing why an additive individual gender coefficient does not represent the full distribution of class composition. Right: reading and mathematics class-level score means, together with a standardized Q-Q diagnostic for Mathematics. In the Q-Q panel, blue points are empirical quantiles and the dashed black line is the Gaussian reference. The green band marks the central range where the Gaussian approximation is most relevant, while the red bands mark lower and upper score extremes. The rightmost extreme is kept as an empirical check rather than fitted separately because it contains only a few repeated score values.
Model
Specification
Coef.
SE
p -value
R2
M1
No controls
β^A =8.096
1.597
4.0×10−7
0.006
M2
School fixed effects
β^A =9.499
1.463
8.5×10−11
0.219
M3
Student controls
β^A =9.380
1.415
3.4×10−11
0.268
M4
Full OLS replication
β^A =9.432
1.419
3.0×10−11
0.269
M5
IV class-size contrast
θ^ =-1.219
0.163
7.0×10−14
–
M6
Random class intercept
β^A =9.099
2.263
5.8×10−5
–
Table 3: Baseline regression estimates for kindergarten mathematics. For M1–M4 and M6, the reported coefficient is β^A , the coefficient of the small-class assignment indicator Aij . For M5, the reported coefficient is the IV class-size coefficient θ^ .
Figure 7: ExactBIC-derived graphs for the STAR HSCM analysis. The left panel shows the directed ExactBIC graph over the flat STAR variables, after adding the unit-level context used by the HSCM analysis. The right panel shows the corresponding collapsed and augmented Q -variable graph for the Mathematics outcome. Blue nodes are unit-level context variables, orange nodes are observed variables or fitted Q -mechanisms, and the green node is the target class-level Mathematics outcome distribution Qy . Solid arrows are retained directed dependencies; dashed red arrows mark factors normalized in the stabilized numerical evaluation.
Symbol
Type
Interpretation
Qa
Bernoulli/proportion
Small-class assignment distribution within a class
Qg
Bernoulli/proportion
Gender composition within a class
Qe
Bernoulli/proportion
Ethnicity composition within a class
Ql
Bernoulli/proportion
Free-lunch composition within a class
Qb
Gaussian score mechanism
Reading score distribution for the class
Qy
Gaussian score mechanism
Mathematics score distribution for the class
Table 4: Class-level Q -variables used by the STAR HSCM evaluator. These are distributional summaries or mechanisms fitted at the class level from the student-level variables in Table 2 .
Run
Target
E[⋅∣do(1)]
E[⋅∣do(0)]
ATE
ExactBIC
Mathematics outcome ( Y )
2402.75
2365.95
36.80
ExactBIC, normalized factors
Reading ( B ) diagnostic
1788.10
1766.47
21.63
Table 5: STAR HSCM estimates and normalized-factor diagnostic. The Mathematics row is the main outcome analysis, where the outcome is denoted Y ; the Reading row is a separate diagnostic run used to inspect the normalized P(Qb∣a,g∣S) factor.
Figure 8: Estimated class-level HSCM response parameters and means in STAR. Reading ( Qb ) and the Mathematics outcome ( Qy ) are represented by Gaussian summaries (μ^i,σ^i2) , and the right panel reports the joint class means used to inspect the relationship between the two score mechanisms. In the right panel, colors encode the class-level urbanicity category S ; the panel is a descriptive diagnostic rather than an oriented causal graph.
Figure 9: Estimated class-level HSCM probability quantities used in the STAR analysis. Binary sub-unit variables are represented by class-level Bernoulli probabilities for Small-class assignment ( Qa ), Gender ( Qg ), Ethnicity ( Qe ), and free lunch ( Ql ); Ql∣e summarizes the free-lunch mechanism conditional on Ethnicity; and S is the observed class-level Urbanicity variable. In the conditional free-lunch scatter, point colors encode the urbanicity category S ; in the S panel itself, the bar color is only a visual grouping choice.
We introduce sHAIL-Causal, the causal specialization of the Saturated Hierarchical Atomic Incremental Learning (sHAIL) paradigm: a sequential staircase procedure that ascends a nested hierarchy of hypothesis classes H_0 < H_1 < ... < H_K once a saturation signal indicates that mastery of the current stage has plateaued. Where general sHAIL leaves the saturation criterion open, sHAIL-Causal instantiates it with a joint criterion of goodness-of-fit saturation and cross-environment invariance, replacing the complexity control of Structural Risk Minimization. We show, theoretically and by simulation, that complexity-only staircases are seduced by confounded predictors that lower empirical risk without reflecting stable causal structure, whereas an invariance-gated staircase provably halts at the true causal predictor set under a per-variable Richness condition. We show that naive greedy search fails to recover the causal set even under Richness, trace the failure to non-monotonicity of the invariance statistic along single-variable paths, and validate a fix combining bounded-exhaustive block-seeding with a calibrated acceptance threshold. We then extend the guarantee to environments arriving sequentially, yielding a confidence guarantee that stays valid at every arrival, which one-shot exhaustive search cannot offer without repeating its full combinatorial search. We close by formalizing the intervention of a wise teacher who lifts a saturated learner off a plateau of boredom.
Ernest Fokoué
School of Mathematics and Statistics, Rochester Institute of Technology
The Half-Trek Criterion (HTC) is the primary graphical tool for determining generic identifiability of causal effect coefficients in linear structural equation models (SEMs) with latent confounders. However, HTC is inherently node-wise: it simultaneously resolves all incoming edges of a node, leaving a gap of "inconclusive" causal effects (15-23% in moderate graphs). We introduce Iterative Identification Closure (IIC), a general framework that decouples causal identification into two phases: (1) a seed function S_0 that identifies an initial set of edges from any external source of information (instrumental variables, interventions, non-Gaussianity, prior knowledge, etc.); and (2) Reduced HTC propagation that iteratively substitutes known coefficients to reduce system dimension, enabling identification of edges that standard HTC cannot resolve. The core novelty is iterative identification propagation: newly identified edges feed back to unlock further identification -- a mechanism absent from all existing graphical criteria, which treat each edge (or node) in isolation. This propagation is non-trivial: coefficient substitution alters the covariance structure, and soundness requires proving that the modified Jacobian retains generic full rank -- a new theoretical result (Reduced HTC Theorem). We prove that IIC is sound, monotone, converges in O(|E|) iterations (empirically <=2), and strictly subsumes both HTC and ancestor decomposition. Exhaustive verification on all graphs with n<=5 (134,144 edges) confirms 100% precision (zero false positives); with combined seeds, IIC reduces the HTC gap by over 80%. The propagation gain is gamma~4x (2 seeds identifying ~3% of edges to 97.5% total identification), far exceeding gamma<=1.2x of prior methods that incorporate side information without iterative feedback.
Ziyi Ding, Xiao-Ping Zhang
Tsinghua Shenzhen International Graduate School, Tsinghua University, Shenzhen, China
Here we introduce Partially Observed Structural Causal Models (POSCMs) as an extension of structural causal models (SCMs) to settings where upstream contexts co-determine both the interaction structure and downstream mechanisms on observed variables. POSCMs thus provide a self-contained causal modeling framework for endogenous graphs, allowing for an intervention hierarchy spanning node- and edge-level contexts and endogenous variable interventions. To define edge interventions, we separate node mechanisms into edge-local transmission channels that can be modified without changing the source node or the rest of the target mechanism. We provide an identifiability theory that clarifies which intervention families would suffice to disentangle structure formation from mechanisms. We then empirically validate these theoretical results in two external simulators: a biophysically detailed virtual human retina and a gene-regulatory analogue. The experiments reproduce non-identifiability under latent context, expose structure-mechanism confounding under latent edges, and recover pathway-level input-output relationships under targeted interventions, consistent with our positive Markov kernel identifiability results. Together, POSCMs provide an intervention-oriented framework for causal systems in which contexts, graph structure, mechanisms, and measurements are jointly generated and only partially observed.
Turan Orujlu, Jordan Matelsky, Martin V. Butz +2
University of Tübingen · MPI for Biological Cybernetics · University of Pennsylvania +1