Causal Concept Explanations for Deep Neural Models
Authors: Josh Rountree, Pulkit Verma, Oswin So, Chuchu Fan, Julie A. Shah
Organizations: Computer Science and Artificial Intelligence Laboratory, MIT · Indian Institute of Technology Madras · Laboratory for Information and Decision Systems, MIT
Concept-based explanations in neural models are important for the development, certification, and adoption of autonomous systems. Yet, interpretability remains correlational. Without methods to guide and quantify causal relationships between explanations, a concept explanation can be provided without being necessary or sufficient for the observed output thereby appearing correct while misleading. We introduce Causal Concept Wrapper Network (CCW-Net), the first use of interventional mediation analysis from fields such as economics, medicine, and epidemiology as a training objective to align causal effects of concept-based explanations with a model's output. We further apply mediation analysis as an evaluation tool to quantify, per concept label combination, how much of any model's output is causally attributable to each concept explanation. As a training objective, CCW-Net shapes a model's concept representations such that they are both necessary and sufficient for the outputs they are credited with and independent of other concepts' contributions. Counterfactual concept representations anchor a set of causal alignment criteria: necessity, sufficiency, and independence. Necessity drives irrelevant concept effects toward zero, sufficiency ensures relevant concept representations fully reconstruct a model's output, and independence enforces the total effect decomposes linearly across concepts. We evaluate CCW-Net across aircraft control, driving, and image classification. CCW-Net provides a step toward concept-based explanations that are not merely interpretable, but quantifiably causal, supporting safer evaluation, certification, and deployment of transparent and trustworthy neural models.
Figures & tables
Figure 1: CCW-Net architecture. A concept encoder gθ maps the frozen backbone’s latent Z to one squashed vector Mj per concept, whose length encodes the concept’s presence and whose direction encodes its context-dependent expression, and a bias-free attention head hϕ maps M to the wrapper output Y^ . Causal alignment losses (weights and the auxiliary presence term omitted) are added to the task and concept losses, so that training shapes, and evaluation measures, how much each concept causally drives Y^ .
Table 1: Aircraft control: fraction of the effect each concept carries (Eq. 7 ), per concept label combination, estimated as in Sec. 5.4 . Active concepts are in bold. All Active and All Inactive are ρactive and ρinactive . Each block’s header gives the model’s test action MSE (tMSE), concept accuracy on the Lead/Lag and Climb/Dive concept sets (%), and DCI disentanglement / completeness / informativeness. Means over ten seeds with two-sided 95% Student- t intervals. CCW-Net shows superior causal effect routing through active concepts compared to prior art.
Table 2: CarRacing: fraction of the effect each concept carries (Eq. 7 ), per concept label combination, estimated as in Sec. 5.4 . Active concepts are in bold. All Active and All Inactive are ρactive and ρinactive . Each block’s header gives the model’s test action MSE (tMSE), concept accuracy on the Straight/Left/Right concept set (%), and DCI disentanglement / completeness / informativeness. Means over ten seeds with two-sided 95% Student- t intervals. CCW-Net shows superior causal effect routing through active concepts compared to prior art.
Concept
Concept Effect (%)
DCI
Model
acc. (%)
All Active ↑
All Inactive ↓
Disentanglement
Completeness
Informativeness
CCW-Net (Ours)
99.6 ± 0.0
91.3 ± 0.3
8.7 ± 0.3
0.895 ± 0.007
0.936 ± 0.007
0.993 ± 0.001
CBM
97.7 ± 0.1
41.6 ± 0.6
58.4 ± 0.6
0.732 ± 0.011
0.789 ± 0.011
0.985 ± 0.001
Table 3: CUB-200: fraction of the effect the active concepts carry, per input (Sec. 5.4 ), averaged over the test split. CBM is the concept bottleneck model of Koh et al. (2020) . No CUB-200 model is trained with the causal alignment losses. Concept accuracy is over the 112 attributes. Means over ten seeds with two-sided 95% Student- t intervals.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Symbol
Definition
Inputs and outputs
X
Input
Y
Expert output
Y^
Wrapper output, hϕ(M)
cj∈{0,1}
Concept label ( 1= active)
Architecture
Appendix
Table 4: Variable dictionary for CCW-Net.
Figure 2: The CCW-Net wrapper SCM. Every path from X to Y^ passes through the concept representation M , and there is no direct Z→Y^ edge. This gives mediator-outcome ignorability within the SCM (Prop. 2 ).
Domain
λcls
λnec
λsuf
λind
λpres
β
Aircraft control
1.0
0.1
0.1
0.1
0.1
0.5
CarRacing
1.0
0.1
0.1
0.1
0.1
1.0
CUB-200, CCW-Net
1.0
—
—
—
—
—
CUB-200, CBM
0.01
—
—
—
—
—
Appendix
Table 5: Loss weights per domain. The CUB-200 models train without causal alignment losses.
Domain
Wrapper learning rate
Flow learning rate
Batch size
Aircraft control
5×10−4
1×10−3
2048
CarRacing, Table 2
1×10−3
1×10−3
32
CarRacing, App. I canonical
1×10−3
1×10−3
128
CUB-200
5×10−4
—
512
Appendix
Table 6: Optimizer settings per domain. The CUB-200 models use no mediator model as they had no causal alignment losses active. App. I lists the batch size of each CarRacing ablation.
Domain
Warmup epochs Ew
Total epochs E
Flow fit epochs
Refit interval Er
Aircraft control
10
100
50
10
CarRacing
10
50
50
10
CUB-200
100
100
—
—
Appendix
Table 7: Training schedule per domain. The mediator model is fit for 50 epochs at the start of alignment and refit every 10 alignment epochs to refresh with the current concept representations. Each alignment step uses S=8 counterfactual draws per concept. The CUB-200 models train the task and concept losses for all 100 epochs.
Concept acc. (%)
Concept Effect (%)
DCI
Model
tMSE
Lead/Lag
Climb/Dive
All Active ↑
All Inactive ↓
Disentanglement
Completeness
Informativeness
Ours
CCW-Net (Ours, 500K samples)
0.123 ± 0.001
96.4 ± 0.1
99.5 ± 0.0
97.3 ± 1.1
2.7 ± 1.1
0.999 ± 0.001
0.748 ± 0.050
0.979 ± 0.000
CCW-Net (Ours, 5M samples)
0.120 ± 0.000
97.5 ± 0.0
99.7 ± 0.0
98.3 ± 0.9
1.7 ± 0.9
1.000 ± 0.000
0.780 ± 0.043
0.986 ± 0.000
CCW-Net (Ours, scalar D=1 )
0.412 ± 0.003
96.2 ± 0.1
99.4 ± 0.0
69.9 ± 11.2
30.1 ± 11.2
0.998 ± 0.001
0.896 ± 0.065
0.978 ± 0.001
No causal alignment losses
Appendix
Table 8: Aircraft control: full ablation matrix. tMSE is the test action MSE, and concept accuracy is on the Lead/Lag and Climb/Dive concept sets (%). With a placebo concept added, the placebo carries 36.2±11.4% of the effect at width 256, and 28.2±9.8% with and 98.2±2.9% without the alignment losses at width 64.
Concept Effect (%)
DCI
Model
tMSE
Concept acc. (%)
All Active ↑
All Inactive ↓
Disentanglement
Completeness
Informativeness
Ours
CCW-Net (Ours, β=1.0 )
0.0025 ± 0.0004
93.4 ± 0.2
75.3 ± 3.8
24.7 ± 3.8
0.914 ± 0.015
0.706 ± 0.042
0.950 ± 0.004
CCW-Net (Ours, scalar D=1 )
0.0420 ± 0.0006
93.0 ± 0.3
92.4 ± 2.7
7.6 ± 2.7
0.866 ± 0.021
0.866 ± 0.021
0.947 ± 0.005
No causal alignment losses
CCW-Net ( D=16 )
0.0032 ± 0.0004
93.4 ± 0.3
63.5 ± 9.0
36.5 ± 9.0
0.913 ± 0.011
0.680 ± 0.034
0.950 ± 0.003
Appendix
Table 9: CarRacing: full ablation matrix. tMSE is the test action MSE, and concept accuracy is on the Straight/Left/Right concept set (%). Every group except the last uses batch size 128 for the canonical, D=16 no-alignment, loss-combination, label-noise, and placebo rows and 32 for the other rows. The last group uses batch size 32. Every row with the presence loss uses β=1 except CCW-Net (Ours, scalar D=1 ) and the two CW-Net rows with alignment, which use β=0.5 . For the Remove rows, the averages cover the label combinations with an active concept, since the removed concept’s own combination has none. § In the Merge rows, the estimator scores only the merged concept’s label combination, because the remaining combination has no comparison that holds the merged concept fixed. With a placebo concept added, the placebo carries 34.6±6.8% of the effect.