Harmful Partial Interventions and Benign Leakage in Concept-based Models

Gregory Verghese; Christopher J. Soelistyo; Mateo Espinosa Zarlenga; Alessandro Barp; Christopher R. S. Banerji

cs.LGOct 15, 2026

Harmful Partial Interventions and Benign Leakage in Concept-based Models

Authors: Gregory Verghese, Christopher J. Soelistyo, Mateo Espinosa Zarlenga, Alessandro Barp, Christopher R. S. Banerji

Organizations: King’s College London, UK · PharosAI, UK · The Francis Crick Institute, UK · University of Oxford, UK ·

Abstract

Artificial intelligence systems deployed in high-risk settings increasingly require models that support meaningful human oversight. Concept-based models (CMs) offer a promising framework by enabling users to inspect and intervene on intermediate, human-understandable concepts. However, concept interventions can counterintuitively worsen downstream predictions, a failure mode we formalise as intervention harm. We distinguish harm after all concepts have been intervened on from harm under partial interventions, the more likely practical use case. Modern CMs often achieve high accuracy by encoding, or leaking, additional task-relevant information in their intermediate concept representations. Recent work argues that such leakage can be benign with respect to intervention harm when concept representations satisfy two conditions: sufficiency and localisation. We prove that these criteria are not sufficient to guarantee that concept interventions improve downstream performance. Across controlled and real-world settings, we find that concept supervision incompleteness is associated with increased intervention harm. Importantly, optimising the model when all concepts are intervened on, a previously proposed leakage mitigation objective, reduces endpoint harm but does not reliably prevent harmful partial corrections. Hence, we introduce a complementary monotonic intervention objective that directly penalises increases in task loss along partial intervention trajectories. Combining endpoint and monotonic intervention training substantially reduces the prevalence and magnitude of partial intervention harm while preserving concept and task predictive performance.

Figures & tables

Figure 1: Endpoint ( $\mathcal{L}_{\text{int}}$ ) and monotonic ( $\mathcal{L}_{\text{mono}}$ ) objectives control complementary components of intervention harm. (A) Adjusted standardised associations of $\omega$ , $\lambda_{\mathrm{int}}$ and $\lambda_{\mathrm{mono}}$ with endpoint harm, maximum partial harm, and harmful-path prevalence. Points show regression coefficients with 95% confidence intervals. (B) Mean endpoint harm, (C) maximum partial harm, and (D) harmful-path prevalence across $\lambda_{\mathrm{int}}$ and $\lambda_{\mathrm{mono}}$ . In these plots, we highlight the cell with the lowest mean.

Figure 2: $\mathcal{L}_{\mathrm{mono}}$ suppresses harmful partial intervention trajectories on CUB . (A) Mean endpoint harm, (B) maximum partial harm, and (C) harmful-path prevalence across ten concept incompleteness strata (higher means higher incompleteness $\Omega_{\mathcal{C}}^{\mathrm{pre}}$ ). We show Hybrid CBMs trained (1) w/o intervention-aware losses ( baseline ) , (2) with $\mathcal{L}_{\mathrm{int}}$ , and (3) with $\mathcal{L}_{\mathrm{int}}+\mathcal{L}_{\mathrm{mono}}$ . Points show means, and bars show 95% confidence intervals. (D–L) We show partial-intervention trajectories from (D, G, J) low-, (E, H, K) intermediate-, and (F, I, L) high-incompleteness strata. Columns show intervention paths for different modalities: (D–F) baseline, (G–I) $\mathcal{L}_{\text{int}}$ only, and (J–L) $\mathcal{L}_{\mathrm{int}}+\mathcal{L}_{\mathrm{mono}}$ . Faint lines show all 512 sampled intervention paths, with the median, most harmful , and least harmful paths highlighted. Harm is measured relative to the observational state, $R(S)-R(\varnothing)$ (i.e., higher is worse).

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 3: Adjusted determinants of intervention harm and model performance. Adjusted standardised associations with (A) endpoint harm; (B) maximum partial harm; (C) harmful-path prevalence; (D) task accuracy; and (E) concept mean-squared error. Points show regression coefficients and bars show 95% cluster-robust confidence intervals across 60 matched synthetic worlds. Outcomes and continuous predictors were standardised. Covariance contrasts use the block-independent construction as the reference; the stop-gradient contrast was estimated among models with

\lambda_{\mathrm{mono}}>0

Figure 4: Robustness to stop-gradient and concept covariance structure. (A) Concept mean-squared error with and without stopping gradients from

\mathcal{L}_{\mathrm{mono}}

through the concept encoder. Values were first averaged within each of 60 matched synthetic worlds over

\lambda_{\mathrm{int}}

and

\omega

; points and bars show means and 95% confidence intervals across worlds. (B–D) Endpoint harm, maximum partial harm, and harmful-path prevalence as functions of

\omega

under block-independent, diagonal, and correlated positive-definite covariance structures. Stop-gradient-enabled models were averaged over the regularisation grid within each matched seed–

k^{\prime}

unit. Points and bars show means and 95% confidence intervals across 20 units per covariance condition and value of

\omega

Figure 5: Effect of the number of monotonic edges sampled per minibatch. Mean (A) maximum partial harm; (B) endpoint harm; (C) harmful-path prevalence; (D) task accuracy; and (E) concept mean-squared error as the number of sampled monotonic edges

m_{\mathrm{mono}}

was varied. The ablation fixed

\lambda_{\mathrm{int}}=10

\lambda_{\mathrm{mono}}=100

, block-independent concept covariance, and stop-gradient. Points and bars show means and 95% confidence intervals across 50 models spanning two values of

k^{\prime}

, five values of

\omega

, and five seeds. Shading denotes

m_{\mathrm{mono}}\geq 5

, beyond which the reduction in maximum partial harm largely plateaued.

Figure 6: Construction of fixed-size concept sets spanning pre-model incompleteness. (A) Distribution of

\Omega_{\mathcal{C}}^{\mathrm{pre}}

across 210 candidate sets containing 50 concepts each. Blue marks identify the 80 sets selected for model training. (B) Pre-model incompleteness of the eight selected sets within each of ten quantile strata; orange lines show stratum means. Higher

\Omega_{\mathcal{C}}^{\mathrm{pre}}

indicates that more task-relevant information remained unexplained by the selected concepts.

Figure 7: Adjusted associations with model performance and concept quality on CUB . Adjusted standardised associations with (A) top-1 accuracy and (B) concept mean-squared error. Each panel shows the association with a one-standard-deviation increase in pre-model incompleteness,

\Omega_{\mathcal{C}}^{\mathrm{pre}}

, and the adjusted differences between endpoint training or endpoint-plus-monotonic training and the baseline regime. Points show regression coefficients and bars show 95% cluster-robust confidence intervals; dashed vertical lines denote no association. Outcomes and

\Omega_{\mathcal{C}}^{\mathrm{pre}}

were standardised. Models included all 240 runs across 80 matched concept sets, with baseline as the regime reference and standard errors clustered by concept-set identifier.

Concept Bottleneck Models

Concept Bottleneck Models route predictions through human-understandable concepts that people can inspect, correct, or intervene on.

See all 88 papers

Jun 9, 2026

In Defense of Information Leakage in Concept-based Models

Concept-based models (CMs), deep neural networks that ground their predictions on representations aligned with human-understandable concepts (e.g., "round", "stripes", etc.), have been shown to learn representations that leak concept-irrelevant information. As the traditional narrative goes, this leakage is undesirable and should be eradicated as it leads to uninterpretable models. In this paper, we posit that this conventional view of leakage in CMs is not only ill-posed, as the evidence of how leakage makes a model less interpretable is often inconclusive, but also bound to lead to impractical CMs under common real-world constraints. Specifically, we argue that in real-world settings where concept incompleteness is the norm, some leakage is often necessary for constructing accurate and intervenable CMs. To this end, we propose that there is such a thing as benign leakage and show that, by optimizing a reframing of the typical CM training objective, CMs can encourage and exploit this form of leakage without sacrificing accuracy or intervenability.

Mateo Espinosa Zarlenga

University of Oxford, UK.

Concept Bottleneck Models Interpretable ML

Ask PDF

Oct 16, 2026

Residual Stochastic Concept Bottleneck Models

Concept-based models (CMs) learn human-interpretable intermediate representations that enable users to understand the model's reasoning and steer its predictions through test-time concept interventions. However, in realistic settings, their performance can be limited by concept incompleteness, where the training concepts fail to capture all task-relevant information. Adding an unconstrained residual pathway to the model's predictive process can overcome the consequences of incompleteness. However, this introduces an inscrutable, leak-prone prediction pathway that may weaken concept interventions, as these often leave residual variables unchanged. To address this limitation, we introduce Residual Stochastic Concept Bottleneck Models (Res-SCBMs), a family of concept-based models that jointly model concept and residual variables through a multivariate Gaussian distribution. Res-SCBMs learn a covariance structure that explicitly captures dependencies between the known concepts and learnt residual variables, allowing interventions to propagate to statistically dependent residual variables through Gaussian conditioning. This extends the intervention capabilities of previous CMs without requiring concept completeness during training. Our results show that, across multiple concept-incomplete benchmarks, Res-SCBMs improve predictive performance over competing baselines while retaining effective test-time concept interventions.

Stephen Marcou, Mateo Espinosa Zarlenga, Sonia Laguna +3

ETH Zurich · University of Oxford · University of Cambridge

Concept Bottleneck Models

Ask PDF

Jun 5, 2025

Towards Reasonable Concept Bottleneck Models

We propose a novel, flexible, and efficient framework for designing Concept Bottleneck Models (CBMs) that enables practitioners to explicitly encode and extend their prior knowledge and beliefs about the concept-concept (

C-C

) and concept-task (

C \to Y

) relationships within the model's reasoning when making predictions. The resulting

\textbf{C}

oncept

\textbf{REA}

soning

\textbf{M}

odels (CREAMs) architecturally encode arbitrary types of

C-C

relationships such as mutual exclusivity, hierarchical associations, and/or correlations, as well as potentially sparse

C \to Y

relationships. Moreover, CREAM can optionally incorporate a regularized side-channel to complement the potentially {incomplete concept sets}, achieving competitive task performance while encouraging predictions to be concept-grounded. To evaluate CBMs in such settings, we introduce a

C \to Y

agnostic metric that quantifies interpretability when predictions partially rely on the side-channel. In our experiments, we show that, without additional computational overhead, CREAM models support efficient interventions, can avoid concept leakage, and achieve black-box-level performance under missing concepts. We further analyze how an optional side-channel affects interpretability and intervenability. Importantly, the side-channel enables CBMs to remain effective even in scenarios where only a limited number of concepts are available.

Nektarios Kalampalikis, Kavya Gupta, Georgi Vitanov +1

Department of Computer Science Saarland University, Germany

Concept Bottleneck Models Interpretable ML

Ask PDF

Abstract

Concept Bottleneck Models

Explore similar work

In Defense of Information Leakage in Concept-based Models

Residual Stochastic Concept Bottleneck Models

Towards Reasonable Concept Bottleneck Models