Concept Bottleneck Models

Also known as CBM

Overview

Concept Bottleneck Models predict through an intermediate set of meaningful attributes, such as visual features or clinical findings, so people can examine how those concepts contribute to an output. Approaches include supervised concept predictors with simple classifiers, vision-language models that propose concepts, and structured bottlenecks that model concept relations, hierarchies, or spatial locations. The methods are tested in settings such as fine-grained image classification, medical diagnosis, text and graph prediction, and multimodal tasks, with evaluation increasingly examining concept accuracy and interventions as well as task accuracy. A central tension is that incomplete or noisy concepts can hurt predictions, while information leakage can preserve accuracy but make explanations or user interventions unreliable.

Latest papers

92 papers9 papers in the last four weeks, down 18% on the four weeks before. 0.1% of all new papers.

Jun 5, 2025

Towards Reasonable Concept Bottleneck Models

We propose a novel, flexible, and efficient framework for designing Concept Bottleneck Models (CBMs) that enables practitioners to explicitly encode and extend their prior knowledge and beliefs about the concept-concept (C−CC-C) and concept-task (C→YC \to Y) relationships within the model's reasoning when making predictions. The resulting C\textbf{C}oncept REA\textbf{REA}soning M\textbf{M}odels (CREAMs) architecturally encode arbitrary types of C−CC-C relationships such as mutual exclusivity, hierarchical associations, and/or correlations, as well as potentially sparse C→YC \to Y relationships. Moreover, CREAM can optionally incorporate a regularized side-channel to complement the potentially {incomplete concept sets}, achieving competitive task performance while encouraging predictions to be concept-grounded. To evaluate CBMs in such settings, we introduce a C→YC \to Y agnostic metric that quantifies interpretability when predictions partially rely on the side-channel. In our experiments, we show that, without additional computational overhead, CREAM models support efficient interventions, can avoid concept leakage, and achieve black-box-level performance under missing concepts. We further analyze how an optional side-channel affects interpretability and intervenability. Importantly, the side-channel enables CBMs to remain effective even in scenarios where only a limited number of concepts are available.
Feb 13, 2025

Hierarchical Vision Transformer with Prototypes for Interpretable Medical Image Classification

Explainability is an important requirement for machine learning models in high-risk domains such as medicine. We introduce HierViT, a hierarchical Vision Transformer that combines competitive predictive performance with multiple mechanisms to support the interpretation of its predictions. The model first predicts clinically relevant visual appearance attributes, which are then used to infer the final target prediction through a hierarchical reasoning process. To facilitate understanding of the intermediate representations, attribute-specific prototypes are visualized as representative image examples, while attribute-specific attention maps highlight image regions associated with each visual feature. Together, these attribute-level components provide complementary views of the model's reasoning by linking representative image examples and spatial attention to semantically defined visual features. We evaluate HierViT on two medical benchmark datasets: LIDC-IDRI for lung nodule assessment and derm7pt for skin lesion classification. Across both datasets, HierViT achieves competitive predictive performance while providing attribute-level prototypes and attention visualizations that support the interpretation of its predictions.