A Concept-Based Interpretable Framework for Clinical Text-based Prediction via Token-Concept Attention without Concept Supervision
Organizations: Department of Data Science, Dana-Farber Cancer Institute, 3 Blackfan St, Boston, 02115, Massachusetts, United States · Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, 610 Walnut St, Madison, 53726, Wisconsin, United States · Department of Medicine, University of Wisconsin-Madison, 610 Walnut St, Madison, 53726, Wisconsin, United States
Abstract
Objective: Concept-based interpretation has emerged as a promising approach for explaining blackbox models using human-understandable, high-level concepts. However, most existing methods require annotated concept labels to train the model to identify concepts in the input, and obtaining such annotations in clinical text is often expensive. This study aims to enable concept-based interpretation for language models without concept supervision on clinical prediction tasks. Methods: We propose a novel framework that enables concept-based interpretation without concept supervision through a tailored cross-attention bottleneck. Clinical text and the text descriptions of medical concepts are first tokenized and embedded by a pretrained language model. A cross-attention module then estimates the presence of each concept for each text token, producing concept relevance scores that capture how concepts are involved throughout the note. These scores are pooled and used in a linear prediction layer to model clinical outcomes in an interpretable manner. Results: The performance of the proposed framework was compared against blackbox models across two tasks: chest trauma severity stratification using a private dataset and an in-hospital mortality prediction task using the publicly available MIMIC-III dataset. The proposed framework achieved AUROC values within 0.02 of the corresponding blackbox models. The framework also enabled cohort- and patient-level interpretation, capturing the overall concept importance across the cohort and patient-specific concept contributions. Conclusion: Our proposed framework realizes concept-based interpretation without requiring ground-truth concept presence labels. It improves the transparency of clinical text predictions while maintaining strong performance in clinical prediction tasks.
Figures & tables
| Chest trauma severity stratification | In-hospital mortality prediction | |||
| Development | Test | Development | Test | |
| Encounters, n | 7,693 | 1,983 | 36,803 | 9,291 |
| Age, median (IQR) | 59 (38,75) | 60 (44,77) | 65 (52,77) | 65 (52,77) |
| Female, n (%) | 3,225 (41.9) | 855 (43.1) | 16,078 (43.7) | 4,078 (43.9) |
| White, n (%) | 7,096 (92.2) | 1,802 (90.9) | 26,449 (71.9) | 6,635 (71.4) |
| Number of pre-truncation tokens by backbone | ||||
| Chest trauma severity | In-hospital mortality | |||
|---|---|---|---|---|
| AUROC | AUPR | AUROC | AUPR | |
| SapBERT blackbox | ||||
| SapBERT + Standard CBM | ||||
| Ours (SapBERT backbone) | † | † | † | † |
| ClinicalBERT | ||||
| ClinicalBERT + Standard CBM | ||||
| Task | SapBERT-Backbone | ClinicalBERT-Backbone | BGE-Backbone |
|---|---|---|---|
| Chest trauma severity | 1) Rib, sternal, and thoracic spine fracture 2) Pyothorax 3) Pneumothorax | 1) Rib, sternal, and thoracic spine fracture 2) Pyothorax 3) Pneumothorax | 1) Pleural plaque 2) Pleural effusion 3) Pneumothorax |
| In-hospital mortality | 1) Hypotension 2) Hepatic failure, not elsewhere specified 3) Pleural effusion in conditions classified elsewhere | 1) Cardiac arrest 2) Adverse effects, not elsewhere specified 3) Respiratory tuberculosis | 1) Tuberculosis of nervous system 2) Heart Failure 3) Respiratory failure |
| Task | Backbone | Concept | LLM confirmed / Our highlighted (%) |
|---|---|---|---|
| Mortality | SapBERT | Hypotension | 38/49 (77.6) |
| ClinicalBERT | Cardiac arrest | 2/21 (9.5) | |
| BGE | Nervous tuberculosis | 48/344 (14.0) | |
| Chest Trauma | SapBERT | Rib fracture | 294/418 (70.3) |
| ClinicalBERT | Rib fracture | 351/643 (54.6) | |
| BGE | Pleural plaque | 251/464 (54.1) |
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
| Category | Hyperparameter | Value / setting |
|---|---|---|
| Concept vocabulary | Concept source | Three-digit ICD-10 concept vocabulary. |
| Concept pooling | [CLS] pooling. | |
| Concept maximum length | 256 tokens. | |
| Concept embedding batch size | ||
| Backbone encoder | Pretrained model | SapBERT . |
| Input maximum length | 512 tokens. |
| Backbone | Perturbation | Masked | Control |
|---|---|---|---|
| SapBERT | Blackbox | ||
| Negative Top-5 | |||
| Negative Top-10 | |||
| Positive Top-5 | |||
| Positive Top-10 | |||
| ClinicalBERT | Blackbox | ||
| Backbone | Perturbation | Perturbed | Control |
|---|---|---|---|
| SapBERT | Blackbox | ||
| Negative Top-5 | |||
| Negative Top-10 | |||
| Positive Top-5 | |||
| Positive Top-10 | |||
| ClinicalBERT | Blackbox | ||
| Chest trauma severity | In-hospital mortality | |||
|---|---|---|---|---|
| AUROC | AUPR | AUROC | AUPR | |
| SapBERT blackbox | ||||
| Without warmup (SapBERT backbone) | ||||
| Ours (SapBERT backbone) | † | † | † | † |
| ClinicalBERT | ||||
| Without warmup (ClinicalBERT backbone) | ||||
| Chest trauma | In-hospital mortality | ||||
|---|---|---|---|---|---|
| Backbone | Top- | Positive-class concepts | Negative-class concepts | Positive-class concepts | Negative-class concepts |
| Tokens removed: mean number (SD) / mean percentage (SD %) | |||||
| SapBERT | 5 | 85 (37) / 21 (10)% | 22 (9) / 5 (2)% | 103 (41) / 25 (12)% | 30 (9) / 8 (3)% |
| 10 | 127 (46) / 32 (11)% | 34 (10) / 8 (3)% | 120 (54) / 32 (15)% | 40 (13) / 10 (4)% | |
| ClinicalBERT | 5 | 101 (46) / 23 (12)% | 29 (11) / 5 (2)% | 114 (57) / 28 (13)% | 36 (10) / 8 (3)% |
| 10 | 143 (58) / 36 (12)% | 49 (11) / 11 (3)% | 152 (69) / 37 (17)% | 46 (15) / 11 (4)% | |
Concept-Based Explanations
Concept-based explanations make AI decisions legible through human-understandable features and test how those features affect predictions.
See all 83 papers