Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction

Jin Mu; Guanhua Chen

cs.CLAug 27, 2026

Making Clinical Language Models Auditable: Concept-Guided Fine-Tuning for Robust Prediction

Authors: Jin Mu, Guanhua Chen

Organizations: University of Wisconsin–Madison

Abstract

Clinical language models can achieve strong in-hospital accuracy yet fail under deployment shifts because they exploit note-specific artifacts (e.g., templates, separators, boilerplate) that do not reflect patient state. We propose CAST (Concept-guided Artifact Suppression Tuning), an SAE-based framework for auditable clinical text classification. CAST uses Sparse Autoencoders to expose sparse, human-auditable features from intermediate Transformer activations, labels SAE latents with an LLM-assisted interpretation pipeline and ICD-10 retrieval constraints, suppresses verified artifact latents via residual subtraction during fine-tuning, and provides post-hoc per-concept attributions for auditing model decisions. On MIMIC-IV discharge-note mortality prediction, CAST improves over its corresponding fine-tuned encoder baselines and remains competitive with strong LLM baselines, while producing a feature-level audit trail of the clinical concepts that support each prediction and the artifact concepts suppressed during training.

Figures & tables

Figure 1: SAE-Guided Fine-Tuning Enables Transparent Clinical Risk Prediction. Unlike standard fine-tuning, which yields opaque risk predictions, our framework encourages models to rely on clinically meaningful SAE concepts and suppress non-clinical artifacts, producing predictions with interpretable clinical evidence and ICD-10 codes.

Figure 2: Framework for clinical concept extraction and controlled fine-tuning. The pipeline consists of three stages: (1) Concept Extraction via a trained Sparse Autoencoder; (2) Concept Interpretation using an LLM to evaluate top activations; and (3) Fine-tuning with Concept Ablation to steer downstream model predictions.

Figure 3: Real clinical note case study: artifact suppression corrects a false positive while preserving clinical evidence.

SAE	Latent	$\boldsymbol{\bar{A}_{j}}$	Concept (ICD-10-CM)
Push prediction $\uparrow$ mortality
BatchTopK	1108	$+0.18$	palliative care (Z51.5)
TopK	6730	$+0.15$	COPD exacerbation (J44.1)
Matryoshka	101	$+0.04$	metastatic disease / secondary malignancy (C79.9)
TopK	915	$+0.02$	acute kidney failure (N17.9)
Push prediction $\downarrow$ mortality

Table 2: Top signed-attribution SAE concepts for ClinicalBERT layer

11

. Positive values push predictions toward mortality; negative values push them away.

Figure 4: Ablation study of concept-guided fine-tuning. Blocking artifact-related concepts improves both PR-AUC and F1, while blocking mortality-related concepts degrades performance.

Appendix figures & tables8 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Prompt template. System prompt and user prompt used for concept interpretation.

Backbone	SAE / Layer	$\boldsymbol{\kappa}_{\textbf{Mort.}}$	$\boldsymbol{\kappa}_{\textbf{Art.}}$	3/3 Mort.	3/3 Art.	Strict/any
ClinicalBERT	BatchTopK / 11	.749	.796	87.7%	93.0%	47.5%
	Matryoshka / 11	.748	.793	87.2%	92.8%	48.7%
	TopK / 11	.746	.806	87.5%	93.3%	50.2%
	BatchTopK / 8	.745	.804	87.0%	93.1%	51.2%
	Matryoshka / 8	.722	.790	85.2%	91.9%	47.6%
	TopK / 8	.733	.801	86.3%	92.9%	48.0%

Appendix

Table 3: LLM-labeling agreement across three independent runs. We report Fleiss’

\kappa

and the fraction receiving a unanimous (

3

-of-

3

) label for mortality (Mort.) and artifact (Art.) decisions. Strict/any is the fraction of latents flagged as artifacts by at least one run that enter the strict-consensus suppression set.

Backbone	SAE type / Layer	ICD-10 (%)	Mortality (%)
ClinicalBERT	TopK / 8	$55.3$	$78.3$
	TopK / 11	$57.6$	$79.2$
	BatchTopK / 8	$55.5$	$78.4$
	BatchTopK / 11	$57.5$	$79.9$
	Matryoshka / 8	$55.5$	$77.4$
	Matryoshka / 11	$57.7$	$78.5$

Appendix

Table 4: Per-configuration summary of interpreted SAE latents. We report the percentage grounded to ICD-10 codes and the percentage classified as mortality-related across backbone architectures, SAE types, and layer depths.

Backbone	Method	F1	AUROC	PR-AUC
ClinicalBERT	Fine-tuning	.2935	.8415	.2429
	TopK CAST	.3061	.8511	.2574
	Matryoshka CAST	.2306	.8538	.2342
Clinical- Longformer	Fine-tuning	.2746	.8464	.2378
	TopK CAST	.2432	.8570	.2485
	Matryoshka CAST	.3025	.8484	.2526

Appendix

Table 5: Layer-

4

results on MIMIC-IV, complementing the layer-

8

and layer-

11

evaluations in Table 1 .

$\boldsymbol{d_{\mathrm{SAE}}}$	$\boldsymbol{k}$	F1	AUROC	PR-AUC
4,096	64	.2703	.8511	.2531
8,192	32	.2569	.8525	.2530
8,192	64	.2961	.8579	.2460
8,192	128	.2801	.8502	.2344
16,384	64	.2881	.8538	.2504

Appendix

Table 6: Dictionary-size and TopK-sparsity sensitivity for ClinicalBERT layer

11

CAST.

Backbone	Layer	SAE	F1	F1 ${}_{\tau^{\star}}$	AUROC	PR-AUC	Brier	NLL	ECE
ClinicalBERT	11	TopK	$\mathbf{.0107}$	$.3389$	$\mathbf{.0002}$	$.1504$	$\mathbf{<.001}$	$\mathbf{<.001}$	$\mathbf{<.001}$
	11	Matryoshka	$.1769$	$.5458$	$.2269$	$.1030$	$\mathbf{<.001}$	$\mathbf{<.001}$	$\mathbf{<.001}$
ClinicalBERT	8	TopK	$.1499$	$.4766$	$\mathbf{.0108}$	$.1705$	$.9982$	$.9925$	$1.0000$
	8	Matryoshka	$.2950$	$.1011$	$\mathbf{.0330}$	$.4500$	$\mathbf{<.001}$	$\mathbf{<.001}$	$\mathbf{<.001}$
Clinical- Longformer	11	TopK	$\mathbf{<.001}$	$.2211$	$\mathbf{.0086}$	$.2335$	$\mathbf{<.001}$	$\mathbf{<.001}$	$\mathbf{<.001}$
	11	Matryoshka	$\mathbf{<.001}$	$.9978$	$.9983$	$.9783$	$\mathbf{<.001}$	$\mathbf{<.001}$	$\mathbf{<.001}$

Appendix

Table 7: One-sided paired-bootstrap

p

-values for TopK and Matryoshka CAST relative to the matched fine-tuning baseline (

B=10{,}000

). The alternative is an improvement by CAST. Bold entries denote

p<.05

SAE	L#	$\boldsymbol{\bar{A}_{j}}$	ICD-10	Concept (LLM description)	Representative activating context
Push prediction $\uparrow$ mortality
TopK	4415	$+0.21$	R19.1	Physical exam finding of a “soft” abdomen, often with descriptors of normal bowel sounds.	/ abd : soft , nt , nd , normal
BatchTopK	1108	$+0.18$	Z51.5	Palliative-care services, often in the context of serious illness or cancer treatment.	second / round of palliative chemo
TopK	6730	$+0.15$	G70.01, J44.1	Worsening of existing symptoms or emergence of new symptoms.	temp > 101.5, worsening pain , / drainage
BatchTopK	7197	$+0.12$	I27.23, J60–J70	Mentions of the lung, particularly lung cancer, lung masses, and other pulmonary pathology.	lung sounds very diminished l lung fie...
TopK	5454	$+0.13$	F41.1, F06.4	Anxiety, often in the context of psychiatric conditions such as depression.	20mg daily for depression / anxiety

Appendix

Table 10: Top signed-attribution clinical concepts

\bar{A}_{j}

on ClinicalBERT layer

11

. Concepts are aggregated across the three SAE variants and shown in descending order of

|\bar{A}_{j}|

within each direction. Concept descriptions are produced by the retrieval-grounded LLM judge (Section 4.4 ); contexts are sampled from the SAE training corpus.

Sub-category	L#	LLM concept description	Representative activating context
formatting	7488	Line breaks and formatting elements, especially the <cr> token in lists.	please take all pills ) / / followup instructions
formatting	4310	Structural transition from response to plan sections in clinical notes.	cr > response : / plan : / chronic o
formatting	1625	Parenthetical confirmation notations, e.g., (x) yes or ( ) no.	services / considered ? ( x ) yes - ( ) no
formatting	8123	List-item or code-snippet delimiters, especially the semicolon character.	care , asking app questions . a ; / loving p ; c
formatting	2201	Phrases indicating that specific details or instructions are provided elsewhere.	of systems is unchanged from admission except as n
de-id	2195	De-identified first- or last-name tokens of the form [FirstName5(NamePattern1) X] .	needed . [ * * first name5 ( namepattern1 ) 1875 *

Appendix

Table 11: Representative entries from the strict-consensus artifact latents on the ClinicalBERT layer

11

TopK SAE. The

3

-of-

3

LLM agreement rule (Section 4.4 ) classifies these latents as documentation-format or de-identification artifacts unrelated to patient state; their decoder contributions are subtracted from

\tilde{h}^{(K)}

during fine-tuning.

Language Model Robustness

Language model robustness studies how reliably language models preserve correct, safe behavior under input changes, distribution shifts, and adversarial pressure.

See all 493 papers

Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations
Mar 18, 2026Sanjay Basu, Sadiq Y. Patel, Parth Sheth +5Mechanistic Interpretability Language Model Steering
University of California San Francisco, San Francisco, CA, USA · Waymark, San Francisco, CA, USA · University of Pennsylvania, Philadelphia, PA, USA +2
L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification
Apr 14, 2026Rishik Kondadadi, John E. OrtegaLearning to Defer Text Classification
University of Minnesota, Northeastern University, Minneapolis, Boston, USA
Training Large Language Models to Predict Clinical Events
May 12, 2026Benjamin Turtel, Paul Wilczewski, Kris SkotheimHealthcare Language Model Calibration
Lightning Rod Labs

Mar 18, 2026J/K move · Enter open · S save

Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations

Sanjay Basu, Sadiq Y. Patel, Parth Sheth, Bhairavi Muralidharan+4

University of California San Francisco, San Francisco, CA, USA · Waymark, San Francisco, CA, USA · University of Pennsylvania, Philadelphia, PA, USA+2

Mechanistic Interpretability Language Model Steering

Open paper AskPDF arXiv

Language models encode task-relevant knowledge in internal representations that far exceeds their output performance, but whether mechanistic interpretability methods can bridge this knowledge-action gap has not been systematically tested. We compared four mechanistic interpretability methods -- concept bottleneck steering (Steerling-8B), sparse autoencoder feature steering, logit lens with activation patching, and linear probing with truthfulness separator vector steering (Qwen 2.5 7B Instruct) -- for correcting false-negative triage errors using 400 physician-adjudicated clinical vignettes (144 hazards, 256 benign). Linear probes discriminated hazardous from benign cases with 98.2% AUROC, yet the model's output sensitivity was only 45.1%, a 53-percentage-point knowledge-action gap. Concept bottleneck steering corrected 20% of missed hazards but disrupted 53% of correct detections, indistinguishable from random perturbation (p=0.84). SAE feature steering produced zero effect despite 3,695 significant features. TSV steering at high strength corrected 24% of missed hazards while disrupting 6% of correct detections, but left 76% of errors uncorrected. Current mechanistic interpretability methods cannot reliably translate internal knowledge into corrected outputs, with implications for AI safety frameworks that assume interpretability enables effective error correction.

Abstract

Language Model Robustness

Explore similar work

Interpretability without actionability: mechanistic methods cannot correct language model errors despite near-perfect internal representations

L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification

Training Large Language Models to Predict Clinical Events