From Latent Biomarkers to Clinical Rules: Embedding-Guided Rule Mining and Attribution-Based Translation for Interpretable Tabular Learning

Majid Lotfian Delouee; Hamed Ayoobi; Sjors G. J. G. In 't Veld; Martijn C. Schut

cs.LGAug 26, 2026

From Latent Biomarkers to Clinical Rules: Embedding-Guided Rule Mining and Attribution-Based Translation for Interpretable Tabular Learning

Authors: Majid Lotfian Delouee, Hamed Ayoobi, Sjors G. J. G. In 't Veld, Martijn C. Schut

Abstract

Clinical decision support tools are most useful when accurate predictions are accompanied by understandable explanations. Rule-based models provide transparency, but rules derived directly from raw clinical measurements may miss patterns arising from interactions between multiple variables. We present a four-step pipeline that mines decision rules in the latent space of an FT-Transformer and translates them back into measurable clinical features. Embedding dimensions that consistently separate patient groups are treated as latent biomarkers, rules are mined using small decision trees, and selected rules are translated using gradient-input saliency and CLS attention attribution. We evaluate the framework on six public clinical and population health datasets at four embedding dimensions. Translated rules outperformed raw-feature rules in five of six datasets, with mean AUROC gains ranging from 0.04 to 0.23. On the heart disease dataset, embedding-space rules reached 0.98 AUROC, but translation reduced this to 0.72, showing that high-performing latent rules cannot always be represented by simple raw-feature conditions. These results show that latent-space rule discovery can uncover predictive patterns while translating them into clinically measurable features that can be evaluated by clinicians.

Figures & tables

Figure 1: Conceptual overview of the proposed pipeline. Clinical patient records are encoded into a compact latent space by a trained FT-Transformer (forward path, blue). Interpretable rules are mined in the latent space and the embedding dimensions involved are identified as latent biomarkers. Gradient and attention attribution then maps each rule back to raw clinical features, producing a translated rule in clinical language (attribution path, red).

Dataset	Domain	Samples	Features	Classes	Maj. %	Notes
Pima Indians Diabetes	Endocrinology	768	8	2	65%	Female Pima Indian patients aged $\geq$ 21 years; binary diabetic outcome [ 23 ]
Cleveland Heart Disease	Cardiology	303	13	2	54%	UCI repository; presence/absence of coronary artery disease [ 10 ]
Adrenoleukodystrophy (ALD)	Neurometabolic disease/ Lipidomics	184	1811	5	58.2%	Lipidomics dataset for five-stage ALD severity [ 11 ]
Adult Census Income	Population Health	48,842	14	2	76%	Annual income ${\leq}$ 50K / ${>}$ 50K; used as a proxy for socioeconomic health determinants [ 2 ]
CDC Diabetes Health Indicators	Endocrinology	253,680	21	2	86%	BRFSS 2015 telephone survey; binary diabetes outcome [ 3 ]
Diabetes 130-US Hospitals	Endocrinology	101,766	47	2	89%	Hospital admissions 1999–2008; outcome is 30-day readmission [ 24 ]

Table 1: Summary of the six clinical and population health datasets used in this study. Sample sizes reflect the full dataset before splitting. Class balance is reported as the percentage of the majority class. Feature counts exclude the target column.

Dataset	Raw-Feature Rule	Embedding Rule	Translated Rule	$\Delta$ AUROC
Pima Indians Diabetes	0.623	0.610	0.779	+0.156
Cleveland Heart Disease	0.675	0.985	0.717	+0.042
Adrenoleukodystrophy (ALD)	0.986 †	0.863	0.862	$-$ 0.124
Adult Census Income	0.618	0.589	0.734	+0.116
CDC Diabetes Health Ind.	0.532	0.549	0.758	+0.226
Diabetes 130-US Hospitals	0.537	0.526	0.753	+0.216

Table 2: Mean test-set AUROC for the three rule types across all six datasets, averaged over selected rules and embedding dimensions (16, 32, 64, 128). The best value per dataset is in bold . Delta shows the AUROC gain of the translated rule over the raw-feature rule. A positive delta indicates that routing through the latent space and translating back to raw features improved performance.

Figure 2: Mean test-set AUROC per dataset for raw-feature rules (gray), embedding-space rules (blue), and translated rules (green), averaged over all selected rules and embedding dimensions within each dataset. Translated rules outperform raw-feature rules in five of six datasets and achieve the highest AUROC among the three rule types in four of six datasets. Two exceptions are annotated: Heart Disease, where near-perfect embedding separation (mean Emb AUROC $=0.99$ ) creates a translation ceiling (8/8 rules, Emb $\gg$ Trans); and ALD, where direct lipid biomarkers yield raw-rule AUROC near 1.0, a baseline translated rules cannot exceed (5/10 rules).

Figure 3: Distribution of test-set AUROC across all 80 individual rules, pooled over six datasets and four embedding dimensions. Each box shows the interquartile range; the horizontal line is the median; whiskers extend to 1.5 times the IQR; individual outliers are plotted. The translated-rule distribution is shifted substantially upward relative to both alternatives: its lower quartile exceeds the upper quartiles of the raw-feature and embedding-rule distributions. The mean improvement over raw-feature rules ( $+0.14$ , bracket) is consistent across the full distribution rather than being driven by outliers.

Figure 4: Test-set AUROC of each translated rule plotted against the AUROC of its corresponding embedding-space rule, for all 80 rules across six datasets (colored by dataset). Points above the diagonal indicate rules where translation improved over the embedding-space rule; points below indicate a translation loss. The heart disease cluster (red crosses), annotated in the upper region, is the only group that falls consistently below the diagonal, confirming that the translation bottleneck is specific to near-perfect latent separation. For all other datasets, translated rules match or substantially exceed the embedding-space AUROC, with gains up to 0.40 AUROC points on individual rules.

Figure 5: Translated rule AUROC plotted against raw-feature rule AUROC for all 80 rules across six datasets. Points above the diagonal (63 of 78 rules with valid raw-feature baselines, 81%) confirm that routing through the latent space and attributing back to raw features improves on direct rule mining for the majority of rules. The two hollow ALD diamonds at the left edge have no raw-feature baseline (classes 2 and 4); their translated AUROCs of 0.82 and 0.89 represent rules found only in the embedding space. Heart disease rules lie above the diagonal despite the translation ceiling seen in Figure 4 : translated rules still outperform raw-feature rules even when they fall well short of the embedding-space rule.

Figure 6: Cohen’s $d$ effect sizes for the top eight clinical features in the Pima Indians Diabetes dataset (left, blue) and the CDC Diabetes Health Indicators dataset (right, teal), measured between rule-positive (low-risk) and rule-negative (high-risk) patients on the test set. All values are negative, indicating that rule-positive patients have consistently lower feature values. Dashed red vertical lines mark $d=0$ . Effect sizes exceeding $|d|=0.8$ (large by Cohen’s convention) are seen for glucose and BMI in Pima Diabetes, and for general health status and high blood pressure in CDC Diabetes, confirming that the attributed features capture genuine clinical differences rather than statistical artifacts.

Figure 7: Approximate distributions of plasma glucose (left), body mass index (centre), and age (right) for rule-positive (blue, non-diabetic prediction) and rule-negative (red, diabetic prediction) patients in the Pima Indians Diabetes dataset (class-0 rule, embedding dimension 128). Curves are Gaussian approximations using group means from the attribution table and pooled standard deviations estimated from Cohen’s $d$ . Dashed vertical lines mark the respective group means. Solid vertical lines in the glucose and BMI panels mark the thresholds of the representative translated rule in Table 3 , which is stated there for the complementary diabetic class; they are shown to locate those thresholds against the two group distributions. The substantial separation between the two groups in glucose and BMI ( $|d|>1.4$ ) confirms that these features genuinely differentiate the patient groups and that the translated rule thresholds are placed at clinically meaningful decision boundaries.

Figure 8: Heatmap of mean combined attribution scores (gradient-input saliency averaged with CLS attention weights) for the eight clinical features of the Pima Indians Diabetes dataset, across four embedding dimensions. Scores are averaged over all selected rules at each dimension. Darker blue indicates higher attribution (stronger influence on the latent rule dimensions). The feature ranking is stable across all four embedding configurations: plasma glucose, BMI, and age consistently account for more than 55% of the total attribution, confirming that the translation step identifies the same core clinical predictors regardless of the embedding size chosen.

Dataset	Embedding Rule	Translated Rule	Raw AUC	Emb. AUC	Trans. AUC
Pima Diabetes	IF $z_{5}>0.44$ AND $z_{11}\leq-0.21$ THEN: diabetic	IF Plasma Glucose $>127.5$ mg/dL AND BMI $>29.8$ kg/m 2 THEN: diabetic	0.60	0.66	0.83
Cleveland Heart	IF $z_{12}\leq 0.15$ AND $z_{1}>0.33$ THEN: disease present	IF Max. Heart Rate $>149$ bpm AND ST Depression $\leq 1.4$ THEN: disease present	0.65	0.99	0.73
CDC Diabetes	IF $z_{3}>0.29$ AND $z_{7}\leq-0.18$ THEN: diabetic	IF BMI $>30.0$ kg/m 2 AND High Blood Pressure $=1$ THEN: diabetic	0.54	0.55	0.76

Table 3: Representative rule translations from three datasets, shown at embedding dimension 32. For each dataset the embedding rule (conditions on latent dimensions

z_{j}

) and the corresponding translated rule (conditions on raw clinical features) are listed alongside test-set AUROC values. Top-attributed features are the five raw features ranked highest by the combined gradient-attention score; only the two features used in the final depth-2 tree are shown in the translated rule.

Clinical Decision Support

Clinical Decision Support uses data, medical knowledge, and AI to help clinicians assess patients and choose tests, diagnoses, and treatments.

See all 244 papers

Medical Heuristic Learning: An LLM-Driven Framework for Interpretable and Auditable Clinical Decision Rules
Jun 15, 2026Wei Xu, Ke Yang, Gang Luo +4Clinical Decision Support Interpretable ML
The Centre for Artificial Intelligence Driven Drug Discovery, Macao Polytechnic University, Macao SAR · The Key Laboratory of Short-Range Radio Equipment Testing and Evaluation, Ministry of Industry and Information Technology, and the Terahertz Science Application Center (TSAC), Beijing Institute of Technology, Zhuhai, China · The Faculty of Education, The University of Hong Kong, Hong Kong SAR +2
MedFeat: Model-Aware and Explainability-Driven Feature Engineering with LLMs for Tabular Prediction
Feb 10, 2026Zizheng Zhang, Yiming Li, Justin Xu +6Interpretable ML Clinical Prediction
Microsoft Research · University of Oxford
Evaluating LLM-Generated Rules for Heart Disease Prediction
Aug 11, 2026Feisal Alaswad, Batoul Aljaddouh, Maher Alrahhal +2Rule-Based Classification Clinical Risk Prediction
Department of Computing Technologies, SRM Institute of Science and Technology, Kattankulathur, Chennai, India · Department of Computer Science, Amity University Dubai, Dubai, UAE · College of Computing and Informatics, University of Sharjah,Sharjah, UAE

Jun 15, 2026J/K move · Enter open · S save

Medical Heuristic Learning: An LLM-Driven Framework for Interpretable and Auditable Clinical Decision Rules

Wei Xu, Ke Yang, Gang Luo, Keli Zheng+3

The Centre for Artificial Intelligence Driven Drug Discovery, Macao Polytechnic University, Macao SAR · The Key Laboratory of Short-Range Radio Equipment Testing and Evaluation, Ministry of Industry and Information Technology, and the Terahertz Science Application Center (TSAC), Beijing Institute of Technology, Zhuhai, China · The Faculty of Education, The University of Hong Kong, Hong Kong SAR+2

Clinical Decision Support Interpretable ML Clinical Prediction

Open paper AskPDF arXiv

Predictive modeling for clinical decision support requires both strong predictive performance and transparent, auditable, and human-reviewable decision logic. Although deep learning and tree-based ensemble methods can achieve high accuracy, their black-box nature remains a major obstacle to trustworthy clinical deployment. Moreover, clinical prediction often operates under practical constraints, including limited sample sizes, severe class imbalance, and feature evolution arising from changes in diagnostic criteria or clinical documentation practices. We propose Medical Heuristic Learning (MHL), a constrained paradigm for LLM-assisted rule learning. Rather than relying on updates to implicit model weights, MHL integrates statistical probes, medical knowledge probes, initial rule synthesis, and iterative rule optimization to construct an executable rule-based expert system. The resulting rule system is expressed entirely using the native logical and control-flow constructs of a programming language. Valid rule versions are recorded and retained along the search trajectory, making the decision logic explicit, interpretable, and auditable. MHL also supports continual learning by using previously validated rules as a starting point and iteratively revising them in response to updated feature information under data drift or feature evolution. MHL is not tied to any specific programming language. Comprehensive experiments on medical datasets show that MHL achieves predictive performance comparable to that of state-of-the-art methods, performs favorably in small-sample and highly imbalanced settings, and supports the transfer and adaptive revision of validated rules under feature evolution. Overall, these findings suggest that non-gradient-based heuristic systems offer an approach to balancing predictive performance and transparency in clinical decision support.

Abstract

Clinical Decision Support

Explore similar work

Medical Heuristic Learning: An LLM-Driven Framework for Interpretable and Auditable Clinical Decision Rules

MedFeat: Model-Aware and Explainability-Driven Feature Engineering with LLMs for Tabular Prediction

Evaluating LLM-Generated Rules for Heart Disease Prediction