Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data
Authors: Robbie Holland, Ashwin Kumar, Mateus A. Esmeraldo, David Le, Andrea Fisher, Jiazhen Pan, Louisa Fay, Magdalini Paschali, Vasiliki Sideri-Lampretsa, Ates Fettahoglu, Stefania Moroianu, Philipp Wesp, Ivan Lopez, Tom Nuno Wolf, McKenzie S. White, Eduardo Pontes Reis, Fatima Rodriguez, Curtis P. Langlotz, Vasilis Syrgkanis, Imon Banerjee, Akshay S. Chaudhari, Sergios Gatidis
Organizations: Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA · Department of Radiology, Mayo Clinic, Phoenix, Arizona, USA · Technical University of Munich, Munich, Germany · Division of Cardiology, Stanford University School of Medicine, Stanford, CA, USA · Stanford University, Stanford, CA, USA · Department of Biomedical Data Science, Stanford University, Stanford, CA, USA · Stanford Cardiovascular Institute, Stanford, CA, USA · Weill Cancer Hub West, Stanford, CA, USA
Automated scientific discovery has the potential to advance understanding of disease mechanisms and accelerate identification of early disease markers. Manual data annotation is infeasible at population scale, motivating methods that automatically extract pre-defined phenotypes in multimodal and unstructured clinical data. Such pre-defined feature sets determine and limit which associations these approaches can discover. This limitation extends to agentic AI systems that use these tools to query population-scale data. In contrast, foundation models (FMs) trained on large volumes of clinical data can perform tasks beyond their training objectives. This suggests their latent embeddings may represent novel or unexplored features that could yield insights into disease mechanisms, yet such latent features remain difficult to interpret. To address this challenge, we introduce Glyph, an automated multistage framework that extracts features from multimodal and unstructured clinical data using FMs and sparse autoencoders, interprets them as coded clinical concepts, and evaluates their value for clinical scientific discovery. We applied Glyph to abdominal CT volumes, CT radiology reports and laboratory measurements from 41,628 patients across different institutions. We identified 397 predictive concepts supported by 4,463 Bonferroni-significant associations. Moreover, a majority of discovered associations were automatically reproducible in held-out cohorts across different sites. Glyph concepts provided complementary associations to literature-derived markers, leading to gains in predictive power. We envision that interpretability can extend beyond a mechanistic understanding of neural networks to connecting implicitly learned features with clinical knowledge and establishing their significance for patient outcomes. We release Glyph at https://github.com/StanfordMIMI/Glyph.
Figures & tables
Figure 1: Transforming foundation model embeddings into engines of clinical discovery. a Glyph can be applied to any data expected to contain useful markers for discovery. b Foundation models compress multimodal clinical data into high-dimensional embeddings, typically used for predefined prediction tasks. The structural contents of embeddings driving these predictions is typically not unexplored. c Glyph instead transforms FM embeddings into engines for discovery, mapping a space of concepts hidden within these embeddings. Glyph goes beyond embeddings by decomposing them into sparse concepts that can be studied individually and in selected combinations (Figure 1 c). d Glyph concepts recover associations established in clinical research and ICD-based PheWAS analyses, while revealing complementary associations not yet discovered by either approach. e–f Glyph evaluates concepts for e associations with future disease after correction for multiple testing, f associations that persist after adjustment for measured confounders, and g clinical interpretability. h All findings are tested for reproducibility across multiple cohorts. i Validated concepts provide an interface for asking complex questions of FM embeddings and grounding the answers in the underlying data.
Figure 2: The Glyph framework. An interactive version is available at https://stanfordmimi.github.io/Glyph-research/ . a At the Discovery Site, Glyph trains sparse autoencoders (SAEs) to decompose foundation model (FM) embeddings into sparse concepts. b Each SAE encoder is then frozen as a concept extractor and its decoder discarded. Concepts are re-extracted zero-shot in held-out internal and external cohorts to assess reproducibility of their interpretations, prognostic associations, and confounder analyses. c The discovered concept space maps the clinical characteristics and disease burden of patients undergoing abdominal CT at the Discovery Site, alongside routine variables such as age and smoking status. We show 397 concepts (9.26% of the 4,285 candidate features) that were associated with at least one of the 141 prognostic outcomes after Bonferroni correction. Edges show correlations between concepts in the top 2.5%. Clinically related concepts were found to cluster into subregions (dashed boxes) corresponding to liver, renal, and cardiovascular disease, and systemic inflammatory and cholestatic illness. Node size scales with the logarithm of the number of patients who share the specific concept. A confounder-adjusted version of this network is shown in Figure 11 .
Figure 3: Glyph concept-wide association study. a Each Glyph concept is tested for association to each of 141 future disease outcomes within 1 to 4 years, yielding 604,185 concept-outcome pairs evaluated using per-standard-deviation odds ratios. b A Discovery filter ( p≤0.05 adjusted for multiple testing via Bonferroni correction, equivalent to p≤8.28×10−8 , and OR >1.0 ) applied on the Discovery Site yields 4,463 (0.739%) promising associations. c Number of promising associations per modality on the Discovery Site, totalling 4,463; these are fixed and tested for reproducibility without calibration on the Internal Test, External Site #1, and External Site #2 cohorts using a 5% FDR threshold (Benjamini–Hochberg), with the percentage replicating shown per modality and cohort. d Volcano plot of all concept-outcome associations and the Discovery filter selection. e Risk profiles and reproducibility statistics for one organ-specific and one systemic-risk concept. f–h Sample counts and observed probabilities of future disease onset, with confidence intervals, across concept-activation bins.
Figure 4: Confounder-adjusted association study. a Glyph concepts act as variables that can be compared with one another and with established clinical variables in a unified feature space, enabling confounder analysis of FM predictions. b For each of the 4,463 promising associations, the ten most strongly associated concepts across other modalities are automatically selected as candidate confounders, alongside six universal clinical covariates. c Number of associations per modality retaining a significant, adjusted OR >1.0 after confounder adjustment per site. d–g Association profiles show that after adjustment I21 (ascites) loses most associations, L127 (cytopenia) remains broadly associated, R85 (cirrhotic liver morphology and varices) retains hepatic associations, and I26 (aortic atherosclerosis) retains cardiovascular and neurocognitive associations. h An example of a knowledge-guided confounder analysis for primary liver cancer.
Figure 5: Glyph identifies associations beyond the clinical literature and adds prognostic value. a Screening 342,647 PubMed papers across 21 outcomes yielded 294 representative studies and 412 literature-derived predictors. b Adding ten Glyph concepts per outcome improved held-out AUC beyond literature markers, with the largest gains in metabolic, haematologic, and infectious outcomes. c On average, Glyph concepts outperform literature markers, even when top performing markers are selected on the Discovery Set. d Glyph concepts provide discoveries that complement the history of clinical research. The additional prognostic benefit varied across axes, as illustrated by the improvement for HFpEF (top) and the larger gain for sepsis (bottom).
Figure 6: The MCP enables more complex analyses that build on a foundational layer of Glyph concepts. In this example, concepts were combined into nodes that competed against one as independent predictors for inclusion in a disease model of septicemia. I76 (an undocumented imaging concept linked to septicemia) provided a predictive contribution independent of hepatic disease markers, fluid accumulation, BMI, inflammatory markers, malignancy, and evidence of clinical intervention, and contributed to prognostic AUC predictive across three cohorts.
Discovery (Train + Val)
Internal Test
External Site #1
External Site #2
Dataset Characteristics
n
%
n
%
n
%
n
%
Accessions (Abdominal CTs)
20,391
–
5,137
–
25,803
–
14,199
–
Unique Patients
14,652
–
3,667
–
17,224
–
6,085
–
Age (mean)
54.1 ± 18.9
–
54.5 ± 19.1
–
60.6 ± 18.9
–
62.1 ± 16.9
–
Sex
Female
11,407
55.9
2,927
57.0
14,039
54.4
6,994
49.3
Table 1: Summary of dataset characteristics across the Discovery, Internal Test, External Site #1, and External Site #2 (Mayo Clinic) cohorts, including the number of unique patients and abdominal CTs, age, sex distribution, smoking status, heavy alcohol use, and BMI. Values are reported as counts (n) and percentages where applicable.
Cohort
Correction
Threshold
Discovery (Train)
Bonferroni
8.28×10−8
Internal Test
Benjamini–Hochberg (5% FDR)
3.46×10−2
External Site #1
Benjamini–Hochberg (5% FDR)
3.72×10−2
External Site #2
Benjamini–Hochberg (5% FDR)
2.44×10−2
Table 2: Significance thresholds for the association filters. The Discovery Site uses a Bonferroni-corrected threshold across all tested concept-outcome pairs; each held-out cohort uses its own Benjamini–Hochberg threshold (5% FDR) computed over the promising associations re-evaluated on that cohort.
Cohort
Correction
Threshold
Discovery (Train)
Benjamini–Hochberg (5% FDR)
2.24×10−2
Internal Test
Benjamini–Hochberg (5% FDR)
1.06×10−2
External Site #1
Benjamini–Hochberg (5% FDR)
9.50×10−3
External Site #2
Benjamini–Hochberg (5% FDR)
1.46×10−2
Table 3: Significance thresholds for the confounder filter. Unlike the association filters (Table 2 ), every cohort, including Discovery, uses a Benjamini–Hochberg threshold (5% FDR) computed over the confounder-adjusted p -values of the promising associations tested on that cohort.
Listing 1: Learning new concepts with Glyph. The example uses DINOv3 for images and Qwen3-Embedding-0.6B for report text. The embedding_map connects each metadata input column to its encoder. fit selects a sparse SAE configuration using validation reconstruction performance, warning if R2<0.85 . hypothesis_search tests sparse concepts against the specified outcomes, and visualize displays the interactive concept space and association results.
Listing 2: Extracting published concepts on new data. Single-concept extraction. Specify a concept code to measure one published concept across a cohort or for a selected study, illustrated with imaging concept I87. Full-space extraction. Omit the concept code to apply the paper’s entire pretrained dictionary for a modality to the supplied cohort, illustrated with the laboratory dictionary. The resulting study-by-concept table can be visualized, and individual concepts such as L127 can be selected directly. The metadata supplies study identifiers and input paths; Glyph computes the matching foundation-model embeddings and concept activations on the user’s data while preserving the paper’s concept indices. Neither outcome labels nor SAE retraining is required for concept extraction and exploration; labels are needed for association testing and significance highlighting.
Listing 3: Running Glyph from a coding agent. Bundled skills guide Claude Code or Codex through configuration, analysis, literature comparisons, visualization, and reporting. The agent installs the package, configures the inputs and outcomes, runs the requested analyses, and summarizes the results. The second prompt applies the paper’s pretrained dictionaries and corresponding foundation models to a new cohort without retraining the SAE.
Figure 7: a, Outcomes were selected by ranking Phecode prediction AUCs on the Discovery Site validation set. After excluding non-disease codes, 141 of the top 150 outcomes were retained. Without refitting, imaging probes transferred with minimal loss across cohorts, whereas report performance declined at External Site #2. b, SAEs fitted at the Discovery Site were applied unchanged across cohorts. Reconstruction remained high for imaging and labs but declined steadily for reports. c, Replacing dense embeddings with sparse concepts preserved most predictive information, with less loss for imaging than reports. d, Median odds ratios of significant associations followed a similar per-site trend to the predictive performance, maintaining high OR for imaging- and labs-derived concepts and steadily dropping for radiology report-derived concepts.
Figure 8: The Glyph MCP brings a new capability; the ability to immediately perform, user-specified requests about high-dimensional unstructured data, such as imaging. Here, Claude Code returns the most highly associated concepts for future risk of myocardial infarction, the majority of which are labs concepts. Responses are grounded by the Glyph layer.
Figure 9: Glyph generates individualised risk explanations for a given outcome. For a query patient (AC42403ee, 74.9% predicted CKD risk, 93.6th percentile), the system surfaces the concepts most responsible for that prediction alongside their attribution scores, activation levels, and population rarity. Cross-referencing against the top population-level risk features reveals which epidemiologically validated signals are active in this patient, grounding the individual explanation in cohort-level evidence.
Figure 10: The Glyph concept layer is flexible to other forms of discovery beyond prognosis. Here, Glyph performs a patients-like-me scenario where the most similar patient to a query is found. Their similarity is then explained in terms of shared concepts, and the rarity of such a co-occurrence at their respective levels of activation.
Figure 11: Confounder-adjusted inter-concept associations. a The full adjusted network. Edges denote directed dependencies from the fitted sparse SEM, retained where the standardized coefficient exceeds ∣w∣≥0.1 ; edge width scales with ∣w∣ and colour denotes sign (red positive, blue negative). Dashed edges involve at least one undocumented concept. b Concepts within two degrees of separation from liver cirrhosis (R85), a disease hub known to link to splenomegaly (I65) and portal hypertension (R309). c Patient age shows adjusted associations with many concepts, particularly cardiovascular disease and spinal degeneration (R1). d In this abdominal CT cohort, labs indicating cholestasis (L108) form the major bridge between modalities, connecting abnormal lab states, such as elevated alkaline phosphatase (L33) and cytopenia (L127), to structural imaging phenotypes such as splenomegaly (I65) and intra-abdominal fluid (I105).
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 12: Glyph concepts are automatically extracted, interpreted, and reproduced zero-shot at external sites. Each concept is interpreted on the Discovery Site by comparing top- versus non-activating examples, then re-extracted at an Internal Test cohort and two external sites where the same interpretation is re-tested on unseen patients; consistent discrimination shows both concepts and interpretations transfer across institutions. a–c A radiology report concept, labeled metastatic disease , re-extracted zero-shot and confirmed at every site. AutoInterp accuracy is highest for reports: as human-written summaries, their concepts are near-fully interpretable but bounded in discovery potential. d–e An imaging concept whose interpretation is not consistently documented in radiology reports, yet was found to rank among the top 10% of imaging predictors of infection, sepsis, wasting, and organ failure. f Remarkably, 52.5% of imaging concepts were interpretable via corresponding radiology reports. g–i A laboratory concept, labelled monocytosis or neutrophilia , confirmed across sites.
Figure 13: Glyph concepts build on pre-coded discovery. Phecodes group ICD codes into manually curated, clinically meaningful concepts for use in research [ 2 ] . We test complementary benefits of data-driven Glyph concepts over Phecodes. a–b Binary Phecodes drawn from the full recorded history yield 3.4 × more significant associations than Glyph concepts, but reproduce less often at Internal Test (49% versus 69%) and have a smaller median effect (OR 1.33 versus 1.45). Recent Phecodes, restricted to the scan-adjacent window, also yield more associations than Glyph concepts but reproduce less often. c Directly corresponding Phecodes were often less specific than imaging- or laboratory-derived Glyph concepts and yielded weaker or nonsignificant associations. In other cases, the broader clinical history captured by Phecodes provided a stronger proxy for future risk. For example, hypertension served as a proxy for atherosclerosis (bottom) and outperformed the corresponding Glyph concept for some outcomes.
Figure 14: Architectural ablation of the sparse autoencoder across 141 disease prediction tasks. Results are organized by modality, with panels a–c corresponding to CT imaging (Merlin), radiology reports (Qwen3-Embedding-8B), and laboratory values (Labs FM), respectively. Within each modality, results are organized by metric, showing sparse autoencoder reconstruction loss (MSE), R2 between original and reconstructed foundation-model embeddings, and mean downstream AUC across the 141 disease prediction tasks. Reconstruction loss and R2 are evaluated on the discovery training set and unseen external data, while downstream AUC assesses whether the Glyph concept space preserves the predictive performance of the underlying foundation models with minimal degradation. Across modalities, diminishing returns in reconstruction are observed beyond K=40 , with R2 improving and subsequently plateauing at this sparsity level.
Outcome
Adjusted / associated (%)
Descriptive profile and representative retained concepts (share; adjusted OR)
Acute/subacute liver necrosis
20 / 60 (33.3%)
Hepatic abnormalities coexist with cytopenia, hypoalbuminaemia and splenic findings. Cirrhotic morphology/varices (R85; 5.0%; OR 1.92); Cholestasis (L108; 5.0%; OR 2.37); Unknown hepatobiliary pattern (I87; 5.0%; OR 2.42); Cytopenia (L127; 5.0%; OR 2.26); Hypoalbuminaemia (L91; 5.0%; OR 2.04); Splenomegaly (I65; 5.0%; OR 2.56).
Liver transplant status
19 / 36 (52.8%)
Broad hepatic, renal and haematological profile, alongside splenic and portal-vascular findings. Cholestasis (L108; 5.3%; OR 3.06); Unknown hepatobiliary pattern (I87; 5.3%; OR 4.31); Cirrhotic morphology/varices (R85; 5.3%; OR 2.11); Azotemia (L84; 5.3%; OR 2.40); Cytopenia (L127; 5.3%; OR 3.09); Splenomegaly (I65; 5.3%; OR 2.99).
Portal hypertension
16 / 59 (27.1%)
Predominantly cirrhotic, splenic, variceal and ascitic findings, with cholestasis. Cirrhotic morphology/varices (R85; 6.2%; OR 2.46); Varices (R414; 6.2%; OR 1.48); Chronic liver disease/congestion (I331; 6.2%; OR 1.50); Splenomegaly (I65; 6.2%; OR 2.79); Ascites/portal-hypertensive or peritoneal findings (I31; 6.2%; OR 1.78); Cholestasis (L108; 6.2%; OR 3.62).
Disorders of bilirubin excretion
13 / 44 (29.5%)
Cholestatic and biliary imaging findings predominate among the illustrated markers. Cholestasis (L108; 7.7%; OR 2.38); Biliary dilatation (I454; 7.7%; OR 1.49); Unknown hepatobiliary pattern (I87; 7.7%; OR 2.44); Raised alkaline phosphatase/globulin (L33; 7.7%; OR 1.77); Biliary obstruction/intervention (I1416; 7.7%; OR 1.33); Pneumobilia (R473; 7.7%; OR 1.48).
Liver abscess/chronic liver disease sequelae
13 / 45 (28.9%)
Structural hepatic and portal-hypertensive findings coexist with abscess/inflammatory concepts. Cirrhotic morphology/varices (R85; 7.7%; OR 2.98); Chronic liver disease/congestion (I331; 7.7%; OR 1.46); Chronic liver morphological changes (R1765; 7.7%; OR 1.61); Varices (R414; 7.7%; OR 1.57); Abscess/fluid collection/fistula (I7223; 7.7%; OR 1.47); Unknown hepatobiliary pattern (I87; 7.7%; OR 2.87).
Chronic passive hepatic congestion
10 / 23 (43.5%)
Cardiac/vascular findings coexist with a hepatobiliary pattern, consistent with a congestive profile. Unknown hepatobiliary pattern (I87; 10.0%; OR 2.73); Ventricular assist device (R283; 10.0%; OR 1.16); Cardiac enlargement/vascular calcification (R487; 10.0%; OR 1.32); Atrial enlargement (R3570; 10.0%; OR 1.09); Aortic atherosclerosis (I26; 10.0%; OR 1.91).
Appendix
Table 4: Adjusted concept profiles across hepatic and biliary outcomes at Discovery.
Outcome
Adjusted / associated (%)
Descriptive profile and representative retained concepts (share; adjusted OR)
Chronic renal failure
21 / 67 (31.3%)
Broad systemic profile alongside renal dysfunction. Cytopenia (L127; 4.8%; OR 1.64); Aortic calcification (I109; 4.8%; OR 1.78); Splenomegaly (I65; 4.8%; OR 1.39); Ascites (I21; 4.8%; OR 1.54); Azotemia (L84; 4.8%; OR 1.75); Impaired renal function (L69; 4.8%; OR 1.51).
Acute renal failure
20 / 83 (24.1%)
Broad systemic profile spanning hepatic, haematological, fluid and vascular abnormalities. Cholestasis (L108; 5.0%; OR 1.67); Cytopenia (L127; 5.0%; OR 1.41); Splenomegaly (I65; 5.0%; OR 1.46); Unknown hepatobiliary pattern (I87; 5.0%; OR 1.39); Ascites (I21; 5.0%; OR 1.37); Atherosclerosis with colonic abnormalities (I106; 5.0%; OR 1.28).
CKD (composite)
18 / 74 (24.3%)
Broad systemic profile spanning hepatic, haematological, fluid and vascular abnormalities. Cholestasis (L108; 5.6%; OR 1.55); Cytopenia (L127; 5.6%; OR 1.35); Unknown hepatobiliary pattern (I87; 5.6%; OR 1.48); Splenomegaly (I65; 5.6%; OR 1.34); Ascites, anasarca and effusions (I25; 5.6%; OR 1.37); Aortic calcification (I109; 5.6%; OR 1.22).
CKD, stage III
15 / 70 (21.4%)
Mixed renal-function, vascular and haematological profile, with a cholestatic signal. Impaired renal function (L69; 6.7%; OR 1.45); Reduced eGFR (L974; 6.7%; OR 1.21); Polycystic renal disease/polyps (R302; 6.7%; OR 1.18); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.43); Thrombocytosis/iron-deficiency pattern (L21; 6.7%; OR 1.38); Cholestasis (L108; 6.7%; OR 1.51).
Hypertensive CKD
15 / 47 (31.9%)
Mixed renal, vascular, hepatic and haematological profile. Azotemia (L84; 6.7%; OR 1.59); Renal failure with hypernatraemia/acidosis (L62; 6.7%; OR 1.47); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.44); Coronary/valvular calcification (R21; 6.7%; OR 1.34); Cholestasis (L108; 6.7%; OR 1.60); Cytopenia (L127; 6.7%; OR 1.50).
Anaemia in CKD
14 / 33 (42.4%)
Anaemia and renal-function markers coexist with vascular findings. Iron-deficiency anaemia (L104; 7.1%; OR 2.81); Low haemoglobin (L61; 7.1%; OR 2.01); Impaired renal function (L69; 7.1%; OR 1.57); Reduced eGFR (L974; 7.1%; OR 1.46); Aortic calcification (I109; 7.1%; OR 2.06); Atherosclerosis with colonic abnormalities (I106; 7.1%; OR 1.87).
Appendix
Table 5: Adjusted concept profiles across renal and urinary outcomes at Discovery.
Outcome
Adjusted / associated (%)
Descriptive profile and representative retained concepts (share; adjusted OR)
Congestive heart failure
16 / 48 (33.3%)
Broad cardiorenal, vascular, hepatic and laboratory profile, with cardiac-device concepts also retained. Aortic atherosclerosis (I26; 6.2%; OR 1.67); Cardiorenal dysfunction (L2358; 6.2%; OR 1.26); Azotemia (L84; 6.2%; OR 1.69); Cholestasis (L108; 6.2%; OR 1.65); Cardiac dysfunction with hypoalbuminaemia (L1488; 6.2%; OR 1.29); Hypoalbuminaemia (L91; 6.2%; OR 1.72).
Ischaemic heart disease (composite)
15 / 49 (30.6%)
Vascular calcification coexists with inflammatory, metabolic and hepatobiliary findings. Coronary/valvular calcification (R21; 6.7%; OR 1.19); Coronary calcification/atherosclerosis (R420; 6.7%; OR 1.20); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.35); Raised inflammatory blood-cell counts (L68; 6.7%; OR 1.21); Dyslipidaemia with inflammation/hyperglycaemia (L496; 6.7%; OR 1.16); Unknown hepatobiliary pattern (I87; 6.7%; OR 1.29).
Myocardial infarction
15 / 55 (27.3%)
Broad vascular, cardiorenal, hepatic, inflammatory and anaemia profile. Aortic atherosclerosis (I26; 6.7%; OR 1.29); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.37); Cardiorenal dysfunction (L2358; 6.7%; OR 1.17); Raised inflammatory blood-cell counts (L68; 6.7%; OR 1.29); Anaemia with hypoalbuminaemia (L94; 6.7%; OR 1.28); Cirrhotic morphology/varices (R85; 6.7%; OR 1.24).
Blood-vessel replacement
14 / 59 (23.7%)
Mixed hepatic, haematological, malignant and procedural profile; the endpoint itself denotes an intervention. Cholestasis (L108; 7.1%; OR 1.60); Cytopenia (L127; 7.1%; OR 1.50); Unknown hepatobiliary pattern (I87; 7.1%; OR 2.11); Raised alkaline phosphatase/globulin (L33; 7.1%; OR 1.51); Biliary/pancreatic duct dilatation (I96; 7.1%; OR 1.59); Peritoneal carcinomatosis (R98; 7.1%; OR 1.57).
Varicose veins
12 / 35 (34.3%)
Portal-hypertensive, hepatic and fluid findings, alongside thrombotic and inflammatory abnormalities. Cirrhotic morphology/varices (R85; 8.3%; OR 2.62); Chronic liver morphological changes (R1765; 8.3%; OR 1.42); Varices (R414; 8.3%; OR 1.65); Ascites (I21; 8.3%; OR 2.03); Splenomegaly (I65; 8.3%; OR 2.89); Venous thrombosis/embolism (R24; 8.3%; OR 1.54).
Heart failure with preserved ejection fraction
11 / 46 (23.9%)
Vascular findings and cholestasis coexist with device and other abdominal concepts. Aortic atherosclerosis (I26; 9.1%; OR 1.77); Atherosclerosis with colonic abnormalities (I106; 9.1%; OR 1.40); Atherosclerosis (R3; 9.1%; OR 1.57); Cholestasis (L108; 9.1%; OR 1.58); Bladder/prostate abnormalities (I95; 9.1%; OR 1.90); Hernia/prior hernia repair (I1154; 9.1%; OR 1.26).
Appendix
Table 6: Adjusted concept profiles across cardiovascular outcomes at Discovery.
Outcome
Adjusted / associated (%)
Descriptive profile and representative retained concepts (share; adjusted OR)
Altered mental status
11 / 66 (16.7%)
Broad systemic profile spanning hepatic, renal, inflammatory and vascular findings. Cholestasis (L108; 9.1%; OR 1.54); Azotemia (L84; 9.1%; OR 1.29); Raised inflammatory blood-cell counts (L68; 9.1%; OR 1.22); Unknown hepatobiliary pattern (I87; 9.1%; OR 1.37); Splenomegaly (I65; 9.1%; OR 1.35); Aortic calcification (I109; 9.1%; OR 1.26).
Coma
7 / 12 (58.3%)
Heterogeneous profile including hepatic/fluid and vascular abnormalities. Cholestasis (L108; 14.3%; OR 1.82); Anasarca and ascites (R34; 14.3%; OR 1.46); Ascites/portal hypertension (R1815; 14.3%; OR 1.16); Atherosclerosis with colonic abnormalities (I106; 14.3%; OR 1.48); Bladder/prostate abnormalities (I95; 14.3%; OR 1.56).
Delirium, dementia and other cognitive disorders
7 / 35 (20.0%)
Anaemia, calcium and skeletal findings coexist with abdominal abnormalities; this broad endpoint is not dementia-specific. Iron-deficiency anaemia (L104; 14.3%; OR 2.33); Low haemoglobin (L61; 14.3%; OR 2.00); Raised calcium (L1507; 14.3%; OR 1.35); Bone demineralisation (R1031; 14.3%; OR 1.33); Abdominal malignancy (I650; 14.3%; OR 1.78); Bowel/biliary obstruction or splanchnic vascular compromise (I7795; 14.3%; OR 1.49).
Dementias
7 / 36 (19.4%)
Vascular/calcific, skeletal and biliary findings, with a bicarbonate signal. Aortic atherosclerosis (I26; 14.3%; OR 1.51); Mitral annular calcification (R901; 14.3%; OR 1.25); Structural spinal abnormalities (R115; 14.3%; OR 1.75); Rib fractures (R67; 14.3%; OR 1.58); Biliary dilatation (R13; 14.3%; OR 1.43); Raised venous bicarbonate (L181; 14.3%; OR 1.49).
Alzheimer’s disease
6 / 23 (26.1%)
Vascular findings recur, alongside bicarbonate and mixed biliary/urothelial abnormalities. Aortic atherosclerosis (I26; 16.7%; OR 1.63); Atherosclerosis with colonic abnormalities (I106; 16.7%; OR 1.83); Infrarenal aortic aneurysm (R415; 16.7%; OR 1.34); Complicated aortic atherosclerosis (R1690; 16.7%; OR 1.32); Raised venous bicarbonate (L181; 16.7%; OR 1.85); Biliary/urothelial abnormalities (I117; 16.7%; OR 1.91).
Delirium due to other conditions
5 / 31 (16.1%)
Mixed blood-cell and vascular signals, with procedural and nonspecific concepts also contributing to the total. Myeloid-cell abnormalities (L39; 20.0%; OR 1.52); Thrombocytosis/iron-deficiency pattern (L21; 20.0%; OR 1.61); Atherosclerosis with colonic abnormalities (I106; 20.0%; OR 1.50).
Appendix
Table 7: Adjusted concept profiles across neurological and cognitive outcomes at Discovery.
Outcome
Adjusted / associated (%)
Descriptive profile and representative retained concepts (share; adjusted OR)
Peritoneal adhesions
19 / 83 (22.9%)
Predominantly abdominal inflammatory and postsurgical findings, with acute-phase and anaemia signals. Abdominal abscess/complex fluid collection (I54; 5.3%; OR 1.94); Abdominal fluid/inflammation (I38; 5.3%; OR 1.60); Anastomotic fluid collection/leak (I39; 5.3%; OR 1.71); Pancreatic inflammation/fluid collection (I59; 5.3%; OR 1.76); Acute-phase response (L124; 5.3%; OR 1.44); Iron-deficiency anaemia (L104; 5.3%; OR 1.59).
Sepsis
19 / 66 (28.8%)
Broad hepatic, haematological, renal, nutritional, vascular and fluid profile. Cholestasis (L108; 5.3%; OR 1.46); Cytopenia (L127; 5.3%; OR 1.33); Unknown hepatobiliary pattern (I87; 5.3%; OR 1.42); Anaemia with hypoalbuminaemia (L94; 5.3%; OR 1.65); Renal failure with hypernatraemia/acidosis (L62; 5.3%; OR 1.46); Aortic calcification (I109; 5.3%; OR 1.32).
MSSA infection
17 / 51 (33.3%)
Broad anaemia, metabolic and cardiorenal profile, with hepatobiliary and inflammatory imaging findings. Acidosis with severe anaemia (L34; 5.9%; OR 2.48); Renal failure with hypernatraemia/acidosis (L62; 5.9%; OR 2.19); Low haemoglobin (L61; 5.9%; OR 2.13); Thrombocytosis with hypoalbuminaemia/anaemia (L45; 5.9%; OR 1.59); Cardiorenal dysfunction (L2358; 5.9%; OR 1.43); Unknown hepatobiliary pattern (I87; 5.9%; OR 3.11).
Septic shock
15 / 51 (29.4%)
Broad renal, haematological and hepatic profile, with a tentatively interpreted wasting/deconditioning concept. Azotemia (L84; 6.7%; OR 1.43); Renal failure with hypernatraemia/acidosis (L62; 6.7%; OR 1.60); Cytopenia (L127; 6.7%; OR 1.64); Anaemia with hypoalbuminaemia (L94; 6.7%; OR 1.73); Cholestasis (L108; 6.7%; OR 1.70); Tentative wasting/deconditioning pattern (I1459; 6.7%; OR 1.39).
Bacteraemia
13 / 68 (19.1%)
Broad hepatic, haematological, renal and nutritional profile. Cholestasis (L108; 7.7%; OR 1.77); Cytopenia (L127; 7.7%; OR 1.49); Azotemia (L84; 7.7%; OR 1.61); Hypoalbuminaemia (L91; 7.7%; OR 1.65); Anaemia with hypoalbuminaemia (L94; 7.7%; OR 1.52); Unknown hepatobiliary pattern (I87; 7.7%; OR 1.89).
Chronic osteomyelitis
12 / 23 (52.2%)
Musculoskeletal, surgical, renal and haematological findings form a heterogeneous profile. Chronic spinal/pelvic abnormalities (I284; 8.3%; OR 1.46); Cytopenia (L127; 8.3%; OR 2.94); Renal failure with hypernatraemia/acidosis (L62; 8.3%; OR 2.53); Coagulopathy with renal impairment (L87; 8.3%; OR 2.76); Red-cell anisocytosis with hypoproteinaemia (L77; 8.3%; OR 2.78); Orthopaedic hardware (I102; 8.3%; OR 2.08).
Appendix
Table 8: Adjusted concept profiles across infection-related outcomes at Discovery.
Outcome
Discovery
Internal Test
External Site #1
External Site #2
Hepatic / biliary
Liver Abscess And Sequelae Of Chronic Liver Disease (571.8)
143 /7,109 /8,079 (1.97%)
42 /2,392 /2,703 (1.73%)
126 /2,558 /23,119 (4.69%)
105 /6,308 /7,786 (1.64%)
Acute And Subacute Necrosis Of Liver (573.4)
127 /7,387 /7,817 (1.69%)
34 /2,472 /2,631 (1.36%)
93 /2,598 /23,112 (3.46%)
22 /6,405 /7,772 (0.34%)
Disorders Of Bilirubin Excretion (277.4)
122 /7,306 /7,903 (1.64%)
27 /2,426 /2,684 (1.10%)
85 /2,596 /23,122 (3.17%)
105 /6,322 /7,772 (1.63%)
Splenomegaly (579.2)
111 /7,008 /8,212 (1.56%)
50 /2,331 /2,756 (2.10%)
120 /2,549 /23,134 (4.50%)
30 /6,307 /7,862 (0.47%)
Cirrhosis Of Liver Without Mention Of Alcohol (571.51)
110 /7,083 /8,138 (1.53%)
39 /2,346 /2,752 (1.64%)
147 /2,487 /23,169 (5.58%)
137 /6,224 /7,838 (2.15%)
Appendix
Table 9: Outcome case frequencies by cohort, grouped by outcome category.
Name (source units)
ID
Measurements
Numeric (%)
Mean (SD)
IQR
A
Anion Gap (mmol/L)
2466
1,870,639
99.4
9.41 (3.61)
4.00
Albumin, Ser/Plas (g/dL)
1751
1,374,127
99.9
3.49 (10.2)
1.20
ALT (SGPT), Ser/Plas (U/L)
1119
1,322,529
98.5
54.8 (158)
27.0
AST (SGOT), Ser/Plas (U/L)
1398
1,320,653
99.2
53.2 (239)
21.0
Alk P’TASE, Total, Ser/Plas (U/L)
1833
1,291,861
100.0
136 (151)
66.0
Appendix
Table 10: Retained laboratory input types and available measurement-level summary statistics.
Objective: Concept-based interpretation has emerged as a promising approach for explaining blackbox models using human-understandable, high-level concepts. However, most existing methods require annotated concept labels to train the model to identify concepts in the input, and obtaining such annotations in clinical text is often expensive. This study aims to enable concept-based interpretation for language models without concept supervision on clinical prediction tasks. Methods: We propose a novel framework that enables concept-based interpretation without concept supervision through a tailored cross-attention bottleneck. Clinical text and the text descriptions of medical concepts are first tokenized and embedded by a pretrained language model. A cross-attention module then estimates the presence of each concept for each text token, producing concept relevance scores that capture how concepts are involved throughout the note. These scores are pooled and used in a linear prediction layer to model clinical outcomes in an interpretable manner. Results: The performance of the proposed framework was compared against blackbox models across two tasks: chest trauma severity stratification using a private dataset and an in-hospital mortality prediction task using the publicly available MIMIC-III dataset. The proposed framework achieved AUROC values within 0.02 of the corresponding blackbox models. The framework also enabled cohort- and patient-level interpretation, capturing the overall concept importance across the cohort and patient-specific concept contributions. Conclusion: Our proposed framework realizes concept-based interpretation without requiring ground-truth concept presence labels. It improves the transparency of clinical text predictions while maintaining strong performance in clinical prediction tasks.
Jifan Gao, Junjie Hu, John Caskey +6
Department of Data Science, Dana-Farber Cancer Institute, 3 Blackfan St, Boston, 02115, Massachusetts, United States · Department of Biostatistics and Medical Informatics, University of Wisconsin-Madison, 610 Walnut St, Madison, 53726, Wisconsin, United States · Department of Medicine, University of Wisconsin-Madison, 610 Walnut St, Madison, 53726, Wisconsin, United States
Clinical decision support tools are most useful when accurate predictions are accompanied by understandable explanations. Rule-based models provide transparency, but rules derived directly from raw clinical measurements may miss patterns arising from interactions between multiple variables. We present a four-step pipeline that mines decision rules in the latent space of an FT-Transformer and translates them back into measurable clinical features. Embedding dimensions that consistently separate patient groups are treated as latent biomarkers, rules are mined using small decision trees, and selected rules are translated using gradient-input saliency and CLS attention attribution. We evaluate the framework on six public clinical and population health datasets at four embedding dimensions. Translated rules outperformed raw-feature rules in five of six datasets, with mean AUROC gains ranging from 0.04 to 0.23. On the heart disease dataset, embedding-space rules reached 0.98 AUROC, but translation reduced this to 0.72, showing that high-performing latent rules cannot always be represented by simple raw-feature conditions. These results show that latent-space rule discovery can uncover predictive patterns while translating them into clinically measurable features that can be evaluated by clinicians.
Majid Lotfian Delouee, Hamed Ayoobi, Sjors G. J. G. In 't Veld +1
Biological systems are governed by structured molecular interactions, where pathways, regulatory circuits, and functional gene relationships shape cellular behavior and disease progression. Much of this knowledge is naturally represented as graphs. However, most biomedical AI models cannot directly use graph-encoded biological knowledge and instead require compressed low-dimensional representations, which can lose important structure and reduce performance, especially in limited-sample clinical studies. Here, we introduce Graph-in-Graph (GiG), a knowledge graph-modulated deep learning framework for data-efficient clinical prediction. GiG represents each patient as a standalone modular graph, in which curated biological knowledge graphs define edges and patient-specific measurements, such as gene expression, define node features. This design allows multiple biological knowledge graphs to be integrated while preserving gene-gene interactions and pathway topology during patient-level representation learning. Across cohorts comprising nearly 9,700 patients and five clinical tasks, including liquid biopsy cancer detection, prostate cancer diagnosis, and 32-class pan-cancer classification, GiG consistently outperforms traditional and state-of-the-art methods, with the largest gains in limited-sample settings. On the challenging prostate cancer diagnosis task, GiG improves macro-F1 by up to 49 percentage points relative to competing methods. Control experiments replacing real pathway graphs with random topologies confirm that these gains arise from biologically grounded knowledge graph structure rather than graph modeling alone. These findings show that knowledge graph-modulated deep learning can improve robustness, interpretability, and sample efficiency in clinical data analysis, and provide a principled framework for integrating biological knowledge graphs into predictive modeling.
Yuwei Xue, Sakib Mostafa, James Zou +5
Department of Radiation Oncology, Stanford University, Stanford, California, USA · Department of Electrical Engineering, Stanford University, Stanford, California, USA · Department of Biomedical Data Science, Stanford University School of Medicine, Stanford, California, USA +6