Glyph: An Interpretability Engine for AI-Driven Scientific Discovery in Clinical Data
Organizations: Stanford Center for Artificial Intelligence in Medicine and Imaging, Stanford University, CA, USA · Department of Radiology, Stanford University, Stanford, CA, USA · Department of Radiology, Mayo Clinic, Phoenix, Arizona, USA · Technical University of Munich, Munich, Germany ·
Abstract
Automated scientific discovery has the potential to advance understanding of disease mechanisms and accelerate identification of early disease markers. Manual data annotation is infeasible at population scale, motivating methods that automatically extract pre-defined phenotypes in multimodal and unstructured clinical data. Such pre-defined feature sets determine and limit which associations these approaches can discover. This limitation extends to agentic AI systems that use these tools to query population-scale data. In contrast, foundation models (FMs) trained on large volumes of clinical data can perform tasks beyond their training objectives. This suggests their latent embeddings may represent novel or unexplored features that could yield insights into disease mechanisms, yet such latent features remain difficult to interpret. To address this challenge, we introduce Glyph, an automated multistage framework that extracts features from multimodal and unstructured clinical data using FMs and sparse autoencoders, interprets them as coded clinical concepts, and evaluates their value for clinical scientific discovery. We applied Glyph to abdominal CT volumes, CT radiology reports and laboratory measurements from 41,628 patients across different institutions. We identified 397 predictive concepts supported by 4,463 Bonferroni-significant associations. Moreover, a majority of discovered associations were automatically reproducible in held-out cohorts across different sites. Glyph concepts provided complementary associations to literature-derived markers, leading to gains in predictive power. We envision that interpretability can extend beyond a mechanistic understanding of neural networks to connecting implicitly learned features with clinical knowledge and establishing their significance for patient outcomes. We release Glyph at https://github.com/StanfordMIMI/Glyph.
Figures & tables
| Discovery (Train + Val) | Internal Test | External Site #1 | External Site #2 | |||||
| Dataset Characteristics | n | % | n | % | n | % | n | % |
| Accessions (Abdominal CTs) | 20,391 | – | 5,137 | – | 25,803 | – | 14,199 | – |
| Unique Patients | 14,652 | – | 3,667 | – | 17,224 | – | 6,085 | – |
| Age (mean) | 54.1 18.9 | – | 54.5 19.1 | – | 60.6 18.9 | – | 62.1 16.9 | – |
| Sex | ||||||||
| Female | 11,407 | 55.9 | 2,927 | 57.0 | 14,039 | 54.4 | 6,994 | 49.3 |
| Cohort | Correction | Threshold |
|---|---|---|
| Discovery (Train) | Bonferroni | |
| Internal Test | Benjamini–Hochberg (5% FDR) | |
| External Site #1 | Benjamini–Hochberg (5% FDR) | |
| External Site #2 | Benjamini–Hochberg (5% FDR) |
| Cohort | Correction | Threshold |
|---|---|---|
| Discovery (Train) | Benjamini–Hochberg (5% FDR) | |
| Internal Test | Benjamini–Hochberg (5% FDR) | |
| External Site #1 | Benjamini–Hochberg (5% FDR) | |
| External Site #2 | Benjamini–Hochberg (5% FDR) |
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
| Outcome | Adjusted / associated (%) | Descriptive profile and representative retained concepts (share; adjusted OR) |
|---|---|---|
| Acute/subacute liver necrosis | 20 / 60 (33.3%) | Hepatic abnormalities coexist with cytopenia, hypoalbuminaemia and splenic findings. Cirrhotic morphology/varices (R85; 5.0%; OR 1.92); Cholestasis (L108; 5.0%; OR 2.37); Unknown hepatobiliary pattern (I87; 5.0%; OR 2.42); Cytopenia (L127; 5.0%; OR 2.26); Hypoalbuminaemia (L91; 5.0%; OR 2.04); Splenomegaly (I65; 5.0%; OR 2.56). |
| Liver transplant status | 19 / 36 (52.8%) | Broad hepatic, renal and haematological profile, alongside splenic and portal-vascular findings. Cholestasis (L108; 5.3%; OR 3.06); Unknown hepatobiliary pattern (I87; 5.3%; OR 4.31); Cirrhotic morphology/varices (R85; 5.3%; OR 2.11); Azotemia (L84; 5.3%; OR 2.40); Cytopenia (L127; 5.3%; OR 3.09); Splenomegaly (I65; 5.3%; OR 2.99). |
| Portal hypertension | 16 / 59 (27.1%) | Predominantly cirrhotic, splenic, variceal and ascitic findings, with cholestasis. Cirrhotic morphology/varices (R85; 6.2%; OR 2.46); Varices (R414; 6.2%; OR 1.48); Chronic liver disease/congestion (I331; 6.2%; OR 1.50); Splenomegaly (I65; 6.2%; OR 2.79); Ascites/portal-hypertensive or peritoneal findings (I31; 6.2%; OR 1.78); Cholestasis (L108; 6.2%; OR 3.62). |
| Disorders of bilirubin excretion | 13 / 44 (29.5%) | Cholestatic and biliary imaging findings predominate among the illustrated markers. Cholestasis (L108; 7.7%; OR 2.38); Biliary dilatation (I454; 7.7%; OR 1.49); Unknown hepatobiliary pattern (I87; 7.7%; OR 2.44); Raised alkaline phosphatase/globulin (L33; 7.7%; OR 1.77); Biliary obstruction/intervention (I1416; 7.7%; OR 1.33); Pneumobilia (R473; 7.7%; OR 1.48). |
| Liver abscess/chronic liver disease sequelae | 13 / 45 (28.9%) | Structural hepatic and portal-hypertensive findings coexist with abscess/inflammatory concepts. Cirrhotic morphology/varices (R85; 7.7%; OR 2.98); Chronic liver disease/congestion (I331; 7.7%; OR 1.46); Chronic liver morphological changes (R1765; 7.7%; OR 1.61); Varices (R414; 7.7%; OR 1.57); Abscess/fluid collection/fistula (I7223; 7.7%; OR 1.47); Unknown hepatobiliary pattern (I87; 7.7%; OR 2.87). |
| Chronic passive hepatic congestion | 10 / 23 (43.5%) | Cardiac/vascular findings coexist with a hepatobiliary pattern, consistent with a congestive profile. Unknown hepatobiliary pattern (I87; 10.0%; OR 2.73); Ventricular assist device (R283; 10.0%; OR 1.16); Cardiac enlargement/vascular calcification (R487; 10.0%; OR 1.32); Atrial enlargement (R3570; 10.0%; OR 1.09); Aortic atherosclerosis (I26; 10.0%; OR 1.91). |
| Outcome | Adjusted / associated (%) | Descriptive profile and representative retained concepts (share; adjusted OR) |
|---|---|---|
| Chronic renal failure | 21 / 67 (31.3%) | Broad systemic profile alongside renal dysfunction. Cytopenia (L127; 4.8%; OR 1.64); Aortic calcification (I109; 4.8%; OR 1.78); Splenomegaly (I65; 4.8%; OR 1.39); Ascites (I21; 4.8%; OR 1.54); Azotemia (L84; 4.8%; OR 1.75); Impaired renal function (L69; 4.8%; OR 1.51). |
| Acute renal failure | 20 / 83 (24.1%) | Broad systemic profile spanning hepatic, haematological, fluid and vascular abnormalities. Cholestasis (L108; 5.0%; OR 1.67); Cytopenia (L127; 5.0%; OR 1.41); Splenomegaly (I65; 5.0%; OR 1.46); Unknown hepatobiliary pattern (I87; 5.0%; OR 1.39); Ascites (I21; 5.0%; OR 1.37); Atherosclerosis with colonic abnormalities (I106; 5.0%; OR 1.28). |
| CKD (composite) | 18 / 74 (24.3%) | Broad systemic profile spanning hepatic, haematological, fluid and vascular abnormalities. Cholestasis (L108; 5.6%; OR 1.55); Cytopenia (L127; 5.6%; OR 1.35); Unknown hepatobiliary pattern (I87; 5.6%; OR 1.48); Splenomegaly (I65; 5.6%; OR 1.34); Ascites, anasarca and effusions (I25; 5.6%; OR 1.37); Aortic calcification (I109; 5.6%; OR 1.22). |
| CKD, stage III | 15 / 70 (21.4%) | Mixed renal-function, vascular and haematological profile, with a cholestatic signal. Impaired renal function (L69; 6.7%; OR 1.45); Reduced eGFR (L974; 6.7%; OR 1.21); Polycystic renal disease/polyps (R302; 6.7%; OR 1.18); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.43); Thrombocytosis/iron-deficiency pattern (L21; 6.7%; OR 1.38); Cholestasis (L108; 6.7%; OR 1.51). |
| Hypertensive CKD | 15 / 47 (31.9%) | Mixed renal, vascular, hepatic and haematological profile. Azotemia (L84; 6.7%; OR 1.59); Renal failure with hypernatraemia/acidosis (L62; 6.7%; OR 1.47); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.44); Coronary/valvular calcification (R21; 6.7%; OR 1.34); Cholestasis (L108; 6.7%; OR 1.60); Cytopenia (L127; 6.7%; OR 1.50). |
| Anaemia in CKD | 14 / 33 (42.4%) | Anaemia and renal-function markers coexist with vascular findings. Iron-deficiency anaemia (L104; 7.1%; OR 2.81); Low haemoglobin (L61; 7.1%; OR 2.01); Impaired renal function (L69; 7.1%; OR 1.57); Reduced eGFR (L974; 7.1%; OR 1.46); Aortic calcification (I109; 7.1%; OR 2.06); Atherosclerosis with colonic abnormalities (I106; 7.1%; OR 1.87). |
| Outcome | Adjusted / associated (%) | Descriptive profile and representative retained concepts (share; adjusted OR) |
|---|---|---|
| Congestive heart failure | 16 / 48 (33.3%) | Broad cardiorenal, vascular, hepatic and laboratory profile, with cardiac-device concepts also retained. Aortic atherosclerosis (I26; 6.2%; OR 1.67); Cardiorenal dysfunction (L2358; 6.2%; OR 1.26); Azotemia (L84; 6.2%; OR 1.69); Cholestasis (L108; 6.2%; OR 1.65); Cardiac dysfunction with hypoalbuminaemia (L1488; 6.2%; OR 1.29); Hypoalbuminaemia (L91; 6.2%; OR 1.72). |
| Ischaemic heart disease (composite) | 15 / 49 (30.6%) | Vascular calcification coexists with inflammatory, metabolic and hepatobiliary findings. Coronary/valvular calcification (R21; 6.7%; OR 1.19); Coronary calcification/atherosclerosis (R420; 6.7%; OR 1.20); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.35); Raised inflammatory blood-cell counts (L68; 6.7%; OR 1.21); Dyslipidaemia with inflammation/hyperglycaemia (L496; 6.7%; OR 1.16); Unknown hepatobiliary pattern (I87; 6.7%; OR 1.29). |
| Myocardial infarction | 15 / 55 (27.3%) | Broad vascular, cardiorenal, hepatic, inflammatory and anaemia profile. Aortic atherosclerosis (I26; 6.7%; OR 1.29); Atherosclerosis with colonic abnormalities (I106; 6.7%; OR 1.37); Cardiorenal dysfunction (L2358; 6.7%; OR 1.17); Raised inflammatory blood-cell counts (L68; 6.7%; OR 1.29); Anaemia with hypoalbuminaemia (L94; 6.7%; OR 1.28); Cirrhotic morphology/varices (R85; 6.7%; OR 1.24). |
| Blood-vessel replacement | 14 / 59 (23.7%) | Mixed hepatic, haematological, malignant and procedural profile; the endpoint itself denotes an intervention. Cholestasis (L108; 7.1%; OR 1.60); Cytopenia (L127; 7.1%; OR 1.50); Unknown hepatobiliary pattern (I87; 7.1%; OR 2.11); Raised alkaline phosphatase/globulin (L33; 7.1%; OR 1.51); Biliary/pancreatic duct dilatation (I96; 7.1%; OR 1.59); Peritoneal carcinomatosis (R98; 7.1%; OR 1.57). |
| Varicose veins | 12 / 35 (34.3%) | Portal-hypertensive, hepatic and fluid findings, alongside thrombotic and inflammatory abnormalities. Cirrhotic morphology/varices (R85; 8.3%; OR 2.62); Chronic liver morphological changes (R1765; 8.3%; OR 1.42); Varices (R414; 8.3%; OR 1.65); Ascites (I21; 8.3%; OR 2.03); Splenomegaly (I65; 8.3%; OR 2.89); Venous thrombosis/embolism (R24; 8.3%; OR 1.54). |
| Heart failure with preserved ejection fraction | 11 / 46 (23.9%) | Vascular findings and cholestasis coexist with device and other abdominal concepts. Aortic atherosclerosis (I26; 9.1%; OR 1.77); Atherosclerosis with colonic abnormalities (I106; 9.1%; OR 1.40); Atherosclerosis (R3; 9.1%; OR 1.57); Cholestasis (L108; 9.1%; OR 1.58); Bladder/prostate abnormalities (I95; 9.1%; OR 1.90); Hernia/prior hernia repair (I1154; 9.1%; OR 1.26). |
| Outcome | Adjusted / associated (%) | Descriptive profile and representative retained concepts (share; adjusted OR) |
|---|---|---|
| Altered mental status | 11 / 66 (16.7%) | Broad systemic profile spanning hepatic, renal, inflammatory and vascular findings. Cholestasis (L108; 9.1%; OR 1.54); Azotemia (L84; 9.1%; OR 1.29); Raised inflammatory blood-cell counts (L68; 9.1%; OR 1.22); Unknown hepatobiliary pattern (I87; 9.1%; OR 1.37); Splenomegaly (I65; 9.1%; OR 1.35); Aortic calcification (I109; 9.1%; OR 1.26). |
| Coma | 7 / 12 (58.3%) | Heterogeneous profile including hepatic/fluid and vascular abnormalities. Cholestasis (L108; 14.3%; OR 1.82); Anasarca and ascites (R34; 14.3%; OR 1.46); Ascites/portal hypertension (R1815; 14.3%; OR 1.16); Atherosclerosis with colonic abnormalities (I106; 14.3%; OR 1.48); Bladder/prostate abnormalities (I95; 14.3%; OR 1.56). |
| Delirium, dementia and other cognitive disorders | 7 / 35 (20.0%) | Anaemia, calcium and skeletal findings coexist with abdominal abnormalities; this broad endpoint is not dementia-specific. Iron-deficiency anaemia (L104; 14.3%; OR 2.33); Low haemoglobin (L61; 14.3%; OR 2.00); Raised calcium (L1507; 14.3%; OR 1.35); Bone demineralisation (R1031; 14.3%; OR 1.33); Abdominal malignancy (I650; 14.3%; OR 1.78); Bowel/biliary obstruction or splanchnic vascular compromise (I7795; 14.3%; OR 1.49). |
| Dementias | 7 / 36 (19.4%) | Vascular/calcific, skeletal and biliary findings, with a bicarbonate signal. Aortic atherosclerosis (I26; 14.3%; OR 1.51); Mitral annular calcification (R901; 14.3%; OR 1.25); Structural spinal abnormalities (R115; 14.3%; OR 1.75); Rib fractures (R67; 14.3%; OR 1.58); Biliary dilatation (R13; 14.3%; OR 1.43); Raised venous bicarbonate (L181; 14.3%; OR 1.49). |
| Alzheimer’s disease | 6 / 23 (26.1%) | Vascular findings recur, alongside bicarbonate and mixed biliary/urothelial abnormalities. Aortic atherosclerosis (I26; 16.7%; OR 1.63); Atherosclerosis with colonic abnormalities (I106; 16.7%; OR 1.83); Infrarenal aortic aneurysm (R415; 16.7%; OR 1.34); Complicated aortic atherosclerosis (R1690; 16.7%; OR 1.32); Raised venous bicarbonate (L181; 16.7%; OR 1.85); Biliary/urothelial abnormalities (I117; 16.7%; OR 1.91). |
| Delirium due to other conditions | 5 / 31 (16.1%) | Mixed blood-cell and vascular signals, with procedural and nonspecific concepts also contributing to the total. Myeloid-cell abnormalities (L39; 20.0%; OR 1.52); Thrombocytosis/iron-deficiency pattern (L21; 20.0%; OR 1.61); Atherosclerosis with colonic abnormalities (I106; 20.0%; OR 1.50). |
| Outcome | Adjusted / associated (%) | Descriptive profile and representative retained concepts (share; adjusted OR) |
|---|---|---|
| Peritoneal adhesions | 19 / 83 (22.9%) | Predominantly abdominal inflammatory and postsurgical findings, with acute-phase and anaemia signals. Abdominal abscess/complex fluid collection (I54; 5.3%; OR 1.94); Abdominal fluid/inflammation (I38; 5.3%; OR 1.60); Anastomotic fluid collection/leak (I39; 5.3%; OR 1.71); Pancreatic inflammation/fluid collection (I59; 5.3%; OR 1.76); Acute-phase response (L124; 5.3%; OR 1.44); Iron-deficiency anaemia (L104; 5.3%; OR 1.59). |
| Sepsis | 19 / 66 (28.8%) | Broad hepatic, haematological, renal, nutritional, vascular and fluid profile. Cholestasis (L108; 5.3%; OR 1.46); Cytopenia (L127; 5.3%; OR 1.33); Unknown hepatobiliary pattern (I87; 5.3%; OR 1.42); Anaemia with hypoalbuminaemia (L94; 5.3%; OR 1.65); Renal failure with hypernatraemia/acidosis (L62; 5.3%; OR 1.46); Aortic calcification (I109; 5.3%; OR 1.32). |
| MSSA infection | 17 / 51 (33.3%) | Broad anaemia, metabolic and cardiorenal profile, with hepatobiliary and inflammatory imaging findings. Acidosis with severe anaemia (L34; 5.9%; OR 2.48); Renal failure with hypernatraemia/acidosis (L62; 5.9%; OR 2.19); Low haemoglobin (L61; 5.9%; OR 2.13); Thrombocytosis with hypoalbuminaemia/anaemia (L45; 5.9%; OR 1.59); Cardiorenal dysfunction (L2358; 5.9%; OR 1.43); Unknown hepatobiliary pattern (I87; 5.9%; OR 3.11). |
| Septic shock | 15 / 51 (29.4%) | Broad renal, haematological and hepatic profile, with a tentatively interpreted wasting/deconditioning concept. Azotemia (L84; 6.7%; OR 1.43); Renal failure with hypernatraemia/acidosis (L62; 6.7%; OR 1.60); Cytopenia (L127; 6.7%; OR 1.64); Anaemia with hypoalbuminaemia (L94; 6.7%; OR 1.73); Cholestasis (L108; 6.7%; OR 1.70); Tentative wasting/deconditioning pattern (I1459; 6.7%; OR 1.39). |
| Bacteraemia | 13 / 68 (19.1%) | Broad hepatic, haematological, renal and nutritional profile. Cholestasis (L108; 7.7%; OR 1.77); Cytopenia (L127; 7.7%; OR 1.49); Azotemia (L84; 7.7%; OR 1.61); Hypoalbuminaemia (L91; 7.7%; OR 1.65); Anaemia with hypoalbuminaemia (L94; 7.7%; OR 1.52); Unknown hepatobiliary pattern (I87; 7.7%; OR 1.89). |
| Chronic osteomyelitis | 12 / 23 (52.2%) | Musculoskeletal, surgical, renal and haematological findings form a heterogeneous profile. Chronic spinal/pelvic abnormalities (I284; 8.3%; OR 1.46); Cytopenia (L127; 8.3%; OR 2.94); Renal failure with hypernatraemia/acidosis (L62; 8.3%; OR 2.53); Coagulopathy with renal impairment (L87; 8.3%; OR 2.76); Red-cell anisocytosis with hypoproteinaemia (L77; 8.3%; OR 2.78); Orthopaedic hardware (I102; 8.3%; OR 2.08). |
| Outcome | Discovery | Internal Test | External Site #1 | External Site #2 |
|---|---|---|---|---|
| Hepatic / biliary | ||||
| Liver Abscess And Sequelae Of Chronic Liver Disease (571.8) | 143 /7,109 /8,079 (1.97%) | 42 /2,392 /2,703 (1.73%) | 126 /2,558 /23,119 (4.69%) | 105 /6,308 /7,786 (1.64%) |
| Acute And Subacute Necrosis Of Liver (573.4) | 127 /7,387 /7,817 (1.69%) | 34 /2,472 /2,631 (1.36%) | 93 /2,598 /23,112 (3.46%) | 22 /6,405 /7,772 (0.34%) |
| Disorders Of Bilirubin Excretion (277.4) | 122 /7,306 /7,903 (1.64%) | 27 /2,426 /2,684 (1.10%) | 85 /2,596 /23,122 (3.17%) | 105 /6,322 /7,772 (1.63%) |
| Splenomegaly (579.2) | 111 /7,008 /8,212 (1.56%) | 50 /2,331 /2,756 (2.10%) | 120 /2,549 /23,134 (4.50%) | 30 /6,307 /7,862 (0.47%) |
| Cirrhosis Of Liver Without Mention Of Alcohol (571.51) | 110 /7,083 /8,138 (1.53%) | 39 /2,346 /2,752 (1.64%) | 147 /2,487 /23,169 (5.58%) | 137 /6,224 /7,838 (2.15%) |
| Name (source units) | ID | Measurements | Numeric (%) | Mean (SD) | IQR |
|---|---|---|---|---|---|
| A | |||||
| Anion Gap (mmol/L) | 2466 | 1,870,639 | 99.4 | 9.41 (3.61) | 4.00 |
| Albumin, Ser/Plas (g/dL) | 1751 | 1,374,127 | 99.9 | 3.49 (10.2) | 1.20 |
| ALT (SGPT), Ser/Plas (U/L) | 1119 | 1,322,529 | 98.5 | 54.8 (158) | 27.0 |
| AST (SGOT), Ser/Plas (U/L) | 1398 | 1,320,653 | 99.2 | 53.2 (239) | 21.0 |
| Alk P’TASE, Total, Ser/Plas (U/L) | 1833 | 1,291,861 | 100.0 | 136 (151) | 66.0 |
AI-Assisted Scientific Research
AI-assisted scientific research uses machine learning and AI agents to support discovery, analysis, experimentation, and scientific communication.
See all 267 papers