HCOE: Hyperbolic Clinical Ontology Embeddings from Biomedical Language Models

Yixuan Li; Weihao Li; Ziyang Song

cs.AISep 25, 2026

HCOE: Hyperbolic Clinical Ontology Embeddings from Biomedical Language Models

Authors: Yixuan Li, Weihao Li, Ziyang Song

Organizations: McGill University · Northwestern University · Ohio University

Abstract

Biomedical language models (LMs) encode textual semantics but do not explicitly preserve medical code hierarchies. We present Hyperbolic Clinical Ontology Embeddings (HCOE) for hierarchy-aware clinical concept representation. HCOE maps frozen BioBERT embeddings into a Poincare ball, combining parent-side and child-side ontology-guided contrastive learning with coarse-to-fine ontology-path aggregation. It uses International Classification of Diseases (ICD) codes organized by Clinical Classifications Software (CCS) and Anatomical Therapeutic Chemical (ATC) medication hierarchies. Evaluations show that HCOE performs best on ICD/ATC clinical relation prediction and CCS-to-PheCode hierarchy transfer. On the MIMIC-IV dataset, HCOE also achieves the best performance on mortality prediction, readmission prediction, medication recommendation, and rare drug prediction.

Figures & tables

Fig. 1: Comparison of 1-hop parent–child relation prediction in the ICD hierarchy. We assess whether BioBERT and HCOE embed each child concept closer to its true parent than to unrelated concepts across five levels. HCOE consistently outperforms BioBERT, with larger gains at finer-grained levels.

Fig. 2: Overview of HCOE. a . An expert-curated medical ontology organizes clinical concepts into coarse-to-fine hierarchy levels. b . HCOE uses hyperbolic contrastive learning to organize biomedical LM-encoded clinical codes, pulling parent–child concepts closer and pushing unrelated concepts apart. c . For a target clinical concept, HCOE aggregates embeddings along its ontology path to form a coarse-to-fine hierarchy-aware representation.

Method	ICD $\rightarrow$ ICD			ATC $\rightarrow$ ATC			ICD $\rightarrow$ ATC			CCS $\rightarrow$ PheCode
Method	Precision	Recall	F-score	Precision	Recall	F-score	Precision	Recall	F-score	Precision	Recall	F-score
HCOE	76.3	74.9	75.6	85.1	69.3	76.3	54.2	59.4	56.7	85.4	76.3	80.6
BioBERT	30.4	75.8	43.3	50.7	45.7	48.1	49.3	57.0	52.9	76.8	64.1	69.9
SapBERT	69.9	64.5	67.1	84.0	66.9	74.5	50.2	55.7	52.9	73.4	75.9	74.7
HiTs	69.3	61.7	65.3	83.0	66.2	73.6	50.5	56.3	53.2	72.8	80.5	76.5
OnT	65.9	59.1	62.3	78.5	62.3	69.5	48.9	55.2	51.9	78.2	71.5	74.7

TABLE I: Clinical relation prediction on ICD

\rightarrow

ICD, ATC

\rightarrow

ATC, ICD

\rightarrow

ATC, and CCS

\rightarrow

PheCode. Thresholds are selected on validation and fixed for testing without variance. The best results in each column are highlighted in bold.

Method	Mortality		Readmission		Drug			Rare Drug
	AUPRC	AUROC	AUPRC	AUROC	AUPRC	F1	Jaccard	Recall@15
HCOE	44.3 (0.8)	87.2 (1.1)	48.7 (0.6)	72.3 (0.6)	76.3 (0.1)	64.3 (0.2)	50.7 (0.2)	44.2 (0.3)
GRAM	38.5 (0.7)	79.1 (0.9)	43.2 (0.8)	68.2 (0.4)	72.0 (0.3)	61.3 (0.4)	48.6 (0.4)	34.5 (0.2)
cui2vec	39.0 (0.4)	80.4 (0.8)	43.7 (0.6)	68.9 (0.3)	75.2 (0.1)	64.3 (0.3)	49.9 (0.4)	36.6 (0.1)
RotatE	39.2 (0.3)	79.6 (0.6)	43.7 (0.4)	69.2 (0.6)	74.1 (0.2)	62.7 (0.4)	48.7 (0.4)	37.4 (0.1)
BioBERT	43.2 (0.9)	86.1 (1.3)	46.5 (0.9)	71.5 (1.2)	73.9 (0.1)	60.3 (0.2)	44.5 (0.2)	40.6 (0.2)

TABLE II: Clinical prediction results on MIMIC-IV for mortality, readmission, drug recommendation, and rare drug prediction tasks. Metrics are reported as mean (standard error) from bootstrap. The best results in each column are highlighted in bold.

Fig. 3: Visualization of HCOE embeddings for ICD diagnosis codes under the endocrine disease category (major category 3) using a Poincaré map. The embeddings of the endocrine disease category exhibit a radial hierarchy, with coarse concepts near the center and fine-grained codes toward the boundary.

Variant	Path aggregation	Contrastive objective	ICD $\rightarrow$ ICD	ATC $\rightarrow$ ATC	ICD $\rightarrow$ ATC	CCS $\rightarrow$ PheCode
Full HCOE	Yes	Parent + Child	75.6	76.3	56.7	80.6
w/o path aggregation	No	Parent + Child	74.3	75.0	55.1	78.6
Parent-side only	Yes	Parent only	71.7	72.7	55.8	75.0
Child-side only	Yes	Child only	63.5	67.4	52.5	73.5
w/o contrastive learning	Yes	None	47.6	51.6	51.2	71.5

TABLE III: Ablation of HCOE on four clinical relation prediction tasks, including ICD

\rightarrow

ICD, ATC

\rightarrow

ATC, ICD

\rightarrow

ATC, and CCS

\rightarrow

PheCode. We report F-score for each task. The best results in each column are highlighted in bold.

$d_{h}$	ICD $\rightarrow$ ICD	ICD $\rightarrow$ ATC	CCS $\rightarrow$ PheCode
128	74.4	56.1	79.2
256	75.6	56.7	80.6
768	72.7	56.3	77.4

TABLE IV: Sensitivity of HCOE to the hyperbolic latent dimension. We report F-score on clinical relation prediction tasks.

Medication Recommendation

See all 18 papers

SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation
Oct 13, 2026Ziyang Song, Lia Shen, Yixuan Li +6Representation Learning Hierarchical Representation Learning
School of Electrical Engineering and Computer Science, Ohio University · Department of Quantitative and Computational Biology, University of Southern California · School of Computer Science, McGill University +2
Hierarchical Modeling of ICD Codes in EHR Foundation Models
Jun 13, 2026Megha Thukral, Dong Gyun Kang, Rudra Pratap Singh +3Graph Representation Learning Hierarchical Representation Learning
School of Interactive Computing, Georgia Institute of Technology, Atlanta, United States · Optum AI, United States
Text-Attributed Knowledge Graph Enrichment with Large Language Models for Medical Concept Representation
Apr 14, 2026Mohsen Nayebi Kerdabadi, Arya Hadizadeh Moghaddam, Chen Chen +2Representation Learning Biomedical KGs
University of Kansas, USA

Oct 13, 2026J/K move · Enter open · S save

SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation

Ziyang Song, Lia Shen, Yixuan Li, Qincheng Lu+5

School of Electrical Engineering and Computer Science, Ohio University · Department of Quantitative and Computational Biology, University of Southern California · School of Computer Science, McGill University+2

Representation Learning Hierarchical Representation Learning

Open paper AskPDF arXiv

Biomedical language models (LMs) provide strong representations for clinical tasks, but they often fail to capture the hierarchical structure among medical concepts. This limitation is especially pronounced for rare and fine-grained clinical codes in long-tailed vocabularies, where hierarchy-agnostic LM representations become less informative as concept frequency decreases. We introduce Semantic Medical Identifier (SMI), a training-free framework for constructing ontology-grounded Semantic IDs and hierarchy-aware medical concept representations. For each diagnosis, procedure, or medication concept, SMI defines its Semantic ID as the coarse-to-fine path through an expert-defined medical ontology. SMI then aggregates frozen biomedical LM embeddings across the corresponding ontology levels to form hierarchy-aware medical concept representations that preserve coarse-to-fine clinical semantics. The base SMI uses uniform aggregation across ontology levels, while a level-weighted variant selects global level weights on validation data. Our theoretical analysis shows that SMI yields more informative medical concept representations than hierarchy-agnostic LM embeddings, particularly for rare and fine-grained clinical concepts. Experiments show that SMI captures meaningful hierarchical structure and improves clinical relation prediction. On MIMIC-IV, SMI improves mortality prediction, readmission prediction, drug recommendation, and rare drug prediction. On eICU, SMI yields more robust patient representations under cross-hospital distribution shift.

Abstract

Medication Recommendation

Explore similar work

SMI: Ontology-Grounded Semantic IDs for Hierarchy-Aware Medical Concept Representation

Hierarchical Modeling of ICD Codes in EHR Foundation Models

Text-Attributed Knowledge Graph Enrichment with Large Language Models for Medical Concept Representation