SNO-PI: Label-Scarce SNOMED CT Entity Linking with Pre-trained Encoders and Curated Terminology Knowledge

Piyawoot Songsiritat

cs.CLOct 19, 2026

SNO-PI: Label-Scarce SNOMED CT Entity Linking with Pre-trained Encoders and Curated Terminology Knowledge

Authors: Piyawoot Songsiritat

Organizations: The University of Queensland, Greater Brisbane Clinical School, Brisbane, Australia

Abstract

We present SNO-PI, the top-performing submission on SNOMED International's SNOMED CT Entity Linking Benchmark as of 18 October 2026. SNO-PI achieves a macro character IoU of 0.5387, outperforming the organisers' reference system (0.4582; based on the 2024 challenge winner's approach) by 18% relative. Linking clinical mentions to SNOMED CT is a zero- and few-shot problem over a very large label space: the 272 annotated MIMIC-IV discharge summaries cover only 6,595 of the roughly 219,000 eligible concepts, and a third of those occur only once. This motivates a design built on transfer from pre-trained models and human-curated knowledge. SNO-PI combines high-recall span proposal, SapBERT candidate retrieval and listwise ensemble reranking with three BERT-family encoders, each further pre-trained with masked language modelling on unlabelled MIMIC-IV notes and then on SNOMED CT synonyms and hierarchy before task fine-tuning. We find that curated knowledge helps most on concepts never seen in training, yet these concepts remain the main source of error.

Figures & tables

Panel of Figure 1 — Figure 1: Label coverage grows slowly with annotation effort. Test gold coverage over random 20-note samples, for random subsets of training notes (3 seeds). Dashed: log-linear extrapolation to all 272 notes.

Panel of Figure 2 — Figure 2: Synthetic illustration of note-level variation: one list item is linked, its neighbours are not.

Panel of Figure 3 — Figure 3: SNO-PI architecture. Solid arrows: inference. Dashed boxes: the training recipe for the encoders. The illustrative input is synthetic.

Knowledge source	Where it enters	Measured effect
SNOMED terms and synonyms	span proposal, retrieval index, option text	only source of candidates for unseen concepts (R@10 .77)
UMLS synonym sets (via SapBERT)	retrieval	R@10 .969 overall
Pre-trained encoders (general and biomedical)	reranker initialisation, ensemble	prior that SNOMED pre-training builds on (SNOMED pre-training row); BioLinkBERT best on unseen in a controlled base-size comparison (.784)
50k unlabelled notes (MLM)	tagger and encoders	tagger F1 +.009; small but consistent reranker gains
SNOMED pre-training	reranker initialisation	concept acc. .26 $\rightarrow$ .85 / .71 (seen / unseen) with no labels (Table 2 ); +.015 macro after fine-tuning (ablation)
SNOMED hierarchy	scope filter, parent in option text, pre-training hard negatives	scope via is-a keeps the $\sim$ 22% of gold annotations a semantic-tag filter would drop; part of SNOMED pre-training gain (+.015, above)

Table 1: Where non-label knowledge enters SNO-PI and what it bought (local test, 174 training notes).

	Concept acc.		Linked
Stage	seen	unseen	(unseen)
SapBERT retriever (top-1)	.285	.550	.216
C-DAPT, raw encoder	.262	.266	.000
+ SNOMED stage	.849	.705	.106
+ note fine-tuning	.948	.829	.725

Table 2: The C-DAPT reranker (Bio_ClinicalBERT after MIMIC-IV MLM) after each training stage, local test. Concept acc.: the gold concept is ranked first among the ten concept options, before the threshold and the none check. Linked: it also passes the

p\geq 0.5

threshold and the none check (unseen concepts only).

System	Macro	Weighted
SNO-PI (rank 1)	0.5387	0.6629
Rank 2 entry	0.4985	0.6423
Rank 3 entry	0.4695	0.6273
Rank 4 entry	0.4657	0.6165
Organisers’ reference (Super Dictionary)	0.4582	0.6234
Rank 6 entry	0.4214	0.5831

Table 3: The public benchmark leaderboard on 18 October 2026.

	Local	Hidden test
Development stage	dev / test	macro / wtd
Three-encoder ensemble	.504 / .511	.500 / .628
+ six members, wider context	.532 / .530	–
+ retrained on all 272 notes	–	.535 / .658
+ Qwen3-8B member (final)	–	.539 / .663

Table 4: Development history; each row adds to the one above. The first row uses MLM and SNOMED pre-training. Local scores are the mean over 1,000 random 20-note subsets (trained on 174 notes). Hidden-test scores are the official results of our submissions (October 2026).

Panel of Figure 4 — Figure 4: Concept prediction coverage increases with training frequency: share of test gold concepts predicted at least once, by the concept’s number of training examples (earlier three-encoder system, local test, 300 random 20-note samples).

	Dev	Test
Gold spans in span set (exact)	95.2%	94.2%
Retrieval recall@10 (seen / unseen)	.974 / .768	.969 / .766
Linked to gold, seen concepts	.888	.858
Linked to gold, unseen concepts	.678	.685
Non-gold spans correctly rejected	91.0%	91.6%
Macro IoU, as is	.532	.530

Table 5: Stage-wise analysis of the six-member system (174 training notes), with oracles applied cumulatively.

Idea	Outcome
sap4 : our own SapBERT retrained on UMLS 2025AB with harder negatives, then mention-tuned on gold spans	UMLS held-out acc@1 69.2% $\rightarrow$ 80.8%, but almost all of the gain is outside SNOMED, and harder negatives lowered unseen-concept recall. Mention tuning gave the best retriever measured (R@10 .955 $\rightarrow$ .962; unseen .738 $\rightarrow$ .785), yet only +.001 / +.002 in the ensemble.
SNOMED is-a hierarchy encoder (HiT [ 21 ] ) as retriever	Worse than SapBERT: R@10 .950 vs .955, unseen .716 vs .738.
Hierarchy-aware SNOMED pre-training loss (soft targets, relation margins); pre-training on every in-scope concept	Within seed noise; unseen accuracy unchanged.
KRISSBERT [ 22 ] : contextual retrieval as a second retriever, with source-agreement features	Recovers 366 of 1,058 SapBERT misses (only 85 unseen); +.005 per three-model ensemble, $\leq$ 0 in the six-model ensemble.
Hierarchy near-miss resolution (parent/child pooling, specificity preference)	Oracle +.085; every rule $\leq$ +.0005, as annotators split a phrase between neighbouring concepts.

Table 6: A selection of negative results (paired

\Delta

macro, dev / test).

Biomedical Entity Linking

See all 11 papers

SNOMED CT Concept Recommendation from Masked Clinical Context
Sep 15, 2026Ali NooriAutomated Medical Coding Clinical Information Extraction
Informatics and Analytics, University of North Carolina Greensboro Greensboro, North Carolina, USA
Neighborhood-Aware Dual Biomedical Entity Linking
Aug 4, 2026Yicheng Tao, Jie LiuLearning to Rank Biomedical Entity Linking
Department of Electrical Engineering and Computer Science, University of Michigan · Gilbert S. Omenn Department of Computational Medicine and Bioinformatics, University of Michigan
LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking
May 13, 2026Adam Remaki, Xavier Tannier, Christel GérardinBiomedical Entity Linking Long-Context Modeling
Sorbonne Université, Inserm, Université Sorbonne Paris Nord, Limics, 75006 Paris, France · Service de médecine interne, Hôpital Tenon, Assistance Publique - Hôpitaux de Paris, Paris, France

Sep 15, 2026J/K move · Enter open · S save

SNOMED CT Concept Recommendation from Masked Clinical Context

Ali Noori

Informatics and Analytics, University of North Carolina Greensboro Greensboro, North Carolina, USA

Automated Medical Coding Clinical Information Extraction Information Retrieval

Open paper AskPDF arXiv

Standardizing clinical language to SNOMED CT supports interoperability, analytics, and reusable phenotyping, but concept recommendation remains difficult when relevant concepts are rare or absent from training data. We present a masked-concept recommendation benchmark using the SNOMED CT Entity Linking Challenge v1.2.1 data derived from MIMIC-IV-Note. The dataset contains 75,491 annotations across 272 discharge summaries, with 204 notes used for training and 68 for historical testing. For each unique note-concept pair, the target mention is masked from a local clinical context and the system ranks SNOMED CT concepts observed during training. We compare a popularity baseline, sparse TF-IDF concept prototypes, dense latent semantic analysis embeddings, sparse-dense fusion, retrieved-note evidence, and a retrieval-augmented hybrid. Sparse TF-IDF performs best, achieving Recall@1 of 14.81%, Recall@10 of 33.43%, MRR of 0.2114, and nDCG@10 of 0.2297. Retrieval augmentation does not improve this baseline, with Recall@10 of 31.99% and MRR of 0.1937. Performance is strongly affected by concept frequency: Recall@10 is 7.74% for concepts appearing in only one or two training notes versus 43.90% for concepts appearing in more than ten. In addition, 9.66% of test note-concept pairs contain concepts unseen during training. These findings show that local lexical context and terminology coverage are major determinants of recommendation quality in low-resource settings and provide a reproducible baseline for future ontology-grounded and biomedical-encoder retrieval systems.

Abstract

Biomedical Entity Linking

Explore similar work

SNOMED CT Concept Recommendation from Masked Clinical Context

Neighborhood-Aware Dual Biomedical Entity Linking

LongBEL: Long-Context and Document-Consistent Biomedical Entity Linking