CLiGNet: Clinical Label-Interaction Graph Network for Medical Specialty Classification from Clinical Transcriptions

Pronob Kumar Barman; Pronoy Kumar Barman

cs.AIMar 24, 2026

CLiGNet: Clinical Label-Interaction Graph Network for Medical Specialty Classification from Clinical Transcriptions

Authors: Pronob Kumar Barman, Pronoy Kumar Barman

Organizations: University of Maryland, Baltimore County, Department of Information Systems, Baltimore, MD, USA · Jagannath University, Department of Statistics, Dhaka, Bangladesh

Abstract

Automated classification of clinical transcriptions into medical specialties is essential for routing, coding, and clinical decision support, yet prior work on the widely used MTSamples benchmark suffers from severe data leakage caused by applying SMOTE oversampling before train test splitting. We first document this methodological flaw and establish a leakage free benchmark across 40 medical specialties (4966 records), revealing that the true task difficulty is substantially higher than previously reported. We then introduce CLiGNet (Clinical Label Interaction Graph Network), a neural architecture that combines a Bio ClinicalBERT text encoder with a two layer Graph Convolutional Network operating on a specialty label graph constructed from semantic similarity and ICD 10 chapter priors. Per label attention gates fuse document and label graph representations, trained with focal binary cross entropy loss to handle extreme class imbalance (181 to 1 ratio). Across seven baselines ranging from TF IDF classifiers to Clinical Longformer, CLiGNet without calibration achieves the highest macro F1 of 0.279, with an ablation study confirming that the GCN label graph provides the single largest component gain (increase of 0.066 macro F1). Adding per label Platt scaling calibration yields an expected calibration error of 0.007, demonstrating a principled trade off between ranking performance and probability reliability. We provide comprehensive failure analysis covering pairwise specialty confusions, rare class behaviour, document length effects, and token level Integrated Gradients attribution, offering actionable insights for clinical NLP system deployment.

Figures & tables

Statistic	Value	Note
Raw records	4,999
After cleaning	4,966	33 empty removed
Medical specialties	40	Single-label
Training set	3,476 (70%)	Stratified
Validation set	745 (15%)	Stratified
Test set	745 (15%)	Stratified

Table 1. MTSamples dataset summary after preprocessing.

Model	Ma-F1 $\uparrow$	Mi-F1 $\uparrow$	Acc $\uparrow$	Ham $\downarrow$
B1: TF-IDF + LR	0.173	0.105	0.105	0.045
B2: TF-IDF + SVC	0.099	0.152	0.152	0.042
B3: ClinBERT + OvR	0.211	0.344	0.344	0.033
B4: BioBERT + OvR	0.172	0.318	0.318	0.034
B7: Longformer + BR	0.262	0.336	0.336	0.033
B6: CLiGNet (no cal.)	0.279	0.362	0.362	0.032

Table 2. Test-set performance across all models. Best result per metric is bolded . B6 achieves the highest macro-F1; B8 additionally provides calibrated probabilities (ECE = 0.007).

Baseline	$n_{01}$	$n_{10}$	$\chi^{2}$	$p$	Sig.
B1: TF-IDF + LR	42	216	116.0	$<$ 0.001
B2: TF-IDF + SVC	54	193	77.1	$<$ 0.001
B3: ClinBERT	120	116	0.04	0.845	ns
B4: BioBERT	100	115	0.91	0.340	ns
B6: no calib.	130	112	1.19	0.274	ns
B7: Longformer	123	125	0.00	0.949	ns

Table 3. McNemar’s test results: B8 vs. each baseline. Bonferroni-corrected

\alpha=0.0083

n_{01}

: samples correct only in baseline;

n_{10}

: correct only in B8.

ID	Removed Component	Ma-F1	Mi-F1	$\Delta$ Ma-F1
B6	(Full, no calibration)	0.279	0.362	—
A1	GCN label graph	0.214	0.336	$-$ 0.066
A2	Focal loss	0.266	0.366	$-$ 0.013
A4	Sliding window	0.271	0.379	$-$ 0.008
A5	Per-label thresholds	0.270	0.338	$-$ 0.009

Table 4. Ablation study: macro-F1 and micro-F1 on the test set. Each variant removes one component from B6 (macro-F1 = 0.279).

True Label	Predicted	Count	% of True
General Medicine	Consult - Hist. & Phy.	26	66.7
Surgery	Gastroenterology	24	14.7
Surgery	Orthopedic	23	14.1
Surgery	Cardiovascular/Pulm.	21	12.9
Urology	Surgery	16	69.6

Table 5. Top-5 confused specialty pairs for B8 on the test set.

Training Samples	Mean F1	Num. Classes
0–20	0.271	16
20–50	0.316	6
50–100	0.447	7
100–500	0.410	8
500+	0.596	1 (Surgery)

Table 6. Mean per-label F1 by training set class size bin.

Word Count	Accuracy
0–200	0.298
200–400	0.338
400–600	0.371
600–1,000	0.353
1,000+	0.286

Table 7. B8 accuracy by document word-count bin.

Explore similar work

Clinical Domain Classification from Medical Transcriptions
Sep 19, 2026Sravani Pottipati, Lakshmikar R. PolamreddyClassification Clinical NLP
Independent Researcher, New York City, USA · Yeshiva University, New York City, USA
L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification
Apr 14, 2026Rishik Kondadadi, John E. OrtegaLearning to Defer Text Classification
University of Minnesota, Northeastern University, Minneapolis, Boston, USA
Clinical named entity recognition in the Portuguese language: a benchmark of modern BERT models and LLMs
Mar 27, 2026Vinicius Anjos de Almeida, Sandro Saorin da Silva, Josimar Chire +8Named Entity Recognition Clinical NLP
Spesia, Curitiba - PR, Brazil · Faculdade de Medicina, Universidade de São Paulo, São Paulo - SP, Brazil · Pontifícia Universidade Católica do Paraná (PUCPR), Curitiba - PR, Brazil +2

Sep 19, 2026J/K move · Enter open · S save

Clinical Domain Classification from Medical Transcriptions

Sravani Pottipati, Lakshmikar R. Polamreddy

Independent Researcher, New York City, USA · Yeshiva University, New York City, USA

Classification Clinical NLP Class-Imbalanced Learning

Open paper AskPDF arXiv

Clinical domain classification plays an important role in organizing and analyzing large volumes of unstructured medical text. However, medical transcription datasets are often highly imbalanced, which can substantially degrade classification performance, particularly for underrepresented clinical specialties. In this work, we present a comparative study of machine learning and transformer-based approaches for clinical domain classification from medical transcriptions. We evaluate six traditional machine learning classifiers---Naive Bayes, Support Vector Machine (SVM), Decision Tree, Random Forest, K-Nearest Neighbors (KNN), and XGBoost---along with two pretrained transformer models, BERT and XLNet, and a few-shot large language model prompting approach. Experiments are conducted on medical transcription data collected from MTSamples, comprising 5,013 samples across 40 clinical specialties. To address severe class imbalance, we investigate two balancing strategies: text augmentation using NLP-based synonym replacement and Synthetic Minority Over-sampling Technique (SMOTE). Experimental results demonstrate that data balancing substantially improves classification performance across the evaluated models. In particular, BERT achieves the highest F1-score of 0.996 on the SMOTE-balanced dataset while requiring lower training time than XLNet. The results highlight the effectiveness of transformer-based representations combined with appropriate data balancing strategies for clinical domain classification and provide a systematic comparison of classical machine learning, transformer models, and few-shot prompting for medical transcription analysis.

Abstract

Explore similar work

Clinical Domain Classification from Medical Transcriptions

L2D-Clinical: Learning to Defer for Adaptive Model Selection in Clinical Text Classification

Clinical named entity recognition in the Portuguese language: a benchmark of modern BERT models and LLMs