ThyCLIPNet: A BiomedCLIP-Guided Lightweight Attention-Enhanced DeepLabV3+ Framework for Robust Thyroid Nodule Segmentation

Tasnim Jahan; Md Easin Arafat; Swakkhar Shatabda

cs.CVOct 3, 2026

ThyCLIPNet: A BiomedCLIP-Guided Lightweight Attention-Enhanced DeepLabV3+ Framework for Robust Thyroid Nodule Segmentation

Authors: Tasnim Jahan, Md Easin Arafat, Swakkhar Shatabda

Organizations: Department of Computer Science and Engineering, United International University, United City, Madani Ave, Dhaka 1212, Bangladesh · Faculty of Informatics, Institute of Industry-Academia Innovation, Department of Data Science and Engineering, Eötvös Lor´and University, P´azm´any P´eter S´et´any 1/C, Budapest 1117, Hungary · Department of Computer Science and Engineering, BRAC University, Pragati Sarani, Merul Badda, Dhaka 1212, Bangladesh

Abstract

Accurate thyroid ultrasound segmentation is often challenged by low contrast, speckle noise, and unclear boundaries. Although recent methods have improved segmentation accuracy, many rely on resource-intensive architectures or lack explicit integration of multiscale features with global biomedical visual guidance. In this paper, we introduce ThyCLIPNet, a lightweight semantic-guided hybrid encoder-decoder framework that integrates BiomedCLIP-derived biomedical semantic guidance into a lightweight multi-scale CNN segmentation pipeline. The encoder integrates MobileNetV2 with efficient channel attention, while atrous spatial pyramid pooling and a custom convolutional block attention module enrich bottleneck features. The decoder combines hierarchical skip connections and lightweight attention refinement with a BiomedCLIP-guided gated fusion pathway that projects vision-only global biomedical embeddings into decoder feature space and selectively integrates them through semantic-local fusion and spatial gating. To the best of our knowledge, ThyCLIPNet is among the first lightweight thyroid ultrasound segmentation frameworks to use BiomedCLIP's vision encoder alone for image-only global semantic guidance without text prompting. Experiments on TG3K, TN3K, DDTI, and PKTN achieve dice similarity coefficients of 96.22%, 87.58%, 84.73%, and 80.70%; intersection over union scores of 92.72%, 77.91%, 73.51%, and 67.64%; and 95th-percentile hausdorff distances of 3.75, 16.38, 18.23, and 10.86, respectively. ThyCLIPNet uses 8.55M parameters and 22.99G FLOPs. Overall, the results support integrating global biomedical semantic guidance with lightweight multi-scale CNN representations for robust and computationally efficient thyroid ultrasound segmentation. Source code: https://github.com/Tasnim-Jahan/ThyCLIPNet. [Abstract shortened for arXiv. See PDF for full abstract.]

Figures & tables

Dataset	Total Images (Original)	Train Images (Original)	Validation Images	Test Images	Augmentation Factor (k)	Augmented Train Images	Final Train Images	Total Images (After Augmentation)	Ultrasound Devices
TG3K [ 62 ]	3,585	2,868	358	359	1	2,868	5,736	6,453	GE Logiq E9
TN3K [ 63 ]	3,493	2,304	575	614	1	2,304	4,608	5,797	GE Logiq E9; ARIETTA 850; RESONA 70B
DDTI [ 64 ]	637	509	63	65	5	2,545	3,054	3,182	Toshiba Nemio 30; Toshiba Nemio MX
PKTN [ 54 ]	1,005	804	100	101	4	3,216	4,020	4,221	Not disclosed

Table 1: Key characteristics and dataset splits for model evaluation.

Figure 1: Illustration of the preprocessing and data‐augmentation pipeline: ultrasound images and masks are resized to 256 $\times$ 256 and normalized; dataset-specific offline augmentations are applied; 512-dimensional BiomedCLIP embeddings are extracted from original and augmented images; dynamic online augmentations are performed on both images and embeddings; and the resulting data are fed into the segmentation model for training.

Figure 2: Architecture of the proposed ThyCLIPNet framework for thyroid ultrasound segmentation, illustrating: (a) MobileNetV2 encoder integrated with ECA modules; (b) multi-scale context capture using an ASPP block fused with a custom CBAM in the bottleneck; (c) global semantic guidance from BiomedCLIP image embeddings integrated through the BiomedCLIP-Guided Gated Fusion (BGF); and (d) three-stage U-Net–style decoder with skip connections and lightweight CBAM for progressive boundary refinement.

Figure 3: Feature bottleneck module: the ECA-refined encoder output

F^{\prime}_{4}

is processed by ASPP block with SpatialDropout to yield

F_{\mathrm{ASPP}}

, which is then refined by custom CBAM to produce

F_{\mathrm{CustomCBAM}}

before decoding.

Figure 4: Lightweight CBAM attention block: decoder feature maps first pass through channel attention (global average pooling

\rightarrow

dense

\rightarrow

relu

\rightarrow

dense

\rightarrow

sigmoid) and then spatial attention (pooled features concatenated

\rightarrow

7\times 7

convolution

\rightarrow

sigmoid) to selectively refine and enhance informative regions before further decoding.

Figure 5: BiomedCLIP-guided Gated Fusion (BGF) module: global biomedical image embeddings are projected, reshaped, and bilinearly upsampled, then combined with decoder features using weighted addition and refined with spatial attention gating before concatenation.

Dataset	Methods	DICE (%) $\uparrow$	IoU (%) $\uparrow$	Accuracy (%) $\uparrow$	Recall (%) $\uparrow$	Precision (%) $\uparrow$	HD95 $\downarrow$
TG3K [ 62 ]	Coarse for Fine [ 73 ]	80.67 $\pm$ 1.09	—	—	—	—	16.63 $\pm$ 0.84
	Weakly‐supervised Framework [ 74 ]	85.92 $\pm$ 1.53	—	93.00	86.00	91.00	13.09 $\pm$ 0.43
	Adapting Vision [ 75 ]	83.04	—	—	—	—	12.55
	SAMTNU [ 83 ]	83.87	73.79	97.64	85.17	—	23.98
	DCDT [ 92 ]	82.04 $\pm$ 1.11	69.55 $\pm$ 1.85	—	—	—	26.68 $\pm$ 0.74
	FMF-Net [ 91 ]	92.20	—	—	88.90	—	0.94

Table 2: Table 2 . Comparison of segmentation effects between the proposed and SOTA methods across all datasets.

Panel of Figure 6 — Figure 6: Multi‐metric bubble‐plot comparison of ThyCLIPNet versus state‐of‐the‐art methods on three thyroid ultrasound datasets (TG3K: top row; TN3K: middle row; DDTI: bottom row). The left column shows DICE versus HD95 (bubble size $\propto 1/\mathrm{HD95}$ ); the center column shows DICE versus Accuracy (bubble size $\propto\mathrm{Accuracy}$ ); and the right column shows dataset‐specific comparisons: Accuracy versus HD95 for TG3K (bubble size $\propto 1/\mathrm{HD95}$ ), DICE versus IoU for TN3K and DDTI (bubble size $\propto\mathrm{IoU}$ ). Bubble color represents the DICE score.

Figure 7: Qualitative visualization of absolute segmentation error maps ( $|GT-\text{Prediction}|$ ) on TG3K, TN3K, DDTI, and PKTN for ThyCLIPNet and eight ablated variants. Each row shows absolute error, highlighting boundary and shape discrepancies.

Figure 8: Qualitative prediction matrix on representative test images from TG3K, TN3K, DDTI, and PKTN. Rows correspond to datasets; columns show the input ultrasound image, ground truth (GT), and predicted masks from ThyCLIPNet, BPAT-UNet, TRFE++, CLIP-TNseg, and UNet++.

Figure 9: Representative hard-case segmentation results across four thyroid ultrasound datasets (TG3K, TN3K, DDTI, PKTN). Each row depicts a difficult example characterized by low contrast, boundary ambiguity, or structural noise. Columns show (from left to right): input image with annotated contours (ground truth in orange, prediction in blue), ground truth mask, predicted mask, and absolute error overlay. DICE and HD95 are reported per case to illustrate how boundary ambiguity elevates HD95 and how errors are typically concentrated near challenging edges.

Dataset	Architecture	DICE (%) $\uparrow$	IoU (%) $\uparrow$	Accuracy (%) $\uparrow$	Recall (%) $\uparrow$	Precision (%) $\uparrow$	HD95 (px) $\downarrow$
TG3K [ 62 ]	ThyCLIPNet w/o ECA	96.08 $\pm$ 0.10	92.45 $\pm$ 0.20	99.31 $\pm$ 0.00	97.07 $\pm$ 0.20	95.11 $\pm$ 0.30	3.63 $\pm$ 0.46
	ThyCLIPNet w/o ASPP	96.12 $\pm$ 0.20	92.53 $\pm$ 0.40	99.32 $\pm$ 0.00	96.87 $\pm$ 0.50	95.38 $\pm$ 0.50	3.96 $\pm$ 0.23
	ThyCLIPNet w/o BiomedCLIP	94.21 $\pm$ 0.20	89.06 $\pm$ 0.40	98.94 $\pm$ 0.00	98.34 $\pm$ 0.40	90.43 $\pm$ 0.60	4.94 $\pm$ 0.42
	ThyCLIPNet w/o custom CBAM	95.91 $\pm$ 0.60	92.15 $\pm$ 1.10	99.28 $\pm$ 0.10	96.80 $\pm$ 0.70	95.04 $\pm$ 0.80	4.02 $\pm$ 0.19
	ThyCLIPNet w/o lightweight CBAM	96.36 $\pm$ 0.10	92.97 $\pm$ 0.20	99.36 $\pm$ 0.00	96.95 $\pm$ 0.10	95.78 $\pm$ 0.20	3.79 $\pm$ 0.23
	ThyCLIPNet w/o both CBAM variants	96.54 $\pm$ 0.20	93.32 $\pm$ 0.30	99.39 $\pm$ 0.00	97.18 $\pm$ 0.10	95.92 $\pm$ 0.20	3.75 $\pm$ 0.13

Table 3: Table 3 . Ablation‐study results on four datasets (TG3K, TN3K, DDTI, and PKTN). Metrics are mean

\pm

standard deviation over five independent runs. DICE, IoU, accuracy, recall, precision and HD95 are shown.

Variants	Params	$\Delta$ Params (%)	Train Time (s)	$\Delta$ Train (%)	Inf Time (s)	$\Delta$ Inf (%)
w/o ECA	8,550,704	0.00	16,070.82	+5.93	0.01305	+2.84
w/o BiomedCLIP	7,821,736	-8.53	8,479.56	-44.11	0.01148	-9.54
w/o lightweight CBAM	8,312,114	-2.79	15,083.26	-0.58	0.01150	-9.38
w/o both CBAM variants	8,270,353	-3.28	20,043.51	+32.12	0.01267	-0.16
w/o (BiomedCLIP+ECA)	7,821,720	-8.53	18,130.88	+19.51	0.01874	+47.68
w/o (BiomedCLIP+ASPP)	4,543,656	-46.86	14,859.38	-2.05	0.00879	-30.73

Table 4: Table 4 . Efficiency comparison on TG3K. Values are reported as the mean over five independent runs.

\Delta

values are relative to the proposed ThyCLIPNet model.

Dataset	Architecture	DICE (%)	IoU (%)	Accuracy (%)	Recall (%)	Precision (%)	HD95 (px)
TG3K	ThyCLIPNet+TTA	96.43 $\pm$ 0.10	93.10 $\pm$ 0.10	99.37 $\pm$ 0.00	97.03 $\pm$ 0.20	95.83 $\pm$ 0.20	3.69 $\pm$ 0.16
TN3K	ThyCLIPNet+TTA	87.92 $\pm$ 0.20	78.45 $\pm$ 0.30	97.03 $\pm$ 0.10	89.49 $\pm$ 0.70	86.42 $\pm$ 0.80	16.09 $\pm$ 0.88
DDTI	ThyCLIPNet+TTA	85.05 $\pm$ 0.50	73.99 $\pm$ 0.80	95.35 $\pm$ 0.20	89.72 $\pm$ 1.40	80.88 $\pm$ 1.80	18.41 $\pm$ 1.28
PKTN	ThyCLIPNet+TTA	80.77 $\pm$ 0.30	67.75 $\pm$ 0.40	98.68 $\pm$ 0.00	80.45 $\pm$ 1.80	81.17 $\pm$ 2.00	10.91 $\pm$ 0.51

Table 5: Table 5 . Performance of ThyCLIPNet + TTA across four datasets metrics as mean

\pm

std.

Model	Params (M)	Inference Time (s/img)	FPS	FLOPs (G)	GPU Mem (GB)
UNet++ [ 95 ]	9.16	0.01359	73.58	69.43	1.81
TRFE+ [ 77 ]	45.44	0.03834	26.34	160.77	0.76
BPAT-UNet [ 81 ]	72.39	0.07648	13.08	128.66	0.55
DeepLabV3+ with MobileNetV2 [ 65 ]	5.85	0.01223	81.77	12.37	0.77
U-Net with MobileNetV2 [ 28 ]	3.75	0.00698	143.27	15.85	0.52
ThyCLIPNet-Proposed	8.55	0.01269	78.81	22.99	0.76

Table 6: Table 6 . Computational benchmark comparison of ThyCLIPNet with representative baselines. Representative lightweight models were re-implemented under identical evaluation settings (input size = 256 × 256, batch size = 1), while remaining architectures were benchmarked under the same hardware and inference environment. FLOPs and parameter counts are architecture-dependent and dataset-independent. Inference times represent median (P50) values over five independent runs; for ThyCLIPNet, computational measurements correspond to the segmentation stage using precomputed BiomedCLIP embeddings. FPS is computed as

1/\text{time}

Model	Params (M)	Inference Time (s/img)	FPS	FLOPs (G)	GPU Mem (GB)
GDSSA-Net [ 53 ]	35.00	0.042	—	—	—
MSAC-DD [ 52 ]	25.70	0.233	—	—	2.55
FCG-Net [ 90 ]	11.35	—	20.25	—	—
FDE-Net [ 97 ]	—	0.036	—	—	—
DualSwinUNet++ [ 98 ]	—	0.180	—	—	—
TNSNet [ 96 ]	46.47	0.080	12.5	—	25.42

Table 7: Table 7 . Comparison with previously reported thyroid segmentation networks. Values are primarily extracted from the corresponding original publications, whereas ThyCLIPNet was benchmarked under the experimental protocol adopted in this study (input size = 256 × 256, batch size = 1).

Figure 10: Six‐panel Grad-CAM interpretability visualizations for two representative cases from each thyroid ultrasound dataset (TG3K: top two rows; TN3K: next two rows; DDTI: next two rows; PKTN: bottom two rows). From left to right: (i) input image, (ii) ground-truth mask, (iii) predicted mask, (iv) Grad-CAM heatmap, (v) overlay of prediction, and (vi) overlay of Grad-CAM heatmap.

Explore similar work

Thyroid Nodule Segmentation in Ultrasound Images with Conformal Uncertainty Estimation
Oct 9, 2026Francesco Teodori, Pierpaolo Brutti, Umberto Ferraro PetrilloUltrasound Image Segmentation Medical Image Segmentation
Department of Statistical Sciences, Sapienza University of Rome, Rome, Italy.
MedSaab-US: A Backpropagation-Free Multi-Scale Wavelet-Saab Framework for Thyroid Nodule Segmentation in Ultrasound Images
Jul 2, 2026Mohammad Amanour RahmanUltrasound Image Segmentation Medical Image Segmentation
Department of Computer Science and Engineering Ahsanullah University of Science and Technology (AUST) Dhaka, Bangladesh
Prototype-Enhanced Multi-View Learning for Thyroid Nodule Ultrasound Classification
Mar 30, 2026Yangmei Chen, Zhongyuan Zhang, Xikun Zhang +4Medical Image Classification Cross-Domain Generalization
Guangdong Institute of Smart Education, Jinan University, Guangzhou 510632, China

Oct 9, 2026J/K move · Enter open · S save

Thyroid Nodule Segmentation in Ultrasound Images with Conformal Uncertainty Estimation

Francesco Teodori, Pierpaolo Brutti, Umberto Ferraro Petrillo

Department of Statistical Sciences, Sapienza University of Rome, Rome, Italy.

Ultrasound Image Segmentation Medical Image Segmentation Segmentation Uncertainty Estimation

Open paper AskPDF arXiv

Thyroid nodules affect a large proportion of the general population. Although most are benign, a non-negligible fraction is malignant. Ultrasound imaging is the primary diagnostic modality, and accurate nodule segmentation is a prerequisite for CAD systems to classify malignancy and assess metastatic risk.\ In this work, we propose a thyroid nodule segmentation framework that integrates a DeepLabV3+, channel-attention modules and post-hoc conformal prediction, providing both accurate mask and pixel-level uncertainty estimates. We address thyroid ultrasound nodule segmentation as the first stage of a pipeline that also includes malignancy classification and metastatic risk prediction. We adopt DeepLabV3+, whose Atrous Spatial Pyramid Pooling (ASPP) captures multi-scale context, with an EfficientNet-B5 backbone for its accuracy-efficiency trade-off. Additionally, we test three channel-attention modules: Squeeze-and-Excitation, CBAM and Grouped Channel Attention (GCA), paired with two loss formulations (Dice-BCE and Boundary).\ Since binary masks do not indicate prediction confidence, we wrap the trained model with a split conformal prediction layer that produces pixel-level uncertainty maps with statistical coverage guarantees.\ We evaluate all configurations on the TN3K public benchmark using a fixed split and directly compare them with state-of-the-art methods. The plain backbone already improves upon previously reported results, with CBAM + Dice-BCE emerging as the best overall configuration.

Abstract

Explore similar work

Thyroid Nodule Segmentation in Ultrasound Images with Conformal Uncertainty Estimation

MedSaab-US: A Backpropagation-Free Multi-Scale Wavelet-Saab Framework for Thyroid Nodule Segmentation in Ultrasound Images

Prototype-Enhanced Multi-View Learning for Thyroid Nodule Ultrasound Classification