Transformer-Based Multi-Region Segmentation and Radiomic Analysis of HR-pQCT Imaging for Osteoporosis Classification

Mohseu Rashid Subah; Mohammed Abdul Gani Zilani; Thomas L. Nickolas; Matthew R. Allen; Stuart J. Warden; Rachel K. Surowiec

cs.CVMar 10, 2026

Transformer-Based Multi-Region Segmentation and Radiomic Analysis of HR-pQCT Imaging for Osteoporosis Classification

Authors: Mohseu Rashid Subah, Mohammed Abdul Gani Zilani, Thomas L. Nickolas, Matthew R. Allen, Stuart J. Warden, Rachel K. Surowiec

Organizations: Weldon School of Biomedical Engineering, Purdue University, West Lafayette, IN, USA · Division of Bone and Mineral Diseases, Washington University Medicine, St. Louis, MO, USA · Department of Anatomy, Cell Biology & Physiology, Indiana University School of Medicine, Indianapolis, IN, USA · Department of Physical Therapy, School of Health & Human Sciences, Indiana University Indianapolis, Indianapolis, IN, USA

Abstract

Osteoporosis is a skeletal disease typically diagnosed using dual-energy X-ray absorptiometry (DXA), which quantifies areal bone mineral density but overlooks bone microarchitecture and surrounding soft tissues. High-resolution peripheral quantitative computed tomography (HR-pQCT) enables three-dimensional microstructural imaging with minimal radiation. However, current analysis pipelines largely focus on mineralized bone compartments, leaving much of the acquired image data underutilized. We introduce a fully automated framework for binary osteoporosis classification using radiomics features extracted from anatomically segmented HR-pQCT images. To our knowledge, this work is the first to leverage a transformer-based segmentation architecture, i.e., the SegFormer, for fully automated multi-region HR-pQCT analysis. The SegFormer model simultaneously delineated the cortical and trabecular bone of the tibia and fibula along with surrounding soft tissues and achieved a mean F1 score of 95.36%. Soft tissues were further subdivided into skin, myotendinous, and adipose regions through post-processing. From each region, 939 radiomic features were extracted and dimensionally reduced to train six machine learning classifiers on an independent dataset comprising 20,496 images from 122 HR-pQCT scans. The best image level performance was achieved using myotendinous tissue features, yielding an accuracy of 80.08% and an area under the receiver operating characteristic curve (AUROC) of 0.85, outperforming bone-based models. At the patient level, replacing standard biological, DXA, and HR-pQCT parameters with soft tissue radiomics improved AUROC from 0.792 to 0.875. These findings demonstrate that automated, multi-region HR-pQCT segmentation enables the extraction of clinically informative signals beyond bone alone, highlighting the importance of integrated tissue assessment for osteoporosis detection.

Figures & tables

Fig. 1: Input HR-pQCT slice to the SegFormer model (left), five region output of the model highlighting the overall soft tissues in green (middle), and refined output with seven distinct regions after soft tissue segmentation (right). The abbreviations indicated in parentheses are used throughout the text.

Fig. 2: Schematic of the proposed SegFormer-based segmentation pipeline. Standardized HR-pQCT inputs are processed through the encoder-decoder framework to generate a five-class semantic map, followed by post-processing and soft-tissue segmentation to obtain the final seven-class output. For each of the four transformer blocks in the encoder, M denotes the number of repeated efficient self-attention and mix feed-forward network units, with M = 3, 4, 18, and 3. In the decoder, H, W, and C denote the height, width, and channel dimension of the input feature maps.

Fig. 3: Overview of the proposed osteoporosis classification framework. (a) From each of the seven segmented regions except skin, 939 radiomics features are extracted and reduced through feature selection. (b) Region-specific models are trained using six machine learning classifiers with five-fold cross-validation on 16,464 images and tested on 4,032 held-out images. (c) Different features are grouped into three subsets: non-radiomics, radiomics tibia, and radiomics soft-tissue. Selected features from the subsets are used to construct patient level models (98 training, 24 testing subjects) for binary osteoporosis prediction.

Fig. 4: Qualitative semantic segmentation performance on a sample HR-pQCT image (7.3% distal tibia region). The top row shows the full HR-pQCT image encompassing all five regions, and the bottom row highlights the fibula. U-Net–based architectures demonstrate noticeable pixel misclassification, which is substantially reduced with the SegFormer model (highlighted in red). The remaining minor inconsistencies are addressed through post-processing.

Model	Anatomical Structure	Precision	Recall	F1 Score	IoU
U-Net	Soft Tissues	98.4 $\pm$ 7.3	99.6 $\pm$ 1.7	98.8 $\pm$ 5.5	98.0 $\pm$ 7.5
	Tibia Cortical	93.2 $\pm$ 5.3	91.7 $\pm$ 3.6	92.4 $\pm$ 3.9	86.1 $\pm$ 6.4
	Tibia Trabecular	98.8 $\pm$ 1.0	99.4 $\pm$ 2.0	99.1 $\pm$ 1.6	98.2 $\pm$ 2.1
	Fibula Cortical	83.8 $\pm$ 16.3	89.2 $\pm$ 11.9	85.5 $\pm$ 12.9	76.7 $\pm$ 18.1
	Fibula Trabecular	97.0 $\pm$ 4.0	77.2 $\pm$ 25.5	82.9 $\pm$ 18.2	74.4 $\pm$ 23.4
Attention U-Net	Soft Tissues	98.6 $\pm$ 5.8	99.6 $\pm$ 2.3	98.9 $\pm$ 4.6	98.2 $\pm$ 6.2

TABLE I: Segmentation performance of different models across anatomical structures (mean

\pm

SD). Bold values indicate the best performance per structure.

Fig. 5: Retained features after the feature selection process. (a) Features grouped by filter classes. (b) Features grouped by feature classes.

Fig. 6: Correlation heatmap of features extracted from the myotendinous tissue region. The mean absolute correlation coefficient of the selected features was 0.358.

Anatomical Structure	Model a	Acc.	Sen.	Spe.	F1 Score	AUROC
Tibia Cortical	LR	76.69	64.34	89.04	0.734	0.777
	SVM	73.59	57.44	89.73	0.685	0.766
	RF	68.53	69.05	68.01	0.687	0.777
	XGBoost	73.64	62.45	84.82	0.703	0.777
	KNN	69.25	50.35	88.14	0.621	0.769
	NB	77.48	67.36	87.60	0.749	0.792

TABLE II: Classification performance of different models across anatomical structures on the held-out test set. Bold values indicate the best performance per structure.

Fig. 7: Classification performance of the Logistic Regression model using features extracted from different anatomical regions.

Anatomical Structure	Model	Acc.	Sen.	Spe.	F1 Score	AUROC
Soft Tissue (10 mm)	LR	77.60	60.96	94.25	0.731	0.844
	SVM	77.78	61.21	94.35	0.734	0.835
	RF	77.65	87.65	67.66	0.797	0.850
	XGBoost	78.17	83.58	72.77	0.793	0.875
	KNN	71.65	59.77	83.53	0.678	0.793
	NB	74.36	59.57	89.14	0.699	0.809

TABLE III: Classification performance of different models across soft tissue regions at varying radial distances from the tibia on the held-out test set. Bold values indicate the best performance per region.

Model	Feature Name a	$\boldsymbol{\beta}$	OR (95 $\%$ CI)	p Value	VIF
Non- Radiomics	Whole Body Total % Fat Mass	-0.744	0.475 (0.257, 0.822)	0.011 *	1.236
	TbN	-0.761	0.467 (0.229, 0.873)	0.024 *	1.135
	CtvBMD	-1.082	0.339 (0.151, 0.684)	0.004 *	1.872
	Stiffness	-0.895	0.409 (0.176, 0.868)	0.026 *	1.393
	(Tb.F/TF)prox	-0.670	0.512 (0.242, 1.000)	0.061	1.881
Radiomics Tibia	TC Original GLCM Autocorrelation	-1.004	0.366 (0.170, 0.720)	0.006 *	1.559

TABLE IV: Results of the logistic regression models, including regression slope coefficients (

\beta

), odds ratios (OR), 95

\%

confidence intervals (CI), variance inflation factors (VIF), and corresponding p-values for the selected variables.

Model	Acc.	Sen.	Spe.	F1 Score	AUROC
Non-Radiomics	0.792	0.667	0.917	0.762	0.792
Radiomics Tibia	0.792	0.833	0.750	0.800	0.826
Radiomics Soft Tissue	0.875	0.917	0.833	0.880	0.875

TABLE V: Classification performance of different models at the patient level on the held-out test set. Bold values indicate the best performance across models.

Radiomics

Radiomics quantifies medical-image shape, intensity, and texture to develop biomarkers for diagnosis, prognosis, treatment response, and biological characterization.

See all 50 papers

Opportunistic Bone-Loss Screening from Routine Knee Radiographs Using a Multi-Task Deep Learning Framework with Sensitivity-Constrained Threshold Optimization
Apr 22, 2026Zhaochen Li, Xinghao Yan, Runni Zhou +8Medical Imaging Medical Diagnosis
ORBIT Lab, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, Liaoning, China. · Department of Radiology, Liaoning Provincial Key Laboratory of Medical Imaging, Liaoning Provincial Key Laboratory of Imaging Technology and Artificial Intelligence, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China. · Rehabilitation Center, Liaoning Provincial Key Laboratory of Medical Imaging, Liaoning Provincial Key Laboratory of Imaging Technology and Artificial Intelligence, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China. +1
LegSegNet: A Public Deep Learning System for Lower Extremity CT Tissue Segmentation and Quantification
May 29, 2026Yuwen Chen, Yaqian Chen, Roy Colglazier +4Medical Image Analysis Medical Image Segmentation
Department of Electrical and Computer Engineering, Duke University, Durham, NC 27708 · Department of Radiology, Duke University, Durham, NC 27708 · Department of Biostatistics & Bioinformatics, Duke University, Durham, NC 27708 +2
CT-IDP: Segmentation-Derived Quantitative Phenotypes for Interpretable Abdominal CT Disease Classification
May 9, 2026Lavsen Dahal, Joseph Y. LoMedical Image Classification Explainable Medical Image Analysis
1. Center for Virtual Imaging Trials, Carl E. Ravin Advanced Imaging Laboratories · 3. Electrical and Computer Engineering, Pratt School of Engineering, Duke University · 4. Medical Physics Graduate Program, Duke University +1

Apr 22, 2026J/K move · Enter open · S save

Opportunistic Bone-Loss Screening from Routine Knee Radiographs Using a Multi-Task Deep Learning Framework with Sensitivity-Constrained Threshold Optimization

Zhaochen Li, Xinghao Yan, Runni Zhou, Xiaoyang Li+7

ORBIT Lab, College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, Liaoning, China. · Department of Radiology, Liaoning Provincial Key Laboratory of Medical Imaging, Liaoning Provincial Key Laboratory of Imaging Technology and Artificial Intelligence, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China. · Rehabilitation Center, Liaoning Provincial Key Laboratory of Medical Imaging, Liaoning Provincial Key Laboratory of Imaging Technology and Artificial Intelligence, Shengjing Hospital of China Medical University, Shenyang, Liaoning, China.+1

Medical Imaging Medical Diagnosis Multi-Task Learning Medical Image Classification

Open paper AskPDF arXiv

Background: Osteoporosis and osteopenia are often undiagnosed until fragility fractures occur. Dual-energy X-ray absorptiometry (DXA) is the reference standard for bone mineral density (BMD) assessment, but access remains limited. Knee radiographs are obtained at high volume for osteoarthritis evaluation and may offer an opportunity for opportunistic bone-loss screening. Objective: To develop and evaluate a multi-task deep learning system for opportunistic bone-loss screening from routine knee radiographs without additional imaging or patient visits. Methods: We developed STR-Net, a multi-task framework for single-channel grayscale knee radiographs. The model includes a shared backbone, global average pooling feature aggregation, a shared neck, and a task-aware representation routing module connected to three task-specific heads: binary screening (Normal vs. Bone Loss), severity sub-classification (Osteopenia vs. Osteoporosis), and weakly coupled T-score regression with optional clinical variables. A sensitivity-constrained threshold optimization strategy (minimum sensitivity >= 0.86) was applied. The dataset included 1,570 knee radiographs, split at the patient level into training (n=1,120), validation (n=226), and test (n=224) sets. Results: On the held-out test set, STR-Net achieved an AUROC of 0.933, sensitivity of 0.904, specificity of 0.773, and AUPRC of 0.956 for binary screening. Severity sub-classification achieved an AUROC of 0.898. The T-score regression branch showed a Pearson correlation of 0.801 with DXA-measured T-scores in a pilot subset (n=31), with MAE of 0.279 and RMSE of 0.347. Conclusions: STR-Net enables single-pass bone-loss screening, severity stratification, and quantitative T-score estimation from routine knee radiographs. Prospective clinical validation is needed before deployment.

Abstract

Radiomics

Explore similar work

Opportunistic Bone-Loss Screening from Routine Knee Radiographs Using a Multi-Task Deep Learning Framework with Sensitivity-Constrained Threshold Optimization

LegSegNet: A Public Deep Learning System for Lower Extremity CT Tissue Segmentation and Quantification

CT-IDP: Segmentation-Derived Quantitative Phenotypes for Interpretable Abdominal CT Disease Classification