Beyond Binary Classification: Multiclass Machine Learning for Diabetes and Prediabetes Risk Prediction Using NHANES
Authors: Aman Mujahid, Anita Penkova
Organizations: Thomas Lord Department of Computer Science, Viterbi School of Engineering, University of Southern California, Los Angeles, 90089, CA, USA · Department of Aerospace and Mechanical Engineering, Viterbi School of Engineering, University of Southern California, Los Angeles, 90089, CA, USA
Detecting diabetes risk early is important for preventing disease progression. Most existing machine learning approaches focus on binary classification, which limits their ability to capture intermediate risk states such as prediabetes. We develop a supervised learning framework to classify adults as normal, prediabetic, or diabetic using National Health and Nutrition Examination Survey data. The framework uses routinely collected demographic, clinical, and lifestyle variables from 15,356 adults, with glycated hemoglobin (HbA1c) used only to define labels and excluded from input features to prevent leakage. We compare conventional machine learning and transformer-based models and examine two binary formulations: normal versus prediabetes, and normal versus combined prediabetes and diabetes. Models are evaluated on a held-out test set using precision, sensitivity, specificity, and F1 score. In the multiclass setting, performance is broadly consistent across models, with accuracy up to 0.72 and macro-averaged F1 of 0.70. Prediabetes remains the most challenging class, with F1 scores of 0.54 to 0.56 for conventional models and 0.53 to 0.55 for transformers, reflecting its overlap with both neighboring states. When the task is reduced to normal versus prediabetes, F1 scores rise modestly to 0.62 to 0.64. A more substantial improvement is observed under a broader screening formulation combining prediabetes and diabetes, with F1 scores of 0.73 to 0.74 across all models. These findings suggest that feature overlap along the biological continuum between normal and diabetic states contributes more to the difficulty of prediabetes detection than model architecture. Multiclass prediction provides a more clinically informative representation of metabolic risk progression but highlights a trade-off between clinically meaningful stratification and predictive accuracy.
Figures & tables
Variable Type
Category
Variables
Continuous Variables
Demographics
Age
Body Measurements
Body Mass Index, Waist Circumference
Blood Pressure
Systolic Blood Pressure, Diastolic Blood Pressure
Laboratory
Fasting Glucose, Total Cholesterol, HDL, LDL, Triglycerides
Lifestyle
Sedentary Behavior
Categorical Variables
Demographics
Sex, Race or Ethnicity
Table 1: Continuous and categorical variables used in the dataset
Class
Samples
Percentage
Normal
9,249
60.2%
Prediabetes
4,279
27.9%
Diabetes
1,828
11.9%
Table 2: Class distribution
Hyperparameter
Multiclass
Binary Formulations
Tree depth
3, 4, 5
2, 3, 4
Learning rate
0.03, 0.05, 0.07
0.01, 0.03, 0.05
Number of trees
400, 600, 800
500, 700, 900
Row subsampling
0.8, 0.9, 1.0
0.7, 0.8, 0.9
Column subsampling
0.8, 0.9, 1.0
0.7, 0.8, 0.9
Minimum child weight
1, 3, 5
3, 5, 7, 10
Table 3: Hyperparameter search space for the primary XGBoost model. Dashes indicate parameters not included in the corresponding search.
Model
Feature Representation
Transformer Configuration
Training
TabTransformer
16-d categorical embeddings with continuous features
2 encoder layers, 4 heads
40 epochs, batch size 256
FT-Transformer
32-d categorical and continuous feature tokens
3 encoder layers, 4 heads
50 epochs, batch size 256
SAINT
32-d feature tokens with feature and sample attention
3 attention stages, 4 heads
50 epochs, batch size 256
Table 4: Summary of transformer model architectures and training configurations.
Model
Macro F1
95% Confidence Interval
XGBoost
0.6980 ± 0.0076
[0.6913, 0.7047]
LightGBM
0.6930 ± 0.0064
[0.6874, 0.6986]
CatBoost
0.6998 ± 0.0096
[0.6914, 0.7082]
HistGradientBoosting
0.6938 ± 0.0057
[0.6888, 0.6988]
Random Forest
0.6980 ± 0.0064
[0.6924, 0.7036]
Logistic Regression
0.6662 ± 0.0078
[0.6594, 0.6730]
Table 5: Cross-validation stability analysis using macro F1 score. Approximate 95% confidence intervals were calculated from the 5-fold cross-validation results.
Figure 1: Confusion matrices for XGBoost across the multiclass and binary prediction formulations.
Model
Accuracy
Balanced Accuracy
Macro F1
Normal Precision
Normal Recall
Normal Specificity
Normal F1
Prediabetes Precision
Prediabetes Recall
Prediabetes Specificity
Prediabetes F1
Diabetes Precision
Diabetes Recall
Diabetes Specificity
Diabetes F1
XGBoost
0.71
0.72
0.70
0.86
0.71
0.83
0.78
0.49
0.65
0.74
0.56
0.73
0.80
0.96
0.76
LightGBM
0.70
0.72
0.69
0.86
0.71
0.82
0.78
0.48
0.64
0.74
0.55
0.71
0.79
0.96
0.75
CatBoost
0.70
0.71
0.69
0.86
0.71
0.83
0.78
0.48
0.64
0.73
0.55
0.70
0.79
0.95
0.74
Ensemble
0.70
0.72
0.70
0.86
0.71
0.83
0.78
0.49
0.65
0.73
0.56
0.72
0.80
0.96
0.75
HistGradientBoosting
0.70
0.71
0.70
0.86
0.71
0.82
0.78
0.49
0.66
0.73
0.56
0.73
0.77
0.96
0.75
Random Forest
0.72
0.71
0.70
0.84
0.75
0.79
0.79
0.50
0.62
0.76
0.56
0.76
0.77
0.97
0.76
Table 6: Performance comparison across the boosting, ensemble, and baseline models, including overall metrics, balanced accuracy, and class-wise precision, recall, specificity, and F1 scores for normal, prediabetes, and diabetes. Recall and specificity are computed for each class by treating that class as positive and the other two classes as negative.
Formulation
Model
Accuracy
Balanced Accuracy
Precision
Recall
Specificity
F1 Score
ROC-AUC
Normal vs Prediabetes
XGBoost
0.73
0.74
0.55
0.76
0.71
0.64
0.8182
Normal vs Prediabetes
LightGBM
0.73
0.74
0.55
0.76
0.71
0.64
0.8169
Normal vs Prediabetes
CatBoost
0.73
0.73
0.55
0.75
0.72
0.64
0.8144
Normal vs Prediabetes
Ensemble
0.73
0.74
0.55
0.76
0.72
0.64
0.8178
Normal vs Prediabetes
HistGradientBoosting
0.73
0.74
0.55
0.76
0.72
0.64
0.8169
Normal vs Prediabetes
Random Forest
0.74
0.72
0.58
0.68
0.77
0.62
0.8055
Table 7: Performance comparison across targeted binary formulations. Metrics are reported for the positive class in each setup, corresponding to prediabetes in the first formulation and to the combined abnormal glycemic state, which includes both prediabetes and diabetes, in the second. Specificity is the proportion of normal individuals correctly classified as normal. Balanced accuracy is also reported to account for class imbalance between the normal and positive classes.
Formulation
Model
Accuracy
Balanced Accuracy
Macro F1
Precision
Recall
Specificity
F1
ROC-AUC
Multiclass
TabTransformer
0.70
0.71
0.69
0.49
0.61
0.75
0.55
0.86
Multiclass
FT-Transformer
0.71
0.72
0.68
0.50
0.56
0.79
0.53
0.86
Multiclass
SAINT
0.68
0.71
0.67
0.46
0.61
0.72
0.53
0.85
Normal vs Prediabetes
TabTransformer
0.72
0.73
0.71
0.55
0.75
0.71
0.63
0.80
Normal vs Prediabetes
FT-Transformer
0.73
0.73
0.71
0.56
0.74
0.73
0.63
0.81
Normal vs Prediabetes
SAINT
0.72
0.73
0.70
0.54
0.77
0.70
0.63
0.81
Table 8: Transformer-based model performance across the multiclass, normal versus prediabetes, and normal versus abnormal glycemic state formulations. For the multiclass task, precision, recall, specificity, and F1 refer to the prediabetes class, with specificity computed by treating prediabetes as positive and the remaining two classes as negative. For the binary tasks, these metrics refer to prediabetes in the normal versus prediabetes formulation and to the combined abnormal glycemic state in the normal versus abnormal formulation, and specificity is the proportion of normal individuals correctly classified as normal.
Figure 2: UMAP visualization of the normal, prediabetes, and diabetes groups in the test data. HbA1c and participant identifiers were excluded, numeric variables were standardized, and categorical variables were one-hot encoded. UMAP was fitted on the training set and the held-out test set was projected into two dimensions, using 30 neighbors, a minimum distance of 0.1, Euclidean distance, and a random seed of 42.
Encoding
Accuracy
Balanced Accuracy
Macro F1
Prediabetes F1
Label
0.71
0.72
0.70
0.56
One-Hot
0.71
0.72
0.69
0.55
Target
0.72
0.72
0.70
0.56
Table 9: Encoding sensitivity analysis for the multiclass XGBoost model.
Figure 3: Prediabetes recall across race or ethnicity groups for the baseline and fairness-aware multiclass XGBoost models.
Figure 4: SHAP summary plot illustrating feature contributions to the prediabetes class in the XGBoost model. Laboratory features are LBXGLU for fasting glucose, LBDLDL for LDL cholesterol, LBDHDD for HDL cholesterol, LBXTR for triglycerides, and LBXTC for total cholesterol. Body and blood pressure features are BMXBMI for body mass index, BMXWAIST for waist circumference, and SBP and DBP for systolic and diastolic blood pressure. Demographic and lifestyle features are RIDAGEYR for age, RIDRETH1 for race or ethnicity, SMQ040 for current cigarette smoking, SMQ020 for having smoked at least 100 cigarettes in life, PAD680 for minutes of sedentary activity, PAQ665 for moderate recreational activities, and MCQ300C for a close relative having diabetes. Engineered features are Age_BMI and Age_Waist for the interactions of age with body mass index and waist circumference, TG_to_HDL for the triglyceride-to-HDL ratio, and PulsePressure for pulse pressure.
Department of Biostatistics, Faculty of Medical Sciences, Tarbiat Modares University, Tehran, Iran · Department of Industrial Engineering, K. N. Toosi University of Technology, Tehran, Iran