Machine Learning Based Prediction of Surgical Outcomes in Chronic Rhinosinusitis from Clinical Data
Authors: Sayeed Shafayet Chowdhury, Karen D'Souza, V. Siva Kakumani, Snehasis Mukhopadhyay, Shiaofen Fang, Rodney J. Schlosser, Daniel M. Beswick, Jeremiah A. Alt, Jess C. Mace, Zachary M. Soler, Timothy L. Smith, Vijay R. Ramakrishnan
Organizations: Purdue University, Indianapolis, IN, USA · Indiana University Indianapolis, Indianapolis, IN, USA · Idaho National Laboratory, Idaho Falls, ID, USA · Department of Otolaryngology-Head and Neck Surgery, Medical University of South Carolina, Charleston, SC, USA · Department of Otolaryngology-Head and Neck Surgery, University of California, Los Angeles, Los Angeles, CA, USA · Department of Otolaryngology-Head and Neck Surgery, University of Utah, Salt Lake City, UT, USA · Department of Otolaryngology-Head and Neck Surgery, Oregon Health Sciences University, Portland, OR, USA · Department of Otolaryngology-Head and Neck Surgery, Indiana University School of Medicine, Indianapolis, IN, USA
Artificial intelligence (AI) has increasingly transformed medical prognostics by enabling rapid and accurate analysis across imaging and pathology. However, the investigation of machine learning predictions applied to prospectively collected, standardized data from observational clinical intervention trials remains underexplored, despite its potential to reduce costs and improve patient outcomes. Chronic rhinosinusitis (CRS), a persistent inflammatory disease of the paranasal sinuses lasting more than three months, imposes a substantial burden on quality of life (QoL) and societal cost. Although many patients respond to medical therapy, others with refractory symptoms often pursue surgical intervention. Surgical decision-making in CRS is complex, as it must weigh known procedural risks against uncertain individualized outcomes. In this study, we evaluated supervised machine learning models for predicting surgical benefit in CRS, using the Sino-Nasal Outcome Test-22 (SNOT-22) as the primary patient-reported outcome. Our prospectively collected cohort from an observational intervention trial comprised patients who all underwent surgery; we investigated whether models trained only on preoperative data could identify patients who might not have been recommended surgery prior to the procedure. Across multiple algorithms, including an ensemble approach, our best model achieved approximately 85% classification accuracy, providing accurate and interpretable predictions of surgical candidacy. Moreover, on a held-out set of 30 cases spanning mixed difficulty, our model achieved 80% accuracy, exceeding the average prediction accuracy of expert clinicians (75.6%), demonstrating its potential to augment clinical decision-making and support personalized CRS care.
Figures & tables
Figure 1: Classic local manifestations of Chronic Rhinosinusitis (CRS). In addition, systemic such as fatigue and depression, and financial burden of direct and indirect costs, together result in a large overall disease burden (Figure reused with permission from [ 17 ] ).
Figure 2: Schematic pipeline of the proposed decision-support approach. Structured fields from the electronic health record (EHR) undergo data cleaning and encoding, after which multiple in-house classifiers (e.g., MLP, logistic regression, SVM) are trained. Model outputs are combined via a majority-vote ensemble to generate a binary surgery recommendation (Yes/No).
Figure 3: Ensemble decision-support pipeline. Structured EHR data are cleaned and encoded, then passed in parallel to multiple base learners ( Random Forest , Logistic Regression , SVM , MLP etc.). Their predictions are aggregated via a voting scheme to produce a binary surgery recommendation (Yes/No). Only preoperative variables are used for inference.
Feature
# Nulls
Age
1
Race
1
Education
10
Household Income
7
Smoker
4
Alcohol
5
Table 1: Feature-wise missing (null) values in the 2R01 cohort restricted to surgery-treated patients.
Feature
# Nulls
Race
2
Education
3
Household Income
1
Obstructive Sleep Apnea (OSA) History
3
Alcohol
3
Baseline Endoscopy Score
3
Table 2: Feature-wise missing (null) values in the 3R01 cohort restricted to surgery-treated patients.
Figure 4: Class distribution in the train (left) and test (right) sets. Bars show the number of CRS patients who obtained the desired surgery outcome (1) versus those who did not obtain the desired outcome (0). The splits retain the overall class imbalance (train: 338 vs. 81; test: 85 vs. 20), which is considered in model training and evaluation.
Figure 5: Exploratory feature and PCA analysis. (a) Explained variance by principal component. Each bar shows the proportion of total variance attributable to an individual component after standardization; early components carry the largest share, followed by a long tail. (b) Cumulative explained variance. The running total of variance captured as components are added; the dashed line marks 95% retained variance, indicating that a moderate subset of components suffices for compact representations. (c) Feature correlation heatmap (Pearson’s correlation coefficient, r ). Blue denotes negative and red positive associations among preprocessed clinical predictors; the matrix helps flag redundancy/collinearity before modeling. (d) PCA score plot (PC1 vs. PC2). Patients are projected onto the first two principal components and colored by class label; the overlap across classes in this linear 2D view suggests that discrimination likely depends on higher-order components and/or non-linear structure.
Figure 6: Held-out test performance across five classifiers (Logistic Regression, SVM, Naïve Bayes, MLP, Random Forest). Each panel shows the classification report (precision, recall, F1-score, support) alongside the confusion matrix (counts).
Figure 7: MLP hyperparameter tuning (single hidden layer). Validation performance vs. hidden-layer width showing accuracy, weighted F1, and class-specific F1 scores. Overall accuracy and class 1 F1 remain relatively stable across widths, whereas the class 0 F1 (our priority) improves with larger layers and peaks for wide settings ( ∼ 400–480 units). Results are from stratified cross-validation on the training split.
Figure 8: Optimized MLP performance on the held-out test set. Left: class-wise precision, recall, and F1 for class 0 and class 1 (dashed line marks overall accuracy, 0.85). Right: macro- and weighted-average metrics, which are close to overall accuracy, indicating stable performance without overfitting to the majority class.
Figure 9: Class-wise comparison of precision, recall, and F1-score across five models on the held-out test set (MLP, Random Forest, Logistic Regression, SVM, Naïve Bayes). Each panel shows grouped bars for class 0 vs. class 1. Overall, class 1 metrics are higher for all models, while the MLP attains the strongest class 0 F1 among the candidates.
Figure 10: Optimized XGBoost results on the held-out test set. Left: class-wise precision, recall, F1 , and overall accuracy bars (class 0 vs. class 1). Right: confusion matrix (counts), yielding accuracy ≈0.83 with strong class 1 recall ( 0.96 ) but modest class 0 recall ( 0.25 ).
Figure 11: Majority-voting ensemble on the held-out test set. Left: classification report (precision, recall, F1 , support) for each class and macro/weighted averages. Right: confusion matrix (counts). The ensemble aggregates predictions from six tuned models (LR, SVM, RF, Naïve Bayes, MLP, XGBoost) with an MLP tie–break, yielding slightly higher overall accuracy than the single MLP while preserving similar class 0 performance.
Figure 12: AdaBoost ensemble on the held-out test set. Left: classification report (precision, recall, F1 , support) for each class and macro/weighted averages. Right: confusion matrix (counts) showing balanced overall performance with stronger class 1 detection and moderate class 0 recall.
Figure 13: Stacking ensemble on the held-out test set. Left: classification report by class and macro/weighted averages. Right: confusion matrix (counts). The meta-learner combines out-of-fold predictions from six tuned base models (LR, SVM, RF, Naïve Bayes, MLP, XGBoost); while overall accuracy is competitive, class 0 detection lags the MLP.
Figure 14: TabPFN on the held-out test set. Left: classification report by class and macro/weighted averages. Right: confusion matrix (counts). While TabPFN delivers excellent class 1 recall ( 0.99 ), class 0 recall ( 0.15 ) lags our optimized MLP, which is prioritized for clinical screening of likely non-responders.
Doctor 1
Doctor 2
Doctor 3
Doctor 4
Accuracy (%)
76.67
66.70
70.00
83.30
Table 3: Thirty-case human benchmarking subset: accuracy by rater. Row 1 reports Doctors 1–4; Row 2 reports Doctors 5–6, the in-house MLP, and ChatGPT.
Rater
Acc
P0
R0
P1
R1
Doctor 1
0.767
0.500
0.286
0.808
0.913
Doctor 2
0.667
0.286
0.286
0.783
0.783
Doctor 3
0.700
0.333
0.286
0.792
0.826
Doctor 4
0.833
0.750
0.429
0.846
0.957
Doctor 5
0.800
0.667
0.286
0.815
0.957
Doctor 6
0.767
0.500
0.143
0.786
0.957
Table 4: Expert benchmarking on the 30-case subset: per-expert performance and panel aggregation. P0 and P1 represent precision for class 0 and 1, respectively, and similarly R0 and R1 represent recall for class 0 and 1, respectively. “Panel of experts” uses majority vote; ties are resolved using confidence-weighted voting.
Figure 15: Human-benchmark subset ( n=30 ). Left (a): MLP prediction confusion matrix ( acc=0.80 . Right (b): ChatGPT prediction confusion matrix ( acc=0.57 . The MLP shows markedly higher recall for class 1 while maintaining comparable recall for class 0, yielding superior overall discrimination on this stratified sample.
Figure 16: Pooled experts: accuracy by case difficulty tier (Hard/Medium/Easy) on the 30-case benchmark subset.
Figure 17: Accuracy gradient across difficulty tiers for each expert and the pooled mean. Most raters show a monotonic Hard → Medium → Easy increase.
Figure 18: Permutation feature importance for the MLP classifier on the held-out test set. Bars show the mean decrease in balanced accuracy when each feature is randomly permuted (larger values indicate greater importance); black ticks denote the standard deviation across permutation repeats. The model is most sensitive to SNOT22_BLN_TOTAL , Age , and BLN_CT_TOTAL , followed by ALLERGY_TESTING , PREVIOUS_SURGERY , and CRS_POLYPS .
Figure 19: (a) SHAP bar (global importance): Mean absolute SHAP values on the test set rank features by their average contribution to the predicted probability of achieving the desired outcome; higher bars indicate greater global influence. (b) SHAP beeswarm: Each point represents a patient; the horizontal axis shows the SHAP value (feature effect on the log-odds/probability), and color encodes the feature value from low → high. Distributions reveal both effect size and heterogeneity.
Electrical and Computer Engineering University of Houston Houston, USA · Otolaryngology Houston Methodist Hospital Houston, USA · Information Science Technology, University of Houston & Otolaryngology, Houston Methodist Hospital Houston, USA
Saw Swee Hock School of Public Health, National University of Singapore, Singapore 117549 · Data Science and Artificial Intelligence Lab, Singapore General Hospital, Singapore 169608 · Duke-NUS Medical School, Singapore 169857 +5
Wanyin Wu, Haoyun Zhao, and Dapeng Tao are with the School of Information Science and Engineering, Yunnan University, Kunming 650500, China. · Kanxue Li and Yibing Zhan are with the School of Computer Science, Wuhan University, Wuhan 430072, China. · Baosheng Yu is with the Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 639798. +1