cs.CVApr 10, 2026

Robust by Design: A Continuous Monitoring and Data Integration Framework for Medical AI

Authors: Mohammad Daouk, Jan Ulrich Becker, Neeraja Kambham, Anthony Chang, Chandra Mohan, Hien Van Nguyen

Organizations: University of Houston, Houston, TX, USA · University Hospital Cologne, Cologne, Germany · Stanford University, Stanford, CA, USA · The University of Chicago, Chicago, IL, USA

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Adaptive medical AI models often face performance drops in dynamic clinical environments due to data drift. We propose an autonomous continuous monitoring and data integration framework that maintains robust performance over time. Focusing on glomerular pathology image classification (proliferative vs. non-proliferative lupus nephritis), our three-stage method uses multi-metric feature analysis and Monte Carlo dropout-based uncertainty gating to decide when to retrain on new data. Only images statistically similar to the training distribution (via Euclidean, cosine, Mahalanobis metrics) and with low predictive entropy are integrated. The model is then incrementally retrained with these images under strict performance safeguards (no metric degradation >5%). In experiments with a ResNet18 ensemble on a multi-center dataset, the framework prevents performance degradation: new images were added without significant change in AUC (~0.92) or accuracy (~89%). This approach addresses data shift and avoids catastrophic forgetting, enabling sustained learning in medical imaging AI.

Figures & tables

Figure 1: Stage-2: Distribution of misclassified instances across different uncertainty values based on the ResNet18 model outputs on the test dataset. This figure highlights the relationship between predictive uncertainty and misclassification frequency, supporting the determination of an optimal uncertainty threshold.
Image IDEuclidean DistanceEuclidean ThresholdCosine SimilarityCosine ThresholdMahalanobis DistanceMahalanobis ThresholdPredictive UncertaintyPredictive Uncertainty Threshold
613.2918.470.83450.747618.8125.100.009110.24682
2811.9318.470.83710.747618.0825.100.001970.24682
5711.9918.470.86880.747617.3625.100.116610.24682
6211.1518.470.84540.747616.1125.100.010790.24682
7013.9918.470.81140.747621.1925.100.007110.24682
Table 1: Distance and similarity metrics along with predictive uncertainty for five new images computed using an ensemble of 5-fold ResNet18 models. For each image, the Euclidean distance, cosine similarity, Mahalanobis distance, and predictive entropy are reported alongside their respective thresholds—80th percentile for Euclidean and Mahalanobis distances, 20th percentile for cosine similarity (both computed on the base dataset), and the predictive entropy threshold computed on the test dataset.
Image IDRetrained AUCAUC % ChangeRetrained AccuracyAccuracy % ChangeRetrained SensitivitySensitivity % ChangeRetrained SpecificitySpecificity % ChangeUncertainty ThresholdUncrtnty % Change
692.01%0.20%88.98%-0.09%59.92%-4.35%96.10%0.60%0.032-87.08%
2892.06%0.26%89.14%0.09%61.09%-2.48%96.00%0.50%0.165-33.10%
5792.03%0.23%88.98%-0.09%60.70%-3.11%95.90%0.40%0.200-18.78%
6292.06%0.26%89.06%0.00%60.70%-3.11%96.00%0.50%0.237-3.94%
7091.99%0.19%89.06%0.00%60.31%-3.73%96.10%0.60%0.024-90.14%
Table 2: Performance metrics of the ensemble of 5-fold ResNet18 models (AUC for distinguishing proliferative from non-proliferative Glomerulonephritis) on the test dataset after retraining with the new image. Metrics such as AUC, accuracy, sensitivity, specificity, and the uncertainty threshold are reported along with the percent change from the original results, demonstrating that retraining results in negligible changes in model performance and uncertainty.
Figure 2: Stage-1: Distributions of Euclidean distance, cosine similarity, and Mahalanobis distance for new image (Image ID: 6) computed over 250 Monte Carlo dropout iterations, overlaid with the corresponding base dataset distributions. This visualization demonstrates how the new image aligns with the base dataset according to the selected metrics.
Figure 3: Stage-3: Comparison of the ensemble of 5-fold ResNet18 models AUC for distinguishing proliferative from non-proliferative Glomerulonephritis on the test dataset before and after retraining with the new image. The negligible difference in the AUC values confirms that the retraining process maintains model performance while integrating the new image.

Explore similar work

CardsList

  1. Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices

    Apr 6, 2026Alexis Burgon, Berkman Sahiner, Nicholas A Petrick +2Adaptive LearningStability-Plasticity Dilemma

    Division of Imaging, Diagnostics, and Software Reliability (DIDSR), Office of Science and Engineering Laboratories (OSEL), Center for Devices and Radiological Health (CDRH), U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD US 20993

  2. AEGIS: An Operational Infrastructure for Post-Market Governance of Adaptive Medical AI Under US and EU Regulations

    Mar 20, 2026Fardin Afdideh, Mehdi Astaraki, Fernando Seoane +1AI GovernanceResponsible AI

    Department of Clinical Science, Intervention and Technology, Karolinska Institutet, 17177 Stockholm, Sweden · Department of Medical Radiation Physics, Stockholm University, Solna, Sweden · Department of Oncology-Pathology, Karolinska Institutet, Solna, Sweden +4

  3. GenMed: A Pairwise Generative Reformulation of Medical Diagnostic Tasks

    May 11, 2026Hantao Zhang, Weidong Guo, Yuhe Liu +5Cross-Modality Medical Image SegmentationDiffusion Model Guidance

    CVLab, École Polytechnique Fédérale de Lausanne (EPFL), Switzerland · Fudan University, Shanghai, China · Beihang University, Beijing, China +3