AEGIS: An Operational Infrastructure for Post-Market Governance of Adaptive Medical AI Under US and EU Regulations

Fardin Afdideh; Mehdi Astaraki; Fernando Seoane; Farhad Abtahi

cs.LGMar 20, 2026

AEGIS: An Operational Infrastructure for Post-Market Governance of Adaptive Medical AI Under US and EU Regulations

Authors: Fardin Afdideh, Mehdi Astaraki, Fernando Seoane, Farhad Abtahi

Organizations: Department of Clinical Science, Intervention and Technology, Karolinska Institutet, 17177 Stockholm, Sweden · Department of Medical Radiation Physics, Stockholm University, Solna, Sweden · Department of Oncology-Pathology, Karolinska Institutet, Solna, Sweden · Department of Clinical Physiology, Karolinska University Hospital, 17176 Stockholm, Sweden · Department of Textile Technology, University of Bor˚as, Bor˚as 50190, Sweden · Department of Medical Technologies, Karolinska University Hospital, 141 57 Huddinge, Sweden · Department of Biomedical Engineering and Health System, KTH Royal Institute of Technology, 14157 Huddinge, Sweden

Abstract

Machine learning systems deployed in medical devices require governance frameworks that ensure safety while enabling continuous improvement. Regulatory bodies including the FDA and European Union have introduced mechanisms such as the Predetermined Change Control Plan (PCCP) and Post-Market Surveillance (PMS) to manage iterative model updates without repeated submissions. This paper presents AI/ML Evaluation and Governance Infrastructure for Safety (AEGIS), a governance framework applicable to any healthcare AI system. AEGIS comprises three modules, i.e., dataset assimilation and retraining, model monitoring, and conditional decision, that operationalize FDA PCCP and EU AI Act Article 43(4) provisions. We implement a four-category deployment decision taxonomy (APPROVE, CONDITIONAL APPROVAL, CLINICAL REVIEW, REJECT) with an independent PMS ALARM signal, enabling detection of the critical state in which no deployable model exists while the released model is simultaneously at risk. To illustrate how AEGIS can be instantiated across heterogeneous clinical contexts, we provide two examples: sepsis prediction from electronic health records and brain tumor segmentation from medical imaging. Both cases use identical governance architecture, differing only in configuration. Across 11 simulated iterations on the sepsis example, AEGIS yielded 8 APPROVE, 1 CONDITIONAL APPROVAL, 1 CLINICAL REVIEW, and 1 REJECT decision, exercising all four categories. ALARM signals were co-issued at iterations 8 and 10, including the critical state where no deployable model exists and the released model is simultaneously failing. AEGIS detected drift before observable performance degradation. These results demonstrate that AEGIS translates regulatory change-control concepts into executable governance procedures, supporting safe continuous learning for adaptive medical AI across diverse clinical applications.

Figures & tables

Dimension	FDA (510(k)/ PCCP )	EU MDR
Core Philosophy	SE	Continuous risk management
Foundational Question	“Is this device as safe and effective as something already proven?”	“Does the ongoing risk-benefit analysis remain acceptable?”
Evidence Paradigm	Comparative (new vs. predicate)	Cumulative (ongoing real-world evidence)
Clinical Data Focus	Premarket; 510(k) may rely on bench testing	Continuous clinical evaluation ( Post-Market Clinical Follow-up (PMCF) )
Change Control Logic	Modifications within PCCP bounds maintain equivalence	Changes assessed against evolving risk profile
Methodological Backbone	Predicate device comparison	ISO 14971:2019 risk management

TABLE I: Regulatory Philosophy Comparison: FDA vs. EU MDR

Fig. 1: DARM architecture. The module manages three datasets: Training ( $\mathcal{D}_{T,k}$ ) for parameter updates, Golden ( $\mathcal{D}_{G}$ ) as static benchmark, and Drifting ( $\mathcal{D}_{D,k}$ ) for drift monitoring.

Parameter Categories	Description
Tolerance	Acceptable deviation from target value
Range	Minimum and maximum bounds for acceptable values
Reference	Reference values against which comparisons are made
Distribution Shift Metrics	KL divergence, JS distance, mean drift
Trustworthy AI (TAI) Metrics	Demographic parity, equalized odds metrics
Model Performance Metrics	Accuracy, sensitivity, specificity, AUC, F1-score

TABLE II: Parameters

Fig. 2: MMM architecture showing metric calculation across performance, fairness, and distribution categories, historical KPI maintenance, and outputs for monitoring and compliance.

Decision	Question answered	Boolean Function f	Operator & Rationale
CHANGE-CONTROL DEPLOYMENT DECISIONS: new candidate model
REJECT	Is new model safe?	$({P_{G,k}}<{P^{\text{fail}}})\lor(\text{any safety metric}<\text{its floor})$	OR — any single safety failure blocks deployment
CLINICAL REVIEW	Does new model need human validation?	$({P^{\text{fail}}}\leq{P_{G,k}}<{P^{\text{fail}}}+{\tau})\lor({\tau}<{P^{\text{ref}}}-{P_{G,k}})$	OR — any metric in buffer zone, or any regression from ${P^{\text{ref}}}$ (fixed performance reference comparison fails).
CONDITIONAL APPROVAL	Does the new model need enhanced monitoring?	$(\mathcal{S}_{D}\in{\mathcal{S}_{D}^{\text{minor}}})\lor(\mathcal{S}_{T}<{\mathcal{S}_{T}^{\text{thr}}})$	OR — minor drift or any TAI concern each independently warrant enhanced monitoring
APPROVE	Does new model pass all checks?	$(({P^{\text{ref}}}-{\tau})\leq{P_{G,k}})\land({P_{G,k-1}}\leq{P_{G,k}})\land(\mathcal{S}_{D}\leq{\mathcal{S}_{D}^{\text{minor}}})\land({\mathcal{S}_{T}^{\text{thr}}}\leq\mathcal{S}_{T})$	AND — all checks must pass; any single failure routes to a lower category
PMS ALARM SIGNAL: currently released model

TABLE III: CDM Decision Categories. Deployment decisions: Four decision categories for a new-model deployment. PMS ALARM signal: ALARM for the currently released model. The two parts are evaluated independently and produce a composite output.

Priority	Check	Condition	Decision
New Model Deployment Hierarchy (evaluate in priority order on $\mathcal{D}_{G}$ and $\mathcal{D}_{D,k}$ )
P1	deployment safety performance floor	${P_{G,k}}<{P^{\text{fail}}}$	REJECT
P2	performance buffer / regression from fixed performance reference	$({P_{G,k}}\in{R_{G}})\lor({\tau}<{P^{\text{ref}}}-{P_{G,k}})$	CLINICAL REVIEW
P3	minor drift / TAI	$(\mathcal{S}_{D}\in{\mathcal{S}_{D}^{\text{minor}}})\lor(\mathcal{S}_{T}<{\mathcal{S}_{T}^{\text{thr}}})$	CONDITIONAL APPROVAL
P4	all pass	all checks passed	APPROVE
PMS ALARM Signal (evaluate in parallel using released model on $\mathcal{D}_{D,k}$ )

TABLE IV: Decision Hierarchy: Priority-Ordered Evaluation

Fig. 3: CDM architecture showing flow from metrics and thresholds through atomic conditions and Boolean aggregation to decision outcomes.

Fig. 4: AEGIS infrastructure integrating DARM , MMM , and CDM at each iteration $k$ .

Parameter	AEGIS Definition	Sepsis Instantiation	Segmentation Instantiation
Data modality	Any supervised learning data	Tabular Electronic Health Record (EHR) time-series (202 features)	3D multi-parametric Magnetic Resonance Imaging (MRI)
Model architecture	Agnostic (wrapper)	RF	nnU-Net ResENC-M
Primary metrics	Domain-configurable	Sens., Spec., ROC-AUC	Dice Similarity Coefficient (DSC) ( Tumor Core (TC) )
MLcps weighting	User-defined via risk assessment	Sens.: 1.5, AUC: 1.3, Bal. Acc.: 1.1, Spec.: 1.0 (raw; normalized internally)	DSC : 1
Drift detection	Pluggable method	Bonferroni-corrected Kolmogorov-Smirnov (KS) drift score	Performance comparison
Clinical thresholds	Risk-assessment derived (ISO 14971:2019)	Sens. $<$ 0.65 ∗	DSC $<$ 0.676 ( TC ) ∗∗

TABLE V: AEGIS Instantiation Parameters

Fig. 5: AEGIS instantiation pattern. The abstract infrastructure (middle) defines domain-agnostic modules. Each instantiation (top and bottom) configures with domain-specific parameters while maintaining identical logic.

Decision	Definition	FDA PCCP Status	EU MDR / AI Act	Trigger
APPROVE	All metrics satisfied	Within PCCP scope	Art. 43(4) exemption	All pass
CONDITIONAL APPROVAL	Enhanced monitoring	Limited PCCP scope	Enhanced PMS	Minor drift or low trust scores
CLINICAL REVIEW	Human validation required	May exceed PCCP	Pre-modification review	Near threshold
ALARM	Released model at risk in field; issued as a PMS signal independently of the new model deployment decision	Activates PMS reporting; may require vigilance report depending on severity	EU MDR Art. 83 PMS obligation; potential Art. 87 serious incident notification	PMS safety performance floor breach; released model regression; major distributional shift
REJECT	Deployment blocked	New 510(k) required	New CA	Safety violation

TABLE VI: Decision Taxonomy with Regulatory Mapping ∗

AEGIS Parameter	Concrete Value	Clinical Justification	CDM Decision
${P^{\text{ref}}}$	Sens.: 0.723, Spec.: 0.933, AUC: 0.922	performance of the first fully approved model on the Golden dataset	fixed performance reference
${P^{\text{fail}}}$	Sens.: $0.65$	derived from published systematic reviews of sepsis prediction performance [ 8 , 30 ]	REJECT
${R_{G}}$	$0.66\leq$ Sens. $0.60\leq$ Spec.	Performance thresholds (buffer zone)	CLINICAL REVIEW
Major drift threshold	$0.90<\mathcal{S}_{D}$	$>90\%$ of 34 key features shifted ( KS test)	ALARM (A3)
Minor drift range	$0.30\leq\mathcal{S}_{D}\leq 0.70$	Moderate distributional change detected	CONDITIONAL APPROVAL
${P^{\text{PMS}}}^{*}$	Sens.: $0.65$	$={P^{\text{fail}}}$	ALARM (A1)

TABLE VII: Mapping of Abstract AEGIS Parameters to Sepsis Prediction Parameters. This table provides the sepsis-specific instantiation of the infrastructure-level CDM priority hierarchy defined in Table IV .

Fig. 6: GLI tumors in pre-operative and post-treatment mpMRI .

Iter.	$\lvert\mathcal{D}_{T,k}\rvert$	Sens.	Spec.	AUC	MLcps	$\mathcal{S}_{D}$	Trigger	Routing Trace ∗	Deploy. Decision	PMS Signal
0	8,134	0.723	0.933	0.922	0.721	0.000	All satisfied	P1 $\checkmark\,$ P2 $\checkmark\,$ P3 $\checkmark\,$ $\rightarrow$ P4	APPR.	—
1	9,760	0.745	0.926	0.922	0.731	0.000	All satisfied	P1 $\checkmark\,$ P2 $\checkmark\,$ P3 $\checkmark\,$ $\rightarrow$ P4	APPR.	—
2	11,386	0.745	0.926	0.923	0.731	0.000	All satisfied	P1 $\checkmark\,$ P2 $\checkmark\,$ P3 $\checkmark\,$ $\rightarrow$ P4	APPR.	—
3	13,012	0.742	0.926	0.923	0.729	0.000	All satisfied	P1 $\checkmark\,$ P2 $\checkmark\,$ P3 $\checkmark\,$ $\rightarrow$ P4	APPR.	—
4	14,638	0.742	0.932	0.923	0.733	0.000	All satisfied	P1 $\checkmark\,$ P2 $\checkmark\,$ P3 $\checkmark\,$ $\rightarrow$ P4	APPR.	—
5	16,268	0.753	0.929	0.924	0.739	0.000	All satisfied	P1 $\checkmark\,$ P2 $\checkmark\,$ P3 $\checkmark\,$ $\rightarrow$ P4	APPR.	—

TABLE VIII: Iteration Summary with Trigger Conditions. The Routing Trace column traces the exact priority-ordered evaluation for the new candidate model at each iteration. The PMS Signal column records ALARM co-issuances where the released model was assessed as at risk.

Fig. 7: AEGIS comprehensive iterative change control results. A) Clinical performance metrics; B) PPV and NPV ; C) ROC-AUC , PR-AUC , and MLcps ; D) Matthews correlation and Kappa; E) Drift score; F) Training size; G) Confusion matrix components; H) Deployment decisions with PMS ALARM markers at iterations 8 and 10; I) MLcps score with CDM decision context.

Iter.	$\lvert\mathcal{D}_{T,k}\rvert$	${P_{G,k}}$	${P_{D,k}}$	Trigger	Routing Trace*	Deploy. Decision	PMS Signal
0	1150	0.726 $\pm$ 0.313	—	fixed performance reference	—	—	—
1	200	0.646 $\pm$ 0.337	0.694 $\pm$ 0.325	safety violation ( $P_{G,1}<0.676$ )	$\rightarrow P1$	REJ.	—
2	400	0.673 $\pm$ 0.337	0.748 $\pm$ 0.289	safety violation ( $P_{G,2}<0.676$ )	$\rightarrow P1$	REJ.	—
3	600	0.690 $\pm$ 0.326	0.745 $\pm$ 0.308	minor drift ( ${P^{\text{rel}}_{G}}=0.690$ ; $P_{D,3}<P_{D,2}$ )	$P1\checkmark\rightarrow P4$	APPR.	ALARM
4	800	0.700 $\pm$ 0.321	0.758 $\pm$ 0.284	All satisfied ( ${P^{\text{rel}}_{G}}=0.700$ )	$P1\checkmark\rightarrow P4$	APPR.	—
5	1000	0.725 $\pm$ 0.325	0.735 $\pm$ 0.325	minor drift ( ${P^{\text{rel}}_{G}}=0.725$ ; $P_{D,5}<P_{D,4}$ )	$P1\checkmark\rightarrow P4$	APPR.	ALARM

TABLE IX: DSC Metric for Tumor Core Segmentation with Routing Trace

AEGIS Parameter	Concrete Value	Clinical Justification	CDM Decision
${P^{\text{ref}}}$	DSC : 0.726	fixed performance reference. performance of the model trained on the full cohort on the Golden set ( TC region)	—
${\tau}$	0.05	Defines ${P^{\text{fail}}}$ boundary. Clinically acceptable margin below fixed performance reference	—
${P^{\text{fail}}}={P^{\text{ref}}}-{\tau}$	0.676	Minimum acceptable TC segmentation for treatment planning	—
${C^{\text{R}}}$	${P_{G,k}}<{P^{\text{fail}}}$	Performance falls below the minimum requirement for safe radiotherapy and surgical planning	REJECT
${C^{\text{NR}}}$	${P^{\text{rel}}_{G}}\leq{P_{G,k}}$	Non-regression condition: must match or exceed released performance reference	APPROVE
${C^{\text{R}}}$	${P_{G,k}}<{P^{\text{rel}}_{G}}$	Regression condition: must be less than released performance reference	REJECT

TABLE X: Mapping of Abstract AEGIS Parameters to Brain Tumor Segmentation Parameters

Dimension	ABCDS (Duke)	FDA AILC	Babic et al.	AEGIS
Focus	Institutional governance	Conceptual lifecycle	Surveillance gaps	Operational implementation
Contribution	Committee structure	Phase-based model	Policy recommendations	Metrics-driven logic
Decision Mechanism	Committee review	Phase guidance	Quarterly updates	Threshold-based, five categories
Monitoring	Periodic human review	Conceptual	Recommended	Continuous with drift detection
Drift Detection	Not specified	Conceptual	Recommended	KS-test with threshold
Regulatory Mapping	Implicit	General	FDA MAUDE	Explicit PCCP / MDR / AI Act

TABLE XI: Comparison with Existing Approaches

Recommendation	AEGIS Implementation
Quarterly deployment updates	DARM iterative cycle; Table VIII shows 11 tracked iterations
Concept drift detection	MMM KS -test (threshold $>$ 0.90) triggers ALARM
Covariate shift monitoring	Golden dataset static; drifting dataset compared against reference
Algorithmic stability	fixed performance reference comparison (1% tolerance)
Subgroup tracking	MMM bias score across demographics

TABLE XII: Mapping Babic et al. Recommendations to AEGIS

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Symbol	Definition
${\tau}$	Tolerance: acceptable deviation from target value
${R_{G}}/{R_{D}}$	Acceptable performance ranges for Golden/Drifting datasets
${P^{\text{ref}}}$	fixed performance reference: performance of the first fully approved model on the Golden dataset. Set at first APPROVE; never updated.
${P^{\text{fail}}}/{P^{\text{PMS}}}$	deployment/ PMS safety performance floor below which REJECT/ALARM is triggered
${P_{G,k}}/{P_{D,k}}$	candidate model performance on Golden/Drifting dataset at iteration $k$
${P^{\text{rel}}_{G}}/{P^{\text{rel}}_{D}}$	fully approved released model performance on Golden/Drifting set

Appendix

TABLE XIII: Mathematical Notation

Explore similar work

Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices
Apr 6, 2026Alexis Burgon, Berkman Sahiner, Nicholas A Petrick +2Adaptive Learning Stability-Plasticity Dilemma
Division of Imaging, Diagnostics, and Software Reliability (DIDSR), Office of Science and Engineering Laboratories (OSEL), Center for Devices and Radiological Health (CDRH), U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD US 20993
From Single Chatbots to Governed Agent Ecosystems: An Agentic AI Pattern Catalogue and Orchestration Framework for Mission-Critical Hospital Information Management Systems
Aug 7, 2026Manideep Dhar, Ritwik Singh, Sharat Chandra Kumar ManikondaHealthcare LLM Agent Orchestration
Chief Operating Officer and Head of Research and Development, Instil-it, Hyderabad, Telangana. · Chief AI Architect Engineer, Instil-it, Hyderabad, Telangana · Chief Executive Officer, Managing Director and Chief Data Scientist, Instil-it, Hyderabad, Telangana
Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring
May 23, 2026Jehanne DussertLLM-as-a-Judge AI Governance
Independent Researcher

Apr 6, 2026J/K move · Enter open · S save

Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices

Alexis Burgon, Berkman Sahiner, Nicholas A Petrick, Gene Pennello+1

Division of Imaging, Diagnostics, and Software Reliability (DIDSR), Office of Science and Engineering Laboratories (OSEL), Center for Devices and Radiological Health (CDRH), U.S. Food and Drug Administration, 10903 New Hampshire Ave, Silver Spring, MD US 20993

Adaptive Learning Stability-Plasticity Dilemma

Open paper AskPDF arXiv

This work addresses challenges in evaluating adaptive artificial intelligence (AI) models for medical devices, where iterative updates to both models and evaluation datasets complicate performance assessment. We introduce a novel approach with three complementary measurements: learning (model improvement on current data), potential (dataset-driven performance shifts), and retention (knowledge preservation across modification steps), to disentangle performance changes caused by model adaptations versus dynamic environments. Case studies using simulated population shifts demonstrate the approach's utility: gradual transitions enable stable learning and retention, while rapid shifts reveal trade-offs between plasticity and stability. These measurements provide practical insights for regulatory science, enabling rigorous assessment of the safety and effectiveness of adaptive AI systems over sequential modifications.

Abstract

Explore similar work

Learning, Potential, and Retention: An Approach for Evaluating Adaptive AI-Enabled Medical Devices

From Single Chatbots to Governed Agent Ecosystems: An Agentic AI Pattern Catalogue and Orchestration Framework for Mission-Critical Hospital Information Management Systems

Who judges the judges? Governance from metrics: a runtime framework for continuous LLM compliance monitoring