Early Risk Stratification of Dosing Errors in Clinical Trials Using Machine Learning

Félicien Hêche; Sohrab Ferdowsi; Anthony Yazdani; Sara Sansaloni-Pastor; Douglas Teodoro

cs.LGFeb 25, 2026

Early Risk Stratification of Dosing Errors in Clinical Trials Using Machine Learning

Authors: Félicien Hêche, Sohrab Ferdowsi, Anthony Yazdani, Sara Sansaloni-Pastor, Douglas Teodoro

Organizations: Department of Radiology and Medical Informatics, University of Geneva, Geneva, Switzerland · Actelion Pharmaceuticals Ltd, Basel, Switzerland

Abstract

Objective: The objective of this study is to develop a machine learning (ML)-based framework for early risk stratification of clinical trials (CTs) according to their likelihood of exhibiting a high rate of dosing errors, using information available prior to trial initiation. Materials and Methods: We constructed a dataset from ClinicalTrials.gov comprising 42,112 CTs. Structured, semi-structured trial data, and unstructured protocol-related free-text data were extracted. CTs were assigned binary labels indicating elevated dosing error rate, derived from adverse event reports, MedDRA terminology, and Wilson confidence intervals. We evaluated an XGBoost model trained on structured features, a ClinicalModernBERT model using textual data, and a simple late-fusion model combining both modalities. Post-hoc probability calibration was applied to enable interpretable, trial-level risk stratification. Results: The late-fusion model achieved the highest AUC-ROC (0.862). Beyond discrimination, calibrated outputs enabled robust stratification of CTs into predefined risk categories. The proportion of trials labeled as having an excessively high dosing error rate increased monotonically across higher predicted risk groups and aligned with the corresponding predicted probability ranges. Discussion: These findings indicate that dosing error risk can be anticipated at the trial level using pre-initiation information. Probability calibration was essential for translating model outputs into reliable and interpretable risk categories, while simple multimodal integration yielded performance gains without requiring complex architectures. Conclusion: This study introduces a reproducible and scalable ML framework for early, trial-level risk stratification of CTs at risk of high dosing error rates, supporting proactive, risk-based quality management in clinical research.

Figures & tables

Figure 1: Overview of the dataset construction steps, from CTs selection to feature and label construction and dataset splitting. Created with BioRender .

Categorical and binary features
Feature name	Data type	Missing (%)	Most frequent class (%)	Description
primaryPurpose	Categorical	0.94	Treatment (84)	Primary objective of the study
masking	Categorical	0.18	None (52)	Masking strategy used in the trial design
sex	Categorical	0.02	All (88)	Sex eligibility criteria for participants
phases	Categorical (multi-label)	0.01	Phase II (41)	Clinical development phase(s) of the trial
armGroupTypes	Categorical (multi-label)	0.60	Experimental (64)	Type(s) of study arms

Table 1: Overview of categorical/binary, numeric, and text features extracted from ClinicalTrials.gov. For each feature type, we report the corresponding summary statistic: most frequent class for categorical/binary variables, mean (SD) for numeric variables, and mean character length (SD) for text fields.

Figure 2: Overview of the dosing-error labeling pipeline. MedDRA high-level group terms were selected and reviewed by clinical pharmacology experts, matched to reported adverse events, aggregated at the trial level, and converted into binary risk labels using a Wilson confidence interval–based threshold. Created with BioRender

Figure 3: Illustration of the bias introduced by splitting CTs chronologically according to initiation dates. Because only completed or terminated trials are included, trials with shorter durations are more likely to fall within later time windows, leading to their overrepresentation in the validation and test sets.

Panel of Figure 4 — Figure 4: Comparison of distributional shifts induced by different temporal splitting strategies. Kernel density estimates of enrollment counts across the training, validation, and test sets are shown for each strategy; distributions are truncated at the 95th percentile for visualization purposes. The left panel shows splitting by trial initiation date, and the right panel shows splitting by trial completion date, which we adopt in this work.

Model	AUC	Brier	F1	F1 (macro)	Rec	Prec	BalAcc	Acc
XGBoost (uncal.)	0.848	0.105	0.293	0.619	0.423	0.224	0.674	0.900
XGBoost	0.848	0.042	0.292	0.619	0.423	0.222	0.674	0.899
ClinicalModernBERT (uncal.)	0.855	0.092	0.304	0.626	0.429	0.235	0.678	0.903
ClinicalModernBERT	0.855	0.042	0.306	0.628	0.426	0.239	0.678	0.905
LateFusion (uncal.)	0.862	0.090	0.300	0.633	0.284	0.318	0.626	0.935
LateFusion	0.862	0.041	0.298	0.632	0.281	0.319	0.625	0.935

Table 2: Classification performance on the test set before and after probability calibration. Best performance per metric is shown in bold.

Risk group	Number of CTs	Number of events	Event rate (%)	Relative risk
LateFusion (uncal.)
Low	401	0	0.00	0.000
Moderate	1773	6	0.34	0.069
High	953	8	0.84	0.171
Very high	3191	296	9.28	1.89
LateFusion

Table 3: Risk stratification of CTs on the test set using calibrated LateFusion predictions. Events correspond to trials labeled as exhibiting an elevated dosing error rate. Relative risks are reported relative to the overall event prevalence in the test set (4.91%).

Risk group	Number of CTs	Number of events	Event rate (%)	Relative risk
Early stage
Low	642	3	0.47	0.235
Moderate	75	1	1.33	0.672
High	50	5	10.00	5.038
Very high	39	7	17.95	9.042
Mid stage

Table 4: Risk stratification by clinical development stage on the test set using calibrated LateFusion predictions. Relative risks are reported with respect to the prevalence within each development stage.

Appendix figures & tables5 assets

Supplementary material from the paper’s appendix.

Appendix

Feature	ID	Meaning
phases	0	NA (not reported)
	1	EARLY_PHASE1
	2	PHASE1
	3	PHASE2
	4	PHASE3
	5	PHASE4

Appendix

Table 5: Overview of mappings between categorical features and their corresponding integer encodings. For multi-label variables, multiple category identifiers may be associated with a single trial.

Name	Data type / encoding	Description
overallStatus	Categorical	Overall recruitment status of the trial
leadSponsorClass	Categorical	Sponsor class
hasProtocol	Binary indicator	Indicates whether a protocol document is available
hasSap	Binary indicator	Indicates whether a statistical analysis plan document is available
hasIcf	Binary indicator	Indicates whether an informed consent form document is available
startDate	Date	Trial initiation date

Appendix

Table 6: Overview of additional information extracted.

Hyperparameter	Search range	Description
n_estimators	100 – 1000	Number of boosting trees
max_depth	3 – 12	Maximum depth of individual decision trees
learning_rate	$[10^{-3},0.3]$	Learning rate controlling the contribution of each tree; sampled on a logarithmic scale
subsample	0.5 – 1.0	Fraction of training samples randomly subsampled for each tree
colsample_bytree	0.5 – 1.0	Fraction of features randomly subsampled for each tree
gamma	0 – 5	Minimum loss reduction required to make a further partition on a leaf node

Appendix

Table 7: XGBoost hyperparameter search space.

	Number of CTs	Number of events	Event rate (%)	Relative risk
XGBoost (uncal.)
Low	79	0	0.00	0.000
Moderate	1311	1	0.08	0.015
High	1252	8	0.64	0.130
Very high	3676	301	8.19	1.669
XGBoost

Appendix

Table 8: Risk stratification results for all evaluated models on the test set. Results are shown for both calibrated and uncalibrated variants using predefined risk group cut-offs applied consistently across models.

Risk group	Number of CTs	Number of events	Event rate (%)	Relative risk
Enrollment $\leq$ 50
Low	2535	16	0.63	0.517
Moderate	328	5	1.52	1.248
High	124	8	6.45	5.283
Very high	43	8	18.60	15.236
Enrollment 51–200

Appendix

Table 9: Risk stratification by enrollment size on the test set using calibrated LateFusion predictions. Relative risks are reported with respect to the baseline prevalence within each enrollment group.

Explore similar work

CaresAI at CT-DEB26: Detecting Dosing Errors In Clinical Trials Using Domain-Specific Transformer Embeddings and Classification Models
Jun 29, 2026Leon Hamnett, Favour Igwezeke, Joseph Itopa Abubakar +1Clinical Decision Support Clinical NLP
CaresAI, Australia · Faculty of Pharmaceutical Sciences, Nsukka Enugu, Nigeria · Menzies School of Health Research, Australia
Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM
Mar 25, 2026Mohammad AL-SmadiClinical NLP Automated Feature Engineering
Qatar University Doha, Qatar
A Latent Risk-Aware Machine Learning Approach for Predicting Operational Success in Clinical Trials based on TrialsBank
Mar 30, 2026Iness Halimi, Emmanuel Piffo, Oumnia Boudersa +11Clinical Risk Prediction
Groupe Sorintellis Inc., Montréal, Québec, Canada · Department of Computer Science and Operations Research, Université de Montréal, Montréal, Québec, Canada · Department of Computer Engineering, École Polytechnique de Montréal, Montréal, Québec, Canada +6

Jun 29, 2026J/K move · Enter open · S save

CaresAI at CT-DEB26: Detecting Dosing Errors In Clinical Trials Using Domain-Specific Transformer Embeddings and Classification Models

Leon Hamnett, Favour Igwezeke, Joseph Itopa Abubakar, Mary Adetutu Adewunmi

CaresAI, Australia · Faculty of Pharmaceutical Sciences, Nsukka Enugu, Nigeria · Menzies School of Health Research, Australia

Clinical Decision Support Clinical NLP Clinical Risk Prediction

Open paper AskPDF arXiv

Medication errors, particularly dosing errors in clinical trials (CT), can lead to patient harm, adverse drug events and worse patient outcomes. Dosing errors are preventable, and early identification can improve trial integrity and mitigate subsequent clinical and financial burden. This study aims to detect dosing errors within CT protocols by evaluating text representations of trial information using transformer-based language models trained on biomedical corpora. CT textual data was encoded using several models, including ClinicalBERT, PubMedBERT, BioBERT, and MedCPT, and integrated with categorical features. These text embeddings were used as input to classical machine learning models and neural network architectures within an experimental framework. Performance was primarily assessed using ROC-AUC with respect to predicting dosage error. Under a logistic regression baseline, BioBERT consistently outperformed alternative encoders, achieving an ROC-AUC of 0.794, a 3.95% improvement over the ClinicalBERT baseline. Combining multiple embeddings did not yield improvements, indicating that domain alignment outweighs representational stacking. Gradient boosting models, support vector classifiers, logistic regression, and residual neural networks achieved the strongest performance for predicting dosage error, achieving ROC-AUCs: 0.821 to 0.853. Overall, the integration of domain-specific transformer embeddings with structured metadata enables discrimination of trials meeting a predefined elevated dosing error risk criterion, advancing safety monitoring and supporting informed regulatory decision-making.

Abstract

Explore similar work

CaresAI at CT-DEB26: Detecting Dosing Errors In Clinical Trials Using Domain-Specific Transformer Embeddings and Classification Models

Automated Detection of Dosing Errors in Clinical Trial Narratives: A Multi-Modal Feature Engineering Approach with LightGBM

A Latent Risk-Aware Machine Learning Approach for Predicting Operational Success in Clinical Trials based on TrialsBank