PRAISE: A Reproducible Workflow for Non-Programmer Scientists to Implement Published AI Models
Authors: Max Achterweust, Stefan Groot Nibbelink, Vincent J. de Beer, Stefan Schipper, Zeynep Yüksektepe, Pedro Andrade da Silva, Aurelia Peppelaar, Martijn J. van der Willigen, Christophe B. Minkenberg, Robert F. van der Willigen
Organizations: HR Datalab EAS, School of Engineering and Applied Sciences, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands · Department of Chemistry, School of Engineering and Applied Sciences, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands · Department of Electrical Engineering, School of Engineering and Applied Sciences, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands · Department of Biology and Medical Laboratory Research, School of Engineering and Applied Sciences, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands · Data Supported Healthcare: Data-Science Unit, Research Center Innovations in Care, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands · School of Communication, Media and Information Technology, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands · HR DataLab AI SusTech, Centre of Expertise HRTech, Rotterdam University of Applied Sciences, Rotterdam, the Netherlands
Artificial Intelligence (AI) models are increasingly developed, published and used across scientific fields. Researchers, students and industry professionals who want to apply these models to their own research questions often lack the programming and data science expertise to do so. Existing guidelines cover single steps, but not the route from a published model to working predictions on one's own data. Here we present the hands-on workflow PRAISE, a Practical, Reproducible AI-model Implementation for Science and Education, that helps non-programmers implement published AI models step by step. Because implementations rarely work the first time, PRAISE anticipates common failures and provides a fallback route for each. As a proof of principle, three BSc chemistry students and their tutor applied PRAISE without prior programming or data science knowledge. They reproduced, validated and extended a state-of-the-art model that predicts enzyme-substrate interactions. This shows that with PRAISE, implementing an AI model from the literature becomes a learnable skill, one that turns non-programmer scientists from users into evaluators and improvers of AI models.
Figures & tables
Figure 1: Visualization of the computational stack required to execute an AI model (left), and the tools that configure the stack (right). During a run, information flows from the top to the bottom of the stack and back. All layers and dependencies must be installed and compatible before the AI model can run. Users primarily work in the development tool and in the environment and package manager. Generative AI (GenAI) can help the user to develop the model and stack.
Figure 2: PRAISE (Practical, Reproducible AI-model Implementation for Science and Education) workflow for implementing scientific AI models to practical case studies, assuming no issues arise. Outputs and checks per step are given. Troubleshooting steps for common issues are shown in Section 5 .
Figure 3: AI use case formulation guide. An AI use case can be constructed by bringing together both problem and solution discovery, until a candidate model is found that fits a defined knowledge gap. The regulatory compliance is performed afterwards.
Figure 4: The PRAISE workflow for implementing scientific AI models to practical case studies, extended with troubleshooting routes. When a checkpoint is not met, the arrow marked with a cross leads to a decision point that helps decide which solution is required. These bring the user one or more steps back, depending on the issue. Such iterations are common in AI model implementations.
Figure 5: Implementation of an AI model to predict enzyme-substrate interactions following the PRAISE workflow. Colored arrows mark the first pass and the four iterations that followed, numbered in the order they occurred. First pass: the initial implementation of ProSmith, which stopped at the first issue. First iteration: the switch from Windows to a cloud-based Linux OS. Second iteration: the retraining on a dataset supplemented with data gathered manually from the literature. Third iteration: the retraining on an additional dataset containing true-negative data. Fourth iteration: the changes to the training pipeline to investigate suspected data leakage.
Model
Iteration
Accuracy
Precision
ROC-AUC
MCC
Standard ProSmith Windows
Original
0.93
n.d.
0.96
0.80
Standard ProSmith Linux
Install & computing → infrastructure selection
0.92
0.94
0.95
0.80
ProSmith manually supplemented database
Validation → data preparation
0.91
0.92
0.95
0.76
ProSmith supplemented with VIPER database
Validation → data preparation
0.93
0.83
0.95
0.75
Standard ProSmith suspected data leak fix
Validation → install & computing
0.90
0.92
0.95
0.74
Table 1: Outcomes of five different ProSmith model training sessions and the corresponding validation scores. Each training was performed once, and the values are only indicative. The suspected data leak fix was investigated and found not to be data leakage.
Department of Electrical and Computer Engineering, University of California San Diego, La Jolla, CA 92093, USA · Department of Medicine, University of California San Diego, La Jolla, CA 92093, USA