Coding-Free and Privacy-Preserving Agentic Framework for Data-Driven Clinical Research
Authors: Taehun Kim, Hyeryun Park, Hyeonhoon Lee, Yushin Lee, Kyungsang Kim, Hyung-Chul Lee
Organizations: Infmedix, Co., Ltd., Seoul, Republic of Korea. · Department of Transdisciplinary Studies, Seoul National University, Seoul, Republic of Korea. · Healthcare AI Research Institute, Seoul National University Hospital, Seoul, Republic of Korea. · Department of Medicine, Seoul National University College of Medicine, Seoul, Republic of Korea. · Department of Transdisciplinary Medicine, Seoul National University Hospital, Seoul, Republic of Korea. · Department of Radiology, Massachusetts General Hospital and Harvard Medical School, Boston, MA, USA.
Clinical data-driven research requires clinical expertise, programming skills, access to patient data, and extensive documentation, creating barriers and slowing the pace for clinicians and external researchers. To address this, we developed the Clinical Agentic Research Intelligence System (CARIS) that automates the workflow: research planning, literature search, cohort construction, Institutional Review Board (IRB) documentation, Vibe Machine Learning (ML), and report generation, with human-in-the-loop refinement. CARIS integrates Large Language Models (LLMs) with modular tools through the Model Context Protocol (MCP), enabling natural language-driven research without coding while allowing users to access only outputs. We evaluated CARIS on three heterogeneous datasets with distinct clinical tasks, where it completed planning and IRB documentation within four iterations, supported Vibe ML, and generated reports, achieving 96% completeness in LLM-based evaluation and 82% in human evaluation. CARIS demonstrates potential to reduce documentation burden and technical barriers, accelerating data-driven clinical research across public and private data environments.
Figures & tables
Figure 1: Overview of Clinical Agentic Research Intelligence System (CARIS). Given a user intent, CARIS autonomously executes a multi-step clinical research workflow by interacting with agents, heterogeneous datasets, and various tools, while allowing user intervention. Users do not have direct access to the file or DB systems.
Figure 2: Overview of the clinical research workflow. (A) User interaction to iteratively generate and refine a structured research plan. (B) Keyword extraction using the PIMO framework and retrieval of relevant PubMed literature. (C) Database exploration, cohort definition, query execution, and dataset generation. (D) Iterative generation and refinement of IRB document sections through user interaction. (E) Data analysis, preprocessing, model training, evaluation, and visualization, followed by the automated generation of a clinical ML report.
Dataset
Data source
Data format
Prediction task
Number of instances
MIMIC-IV
Beth Israel Deaconess Medical Center
RDB
ICU readmission
83,101
INSPIRE
Seoul National University Hospital
RDB
Postoperative AKI
100,474
SyntheticMass
Synthetic Massachusetts patient data
OMOP-CDM
Prediabetes-to-diabetes
2,556
Table 1: Summary of datasets and target tasks. RDB: Relational Database.
Figure 3: Revision patterns in IRB document generation. (A) Sections revised across iterations. Each dot represents a revision, with colors indicating section types. (B) Distribution of revision items per section. Numbers inside wedges indicate item counts, and percentages indicate their proportions.
Figure 4: Radar chart of IRB document evaluation results across four criteria. Each axis represents the pass rate (%) for each criteria assessed across three tasks.
Task (dataset)
Model
Feature selection(#)
AUROC
Precision
Recall
F1
ICU readmission (MIMIC)
XGBoost
MI + RFE (55)
71.50%
73.24%
51.24%
49.97%
XGBoost
SelectKBest MI (37)
71.09%
68.42%
50.92%
49.38%
Preoperative AKI (INSPIRE)
LightGBM
RFE (31)
85.77%
80.00%
53.88%
56.24%
LightGBM
Boruta with SHAP (30)
85.61%
78.64%
53.80%
56.08%
Prediabetes-to-diabetes (SyntheticMass)
XGBoost
LLM features + HDL (10)
88.58%
74.79%
68.62%
70.72%
XGBoost
LLM features + HDL + TG (11)
88.03%
70.98%
67.04%
68.50%
Table 2: Performance comparison of the top two ML models for each task under different feature selection methods. RF: Random Forest; MI: Mutual Information; RFE: Recursive Feature Elimination; TG: Triglycerides.
Figure 5: Checklist coverage of the final report across nine criteria. (A) Evaluation results by Claude Sonnet. (B) Evaluation results by human evaluator. Each axis represents criteria with checklist item count in parentheses. The dashed line indicates the average across all three reports.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Agent
Tool name
Description
Research planning
get_title_answer
Get the user’s research topic and refine it into a title
get_title
Generate questions to clarify the research from the title
stream_generator
Collects user responses to the generated questions
get_querys
Generates content for each research plan section
get_update_answer
Updates generated content based on user revision requests
get_pubmed_search
Summarizes proposed research, creates search keywords, evaluates the searched papers
Appendix
Table 1: Key Tools for Clinical Research Process Automation
Figure 1: User interface to interact with LLMs and modular MCP tools.
Figure 2: Revision form used to capture evaluator feedback, along with results across three tasks.
Figure 3: Evaluation form used to assess the final IRB document.
Figure 4: Evaluation results on the final IRB document across tasks.
Figure 5: IRB document evaluation results on the MIMIC ICU readmission prediction task.
Figure 6: IRB document evaluation results on the INSPIRE preoperative AKI prediction task.
Figure 7: IRB document evaluation results on the SyntheticMass ICU readmission prediction task.
Figure 8: Evaluation form used to assess the generated IRB document.
Figure 9: Claude Sonnet evaluation results on the final report (section 1-3) across tasks.
Figure 10: Claude Sonnet Evaluation results on the final report (section 4-6) across tasks.
Figure 11: Claude Sonnet evaluation results on the MIMIC ICU readmission prediction final report.
Figure 12: Claude Sonnet evaluation results on the INSPIRE preoperative AKI prediction final report.
Figure 13: Claude Sonnet evaluation results on SyntheticMass ICU readmission prediction final report.
Figure 14: Human evaluation results on the final report (section 1-3) across tasks.
Figure 15: Human evaluation results on the final report (section 4-6) across tasks.
Figure 16: Human evaluation results on the MIMIC ICU readmission prediction final report.
Figure 17: Human evaluation results on the INSPIRE preoperative AKI prediction final report.
Figure 18: Human evaluation results on SyntheticMass ICU readmission prediction final report.
With the emergence of large language models (LLMs) and AI agent frameworks, the human-AI co-work paradigm known as Vibe Coding is changing how people code, making it more accessible and productive. In scientific research, where workflows are more complex and the burden of specialized labor limits independent researchers and those in low-resource areas, the potential impact is even greater, particularly in biomedicine, which involves heterogeneous data modalities and multi-step analytical pipelines. In this paper, we introduce Vibe Medicine, a co-work paradigm in which clinicians and researchers direct skill-augmented AI agents through natural language to execute complex, multi-step biomedical workflows, while retaining the role of research director who specifies objectives, reviews intermediate results, and makes domain-informed decisions. The enabling infrastructure consists of three layers: capable LLMs, agent frameworks such as OpenClaw and Hermes Agent, and the OpenClaw medical skills collection, which includes more than 1,000 curated skills from multiple open-source repositories. We analyze the architecture and skill categories of this collection across ten biomedical domains, and present case studies covering rare disease diagnosis, drug repurposing, and clinical trial design that demonstrate end-to-end workflows in practice. We also identify the principal risks, such as hallucination, data privacy, and over-reliance, and outline directions toward more reliable, trustworthy, and clinically integrated agent-assisted research that advances research and technological equity and reduces health care resource disparities.
Zihao Wu, Steven Xu, Bowen Chen +8
School of Computing University of Georgia Athens GA 30602 USA · Department of Biomedical Engineering New Jersey Institute of Technology Newark NJ 07102 USA · Department of Computer Science and Engineering University of Texas at Arlington Arlington TX 76019 USA
Large language models (LLMs) and agents are now widely used tools in code development, with data typically sent to third-party cloud-based models. Their adoption in research using personal data is constrained by governance requirements that typically prohibit data transmission to external services. Locally deployable open-weight models offer an alternative since sensitive data never leave the local environment. We introduce an open-source framework for evaluating the efficacy of AI agents powered by open-weight LLMs on one of the most persistent bottlenecks in research on longitudinal population studies: data preparation. The framework comprises: a curated ground-truth dataset (cleaning scripts preparing six sweeps of data from a British cohort study), task definitions encompassing tasks such as category harmonization and multi-wave merging, and automated routines for evaluating the LLM-produced R code and outputted data. We benchmark LLMs across the (consumer grade) deployment spectrum to assess their efficacy in 20 data preparation tasks (creation of 102 variables). Current state-of-the-art, 31-35B parameter models almost saturated our benchmark ('average task completion' up to 87.9%). The performance of open-weight LLMs running on consumer-grade hardware shows promise of a viable path toward AI-assisted data preparation in governance-restricted research settings. Our framework is publicly available at: https://github.com/UCL-ARC/RRBench.
Mack Nixon, Liam Wright, Yevgeniya Kovalchuk +4
Centre for Advanced Research Computing, University College London, London, UK · Centre for Longitudinal Studies, University College London, London, UK · UK Longitudinal Linkage Collaboration, University of Bristol, Bristol, UK
Clinical reasoning agents based on large language models (LLMs) aim to automate tasks such as intensive care unit (ICU) monitoring and patient state tracking from electronic health records (EHRs). Existing systems typically rely on manually curated clinical tools or skills for concepts such as sepsis detection and organ failure assessment. However, maintaining these tool libraries requires substantial expert effort, while zero-shot querying or code generation often produces inefficient and unreliable reasoning chains, especially under institution-specific clinical policies. We introduce CodeClinic, a benchmark built on MIMIC-IV for evaluating whether LLM agents can synthesize and compose reusable clinical skills instead of relying on fixed toolboxes. The benchmark contains two complementary tasks: longitudinal ICU surveillance and compositional information seeking. The longitudinal setting simulates monitoring patient trajectories with structured decisions every four hours across 25 findings and eight clinical families, while the compositional setting spans 63k instances across 259 tasks in nine domains and is stratified by compositional dependency depth to evaluate increasingly complex multi-step reasoning. We further propose an offline autoformalization pipeline that converts natural-language clinical guidelines into reusable and verified Python skill libraries through iterative LLM refinement. Compared with zero-shot code generation, the resulting libraries improve consistency while reducing per-query token usage by up to 40%.
Timothy Ossowski, Xinchi Liu, Danyal Maqbool +6
University of Wisconsin–Madison · 2Microsoft Research