ScienceBuddy-PV: A Verifier-Governed Recursive Improvement Protocol for Panvascular Intervention Research
Authors: Lingsen You, Yujun Guo, Xinyu Zhong, Zijie Feng, Wentong Wang, Li Shen, Junbo Ge
Organizations: Department of Cardiology, Zhongshan Hospital, Fudan University, Shanghai Institute of Cardiovascular Diseases, State Key Laboratory of Cardiovascular Diseases, NHC Key Laboratory of Ischemic Heart Diseases China, National Clinical Research Center for Interventional Medicine, Key Laboratory of Viral Heart Diseases, Chinese Academy of Medical Sciences, Shanghai 200032, China. · Institutes of Biomedical Sciences, Fudan University, Shanghai 200030, China. · College of Biomedical Engineering, Fudan University, Shanghai 200438, China. · Department of Cardiology, Shidong Hospital, Yangpu District, Shanghai 200438, China. · Department of Chemical and Environmental Engineering, University of Nottingham, University Park, Nottingham NG7 2RD, United Kingdom.
ScienceBuddy-PV adapts the ScienceBuddy recursive-in-recursive design to panvascular intervention research through a structured evidence ledger, device-vessel ontology, abstention contract and retained failure bank. Recursive harness optimization and outer-loop model learning, including reinforcement learning, are specified but not executed. We audit a fixed component, rather than the complete agent system, using separate ARCADE syntax and stenosis tasks (1,000 training, 200 validation and 300 test images per task). A shared task-masked convolutional model was trained once per representation setting; its parameters and the heuristic support gate were then held fixed, with the stenosis threshold selected on validation data. On the locked test sets, syntax Dice was 0.475 (95% CI 0.459-0.490) and stenosis Dice was 0.231 (95% CI 0.214-0.249). Stenosis pixel AUROC was 0.957 and AUPRC was 0.180 at prevalence 0.00975. The gate accepted 76.7% of stenosis-frame outputs versus 100% for a score-only comparator. In a constructed mask audit, it accepted 44.3% of reference candidates and rejected 55.7%, exposing the cost of abstention. Acceptance does not establish correctness or clinical safety. This public-data component audit makes support and abstention measurable and provides a reproducible protocol for future collaborative agent studies. It does not demonstrate recursive improvement, patient-level validity or clinical device recommendations.
Figures & tables
Model
Threshold
Syntax Dice
Stenosis Dice
AUROC
AUPRC
Orthogonal probe
0.80
0.475
0.231
0.957
0.180
No orthogonality
0.75
0.457
0.236
0.956
0.186
Table 1: Locked test metrics. Values are image-level estimates from the public ARCADE test environments.
Figure 1: ScienceBuddy-PV workflow. Separate public task environments feed task-masked training; a locked verifier exposes support and abstention; actions and fault injections form audit environments. No paired syntax and stenosis observation is assumed.
Figure 2: Fixed-verifier action audit at the validation-selected score threshold. Constructed geometric transformations of predicted candidate masks alter the verifier state and usually trigger abstention; these are not learned treatment transitions.
Figure 3: Locked model under controlled distribution shifts.
Figure 4: Constructed fault-bank audit. Acceptance of reference and transformed candidate masks under the fixed support gate. Transformation labels are audit definitions rather than independently adjudicated errors.
Figure 5: Locked test metrics for the model trained with an orthogonality penalty. AUPRC is multiplied by 10 only for visual comparability; numerical values are reported in the text.
Figure 6: Validation-only threshold selection. The dashed line marks the score threshold frozen before test evaluation.
Figure 7: Retrospective test-threshold diagnostic showing gate coverage and its complement, abstention. This identity does not estimate selective prediction risk. The reported operating score threshold was selected on validation data and then locked.
Figure 8: Example held-out image, vessel-support prediction and stenosis score. These overlays are for technical inspection and are not clinical measurements.
Figure 9: Saved optimization histories for the matched representation comparison. Curves show objective value and validation Dice over three training epochs; the test model and threshold remained frozen for the locked audit.
Figure 10: Pixel-level discrimination on the untouched stenosis test environment. The precision–recall panel displays the low foreground prevalence explicitly; curves are computed from frozen model scores.
Figure 11: Retrospective per-frame stenosis Dice and evidence overlap. Points are held-out frames; colors show whether the fixed heuristic core-vessel gate accepted or abstained at the validation-selected score threshold.
Figure 12: Retrospective validation score-threshold diagnostic. The operating threshold (0.80) was selected on validation data before test evaluation; the panels show Dice, coverage of the fixed heuristic gate and pixel precision–recall.
Figure 13: Retrospective view of locked test metrics and the controlled-shift audit matrix. Values reproduce the saved summary; shifts are stress environments rather than external-cohort validation.
Figure 14: Five retrospectively selected, deterministic held-out examples spanning the evidence boundary. Reference candidate masks are shown for technical comparison; acceptance is the fixed heuristic gate decision at the validation-selected score threshold.
Figure 15: Retrospective locked-evaluation diagnostics. (A) Pixel ROC and precision–recall curves on the untouched stenosis test environment. (B) Pixel-level reliability diagram with observed prevalence and Brier score. (C–D) Core-vessel overlap and candidate-area state space at the validation-selected prediction score threshold, with fixed heuristic gate boundaries. These are image-level technical quantities, not patient-level risk estimates.
Figure 16: Matched representation ablation. Locked test metrics for the orthogonality and no-penalty models; values are image-level estimates from the frozen ARCADE test environments.
Figure 17: Retrospective examples of evidence and constructed-fault states. Held-out angiography frames are paired with reference candidates, predicted candidates, predicted vessel support and reflected candidate-mask transformations for technical inspection only. Transformation labels are not independently adjudicated errors.
School of Mathematical Sciences, Ocean University of China, Qingdao, 266100, Shandong, China. · Department of Cardiology, Qilu Hospital (Qingdao), Cheeloo College of Medicine, Shandong University, No. 758 Hefei Road, Qingdao, 266000, Shandong, China.
BCN Medtech, Universitat Pompeu Fabra, Barcelona, Spain · Universitätsklinikum Erlangen, Department of Radiology of the Uniklinikum Erlangen (UKER), Erlangen, Germany · University Hospital Erlangen, Imaging Science Institute, Erlangen, Germany +10