ScienceBuddy-PV: A Verifier-Governed Recursive Improvement Protocol for Panvascular Intervention Research
Authors: Lingsen You, Yujun Guo, Xinyu Zhong, Zijie Feng, Wentong Wang, Li Shen, Junbo Ge
Organizations: Department of Cardiology, Zhongshan Hospital, Fudan University, Shanghai Institute of Cardiovascular Diseases, State Key Laboratory of Cardiovascular Diseases, NHC Key Laboratory of Ischemic Heart Diseases China, National Clinical Research Center for Interventional Medicine, Key Laboratory of Viral Heart Diseases, Chinese Academy of Medical Sciences, Shanghai 200032, China. · Institutes of Biomedical Sciences, Fudan University, Shanghai 200030, China. · College of Biomedical Engineering, Fudan University, Shanghai 200438, China. · Department of Cardiology, Shidong Hospital, Yangpu District, Shanghai 200438, China. · Department of Chemical and Environmental Engineering, University of Nottingham, University Park, Nottingham NG7 2RD, United Kingdom.
ScienceBuddy-PV adapts the ScienceBuddy recursive-in-recursive design to panvascular intervention research through a structured evidence ledger, device-vessel ontology, abstention contract and retained failure bank. Recursive harness optimization and outer-loop model learning, including reinforcement learning, are specified but not executed. We audit a fixed component, rather than the complete agent system, using separate ARCADE syntax and stenosis tasks (1,000 training, 200 validation and 300 test images per task). A shared task-masked convolutional model was trained once per representation setting; its parameters and the heuristic support gate were then held fixed, with the stenosis threshold selected on validation data. On the locked test sets, syntax Dice was 0.475 (95% CI 0.459-0.490) and stenosis Dice was 0.231 (95% CI 0.214-0.249). Stenosis pixel AUROC was 0.957 and AUPRC was 0.180 at prevalence 0.00975. The gate accepted 76.7% of stenosis-frame outputs versus 100% for a score-only comparator. In a constructed mask audit, it accepted 44.3% of reference candidates and rejected 55.7%, exposing the cost of abstention. Acceptance does not establish correctness or clinical safety. This public-data component audit makes support and abstention measurable and provides a reproducible protocol for future collaborative agent studies. It does not demonstrate recursive improvement, patient-level validity or clinical device recommendations.
Figures & tables
Model
Threshold
Syntax Dice
Stenosis Dice
AUROC
AUPRC
Orthogonal probe
0.80
0.475
0.231
0.957
0.180
No orthogonality
0.75
0.457
0.236
0.956
0.186
Table 1: Locked test metrics. Values are image-level estimates from the public ARCADE test environments.
Figure 1: ScienceBuddy-PV workflow. Separate public task environments feed task-masked training; a locked verifier exposes support and abstention; actions and fault injections form audit environments. No paired syntax and stenosis observation is assumed.
Figure 2: Fixed-verifier action audit at the validation-selected score threshold. Constructed geometric transformations of predicted candidate masks alter the verifier state and usually trigger abstention; these are not learned treatment transitions.
Figure 3: Locked model under controlled distribution shifts.
Figure 4: Constructed fault-bank audit. Acceptance of reference and transformed candidate masks under the fixed support gate. Transformation labels are audit definitions rather than independently adjudicated errors.
Figure 5: Locked test metrics for the model trained with an orthogonality penalty. AUPRC is multiplied by 10 only for visual comparability; numerical values are reported in the text.
Figure 6: Validation-only threshold selection. The dashed line marks the score threshold frozen before test evaluation.
Figure 7: Retrospective test-threshold diagnostic showing gate coverage and its complement, abstention. This identity does not estimate selective prediction risk. The reported operating score threshold was selected on validation data and then locked.
Figure 8: Example held-out image, vessel-support prediction and stenosis score. These overlays are for technical inspection and are not clinical measurements.
Figure 9: Saved optimization histories for the matched representation comparison. Curves show objective value and validation Dice over three training epochs; the test model and threshold remained frozen for the locked audit.
Figure 10: Pixel-level discrimination on the untouched stenosis test environment. The precision–recall panel displays the low foreground prevalence explicitly; curves are computed from frozen model scores.
Figure 11: Retrospective per-frame stenosis Dice and evidence overlap. Points are held-out frames; colors show whether the fixed heuristic core-vessel gate accepted or abstained at the validation-selected score threshold.
Figure 12: Retrospective validation score-threshold diagnostic. The operating threshold (0.80) was selected on validation data before test evaluation; the panels show Dice, coverage of the fixed heuristic gate and pixel precision–recall.
Figure 13: Retrospective view of locked test metrics and the controlled-shift audit matrix. Values reproduce the saved summary; shifts are stress environments rather than external-cohort validation.
Figure 14: Five retrospectively selected, deterministic held-out examples spanning the evidence boundary. Reference candidate masks are shown for technical comparison; acceptance is the fixed heuristic gate decision at the validation-selected score threshold.
Figure 15: Retrospective locked-evaluation diagnostics. (A) Pixel ROC and precision–recall curves on the untouched stenosis test environment. (B) Pixel-level reliability diagram with observed prevalence and Brier score. (C–D) Core-vessel overlap and candidate-area state space at the validation-selected prediction score threshold, with fixed heuristic gate boundaries. These are image-level technical quantities, not patient-level risk estimates.
Figure 16: Matched representation ablation. Locked test metrics for the orthogonality and no-penalty models; values are image-level estimates from the frozen ARCADE test environments.
Figure 17: Retrospective examples of evidence and constructed-fault states. Held-out angiography frames are paired with reference candidates, predicted candidates, predicted vessel support and reflected candidate-mask transformations for technical inspection only. Transformation labels are not independently adjudicated errors.
Conventional pixel-wise loss functions fail to enforce topological constraints in coronary vessel segmentation, producing fragmented vascular trees despite high pixel-level accuracy. We present ARIADNE, a two-stage framework coupling preference-aligned perception with RL-based diagnostic reasoning for topologically coherent stenosis detection. The perception module employs DPO to fine-tune the Sa2VA vision-language foundation model using Betti number constraints as preference signals, aligning the policy toward geometrically complete vessel structures rather than pixel-wise overlap metrics. The reasoning module formulates stenosis localization as a Markov Decision Process with an explicit rejection mechanism that autonomously defers ambiguous anatomical candidates such as bifurcations and vessel crossings, shifting from coverage maximization to reliability optimization. On 1,400 clinical angiograms, ARIADNE achieves state-of-the-art centerline Dice of 0.838, reduces false positives by 41% compared to geometric baselines. External validation on multi-center benchmarks ARCADE and XCAD confirms generalization across acquisition protocols. This represents the first application of DPO for topological alignment in medical imaging, demonstrating that preference-based learning over structural constraints mitigates topological violations while maintaining diagnostic sensitivity in interventional cardiology workflows.
Zhan Jin, Yu Luo, Yizhou Zhang +5
School of Mathematical Sciences, Ocean University of China, Qingdao, 266100, Shandong, China. · Department of Cardiology, Qilu Hospital (Qingdao), Cheeloo College of Medicine, Shandong University, No. 758 Hefei Road, Qingdao, 266000, Shandong, China.
Surgical resection remains the only potentially curative treatment for pancreatic ductal adenocarcinoma (PDAC), and eligibility depends on accurate assessment of vascular invasion (VI), i.e., tumor extension into adjacent critical vessels. Despite its importance for preoperative staging and surgical planning, computational VI assessment remains underexplored. Two major challenges are the lack of public datasets and the diagnostic ambiguity at the tumor-vessel interface, which leads to substantial inter-rater variability even among expert radiologists. To address these limitations, we introduce the CURVAS-PDACVI Dataset and Challenge, an open benchmark for uncertainty-aware AI in PDAC staging based on a densely annotated dataset with five independent expert annotations per scan. We also propose a multi-metric evaluation framework that extends beyond spatial overlap to include probabilistic calibration and VI assessment. Evaluation of six state-of-the-art methods shows that strong global volumetric overlap does not necessarily translate into reliable performance at clinically critical tumor-vessel interfaces. In particular, methods optimized for binary segmentation perform competitively on average overlap metrics, but often degrade in high-complexity cases with low expert consensus, either collapsing in volume or overextending at uncertain boundaries. In contrast, methods that model inter-rater disagreement produce better calibrated probabilistic maps and show greater robustness in these ambiguous cases. The benchmark highlights the limitations of volumetric accuracy as a proxy for localized surgical utility, motivating uncertainty-aware probabilistic models for preoperative decision-making.
M. Riera-Marín, O. K. Sikha, J. Rodríguez-Comas +23
BCN Medtech, Universitat Pompeu Fabra, Barcelona, Spain · Universitätsklinikum Erlangen, Department of Radiology of the Uniklinikum Erlangen (UKER), Erlangen, Germany · University Hospital Erlangen, Imaging Science Institute, Erlangen, Germany +10
We introduce and release ScienceBuddy, an interactive scientific research workspace that brings continually improving scientific agents into researchers' everyday workflows. ScienceBuddy supports researchers in carrying out scientific tasks while transforming their requests, feedback, and execution evidence into tasks and evaluation rubrics for continual learning. At its core is recursive-in-recursive self-improvement, a paradigm that couples harness evolution with model reinforcement learning: the inner recursion improves the harness with the model fixed, while the outer recursion trains the model under the improved harness. Harness evolution shapes training experience, and model learning creates new opportunities for harness adaptation. We present case studies of researcher interaction, harness refinement, and model learning, with the benchmark cases spanning four scientific task families. By releasing ScienceBuddy as a research product, we make this paradigm available to the scientific community and take a step toward discovery intelligence: scientific AI that advances through sustained collaboration with researchers and evolves alongside the research it supports. Website: http://science-buddy.io