Organizations: Wanyin Wu, Haoyun Zhao, and Dapeng Tao are with the School of Information Science and Engineering, Yunnan University, Kunming 650500, China. · Kanxue Li and Yibing Zhan are with the School of Computer Science, Wuhan University, Wuhan 430072, China. · Baosheng Yu is with the Lee Kong Chian School of Medicine, Nanyang Technological University, Singapore 639798. · Hua Jin is with First People’s Hospital of Yunnan Province, Kunming 650032, China.
Accurate prediction of surgical duration is pivotal for hospital resource management. Although recent supervised learning approaches-from machine learning (ML) to fine-tuned large language models (LLMs)-have shown strong performance, they remain constrained by the need for high-quality labeled data and computationally intensive training. In contrast, zero-shot LLM inference offers a promising training-free alternative but it lacks grounding in institution-specific clinical context (e.g., local demographics and case-mix distributions), making its predictions clinically misaligned and prone to instability. To address these limitations, we present PREBA, a retrieval-augmented framework that integrates PCA-weighted retrieval and Bayesian averaging aggregation to ground LLM predictions in institution-specific clinical evidence and statistical priors. The core of PREBA is to construct an evidence-based prompt for the LLM, comprising (1) the most clinically similar historical surgical cases and (2) clinical statistical priors. To achieve this, PREBA first encodes heterogeneous clinical features into a unified representation space enabling systematic retrieval. It then performs PCA-weighted retrieval to identify clinically relevant historical cases, which form the evidence context supplied to the LLM. Finally, PREBA applies Bayesian averaging to fuse multi-round LLM predictions with population-level statistical priors, yielding calibrated and clinically plausible duration estimates. We evaluate PREBA on two real-world clinical datasets using three state-of-the-art LLMs, including Qwen3, DeepSeek-R1, and HuatuoGPT-o1. PREBA significantly improves performance-for instance, reducing MAE by up to 40% and raising R^2 from -0.13 to 0.62 over zero-shot inference-and it achieves accuracy competitive with supervised ML methods, demonstrating strong effectiveness and generalization.
Figures & tables
Fig. 1: The proposed PREBA framework. for surgical duration prediction, consisting of three key modules: (i) Heterogeneous Biomedical Feature Embedding, (ii) PCA-Weighted Retrieval-Augmented Generation, and (iii) Bayesian Averaging Aggregation.
Fig. 2: PCA-based feature importance analysis showing the top 20 clinical features ranked by their contribution to surgical duration prediction.
Fig. 3: Key characteristics of In-hospital Dataset. (a) Patient and Surgical Characteristics: distributions of gender, age, and surgical grade. (b) Surgical duration distribution with key statistics. (c) Department-stratified duration patterns across 10 selected clinical departments.
Fig. 4: Structured prompt template for retrieval-augmented surgical duration prediction. The prompt integrates (1) system role definition, (2) similar case demonstrations retrieved via PCA-weighted similarity, (3) statistical priors from historical data, and (4) the query case.
Dataset
Count
Mean±Std
Range (min)
Train
125,369
155.8±83.7
[8, 810]
Validation
3,470
149.9±82.0
[17, 720]
Test
3,411
154.3±81.2
[11, 745]
Total
132,250
155.6±83.6
[8, 810]
TABLE I: Surgical duration statistics across dataset splits
Model Setting
Model
In-hospital Dataset
MMSDP [ 12 ]
MAE ↓
RMSE ↓
R 2 ↑
MAPE (%) ↓
MAE ↓
RMSE ↓
R 2 ↑
MAPE (%) ↓
Zero-Shot
Qwen3-8B [ 31 ]
58.62
86.49
-0.13
37.66
79.20
93.91
-1.63
41.68
DeepSeek-R1-8B [ 32 ]
62.29
93.32
-0.32
38.66
82.32
103.21
-2.14
42.89
HuatuoGPT-o1-7B [ 33 ]
52.77
76.81
0.11
32.72
81.70
101.04
-2.05
42.20
Random Few-Shot
Qwen3-8B [ 31 ]
48.85
74.80
0.15
31.40
52.68
73.27
-0.60
29.48
DeepSeek-R1-8B [ 32 ]
54.23
81.64
-0.01
32.22
48.08
65.52
-0.28
25.85
TABLE II: Performance comparison with various LLMs across different settings
Fig. 5: N-Shot Ablation Study. Unified Performance-Efficiency Analysis on Qwen3-8B. Comprehensive visualization showing MAE (blue) and RMSE (green) on left axis, R² (red) and MAPE (orange) on right axis, with inference time color gradient (light to dark) displayed above the x-axis.
Type
Model Setting
Model
MAE ↓
RMSE ↓
R 2 ↑
MAPE (%) ↓
ML
Full-Parameter Training
SVR [ 55 ]
44.63
67.78
0.30
30.51
Decision Tree [ 9 ]
49.80
70.20
0.25
35.89
Gradient Boosting Trees [ 56 ]
36.61
49.85
0.62
27.56
Bagging [ 35 ]
36.18
50.16
0.62
26.58
Random Forest [ 10 ]
35.01
47.98
0.65
26.11
Quantile Random Forest [ 34 ]
33.80
47.65
0.66
23.74
TABLE III: Performance comparison with machine learning methods
Model
Feature Encoding
MAE
RMSE
R 2
MAPE (%)
Qwen3-8B
Pure BERT
40.83
57.11
0.50
30.24
Heterogeneous Encoder
39.96
56.27
0.52
29.01
TABLE IV: Ablation study on feature encoding strategies
Model
W/O PCA Weighting
MAE
RMSE
R 2
MAPE (%)
Qwen3-8B
✗
39.52
56.13
0.54
29.13
✓
37.61
52.92
0.58
27.74
TABLE V: Ablation study on PCA-weighted retrieval
Model
Prior Statistics
MAE
RMSE
R 2
MAPE (%)
Qwen3-8B
✗
39.96
56.27
0.52
29.01
✓
37.61
52.92
0.58
27.74
TABLE VI: Ablation study on the effect of statistical prior knowledge
Model
W/O Postprocess
MAE
RMSE
R 2
MAPE (%)
Qwen3-8B
✗
39.96
56.27
0.52
29.01
✓
38.65
54.51
0.55
27.78
TABLE VII: Ablation study on the effect of post-processing
Fig. 6: Case study visualizations of PREBA on the in-hospital dataset using Qwen3-8B. The figure shows one query case (grey) and its top-8 retrieved historical surgeries (blue), together with their similarity scores and observed durations, illustrating how institution-specific evidence is assembled for the LLM. The bottom panel (yellow) summarizes feature consistency and reference 95% CI and ground-truth duration with the PREBA prediction.
Fig. 7: Ablation study on the effect of post-processing scale.
Fig. 8: Ablation study on the effect of multi-round prediction.
Model
Model Size
Performance Metrics
Time(s)
MAE
RMSE
R 2
MAPE (%)
Qwen3
4B
35.99
50.42
0.61
26.62
9.24
8B
35.19
50.08
0.62
25.39
11.68
14B
34.81
49.13
0.63
25.60
18.43
32B
33.43
47.57
0.66
24.26
26.51
TABLE VIII: Ablation study on model scale
Model
Strategy
MAE
RMSE
R 2
MAPE (%)
Qwen3-8B
Median
36.59
51.15
0.60
26.70
Majority Voting
36.95
51.70
0.59
27.01
Quantile Average
36.52
50.97
0.61
26.71
Simple Average
36.57
51.00
0.61
26.83
Bayesian Average (Ours)
35.19
50.08
0.62
25.39
HuatuoGPT-o1
Median
36.33
50.60
0.61
26.58
TABLE IX: Ablation study on prediction aggregation strategies
Model
Prior Weight
MAE
RMSE
R 2
MAPE (%)
HuatuoGPT-o1
0.3
35.44
49.48
0.63
25.93
0.6
35.09
49.36
0.63
25.51
0.9
34.98
49.59
0.63
25.23
1.2
35.08
50.07
0.62
25.07
1.5
35.34
50.73
0.61
24.99
TABLE X: Ablation study on the prior weight of Bayesian averaging
Longitudinal clinical notes contain rich evidence of how patients evolve over time, but converting this signal into training supervision for clinical prediction remains challenging. We extend Foresight Learning to clinical prediction by converting time-ordered MIMIC-III notes into examples consisting of past patient context, a natural-language question about a possible future event, and a label resolved from later documentation. This process yields 6,900 prediction examples from 702 admissions across medications, procedures, organ support, microbiology, and mortality. A small LoRA adapter trained on these examples improves over the prompted base model, reducing expected calibration error from 0.1269 to 0.0398 and Brier score from 0.199 to 0.145, while slightly outperforming GPT-5 point estimates on held-out questions. The approach enables reusable clinical prediction supervision from longitudinal notes without hand-engineered structured features or endpoint-specific classifiers.
Deploying clinical ML is slow and brittle: models that work at one hospital often degrade under distribution shifts at the next. In this work, we study a simple question -- can large language models (LLMs) create portable patient embeddings i.e. representations of patients enable a downstream predictor built on one hospital to be used elsewhere with minimal-to-no retraining and fine-tuning. To do so, we map from irregular ICU time series onto concise natural language summaries using a frozen LLM, then embed each summary with a frozen text embedding model to obtain a fixed length vector capable of serving as input to a variety of downstream predictors. Across three cohorts (MIMIC-IV, HIRID, PPICU), on multiple clinically grounded forecasting and classification tasks, we find that our approach is simple, easy to use and competitive with in-distribution with grid imputation, self-supervised representation learning, and time series foundation models, while exhibiting smaller relative performance drops when transferring to new hospitals. We study the variation in performance across prompt design, with structured prompts being crucial to reducing the variance of the predictive models without altering mean accuracy. We find that using these portable representations improves few-shot learning and does not increase demographic recoverability of age or sex relative to baselines, suggesting little additional privacy risk. Our work points to the potential that LLMs hold as tools to enable the scalable deployment of production grade predictive models by reducing the engineering overhead.
Zongliang Ji, Yifei Sun, Andre Amaral +2
University of Toronto, Canada · Vector Institute, Canada · Sunnybrook Health Sciences Centre, Canada
Task-specific lightweight models for surgical phase recognition excel at capturing temporal dynamics but generalize poorly under domain shift. Conversely, surgical foundation models (FMs) offer superior transferability via large-scale pretraining, yet their lack of explicit temporal modeling often yields temporally inconsistent predictions, leading to degraded performance. To exploit the complementary strengths of both paradigms, we propose Large-Small Temporal adaptation (LaST), a novel large-small collaborative framework that enables zero-shot adaptation to unseen clinical domains. In LaST, the FM initiates the pipeline by generating frame-level phase priors that serve as initial weak supervision. To effectively utilize these noisy phase priors, we introduce an iterative temporal refinement scheme that integrates dynamic quality control to filter reliable predictions and dual-model cross-learning to mitigate confirmation bias. Simultaneously, the lightweight model leverages its intrinsic temporal modeling ability to progressively correct inconsistent predictions and enhance overall accuracy across iterations. At the end, a cycle replay strategy is employed to close the loop: the refined, more accurate predictions are utilized as upgraded supervision signals for the subsequent iterations, fostering a self-reinforcing evolution of both label quality and model capability. Extensive experiments demonstrate that LaST achieves robust adaptation to unseen domains for zero-shot surgical phase recognition, outperforming the baseline (PeskaVLP) by 24.85%-43.17% in accuracy and even surpassing fully supervised linear probing and several state-of-the-art few-shot approaches. Codes will be released at https://github.com/YIYIZH/LaST.
Yiyi Zhang, Ying Zheng, Wenxin Fan +5
Department of Computer Science and Engineering, The Chinese University of Hong Kong, Hong Kong, China · Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong, China · Department of Surgery, The Chinese University of Hong Kong, Hong Kong, China +1