Agentic neural architecture search for clinical prediction from electronic health records: Cross-hospital knowledge transfer and computational efficiency
Authors: Deyi Li, Qi Xu, Lingyao Li, Tiansheng Wang, Muxuan Liang, Mei Liu
Organizations: Department of Health Outcomes and Biomedical Informatics, College of Medicine, University of Florida, Gainesville, FL, USA · College of Information Science, University of Arizona, Tucson, AZ, USA · Department of Pharmaceutical Health Outcomes and Policy, College of Pharmacy, University of Houston, Houston, TX, USA · Department of Epidemiology, Gillings School of Global Public Health, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA · Department of Biostatistics, University of Texas MD Anderson Cancer Center, Houston, TX, USA
Transformer architectures for clinical prediction from electronic health records (EHRs) often require costly manual tuning and may vary across hospitals and tasks. We introduce ATHENA (Agentic Transfer across Hospitals for EHR Neural Architecture Search), a knowledge-guided agentic neural architecture search framework for Transformer-based EHR modeling. A weight-sharing supernet is pretrained once per hospital, allowing candidate architectures to inherit weights and be evaluated through fine-tuning rather than independent pretraining. A two-layer cross-hospital prior supplies high-performing source architecture examples and task-specific component preferences summarized by SHapley Additive exPlanations (SHAP)-based meta-regression. These priors guide a multi-agent large language model search using target-hospital validation feedback. Across six clinical prediction tasks at one held-out OneFlorida+ site and one external MIMIC-IV site, ATHENA significantly outperforms all four baselines in 9 of 12 site-task evaluations under a strict equal-compute comparison. In a separate comparison using a common pretrained AutoFormer supernet to isolate search strategy performance, ATHENA ranks first in 9 of 12 evaluations at a maximum search budget of 30 candidate evaluations. It also shows more consistent architecture selection across repeated searches. ATHENA supports more efficient architecture selection for Transformer-based clinical prediction models.
Figures & tables
Fig 1: The ATHENA framework. Stage 1 evaluates sampled architectures across source hospitals and tasks to obtain architecture–performance metadata. The metadata yield a retrieval prior of high-performing source architectures and a SHAP-based architecture-effect prior. Stage 2 combines these priors with target-validation feedback to guide agentic NAS while adaptively balancing exploration and exploitation .
Fig 2: Illustration of the four configurable Transformer architecture parameters in the NAS space.
Aspect
Layer 1 : task-driven retrieval
Layer 2 : architecture-effect prior
Input
Source and target training-set task descriptors; evaluated source architecture–performance records.
Architecture–performance records for the same task across source hospitals.
Estimate cross-hospital performance associations for architectural choices.
Output
Top K=5 evaluated source configurations and their metrics.
SHAP-based level preferences, confidence intervals, feature importance, and pairwise rules.
Use
Concrete examples for candidate proposal and refinement.
Directional evidence for candidate proposal and refinement.
Table 1: Complementary roles of the two cross-hospital prior layers.
Agent
Implementation
Main function
Output
Proposal Agent Ap
LLM
Proposes architectures using task information, cross-hospital priors, search history, and the current strategy; revises proposals based on critique feedback.
Candidate architectures and design rationales.
Critic Agent Ac
LLM
Reviews proposed designs for constraint compatibility and novelty, flags concerns about prior preferences, and recommends revisions.
Structured critiques and revision recommendations.
Experiment Agent
Deterministic evaluator Ae and LLM strategy module As
Ae fine-tunes accepted subnetworks and computes target-validation metrics; As uses updated search feedback to select exploration or exploitation for the next round.
Validation records and the next-round strategy with its rationale.
Table 2: Roles of the three agents in ATHENA.
Fig 3: The Proposal Agent prompt template. The implementation populates each block with task-specific values before invoking the LLM.
Pretraining
Fine-tuning (Binary)
Cohort
Patients
Share
Train
Val.
Test
Prior source pool
Site A
22,186
42.0%
6,158
12,234
12,253
Site B
29,222
52.4%
5,300
10,566
10,647
Site C
37,152
42.0%
10,134
20,594
20,647
Site D
37,259
63.9%
4,204
8,409
8,410
Table 3: Pretraining-pool sizes and patient-level fine-tuning splits.
Cohort
Task
Train
Val.
Test
Prior source pool
Site A
Phenotype (6M)
11,083
8,147
8,405
Phenotype (12M)
12,235
9,124
9,100
Drug Recommendation †
78,999
59,249
59,251
Site B
Phenotype (6M)
9,229
6,892
6,921
Phenotype (12M)
10,285
7,792
7,643
Table 4: Pretraining-pool sizes and patient-level fine-tuning splits (continued).
Task
ρ (AUROC)
ρ (AUPRC)
Mortality
0.544
0.567
Stay > 7d
0.809
0.806
Readmission (3M)
0.570
0.412
Phenotype (6M)
0.951
0.944
Phenotype (12M)
0.940
0.944
Drug Recommendation
0.944
0.958
Table 5: Supernet ranking fidelity on MIMIC-IV.
OneFlorida+ held-out
MIMIC-IV
N
Shared
Scratch
Speedup
Shared
Scratch
Speedup
5
40
160
4.0 ×
34
150
4.4 ×
10
50
320
6.4 ×
39
300
7.7 ×
20
70
640
9.1 ×
49
600
12.2 ×
30
90
960
10.7 ×
59
900
15.3 ×
Table 6: Search compute under weight sharing, in GPU-minutes on one NVIDIA L4.
Target
Task
Random
EA
GENIUS
CoLLM-NAS
ATHENA
OneFlorida+
Mortality
58.95 ± 1.01
58.95 ± 1.01
59.38 ± 0.43
58.86 ± 2.07
58.85 ± 0.84
OneFlorida+
Stay > 7d
71.76 ± 2.31
71.76 ± 2.31
71.56 ± 0.33
71.16 ± 4.24
73.90 ± 0.27 ∗
OneFlorida+
Readmission (3M)
53.18 ± 0.38
53.18 ± 0.38
53.45 ± 0.43
53.20 ± 0.28
52.56 ± 0.24
OneFlorida+
Phenotype (6M)
29.42 ± 2.72
29.42 ± 2.72
29.76 ± 0.29
30.48 ± 0.96
31.06 ± 0.55
OneFlorida+
Phenotype (12M)
30.96 ± 2.37
30.96 ± 2.37
31.46 ± 0.14
31.70 ± 0.81
32.54 ± 0.40 ∗
OneFlorida+
Drug Recommendation
18.91 ± 0.54
18.91 ± 0.54
19.06 ± 0.15
19.08 ± 0.25
19.34 ± 0.07 ∗
Table 7: Test AUPRC across target hospitals and prediction tasks under an equal-compute comparison between ATHENA and baselines following the conventional pretrain-then-finetune paradigm, with all methods evaluated under a compute budget matched to that of ATHENA’s search with a maximum budget of 30 architecture evaluations (mean ± standard deviation (SD) over five random seeds).
Target
Task
Random Search
EA
GENIUS
CoLLM-NAS
ATHENA
(a) Search budget = 5 evaluations
OneFlorida+
Mortality
57.78 ± 0.90
57.78 ± 0.90
58.04 ± 0.34
57.72 ± 1.36
58.23 ± 1.72
OneFlorida+
Stay > 7d
73.25 ± 0.33
73.25 ± 0.33
74.00 ± 0.34
73.59 ± 0.45
74.03 ± 0.37
OneFlorida+
Readmission (3M)
52.09 ± 0.42
52.09 ± 0.42
52.06 ± 0.58
52.16 ± 0.40
52.26 ± 0.27
OneFlorida+
Phenotype (6M)
29.65 ± 0.59
29.53 ± 0.40
29.28 ± 0.87
29.29 ± 1.50
30.42 ± 0.51
OneFlorida+
Phenotype (12M)
30.86 ± 0.71
30.74 ± 0.69
30.39 ± 0.57
30.23 ± 0.60
32.17 ± 0.42
Table 8: Test AUPRC across target hospitals and prediction tasks under increasing NAS budgets of 5, 20, and 30 architecture evaluations (mean ± standard deviation over five random seeds).
Fig 4: Search trajectories on the held-out OneFlorida+ target. Panels show ( a ) Mortality, ( b ) Stay > 7d, ( c ) Readmission (3M), ( d ) Phenotype (6M), ( e ) Phenotype (12M), and ( f ) Drug Recommendation. Lines and shaded bands show the mean and SD, respectively, of the best validation AUPRC attained across five random seeds as a function of the number of evaluated architectures. Curves may end before the full budget when proposal saturation yields no new legal architecture.
Fig 5: Validation AUPRC versus parameter count on the held-out OneFlorida+ target. Markers identify the search method and the dashed line denotes the empirical Pareto frontier across evaluated architectures.
OneFlorida+
MIMIC-IV
Method
Params (M)
Evals
Params (M)
Evals
Random Search
1.28
30.0
2.34
30.0
EA
1.33
30.0
2.11
30.0
GENIUS
1.90
23.6
2.42
20.3
CoLLM-NAS
1.28
10.7
1.93
9.1
ATHENA
1.55
22.5
2.58
19.6
Table 9: Search behavior at a budget of 30 evaluations, averaged over the six tasks and five random seeds.
Target
Task
Random Search
EA
GENIUS
CoLLM-NAS
ATHENA
OneFlorida+
Mortality
—
(64, 2, 2, 2) (2)
(256, 2, 8, 1) (2)
—
—
OneFlorida+
Stay > 7d
—
—
—
—
(128, 4, 4, 1) (2)
OneFlorida+
Readmission (3M)
(128, 1, 8, 1) (2)
—
—
(128, 4, 4, 2) (2)
(128, 2, 8, 2) (2)
OneFlorida+
Phenotype (6M)
—
(256, 1, 8, 2) (2)
(256, 2, 4, 1) (2)
—
(256, 1, 4, 1) (2)
OneFlorida+
Phenotype (12M)
—
(256, 1, 8, 1) (4)
—
(128, 2, 4, 2) (2)
(256, 1, 8, 1) (4)
OneFlorida+
Drug Recommendation
(256, 2, 8, 2) (2)
(256, 2, 4, 2) (2)
(256, 2, 8, 4) (3)
(256, 4, 8, 2) (3)
—
Table 10: Modal architecture selected by each method, reported as the tuple (embed_dim, depth, heads, mlp_ratio), with the number of random seeds selecting that architecture (out of five) in parentheses.
Fig 6: Cross-hospital prior ablation on the external MIMIC-IV target. Bars and error bars show the mean and SD, respectively, of test AUPRC across five random seeds for exact task-matched retrieval, L1-only retrieval without the Layer 2 architecture-effect prior, LOTO Retrieval excluding exact task matches, Cold Start without either prior layer, and CoLLM-NAS as the representative baseline.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Mortality (binary): Whether the patient dies during the target admission ( 1 if a recorded in-hospital death time is present, 0 otherwise). The outcome is assessed only during the target admission, from admission to discharge or death, with no post-discharge follow-up.
Stay > 7d (binary): Whether the length of stay of the target admission exceeds seven days ( 1 if >7 days, 0 otherwise), measured from admission to discharge. The outcome is defined for the single target admission rather than as a cumulative measure across admissions, and all admissions are included regardless of discharge disposition.
Readmission (3M) (binary): Whether the patient has a subsequent hospital admission within 3 months (90 days) of the target admission ( 1 if a subsequent admission occurs within 90 days, 0 otherwise). The interval is measured from the discharge date of the target admission to the admission date of the subsequent admission.
Phenotype (6M) (multilabel, 18 classes): The 18 predefined phenotype categories present among the diagnoses recorded at the patient’s next hospital admission, provided that the next admission occurs within 6 months (180 days) of the target admission. Only target admissions followed by a qualifying next admission are included. The 18 categories are: (1) acute and unspecified renal failure; (2) acute cerebrovascular disease; (3) acute myocardial infarction; (4) cardiac dysrhythmias; (5) chronic kidney disease; (6) chronic obstructive pulmonary disease; (7) conduction disorders; (8) congestive heart failure (nonhypertensive); (9) coronary atherosclerosis and related; (10) disorders of lipid metabolism; (11) essential hypertension; (12) fluid and electrolyte disorders; (13) gastrointestinal hemorrhage; (14) hypertension with complications; (15) other liver diseases; (16) other lower respiratory disease; (17) pneumonia; and (18) septicemia (except in labor).
Phenotype (12M) (multilabel, 18 classes): The same 18 phenotype categories are used as in the 6-month task, but the next hospital admission must occur within 12 months (365 days) of the target admission. The label is an 18-dimensional binary vector, and only target admissions followed by a qualifying next admission are included.
Drug Recommendation (multilabel, 55 classes): The medication classes prescribed during the target admission, based on the admission diagnoses and the patient’s prior visit history. Ground truth consists of physician-prescribed drugs mapped to Anatomical Therapeutic Chemical (ATC) level-4 classes. Of the 140 ATC level-4 classes, the 55 classes present in ≥1% across admissions (prevalence, 1.0–21.8%) are retained. The resulting label is a 55-dimensional binary vector. For model input, diagnoses from the target admission are retained, whereas medication, laboratory, and procedure tokens from that admission are masked; all tokens from prior visits remain available. The 55 ATC level-4 classes are: (1) antithrombotic agents (B01A); (2) other analgesics and antipyretics (N02B); (3) stomatological preparations (A01A); (4) laxatives (A06A); (5) drugs for peptic ulcer and gastro-oesophageal reflux (A02B); (6) other mineral supplements (A12C); (7) lipid-modifying agents (C10A); (8) irrigating solutions (B05C); (9) calcium (A12A); (10) beta-blocking agents (C07A); (11) opioids (N02A); (12) antidepressants (N06A); (13) antiemetics and antinauseants (A04A); (14) ACE inhibitors, plain (C09A); (15) antacids (A02A); (16) thyroid preparations (H03A); (17) pancreatic hormones (H04A); (18) insulins and analogues (A10A); (19) selective calcium-channel blockers (C08C); (20) other beta-lactam antibacterials (J01D); (21) high-ceiling diuretics (C03C); (22) vitamin B12 and folic acid (B03B); (23) intestinal antiinfectives (A07A); (24) anxiolytics (N05B); (25) adrenergics, inhalants (R03A); (26) antiepileptics (N03A); (27) other cardiac preparations (C01E); (28) thiazide diuretics (C03A); (29) cardiac vasodilators (C01D); (30) topical antihemorrhoidals (C05A); (31) class I and III antiarrhythmics (C01B); (32) angiotensin-II receptor blockers (C09C); (33) quinolone antibacterials (J01M); (34) antipsychotics (N05A); (35) antigout preparations (M04A); (36) potassium (A12B); (37) intestinal anti-inflammatory agents (A07E); (38) penicillins (J01C); (39) general anesthetics (N01A); (40) arteriolar smooth-muscle agents (C02D); (41) anti-inflammatory and antirheumatic NSAIDs (M01A); (42) dermatological corticosteroids (D07A); (43) propulsives (A03F); (44) antipruritics (D04A); (45) hypnotics and sedatives (N05C); (46) other dermatological preparations (D11A); (47) drugs used in addictive disorders (N07B); (48) cardiac stimulants, excluding glycosides (C01C); (49) topical nasal decongestants (R01A); (50) potassium-sparing agents (C03D); (51) direct-acting antivirals (J05A); (52) sulfonamides and trimethoprim (J01E); (53) cough suppressants (R05D); (54) bacterial vaccines (J07A); and (55) blood glucose-lowering drugs, excluding insulins (A10B).
Appendix
Table S1: Definitions of the six clinical prediction tasks. Each task is defined relative to the target admission , i.e., the hospital admission being modeled. ACE denotes angiotensin-converting enzyme; NSAIDs denote nonsteroidal anti-inflammatory drugs.
OneFlorida+ Source Sites
Internal Target
External Target
Characteristic
Site A
Site B
Site C
Site D
Site E
MIMIC-IV
Demographics
Patients, n
52,831
55,735
88,527
58,282
84,479
60,706
Admissions, n
144,313
119,579
200,911
97,303
135,955
84,492
Female, n (%)
25,428 (48.13)
27,594 (49.51)
39,775 (44.93)
27,952 (47.96)
46,565 (55.12)
30,171 (49.7)
Utilization
Appendix
Table S2: Baseline characteristics of the six study cohorts. Sites A–D are OneFlorida+ source sites used to construct the cross-hospital architecture prior, Site E is the held-out OneFlorida+ internal target, and MIMIC-IV is the external target. Continuous variables are reported as mean (SD) or median [IQR] and categorical variables as n (%). SD denotes standard deviation and IQR denotes interquartile range. Comorbidities are reported as the number and percentage of admissions with the corresponding diagnosis.
Task
Feature pair
Rule supplied to the prompt
Mortality
embed_dim × heads
Prefer embed_dim ∈{128,256} , with the preference for 256 applying when heads ≥2 ; discourage embed_dim ∈{32,64} .
Stay > 7d
depth × embed_dim
Prefer depth ≥2 ; discourage depth =1 across all embed_dim levels.
Readmission (3M)
heads × embed_dim
Prefer heads ∈{4,8} ; discourage heads ∈{1,2} across all embed_dim levels.
Phenotype (6M)
embed_dim × depth
Prefer embed_dim ∈{128,256} ; discourage embed_dim ∈{32,64} across all depth levels.
Phenotype (12M)
embed_dim × depth
Prefer embed_dim ∈{128,256} ; discourage embed_dim ∈{32,64} across all depth levels.
Drug Recommendation
embed_dim × depth
Prefer embed_dim ∈{128,256} ; discourage embed_dim ∈{32,64} across all depth levels.
Appendix
Table S3: Layer 2 pairwise architecture-effect rules supplied to the search prompt ( Pmeta ). For each task, the interaction model uses the two most influential architecture features. The table reports feature-level combinations with estimated effects whose 95% confidence intervals (CIs) exclude zero.
Target
Task
Random Search
EA
GENIUS
CoLLM-NAS
ATHENA
OneFlorida+
Mortality
89.15 ± 0.72
89.71 ± 0.51
89.07 ± 1.15
88.94 ± 1.49
89.30 ± 0.95
OneFlorida+
Stay > 7d
90.32 ± 0.31
90.31 ± 0.13
90.46 ± 0.13
90.39 ± 0.30
90.30 ± 0.25
OneFlorida+
Readmission (3M)
70.22 ± 0.28
70.52 ± 0.48
70.38 ± 0.19
70.29 ± 0.37
70.51 ± 0.35
OneFlorida+
Phenotype (6M)
78.74 ± 0.55
78.82 ± 0.31
78.39 ± 0.32
78.18 ± 0.71
78.82 ± 0.25
OneFlorida+
Phenotype (12M)
79.86 ± 0.40
79.84 ± 0.44
79.64 ± 0.26
78.85 ± 0.25
80.05 ± 0.10
OneFlorida+
Drug Recommendation
71.21 ± 0.08
71.26 ± 0.08
71.31 ± 0.07
71.20 ± 0.18
71.28 ± 0.09
Appendix
Table S4: Test area under the receiver operating characteristic curve (AUROC) at a search budget of 30 architecture evaluations (mean ± standard deviation over five random seeds). Within each hospital–task comparison, the best mean is shown in bold and the second-best mean is underlined ; ties share the same mark. Avg. Rank denotes the mean rank across the 12 hospital–task comparisons (1 = best; ties receive the average rank), with the number of best-in-row comparisons shown in parentheses. EA denotes evolutionary algorithm.
Fig S1: Cross-hospital architecture prior learned by the Layer 2 meta-regression across four OneFlorida+ source sites. Rows denote the six prediction tasks: ( a ) Mortality, ( b ) Stay > 7d, ( c ) Readmission (3M), ( d ) Phenotype (6M), ( e ) Phenotype (12M), and ( f ) Drug Recommendation; columns denote architecture dimensions. Points show mixed-effects SHapley Additive exPlanations (SHAP) estimates with 95% CIs for each hyperparameter level. Green indicates preferred levels (CI > 0), red indicates discouraged levels (CI < 0), and gray indicates inconclusive levels whose CIs span 0. Gray points at zero without CIs denote reference levels.
Fig S2: Search trajectories on the held-out MIMIC-IV target for ( a ) Mortality, ( b ) Stay > 7d, ( c ) Readmission (3M), ( d ) Phenotype (6M), ( e ) Phenotype (12M), and ( f ) Drug Recommendation. Lines and shaded bands show the mean and SD, respectively, of the best validation AUPRC attained across five random seeds as a function of the number of evaluated architectures. Curves may end before the full budget when proposal saturation yields no new legal architecture.
Fig S3: Validation AUPRC versus parameter count on the held-out MIMIC-IV target for ( a ) Mortality, ( b ) Stay > 7d, ( c ) Readmission (3M), ( d ) Phenotype (6M), ( e ) Phenotype (12M), and ( f ) Drug Recommendation. Markers denote search methods, and the dashed line indicates the empirical Pareto frontier across evaluated architectures. Validation performance guided architecture search and selection; the selected architecture was evaluated only once on the held-out test set.
Fig S4: Cross-hospital prior ablation on the held-out OneFlorida+ target. Bars and error bars show the mean and SD, respectively, of test AUPRC across five random seeds for exact task-matched retrieval, L1-only retrieval without the Layer 2 architecture-effect prior, Leave-One-Task-Out (LOTO) Retrieval excluding exact task matches, Cold Start without either prior layer, and CoLLM-NAS as the representative baseline.
Automated question answering (QA) over electronic health records (EHRs) demands precise evidence retrieval, faithful answer generation, and explicit grounding of answers in clinical notes. In this work, we present Neural1.5, our method for the ArchEHR-QA 2026 shared task at CL4Health@LREC 2026, which comprises four subtasks: question interpretation, evidence identification, answer generation, and evidence alignment. Our approach decouples the task into independent, modular stages and employs DSPy"s MIPROv2 optimizer to automatically discover high-performing prompts, jointly tuning instructions and few-shot demonstrations for each stage. Within every stage, self-consistency voting over multiple stochastic inference runs suppresses spurious errors and improves reliability, while stage-specific verification mechanisms (e.g., self-reflection and chain-of-verification for alignment) further refine output quality. Among all teams that participated in all four subtasks, our method ranks second overall (mean rank 4.00), placing 4th, 1st, 4th, and 7th on Subtasks 1-4, respectively. These results demonstrate that systematic, per-stage prompt optimization combined with self-consistency mechanisms is a cost-effective alternative to model fine-tuning for multifaceted clinical QA.
Autoregressive foundation models for electronic health records can make predictions for diverse clinical tasks without finetuning. These methods work by generating possible "synthetic futures" for patients, and then inferring downstream predictions over these simulated trajectories. Despite their strong capabilities, these methods have several flaws. (1) Inference is extremely computationally expensive, as each prediction requires simulating many trajectories, each with many observations. (2) Their performance is noisy due to the variance inherent to simulation, which in particular reduces efficacy on rare events, which are often of high importance clinically. (3) They are not promptable, as their only input is the patient's medical history, requiring indirect aggregation at inference time. We introduce EveryQuery, a promptable foundation model for structured EHR data. Building on the insight that clinical prediction tasks can be written in a structured query language, EveryQuery takes a query as a prompt alongside the patient's history and estimates the outcome directly. This allows EveryQuery to make predictions across diverse clinical tasks, such as classification and survival modeling, without simulation or finetuning. Across three EHR datasets, EveryQuery outperforms a competitive autoregressive baseline on clinically relevant classification and time-to-event tasks, with mean AUROC gains of 14.3% and 12.7% on MIMIC-IV and a large academic medical center and comparable performance on the smaller NWICU dataset. Its advantage is largest for rare outcomes, and it requires 580x less inference compute.
Electronic health record (EHR) question answering is often handled by LLM-based pipelines that are costly to deploy and do not explicitly leverage the hierarchical structure of clinical data. Motivated by evidence that medical ontologies and patient trajectories exhibit hyperbolic geometry, we propose HypEHR, a compact Lorentzian model that embeds codes, visits, and questions in hyperbolic space and answers queries via geometry-consistent cross-attention with type-specific pointer heads. HypEHR is pretrained with next-visit diagnosis prediction and hierarchy-aware regularization to align representations with the ICD ontology. On two MIMIC-IV-based EHR-QA benchmarks, HypEHR approaches LLM-based methods while using far fewer parameters. Our code is publicly available at https://github.com/yuyuliu11037/HypEHR.
Yuyu Liu, Sarang Rajendra Patil, Mengjia Xu +1
Department of Computer Science, Stony Brook University · Department of Data Science, New Jersey Institute of Technology · Department of Biomedical Informatics, Stony Brook University