cs.LGOct 8, 2026

ActiveMedAgent: Cost-Aware Trajectory Learning for Multimodal Medical Diagnosis

Authors: Weiwei Ma, Xiaobing Yu, Peijie Qiu, Jin Yang, Zhaoqi An, Xuanzhao Dong, Xiaoqi Zhao, Xiaofeng Liu

Organizations: Washington University in St. Louis · Yale University · Icahn School of Medicine at Mount Sinai · Arizona State University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Clinical diagnosis is inherently sequential: clinicians escalate from cheap to costly tests only when additional evidence is expected to resolve diagnostic uncertainty. We present ActiveMedAgent, a framework that brings this cost-aware sequential logic to multimodal medical AI. Given a frozen, API-accessed vision-language model, ActiveMedAgent tracks probability distributions over candidate diagnoses and scores each acquisition by its per-step diagnostic utility minus cost. A lightweight MLP controller is then trained offline on these scored trajectories, learning when to request additional evidence and when to commit. Across three commonly used benchmarks, trajectory-based policy learning consistently outperforms both unguided acquisition and full-modality baselines. Notably, we identify an information overload effect. In 175 cases, the agent produces a correct diagnosis with fewer channels while the full-modality baseline fails, showing that learning what to omit can be as important as learning what to acquire.

Figures & tables

Figure 1: Successful selective acquisition case. PolicyNet commits after two low-cost evidence channels, while Full Modality receives all four channels and ranks the correct diagnosis lower. This illustrates the information-overload mechanism that motivates cost-aware trajectory learning.
Figure 2: ActiveMedAgent framework overview. Left: Diagnostic channels organized into a tiered cost hierarchy. Top: Trajectory collection via structured VLM interaction. Middle: Cost-aware policy learning for channel acquisition. Bottom: Inference-time decision loop with adaptive stopping.
Figure 3: PolicyNet vs. Full Modality. (a) Accuracy difference between PolicyNet and Full Modality. (b) MRR difference between PolicyNet and Full Modality. (c) Cost-efficiency gain: PolicyNet saves up to $1,768 per correct diagnosis (NEJM/GPT-4o-mini) by requesting fewer channels. Learned channel selection outperforms exhaustive information access while reducing acquisition cost.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

λ\lambdaTop-1Mean cost ($)Avg. K
0.00.625$7883.0
0.10.623$7712.9
0.50.621$7142.9
1.00.620$6552.8
2.00.618$5672.7
5.00.614$4152.5
Appendix
Table 11: Cost-penalty λ\lambda sweep. Policy gracefully trades accuracy for cost. At λ=5\lambda{=}5 , accuracy drops only 1.4 pp but cost drops 55%55\% .
Training →\to TestEvaluation setTop-1
NEJM (in-distribution)NEJM0.625
OLIVES (in-distribution)OLIVES0.660
NEJM →\to OLIVES (zero-shot)OLIVES0.610
OLIVES →\to NEJM (zero-shot)NEJM0.565
Random baselineeither0.200
Appendix
Table 14: Zero-shot dataset transfer with channel-name-embedding output head. Transfer loses ∼5\sim 5 pp but still beats random by a wide margin, demonstrating the policy is not purely dataset-specific.
Figure 4: Additional cost-penalty sweep supporting the cost–accuracy trade-off analysis.
Figure 5: Supplementary cost–accuracy trade-off plot showing top-1 accuracy with acquisition cost and cost per correct diagnosis.
Figure 6: Prompt paraphrase robustness across acquisition policies.
SubgroupTop-1ECEN
Pediatric (age < 18)0.5120.165n=42
Adult (18-64)0.5890.142n=98
Geriatric (>= 65)0.5470.151n=60
Male0.5840.139n=109
Female0.5660.148n=91
Image-dominant case0.6010.131n=130
Appendix
Table 19: Per-subgroup performance and calibration. Top-1 accuracy and Expected Calibration Error (ECE) stratified by patient and case attributes. ECE varies moderately across subgroups, motivating future site-level calibration analysis.
Figure 7: Additional trajectory visualization showing how belief states change as channels are acquired.
Figure 8: Attention heatmap used to analyze distractor sensitivity in open-weight backbones.
Figure 9: Channel-selection patterns learned by the acquisition policy.
Figure 10: Reliability analysis for belief-derived policy features.

Explore similar work

CardsList

  1. SCOPE-AD: Sequential cost-aware ordinal-belief planning with energy-based models for diagnostic agents

    Oct 1, 2026Ziwen Yu, Ivan Koychev, Elizabeth Coulthard +7Cost-Aware InferenceModel-Based Planning

    University of Bristol, Bristol, UK · Imperial College London, London, UK · Hunan Normal University, Changsha, China +1

  2. MedTRACE: Tool-Augmented Multimodal Clinical Reasoning Agents for Evidence-Grounded Decision-Making

    Sep 13, 2026Ji Lu, Lifei Liu, Haoran Yu +5Medical DiagnosisClinical Decision-Making

    Vanderbilt University, United States · Wichita State University, United States · University of Florida, United States +4

  3. MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments

    May 8, 2026Yicheng Gao, Xiaolin Zhou, Yahan Li +2HealthcareMedical Diagnosis

    University of Southern California · Arizona State University