Tool-augmented medical agents treat whatever their tools return as evidence. We show that this default is costly: medical tools fail silently, and a frozen reasoner cannot ignore what enters its context, so many cases it would have answered correctly are flipped to wrong once tool outputs are added. We call this evidence hijacking. V-information explains the phenomenon: the reasoner has lost the option to ignore, so a tool output can carry negative usable information, and moving the decision of what to keep outside the reasoner restores that option. The theory further shows that part of any filtering gain comes merely from letting the reasoner see less, and only selection by content lets it see better. Building on this, we recast tool-augmented reasoning as evidence governance, which separates deciding what counts as evidence from the reasoner that answers, and propose context-independent evidence governance (CIEG), the first medical agent framework that rules, after acquisition, on whether each individual tool output may enter as evidence. Every tool output is written as a card that can be checked against the raw input; an independent checker that sees only the input and the cards decides which cards become evidence; and the reasoner answers from the admitted cards alone, with no training anywhere. On medical question answering, fundus glaucoma screening, and chest X-ray pneumonia classification, it achieves the highest accuracy among models of comparable scale, and significantly outperforms state-of-the-art tool-using agents and multi-agent systems built on the same backbone at lower cost.
Figures & tables
Figure 2: Context-independent evidence governance (CIEG). Acquisition: tool agents call their tools on the input and write each result as a verifiable tool card, which shows the finding together with the material it rests on. Admission: the Checker checks the cards for fidelity to input and for consistency with one another, and rules on which cards are admitted. Answering: the Solver answers from input and the admitted cards alone, and its decisions are combined by vote.
Imaging ⋅ bAcc
Text ⋅ Acc (%)
Method
Glaucoma
Chest X-ray
MedQA
MedMCQA
NEJM
General MLLMs
Qwen3-VL-4B
0.620 †
0.776 †
66.77 †
58.71 †
47.94 †
Qwen3-VL-8B (backbone)
0.636 †
0.676 †
74.94 †
64.40 †
55.73 †
InternVL3-8B
0.511 †
0.692 †
64.34 †
56.61 †
46.87 †
Medical MLLMs
Table 1: Main results. bAcc on imaging tasks; accuracy (%) on the three medical QA datasets. † : CIEG is significantly better ( p<0.05 ). Best per column in bold. General and medical MLLMs, like ours, take the majority vote over 8 samples; the other baselines follow their original pipelines. ∗ Training data include chest X-rays and may overlap in source with the test set. § Official 13B weights. ‡ Reproduced with official settings on the text-only Qwen3-8B [ 52 ] , as Qwen3-VL-8B lacks the thinking mode its retrieval queries require.
Glaucoma
Chest X-ray
Med. QA
Method
Tokens
Time
Tokens
Time
Tokens
Time
MedAgents
55.9k
39.6
82.8k
50.2
72.8k
37.9
MDAgents
164.6k
120.4
40.8k
19.7
78.3k
155.1
MAM
36.6k
51.0
54.9k
52.7
38.6k
50.2
Ours
25.3k
5.5
33.3k
8.0
34.6k
39.0
Table 2: Per-case cost. Tokens: prompt plus generation per case over the full test set. Time: median end-to-end latency (s) on a 50-case subset.
Figure 3: Hijacking grows with erroneous evidence. Glaucoma screening accuracy on 400 cases, grouped by the number of erroneous tool cards per case (out of 3). Gray: no cards; orange: admit all; blue: CIEG. Error bars are 95% bootstrap confidence intervals; the shaded band marks the gap between admit-all and ours.
Admission
Glaucoma
Chest X-ray
Med. QA
No cards g0
0.636 †
0.676 †
65.02 †
Admit all g1
0.627 †
0.821 †
72.31 †
Budget-matched rand. π(g)
0.637 †
0.803 †
71.82 †
Rejected only
0.659 †
0.579 †
69.66 †
Ours
0.730
0.837
72.86
Table 3: Same card pool, same Solver; only the admitted cards change. Metrics as in Tab. 1 . No cards uses the same pipeline and answer template as ours, with the Checker rejecting every card. Budget-matched random admits, per case, as many random cards as ours. Rejected only admits only the cards the Checker rejects.
Figure 4: Theory meets practice. (a) Budget term B , selection term S , and total gain T of Eq. 6 on the three tasks (nats, 95% CI). On imaging tasks, all card subsets of each case are enumerated to compute I(π(g)) exactly; on question answering, it is estimated by budget-matched random sampling with a single Solver sample. (b) First-order term of Eq. 7 versus the measured selection term. Each point is a gate (483 on glaucoma, 335 on chest X-ray); the star marks CIEG; the dashed line is y=x .
Configuration
Glaucoma
Chest X-ray
Med. QA
No cards g0
0.636 †
0.676 †
65.02 †
ReAct (judge in context)
0.544 †
0.754 †
67.29 †
Solver calls tools itself
0.606 †
0.814
69.11 †
Ours
0.730
0.837
72.86
Table 4: Where to judge tool outputs. Same backbone and tool pool.
Checker input
AUROC ↑
Adm. corr. ↑
Adm. err. ↓
S↑
Random gate
0.50/0.50
–
–
0
Cards only
0.87 † /0.81 †
83/74
27/31
+ 0.105 †
Input + cards (ours)
0.93 / 0.85
87 / 76
19 / 22
+ 0.152
Table 5: Checker quality on glaucoma. A card is erroneous if its vCDR deviates from the ground truth by more than 0.10. AUROC and admission rates (Adm., %) of correct and erroneous cards are within class (glaucomatous/non-glaucomatous). S is the selection term of Eq. 6 .
Foundation models can serve as clinical agents through tool-use harnesses. However, conventional medical benchmarks assess reasoning over preselected evidence rather than the ability to seek it across clinical records and longitudinal imaging. We propose CASE: a series of role-specific Clinical Agents for Seeking Evidence, together with a tool-use harness and an agentic post-training framework for compact vision-language policy models. We further introduce a longitudinal multimodal benchmark built on UK Biobank, comprising 50,401 clinical questions derived from real-world ICD-10-coded diagnoses of 4,739 participants. Each question links to a patient-specific environment containing clinical context and multi-sequence MRI from baseline and follow-up visits, where agents autonomously select which visits, organs, modalities, slices, and specialist tools to inspect and compare. Supervised fine-tuning transfers evidence-seeking workflows from 147,434 frontier-model interaction trajectories, followed by agentic reinforcement learning on the learner's own environment interactions. Privileged on-policy self-distillation and rubric-based LLM feedback refine evidence-to-conclusion reasoning without prescribing tool sequences. Experiments show that CASE moves beyond question-answer imitation toward transferable investigation policies, strengthening evidence-grounded longitudinal reasoning. Under matched evaluation conditions, our Qwen3-VL-8B-Instruct based agent achieves over 16% and 10% relative improvements in answer accuracy over GPT-5.4 and Claude Opus 4.8. Project page available at: https://vinyehshaw.github.io/case-project/.
Minye Shao, Chaohui Yu, Yixuan Wu +6
Durham University · DAMO Academy, Alibaba Group · Hupan Laboratory +1
Chest X-ray (CXR) interpretation often requires integrating visual findings with task-specific evidence rather than relying on direct visual recognition alone. While tool-augmented medical agents can supply such evidence via external modules, they often lack explicit mechanisms for learning when auxiliary evidence is needed and for preventing inapplicable quantitative evidence from entering diagnostic reasoning. To address this, we propose X-Ragent, a post-training framework for tool-augmented reinforcement learning that decouples learned auxiliary-evidence seeking from validity-gated quantitative measurement acquisition for evidence-grounded CXR reasoning. We construct Rad-CoT, a 7.7K-sample corpus combining causally ordered tool-augmented trajectories with report-style diagnostic reasoning, and use it to initialize Qwen3-VL-8B via agentic supervised fine-tuning (SFT). We then optimize the policy with Group Relative Policy Optimization (GRPO) using an accuracy-gated reward that applies process-level terms for measurement consistency and auxiliary-tool cost only to correct trajectories, preventing plausible process behavior from compensating for an incorrect diagnosis. Across three CXR benchmarks, X-Ragent achieves the strongest results, including an 8.6-point improvement on the fully held-out ChestAgentBench. By seeking evidence selectively and admitting only valid measurements, X-Ragent supports more faithful and evidence-grounded CXR reasoning.
Yi Wang, Zhenlong Yuan, Yuchan Guo +9
Institute of Computing Technology · University of Chinese Academy of Sciences · Carnegie Mellon University +6
Medical AI agents increasingly use external tools for diagnosis, treatment recommendation, and evidence retrieval, yet most existing approaches assume that task-appropriate tools are reliable within their intended scope. This assumption is fragile in real clinical settings, where even relevant tools may fail on challenging instances and lead to unsafe downstream decisions. To address this issue, we study medical tool use under imperfect-tool settings to correct failure instances missed by individual tools. Instance-dependent failure patterns create a gap between the best fixed single tool and an ideal instance-wise selector, which we refer to as the Single-Oracle risk gap. The core challenge is that conventional task-level tool selection cannot realize this gap, as it is inherently bounded by the performance of the best single tool. Motivated by this observation, we therefore account for instance-level heterogeneity and formulate tool use as an instance-level selection problem. Particularly, we propose a GRPO-based reinforcement learning framework with rewards for probabilistic risk minimization and disagreement-aware synergy learning, which promotes instance-level correction of erroneous tool consensus. Furthermore, an entropy-guided sampling strategy is adopted to upweight high-disagreement instances, which provide stronger signals for learning instance-specific tool synergy. These two components complement each other in mitigating instance-level heterogeneity and improving tool synergy. Experiments on two tasks and seven medical benchmarks show that our method consistently achieves robust and stable improvements over a broad range of baselines, highlighting the importance of synergy-aware tool use for reliable medical agentic systems.
Yunhui Gan, Tan Pan, Kaiyu Guo +5
Fudan University · Shanghai Academy of Artificial Intelligence for Science · Shanghai Innovation Institute +2