Learning When to Trust Medical Knowledge: Regression-Safe Routing for Clinical Reasoning

Hui Zhang; Yujia Zeng; Yuhao Xin

cs.AIOct 14, 2026

Learning When to Trust Medical Knowledge: Regression-Safe Routing for Clinical Reasoning

Authors: Hui Zhang, Yujia Zeng, Yuhao Xin

Organizations: Beijing Institute of Technology · Shanghai Jiao Tong University · Xi’an Jiaotong University

Abstract

Clinical reasoning LLMs need external medical knowledge, but retrieved evidence and reasoning chains can be outdated, poisoned, or hallucinated. Harmful knowledge can cause regression: changing a correct answer into an incorrect one. We construct a medical knowledge-poisoning benchmark covering retrieved evidence and reasoning chains, with seven attack families filtered by a three-judge "subtle-but-wrong" vote and paired clean exams that separate poisoning effects from question difficulty. No fixed knowledge-use strategy is robust across these settings: strong RAG methods trade robustness to poisoned evidence against accuracy when evidence is useful. We formulate knowledge use as per-instance routing and introduce Med-TrustRouter, a lightweight controller that keeps the solver frozen. From leakage-free runtime signals, it estimates each action's expected correctness gain and uncertainty, preserving the base answer unless another action is confidently better. Across seven medical reasoning settings, Med-TrustRouter achieves 78.9 macro and 69.2 worst-case accuracy, matching the strongest competitor on macro accuracy while substantially improving the worst-case floor. Its gains transfer across solvers and evidence judges, unlike a hand-coded router using the same signals. Similar performance across linear, tree, and neural action-value heads suggests that the benefit comes from routing rather than estimator complexity. These results support learning when to trust external knowledge for robust clinical reasoning.

Figures & tables

Figure 1: Overview of Med-TrustRouter. For each instance, the system (i) runs five candidate knowledge-use actions, (ii) computes runtime signals from the candidates and the evidence judge, (iii) estimates per-action correctness, gain, and bootstrap uncertainty with per-action value heads, and (iv) selects an action through a risk-adjusted rule with a do-no-harm fallback and a trusted-evidence gate; the emitted answer is always one of the five precomputed candidates. Value heads and thresholds are fit offline on learn/dev splits. The solid path traces a clean literature-grounded instance: the judge marks $z$ trustworthy and the gate commits the candidate from Use-Retrieval , $\hat{y}_{1}$ . Dashed exits mark the two routes this instance skips: committing an action that passes the safety test, and the do-no-harm fallback to $a_{0}$ ; this fallback is the typical path of a poisoned instance, where the judge rejects $z$ and the actions disagree.

Adversarial setting	Channel	Families	Items
Med-Adv	evidence	4	2,400
Reflection-Adv	reasoning	3	1,503
MedReason-MedAdv	evidence †	4	241
MedReason-ReflAdv	reasoning †	3	301

Table 1: Benchmark settings used throughout the experiments. The four adversarial settings poison either retrieved evidence or generated reasoning chains, while MedQA, MedMCQA, and PubMedQA provide clean controls; counts are accepted instances after filtering. † Poisons use the harder MedReason source.

Single knowledge-use actions
Method	M-Adv	MMCQA	MedQA	MR-MAdv	MR-RAdv	PubMQA	Refl-Adv	Macro	Worst
Direct	86.41	74.90	85.71	72.94	68.04	48.13	88.67	74.97	48.13
Retrieval (RAG) ( Lewis et al. 2020 )	54.69	75.30	87.05	34.12	21.65	69.71	52.16	56.38	21.65
Reflection	57.82	74.90	84.82	41.18	30.93	72.20	60.43	60.33	30.93
Verifier	85.87	71.66	83.93	65.88	71.13	46.89	88.85	73.46	46.89
Heuristic / fixed-policy baselines

Table 2: Main results on the DeepSeek+glm test split (accuracy %). Rows compare single actions, heuristic/fixed policies, and recent RAG-style competitors under the same leakage-free protocol. Med-TrustRouter is the only method that leads on both macro and worst-case accuracy. Per-setting columns are seed 0, with Macro the mean and Worst the minimum over the seven settings. Baselines have no fitted controller and are reported from this single seed-0 run; Med-TrustRouter is refit across five seeds, so Macro and Worst are the five-seed means (

78.94{\pm}0.16

69.18{\pm}0.88

Figure 2: Macro versus worst-case accuracy under different call budgets. Each point is a method on the seven DeepSeek+glm test settings, and bubble area is the average number of LLM calls per instance; no lower-budget method reaches the worst-case floor of Med-TrustRouter.

Action-value backend	DeepSeek	Kimi
	Macro / Worst	Macro / Worst
Logistic (linear, default)	78.94 / 69.18	81.61 / 68.24
Gradient-boosted (LightGBM)	79.11 / 68.04	80.44 / 65.88
Neural MLP	78.83 / 68.00	81.52 / 67.06
Nearest-centroid	78.12 / 66.39	80.08 / 67.06

Table 3: Backend sensitivity (Macro/Worst accuracy %). Replacing the linear default with tree, neural, or centroid action-value heads keeps performance in a tight band, indicating a stable routing signal across estimator families.

Figure 3: Model transfer and rule transfer (Macro/Worst accuracy %). Each group changes the solver and/or evidence judge. The hand-coded router confirms that the runtime signals are informative on the solver for which it was designed, but it collapses on the small qwen3-8b solver whereas learned action-value routing does not.

Configuration	Macro	Worst
Med-TrustRouter (full)	78.77	68.24
$-$ Reconcile-Knowledge (4 actions)	76.89	63.92
$-$ Evidence gate	78.74	63.90
$-$ Macro calibration	78.80	68.24
$+$ Leaky features (no drop)	79.30	70.10

Table 4: Component ablations for the locked DeepSeek logistic controller at the seed-0 operating point (test split, accuracy %). Rows are compared against the seed-0 reference

78.77

68.24

, not the five-seed headline in Table 2 (

78.94

69.18

); leaky features are an invalid diagnostic, not a usable configuration.

Figure 4: Feature-importance transparency audit for the LightGBM backend. Bars show mean absolute TreeSHAP contribution share over action heads and test items; the dominant signals are runtime confidence, cross-action agreement, retrieval support, and evidence trust.

Figure 5: Per-instance action selection across the seven clean and adversarial settings. Bars show the fraction of test items routed to each action: clean exams mostly preserve the direct answer, while poisoned settings shift toward verification and knowledge reconciliation.

Figure S1: Poisoning benchmark construction. Clean multiple-choice seeds from MedQA and MedReason are retained as paired clean controls while a generator LLM synthesizes adversarial evidence or reasoning-chain artifacts. Gate 1 screens attack-family viability at the family level. Candidate artifacts then pass structure/entity checks, self-validation, and a three-judge subtle-but-wrong vote before entering the accepted poison pool. Gate 2 evaluates downstream efficacy by comparing naive consumers on accepted poisons against paired clean controls. The release contains clean controls, poisoned artifacts, channel/family metadata, and split manifests.

Setting	Channel	Attack families	Accepted	Evaluated
Med-Adv	Evidence	Outdated guideline; look-alike disease; contraindication flip; subtle distractor support	2,406	2,400
Reflection-Adv	Reasoning chain	Plausible-but-wrong CoT; premature conclusion; over-checking flip CoT	1,503	1,503
MedReason-MedAdv	Evidence	Same four evidence families on the harder MedReason source	241	241
MedReason-ReflAdv	Reasoning chain	Same three reasoning-chain families on the harder MedReason source	301	301

Table S1: Accepted poisoned-artifact counts (“Accepted”) and evaluated release counts (“Evaluated”) from the generation logs. Med-Adv has 2,406 accepted artifacts available in the raw generation pool; the evaluated release is balanced to 2,400 artifacts.

Family	MedQA	MedReason	Poisoned-artifact behavior
Evidence channel
Outdated guideline	600	60	Plausible but outdated or superseded clinical framing.
Look-alike disease	604	60	Shifts interpretation toward a clinically similar distractor.
Contraindication flip	600	60	Reverses treatment or contraindication implications.
Subtle distractor support	602	61	Selectively supports an incorrect answer option.
Reasoning-chain channel

Table S2: Per-family accepted counts in the artifact generation logs, split by source dataset (MedQA-derived vs. MedReason-derived items).

Subset	Items	Plaus. ${=}2$	Subtle ${=}2$	W.-supp. ${=}2$	Strict	Perm.
All audited artifacts	196	91.3%	37.2%	85.7%	33.2%	93.9%
Evidence channel	112	97.3%	65.2%	82.1%	58.0%	89.3%
Reasoning channel	84	83.3%	0.0%	90.5%	0.0%	100.0%
contraindication_flip	28	100.0%	92.9%	92.9%	92.9%	96.4%
look_alike_disease	28	100.0%	85.7%	67.9%	67.9%	85.7%
subtle_distractor_support	28	96.4%	82.1%	75.0%	71.4%	82.1%

Table S3: Human medical validation results from a single independent medically-trained reviewer over 196 accepted artifacts. The three top-level columns report the fraction scored at the top level (

{=}2

) for medical plausibility, subtlety, and wrong-option support; the last two columns report the strict and permissive pass rates. Strict pass requires all three scores at 2; permissive requires all three

{\geq}1

. Wrong-option support uses the pre-defined target when available, otherwise any non-gold option.

Alias	Role	Developer (API host)	Decoding
DeepSeek-V4-Pro	Main solver	DeepSeek (own API)	temp. $0$
glm-5	Main evidence judge	Zhipu GLM (DashScope)	temp. $0$
Kimi-k2.5	Transfer solver; poison generator	Moonshot Kimi (DashScope)	temp. $0$
Qwen3.7-Plus	Transfer evidence judge	Alibaba Qwen (DashScope)	temp. $0$
qwen3-8b	Small open solver	Alibaba Qwen (DashScope)	temp. $0$

Table S4: Hosted models used in the experiments, accessed through provider inference APIs in June 2026 (DeepSeek: api.deepseek.com ; DashScope: dashscope.aliyuncs.com ). Aliases match the released model_registry ; each is the provider’s published version at that time, run with reasoning traces disabled.

Threshold	Candidate values				Locked
Risk aversion $\lambda$	$0.00$	$0.25$	$0.50$	$1.00$	$0.25$
Uncertainty ceiling $u$	$0.30$	$0.45$	$0.60$	$1.00$	$0.30$
Score floor $\delta$	$-0.05$	$0.00$	$0.02$	$0.05$	$-0.05$
Margin $m$	$-0.05$	$0.00$	$0.02$		$0.02$

Table S5: Threshold calibration grid. The locked operating point is selected on the dev split under the harm-rejection floor and reused unchanged at test time.

Method class	Uses poison artifact	Extra judge	Dev tuning	Recorded evidence
Direct answer	No	No	No	Prompt in Section B.2 ; parser in the baseline cache.
Single retrieval action	Evidence channel only	No	No	Evidence prompt, parser, and cached outputs.
Self-reflection action	Reasoning channel only	No	No	Reflection prompt, chain source, and cached outputs.
Verifier action	Candidate answer + question	Yes	No	Verifier prompt, judge model-family separation, and answer-selection rule.
Recent RAG competitors	Method dependent	Method dependent	Only if reported	Prompt, call budget, poisoned-artifact access policy, and multiple-choice adaptation notes.
Learned router	Cached actions + runtime signals	Evidence judge only	Thresholds on dev	Feature list, forbidden-feature filter, threshold grid, and locked seed protocol.

Table S6: Reproduction information required for each method class.

Family	#	Features
Evidence judge scalars	15	evidence_available ; evidence_reliability ; evidence_relevance_score ; evidence_sufficiency_score ; evidence_consistency ; direct_option_support ; prompted_option_support ; retrieval_top_score ; retrieval_margin ; evidence_poison_signal ; evidence_poison_unknown ; evidence_predicted_poison ; reflection_poison_signal ; reflection_poison_unknown ; reflection_predicted_poison .
Evidence judge one-hot	19	evidence_judge_trust_ {trust, reject, uncertain, no_evidence, missing}; evidence_judge_relevance_ {high, partial, low, none, missing}; evidence_judge_sufficiency_ {sufficient, partial, insufficient, none, missing}; reflection_judge_trust_ {trust, reject, uncertain, missing}.
Agreement	12	prompted_equals_direct ; naive_equals_direct ; verifier_equals_direct ; prompted_disagrees_direct ; naive_disagrees_direct ; verifier_disagrees_direct ; verifier_equals_reflection ; verifier_disagrees_reflection ; reflection_is_outlier ; n_agree_with_direct ; direct_is_majority ; verifier_equals_majority .
Confidence	2	direct_confidence ; direct_vote_entropy .
Validity	5	has_prompted_rag ; has_naive_reflection ; has_verifier ; prompted_valid ; naive_valid .
Excluded (leakage)	21	question_chars ; question_tokens ; option_count ; context_chars ; dataset_ {medqa, medmcqa, pubmedqa, medreason, med_adv, reflection_adv}; evidence_status_ {none, clean, conflicting, poisoned, unknown}; harm_type_ {none, retrieval, reflection, verifier, counterfactual, constraint}.

Table S7: The

53

leakage-free router features by family, and the

21

excluded fields (dataset identity, evidence-status and harm-type labels, and raw length/structure counts) that would let the controller key on the benchmark rather than the instance. The forbidden set is enforced by the feature filter used in training and evaluation.

Control	Calls/question	Macro	Worst	Interpretation
Five-action plurality vote	5.81	75.81	59.75	Same action pool and budget, no learned routing.
Five-action confidence-weighted vote	5.81	77.20	51.87	Same action pool and budget, non-learned weighting.
Five-action per-instance oracle	5.81	90.88	81.33	Non-deployable upper bound for the action pool.
Med-TrustRouter	5.81	78.94	69.18	Learned router, five-seed headline.

Table S8: Same-action-pool controls (accuracy %). All rows use exactly the five candidate actions Med-TrustRouter routes over, at the same

5.81

-call budget. Non-learned plurality and confidence-weighted votes trail the learned router on both axes—most sharply on worst-case—while the oracle marks the pool ceiling. Controls are deterministic over the locked candidate-action cache; the Med-TrustRouter row is the five-seed headline.

Ablation	Macro	Worst	Purpose
Full locked Med-TrustRouter seed-0	78.77	68.24	Reference operating point for appendix ablations.
$-$ Evidence-judge signal	77.30	56.02	Trust, relevance, sufficiency, poison features: decisive for worst-case.
$-$ Agreement signal	75.57	58.76	Cross-action agreement features: largest macro drop.
Internal-only action pool	76.62	61.83	Drop retrieval + reconcile: heterogeneity matters ( $-6.4$ worst).
Five-action plurality vote	75.81	59.75	Learned routing replaced by naive same-budget vote.

Table S9: Extended ablations at the locked seed-0 operating point (accuracy %). Rows are compared against the seed-0 reference

78.77

68.24

(not the five-seed headline). Removing either routing signal, homogenizing the action pool, or replacing learned routing with a naive same-budget vote each collapses worst-case accuracy, locating the gain in the leakage-free routing signal, action-pool heterogeneity, and learning.

Adversarial setting	C $\to$ C	C $\to$ W	W $\to$ C	W $\to$ W	AHR	Regr.	Corr.
	Outcome counts (base $\to$ router)				Rates (%)
Med-Adv	769	32	48	78	91.79	4.00	38.10
Reflection-Adv	465	28	31	32	88.19	5.68	49.21
MedReason-MedAdv	54	8	4	19	82.98	12.90	17.39
MedReason-ReflAdv	59	7	10	21	85.71	10.61	32.26

Table S10: Regression decomposition for the locked seed-0 Med-TrustRouter test predictions. C and W mark correct and wrong answers before (base) and after (router) routing. AHR, regression rate, and correction rate are percentages. Values are reconstructed from the same locked cache used for the main-table seed-0 per-setting cells.

Figure S2: Evidence-judge noise sensitivity for the locked logistic controller (test split accuracy %). We randomly flip judge-derived trust and poison features before rerouting; the smooth decline shows that the judge is useful but not an all-or-nothing oracle.

Method	Calls/q	Input tokens/q	Output tokens/q	USD/1k q	Notes
Direct	1.00	237.10	48.68	0.866	Single action.
Retrieval	1.00	487.74	67.97	1.520	Single retrieval action.
Reflection	2.00	448.74	50.80	1.306	Answer, then self-reflect.
Verifier	2.00	0.00	83.90	0.672	Answer, then independent verify; input tokens are a lower bound (final output only cached).
Astute RAG	2.00	0.00	103.21	0.824	Recall, then reconcile; input tokens are a lower bound (final output only cached).
Med-TrustRouter routing cache	5.81	1715.05	438.64	6.942	Sum of direct, retrieval, reflection, verifier, evidence verification, and reconciliation components.

Table S11: Per-question cost estimates. Token columns are character-length estimates of stored prompt/output text under deepseek-v4-pro pricing, not billed usage; input tokens for Verifier and Astute RAG are lower bounds, since only their final outputs were cached.

Method (options hidden)	Macro	Worst	Note
Med-TrustRouter (frozen)	80.63	66.67	MCQ-trained controller, no retraining.
Reconcile-Knowledge (Astute)	80.28	64.71	Strongest fixed action on this split.
Preserve-Direct	78.81	60.61	Base answer; do-no-harm anchor.
Verify-Answer	78.38	56.06	Independent judge, question only.
Use-Retrieval	60.17	23.81	Collapses under poisoned evidence.
Reflection	60.14	23.81	Collapses under poisoned reasoning.

Table S12: Open-form generation probe (accuracy %). All actions answer with the options hidden; free-text answers are mapped back to the option set by a fixed matcher that never sees the evidence, and the controller is the locked seed-0 model applied without retraining. Scored on the

282

504

sampled items on which every action produced an option-mappable answer.

Figure S3: Routing-path utilization on the

2377

-item DeepSeek+glm test split (locked logistic controller, seed 0). Each bar gives, for the overall split and for each of the seven settings, the share of instances taking the trusted-evidence gate, the general risk-adjusted path, or the do-no-harm fallback to

a_{0}

. Settings are ordered by increasing fallback share. The gate concentrates on clean literature-grounded PubMedQA, while the do-no-harm fallback dominates the poisoned Med-Adv and Reflection-Adv settings.

Medical Reasoning

Medical reasoning studies how AI systems use clinical evidence, patient context, and multimodal data to make and explain medical decisions.

See all 31 papers

FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning
Jul 1, 2026Zhiyun Zhang, Liwen Sun, Xiang Qian +1RL for Language Model Reasoning Process Supervision
Carnegie Mellon University · Stanford University School of Medicine · Xlue
Medical Reasoning with Large Language Models: A Survey and MR-Bench
Mar 17, 2026Xiaohan Ren, Chenxiao Fan, Wenyin Ma +4Medical Reasoning Clinical Reasoning
School of Cyber Science and Technology, University of Science and Technology of China, Anhui, China · School of Artificial Intelligence and Data Science, University of Science and Technology of China, Anhui, China · Department of Infectious Disease, The First Affiliated Hospital of USTC, Anhui, China +2
MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoning
Aug 31, 2026Jiangwang Chen, Chenghao Zhang, Hengxing CaiLLM Hallucination Mitigation Evidence-Grounded Reasoning
Tsinghua University · DP Technology

Jul 1, 2026J/K move · Enter open · S save

FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning

Zhiyun Zhang, Liwen Sun, Xiang Qian, Chenyan Xiong

Carnegie Mellon University · Stanford University School of Medicine · Xlue

RL for Language Model Reasoning Process Supervision Medical Reasoning Medical LLMs

Open paper AskPDF arXiv

Faithful reasoning is essential in medicine, where clinical decisions require transparent justification grounded in reliable evidence. Current medical LLMs either lack active access to evidence or use retrieved evidence without supervising how it should be appraised and applied during reasoning. To address this, we formalize evidence-based medicine principles as process-level criteria and introduce FaithMed, a framework that combines clinician-designed, automatically refined rubrics with reinforcement learning using step-level process reward assignment and advantage grouping. Across seven medical benchmarks, FaithMed improves over agentic-search baselines (+9% on average) and outcome-only RL (+5.8%), while raising average evidence-based medicine rubric scores over agentic-search Qwen3 baselines (+15.5%). This work demonstrates that explicit step-level supervision can improve both task success and the faithfulness of the reasoning process. Code is available at https://github.com/cxcscmu/FaithMed.

Abstract

Medical Reasoning

Explore similar work

FaithMed: Training LLMs For Faithful Evidence-Based Medical Reasoning

Medical Reasoning with Large Language Models: A Survey and MR-Bench

MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoning