Latent Decision Flow Rewards for Clinical Guideline Reasoning

Gaurav Bhatt; Marinka Zitnik

cs.LGOct 28, 2026

Latent Decision Flow Rewards for Clinical Guideline Reasoning

Authors: Gaurav Bhatt, Marinka Zitnik

Organizations: Harvard University

Abstract

Clinical practice guidelines encode evidence-based recommendations as conditional decision flows, which makes them a natural source of process supervision for training reasoning models. Two properties of clinical reasoning make this supervision difficult to use. A decision flow consistent with a guideline does not by itself determine the correct answer for a given patient, which we call the path--answer gap, and several valid decision flows can support the same clinical decision, which we call path multiplicity. Current training approaches address parts of this problem. Answer rewards supervise the final decision, and process rewards computed against a reference flow supervise one valid decision flow per case. Supervised fine-tuning on annotated flows teaches guideline-consistent reasoning but constrains the model to one trajectory per case and can reduce answer accuracy. We introduce GUIDER, a method that jointly optimizes a decision flow controller and a reasoning policy. The controller learns to generate plausible decision flows from guideline-derived supervision, and the policy learns to generate a decision flow and a final answer for each patient case. GUIDER computes the process reward for the policy over several controller-generated flows, which accounts for path multiplicity, and gates this reward on answer correctness, which addresses the path--answer gap. Joint training allows the process supervision to evolve with the policy. On MedGUIDE, GUIDER with Qwen3-14B achieves 82.5 weighted accuracy and 73.5 guideline adherence, 3.3 and 3.1 points above the strongest clinical reinforcement learning baseline. With Qwen3-14B and Gemma4-12B backbones, GUIDER also improves over CRPO, MRPO, and FaithMed on the out-of-domain benchmarks AMEGA-LLM, RealMedQA, and CancerGUIDE.

Figures & tables

Figure 1: Path multiplicity and path–answer gap in guideline decision flows. Left. A hypothetical patient case in which two valid decision flows lead to the same clinical answer, which illustrates path multiplicity. Right. A clinical question on the use of high-sensitivity troponin testing. Decision flows from the NICE and SBC guidelines support the same answer, which illustrates path multiplicity, and the complete answer extends beyond the final step of each flow, which illustrates the path–answer gap. The NICE flow follows NICE CG95 (Recommendations 1.2.5.1–1.2.5.7) ( National Institute for Health and Care Excellence, 2016 ) , and the SBC flow follows the Brazilian Society of Cardiology guideline (Table 4) ( Feitosa-Filho et al., 2015 ) .

Figure 2: Overview of GuIDeR . The controller learns to generate decision flows from guideline-derived reference flows and samples several candidate flows for each patient case. The policy generates its own decision flow and final answer from the patient case. GuIDeR computes an answer-gated process reward from the agreement between the policy flow and the controller flows, and it optimizes the policy with GRPO and the controller with guideline supervision in a joint objective.

Training Strategy	MedGUIDE		AMEGA-LLM	RealMedQA	CancerGUIDE
Training Strategy	WAcc.	Adh.	AMEGA-LLM	RealMedQA	CancerGUIDE
Base (Qwen3-14B)	52.3	47.2	17.2	1.62	1.71
SFT (ans)	78.0	53.1	20.3	1.73	1.73
SFT (ans) $\rightarrow$ GRPO (ans)	80.1	54.5	20.5	1.81	1.75
SFT (ans+flow)	64.0	71.3	18.8	1.82	1.80
SFT (ans+flow) $\rightarrow$ GRPO (ans+flow)	68.2	71.6	19.1	1.85	1.82

Table 1: Comparison of training strategies with Qwen3-14B. We compare the base model, supervised fine-tuning (SFT) and GRPO with answer supervision, SFT and GRPO with answer and decision flow supervision, and GuIDeR . “ans” denotes supervision on the final answer, which is the answer target for SFT and the answer correctness reward for GRPO. “ans+flow” adds supervision on the annotated reference decision flow, which is the flow target for SFT and a process reward computed against the reference flow for GRPO. The arrow denotes GRPO initialized from the corresponding SFT checkpoint. We report weighted accuracy (WAcc.) and guideline adherence (Adh.) on MedGUIDE, the aggregate score on AMEGA-LLM, and LLM judge scores on RealMedQA and CancerGUIDE. Higher is better for all metrics.

Gemma4-12B
Model	MedGUIDE		AMEGA-LLM	RealMedQA	CancerGUIDE
Model	WAcc.	Adh.	AMEGA-LLM	RealMedQA	CancerGUIDE
GPT-5.6-Sol	70.0	67.2	19.1	2.8	2.6
Gemini-Flash-3.7	67.2	61.5	18.8	2.4	2.3
Meditron-3-8B ( Theimer-Lienhard et al., 2026 )	34.7	62.3	18.3	1.5	1.9
Fastino-Healthcare-30B ( Atreja et al., 2026 )	36.7	41.3	16.1	1.6	1.3

Table 2: Performance across clinical reasoning benchmarks . For fine-tuned methods, we report mean

\pm

standard deviation over three random seeds. We report weighted accuracy (WAcc.) and guideline adherence (Adh.) on MedGUIDE, aggregate score on AMEGA-LLM, and LLM-judge scores on RealMedQA and CancerGUIDE. Higher is better for all metrics.

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

# Decision Flows (M)	MedGUIDE		AMEGA-LLM	RealMedQA	CancerGUIDE
# Decision Flows (M)	WAcc.	Adh.	AMEGA-LLM	RealMedQA	CancerGUIDE
1	77.1	70.2	20.3	1.6	1.9
2	80.3	72.5	21.6	1.7	2.1
4	82.5	73.5	22.3	1.9	2.4
6	82.4	74.1	22.1	1.9	2.3
8	82.3	74.5	22.2	1.9	2.3

Appendix

Table 3: Effect of the number of sampled decision flows on Qwen3-14B. We vary the number of controller-sampled flows

M

used for Monte Carlo process supervision. We report weighted accuracy (WAcc.) and guideline adherence (Adh.) on MedGUIDE, aggregate score on AMEGA-LLM, and LLM-judge scores on RealMedQA and CancerGUIDE. Higher is better for all metrics.

# Decision Nodes (N)	MedGUIDE		AMEGA-LLM	RealMedQA	CancerGUIDE
# Decision Nodes (N)	WAcc.	Adh.	AMEGA-LLM	RealMedQA	CancerGUIDE
2	77.2	68.5	20.3	1.5	1.8
4	82.3	74.5	22.2	1.9	2.3
6	81.2	74.4	22.4	1.8	2.4
8	80.5	72.8	21.5	1.8	2.1

Appendix

Table 4: Effect of the number of decision nodes in sampled decision flows on Qwen3-14B. We vary the maximum number of decision nodes

N

in each controller-sampled flow and evaluate its effect on in-domain and out-of-domain performance. We report weighted accuracy (WAcc.) and guideline adherence (Adh.) on MedGUIDE, aggregate score on AMEGA-LLM, and LLM-judge scores on RealMedQA and CancerGUIDE. Higher is better for all metrics.

Panel of Figure 3 — Figure 3: LLM instructions for the decision-flow controller. Given a clinical question, the controller generates a compact guideline-style decision flow. The generated flows represent plausible clinical decision pathways and are used to provide process-level supervision during policy optimization.

Panel of Figure 4 — Figure 4: LLM instructions for the clinical reasoning policy. Given a clinical question, the policy generates its own compact guideline-style decision flow followed by the final answer. The policy does not observe controller-generated flows, allowing it to independently reason over the clinical case while receiving process-level supervision through the optimization objective. $N$ denotes the maximum number of decision-flow nodes.

Panel of Figure 5 — Figure 5: MedGUIDE Guideline Adherence Judge Template. For each MedGUIDE example, the judge receives the patient vignette, the supervised gold decision path, the gold answer option, the model’s parsed answer, and the model-generated decision flow.

Panel of Figure 6 — Figure 6: AMEGA judge template. AMEGA-LLM provides a set of weighted clinical criteria for each case–question pair. The judge evaluates each criterion independently.

Panel of Figure 7 — Figure 7: RealMedQA judge template. For RealMedQA, the judge compares the generated clinical decision flow with the reference NICE recommendation.

Panel of Figure 8 — Figure 8: CancerGuide judge template. For CancerGUIDE, the judge evaluates whether the generated cancer-treatment recommendation is clinically equivalent to the reference recommendation.

Panel of Figure 9 — Figure 9: Example clinical question and generated decision flow from MedGUIDE. Given a patient with relapsed extensive-stage small-cell lung cancer and poor performance status, the model generates a compact decision flow identifying the clinically relevant guideline conditions and arrives at the correct treatment recommendation. The generated flow is consistent with the gold guideline recommendation while expressing it as a concise sequence of decision states.

RL for Language Model Reasoning

Reinforcement learning for language model reasoning uses reward-driven training to improve multi-step problem solving, tool use, and reliable answers.

See all 1,103 papers

MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning
May 26, 2026Yuhao Shen, Lang Cao, Simo Du +4Healthcare Clinical Decision Support
University of Illinois Urbana-Champaign · 2The Chinese University of Hong Kong, Shenzhen · 3Albert Einstein College of Medicine
More Reasoning Is Not Always Better: Executing Clinical Guidelines with LLMs and Rules
Oct 11, 2026Xiaoqian Lu, Guangfu Guo, Long Cheng +1Clinical Decision Support Medical LLMs
University of Ottawa · Clemson University
CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubrics
May 10, 2026Aishik Nagar, Arun-Kumar Kaliya-Perumal, Yu-Hsuan Han +5Healthcare Clinical Decision Support
ASUS Intelligent Cloud Services (AICS) · Rehabilitation Research Institute of Singapore, Nanyang Technological University · Department of Family Medicine,2026 Taipei Veterans General Hospital +2

May 26, 2026J/K move · Enter open · S save

MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning

Yuhao Shen, Lang Cao, Simo Du, Yuqing Wang+3

University of Illinois Urbana-Champaign · 2The Chinese University of Hong Kong, Shenzhen · 3Albert Einstein College of Medicine

Healthcare Clinical Decision Support Clinical Reasoning Medical LLMs

Open paper AskPDF arXiv

Clinical practice guidelines (CPGs) encode evidence-based decision logic that clinicians apply by evaluating patient variables, conditional criteria, and recommendation rules. However, existing methods often use CPGs as free-text training data or retrieval sources, underutilizing their procedural decision structure. To better exploit this structure, we introduce a guideline-derived training pipeline that transforms CPG recommendations into executable clinical decision logic and uses it to generate factual and counterfactual question-answering data. Theses data teach models both guideline-supported decisions and how decisions change under different patient conditions. Post-training a medical LLM on the generated data yields MedGuideX. Across four clinical reasoning benchmarks, MedGuideX achieves a 10.28% relative improvement in average accuracy. Physician evaluation further shows that MedGuideX better recovers clinician authored reasoning steps and produces physician-preferred rationales in faithfulness, validity, completeness, and clarity. Overall, our results show that executable decision logic from CPGs can be transformed into scalable supervision for building reliable medical LLMs.

Abstract

RL for Language Model Reasoning

Explore similar work

MedGuideX: Internalizing Decision Logic from Executable Guidelines into Large Language Models for Clinical Reasoning

More Reasoning Is Not Always Better: Executing Clinical Guidelines with LLMs and Rules

CLR-voyance: Reinforcing Open-Ended Reasoning for Inpatient Clinical Decision Support with Outcome-Aware Rubrics