Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent

Yunqi Zhu; Wensheng Zhang; Xuebing Yang

cs.AISep 4, 2026

Constructing and Evaluating Clinical Reasoning Trajectories for Medical Agent

Authors: Yunqi Zhu, Wensheng Zhang, Xuebing Yang

Organizations: Guangzhou University, Guangzhou, China · Institute of Automation, Chinese Academy of Sciences, Beijing, China

Abstract

Evaluation of medical artificial intelligence agents remains predominantly answer-centric, assessing only the correctness of final outputs while overlooking the quality of intermediate reasoning. In clinical settings, however, a correct answer reached through fabricated evidence or incoherent logic is as dangerous as an incorrect one. We propose MedTraj, a framework that treats reasoning trajectories as critical objects for construction, evaluation, and optimization. The pipeline generates structured multi-step reasoning chains from medical reasoning sources. Each trajectory is then parsed into clinical observations, evidence, numbered reasoning steps, and a final conclusion, and scored across five quality dimensions: coherence, evidence support, hallucination, completeness, and traceability. Controlled error injection introduces targeted faults into otherwise correct trajectories to establish causal links between specific reasoning failures and measurable quality degradation. Building on this, step-level filtering based on marginal contribution identifies which individual reasoning steps drive or undermine trajectory quality. Finally, quality-weighted context learning feeds trajectory evaluations back into the model at inference time, allowing it to learn from both strong and weak reasoning demonstrations. Experiments across CareQA, PubMedQA, and CECMed demonstrate that trajectory context consistently improves reasoning coherence, with gains of +0.029 to +0.041 over a zero-shot baseline. On CECMed, quality-weighted context nearly doubles the correctness over the zero-shot baseline while cutting the hallucination ratio by 87%. Marginal-contribution analysis further shows that a small minority of reasoning steps carry most of the quality signal, and that extending chains beyond four steps yields diminishing returns.

Figures & tables

Type	Description
Points Only	Clinical observations with no reasoning or conclusion
Points+Reasoning	Observations and reasoning but no conclusion
Reasoning+Result	A reasoning chain without explicit point extraction
Points+Reasoning+Result	The complete observation-to-conclusion chain
Points+Evidence+Reasoning+Result	Observation-to-conclusion chain with explicit evidence citations
Points+Reasoning+Subreasoning	Hierarchical chain with nested sub-inferences

TABLE I: Structural types of parsed trajectories.

Mode	Description	Error type
Inject irrelevant	Insert steps from another question’s trajectory at a random position	Redundant reasoning
Swap steps	Shuffle the order of selected reasoning steps	Structural disruption
Flip numeric	Multiply numerical values by a random factor uniformly drawn from 2 to 10	Factual / hallucination
Remove key point	Delete critical steps (retaining $\geq 50\%$ )	Incompleteness
Replace unrelated	Replace steps with another question’s reasoning	Causal contradiction
Wrong-then-correct	Prepend $N$ erroneous steps before the correct chain	Positional: bad prefix

TABLE II: Error injection modes.

	CareQA (Open-Ended)				PubMedQA (Open-Ended)				CECMed (Severity)
Method	Corr.	Coh.	Evi.	Hal.	Corr.	Coh.	Evi.	Hal.	Corr.	Coh.	Evi.	Hal.
NC	.229	.538	.430	.523	.656	.812	.800	.140	.390	.734	.692	.189
QA	.232	.540	.429	.546	.651	.809	.798	.147	.436	.745	.705	.176
TC	.244	.567	.450	.510	.708	.853	.851	.104	.642	.773	.767	.063
QWC	.246	.576	.459	.507	.730	.853	.851	.104	.738	.799	.797	.025
SFT	.220	.549	.437	.515	.656	.820	.814	.145	.408	.748	.704	.175

TABLE III: LLM evaluation results across three datasets. Blue marks the best method and grey marks the second-best method.

Fig. 1: Growth curves of trajectory value, answer score, and hallucination under different error types.

Fig. 2: (a) Step type distribution. (b) Proportion of each step type by position. (c) Distribution of marginal contribution $\delta$ .

Fig. 3: (a) Mean marginal contribution $\delta$ by step position ( $\pm$ 1 std). (b) Cumulative quality curves by step type.

Fig. 4: Trajectory value, answer score, and hallucination ratio as reasoning steps accumulate (mean $\pm$ std band).

Cluster	Proportion	Traj. Value	Match Score	Hal.
High-quality effective	82.0%	0.999	0.979	0.029
Coherent but wrong	12.4%	0.942	0.371	0.425
Low-quality invalid	5.5%	0.131	0.175	0.826

TABLE IV: Trajectory clusters from the generation pipeline.

Appendix figures & tables2 assets

Supplementary material from the paper’s appendix.

Appendix

Fig. 5: Distributions of correctness, coherence, evidence score, and hallucination across all methods and datasets. (red diamond = mean)

Panel of Figure 10 — Fig. 6: Distributions of evaluation dimensions under each injection mode.

Explore similar work

Aug 31, 2026cs.AI

MedAgent-R1: Faithfulness-Aware Reinforcement Learning for Evidence-Grounded Medical Reasoning

When medical AI systems hallucinate clinical reasoning, the consequences extend beyond incorrect answers: fabricated justifications that superficially reference retrieved evidence can mislead clinicians into unsafe treatment decisions. Medical reasoning agents must therefore produce not only correct answers but also faithful justifications that clinicians can verify against cited evidence. We identify a systematic failure mode in RL-trained retrieval agents: outcome-only rewards improve accuracy while degrading faithfulness, a phenomenon we term confident hallucination. The agent learns to answer from parametric memory and backfill plausible but unsupported justifications; citation fabrication rates rise from 16.5% to 31.8% even as accuracy improves by 5 points over the supervised baseline. We address this with a faithfulness-gated reward design: accuracy credit is conditioned on evidence grounding via a hard gate, complemented by retrieval validity and conciseness signals that close exploitation paths unique to agentic retrieval. The resulting system, MedAgent-R1, reduces citation fabrication from 31.8% to 4.7% and raises evidence completeness from 58.7 to 82.6 while maintaining 75.1% accuracy, with 13.2-point gains on HealthBench Safety. Under the same agentic retrieval setup, MedAgent-R1 outscores GPT-4o on faithfulness-specific dimensions (Factual Support 4.55 vs. 4.25; Overclaiming 4.40 vs. 4.15) while remaining below GPT-4o in overall accuracy, suggesting that explicit faithfulness training yields evidence-grounding gains not achieved by scaling alone.

Jiangwang Chen, Chenghao Zhang, Hengxing Cai

Tsinghua University · DP Technology

LLM Hallucination Mitigation Evidence-Grounded Reasoning RL for Language Model Reasoning Agentic RAG Medical Reasoning

Ask PDF

May 16, 2026cs.CL

SEMA-RAG: A Self-Evolving Multi-Agent Retrieval-Augmented Generation Framework for Medical Reasoning

Retrieval-Augmented Generation (RAG) is widely employed to mitigate risks such as hallucinations and knowledge obsolescence in medical question answering, yet its predominantly single-round, static retrieval paradigm misaligns with the multi-stage process of clinical reasoning. This compressed workflow induces two structural deficiencies: question-to-query translation often lacks clinically grounded semantic interpretation, and retrieval lacks iterative sufficiency feedback, making it difficult to form reliable evidence chains. We argue that both issues stem from a deeper cause: overloading a single reasoning chain with heterogeneous tasks of interpretation, exploration, and adjudication. The remedy is to reconstruct the workflow via task decoupling and dynamic multi-round exploration. To this end, we propose SEMA-RAG, a Self-Evolving Multi-Agent RAG framework for medical question answering, which assigns these roles to three specialist agents: the Interpreter Agent for clinical schema interpretation, the Explorer Agent for sufficiency-driven self-evolving retrieval, and the Arbiter Agent for evidence adjudication and answer selection. Across five benchmarks and five LLM backbones, SEMA-RAG improves the strongest baseline by +6.46 accuracy points on average, measured per backbone.

Yongfeng Huang, Ruiying Chen, James Cheng

CSE, The Chinese University of Hong Kong · Wuhan University of Technology

Retrieval-Augmented Generation Healthcare Clinical QA Agentic RAG Medical Reasoning

Ask PDF

May 27, 2026cs.AI

Better Accuracies, Worse Reasoning: A Step-Level Audit of Medical Chain-of-Thought Distillation

Chain-of-thought (CoT) distillation trains a smaller model to imitate a teacher's reasoning trace, but it is typically evaluated by final-answer metrics including accuracy. We ask whether gains in answer quality are accompanied by improvements in the trace. In medical QA, where short answer options can leave a richer clinical justification under-specified, a Qwen3-8B student distilled from a DeepSeek-V3-family teacher improves on MedQA-USMLE answer metrics (SC@64 74.7% to 84.4%; expected calibration error (ECE) 0.096 to 0.034). Yet under a Kimi-K2.6 style-blind LLM-judge audit, its error rate over non-abstained steps rises from 30.6% to 50.3%. In this primary medical setting, answer quality and trace factuality move in opposite directions. This before--after pattern persists across evaluators, teacher strengths, student scales and families, medical benchmarks, and style, segmentation, and answer-correctness controls. A 150-step blinded audit by a clinical expert reproduces the same ordering. Boundary checks narrow the scope of the claim: the risk appears when a compact answer under-constrains the rationale and a capable student can imitate expert-like form without reliably grounding each local claim. Standard answer metrics and aggregate hedging rates do not reveal the shift. When such traces are released or reused, answer-level metrics alone are insufficient.

Zhaoyang Jiang, Xuanqi Peng, Fei Teng +5

School of Health & Wellbeing, University of Glasgow, Glasgow, UK · Department of Respiratory and Critical Care Medicine, Shanghai Sixth People’s Hospital, Shanghai Jiao Tong University School of Medicine, Shanghai, China · School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China +1

CoT Distillation CoT Evaluation Clinical QA Medical Reasoning Teacher-Student Learning

Ask PDF