Vision-Language Models (VLMs) have enabled interpretable medical diagnosis by integrating visual perception with linguistic reasoning. Yet, existing medical chain-of-thought (CoT) models lack explicit mechanisms to represent and enforce causal reasoning, leaving them vulnerable to spurious correlations and limiting their clinical reliability. We pinpoint three core challenges in medical CoT reasoning: how to adaptively trigger causal correction, construct high-quality causal-spurious contrastive samples, and maintain causal consistency across reasoning trajectories. To address these challenges, we propose MedCausalX, an end-to-end framework explicitly models causal reasoning chains in medical VLMs. We first introduce the CRMed dataset providing fine-grained anatomical annotations, structured causal reasoning chains, and counterfactual variants that guide the learning of causal relationships beyond superficial correlations. Building upon CRMed, MedCausalX employs a two-stage adaptive reflection architecture equipped with ⟨causal⟩ and ⟨verify⟩ tokens, enabling the model to autonomously determine when and how to perform causal analysis and verification. Finally, a trajectory-level causal correction objective optimized through error-attributed reinforcement learning refines the reasoning chain, allowing the model to distinguish genuine causal dependencies from shortcut associations. Extensive experiments on multiple benchmarks show that MedCausalX consistently outperforms state-of-the-art methods, improving diagnostic consistency by +5.4 points, reducing hallucination by over 10 points, and attaining top spatial grounding IoU, thereby setting a new standard for causally grounded medical reasoning. The code and dataset are available at https://github.com/zhcz328/MedCausalX.
Figures & tables
Figure 1: Existing medical VLMs often lack causality-awareness, which manifests as (1) spurious CoT reasoning attends to mislocalized anatomy or (2) overlooking causal evidence for pathology diagnosis. MedCausalX can achieve trustworthy diagnosis by adaptive causal verification and correction during inference.
Figure 2: Overview of MedCausalX framework. We first construct CRMed dataset via fine-grained annotations, then factorize reasoning via a structural causal model (SCM) with reflective tokens ( <causal> , <verify> ) enabling two-stage adaptive reasoning. After causal Supervised Fine-Tuning (SFT), MedCausalX is subsequently optimized via dual-policy that couples off-policy preference alignment (DPO) with on-policy causal reinforcement (GRPO), bridging stability and adaptability to enhance the model’s causal reasoning capability.
Method
Region-to-Text Identification
Text-to-Region Detection
Structure Identification
Lesion Identification
Single-Region Detection
Multi-Region Detection
BLEU-1
F1
Recall
BertScore
Acc
BLEU-1
F1
Recall
BertScore
Acc
Obj-F1
Reg-F1
Align-F1
IoU
Obj-F1
Reg-F1
Align-F1
IoU
Qwen2.5-VL [ 2 ]
0.15
0.25
0.58
52.34
0.12
0.14
0.23
0.55
48.56
0.11
57.82
8.15
6.23
17.45
49.35
5.87
4.65
14.82
InternVL-2 [ 10 ]
0.13
0.21
0.52
51.78
0.11
0.13
0.20
0.51
47.92
0.10
56.60
7.24
5.45
16.61
48.23
5.41
4.18
13.99
Med-Flamingo [ 64 ]
0.13
0.72
4.90
24.87
0.01
0.12
0.68
4.72
23.45
0.01
18.47
2.35
1.82
5.23
12.68
1.54
1.18
3.47
LLaVA-Med [ 41 ]
0.43
1.15
10.69
34.53
1.74
0.38
1.05
9.85
32.18
1.52
24.83
3.76
2.94
7.85
18.95
2.42
1.87
5.36
Table 1: Comprehensive quantitative comparison on diverse region-centric tasks, including region-to-text identification and text-to-region detection. Abbreviations in this Table: Obj-F1 denotes Object F1, Reg-F1 denotes Region F1, Align-F1 denotes Alignment F1, and Acc denotes Accuracy. The best results are highlighted in bold , while the second-best are underlined .
Method
SLAKE [ 51 ]
VQA-RAD [ 37 ]
PathVQA [ 23 ]
PMC-VQA [ 103 ]
Average
Acc
Diag-C
Hall ↓
Acc
Diag-C
Hall ↓
Acc
Diag-C
Hall ↓
Acc
Diag-C
Hall ↓
Acc
Diag-C
Hall ↓
General Vision-Language Models
Qwen2.5-VL [ 2 ]
68.5 1.8
58.3 2.1
62.5 1.9
52.4 2.3
54.2 2.0
65.8 1.7
48.3 2.1
52.8 1.9
68.2 2.2
58.2 1.6
56.7 1.8
63.5 2.0
56.9 1.9
55.5 1.9
65.0 1.9
InternVL-2 [ 10 ]
70.8 1.6
61.2 1.8
59.8 2.1
55.1 2.0
57.5 1.9
62.5 1.8
52.7 1.9
55.3 2.0
65.3 2.1
61.5 1.5
59.2 1.7
60.8 1.9
60.0 1.8
58.3 1.9
62.1 2.0
Medical-Specific Vision-Language Models
Med-Flamingo [ 64 ]
33.2 2.7
38.5 2.5
75.2 2.3
44.6 2.6
42.8 2.4
71.8 2.5
47.9 2.4
45.6 2.3
68.5 2.6
35.9 2.8
40.5 2.5
73.8 2.4
40.4 2.6
41.9 2.4
72.3 2.5
Table 2: Performance comparison on medical VQA benchmarks with causal reasoning evaluation. Metrics include answer Accuracy (Acc), Diagnostic Consistency (Diag-C), and Hallucination Rate (Hall, lower is better). Results are reported as mean std over 5-fold cross-validation. Best results in bold , second-best underlined . ’*’ denotes fine-tuned models.
Figure 3: Comparison of causal reasoning in medical VQA and region-grounded diagnosis. (a) MedRegA and Med-R1 exhibit spurious correlations and mistakes, while MedCausalX captures correct predictions via structured causal decomposition. (b) Two-stage adaptive causal reasoning and verification, producing refined bounding boxes and logically consistent diagnostic descriptions. Attention maps are used to visualize the model’s focus. GT: ground truth; A1–A3: predictions from MedCausalX, Med-R1, and MedRegA, respectively.
Method
BLEU-1
Region Acc
Align Acc
IoU
Qwen2.5-VL [ 2 ]
18.43
22.30
8.15
17.45
InternVL [ 11 ]
19.46
57.00
12.00
38.00
Med-Flamingo [ 64 ]
21.67
-
-
-
MedDr [ 22 ]
31.27
-
-
-
MedRegA [ 82 ]
33.18
76.59
62.29
52.07
MedCausalX
37.18
81.12
67.38
55.71
Table 3: Performance comparison on chest X-ray report generation benchmark. Our method employs two-stage causal reasoning with adaptive verification based on MedCausalX-32B model. Best results are highlighted in bold , second-best are underlined .
Configuration
BLEU-1
Region Acc
Align Acc
IoU
MedCausalX (Full)
37.18
81.12
67.38
55.71
Main Component Ablation
✗ CRMed Dataset
26.47
41.20
18.35
26.55
✗ Reflective Tokens
29.93
51.20
32.50
38.46
✗ Causal SFT
26.02
43.80
21.75
28.55
✗ RL Training
26.96
63.50
47.30
38.09
Table 4: Systematic ablation of core components and training stage variants on MIMIC-CXR benchmark evaluates each module’s contribution to causal reasoning and verification capabilities.
Figure 4: Radiologists’ evaluation and comparative results. Left: 3-point clinical criteria for spatial localization (Q1), diagnostic description (Q2). Right: radar chart comparing MedRegA, Med-R1, and MedCausalX on localization (blue) and description (orange); lower scores indicate better performance.
Figure 5: Modality distribution and factual-counterfactual composition of CRMed dataset. The dataset spans multiple imaging modalities with constructed counterfactual variants through controlled interventions, enabling robust causal reasoning evaluation.
Hyperparameter
Value
Base Model
Qwen2.5-VL-32B
LoRA Rank ( r )
64
LoRA Alpha ( α )
128
LoRA Dropout
0.05
Target Modules
q_proj, k_proj, v_proj
Reflective Tokens
⟨CAUSAL⟩ , ⟨VERIFY⟩
Table 6: Complete hyperparameter configuration for MedCausalX training and inference.
Model Scale
Region-to-Text Identification
Text-to-Region Detection
Structure Identification
Lesion Identification
Single-Region Detection
Multi-Region Detection
BLEU-1
F1
Recall
BertScore
Acc
BLEU-1
F1
Recall
BertScore
Acc
Obj-F1
Reg-F1
Align-F1
IoU
Obj-F1
Reg-F1
Align-F1
IoU
MedCausalX-3B
53.42
54.68
55.20
52.15
48.20
38.29
39.12
40.35
48.65
35.12
68.45
21.47
18.23
24.49
55.18
14.23
12.47
19.82
MedCausalX-14B
73.49
74.23
75.80
54.20
55.30
56.43
57.80
59.25
70.83
52.80
79.82
34.62
28.15
33.75
65.75
19.56
17.85
26.18
MedCausalX-32B
79.82
80.15
81.25
88.45
70.15
63.27
64.35
65.80
81.90
61.35
82.45
46.82
44.92
44.35
72.38
29.51
26.82
35.76
Table 7: Model scaling analysis across region-centric medical reasoning tasks. We evaluate MedCausalX variants with 3B, 14B, and 32B parameters to assess the impact of model capacity on causal reasoning performance. Best results are highlighted in bold , second-best are underlined . All models trained with identical data and optimization strategies.
Model
VQA-RAD
SLAKE
PathVQA
PMC-VQA
Average
Vision-Language Models (Zero-shot)
LLaVA-1.5-13B [ 55 ]
53.2
58.5
52.8
56.3
55.2
InternVL-2-40B [ 10 ]
59.8
65.2
61.5
67.1
63.4
MedRegA-40B [ 82 ]
76.9
84.1
68.5
79.5
77.3
Medical CoT Methods (Qwen2.5-VL-32B Backbone)
Med-R1 [ 36 ]
76.9
85.4
69.5
81.9
78.4
Table 8: Model scale and architecture comparison across medical VQA benchmarks. Light blue background indicates selected configuration.
Figure 6: Comparison of model efficiency and performance across medical VLMs. MedCausalX achieves the highest composite score at 32B parameters, demonstrating superior causal reasoning capabilities while maintaining computational efficiency.
Config.
Value
VQA-RAD
SLAKE
PathVQA
PMC-VQA
Average
(a) Training Stage Order
Seq.
GRPO→DPO
73.5
81.4
68.2
78.9
75.5
DPO→GRPO
79.8
87.2
73.2
84.6
81.2
(b) GRPO Group Size G
G
4
76.8
84.3
70.5
81.7
78.3
6
78.5
86.1
72.1
83.5
80.1
Table 9: Ablation study on training strategy configuration. Light blue background indicates selected configuration, bold values denote best performance in each row.
Config.
Value
VQA-RAD
SLAKE
PathVQA
PMC-VQA
Average
(a) Reward Weight Configuration λacc/λcausal
λ
1.0/0.0
71.5
78.9
65.3
76.2
73.0
0.7/0.3
76.8
84.1
70.5
81.7
78.3
0.5/0.5
79.8
87.2
73.2
84.6
81.2
0.3/0.7
78.2
85.7
71.8
83.1
79.7
0.0/1.0
67.3
74.5
61.2
71.8
68.7
Table 10: Ablation of reward components and error localization threshold. Light blue background indicates selected configuration.
Method
OVQA
EndoVis-VQA
Skin-VQA
Average
GPT-4o [ 28 ]
43.5
39.8
46.1
43.1
LLaVA-Med [ 41 ]
47.2
44.3
49.5
47.0
InternVL-2 [ 10 ]
49.8
46.7
51.2
49.2
Med-R1 [ 36 ]
44.8
40.5
46.3
43.9
MedVLM-R1 [ 67 ]
50.3
47.1
49.8
49.1
MedCausalX (Ours)
58.4
54.2
59.7
57.4
Table 11: Out-of-domain evaluation on specialized medical VQA datasets. Models trained on core benchmarks and evaluated zero-shot on unseen domains.
Figure 7: Training dynamics of on-policy GRPO optimization. MedCausalX (Full) with DPO pretraining achieves higher rewards and maintains stable, compact reasoning compared to direct GRPO training from preliminary SFT baseline.
Figure 8: Structured prompt template for MedCausalX showing system instructions for pathological analysis approach and output format specifications enabling causal chain generation.
Figure 9: Chest radiograph demonstrating bilateral lung detection where MedCausalX accurately quantifies two distinct pulmonary structures through structured anatomical enumeration and spatial boundary verification.
Figure 10: Abdominal examination section validating gastric cardia identification through causal anatomical landmark detection, confirming presence via systematic spatial relationship analysis with esophagogastric junction.
Figure 11: Echocardiographic image demonstrating successful causal localization of endocardial lesion positioned adjacent to lower intraventricular septum through structured anatomical-pathological reasoning chain.
Medical vision-language models (VLMs) generate chain-of-thought (CoT) reasoning before answering clinical questions, but whether this reasoning causally influences predictions remains unclear. We present CoT-Mediate, a behavioral framework that perturbs a single clinically meaningful attribute within a model's own generated reasoning and measures whether the resulting prediction follows the edited reasoning. Our framework combines a dual-arm protocol comparing re-prompted evidence with prefix-forced continuation, together with a provenance-controlled intervention that varies only the attributed source of identical reasoning to disentangle reasoning mediation from sycophancy. We evaluate LLaVA-Med and MedGemma on 1,000 VQA-RAD samples each. Prefix-forced continuation consistently yields higher mediation faithfulness than re-prompting, while the provenance analysis reveals distinct model-specific deference behaviors. Across both models, removing visual evidence increases reliance on injected reasoning, whereas laterality is the least faithfully tracked clinical attribute. These results show that the mechanism used to inject reasoning substantially affects measured faithfulness and that contextual position, rather than stated provenance, is the primary determinant of whether medical VLMs use their generated reasoning.
Supratik Bhowal, Subhrajyoti Basu, Aritra Gir Mahanta +1
Department of Computer Science (AIML) IEM Kolkata, School of UEMK Kolkata, India · Department of Computer Science and Business System Heritage Institute of Technology Kolkata, India · Department of Computer Science and Engineering Indian Institute of Information Technology, Kalyani Kalyani, India
High-stakes clinical use of large vision-language models (LVLMs) requires reasoning that is grounded in visual evidence and clinical knowledge, not just correct final answers. We introduce OpenMedReason, a large-scale, open multimodal medical reasoning corpus comprising approximately 450K image-question-answer instances whose reasoning traces are primarily derived from curated biomedical, human-authored scientific articles. OpenMedReason provides high-fidelity supervision beyond synthetic chains of thought, covering diverse medical domain vision modalities such as radiological scans, microscopic images, visible light photographs, charts, and others. We complement it with OpenMedReason-Bench, a held-out benchmark that allows fine-grained evaluation of LVLMs along three complementary axes of capability, including perception, medical knowledge, and rationale, enabling diagnostic evaluation beyond final-answer accuracy. OpenMedReason is a rich training resource that exhibits its effectiveness in both supervised fine-tuning (SFT) and reinforcement-based alignment. Training with OpenMedReason yields a 20% average improvement in VQA accuracy over the base model and achieves performance within 4.2% of the strongest comparable-scale medical LVLMs. Fine-grained performance analysis confirms that the gains are not concentrated in any single axis: OpenMedReason improves perception, medical knowledge, and rationale jointly, and its reasoning traces are preferred over those of the base model in 86.1% of pairwise comparisons. We release the code and dataset at huggingface.co/datasets/neginb/OpenMedReason.
Negin Baghbanzadeh, Pritam Sarkar, Michael Colacci +6
York University · Vector Institute · University of British Columbia +5
Large vision-language models (VLMs) often benefit from chain-of-thought (CoT) prompting in general domains, yet its efficacy in medical vision-language tasks remains underexplored. We report a counter-intuitive trend: on medical visual question answering, CoT frequently underperforms direct answering (DirA) across general-purpose and medical-specific models. We attribute this to a medical perception bottleneck: subtle, domain-specific cues can weaken visual grounding, and CoT may compound early perceptual uncertainty rather than correct it. To probe this hypothesis, we introduce two training-free, inference-time grounding interventions: (i) perception anchoring via region-of-interest cues and (ii) description grounding via high-quality textual guidance. Across multiple benchmarks and model families, these interventions improve accuracy, mitigate CoT degradation, and in several settings reverse the CoT--DirA inversion. Our findings suggest that reliable clinical VLMs require robust visual grounding and cross-modal alignment, beyond extending text-driven reasoning chains. Code is available here.
Yuan Wu, Zongxian Yang, Jiayu Qian +5
Department of Computer Science, City University of Hong Kong (Dongguan) · College of Computing and Data Science (CCDS), Nanyang Technological University · School of Computer Science, Northwestern Polytechnical University