RVLM: Recursive Vision-Language Models with Adaptive Depth
Authors: Nicanor Mayumu, Zeenath Khan, Melodena Stephens, Patrick Mukala, Farhad Oroumchian
Organizations: Department of Computer Science, University of Wollongong in Dubai, Dubai Knowledge Park, Dubai, UAE · Department of Academic Affairs, Mohammed Bin Rashid School of Government, City Walk, Building B02, Dubai, UAE
Medical AI systems face two fundamental limitations. First, conventional vision-language models (VLMs) perform single-pass inference, yielding black-box predictions that cannot be audited or explained in clinical terms. Second, iterative reasoning systems that expose intermediate steps rely on fixed iteration budgets wasting compute on simple cases while providing insufficient depth for complex ones. We address both limitations with a unified framework. RVLM replaces single-pass inference with an iterative generate-execute loop: at each step, the model writes Python code, invokes vision sub-agents, manipulates images, and accumulates evidence. Every diagnostic claim is grounded in executable code, satisfying auditability requirements of clinical AI governance frameworks. RRouter makes iteration depth adaptive: a lightweight controller predicts the optimal budget from task-complexity features, then monitors progress and terminates early when reasoning stalls. We evaluate on BraTS 2023 Meningioma (brain MRI) and MIMIC-CXR (chest X-ray) using Gemini 2.5 Flash without fine-tuning. Across repeated runs, RVLM shows high consistency on salient findings (e.g., mass presence and enhancement) and can detect cross-modal discrepancies between Fluid-Attenuated Inversion Recovery (FLAIR) signal characteristics and segmentation boundaries. On MIMIC-CXR, it generates structured reports and correctly recognises view-specific artefacts. Code: https://github.com/nican2018/rvlm.
Figures & tables
Figure 1: Rvlm System Architecture. The core VLM MV operates within a persistent Python REPL environment EV that maintains state across iterations. The environment provides first-class image storage ( context_images ), vision-specific query primitives, and full programmatic image manipulation. At each iteration t , the model generates executable code, which is executed in the REPL; outputs are appended to the message history. The loop terminates upon emission of a FINAL or FINAL_VAR signal.
Case description
H
V
R
T
s
n∗
Single ET, small (00008-000)
0.00
9.83
1
0
0.14
3
Single ET, large
0.00
45.0
1
0
0.35
4
Two regions (ET+ED)
1.00
20.0
2
0
0.58
5
Three regions, high volume
1.50
30.0
3
1
0.86
6
Three regions, tiny NCR
1.10
18.0
3
1
0.73
6
Table 1: RecursionRouter complexity scoring for representative BraTS-MEN cases. H = label entropy (bits), V = total volume (cc), R = sub-region count, T = tiny region flag, s = composite score, n∗ = recommended budget.
Trust Dimension
Rvlm Mechanism
Governance Implication
Procedural Transparency
Stepwise REPL reasoning; one analysis step per iteration with stored intermediate variables and execution logs.
Clinicians audit the reasoning path, and clinician-readable PDF report with explicit AI disclaimer; supports EU AI Act Art. 13
Auditability & Traceability
Complete REPL execution log including generated code, sub-VLM calls, stdout outputs, and stored variables; runs are reproducible given identical inputs and model weights
Full post-hoc audit trail; satisfies ISO/IEC 42001 audit requirements
Reliability & Verification
Cross-modal verification: independent per-modality descriptions, followed by explicit discrepancy detection across modalities
Reduces single-source hallucination; mirrors clinical cross-referencing practice
Human-in-the-Loop
Inspectable intermediate states; clinical PDF with AI disclaimer; human-readable code
Clinician retains decision authority; supports EU AI Act Art. 14
Explainability
Evidence chain linking pixel-level observations, intermediate REPL variables, and final diagnostic synthesis
Reasoning traceable from pixel observation to clinical conclusion
Adaptive Efficiency
RecursionRouter matches compute to case complexity; stall detection prevents wasted iterations
Enables cost-effective deployment at scale without sacrificing depth on complex cases
Table 3: MRI modality characteristics and diagnostic utility.
Sub-region
Volume (cc)
Share
NCR - Necrotic Core
0.00
0%
ED - Peritumoral Oedema
0.00
0%
ET - Enhancing Tumour
9.83
100%
Total
9.83
100%
Table 4: Ground-truth mask statistics for BraTS-MEN-00008-000.
Feature
MEN-00004
MEN-00008
Enhancement pattern
Homogeneous
Irregular
Dural tail sign
✓
✗
CSF cleft
✓
Not assessed
Multifocal lesions
✗
✓
Peritumoral edema
Moderate
Extensive
Mass effect
Mild
Severe
Table 5: Cross-patient comparison of Rvlm diagnostic findings.
Radiological Finding
Identified
Rate
Extra-axial mass
9/9
100%
Intense enhancement
9/9
100%
Peritumoral edema
8/9
89%
Mass effect / ventricular compression
8/9
89%
Dural attachment suggested
7/9
78%
WHO grade suggestion
7/9
78%
Table 6: Diagnostic consistency for BraTS-MEN-00008-000 ( n=9 runs).
Diagnostic Feature
Single-Pass
Rvlm
Tumour identification
✓
✓
Enhancement characterisation
Partial
Detailed
Dural tail sign
✗
✓
Edema localisation
Generic
Specific
CSF cleft mention
✗
✓
WHO grade suggestion
✗
Grade 1
Table 7: Qualitative comparison on BraTS-MEN-00004-000.
Figure 2: Auto-generated clinical PDF report-BraTS-MEN-00004-000. Five labelled sections, mask-statistics table, and AI disclaimer footer. Intended for clinical review, not autonomous diagnosis.
Figure 3: Auto-generated clinical PDF report-MIMIC-CXR subject 10000032. Five clinical sections, a Ground Truth Reference block for radiologist comparison, and an AI disclaimer. The model correctly identifies the AP portable projection and qualifies its cardiac size estimate accordingly.
Vision-Language Models (VLMs) facilitate medical visual question answering (MedVQA) by jointly interpreting images and text. However, existing models typically depend on large architectures and closed-set answers, which limits their efficiency and potential clinical applicability. To overcome these shortcomings, we introduce RoiMAM, an efficient VLM. It integrates a training-free ROI Generation Module with Semantic Selective Suppression to focus on lesion-relevant regions, alongside a Text Prompt Enhancer module that provides modality-specific context without introducing training parameters. Compared to the widely used MedVInT-TD model, our design achieves efficient and accurate diagnosis at less than 20% of the model size, while improving accuracy by approximately 2% on SLAKE and 4.6% on PMC-VQA.
Jiayan Yang, Zhuoyu Wu, Wenqi Fang
Shenzhen Institutes of Advanced Technology, Chinese Academy of Science · CyPhi(ΨΦ) AI Research Lab, School of IT, Monash University, Malaysia Campus
Lesion detection, symptom tracking, and visual explainability are central to real-world medical image analysis, yet current medical Vision-Language Models (VLMs) still lack mechanisms that translate their broad knowledge into clinically actionable outputs. To bridge this gap, we present MEDIC-AD, a clinically oriented VLM that strengthens these three capabilities through a stage-wise framework. First, learnable anomaly-aware tokens (<Ano>) encourage the model to focus on abnormal regions and build more discriminative lesion centered representations. Second, inter image difference tokens (<Diff>) explicitly encode temporal changes between studies, allowing the model to distinguish worsening, improvement, and stability in disease burden. Finally, a dedicated explainability stage trains the model to generate heatmaps that highlight lesion-related regions, offering clear visual evidence that is consistent with the model's reasoning. Through our staged design, MEDIC-AD steadily boosts performance across anomaly detection, symptom tracking, and anomaly segmentation, achieving state-of-the-art results compared with both closed source and medical-specialized baselines. Evaluations on real longitudinal clinical data collected from real hospital workflows further show that MEDIC-AD delivers stable predictions and clinically faithful explanations in practical patient-monitoring and decision-support workflows
Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho +7
AIDAS Laboratory, Seoul National University · Samsung Changwon Hospital · Samsung Medical Center +1
Vision-language models have demonstrated impressive capabilities in general medical visual question answering, yet due to limited interpretability, it remains unclear whether their predictions reflect evidence-grounded clinical reasoning or reliance on spurious priors. We introduce Med-R2 Bench, a hierarchical benchmark aligned with the clinical workflow to evaluate adversarial robustness with visual grounding. We design stepwise QA tasks to assess whether reasoning chains are strictly grounded in visual evidence across the four clinical stages, and employ adversarial perturbations to test robustness against misleading cues. Med-R2 comprises 42,432 images, 31 task categories, and 110,406 QA pairs. Evaluation across 14 VLMs reveals a sequential performance degradation along the four-stage clinical workflow. Adversarial experiments show that models rely heavily on correct prompts to guess answers. Even when provided with explicit visual cues, the models struggle to accurately align textual descriptions. Finally, we demonstrate stepwise fine-tuning using our hierarchical data significantly improves reasoning robustness, highlighting its potential to drive future improvements in evidence-based medical AI.
Wen Ma, Fucheng Niu, Zhiting Fan +3
Zhejiang University, Zhejiang, China · Guangdong Institute of Intelligence Science and Technology