ARISE: Adaptive Agentic Reasoning with Image-grounded Self-Evaluation for Interpretable IBD Assessment

Pronoma Banerjee; Anuva Shah; Jason Wu; Md. Masudur Rahman; Sanjay Mohanty; Satya Kurada; Juan P. Wachs

cs.CVOct 3, 2026

ARISE: Adaptive Agentic Reasoning with Image-grounded Self-Evaluation for Interpretable IBD Assessment

Authors: Pronoma Banerjee, Anuva Shah, Jason Wu, Md. Masudur Rahman, Sanjay Mohanty, Satya Kurada, Juan P. Wachs

Organizations: Purdue University, West Lafayette, IN 47907, USA · Indiana University School of Medicine, Indianapolis, IN 46202, USA

Abstract

Inflammatory bowel disease (IBD) requires frequent imaging-based assessment, yet interpretation of modalities such as wireless capsule endoscopy (WCE) and intestinal ultrasound remains heavily dependent on specialist expertise. Vision-Language Models (VLMs) demonstrate significant potential in multimodal medical image analysis, but their clinical adoption is hindered by their insufficient domain-specific reasoning, susceptibility to hallucination, scarcity of high quality training data in fine-grained diagnostics and limited interpretability. We introduce ARISE (Adaptive Agentic Reasoning with Image-grounded Self-Evaluation), an autonomous planning framework that models few-shot medical image understanding as a sequential agentic workflow. ARISE structures agent execution into a transparent 5-stage workflow: hypothesis generation, image-grounded evidence summarization, evidence-conditioned refinement, symbolic verification, and final diagnosis. We apply ARISE to IBD assessment across two independent patient cohorts: wireless capsule endoscopy (WCE) images for Crohn's disease and B-mode ultrasound data for ulcerative colitis. ARISE consistently improves diagnostic performance over baseline VLMs while exposing where reasoning succeeds or fails, providing a more interpretable basis for clinical decision support and realistic deployment.

Figures & tables

Figure 1: An overview of the ARISE workflow for the Crohn’s disease WCE Pathological vs Non-Pathological image classification. The workflow starts with initial hypotheses generation and a parallel summarization of visual cues from select examples. The initial hypotheses are first updated using the summary of visual cues, followed by a logical consistency check via an SMT solver. The final hypotheses are used for the final diagnosis stage.

Figure 2: Example of stages S2 and S3 on the Crohn’s disease Ulceration Diagnosis task with GPT-5.4

Model	Task 1	Task 2	Task 3
GPT-5.4 (Base)	0.69	0.66	0.32
GPT-5.4 (ARISE)	0.88	0.83	0.79
GPT-4o-mini (Base)	0.61	0.61	0.29
GPT-4o-mini (ARISE)	0.88	0.81	0.74
Gemini-2.5 (Base)	0.66	0.33	0.37
Gemini-2.5 (ARISE)	0.85	0.71	0.58

Table 1: Base and ARISE classification accuracy across the three Crohn’s disease WCE tasks. Task 1: Pathological vs non-pathological image classification, Task 2: Ulceration diagnosis, Task 3: 6-lesion classification

Model	Accuracy	F1	Precision	Recall
GPT-5.4 (Base)	0.60	0.66	0.69	0.63
GPT-5.4 (ARISE)	0.87	0.78	0.83	0.74
GPT-4o-mini (Base)	0.59	0.58	0.58	0.47
GPT-4o-mini (ARISE)	0.77	0.75	0.76	0.74
Gemini-2.5 (Base)	0.26	0.37	0.63	0.22
Gemini-2.5 (ARISE)	0.71	0.78	0.76	0.72

Table 2: Colon wall stratification status classification

Figure 3: Stage-wise agentic scores across models on the Crohn’s disease 6-lesion classification task for the different workflow stages: hypothesis generation (S1), visual grounding (S2), refinement (S3), symbolic verification (S4), and final diagnosis (S5). Rule-based stages are consistently strong, while evidence-guided refinement shows the greatest difficulty and model variation.

Figure 4: Overall, agentic, and task scores for the 4 evaluated agents on the Crohn’s disease 6-lesion classification task.

Figure 5: Qualitative example of improvement of visual grounding with ARISE. The base VLM produces ambiguous visual cues, ultimately leading to the wrong classification. ARISE improves visual grounding and ultimately yields the correct classification.

Figure 6: Illustration of an AI-enabled decision-support interface for explainable IBD assessment using intestinal ultrasound, designed to make model outputs accessible and interpretable to nurses.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Task	Modality	Output labels	Support	ICL	Test
Normal vs. pathological classification	WCE	Normal; pathological	7	18	1434
Six-lesion classification	WCE	AU; S; O; E; U3–10; U $>$ 10	7	12	535
Ulceration diagnosis	WCE	Ulceration; no ulceration	7	12	535
Colon-wall stratification status	B-mode ultrasound	Preserved; lost	4	10	211

Appendix

Table A1: Dataset and task splits used in the experiments. AU: aphthoid ulceration; S: stenosis; O: edema; E: erythema; U3-10: ulceration between 3 and 10 mm; U

>

10: ulceration

>10

mm.

Model	Accuracy	F1	Precision	Recall
GPT-5.4 (Base)	0.69	0.65	0.59	0.72
GPT-5.4 (ARISE)	0.88	0.86	0.85	0.89
GPT-4o-mini (Base) )	0.61	0.73	0.60	0.77
GPT-4o-mini (ARISE)	0.88	0.88	0.89	0.88
Gemini-2.5 (Base)	0.66	0.59	0.77	0.62
Gemini-2.5 (ARISE)	0.85	0.78	0.82	0.76

Appendix

Table A2: Pathological vs Non-Pathological WCE image

Model	Accuracy	F1	Precision	Recall
GPT-5.4 (Base)	0.66	0.76	0.79	0.73
GPT-5.4 (ARISE)	0.83	0.78	0.83	0.74
GPT-4o-mini (Base)	0.61	0.67	0.86	0.59
GPT-4o-mini (ARISE)	0.81	0.84	0.88	0.81
Gemini-2.5 (Base)	0.33	0.18	0.78	0.10
Gemini-2.5 (ARISE)	0.71	0.78	0.86	0.72

Appendix

Table A3: WCE - Diagnosing ulceration

Model	Accuracy	F1	Precision	Recall
GPT-5.4 (Base)	0.32	0.30	0.35	0.32
GPT-5.4 (ARISE)	0.79	0.76	0.81	0.75
GPT-4o-mini (Base)	0.29	0.34	0.32	0.26
GPT-4o-mini(ARISE)	0.74	0.75	0.77	0.74
Gemini-2.5 (Base)	0.37	0.35	0.47	0.37
Gemini-2.5(ARISE)	0.58	0.53	0.58	0.63

Appendix

Table A4: 6 lesion classification from Crohn’s disease WCE

Medical Diagnosis

Medical diagnosis uses clinical data, tests, and images to identify diseases, assess risk, and guide care.

See all 261 papers

RVLM: Recursive Vision-Language Models with Adaptive Depth
Mar 25, 2026Nicanor Mayumu, Zeenath Khan, Melodena Stephens +2VLM Reasoning Medical VLMs
Department of Computer Science, University of Wollongong in Dubai, Dubai Knowledge Park, Dubai, UAE · Department of Academic Affairs, Mohammed Bin Rashid School of Government, City Walk, Building B02, Dubai, UAE
MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
Mar 28, 2026Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho +7Medical VLMs Medical Image Anomaly Detection
AIDAS Laboratory, Seoul National University · Samsung Changwon Hospital · Samsung Medical Center +1
CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework
Mar 2, 2026Yuexi Du, Jinglu Wang, Shujie Liu +2Medical VLMs Medical Reasoning
Microsoft Research Asia · Department of Biomedical Engineering, Yale University · Department of Radiology & Biomedical Imaging, Yale University

Mar 25, 2026J/K move · Enter open · S save

RVLM: Recursive Vision-Language Models with Adaptive Depth

Nicanor Mayumu, Zeenath Khan, Melodena Stephens, Patrick Mukala+1

Department of Computer Science, University of Wollongong in Dubai, Dubai Knowledge Park, Dubai, UAE · Department of Academic Affairs, Mohammed Bin Rashid School of Government, City Walk, Building B02, Dubai, UAE

VLM Reasoning Medical VLMs Explainable Medical Image Analysis Tool Use in VLMs Radiology VLMs Efficient VLM Inference

Open paper AskPDF arXiv

Medical AI systems face two fundamental limitations. First, conventional vision-language models (VLMs) perform single-pass inference, yielding black-box predictions that cannot be audited or explained in clinical terms. Second, iterative reasoning systems that expose intermediate steps rely on fixed iteration budgets wasting compute on simple cases while providing insufficient depth for complex ones. We address both limitations with a unified framework. RVLM replaces single-pass inference with an iterative generate-execute loop: at each step, the model writes Python code, invokes vision sub-agents, manipulates images, and accumulates evidence. Every diagnostic claim is grounded in executable code, satisfying auditability requirements of clinical AI governance frameworks. RRouter makes iteration depth adaptive: a lightweight controller predicts the optimal budget from task-complexity features, then monitors progress and terminates early when reasoning stalls. We evaluate on BraTS 2023 Meningioma (brain MRI) and MIMIC-CXR (chest X-ray) using Gemini 2.5 Flash without fine-tuning. Across repeated runs, RVLM shows high consistency on salient findings (e.g., mass presence and enhancement) and can detect cross-modal discrepancies between Fluid-Attenuated Inversion Recovery (FLAIR) signal characteristics and segmentation boundaries. On MIMIC-CXR, it generates structured reports and correctly recognises view-specific artefacts. Code: https://github.com/nican2018/rvlm.

Abstract

Medical Diagnosis

Explore similar work

RVLM: Recursive Vision-Language Models with Adaptive Depth

MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence

CARE: Towards Clinical Accountability in Multi-Modal Medical Reasoning with an Evidence-Grounded Agentic Framework