An Explainable Vision-Language Model Framework with Adaptive PID-Tversky Loss for Lumbar Spinal Stenosis Diagnosis

Md. Sajeebul Islam Sk.; Md. Mehedi Hasan Shawon; Md. Golam Rabiul Alam

cs.CVApr 2, 2026

An Explainable Vision-Language Model Framework with Adaptive PID-Tversky Loss for Lumbar Spinal Stenosis Diagnosis

Authors: Md. Sajeebul Islam Sk., Md. Mehedi Hasan Shawon, Md. Golam Rabiul Alam

Organizations: Department of Computer Science and Engineering, , Dhaka, 1212, , Bangladesh. · Department of Electrical and Computer Engineering, , College Park, 20742, Maryland, USA.

Abstract

Lumbar Spinal Stenosis (LSS) diagnosis remains a critical clinical challenge, with diagnosis heavily dependent on labor-intensive manual interpretation of multi-view Magnetic Resonance Imaging (MRI), leading to substantial inter-observer variability and diagnostic delays. Existing vision-language models simultaneously fail to address the extreme class imbalance prevalent in clinical segmentation datasets while preserving spatial accuracy, primarily due to global pooling mechanisms that discard crucial anatomical hierarchies. We present an end-to-end Explainable Vision-Language Model framework designed to overcome these limitations, achieved through two principal objectives. We propose a Spatial Patch Cross-Attention module that enables precise, text-directed localization of spinal anomalies with spatial precision. A novel Adaptive PID-Tversky Loss function by integrating control theory principles dynamically further modifies training penalties to specifically address difficult, under-segmented minority instances. By incorporating foundational VLMs alongside an Automated Radiology Report Generation module, our framework demonstrates considerable performance: a diagnostic classification accuracy of 90.69%, a macro-averaged Dice score of 0.9512 for segmentation, and a CIDEr score of 92.80%. Furthermore, the framework shows explainability by converting complex segmentation predictions into radiologist-style clinical reports, thereby establishing a new benchmark for transparent, interpretable AI in clinical medical imaging that keeps essential human supervision while enhancing diagnostic capabilities.

Figures & tables

Figure 1: Detailed Model Architecture, the proposed multimodal vision-language framework for Lumbar Spinal Stenosis (LSS) diagnosis.

Model	Evaluation Metric	Grade A	Grade B&C	Grade D	Macro Average	Accuracy
LLaVA-Med	Precision	0.9366	0.8495	0.9387	0.9083	90.69%
	Recall	0.8892	0.8890	0.9425	0.9069
	F1-Score	0.9123	0.8688	0.9406	0.9072
BiomedCLIP	Precision	0.9225	0.7867	0.9407	0.8833	87.77%
	Recall	0.9124	0.8740	0.8468	0.8777
	F1-Score	0.9175	0.8280	0.8913	0.8789

Table 1: Diagnostic metrics for Vision-Language Models in the classification of lumbar spinal stenosis. The table shows precision, recall, F1-score and accuracy for three anatomical grades (A: normal, B&C: mild-to-moderate, D: severe) using three different model types: LLaVA-Med, BiomedCLIP and SmolVLM. Bold values indicate the best-performing model for each metric.

Figure 2: Classification performance comparison across multi-modal VLM models. (a–c) Confusion matrices from the clinical test set for BiomedCLIP, LLaVA-Med, and SmolVLM, respectively, displaying predicted versus true severity grades (A: normal, B&C: mild-to-moderate stenosis, D: severe stenosis). (d) Receiver operating characteristic (ROC) curves quantifying model discrimination performance across severity grades.

Model	Dice Score (CI)	IoU	Precision	Recall	Specificity
LLaVA-Med	0.9624 (0.951-0.973)	0.9288	0.9655	0.9610	0.9412
BiomedCLIP	0.9512 (0.938-0.964)	0.9147	0.9570	0.9574	0.9279
SmolVLM	0.8603 (0.841-0.879)	0.7566	0.8402	0.8841	0.8567

Table 2: Pixel-level segmentation performance metrics for Vision-Language Models in lumbar spinal stenosis. The table reports Dice Score with 95% Confidence Intervals (CI), Intersection over Union (IoU), Precision, Recall (Sensitivity), and Specificity across LLaVA-Med, BiomedCLIP, and SmolVLM. Bold values indicate the best-performing model for each metric.

Model	Metric	Grade A	Grade B&C	Grade D
LLaVA-Med	Dice Score	0.9650	0.9422	0.9800
	IoU	0.9310	0.8954	0.9600
	Sensitivity	0.9650	0.9380	0.9800
BiomedCLIP	Dice Score	0.9754	0.8965	0.9816
	IoU	0.9056	0.8705	0.9680
	Sensitivity	1.0000	0.8955	0.9765

Table 3: Quantitative evaluation of segmentation performance across clinical severity grades (A: normal, B&C: mild-to-moderate, and D: severe). The table reports Dice Score, Intersection over Union (IoU), and Sensitivity for three Vision-Language Models. Bold values indicate the best-performing model for each metric and grade.

Figure 3: Segmentation-based severity classification performance across multi-modal VLM models. (a–c) Confusion matrices mapping pixel-level segmentation outputs to clinical severity grades for BiomedCLIP, LLaVA-Med, and SmolVLM (all trained with the proposed Adaptive PID-Tversky loss). (d) Receiver operating characteristic (ROC) curves quantifying the models’ spatial discrimination performance derived from segmentation masks across clinical severity levels.

Figure 4: A detailed pixel-level segmentation analysis that compares the predictions of the BiomedCLIP model to expert-annotated ground truths for different levels of stenosis severity (Grade A, Grade B&C, and Grade D). There are three rows in the figure, each with a label: (a), (b), and (c). Each row shows a different patient case and stenosis grade. The first two images in each column show the model input: (1) the original axial MRI scan of the lumbar spine and (2) the ground truth stenosis region (highlight in dark maroon with the exact coverage percentage of the spine area and total annotated pixels reported). The last two images show the model predicted output: (3) the BiomedCLIP model’s predicted stenosis segmentation mask (represented in green, with the predicted coverage percentage of the spine area, mean confidence score, and maximum confidence score reported) and (4) the overlay comparison analysis showing pixel-level performance metrics (Dice coefficient and IoU) along with highlighted yellow areas for correctly identified true positives, red for missed regions (false negatives), and green for false positive with the exact pixel counts and percentages provided.

Model	BLEU-1	BLEU-2	BLEU-3	BLEU-4	ROUGE-1	ROUGE-2	ROUGE-L	METEOR	CIDEr	Jaccard	TF-IDF	Dist-1	Dist-2
LLaVA-Med	88.15%	87.50%	86.85%	86.10%	90.50%	88.95%	90.45%	91.05%	92.80%	88.10%	88.50%	0.0032	0.0065
BiomedCLIP	84.78%	84.00%	83.40%	82.82%	87.26%	84.83%	87.02%	87.22%	89.38%	84.25%	85.34%	0.0029	0.0059
SmolVLM	86.90%	86.12%	85.44%	84.84%	89.14%	86.82%	88.91%	89.13%	90.94%	86.34%	86.85%	0.0028	0.0057

Table 4: Natural Language Generation (NLG) performance metrics of different Vision-Language Models with Adaptive PID-Tversky loss function for the Automated Radiology Report Generation (ARRG) module.

Figure 5: Report generation performance comparison across multi-modal VLM models. (a–c) Confusion matrices from the clinical test set for BiomedCLIP, SmolVLM, and LLaVA-Med, respectively. (d) ROC curves quantifying model discrimination performance across severity grades derived from the semantic content of the automated reports.

Model	Metric	Grade A	Grade B&C	Grade D
BiomedCLIP	Precision	0.8908	0.7968	0.9234
	Recall	0.9261	0.8219	0.8577
	F1-score	0.9081	0.8092	0.8894
	Dice Score	0.9833	0.8147	0.9543
	IoU	0.8960	0.7505	0.9200
SmolVLM	Precision	0.8969	0.7682	0.8426

Table 5: Performance of implicit mask segmentation and severity classification during the text generation process. Bold values indicate the best-performing model for each metric.

Figure 6: Detailed qualitative performance of the fine-tuned SmolVLM vision-language model in generating automatic radiology reports from lumbar spine MRI images across three different grades of spinal canal stenosis. The figure consists of three panels labeled (a), (b), and (c), each showing (left) the original patient MRI image and (right) two text boxes containing the model’s VLM Output (predicted report) and the Radiologist Report (Ground Truth) for direct comparison.

Model	Loss Function	Dice	IoU	Prec.	Recall	Spec.
LLaVA-Med	Standard Cross-Entropy	0.8512	0.7620	0.8120	0.8950	0.8500
	Adaptive PID-Tversky	0.9624	0.9288	0.9655	0.9610	0.9412
BiomedCLIP	Standard Cross-Entropy	0.7411	0.7133	0.7496	0.7470	0.9221
	Adaptive PID-Tversky	0.9512	0.9147	0.9570	0.9574	0.9279
SmolVLM	Standard Cross-Entropy	0.8464	0.7357	0.8082	0.8924	0.8261
	Adaptive PID-Tversky	0.8603	0.7566	0.8402	0.8841	0.8567

Table 6: Performance matrix for ablation studies of the proposed Adaptive PID-Tversky loss against the standard Cross-Entropy loss. Bold values indicate the best-performing model for each metric.

Radiology VLMs

Radiology vision-language models (VLMs) align medical images and text to support grounded diagnosis, reporting, retrieval, and clinical reasoning.

See all 109 papers

Beyond Fluency: A Clinical Benchmark and Anomaly-Enhanced Baseline for Spine MRI Report Generation
Aug 7, 2026Bruno Palau, Franziska Vogt, Daria Laslo +4VLM Evaluation Radiology Report Generation
ETH Zurich, Switzerland · 2Swiss Institute of Bioinformatics (SIB), Switzerland
MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence
Mar 28, 2026Woohyeon Park, Jaeik Kim, Sunghwan Steve Cho +7Medical VLMs Medical Image Anomaly Detection
AIDAS Laboratory, Seoul National University · Samsung Changwon Hospital · Samsung Medical Center +1
Bridging visual saliency and large language models for explainable deep learning in medical imaging
May 7, 2026Paul Valery Nguezet, Elie Tagne Fute, Yusuf Brima +2Radiology Report Generation Medical Image Classification
Department of Mathematics and Computer science, University of Dschang, Dschang, 67, Cameroon. · Computer Vision, Institute of Cognitive Science, Osnabrück University, Osnabrueck, D-49090, Lower Saxony, Germany. · Department of Mathematics, Rhodes University, Grahamstown, Eastern Cape, 6140, Eastern Cape, South Africa. +1

Aug 7, 2026J/K move · Enter open · S save

Beyond Fluency: A Clinical Benchmark and Anomaly-Enhanced Baseline for Spine MRI Report Generation

Bruno Palau, Franziska Vogt, Daria Laslo, Haobo Li+3

ETH Zurich, Switzerland · 2Swiss Institute of Bioinformatics (SIB), Switzerland

VLM Evaluation Radiology Report Generation Medical Image Grounding Explainable Medical Image Analysis Radiology Report Evaluation

Open paper AskPDF arXiv

Radiology reporting is time-consuming and subject to inter-rater variability, making automated report generation an attractive clinical application for Vision-Language Models (VLMs). We benchmark state-of-the-art VLMs on lumbar spine MRI with a focus on diagnostic accuracy and demonstrate that standard lexical and semantic metrics poorly reflect clinical correctness: fluent, well-structured reports can score highly while containing clinically meaningful diagnostic errors. To address this failure mode, we propose an architecture-agnostic framework that augments VLM inputs with spatially localized, disc-level anomaly heatmaps generated by a semi-supervised U-Net++ model. These heatmaps both improve anatomical sensitivity through explicit visual grounding and provide an independent interpretability output for clinical oversight, moving us closer to diagnostically reliable, visually grounded VLMs for lumbar spine MRI interpretation.

Abstract

Radiology VLMs

Explore similar work

Beyond Fluency: A Clinical Benchmark and Anomaly-Enhanced Baseline for Spine MRI Report Generation

MEDIC-AD: Towards Medical Vision-Language Model's Clinical Intelligence

Bridging visual saliency and large language models for explainable deep learning in medical imaging