GateSPINE: Gated Cross-View Fusion for Lumbar Spine MRI Report Generation

Hoang Nguyen Van; Cuong Vuong Tuan; Trang Mai Xuan; Bien Tran Van; Nam Tran Van; Thien Van Luong

cs.CVSep 30, 2026

GateSPINE: Gated Cross-View Fusion for Lumbar Spine MRI Report Generation

Authors: Hoang Nguyen Van, Cuong Vuong Tuan, Trang Mai Xuan, Bien Tran Van, Nam Tran Van, Thien Van Luong

Organizations: Applied AI Lab, Phenikaa University, Hanoi, Vietnam · Medical Imaging & Radiological Technology Department, Faculty of Medical Technology, Phenikaa School of Medicine & Pharmacy, Phenikaa University, Vietnam · Radiology & Functional Exploration Center, Phenikaa University Hospital, Vietnam · Business AI Lab, College of Technology, National Economics University, Vietnam

Abstract

Automated report generation can ease the burden radiolo gists face when interpreting multi-sequence MRI studies. Unlike CT, MRI examinations comprise multiple sequences and imaging planes, each con tributing complementary diagnostic information. Existing methods en code a study as a single volume and combine multiple acquisitions by fixed rules. Findings visible in only one plane are thus diluted and of ten missed, lowering recall on clinical efficacy metrics, where a missed abnormality is most costly. We propose GateSPINE, a vision-language framework that fuses sagittal T1 and T2 volumes with a training-free operator, encodes the fused sagittal and axial volumes with two parallel 3D encoders, and decodes their combined representation into a report. Its core mechanism is a gated cross view fusion module that predicts, per feature channel and token, how much of each view to admit, so the more informative view dominates at each spatial location. We evaluate GateSPINE on three lumbar MRI datasets, comprising two public bench marks and a private cohort collected from Phenikaa University Hospital, using both natural language generation (NLG) and clinical efficacy (CE) metrics. GateSPINE achieves the highest CE F1 through improved re call on all three datasets; on SPIDER, which lacks an axial sequence, this reflects the sagittal fusion component rather than the gated cross-view mechanism, which is validated on the two cohorts with both imaging planes. GateSPINE also remains competitive on standard NLG metrics.

Figures & tables

Figure 1: Overview of the GateSPINE framework. Sagittal T1 and T2 volumes are fused by the TTD module, while the axial T2 volume is processed in parallel. Two ViT3D encoders extract features from the fused sagittal volume and the axial volume, and the Gated Cross-View Fusion layer combines them. The 3D Projector then compresses the fused features into visual tokens for the VLM decoder, which generates the final report.

	Method	NLP					CE Micro			CE Macro
	Method	BL-4	RG-1	RG-L	MTR	BERT	Pre.	Rec.	F1	Pre.	Rec.	F1
PhenikaaMec	CT2Rep [ 10 ]	25.91	57.97	44.21	45.58	89.78	66.67	2.37	4.58	61.11	2.08	3.99
	Dia-LLaMA [ 8 ]	28.16	58.99	45.05	50.57	90.88	74.79	52.05	61.38	71.79	47.75	53.55
	Med3DVLM [ 22 ]	4.35	36.99	20.27	22.78	87.03	36.51	6.69	11.30	28.86	6.68	10.19
	SPINE [ 11 ]	25.04	56.84	42.71	45.99	90.43	71.24	64.02	67.49	65.23	57.17	58.78
	GateSPINE	26.46	60.24	46.15	48.63	91.16	75.33	67.35	71.12	65.30	59.69	60.50

Table 1: Comparison with state-of-the-art methods on language-quality (NLP) and clinical-efficacy (CE) metrics across three datasets. Best and second-best results in bold and underlined .

Figure 2: F1 score for each clinical finding on PhenikaaMec.

	Sequence			NLP Metrics					CE (F1)
Dataset	T1	T2	Ax	BL-4	RG-1	RG-L	MTR	BERT	Micro	Macro
PhenikaaMec	✓			19.69	54.12	39.20	40.81	90.30	31.12	17.53
		✓		21.81	54.36	41.16	42.10	90.18	56.25	46.50
			✓	22.11	55.64	42.23	44.53	90.27	48.37	39.77
	✓	✓		23.50	55.52	42.63	43.90	90.43	49.51	45.16
	✓	✓	✓	26.46	60.24	46.15	48.63	91.16	71.12	60.50

Table 2: Ablation on input MRI sequences. Best in bold , second best underlined .

Dataset	Fusion	Decoder	NLP Metrics					CE (F1)
Dataset	Fusion	Decoder	BL-4	RG-1	RG-L	MTR	BERT	Micro	Macro
PhenikaaMec	Concatenation	MedGemma	23.61	56.80	43.58	44.67	90.59	53.11	46.65
	Dynamic	MedGemma	24.37	56.60	44.31	45.37	90.73	63.01	53.46
	Gate	MedGemma	26.46	60.24	46.15	48.63	91.16	71.12	60.50
	Gate	Phi-3-mini	20.94	53.47	39.28	41.51	89.93	56.32	46.89
Lumbar	Concatenation	MedGemma	18.56	44.25	40.79	40.56	89.59	53.90	23.47

Table 3: Ablation on the cross-view fusion mechanism (rows 1–3, MedGemma decoder) and on the language decoder (rows 3–4, gate fusion; MedGemma 4B vs. Phi-3-mini 3.8B). Best in bold , second best underlined among fusion variants.

Figure 3: Generated report comparison: Ground-truth, Ours, and SPINE, one case per dataset.

Explore similar work

A multi-agent system for spine MRI report generation from multi-sequence imaging
Jun 8, 2026Zhiping Xiao, Junwei Yang, Gongbo Sun +12Magnetic Resonance Imaging Dataset Multimodal Clinical Data
1Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA · School of Computer Science, Peking University, Beijing, China · Department of Computer Sciences, University of Wisconsin–Madison, Madison, WI, USA +3
Beyond Fluency: A Clinical Benchmark and Anomaly-Enhanced Baseline for Spine MRI Report Generation
Aug 7, 2026Bruno Palau, Franziska Vogt, Daria Laslo +4Medical Vision-Language Models Radiology Report Generation
ETH Zurich, Switzerland · 2Swiss Institute of Bioinformatics (SIB), Switzerland
SpineReport: Automated 3D Quantification and Reporting of Lumbar Spine Degeneration on MRI
Jun 8, 2026Nathan Molinier, Adrian A. Marth, Reto Sutter +6Morphometric Analysis Multi Organ Computed Tomography Segmentation
1NeuroPoly, Institute of Biomedical Engineering, Polytechnique Montreal, Montreal, QC, Canada · 2Mila, Quebec AI Institute, Montreal, QC, Canada · Department of Radiology, Balgrist University Hospital, Zurich, Switzerland +7

Jun 8, 2026, cs.CVJ/K move · Enter open · S save

A multi-agent system for spine MRI report generation from multi-sequence imaging

Zhiping Xiao, Junwei Yang, Gongbo Sun, Han Zhang+11

1Paul G. Allen School of Computer Science & Engineering, University of Washington, Seattle, WA, USA · School of Computer Science, Peking University, Beijing, China · Department of Computer Sciences, University of Wisconsin–Madison, Madison, WI, USA+3

Magnetic Resonance Imaging Dataset Multimodal Clinical Data Report Diagnosis Digital Pathology Encoders

Open paper AskPDF arXiv

Spinal pathology is a leading cause of pain and disability worldwide. Spine MRI is central to clinical evaluation, yet its interpretation remains complex and time-consuming, requiring integration of information across multiple imaging sequences and anatomical regions. Despite recent advances in automated MRI analysis, effectively combining multi-sequence data while preserving sequence-specific diagnostic information remains an open challenge. Here we present SpineAgent, a multi-agent framework for spine MRI report generation built upon a multi-sequence foundation model trained on routine clinical data from 32,047 patients and 453,683 MRI series, comprising a total of 13,441,191 MRI slices. To accommodate diverse modalities of sequences, we first pre-train two DINOv3-based encoders separately on T1- and T2-weighted sequences. We then introduce a continual training strategy that learns a synthesizer to embed images of other sequences using the T1 and T2 encoders, producing patient-level embedding that integrates various signals across MRI sequences. Using these embeddings, SpineAgent achieves state-of-the-art performance, and demonstrates strong generalizability under cross-manufacturer and cross-cohort evaluation. Beyond classification, SpineAgent enables pathology localization by identifying findings-relevant slices and segmenting pathological regions. It also supports multimodal image-report retrieval, providing a solid foundation for scalable and explainable MRI report generation. We further integrate these validated capabilities of SpineAgent into 37 specialized agents. Finally, we incorporate their outputs as structured tokens within a Medical Report Agent trained end-to-end for report generation. Through both automated metrics and expert evaluation by five radiologists, SpineAgent achieves leading performance in spine MRI report generation.

Abstract

Explore similar work

A multi-agent system for spine MRI report generation from multi-sequence imaging

Beyond Fluency: A Clinical Benchmark and Anomaly-Enhanced Baseline for Spine MRI Report Generation

SpineReport: Automated 3D Quantification and Reporting of Lumbar Spine Degeneration on MRI