cs.CVMar 12, 2026

LoV3D: Grounding Cognitive Prognosis Reasoning in Longitudinal 3D Brain MRI via Regional Volume Assessments

Authors: Zhaoyang Jiang, Zhizhong Fu, David McAllister, Yunsoo Kim, Honghan Wu

Organizations: School of Health & Wellbeing, University of Glasgow, Glasgow, UK · School of Life Science and Technology, University of Electronic Science and Technology of China, Chengdu, China · Institute of Health Informatics, University College London, London, UK

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Longitudinal brain MRI is essential for characterizing the progression of neurological diseases such as Alzheimer's disease assessment. However, current deep-learning tools fragment this process: classifiers reduce a scan to a label, volumetric pipelines produce uninterpreted measurements, and vision-language models (VLMs) may generate fluent but potentially hallucinated conclusions. We present LoV3D, a pipeline for training 3D vision-language models, which reads longitudinal T1-weighted brain MRI, produces a region-level anatomical assessment, conducts longitudinal comparison with the prior scan, and finally outputs a three-class diagnosis (Cognitively Normal, Mild Cognitive Impairment, or Dementia) along with a synthesized diagnostic summary. The stepped pipeline grounds the final diagnosis by enforcing label consistency, longitudinal coherence, and biological plausibility, thereby reducing the risks of hallucinations. The training process introduces a clinically-weighted Verifier that scores candidate outputs automatically against normative references derived from standardized volume metrics, driving Direct Preference Optimization without a single human annotation. On a subject-level held-out ADNI test set (479 scans, 258 subjects), LoV3D achieves 93.7% three-class diagnostic accuracy (+34.8% over the no-grounding baseline), 97.2% on two-class diagnosis accuracy (+4% over the SOTA) and 82.6% region-level anatomical classification accuracy (+33.1% over VLM baselines). Zero-shot transfer yields 95.4% on MIRIAD (100% Dementia recall) and 82.9% three-class accuracy on AIBL, confirming high generalizability across sites, scanners, and populations. Code is available at https://github.com/Anonymous-TEVC/LoV-3D.

Figures & tables

Figure 1: Model paradigms and the LoV3D training pipeline. Top : Published MRI classifiers produce only a two-class diagnostic label from MRI inputs. Middle and Bottom : multimodal VLM models that take a MRI and text prompt, and produce an output including (3-class) classification and a detailed LLM output. Specifically, Middle : 3D medical VLMs (RadFM, M3D-LaMed) and an ungrounded ablation (same pipeline, encoder warmed via diagnosis classification instead of regional regression). Bottom : LoV3D training introduces a three-stage learning paradigm to ground the VLM’s cognitive prognosis reasoning. Stage 0 warms the encoder via a regional volume regression task; Stages 1a–b learn structured clinical reasoning through projector alignment and LoRA SFT for a multi-task learning (selecting relevant brain regions; classifying region deterioration categories; the longitudinal progression classification); Stage 2 refines outputs via Verifier-guided DPO (dashed border marks optimized modules) for a more reasonable diagnostic summary generation. LoV3D produces both a classification result and structured LLM output (right) comprising imaging observations, chain-of-thought reasoning, verifiable facts, and a diagnostic summary. Fire/snowflake icons denote trainable/frozen modules.
MethodDX (%)F 1κ\kappaRegion (%)AD/CN (%)
Classification baselines
Liu et al. [ 17 ] ‡————91.1
Lian et al. [ 15 ] ‡————90.3
Zhang et al. [ 28 ] ‡————93.2
ResNet-50 (ours) ∗58.958.1.461—87.8
3D medical VLM baselines
Table 1: Comparison on the ADNI test set (258 subjects). DX = three-way diagnostic accuracy, F 1 = macro-averaged F 1 , κ\kappa = Cohen’s weighted kappa, Region = mean anatomical accuracy across five AD-signature regions, AD/CN = binary accuracy on the CN and Dementia subset (MCI subjects excluded, MCI predictions mapped to CN). ∗ Same MONAI ResNet-50 backbone with a linear head, trained on the same split. † VLM baselines produce no valid JSON; metrics are from heuristic extraction. ‡ Published results on different ADNI subsets; shown for reference. LoV3D (no-grounding) uses the full LoV3D pipeline but replaces Stage 0 regional volume regression with three-class diagnosis classification. LoV3D is evaluated on all 479 test scans (258 subjects, multiple visits).
 Diagnosis Report QualityClinical Safety
StageAccF 1κ\kappaRegionBLEU-4ROUGE-LF.Abn ↓\downarrowF.Sev ↓\downarrow
1a (projector)89.188.0.85179.7.431.6357.26.3
1b (+ LoRA)93.392.5.90581.6.354.5587.14.1
2 (+ DPO)93.793.3.91182.6.584.7635.02.2
Table 2: Stage-wise progression on the ADNI test set (479 scans). Each stage builds on the previous checkpoint. Region = mean accuracy across five AD-signature regions. F.Abn./F.Sev. = false abnormal/severe rate on normal anatomy ( ↓\downarrow = lower is better). Reasoning completeness is 100% and hallucination rate <{<} 1% at all stages.
DatasetMethodClassesSubjectsAcc (%)Bal. Acc (%)
MIRIADYee et al. [ 27 ]269—95.7
Lu et al. [ 19 ]26993.694.9
LoV3D (ours)26995.493.2
AIBLLteif et al. [ 18 ]366173.4—
Batool et al. [ 2 ]366176.461.4
LoV3D (ours)362182.974.2
Table 3: Cross-site generalization. LoV3D is evaluated zero-shot (no fine-tuning) on two independent datasets. Prior work results are reproduced from published numbers on the same datasets used as pure external test sets. MIRIAD baselines were trained on ADNI; AIBL baselines were trained on NACC.
More in this topic

Longitudinal Medical Image Analysis

Longitudinal medical image analysis studies how scans change over time to track disease, predict outcomes, and guide clinical decisions.

See all 100 papers

Explore similar work

CardsList

  1. 3D MRI-Based Alzheimer's Disease Classification Using Multi-Modal 3D CNN with Leakage-Aware Subject-Level Evaluation

    Mar 18, 2026Md Sifat, Sania Akter, Akif Islam +5Alzheimer's Disease ClassificationMultimodal Classification

    Department of Computer Science and Engineering, University of Rajshahi, Rajshahi, Bangladesh · School of Computer Science and Engineering, University of Aizu, Aizuwakamatsu, Japan · Department of Computer Science and Engineering, Pabna University of Science and Technology, Pabna, Bangladesh

  2. Brain3D: Brain Report Automation via Inflated Vision Transformers in 3D

    Feb 25, 2026Mariano Barone, Francesco Di Serio, Giuseppe Riccio +4Radiology Report GenerationMedical VLMs

    University of Naples Federico II, Department of Electrical Engineering and Information Technology, Italy · Northwestern University, Dept. of Computer Science, McCormick School of Engineering and Applied Science, United States · Pegaso University, Department of Information Science and Technology, Italy

  3. Brain-Adapter: A Dual-Stream Vision-Language MIL Framework for Comprehensive 3D CT Diagnosis of Acute Intracranial Pathologies

    Jun 22, 2026Zhenyu Yi, Zhiyun Song, Yusong Sun +6VLM Adaptation3D Medical Imaging

    School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, China · Department of Computing, Imperial College London, London, UK