Unleashing Video Language Models for Fine-grained HRCT Report Generation
Organizations: Bioengineering Department, Imperial College London, London, UK · School of Biomedical Engineering & lmaging Sciences, King’s College London, London,UK
Abstract
Generating precise diagnostic reports from High-Resolution Computed Tomography (HRCT) is critical for clinical workflow, yet it remains a formidable challenge due to the high pathological diversity and spatial sparsity within 3D volumes. While Video Language Models (VideoLMs) have demonstrated remarkable spatio-temporal reasoning in general domains, their adaptability to domain-specific, high-volume medical interpretation remains underexplored. In this work, we present AbSteering, an abnormality-centric framework that steers VideoLMs toward precise HRCT report generation. Specifically, AbSteering introduces: (i) an abnormality-centric Chain-of-Thought scheme that enforces abnormality reasoning, and (ii) a Direct Preference Optimization objective that utilizes clinically confusable abnormalities as hard negatives to enhance fine-grained discrimination. Our results demonstrate that general-purpose VideoLMs possess strong transferability to high-volume medical imaging when guided by this paradigm. Notably, AbSteering outperforms state-of-the-art domain-specific CT foundation models, which are pretrained with large-scale CTs, achieving superior detection sensitivity while simultaneously mitigating hallucinations.
Figures & tables
| NLG Metrics | CE Micro | CE Macro | CE Weighted | CE Sample | ||||||||||||||||
| Method | Dataset | BL-1 | BL-2 | BL-3 | BL-4 | MT-R | RG-L | BERT | P | R | F1 | P | R | F1 | P | R | F1 | P | R | F1 |
| CT2Rep [ 9 ] | CT-RATE | 47.91 | 38.23 | 32.41 | 28.04 | 45.62 | 45.43 | 88.10 | 26.39 | 10.50 | 14.10 | 17.03 | 12.59 | 10.65 | 24.44 | 10.37 | 11.35 | 17.36 | 9.07 | 10.86 |
| RadFM [ 18 ] | CT-RATE | 50.20 | 33.88 | 24.09 | 17.02 | 36.81 | 30.46 | 86.17 | 36.10 | 13.48 | 19.63 | 28.51 | 10.16 | 13.05 | 36.10 | 13.48 | 17.74 | 20.47 | 10.11 | 12.14 |
| Reg2RG [ 5 ] | CT-RATE | 44.89 | 33.43 | 26.42 | 21.08 | 38.25 | 24.41 | 86.18 | 28.47 | 11.06 | 15.93 | 19.76 | 8.17 | 10.48 | 26.19 | 11.06 | 14.51 | 19.81 | 11.51 | 12.19 |
| CT-CTCHAT [ 8 ] | CT-RATE | 42.81 | 31.07 | 23.38 | 17.63 | 37.62 | 32.50 | 86.35 | 25.13 | 37.48 | 30.08 | 19.97 | 29.68 | 21.66 | 25.70 | 37.48 | 28.35 | 25.25 | 34.35 | 25.31 |
| M3D-8B [ 2 ] | CT-RATE | 44.95 | 34.36 | 27.83 | 22.98 | 40.75 | 37.76 | 87.52 | 47.60 | 28.54 | 35.69 | 41.62 | 22.41 | 26.74 | 47.06 | 28.54 | 33.13 | 35.04 | 23.21 | 25.21 |