DeepFAN, a transformer-based deep learning model for human-artificial intelligence collaborative assessment of incidental pulmonary nodules in CT scans: a multi-reader, multi-case trial
Authors: Zhenchen Zhu, Ge Hu, Weixiong Tan, Kai Gao, Chao Sun, Zhen Zhou, Kepei Xu, Wei Han, +11 more
Organizations: Department of Radiology, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China. · Theranostics and Translational Research Center, National Infrastructures for Translational Medicine, Institute of Clinical Medicine, State Key Laboratory of Complex Severe and Rare Diseases, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, China. · Artificial Intelligence Lab, Deepwise Healthcare, Beijing, China. · Department of Radiology, Peking University People’s Hospital, Beijing, China. · Department of Epidemiology and Health Statistics, Institute of Basic Medicine Sciences, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China. · Department of Biostatistics, Peking University First Hospital, Beijing, China. · Department of Radiology, Huangshi Central Hospital, Affiliated Hospital of Hubei Polytechnic University, Hubei Province, China. · Key Laboratory of Cerebrovascular Disease lmaging and Artificial Intelligence, Huangshi，Hubei Province, China · Department of Radiology, Wuhan Third Hospital, Tongren Hospital of Wuhan University, Wuhan, Hubei Province, China. · 4+4 Medical Doctor Program, Chinese Academy of Medical Sciences & Peking Union Medical College, Beijing, China. · Department of Medicine Imaging, School of Clinical Medicine, Southwest Medical University, Luzhou, China. · School of Computing and Data Science, The University of Hong Kong, Hong Kong SAR, China.
The widespread adoption of CT has notably increased the number of detected lung nodules. However, current deep learning methods for classifying benign and malignant nodules often fail to comprehensively integrate global and local features, and most of them have not been validated through clinical trials. To address this, we developed DeepFAN, a transformer-based model trained on over 10K pathology-confirmed nodules and further conducted a multi-reader, multi-case clinical trial to evaluate its efficacy in assisting junior radiologists. DeepFAN achieved diagnostic area under the curve (AUC) of 0.939 (95% CI 0.930-0.948) on an internal test set and 0.954 (95% CI 0.934-0.973) on the clinical trial dataset involving 400 cases across three independent medical institutions. Explainability analysis indicated higher contributions from global than local features. Twelve readers' average performance significantly improved by 10.9% (95% CI 8.3%-13.5%) in AUC, 10.0% (95% CI 8.9%-11.1%) in accuracy, 7.6% (95% CI 6.1%-9.2%) in sensitivity, and 12.6% (95% CI 10.9%-14.3%) in specificity (P<0.001 for all). Nodule-level inter-reader diagnostic consistency improved from fair to moderate (overall k: 0.313 vs. 0.421; P=0.019). In conclusion, DeepFAN effectively assisted junior radiologists and may help homogenize diagnostic quality and reduce unnecessary follow-up of indeterminate pulmonary nodules. Chinese Clinical Trial Registry: ChiCTR2400084624.
The accurate classification of benign and malignant pulmonary nodules in CT scans is critical for early lung cancer screening, yet remains challenging due to the multi-scale and heterogeneous nature of pulmonary nodules. While deep learning offers potential for auxiliary diagnosis, most existing models act as "black boxes", lacking the transparency and explainability required for trustworthy clinical integration. To address this issue, we propose M3Net, a novel 3D network for pulmonary nodule classification inspired by the hierarchical diagnostic workflow of radiologists, which integrates multi-scale contextual information from fine-grained structures to global anatomical relationships. Our framework constructs a progressive multi-scale input, from fine-grained nodule structures to local semantics and global spatial relationships. M3Net employs scale-specific encoders and ensures cross-scale semantic consistency through latent space projection and mutual information maximization. Extensive experiments on the public LIDC-IDRI dataset and a self-collected clinical dataset (USTC-FHLN) demonstrate that our method achieves state-of-the-art performance, with accuracies of 86.96% and 84.24% respectively, outperforming the best baseline by 3.26% and 2.17%. The results validate that M3Net provides a more robust and clinically relevant solution for pulmonary nodule classification. The code is available at https://github.com/jylEcho/M3-Net.
Jinyue Li, Yuzhou Yu, Jingjing Yang +7
Hefei Cancer Hospital of CAS, Institute of Health and Medical Technology, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei, 230031, PR China · University of Science and Technology of China, Hefei, 230026, PR China · Graduate School, Bengbu Medical College, Bengbu, 233030, PR China +4
Lung cancer remains one of the leading causes of cancer-related mortality worldwide, and Computed Tomography (CT) is a primary imaging tool for screening and followup assessment. After pulmonary nodule detection, radiologists manually assess anatomical location, diameter, margin characteristics, and attenuation type to support risk assessment and clinical decision-making. However, this post-detection workflow is time-consuming and can be affected by inter-observer variability. Existing Artificial Intelligence methods often focus on isolated tasks, limiting their use as a unified, clinically grounded interpretation framework. This study presents FZ-VLM, a two-stage Florence-Zephyr Vision Language Model framework for unified structured pulmonary nodule characterization in lung CT. The framework uses a fine-tuned Florence-2 model to extract radiological attributes from expert-annotated 2D axial CT slices, while a Zephyr-7B model uses these attributes to generate nodule descriptions, follow-up recommendations, and longitudinal analyses. Results showed that the Stage 1 model achieved 77.18% accuracy for anatomical location, 67.96% accuracy for margin characteristics, and 79.13% accuracy for attenuation type, with a Mean Absolute Error of 2.58 mm for diameter estimation, outperforming evaluated GPT-4-based baselines as well as the human baseline. Expert radiologist evaluation of Stage 2 showed 93.9% accuracy, 98.6% completeness score, 76.1% clinical relevance, and an overall score of 89.5%. Safety analysis showed that most outputs were clinically safe, although some follow-up recommendations still required expert review. To the best of our knowledge, this study presents the first two-stage Vision-Language Model framework for structured nodule characterization and clinical decision-making.
Pramit Dutta, Jenita Manokaran, Richa Mittal +2
College of Engineering, University of Guelph, Guelph, ON N1G 2W1, Canada · Holland Bloorview Kids Rehabilitation Hospital, Toronto, ON M4G 1R8, Canada · Guelph General Hospital, Guelph, ON N1E 4J4, Canada +1
Low dose Computed Tomography (CT) is the standard imaging modality for lung cancer screening. On CT scans, lung cancer typically manifests as pulmonary nodules, whose manual identification is time-consuming and prone to observer variability, image quality limitations, and suboptimal reading conditions. Moreover, subtle temporal changes are often difficult to quantify reliably in CT scans from the same patient across follow-up examinations. Deep learning-based approaches for the automatic detection and segmentation of lung nodules have demonstrated improved accuracy and efficiency. However, most existing approaches provide only coarse region-of-interest localisation, rely on prior lesion annotations or human supervision, or treat detection and segmentation as separate tasks. In this work, we present NoduleFlow, a framework for lung nodule detection, classification, and segmentation directly from series of CT volumes. Our proposal consists of four interconnected modules and operates in an automated manner at inference time, without requiring prior lesion annotations, manual initialisation, or region-of-interest pre-selection. We further investigate whether nodule localization benefits from the propagation of candidate regions across follow-up scans through inter-scan registration fields, as opposed to conventional independent per-scan analysis. The generalisability of the framework is systematically evaluated across four independent datasets. The results underscore the potential of automated pipelines for lung nodule analysis in screening and diagnostic settings.
Giulia Romoli, Filippo Ruffini, Francesco Di Feola +3
Department of Diagnostics and Intervention, Radiation Physics, Biomedical Engineering, Umeå University, Universitetstorget 4, Umeå, Sweden. · Unit of Artificial Intelligence and Computer Systems, Department of Engineering, Università Campus Bio-Medico di Roma, Via Alvaro del Portillo 21, Rome, Italy. · Department of Naval, Electrical, Electronic, and Telecommunications Engineering, University of Genoa, Via Opera Pia 11A, Genoa, Italy.