How Sensitive Are Radiomic AI Models to Acquisition Parameters?

Radiomics is the established approach for CT-based lung cancer phenotyping, yet comparisons with foundation models rarely isolate contributions of feature extractor, classification head, and segmentation choice, or test cross-cohort robustness. We benchmark five feature extractors (Curia, Curia-2, DINOv3, Radiomics2D, Radiomics3D), seven classification heads (TabPFN, TabICL, XGBoost, CatBoost, Random Forest, logistic regression, Ridge), and three segmentation regimes on five tasks: tumor volume and stage classification, 2-year survival prediction, histology classification, and age prediction. Models are trained on LUNG1 (n=338) and evaluated on an internal test set (n=84) and the external LUNG2 cohort (n=211), with worst-case cross-cohort performance as the primary metric. The dominant design factor is task-dependent: segmentation drives volume and stage classification, while classifier choice drives survival, histology, and age prediction. Radiomics is competitive for tumor volume, tumor stage and survival (partly due to label-derivation effects for the former); Curia variants reach comparable peak scores for survival; DINOv3 falls slightly short across tasks. Patch and slice aggregation have negligible impact. We recommend Curia with tumor segmentation and a CatBoost head as a safe default, achieving the best mean rank across the three primary clinical tasks, though task-specific selection consistently outperforms any cross-task default. When tumor delineations are unavailable, Curia-2 with lung segmentation and logistic regression offers a competitive alternative. All pipelines use a two-stage design suited to small cohort sizes where end-to-end fine-tuning would risk overfitting.

Artificial intelligence (AI) has achieved remarkable success in medical imaging, but it is widely recognized that these models often perform inconsistently across real-world clinical settings. Such inconsistencies occur when patient demographics and imaging protocols vary, for example, in detecting small tumors, analyzing scans from different contrast phases, or evaluating patients of different ages or sexes. To quantify these inconsistencies, we develop a large-scale, open benchmark of 85,355 CT scans that systematically evaluates 12 tumor-detection AI models across tumor size, location, patient subgroup, and imaging protocol. We leverage large language models (LLMs) to extract and organize subgroup information from clinical data, which makes the analysis both scalable and reproducible. Our benchmark reveals that current state-of-the-art AI models, optimized for average accuracy, perform poorly in rare or underrepresented subgroups, such as young, female African Americans. However, collecting sufficient annotated data for these rare cases is often impractical. The benchmark provides a foundation for building more reliable and robust AI models for tumor detection and highlighting the need for rigorous, subgroup-level evaluation in medical imaging and computer vision. Datasets, code

Background: Sensitivity of AI-assisted lung nodule detection systems is known to vary with CT acquisition parameters including radiation dose, reconstruction kernel, and slice thickness. However, the dependence of detection probability on nodule position within the reconstruction cycle -- the z-phase -- has not, to the author's knowledge, been characterized for deep learning-based detection systems. Methods: A retrospective analysis was performed using the LIDC-IDRI dataset. Detection results from a previously validated 154-case perturbation study were re-analyzed. For each consensus nodule (>=4-reader agreement), z-phase was defined as the fractional position of the nodule center within the reconstruction cycle, folded to [0, 0.5]. Detection sensitivity was stratified by z-phase bin, reconstruction interval (1mm, 3mm, 5mm), and by the ratio of reconstruction interval to nodule diameter (d/D). Results: At 5mm reconstruction interval, sensitivity was 71.6% vs 84.8% at 1mm baseline. Within the 5mm condition, sensitivity varied by 17.6 percentage points across z-phase bins. Stratified by d/D ratio, sensitivity was 92.4% for d/D < 0.5, 78.0% for 0.5 <= d/D < 1.0, and 61.4% for d/D >= 1.0, with a systematic z-phase effect present only in the d/D >= 1.0 stratum. Conclusions: AI detection sensitivity depends on the ratio of reconstruction interval to nodule diameter. When this ratio approaches or exceeds 1.0 -- as occurs for 3-6mm nodules at 5mm reconstruction -- z-phase becomes the dominant source of per-study detection variance. This stochastic effect is invisible to protocol-level quality metrics and not reflected in AI confidence scores.