Interactive image diagnosis using test-time backpropagation from the auxiliary task

Computer-aided diagnosis (CADx) has become vital in medical imaging, but automated systems often struggle to replicate the nuanced process of clinical interpretation. Expert diagnosis requires a comprehensive analysis of how abnormalities relate to each other across various views and time points, but current multi-view CADx methods frequently overlook these complex dependencies. Specifically, they fail to model the crucial relationships within a single view and the dynamic changes lesions exhibit across different views. This limitation, combined with the common challenge of incomplete data, greatly reduces their predictive reliability. To address these gaps, we reframe the diagnostic task as one of relationship modeling and propose GIIM, a novel graph-based approach. Our framework is uniquely designed to simultaneously capture both critical intra-view dependencies between abnormalities and inter-view dynamics. Furthermore, it ensures diagnostic robustness by incorporating specific techniques to effectively handle missing data, a common clinical issue. We demonstrate the generality of this approach through extensive evaluations on diverse imaging modalities, including CT, MRI, and mammography. The results confirm that our GIIM model significantly enhances diagnostic accuracy and robustness over existing methods, establishing a more effective framework for future CADx systems.

While multi-task learning based automatic radiology report generation (RRG) is widely adopted to ensure clinical consistency, most focus on architectural designs yet remain limited to coarse linear scalarization strategies. These strategies cannot effectively balance the hard constraints of discriminative clinical supervision with the smoothness requirements of report generation. To address these problems, we analyze the failure mechanism of linear scalarization from the perspective of gradient dynamics, utilizing the stochastic differential equation (SDE) framework to characterize it as a "Double Dilemma" of drift term deviation and diffusion term decay. Based on this, we propose a backbone-agnostic optimizer named Conflict-Averse Magnitude-Enhanced Gradient Descent (CAME-Grad). Through conflict-averse direction rectification and magnitude-enhanced energy injection, the algorithm not only ensures geometric validity, but also avoids local optimal solutions. Then, the adaptive gradient fusion mechanism is used to establish a dynamic balance between the theoretical optimal direction and the task-specific inductive bias. Experiments show that as a universal plug-and-play optimizer, CAME-Grad brings substantial and consistent improvements across eight diverse RRG methods, elevating overall clinical efficacy performance by an average of 2.3% on MIMIC-CXR and 1.9% on IU X-Ray. Our code is available at https://github.com/vpsg-research/CAME-Grad.

OncoVision is a privileged-information training framework that uses mammography images and clinical features during training and performs inference from mammographic images alone. Employing an attention-based encoder-decoder backbone, it jointly segments four regions of interest (masses, calcifications, axillary findings, and breast tissue) with accuracy exceeding the nnU-Net baseline and predicts ten structured clinical features, including BI-RADS category. We developed two late-fusion strategies, Independent and Dependent, that integrate imaging, radiomic, and clinical information during training to improve diagnostic precision and potentially reduce inter-observer variability. Radiomic features extracted from predicted masks provide shape, intensity, and texture descriptors that complement the learned CNN representations. We evaluated OncoVision in a retrospective multi-reader study with six board-certified radiologists, assessing diagnostic confidence, reading time, and segmentation accuracy with and without AI assistance. In a paired reader-assistance evaluation, OncoVision was associated with higher diagnostic confidence for junior and senior radiologists, reduced reading time by up to 61%, and achieved segmentation accuracy comparable to or exceeding that of radiologists for mass lesions. We operationalized OncoVision as a secure web application, now deployed at a partner hospital, that generates structured reports with dual-confidence scoring and attention-weighted visualizations for real-time diagnostic support. The platform is designed for integration into clinical workflows, with the goal of supporting screening access in underprivileged regions. By combining accurate segmentation with clinical intuition, OncoVision advances AI-assisted mammographic interpretation, offering a scalable and accessible approach to earlier and more consistent image interpretation.