A General-Purpose Dual-Path Interaction Framework for Multi-View Medical Image Segmentation: Alleviating Data Volume Dependency in Small-Sample Regimes
Organizations: College of Medicine and Biological Information Engineering, Northeastern University, Shenyang, Liaoning, 110169, China · Department of Ultrasound, The Fourth Affiliated Hospital of China Medical University, Shenyang, Liaoning, 110000, China · Department of Cardiology, Qilu Hospital of Shandong University · The Key Laboratory of Intelligent Computing in Medical Image, Ministry of Education, Northeastern University, Shenyang, Liaoning, 110819, China
The performance of medical image segmentation depends on a large amount of annotated data, and most segmentation models experience a significant drop in performance in low-sample-size scenarios. Existing semi-supervised and weakly supervised methods rely on specific network architectures, while traditional multi-view-assisted segmentation methods require strictly paired samples, which limits their clinical applications. To address these issues, we propose DPIF, a general-purpose plug-and-play dual-path interaction framework compatible with arbitrary encoder-decoder segmentation networks, which leverages cross-view anatomical complementarity to reduce annotation reliance. DPIF first performs Sinkhorn optimal transport-based cross-view distribution alignment to establish global semantic correspondence without sample-level pairing. It then adopts conditional mutual information-guided feature decoupling to enable bidirectional knowledge transfer only within the shared semantic space, suppressing view-specific noise. Finally, persistent homology-based topological regularization encodes anatomical invariants as unsupervised priors to mitigate overfitting. Experiments were conducted using 24 baseline models on a private multicenter echocardiography dataset and two public datasets. The results show that DPIF-enhanced models outperform single-view baseline models when trained on comparable amounts of data. Notably, most dual-path variants trained on 40% - 60% of the data achieve performance comparable to that of single-view models trained on the full dataset. This demonstrates that DPIF provides a general-purpose and scalable solution for data-scarce multi-view medical image segmentation problems.
Figures & tables
Figure 1: DPIF Framework and Functional Modules. (A) Schematic diagram of the dual-branch interaction framework deployment. (B) Core interaction module, used to connect the two segmentation pipelines. (C) Two-layer feature decoupling method, which distinguishes between shared and proprietary features. (D) Feature distribution alignment algorithm, which enables bidirectional semantic interaction and enhancement. (E) Segmentation uncertainty-guided weight interaction method, which enables adaptive adjustment of interaction intensity.
Figure 2: Details on the use of the three datasets. For each dataset, 20% is randomly allocated as an independent test set, and the remaining data is used for training and validation using five-fold cross-validation.
Figure 3: Visualization of the segmentation results and the bar chart showing the percentage increase in DSC. (A)–(C) show the validation results of the segmentation results against the mask, where red indicates the overlapping regions, purple indicates over-segmented regions, and green indicates under-segmented regions. (D)–(F) show the percentage improvements in the DSC metric achieved by the dual-path method.
Figure 4: Comparison of Performance Improvements Achieved by the Multi-View Approach Across Three Datasets.
Figure 5: Ablation analysis regarding the amount of training data. The dashed line represents the model’s DSC for training on the entire dataset in a single-path pipeline.
Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.
Precise medical image segmentation is crucial for clinical diagnosis and treatment planning, yet relies heavily on expensive expert annotations. Semi-supervised medical image segmentation (SSMIS) offers a cost-effective solution but typically operates under the assumption of independent and identically distributed (i.i.d.) data, defaulting to random sampling. While statistically valid at scale, this strategy suffers from severe representation bias in low-data regimes, failing to capture the heterogeneous medical data manifold. To address this, we propose a highly data-efficient framework driven by distribution alignment. First, we introduce an offline Distribution-Aware Sample Selection strategy. By leveraging Vision Foundation Models (VFMs) and our designed Density-K-Center algorithm, we explicitly identify representative structural anchors, establishing a more representative labeled domain. Second, to bridge the remaining distribution gap, we propose the Memory-guided Copy-Paste (MCP) module. Tailored for the inherent class imbalance in medical scans, MCP leverages a semantic memory mechanism to retrieve historically consistent priors for cross-domain alignment, encouraging semantic consistency. Coupled with an easy-to-hard progressive schedule, this framework effectively mitigates early-stage pseudo-label noise. Extensive experiments on six diverse 2D and 3D datasets demonstrate strong segmentation performance, particularly in extremely low-labeled scenarios (\eg, 1/16 ratio).
Weihao Yan, Yeqiang Qian, Yi Dong +1
School of Automation and Intelligent Sensing, Shanghai Jiao Tong University, Shanghai, China · Department of Ultrasound, Xinhua Hospital Affiliated to Shanghai Jiao Tong University School of Medicine, Shanghai, China
Deep learning has greatly advanced medical image segmentation, but its success relies heavily on fully supervised learning, which requires dense annotations that are costly and time-consuming for 3D volumetric scans. Barely-supervised learning reduces annotation burden by using only a few labeled slices per volume. Existing methods typically propagate sparse annotations to unlabeled slices through geometric continuity to generate pseudo-labels, but this strategy lacks semantic understanding, often resulting in low-quality pseudo-labels. Furthermore, medical image segmentation is inherently a pixel-level visual understanding task, where accuracy fundamentally depends on the quality of local, fine-grained visual features. Inspired by this, we propose RADA, a novel Region-Aware Dual-encoder Auxiliary learning pipeline which introduces a dual-encoder framework pre-trained on Alpha-CLIP to extract fine-grained, region-specific visual features from the original images and limited annotations. The framework combines image-level fine-grained visual features with text-level semantic guidance, providing region-aware semantic supervision that bridges image-level semantics and pixel-level segmentation. Integrated into a triple-view training framework, RADA achieves SOTA performance under extremely sparse annotation settings on LA2018, KiTS19 and LiTS, demonstrating robust generalization across diverse datasets.
Adapting foundation models to medical segmentation typically requires either backbone fine-tuning or high-capacity task-specific decoders, both of which are difficult to fit reliably when annotations are scarce. We show that frozen DINOv3 features already contain useful structural and boundary cues for medical segmentation, and that the main bottleneck lies in how these features are read out. We propose DINO-MVR, a Multi-View Readout framework for annotation-efficient medical segmentation. DINO-MVR trains only lightweight MLP probes on features from the final three transformer blocks of a frozen DINOv3 backbone, without updating the backbone itself. At inference, each input is interpreted through complementary resolutions and test-time augmentations, whose probability maps are combined by entropy-weighted fusion and refined with simple spatial regularization. For volumetric inputs, Gaussian z-axis smoothing further improves inter-slice consistency. Under fixed evaluation protocols on endoscopy, dermoscopy, and MRI benchmarks, DINO-MVR achieves strong readout-only performance, including 0.895 Dice on Kvasir-SEG, 0.897 Dice on ISIC 2018, and 0.908 Dice on BraTS FLAIR whole-tumor segmentation. With only five annotated BraTS patients, it recovers 98.4% of the performance obtained by the 40-patient BraTS reference run. These results suggest that frozen self-supervised vision backbones can support accurate medical segmentation when paired with an effective multi-view readout.
Wei Jiang, Feng Liu, Nan Ye +1
The University of Queensland · The University of Newcastle