cs.CVOct 3, 2026

UniPro: Unified Multi-Mode Medical Image Segmentation from 2D Images to 3D Volumes via Propagation

Authors: Bangwei Guo, Yunhe Gao, Meng Ye, Yang Zhou, Difei Gu, Guoning Zhang, Leon Axel, Dimitris Metaxas

Organizations: Rutgers University · Stanford University · The University of Texas at Arlington · New York University

Read PDF
Ask AI
Cite

Copy citation

DOINot available for this paper

Abstract

Medical image segmentation remains fragmented along two axes: segmentation paradigms and data dimensionality. Existing methods are typically developed separately for semantic, in-context, and interactive segmentation, and are further specialized to either native 2D images or 3D volumetric data. In clinical practice, however, segmentation workflows take many forms: a case may be initialized by semantic prediction, reference-guided segmentation, or user interaction. Regardless of how it begins, fine-grained refinement is naturally performed on 2D views; for volumetric scans, such 2D edits must propagate coherently to the rest of the volume. We present UniPro, a unified model that bridges segmentation paradigms and data dimensionality, using propagation to extend 2D segmentation to 3D volumes. Our key insight is that volumetric propagation and in-context segmentation share the same reference-conditioned prediction mechanism, differing only in whether the reference image-mask pairs come from other cases or from previously segmented neighboring slices. Building on this view, UniPro supports semantic, in-context, interactive, and propagation-based segmentation within a single slice-based framework, using class priors, reference exemplars, user clicks, and neighboring-slice predictions as mode-specific conditioning inputs. To improve propagation reliability, UniPro further incorporates bidirectional and 3D supervision to regularize slice-wise propagation beyond per-slice losses. Extensive experiments across diverse modalities and anatomies show that UniPro achieves strong performance across all segmentation settings, enabling annotation-efficient 3D segmentation from sparse 2D initialization and reducing slice-by-slice correction effort.

Figures & tables

Figure 1: Comparison of medical image segmentation paradigms. Existing task-specific, universal, and propagation-based models cover only part of the design space, with limitations in segmentation modes, data dimensionality, or propagation capability. UniPro unifies multi-mode segmentation, native 2D and 3D data, and volumetric propagation in a single slice-based framework.
Figure 2: UniPro overview. (a) A unified slice-based segmentation pipeline for semantic, in-context, interactive, and propagation-based segmentation. UniPro uses a shared image encoder and decoder, with different settings instantiated through mode-specific conditioning inputs. Conditioning signals are converted into dense 2-D fusion features and sparse 1-D queries, which are jointly fed to the MoE Transformer decoder. Notably, propagation is formulated as in-context segmentation over neighboring slices, enabling coherent volumetric segmentation. (b) Two-stage training strategy. Stage 1 jointly pretrains semantic, in-context, and interactive segmentation with the shared backbone. Stage 2 further adapts the model to propagation with bidirectional and 3D consistency supervision.
MethodInternalExternalUnseen
Abd-1KCHAOSM&MsLiTSKiTSBTCVACDCBraTS
3D Upper Bound
nnInteractive76.3883.0478.5849.7663.0867.4479.8751.25
General Pretrained Model
SAM245.0056.6147.6023.0851.6137.1050.7132.38
Propagation Model
Table 1: Propagation-based segmentation (3D Dice, %, ↑\uparrow ). All methods are initialized from a single positive click on one seed slice and propagated to the remaining slices.
Method3D Datasets2D Datasets
InternalExternalInternalExternal
Abd-1KCHAOSM&MsLiTSKiTSBTCVACDCBreastBKAIChestISICBUSUW-SC
3D Upper Bound
nnU-Net (3D)88.4590.1585.4763.1481.2886.0288.59––––––
Semantic segmentation
nnU-Net (2D)79.1380.2985.4260.1275.9970.7384.6471.6883.4594.9586.7668.0079.78
Table 2: Semantic and in-context segmentation across internal and external datasets (Dice, %, ↑\uparrow ). All in-context models use one-shot inference; propagation is applied only to 3D volumes. nnU-Net (3D) is reported as a fully supervised upper bound. Best results in each block are in bold.
Table 3: Ablation studies on Stage-2 propagation training.
Figure 3: Ablation analysis of UniPro’s 3D propagation behavior.
Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 4: Transformer decoder with MoE-FFN. Queries attend to image features via cross-attention, followed by self-attention and MoE-FFN.
Dataset# clsModality3D Volumes2-D Slices
TrainValidateTestTrainValidateTest
AbdomenCT-1K [ 40 ]4CT35220763995625578665
CHAOS [ 31 , 30 ]4MRI36915498413091945
M&Ms [ 9 ]3MRI300-3402475-2821
LiTS [ 6 ]1CT104-275687-1625
KiTS [ 26 ]1CT168-425269-1609
Appendix
Table 4: Datasets statistics.
DatasetFull-vol DiceTarget-slice DiceΔ\Delta
Abd-1K86.1188.62+2.51
CHAOS87.0589.70+2.65
M&M86.0687.64+1.58
LiTS63.3266.95+3.63
KiTS73.0576.95+3.90
BTCV82.1984.40+2.21
Appendix
Table 5: Comparison between standard full-volume Dice (%) and target-containing-slice Dice. Results are averaged over organs within each dataset when applicable. Δ\Delta denotes target-containing-slice Dice minus full-volume Dice, reported in percentage points.
Seed sliceq=0.2q=0.2q=0.4q=0.4q=0.5q=0.5q=0.6q=0.6q=0.8q=0.8Random
Mean Dice89.8191.0390.9890.5389.4690.12
Δ\Delta vs. median-1.17+0.050.00-0.45-1.52-0.86
Appendix
Table 6: Seed-position robustness on ACDC under ground-truth seed-mask initialization. GT-init removes seed-slice segmentation error and isolates the effect of seed location on propagation. All results are 3D Dice on target-containing slices.
DatasetMeanSD95% CI
Abd-1K88.6212.64[87.19, 90.05]
CHAOS89.709.05[87.37, 92.04]
M&Ms87.6410.31[87.01, 88.28]
LiTS66.9528.60[55.15, 78.76]
KiTS76.9518.94[71.04, 82.85]
BTCV84.4014.70[81.32, 87.48]
Appendix
Table 7: Statistical analysis of propagation performance. (a) UniPro’s mean Dice (%), SD, and 95% CI. (b) Paired comparisons with MedSAM2, where Δ\Delta denotes the Dice difference in percentage points. Bold pp -values indicate significance at the unadjusted p<0.05p<0.05 level.
MethodDice ( ↑\uparrow )
Vol2Flow83.81
Sli2Vol+86.05
UniPro (GT seed)94.03
Appendix
Table 8: Liver propagation on AbdomenCT-1K with the same seed slice and ground-truth seed mask for all methods. Dice scores are percentages.
ModelParametersNative inputTime (s)Memory (GB)
3D nnU-Net31.84M1283128^{3} patch9.33.60
UniPro32.84M8×3×512×5128\times 3\times 512\times 512 chunk28.01.93
Appendix
Table 9: Inference runtime and memory on AbdomenCT-1K using one NVIDIA Quadro RTX 8000 GPU and batch size 1. Time is the median per volume; memory is peak GPU memory during inference.
Training strategyACDC Dice (GT seed)
Stage I only87.29
Stage II only, without Stage-I pretraining88.12
Full two-stage UniPro90.98
Appendix
Table 10: Training-stage ablation using only cardiac training data. All variants are evaluated on ACDC with the same ground-truth seed masks. Dice scores are percentages.
Figure 5: Qualitative comparison on pancreas segmentation under the single-click propagation setting. Each row shows a different method propagating from a single positive click ( ★\bigstar ) on the seed slice ( z=34z=34 ) to representative slices at z=22,28,43,48z=22,28,43,48 , with the resulting 3D reconstruction shown in the rightmost column. UniPro produces masks that closely follow the ground-truth pancreas boundary across the entire volume, while baselines exhibit boundary leakage (PAM, iSegFormer), partial coverage (nnInteractive, MedSAM2), or large over-segmentation toward neighboring organs (SAM2). Per-volume 3D Dice is reported next to each method name.
Figure 6: Selected success and failure cases on LiTS, KiTS, and BraTS. Each row shows the seed slice (red border), four additional slices ordered by absolute distance from the seed, and 3D ground-truth and predicted surfaces. Green contours and orange overlays denote ground truth and predictions; 3D surfaces use the same colors. zz is the slice index and Δ\Delta the absolute index distance in the preprocessed volume.
More in this topic

Medical Image Segmentation

Medical image segmentation delineates organs, lesions, and other structures in scans to support measurement, diagnosis, treatment planning, and research.

See all 788 papers

Explore similar work

CardsList
Jul 14, 2026

UniMedSeg: Unified In-Context Learning for Multi-Paradigm 2D/3D Medical Image Segmentation

Medical image segmentation foundation models are expected to generalize across diverse clinical scenarios, yet existing universal methods remain fragmented by prompt paradigms and spatial dimensions. Visual in-context learning, interactive segmentation, and language-guided segmentation are typically handled by paradigm-specific models, while 2D and 3D images are also modeled separately. Such isolation prevents heterogeneous annotations and data from being jointly absorbed by a single scalable model and limits cross-paradigm knowledge transfer. To address this bottleneck, we propose UniMedSeg, a Transformer-centric universal segmentation framework that maps visual examples, geometric interactions, language instructions, and 2D/3D images into a shared sequence space, enabling heterogeneous medical supervision to be jointly learned through a unified in-context interface without prompt- or dimension-specific branches. To overcome the long-sequence memory bottleneck caused by visual contexts, we introduce Decoupled Split Attention, which reduces attention complexity to linear while preserving hardware-friendly computation and focused context-target interaction. Extensively trained and evaluated on a large corpus curated from 27 public datasets, UniMedSeg achieves state-of-the-art performance across visual in-context, interactive, and language-guided segmentation without task-specific fine-tuning, demonstrating strong generalization on diverse held-out tasks. The code and model weights are publicly available at https://github.com/Lii1228/UniMedSeg

Yunzhou Li, Jiesi Hu, Yanwu Yang +5

Harbin Institute of Technology at Shenzhen, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China · University Hospital Tübingen, Tübingen, Germany +2

Interactive Medical Image SegmentationMedical Imaging Foundation ModelsText-Guided Image SegmentationMedical Image Segmentation3D Medical Image Segmentation
AskPDF
Aug 12, 2026

Inference-Time Orthogonal Seeding Enables Geometry-Aligned 3D Organ Segmentation for Slice-Propagation Methods

Dense voxel-level annotation remains a major bottleneck in 3D medical image segmentation. Single-slice propagation methods such as Sli2Vol reduce this burden by propagating one annotated seed slice through a volume using label-free registration. However, axial-only propagation accumulates errors with distance from the seed, especially in surface-distance metrics, because it ignores coronal and sagittal evidence and therefore underuses the 3D information available in CT/MRI volumes. To better leverage volumetric geometry, we study how key training and inference choices affect slice-propagation models, including single-axis versus multi-axis label-free registration, single-seed versus multi-seed propagation, and orthogonal seed configurations. Instead of propagating from a single axial seed, we use three orthogonal seeds---one axial, one coronal, and one sagittal---and fuse their propagated labels with a simple label-free rule. Our results show that the training paradigm has limited impact: an axially trained network applied to off-axis seeds captures nearly all the improvement, while explicit three-axis training adds little. Instead, performance is driven by inference-time seed geometry, especially orthogonality rather than the number of annotated slices, as a budget-matched three-axial control provides no benefit and can even degrade performance. On a multi-organ CT cohort, orthogonal seeding with the axial Sli2Vol backbone improves Dice by 21.9%, Normalized Surface Dice by 25.5%, and reduces Average Hausdorff Distance by 53.5% over the single-axis baseline.

Md Rakibul Haque, Tushar Kataria, Shireen Y. Elhabian

Kahlert School of Computing, University of Utah, Salt Lake City, USA · Scientific Computing and Imaging Institute, University of Utah, Salt Lake City, USA

Multi-Organ SegmentationMedical Image Segmentation3D Medical Image Segmentation
AskPDF
Mar 7, 2026

LightMedSeg: Lightweight 3D Medical Image Segmentation with Learned Spatial Anchors

Accurate and efficient 3D medical image segmentation is essential for clinical AI, where models must remain reliable under stringent memory, latency, and data availability constraints. Transformer-based methods achieve strong accuracy but suffer from excessive parameters, high FLOPs, and limited generalization. We propose LightMedSeg, a modular UNet-style segmentation architecture that integrates anatomical priors with adaptive context modeling. Anchor-conditioned FiLM modulation enables anatomy-aware feature calibration, while a local structural prior module and texture-aware routing dynamically allocate representational capacity to boundary-rich regions. Computational redundancy is minimized through ghost and depthwise convolutions, and multi-scale features are adaptively fused via a learned skip router with anchor-relative spatial position bias. Despite requiring only 0.48M parameters and 14.64~GFLOPs, LightMedSeg achieves segmentation accuracy within a few Dice points of heavy transformer baselines. Therefore, LightMedSeg is a deployable and data-efficient solution for 3D medical image segmentation. Code will be released publicly upon acceptance.

Kavyansh Tyagi, Vishwas Rathi, Puneet Goyal

National Institute of Technology Kurukshetra, India · Indian Institute of Technology Ropar, India

Medical Image Segmentation3D Medical Image SegmentationU-Net
AskPDF