DAUPNet: Domain-Aware Uncertainty Modeling for Reliable Prototype Discrimination in Cross-Domain Few-Shot Semantic Segmentation
Organizations: Department of Mechanical Science and Engineering, Huazhong University of Science and Technology, China
Abstract
Cross-domain few-shot semantic segmentation (CD-FSS) has predominantly been formulated as learning domain-invariant representations or improving support-query correspondence. Nevertheless, large domain shifts still make prototype matching unreliable: inconsistent hierarchical responses corrupt the support representation, deterministic prototypes cannot express boundary and appearance ambiguity, and treating prototypes with different reliability equally during optimization weakens foreground-background separation. We therefore propose DAUPNet, a unified framework that reformulates cross-domain prototype matching as uncertainty-aware prototype discrimination. DAUPNet first harmonizes hierarchical support-query features to provide stable evidence, then represents foreground and background prototypes probabilistically, and finally uses their estimated uncertainty to regulate contrastive optimization. On four standard target domains, DAUPNet achieves 72.6% and 76.7% average mIoU in the 1-shot and 5-shot settings, respectively, including substantial gains on the two medical domains. These results demonstrate that modeling prototype uncertainty and incorporating it into optimization provides a robust and interpretable approach to CD-FSS under severe domain shift. The code is available at https://github.com/madness-Lei/DAUPNet
Figures & tables
| Method | Venue | Backbone | DeepGlobe | ISIC | Chest X-ray | FSS | Mean | |||||
| 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | 1-shot | 5-shot | |||
| SSP | ECCV-22 | ResNet-50 | 40.0 | 48.7 | 35.5 | 45.9 | 74.4 | 74.3 | 78.9 | 80.6 | 57.2 | 62.4 |
| PerSAM [ 21 ] | ICLR-24 | ViT-base | 36.1 | 40.7 | 23.3 | 25.3 | 30.0 | 30.1 | 60.9 | 66.5 | 37.6 | 40.6 |
| PATNet | ECCV-22 | ResNet-50 | 37.9 | 43.0 | 41.2 | 53.6 | 66.6 | 70.2 | 78.6 | 81.2 | 56.1 | 62.0 |
| APM | NeurIPS-24 | ResNet-50 | 40.9 | 44.9 | 41.7 | 51.2 | 78.3 | 82.8 | 79.3 | 81.8 | 60.0 | 65.2 |
| ABCDFSS | CVPR-24 | ResNet-50 | 42.6 | 49.0 | 45.7 | 53.3 | 79.8 | 81.4 | 74.6 | 76.2 | 60.7 | 65.0 |
| Exp. | Setting | HFH | PPR | VACL | DeepGlobe | ISIC | Chest X-ray | FSS | Mean | Delta |
| 1 | Baseline | 41.53 | 68.63 | 78.31 | 74.45 | 65.73 | 0.00 | |||
| 2 | +HFH | ✓ | 43.42 | 70.86 | 87.95 | 76.57 | 69.70 | +3.97 | ||
| 3 | +HFH+PPR | ✓ | ✓ | 45.19 | 71.41 | 89.96 | 77.16 | 70.93 | +5.20 | |
| 4 | +HFH+PPR+VACL | ✓ | ✓ | ✓ | 46.74 | 74.41 | 91.50 | 80.89 | 73.39 | +7.66 |
| DeepGlobe | ISIC | Chest X-ray | FSS | Mean | Mean | |
| 0.000 | 45.19 | 71.41 | 89.96 | 77.16 | 70.93 | 0.00 |
| 0.005 | 45.74 | 72.41 | 89.89 | 78.73 | 71.69 | +0.76 |
| 0.010 | 45.79 | 72.85 | 90.94 | 81.04 | 72.66 | +1.73 |
| 0.020 | 46.74 | 74.41 | 91.50 | 80.89 | 73.39 | +2.46 |
| 0.050 | 44.79 | 70.90 | 90.99 | 77.39 | 71.02 | +0.09 |
Appendix figures & tables9 assets
Supplementary material from the paper’s appendix.
Appendix
| Exp. | Prototype Source | Variance | mIoU |
| 1 | Single deterministic | 77.30 | |
| 2 | Deepest-layer fg/bg | 78.03 | |
| 3 | Deepest-layer fg/bg | ✓ | 79.32 |
| 4 | Multi-level fg/bg | 78.18 | |
| 5 | Multi-level fg/bg | ✓ | 80.89 |
| Exp. | Setting | Representation Path | VACL Input | Four-Domain Mean |
| 1 | Baseline | baseline feature | no VACL | 65.73 |
| 2 | Baseline + VACL | baseline feature | deterministic prototypes | 66.67 |
| 3 | HFH + VACL w/o PPR | HFH feature | HFH deterministic prototypes | 70.21 |
| 4 | HFH + PPR + VACL | HFH + PPR feature | PPR probabilistic prototypes and variances | 73.39 |
| Exp. | Setting | Variance Weight | Rank Weight | FG-BG Separation | FG Attraction | BG Attraction | mIoU |
| 1 | w/o VACL | 77.16 | |||||
| 2 | Standard contrastive | ✓ | ✓ | ✓ | 78.42 | ||
| 3 | Variance-aware contrastive | ✓ | ✓ | ✓ | ✓ | ✓ | 80.89 |
| 4 | Negative only | ✓ | ✓ | ✓ | 78.72 | ||
| 5 | Positive only | ✓ | ✓ | ✓ | ✓ | 79.59 | |
| 6 | FG positive + negative | ✓ | ✓ | ✓ | ✓ | 79.60 |
| Target Domain | MMD Distance | Baseline | DAUPNet | mIoU |
| DeepGlobe | 0.3821 | 41.53 | 46.74 | 5.21 |
| ISIC | 0.4546 | 68.63 | 74.41 | 5.78 |
| Chest X-ray | 0.4906 | 78.31 | 91.50 | 13.19 |
| FSS-1000 | 0.2072 | 74.45 | 80.89 | 6.44 |
| Method | Input Size | Params (M) | FLOPs (G) | FPS | Memory (MiB) |
| Baseline | 416 | 28.100673 | 96.035 | 61.88 | 323.90 |
| DAUPNet | 416 | 28.439873 | 99.978 | 51.56 | 376.13 |
| Baseline | 400 | 28.100673 | 89.555 | 62.87 | 305.98 |
| DAUPNet | 400 | 28.439873 | 93.204 | 51.81 | 355.96 |
| Exp. | PPR Configuration | DeepGlobe | ISIC | Chest X-ray | FSS-1000 | Mean |
| 1 | Deepest + variance | 45.78 | 72.84 | 89.97 | 79.32 | 71.98 |
| 2 | Multi-level + variance | 46.74 | 74.41 | 91.50 | 80.89 | 73.39 |
Few-Shot Semantic Segmentation
Few-shot semantic segmentation labels pixels in new image classes using only a small set of annotated examples.
See all 41 papers