DORMS: Image Segmentation via Depth-Ordered Region Merge Trees
Organizations: Department of Electrical Engineering, University of South Florida
Abstract
We introduce DORMS (Depth-Ordered Region Merging for Segmentation), a purely algorithmic approach to image segmentation from monocular depth maps, requiring no neural networks or learned parameters in the segmentation process. An off-the-shelf monocular depth estimator provides the input depth maps. Our method constructs depth-ordered merge trees through a deterministic union-find sweep, combining component area, depth persistence, and spatial contact constraints to guide region merging. A threshold search determines the segmentation granularity according to a specified target region count, while global depth detrending improves the separation of local geometric structures. Extensive experiments on COCO Panoptic, Cityscapes, and ADE20K demonstrate consistent improvements over classical segmentation baselines and competitive performance against zero-shot neural segmentation models. Notably, our method outperforms all evaluated zero-shot models on Cityscapes, while remaining competitive with several approaches on COCO and ADE20K. These results highlight the potential of purely algorithmic geometric reasoning for image segmentation without learned segmentation models.
Figures & tables
| Dataset / Method | Without | With | |
|---|---|---|---|
| COCO / DORMS | 0.2037 | 0.2332 | +0.0295 |
| Cityscapes / DORMS (stereo) | 0.0612 | 0.1045 | +0.0433 |
| ADE20K / DORMS | 0.1753 | 0.2206 | +0.0453 |
| COCO / Felzenszwalb | 0.1496 | 0.1516 | +0.0020 |
| COCO / SLIC | 0.1712 | 0.1626 | -0.0086 |
| Disparity preprocessing | H-mIoU | Covering |
|---|---|---|
| Original disparity | 0.0612 | 0.2963 |
| + Depth detrending | 0.1045 ( ) | 0.4751 ( ) |
| + Disparity preprocessing | 0.1163 ( ) | 0.4994 ( ) |
Appendix figures & tables20 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | |||
|---|---|---|---|
| COCO Panoptic | 6 | 16 | 0.2% |
| Cityscapes | 30 | 3 | 10% |
| ADE20K | 8 | 8 | 2% |
| 4 | 5 | 6 | 7 | 8 | 10 | |
|---|---|---|---|---|---|---|
| H-mIoU | 0.20114 | 0.20273 | 0.20357 | 0.20102 | 0.19624 | 0.18822 |
| Dataset | Before | After | Removed | Below |
|---|---|---|---|---|
| COCO | 6.054 | 5.927 | 0.127 | 371 (7.4%) |
| Cityscapes | 30.034 | 25.842 | 4.192 | 460 (92.0%) |
| ADE20K | 8.021 | 7.886 | 0.135 | 175 (8.8%) |
| Dataset | DA-V2 checkpoint | Parameters |
|---|---|---|
| COCO Panoptic | Small | 24.8M |
| Cityscapes | Large | 335.3M |
| ADE20K | Small | 24.8M |
| Depth source | Original | Detrended | |
|---|---|---|---|
| Stereo disparity | 0.0648 | 0.1023 | +0.0375 |
| Marigold | 0.0807 | 0.1195 | +0.0388 |
| Depth Anything V1 | 0.0884 | 0.1625 | +0.0741 |
| ZoeDepth | 0.0922 | 0.1307 | +0.0385 |
| Depth Anything V2 | 0.1110 | 0.1919 | +0.0809 |
| Dataset | Split | Images | GT regions/image |
|---|---|---|---|
| COCO Panoptic | val2017 | 5,000 | 11.35 |
| Cityscapes | Fine val | 500 | 29.34 |
| ADE20K | Val | 2,000 | 15.21 |
| Method | Input | Configuration | Setting | #R | H-mIoU | Covering |
| DiffCut | RGB | SSD-1B backbone | Zero-shot | 6.31 | 0.3245 | 0.6668 |
| SAM2 | RGB | 16 points | Zero-shot | 10.11 | 0.2783 | 0.6002 |
| SAM2 | RGB | 32 points | Zero-shot | 14.59 | 0.2648 | 0.6107 |
| CLIPSeg | RGB | 133-category vocabulary | Zero-shot | 13.31 | 0.2471 | 0.6684 |
| DORMS | DA-V2 Small | Depth-only | 5.93 | 0.2332 | 0.5119 | |
| SLIC | Depth (raw) | Classical | 5.16 | 0.1712 | 0.3823 |
| Method | Input | Configuration | Setting | #R | H-mIoU | Covering |
| DORMS | DA-V2 Large | Depth-only | 25.84 | 0.2086 | 0.5866 | |
| CLIPSeg | RGB | 19-category vocabulary | Zero-shot | 9.24 | 0.1506 | 0.6713 |
| SAM2 | RGB | 32 points | Zero-shot | 41.35 | 0.1464 | 0.4356 |
| SAM2 | RGB | 16 points | Zero-shot | 22.77 | 0.1365 | 0.4063 |
| DiffCut | RGB | SSD-1B backbone | Zero-shot | 5.54 | 0.1309 | 0.6834 |
| SAM | RGB | 16 points | Zero-shot | 26.53 | 0.1234 | 0.4536 |
| Method | Input | Configuration | Setting | #R | H-mIoU | Covering |
| DiffCut | RGB | SSD-1B backbone | Zero-shot | 6.20 | 0.2841 | 0.6329 |
| SAM2 | RGB | 32 points | Zero-shot | 11.92 | 0.2696 | 0.5474 |
| SAM2 | RGB | 16 points | Zero-shot | 8.66 | 0.2615 | 0.5357 |
| DORMS | DA-V2 Small | Depth-only | 7.89 | 0.2206 | 0.5126 | |
| CLIPSeg | RGB | 150-category vocabulary | Zero-shot | 17.25 | 0.2126 | 0.6488 |
| SAM | RGB | 16 points | Zero-shot | 26.95 | 0.1946 | 0.5558 |
| Quartile | range | Without | With | H-mIoU | 95% CI | -value | Improved images |
|---|---|---|---|---|---|---|---|
| Q1 | [0.0003, 0.3640] | 0.2127 | 0.2284 | +0.0158 | [0.0117, 0.0198] | 761 / 1250 | |
| Q2 | [0.3641, 0.5883] | 0.1954 | 0.2341 | +0.0387 | [0.0344, 0.0431] | 915 / 1250 | |
| Q3 | [0.5884, 0.7538] | 0.1974 | 0.2428 | +0.0454 | [0.0408, 0.0502] | 929 / 1250 | |
| Q4 | [0.7539, 0.9939] | 0.2092 | 0.2274 | +0.0182 | [0.0123, 0.0241] | 802 / 1250 |
| Method | Without | With | |
|---|---|---|---|
| Felzenszwalb (depth) | 0.1496 | 0.1516 | +0.0020 |
| SLIC (depth) | 0.1712 | 0.1626 | -0.0086 |
| Preprocessing variant | H-mIoU | Covering |
|---|---|---|
| Fill split | 0.1184 | 0.4287 |
| Log transform | 0.1170 | 0.5100 |
| Fill far | 0.1152 | 0.3875 |
| Median ( ) | 0.1143 | 0.4107 |
| Bilateral filtering | 0.1142 | 0.3957 |
| Half resolution | 0.1131 | 0.4007 |
| Disparity preprocessing + detrending | Detrending only | |||||
| H-mIoU | Covering | #R | H-mIoU | Covering | #R | |
| 4 | 0.0688 | 0.4708 | 4.0 | – | – | – |
| 5 | 0.0772 | 0.4887 | 4.9 | – | – | – |
| 7 | 0.0892 | 0.4994 | 6.9 | 0.0858 | 0.4751 | 7.0 |
| 10 | 0.1002 | 0.4969 | 9.9 | – | – | – |
| 15 | 0.1112 | 0.4913 | 14.4 | 0.1014 | 0.4343 | 14.6 |
| Method | CPU (s) | GPU (s) | CPU images repeats |
| Proposed segmentation stage | |||
| Ours (full search) | 1.1230 | – | |
| Ours (fixed ) | 0.0328 | – | |
| Classical methods | |||
| Felzenszwalb (depth) | 0.2719 | – | |
| SLIC (depth) | 0.0330 | – | |
| Method | COCO | ADE20K | Cityscapes |
|---|---|---|---|
| Ours (segmentation) | 231 | 233 | 465 |
| Ours (complete pipeline) | 1128 | 1311 | 3210 |
| Felzenszwalb | 231 | 257 | 811 |
| SLIC | 157 | 162 | 321 |
| Method | Params (M) | Alloc. | Reserv. | Process VRAM | Host RSS |
|---|---|---|---|---|---|
| Depth Anything V2 Small | 24.8 | 219 | 272 | 732 | 1407 |
| Depth Anything V2 Large | 335.3 | 1500 | 1758 | 2218 | 2246 |
| GroupViT | 55.7 | 278 | 326 | 786 | 1693 |
| Mask2Former | 47.4 | 375 | 510 | 972 | 1773 |
| SAM (16 points) | 93.7 | 2426 | 3880 | 4342 | 1698 |
| SAM (32 points) | 93.7 | 2426 | 3880 | 4342 | 1708 |
| COCO (held-out) | Cityscapes (val) | |||
|---|---|---|---|---|
| H-mIoU | Veto off | Veto on | ||
| 4 | 0.2241 | 10 | 0.1543 | 0.1550 |
| 5 | 0.2329 | 20 | 0.1952 | 0.1996 |
| 6 | 0.2357 | 25 | 0.1987 | 0.2067 |
| 7 | 0.2300 | 30 | 0.1945 | 0.2086 |
| 8 | 0.2252 | 35 | 0.1844 | 0.2061 |
Semantic Segmentation
Semantic segmentation assigns a class label to every image pixel, enabling detailed scene understanding across domains and sensors.
See all 447 papers