A Contrastive Fewshot RGBD Traversability Segmentation Framework for Indoor Robotic Navigation
Authors: Qiyuan An, Tuan Dang, Fillia Makedon
Organizations: Department of Computer Science and Engineering, University of Texas at Arlington, 1225 West Mitchell, Arlington, TX 76019, USA · Uber, Sunnyvale, CA 94086 USA · Cognitive Robotics Lab, Department of Electrical Engineering and Computer Science, University of Arkansas, Fayetteville, AR, USA
Indoor traversability segmentation aims to identify safe, navigable free space for autonomous agents, which is critical for robotic navigation. Pure vision-based models often fail to detect thin obstacles, such as chair legs, which can pose serious safety risks. We propose a multi-modal segmentation framework that leverages RGB images and sparse 1D laser depth information to capture geometric interactions and improve the detection of challenging obstacles. To reduce the reliance on large labeled datasets, we adopt the few-shot segmentation (FSS) paradigm, enabling the model to generalize from limited annotated examples. Traditional FSS methods focus solely on positive prototypes, often leading to overfitting to the support set and poor generalization. To address this, we introduce a negative contrastive learning (NCL) branch that leverages negative prototypes (obstacles) to refine free-space predictions. Additionally, we design a two-stage attention depth module to align 1D depth vectors with RGB images both horizontally and vertically. Extensive experiments on our custom-collected indoor RGB-D traversability dataset demonstrate that our method outperforms state-of-the-art FSS and RGB-D segmentation baselines, achieving up to 9% higher mIoU under both 1-shot and 5-shot settings. These results highlight the effectiveness of leveraging negative prototypes and sparse depth for robust and efficient traversability segmentation.
Figures & tables
Fig. 1 : Illustration of our RGB-D few-shot segmentation framework. The support and query inputs consist of RGB images ( SRGB , QRGB ) and depth vectors ( SD , QD ), encoded separately and fused via a multi-modal fusion block to produce support and query features ( s , q ). The support mask Smask is used to mask-pool on s , yielding both positive and negative prototypes ( s+ , s− ). The query feature q is then refined into free-space and obstacle representations ( q+ , q− ), which are concatenated and digested by a lightweight decoder to generate the final query mask ( qmask ).
Fig. 2 : Proposed contrastive few-shot RGB-D segmentation framework. RGB and depth inputs are embedded with modality-specific backbones, fused into unified support and query features, and refined through prototype-based contrastive learning. A lightweight decoder then predicts the query segmentation mask for indoor freespaces.
Fig. 3 : Two-stage attention depth backbone. It transforms 1D depth vectors into spatially aligned embeddings by applying horizontal attention (beam alignment) followed by vertical attention (height projection), producing refined depth features for multi-modal fusion.
Fig. 4 : Summit-XL Steel platform
Building
ELB
ERB
NH
UC
RGB-D pairs
8732
5590
4610
36072
Manually labeled
649
658
433
521
Building
WH
Mocap
Heracleia
Total
RGB-D pairs
10899
10872
17295
91951
Manually labeled
292
-
-
2553
TABLE I : Statistics of the custom-collected indoor RGB-D traversability dataset.
Backbone
Method
1-shot
5-shot
Trainable / Total Params
Freespace
Obstacles
mIoU
Freespace
Obstacles
mIoU
CMX [ 32 ]
PANet [ 25 ]
74.8
52.01
63.4
76.24
54.81
65.52
2.5M / 65.7M
CWT [ 14 ]
83.51
64.82
74.16
84.87
64.38
74.68
1.2M / 98M
BAM [ 12 ]
86.03
73.39
78.91
85.67
74.69
80.19
2.8M / 102M
NCL (ours)
91.5
82.55
87.03
91.84
83.94
87.91
4.6M / 59.7M
DFormer [ 30 ]
PANet
76.45
53.16
64.8
78.54
56.46
67.5
4.3M / 14.7M
TABLE II : Quantitative results on the indoor traversability dataset under 1-shot and 5-shot settings. Our proposed NCL achieves the best mIoU across all settings while requiring only a small fraction of trainable parameters, since most backbone weights are frozen during adaptation.
Fig. 5 : Qualitative results on indoor traversability segmentation. Each row shows (1) the query RGB image, (2) the corresponding depth vector, (3) predictions without the two-stage depth attention module and without NCL, (4) predictions with the depth module but without NCL, and (5) predictions from the full model. The proposed depth module helps separate floors from walls/ceilings, while the NCL branch further improves recognition of thin obstacles (e.g., chair legs).
1-shot
Backbone
Setting
Freespace
Obstacles
mIoU
DFormer [ 30 ]
- H - W
85.84
69.1
77.47
- H + W
88.63
71.44
80.03
+ H - W
90.32
76.74
83.53
+ H + W
93.52
84.37
88.95
TABLE III : Ablation study on the two-stage depth attention module, where H and W represent horizontal and vertical attention blocks respectively.
1-shot
Backbone
Setting
Freespace
Obstacles
mIoU
DFormer [ 30 ]
+ p2p - n2p
88.36
72.96
80.66
+ p2p + n2p
93.52
84.37
88.95
TABLE IV : Ablation study on the negative contrastive learning branch n2p , where p2p represents the traditional positive prototype matching branch.
Field Robotics Engineering and Science Hub (FRESH), Illinois Autonomous Farm, University of Illinois at Urbana-Champaign (UIUC), IL. · Mobile Robotics Group, University of S˜ao Paulo, S˜ao Carlos, SP, Brazil.