A Contrastive Fewshot RGBD Traversability Segmentation Framework for Indoor Robotic Navigation
Authors: Qiyuan An, Tuan Dang, Fillia Makedon
Organizations: Department of Computer Science and Engineering, University of Texas at Arlington, 1225 West Mitchell, Arlington, TX 76019, USA · Uber, Sunnyvale, CA 94086 USA · Cognitive Robotics Lab, Department of Electrical Engineering and Computer Science, University of Arkansas, Fayetteville, AR, USA
Indoor traversability segmentation aims to identify safe, navigable free space for autonomous agents, which is critical for robotic navigation. Pure vision-based models often fail to detect thin obstacles, such as chair legs, which can pose serious safety risks. We propose a multi-modal segmentation framework that leverages RGB images and sparse 1D laser depth information to capture geometric interactions and improve the detection of challenging obstacles. To reduce the reliance on large labeled datasets, we adopt the few-shot segmentation (FSS) paradigm, enabling the model to generalize from limited annotated examples. Traditional FSS methods focus solely on positive prototypes, often leading to overfitting to the support set and poor generalization. To address this, we introduce a negative contrastive learning (NCL) branch that leverages negative prototypes (obstacles) to refine free-space predictions. Additionally, we design a two-stage attention depth module to align 1D depth vectors with RGB images both horizontally and vertically. Extensive experiments on our custom-collected indoor RGB-D traversability dataset demonstrate that our method outperforms state-of-the-art FSS and RGB-D segmentation baselines, achieving up to 9% higher mIoU under both 1-shot and 5-shot settings. These results highlight the effectiveness of leveraging negative prototypes and sparse depth for robust and efficient traversability segmentation.
Figures & tables
Fig. 1 : Illustration of our RGB-D few-shot segmentation framework. The support and query inputs consist of RGB images ( SRGB , QRGB ) and depth vectors ( SD , QD ), encoded separately and fused via a multi-modal fusion block to produce support and query features ( s , q ). The support mask Smask is used to mask-pool on s , yielding both positive and negative prototypes ( s+ , s− ). The query feature q is then refined into free-space and obstacle representations ( q+ , q− ), which are concatenated and digested by a lightweight decoder to generate the final query mask ( qmask ).
Fig. 2 : Proposed contrastive few-shot RGB-D segmentation framework. RGB and depth inputs are embedded with modality-specific backbones, fused into unified support and query features, and refined through prototype-based contrastive learning. A lightweight decoder then predicts the query segmentation mask for indoor freespaces.
Fig. 3 : Two-stage attention depth backbone. It transforms 1D depth vectors into spatially aligned embeddings by applying horizontal attention (beam alignment) followed by vertical attention (height projection), producing refined depth features for multi-modal fusion.
Fig. 4 : Summit-XL Steel platform
Building
ELB
ERB
NH
UC
RGB-D pairs
8732
5590
4610
36072
Manually labeled
649
658
433
521
Building
WH
Mocap
Heracleia
Total
RGB-D pairs
10899
10872
17295
91951
Manually labeled
292
-
-
2553
TABLE I : Statistics of the custom-collected indoor RGB-D traversability dataset.
Backbone
Method
1-shot
5-shot
Trainable / Total Params
Freespace
Obstacles
mIoU
Freespace
Obstacles
mIoU
CMX [ 32 ]
PANet [ 25 ]
74.8
52.01
63.4
76.24
54.81
65.52
2.5M / 65.7M
CWT [ 14 ]
83.51
64.82
74.16
84.87
64.38
74.68
1.2M / 98M
BAM [ 12 ]
86.03
73.39
78.91
85.67
74.69
80.19
2.8M / 102M
NCL (ours)
91.5
82.55
87.03
91.84
83.94
87.91
4.6M / 59.7M
DFormer [ 30 ]
PANet
76.45
53.16
64.8
78.54
56.46
67.5
4.3M / 14.7M
TABLE II : Quantitative results on the indoor traversability dataset under 1-shot and 5-shot settings. Our proposed NCL achieves the best mIoU across all settings while requiring only a small fraction of trainable parameters, since most backbone weights are frozen during adaptation.
Fig. 5 : Qualitative results on indoor traversability segmentation. Each row shows (1) the query RGB image, (2) the corresponding depth vector, (3) predictions without the two-stage depth attention module and without NCL, (4) predictions with the depth module but without NCL, and (5) predictions from the full model. The proposed depth module helps separate floors from walls/ceilings, while the NCL branch further improves recognition of thin obstacles (e.g., chair legs).
1-shot
Backbone
Setting
Freespace
Obstacles
mIoU
DFormer [ 30 ]
- H - W
85.84
69.1
77.47
- H + W
88.63
71.44
80.03
+ H - W
90.32
76.74
83.53
+ H + W
93.52
84.37
88.95
TABLE III : Ablation study on the two-stage depth attention module, where H and W represent horizontal and vertical attention blocks respectively.
1-shot
Backbone
Setting
Freespace
Obstacles
mIoU
DFormer [ 30 ]
+ p2p - n2p
88.36
72.96
80.66
+ p2p + n2p
93.52
84.37
88.95
TABLE IV : Ablation study on the negative contrastive learning branch n2p , where p2p represents the traditional positive prototype matching branch.
Safe autonomous navigation requires reliable estimation of environmental traversability. Traditional methods have relied on semantic or geometry-based approaches with human-defined thresholds, but these methods often yield unreliable predictions due to the inherent subjectivity of human supervision. While self-supervised approaches enable robots to learn from their own experience, they still face a fundamental challenge: the positive-only learning problem. To address these limitations, recent studies have employed Positive-Unlabeled (PU) learning, where the core challenge is identifying positive samples without explicit negative supervision. In this work, we propose GSAT, which addresses these limitations by constructing a positive hypersphere in latent space to classify traversable regions through anomaly detection without requiring additional prototypes (e.g., unlabeled or negative). Furthermore, our approach employs joint learning of anomaly classification and traversability prediction to more efficiently utilize robot experience. We comprehensively evaluate the proposed framework through ablation studies, validation on heterogeneous real-world robotic platforms, and autonomous navigation demonstrations in simulation environments.
Dongjin Cho, Miryeong Park, Juhui Lee +2
the Electrical and Computer Engineering and INHA Future Mobility IPCC, Inha University, Incheon, South Korea.
Advances in robotics and autonomous navigation systems hinge on the ability to predict terrain traversability accurately. Traditional methods for generating datasets to train these prediction models often involve putting robots into potentially hazardous environments, posing risks to equipment and safety. To solve this problem, we present ZeST, a novel approach that treats repeated VLM outputs as stochastic measurements and fuses them into an uncertainty-aware posterior. Our approach not only performs zero-shot traversability and mitigates the risks associated with real-world data collection but also accelerates the development of advanced navigation systems, offering a cost-effective and scalable solution. To support our findings, we present navigation results, in both controlled indoor and unstructured outdoor environments. As shown in the experiments, ZeST provides safer navigation with a 90-100% success rate while no baseline exceeds 60%, and keeps complete success under query latencies up to 3.6s.
Shreya Gummadi, Mateus V. Gasparino, Gianluca Capezzuto +2
Field Robotics Engineering and Science Hub (FRESH), Illinois Autonomous Farm, University of Illinois at Urbana-Champaign (UIUC), IL. · Mobile Robotics Group, University of S˜ao Paulo, S˜ao Carlos, SP, Brazil.
Visual traversability estimation is central to autonomous navigation, yet most approaches either rely on prompt-driven Vision-Language Model (VLM) or decouple traversability from trajectory planning, requiring separate planners with heavy mapping, manual tuning, and extended deployment time. We propose EmbodiedDiffusion, a diffusion-based framework that simultaneously predicts traversability maps and generates feasible trajectories from RGB images using planner-free synthetic supervision and embodiment conditioning for cross-platform transfer. The framework distills category-level traversability semantics from a VLM teacher into a lightweight student model during training, enabling prompt-free, real-time inference at deployment. A modular FiLM-based conditioning mechanism isolates embodiment-specific reasoning into a compact trainable subset of the network, allowing rapid adaptation to new robot platforms without retraining the visual backbone or the trajectory diffusion model. Across indoor environments with quadruped and aerial robots, EmbodiedDiffusion achieves 80-100% navigation success in the full-data regime with real-time inference (90 ms) and adapts to new platforms using only 10 min of visual data collection, demonstrating scalable, unified traversability reasoning and trajectory generation for heterogeneous robots.
Iana Zhura, Sausar Karaf, Faryal Batool +7
Intelligent Space Robotics Laboratory, Center for Digital Engineering, Skolkovo Institute of Science and Technology.