Reliable recognition of accessibility-critical objects (e.g., audible pedestrian signals, door-activation buttons, and handrails) is essential for assistive navigation technologies that support safe, independent mobility for blind and low-vision (BLV) users. Yet these categories are underrepresented in existing large-scale vision datasets, and the few datasets that include them suffer from limited class coverage, inconsistent annotations, and poor diversity. We introduce NavAble, a large-scale dataset spanning 11 accessibility-critical object classes, combining 41K curated open-source images with 8K newly collected, densely annotated real-world images. To extend scale and diversity, we develop a synthetic data generation pipeline that renders high-fidelity 3D assets across 37 environments under diverse styles and viewpoints, yielding 452K frames with rich ground-truth annotations from 565 assets. The pipeline minimizes manual effort by combining automated web image crawling, VLM-based filtering, SAM 3D-based asset generation, and convenient camera trajectory configuration, enabling large-scale data generation at a diversity unattainable through manual collection. Experiments show that fine-tuning existing segmentation models on the NavAble Dataset achieves reliable accessibility-object segmentation (>80% mIoU on our held-out test set). We also demonstrate the effectiveness of synthetic data augmentation, which yields up to +3.5 mIoU over real-world-only fine-tuning. Beyond benchmark evaluation, validation on egocentric data from two mobility-assistive robots demonstrates improved perception reliability in real-world navigation scenarios. Together, NavAble establishes a scalable foundation for BLV navigation perception, directly addressing the data scarcity and embodiment gaps. The dataset and pipeline are publicly available at https://huggingface.co/datasets/NavAble/NeurIPS_2026_BLV.
Figures & tables
Figure 1: Overview of NavAble Dataset. Existing datasets limited object classes (e.g., lacking APS or door buttons). NavAble Dataset provides a curated multi-source collection of open-source, newly collected real-world data, and synthetic data across 11 accessibility-critical object categories.
Figure 2: Inconsistent and low-quality fragmented online data (left) are curated and normalized to merge with our newly collected high-quality dense annotation data (right).
Figure 3: Overview of NavAble Generator . (Left) Based on the user’s object of interest, we generate 3D assets from images by using foundation models. (Center) Our generator only requires two pieces of information from the user: the asset location and the camera trajectory. Once those are given, the generator creates augmented data. (Right) NavAble generator creates data in diverse scenes, lighting conditions, and environments via Isaac Sim.
Table 1: Quantitative metrics and qualitative comparison across segmentation models. mIoU, precision, and recall on the held-out NavAble -Real test split. The lower block shows one qualitative example for different classes (APS button, bus stop sign, crosswalk, elevator, top to bottom): the leftmost panel is the ground-truth mask and the remaining panels show each model’s prediction in the same column order as the metrics above.
Box
Point
Model
AP
AP 50
Pt-in-mask
PCK .05
PCK .10
Gemini Robotics-ER 1.6
33.8
48.9
69.0
68.7
85.5
Gemini 3 Pro
31.5
45.9
68.6
61.2
79.4
Qwen3-VL-32B
20.0
29.7
59.0
59.7
79.2
MolmoPoint-8B
–
–
74.2
55.0
73.6
Table 2: State-of-the-art VLMs on the NavAble real-world test set. N=363 images, 10 evaluable categories; turnstile is held out. Box metrics are COCO-style AP (%). Pt-in-mask (%) is the fraction of predicted points that fall inside the ground-truth segmentation polygon (RoboPoint convention). PCK .05 / PCK .10 are point-localisation accuracy at 5 % and 10 % of the image diagonal. MolmoPoint-8B does not output bounding boxes; only its point-localisation metrics apply. Best per column in bold.
Cane-shaped
Quadruped
Model
mIoU
mPrec
mRec
mIoU
mPrec
mRec
SAN-ZS
10.27
10.63
45.34
0 9.95
10.78
32.51
SegFormer (R)
11.94
24.69
39.93
21.56
30.73
38.90
SegFormer (R+S)
16.17
31.44
38.22
23.08
34.24
39.25
Table 4: OOD embodiment evaluation on egocentric splits from the quadruped and cane-shaped platforms. Best per platform in bold .
Figure 4: Embodiment evaluation. (Left) The two BLV-assistive platforms used for out-of-distribution evaluation: the cane-shaped mobility assistive device and the quadruped robot, both streaming egocentric footage from onboard cameras during live navigation. (Right) Qualitative SegFormer outputs on frames from these platforms; each row shows ground truth, the Real Data Only model, and the Real + Syn Data model.
Appendix figures & tables8 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 5: Samples of our synthetic 3D assets. NavAble includes synthetic assets of 15 APS buttons, 91 door buttons, 66 pedestrian signals, 74 elevator buttons, 111 elevators, 61 bus stops, 10 bus stop signs, 48 revolving doors, 5 escalators, 29 handrails, and 10 crosswalks. Except for crosswalks and escalators, all assets are generated through our synthetic data pipeline.
Class
Real
+Cur
+Syn
background
95.76
95.73
96.14
elevator
82.37
81.78
81.89
elevator_button
91.19
89.82
92.83
door_button
76.34
74.20
77.99
crosswalk
85.11
87.39
91.95
pedestrian_signal
42.89
56.30
51.70
Appendix
Table 6: Per-class IoU (%) for Mask2Former (Swin-L). Best configuration per class in bold.
Class
Real
+Cur
+Syn
+Cur+Syn †
elevator
78.58
79.38
68.83
81.52
elevator_button
88.73
87.18
92.90
83.01
door_button
68.03
66.29
75.26
69.09
crosswalk
90.72
84.06
91.60
90.63
pedestrian_signal
40.14
49.01
57.92
48.38
aps_button
46.92
48.27
52.19
49.40
Appendix
Table 7: Per-class IoU (%) for SAN (CLIP ViT-B/16). Best configuration per class in bold.
Class
Real
+Cur
+Syn
+Cur+Syn
background
93.53
93.71
93.37
94.03
elevator
73.07
76.29
66.76
72.21
elevator_button
76.18
80.75
84.41
82.60
door_button
72.02
73.83
77.65
77.60
crosswalk
79.44
81.18
85.76
83.35
pedestrian_signal
32.61
36.72
43.79
41.76
Appendix
Table 8: Per-class IoU (%) for SegFormer (MiT-B5). Best configuration per class in bold.
Class
Real
+Cur
+Syn
+Cur+Syn
background
93.47
93.46
93.43
92.78
elevator
72.90
75.48
81.57
77.37
elevator_button
85.18
83.47
83.68
78.61
door_button
71.92
70.10
72.19
68.39
crosswalk
81.12
78.01
76.44
76.87
pedestrian_signal
32.28
31.85
38.73
40.96
Appendix
Table 9: Per-class IoU (%) for DeepLabv3+ (ResNet-101). Best configuration per class in bold.
Class
Real
+Cur
+Syn
+Cur+Syn
background
93.80
91.72
94.24
93.69
elevator
77.70
77.18
79.97
77.19
elevator_button
74.34
72.07
82.71
81.82
door_button
62.57
60.46
71.89
69.13
crosswalk
81.11
69.85
72.82
75.32
pedestrian_signal
38.70
39.62
34.58
39.11
Appendix
Table 10: Per-class IoU (%) for EncNet (ResNet-101). Best configuration per class in bold.
General-purpose robot models increasingly rely on large and diverse datasets. For embodied 3D navigation, however, existing data sources face a fundamental trade-off: simulated data can be generated at scale but often suffer from the visual sim-to-real gap, whereas real-world flight data provide realistic observations but are costly to collect. This paper studies another direction: the use of high-fidelity visual generative models as scalable data engines for embodied 3D navigation. We introduce NavGen, a text-to-video data generation pipeline that produces diverse vision-language navigation (VLN) episodes across indoor and outdoor scenes. We also propose a style-diversification method that scales up long-tail data that are difficult and costly to collect. The resulting dataset contains approximately 400K navigation episodes. We evaluate our dataset against existing UAV navigation datasets across multiple metrics, and find that the model trained on our data generally improves with scale, outperforming those trained on existing datasets. To validate real-world transferability, we deploy the trained model in world-action-model paradigm to real-world flying experiments. The final model achieves a 75% success rate across different navigation tasks and environments.
Independent sidewalk mobility is essential for blind and visually impaired pedestrians (BVIPs), yet smartphone-based assistive navigation requires perception models that distinguish walkable sidewalks from adjacent unsafe regions. This study presents a safety-oriented semantic segmentation framework for future mobile guidance. We introduce SENSATION-DS, a chest-height pedestrian-view dataset with 2,752 image-mask pairs and nine-class navigation-relevant taxonomy. External urban and sidewalk datasets were harmonized to this label space, and five segmentation architectures were evaluated using staged target-domain adaptation with mask-conditioned synthetic images and Segment Anything Model 2 (SAM2) pseudo-labels. Models were assessed using mean Intersection over Union (mIoU), road- and sidewalk-specific metrics, Road-as-Sidewalk Error Rate as a proxy false-safe measure, and Android Open Neural Network Exchange benchmarking. Synthetic augmentation generally improved segmentation accuracy, whereas SAM2 pseudo-labels more consistently reduced Road-as-Sidewalk errors. UPerNet-MobileNetV3 achieved the highest offline mIoU (0.715 +/- 0.006), while DeepLabV3Plus-MobileNetV3 achieved the lowest Road-as-Sidewalk Error Rate (0.079) and highest Android runtime at 512x384 (7.383 FPS). These results show that assistive sidewalk perception should be evaluated jointly by segmentation accuracy, proxy false-safe behavior, and smartphone deployment feasibility, while real-world benefit requires validation with BVIP users. This evaluation supports selecting models that balance accurate perception, conservative error behavior, and practical runtime.
Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany · Department of Telecommunications, National University of Science and Technology “Politehnica” Bucharest, Romania
Visual impairment affects hundreds of millions of people worldwide, severely limiting their ability to navigate urban environments safely and independently. While wearable assistive devices offer a promising platform for real-time hazard detection, existing approaches rely on task-specific vision pipelines that lack flexibility and generalizability. In this work, we propose an event map framework based on visual question answering that leverages Vision-Language Models (VLMs) for pedestrian scene description and hazard identification across diverse real-world environments, using a three-level hierarchical query structure to enable fine-grained scene understanding without task-specific retraining. Model responses are aggregated into a weighted risk scoring system that maps street segments into four discrete safety categories, producing navigable risk-aware event maps for route planning. To support evaluation and future research, we introduce a geographically diverse dataset spanning 20 cities across six continents, comprising over 800 annotated images and 18,000 answered questions. We benchmark four VQA architectures -ViLT, LLaVA, InstructBLIP, and Qwen-VL- and find that generative Multimodal Large Language Models (MLLMs) substantially outperform classification-based approaches, with Qwen-VL achieving the best overall balance of precision and recall. These results demonstrate the viability of MLLMs as a flexible and generalizable foundation for assistive navigation systems for visually impaired people.
Antoni Valls, Jordi Sanchez-Riera
Institut de Rob`otica i Inform`atica Industrial, CSIC-UPC, Llorens i Artigas 4-6, 08028 Barcelona, Spain