GuideTWSI: A Diverse Tactile Walking Surface Indicator Dataset from Synthetic and Real-World Images for Blind and Low-Vision Navigation

Hochul Hwang; Soowan Yang; Anh N. H. Nguyen; Parth Goel; Krisha Adhikari; Sunghoon I. Lee; Joydeep Biswas; Nicholas A. Giudice

cs.ROMar 7, 2026

GuideTWSI: A Diverse Tactile Walking Surface Indicator Dataset from Synthetic and Real-World Images for Blind and Low-Vision Navigation

Authors: Hochul Hwang, Soowan Yang, Anh N. H. Nguyen, Parth Goel, Krisha Adhikari, Sunghoon I. Lee, Joydeep Biswas, Nicholas A. Giudice, +1 more

Organizations: University of Massachusetts Amherst · Daegu Gyeongbuk Institute of Science and Technology · The University of Texas at Austin · University of Maine · University of Massachusetts Amherst, 140 Governors Dr, U.S.

Abstract

Tactile Walking Surface Indicators (TWSIs) are safety-critical landmarks that blind and low-vision (BLV) pedestrians use to locate crossings and hazard zones. From our observation sessions with BLV guide dog handlers, trainers, and an O&M specialist, we confirmed the critical importance of reliable and accurate TWSI segmentation for navigation assistance of BLV individuals. Achieving such reliability requires large-scale annotated data. However, TWSIs are severely underrepresented in existing urban perception datasets, and even existing dedicated paving datasets are limited: they lack robot-relevant viewpoints (e.g., egocentric or top-down) and are geographically biased toward East Asian directional bars - raised parallel strips used for continuous guidance along sidewalks. This narrow focus overlooks truncated domes - rows of round bumps used primarily in North America and Europe as detectable warnings at curbs, crossings, and platform edges. As a result, models trained only on bar-centric data struggle to generalize to dome-based warnings, leading to missed detections and false stops in safety-critical environments.

Figures & tables

Fig. 1: Objective of the proposed research and tactile walking surface indicator (TWSI). (1) A guide dog stopping at a TWSI and curb, (2) a robot mimicking this stop behavior, and (3) different TWSI types.

Fig. 2: Photorealistic synthetic tactile paving dataset generation. Our pipeline uses various sidewalk environments in Unreal Engine 4 with varying viewpoints, lighting, and weather conditions to simulate real-world variability. We use AirSim for automatic annotation of depth, instance masks, and bounding boxes. This enables the creation of a high-quality dataset of 15K samples that significantly enhances TWSI segmentation when fused with real-world data.

ID	Age	Gender	Vision Level	Experience ∗
GH01	63	F	Totally blind	36
GH02	66	F	Legally blind	9
GT01	-	F		-
GT02	54	M		21
GT03	35	F		4
GT04	28	F		7

TABLE I: Observation Session Participant Demographics

Dataset	Scale	Type	Geography	Modalities
SideGuide [ 17 ]	$\sim$ 8.2K	Real/bars	Korea	RGB, BBX, Seg.
Tenji10K [ 24 ]	10K	Real/bars	Japan	RGB, Seg.
TP [ 28 ]	$\sim$ 1.4K	Real/bars	China	RGB, Seg.
RBar-22K (compiled)	$\sim$ 22K	Real/bars	Mostly Asia	RGB, Seg.
RDome-2K (ours)	2.4K+	Real/domes	United States	RGB, Seg.
SDome-15K (ours)	15K+	Synthetic/domes	Simulated	RGB+D, BBx, Seg.

TABLE II: GuideTWSI dataset comparison.

Fig. 3: Real robot data collection and hardware experiment. (a) We collected data with a quadruped robot across multiple sites in suburban, campus, and rural environments featuring various truncated domes. Data were gathered at different times of day under diverse truncated dome appearances. (b) We evaluated the robot’s reliable stopping at truncated domes in unseen areas. We present the hardware experiment setup and segmentation inference results.

	Real Data Only				Real + Synthetic Data
Method	Prec.	Rec.	mAP50-95	mIoU	Prec.	Rec.	mAP50-95	mIoU	$\Delta$ mIoU
YOLOv11-seg-N	0.7958	0.6924	0.5934	0.6161	0.8718	0.8084	0.7288	0.7308	+0.1147
YOLOv11-seg-X	0.8838	0.8204	0.7362	0.7389	0.9102	0.8588	0.8188	0.7887	+0.0498
Mask2Former	0.9458	0.5975	0.4798	0.5777	0.9611	0.8669	0.7829	0.8375	+0.2598
SAM2.1+UNet	0.8680	0.5165	0.3475	0.4789	0.9704	0.7031	0.5627	0.6883	+0.2094
DINOv3+RegCls	0.9027	0.7804	0.6176	0.7322	0.8667	0.8924	0.6933	0.7926	+0.0604

TABLE III: Impact of synthetic data augmentation on truncated dome segmentation (RBar-train (+SDome-15K) → RDome-2K)

Panel of Figure 7 — Fig. 4: Qualitative comparison of segmentation models trained on real data only vs. real + synthetic data. Each row shows a test sample alongside predictions from YOLOv11-seg-N, YOLOv11-seg-X, and DINOv3+EOMT. Highlighted regions in the images show that models trained with synthetic data produce sharper boundaries and fewer missed detections, especially under challenging textures and lighting.

Fig. 5: Hardware configuration and segmentation visualization. (Top) Hardware configuration with a downward-facing RGB camera mounted at a $70^{\circ}$ angle. (Bottom) Segmentation masks from the fine-tuned model trigger a stop command as the robot walks forward at $0.8\text{\,}\mathrm{m}\text{/}\mathrm{s}$ .

Environment	N	Mean Distance (cm)	Success Rate
Urban	6	$35.2\pm 9.2$	29/30
Urban2	5	$47.1\pm 15.6$	25/25
Suburban	6	$38.2\pm 14.4$	32/34
Residential	3	$34.6\pm 14.2$	14/15
Overall	20	$\mathbf{39.0\pm 14.3}$	100/104 (96.15%)

TABLE IV: Robot stopping performance across different environments.

Explore similar work

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation
Jul 23, 2026Hakan Calim, Anamaria Dumitrescu, Adarsh Bhandary Panambur +2Visual Navigation Semantic Segmentation
Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany · Department of Telecommunications, National University of Science and Technology “Politehnica” Bucharest, Romania
NavAble: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation
Oct 14, 2026Hochul Hwang, Jahir Sadik Monon, Soowan Yang +9Visual Navigation Synthetic Data Generation for Robotics
University of Massachusetts Amherst · Daegu Gyeongbuk Institute of Science and Technology · Boston University
WalkGPT: Grounded Vision-Language Conversation with Depth-Aware Segmentation for Pedestrian Navigation
Mar 11, 2026Rafi Ibn Sultan, Hui Zhu, Xiangyu Zhou +4Vision-Language Grounding Large Vision-Language Models
Department of Computer Science, Wayne State University · Department of Radiation Oncology, Henry Ford Health · Department of Electrical and Computer Engineering, The Ohio State University +1

Jul 23, 2026J/K move · Enter open · S save

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation

Hakan Calim, Anamaria Dumitrescu, Adarsh Bhandary Panambur, Huzaifa Asif+1

Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany · Department of Telecommunications, National University of Science and Technology “Politehnica” Bucharest, Romania

Visual Navigation Semantic Segmentation

Open paper AskPDF arXiv

Independent sidewalk mobility is essential for blind and visually impaired pedestrians (BVIPs), yet smartphone-based assistive navigation requires perception models that distinguish walkable sidewalks from adjacent unsafe regions. This study presents a safety-oriented semantic segmentation framework for future mobile guidance. We introduce SENSATION-DS, a chest-height pedestrian-view dataset with 2,752 image-mask pairs and nine-class navigation-relevant taxonomy. External urban and sidewalk datasets were harmonized to this label space, and five segmentation architectures were evaluated using staged target-domain adaptation with mask-conditioned synthetic images and Segment Anything Model 2 (SAM2) pseudo-labels. Models were assessed using mean Intersection over Union (mIoU), road- and sidewalk-specific metrics, Road-as-Sidewalk Error Rate as a proxy false-safe measure, and Android Open Neural Network Exchange benchmarking. Synthetic augmentation generally improved segmentation accuracy, whereas SAM2 pseudo-labels more consistently reduced Road-as-Sidewalk errors. UPerNet-MobileNetV3 achieved the highest offline mIoU (0.715 +/- 0.006), while DeepLabV3Plus-MobileNetV3 achieved the lowest Road-as-Sidewalk Error Rate (0.079) and highest Android runtime at 512x384 (7.383 FPS). These results show that assistive sidewalk perception should be evaluated jointly by segmentation accuracy, proxy false-safe behavior, and smartphone deployment feasibility, while real-world benefit requires validation with BVIP users. This evaluation supports selecting models that balance accurate perception, conservative error behavior, and practical runtime.

Abstract

Explore similar work

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation

NavAble: A Large-Scale Dataset and Synthetic Data Generation Pipeline for Blind Navigation

WalkGPT: Grounded Vision-Language Conversation with Depth-Aware Segmentation for Pedestrian Navigation