Lightweight Machine Learning-Driven Monocular Sidewalk Path Extraction for Embedded Micromobility Navigation

Lkhanaajav Mijiddorj; Yang Yan; Tyler Beringer; Bilguunzaya Mijiddorj; Alex N. Ho; Bin Xu; Binbin Weng

cs.CVAug 25, 2026

Lightweight Machine Learning-Driven Monocular Sidewalk Path Extraction for Embedded Micromobility Navigation

Authors: Lkhanaajav Mijiddorj, Yang Yan, Tyler Beringer, Bilguunzaya Mijiddorj, Alex N. Ho, Bin Xu, Binbin Weng

Organizations: School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA · School of Aerospace and Mechanical Engineering, University of Oklahoma, Norman, OK 73019, USA

Abstract

Sidewalk-scale path extraction demands perception and planning that run reliably on compact, low-power hardware in cluttered, map-sparse environments. We present a monocular vision pipeline for sidewalk path extraction in micromobility systems that progresses through three design iterations, from a skeleton-graph baseline through distance-transform corridor planning to a lightweight image-space architecture, and provides a systematic comparison of five path-planning methods across both bird's-eye-view (BEV) and image-space domains. A compact SegFormer-B0 student model, trained with a semi-supervised teacher-student framework using OneFormer Swin-L pseudo-labels, achieves a hand-annotated IoU of 0.946 at 11.7 ms per frame, improving over the baseline checkpoint (IoU 0.758, 18.9 ms). In a controlled planner comparison on 32 hand-labeled frames, image-space midpoint planning achieves the lowest lateral center error (14.3 px) at 2.2 ms, a 421x speedup over BEV distance-transform planning (926.8 ms, 65.0 px center error), while maintaining comparable mask-path alignment (98.5% versus 98.6%). A full-video replay across six campus sequences (22,679 frames) confirms that the improved segmentation reduces temporal instability from 1.46% to 0.33% and increases template-path availability from 73.7% to 79.3%. We further show that BEV-only path extraction is fragile in monocular settings: in one profiled run, 99.3% of frames produced no valid BEV path. The final recommended architecture, image-space midpoint primary, image-space distance-transform fallback, and BEV reserved for visualization, runs the full perception-to-path stack in under 50 ms per frame on CPU, making it suitable for embedded pedestrian-speed micromobility systems.

Figures & tables

Panel of Figure 1 — Figure 1: Hardware context for this study. (a) Electric scooter platform used for data collection, with a forward-facing monocular camera rigidly mounted to the handlebar stem. (b) Sample sidewalk scene as seen by the camera; all experiments are conducted offline on video recorded from this rig.

Figure 2: Overview of the sidewalk navigation pipeline. Hand-annotated and pseudo-labeled data are merged to train a student segmentation model, followed by geometric processing to extract traversable paths. The system supports both BEV-domain and image-space planning; our experiments compare five methods across both domains.

Setting	Value
Student model	SegFormer-B0
Teacher model	OneFormer Swin-L pretrained on ADE20K
Input resolution	$640\times 360$
Pseudo-label confidence threshold	0.60
Optimizer	AdamW
Initial learning rate	$5\times 10^{-5}$

Table 1: Known training settings of the SegFormer-B0 student model.

Iteration	Teacher	Train Pairs	Val IoU
Baseline	SegFormer-B2	300 + 2,300	0.758 $\dagger$
OneFormer (4 vid)	OneFormer Swin-L	400	0.944
OneFormer (mixed)	OneFormer Swin-L	1,419	0.960

Table 2: Segmentation model training progression. All students are SegFormer-B0; validation IoU is on held-out splits.

Figure 3: Resolution sweep for SegFormer-B0. Each column shows input overlay (left) and resulting segmentation mask (right). Higher resolutions improve edge detail but increase latency.

Resolution	Latency (ms)	FPS
$160{\times}90$	19.84	50.40
$320{\times}180$	25.35	39.45
$480{\times}270$	42.05	23.78
$640{\times}360$	46.00	21.74
$768{\times}432$	93.55	10.69
native $540{\times}540$	72.80	13.74

Table 3: SegFormer-B0 inference latency and throughput at various input resolutions (CPU-only).

Panel of Figure 4 — Figure 4: Segmentation pipeline. (a) RGB input, (b) raw sidewalk mask in image plane, (c) BEV projection of cleaned mask. Morphological filtering and connected-component analysis suppress spurs and disconnected blobs.

Model	IoU	Prec.	Recall	F1	ms
Baseline	0.758	0.910	0.835	0.851	18.9
Candidate	0.946	0.983	0.962	0.972	11.7
Cand.+confhold	0.903	0.987	0.914	0.947	13.1

Table 4: Segmentation quality on 32 hand-annotated frames. “Baseline” uses the original SegFormer-B2 teacher; “Candidate” uses the OneFormer Swin-L teacher with optimized threshold.

Figure 5: Segmentation quality comparison on 32 hand-annotated frames. The OneFormer-trained candidate improves all metrics while reducing inference time.

Metric	Baseline	Candidate	$\Delta$
Mean Seg IoU	0.909	0.925	+0.016
Unstable Rate [%]	1.46	0.33	$-$ 1.12 pp
Has-Path Rate [%]	100.0	100.0	—
Template Success [%]	73.7	79.3	+5.6 pp
Fallback Rate [%]	19.0	14.3	$-$ 4.7 pp
Mean Heading $\Delta$ [deg]	0.209	0.201	$-$ 0.008

Table 5: Full-video replay metrics across six campus sequences ( 22 679 frames).

BEV-domain methods
Planner	Path [%]	Inside-GT	Center Err. [px]	ms
Skeleton-Graph	100	0.971	76.6	380.3
DT Ridge (full)	100	0.986	65.0	926.8
DT Ridge (near)	100	0.986	79.0	1603.0
Image-space methods
Midpoint	100	0.985	14.3	2.2

Table 6: Planner comparison on 32 hand-annotated frames (candidate mask). Runtime is mean per-frame wall time.

Figure 6: Planner comparison on 32 hand-annotated frames. Left: lateral center error (px) with runtime annotated. Right: inside-GT ratio. Image-space methods (green) dominate BEV methods (blue) in both accuracy and speed.

Panel of Figure 7 — Figure 7: BEV skeleton-graph pipeline (baseline method). (a) Skeletonized BEV mask after Guo–Hall thinning, (b) candidate paths from Dijkstra search, (c) smoothed path reprojected to camera view. While topologically informative, this method is the slowest ( $380.3\text{\,}\mathrm{m}\mathrm{s}$ ) and least accurate of the five compared planners.

Metric	Value
Frames with valid path ( dt_ridge )	10 (0.2%)
Frames with held path ( dt_ridge_hold )	20 (0.5%)
Frames with no path ( none )	4,377 (99.3%)
Mean BEV mask occupancy ratio	0.0002

Table 7: BEV path extraction reliability over 4,407 profiled frames.

Figure 8: BEV path extraction outcome over 4,407 frames. 99.3% of frames produced no valid BEV path, demonstrating the fragility of monocular BEV planning.

Module	BEV Skeleton	Image-Space
SegFormer Inference	11.7 ms	11.7 ms
Mask Refinement	8.5 ms	3.0 ms $\dagger$
BEV Projection	0.9 ms	—
BEV Cleanup	14.6 ms	—
Planner	380.3 ms	2.2 ms
Total	416.0 ms	16.9 ms

Table 8: Per-module runtime comparison at

640{\times}360

(CPU-only).

Component	Cost (ms)	Recommendation
BEV DT Planner	926.8	Replace
BEV Graph Planner	380.3	Replace
YOLO Detection (CPU)	39.0	Gate or GPU
BEV Warp + Cleanup	15.5	Skip in img mode
Seg ( $512{\times}288$ )	75.3	Use $640{\times}360$
Predictor disabled	+73.0	Always enable

Table 9: Runtime offenders ranked by cost. The image-space architecture eliminates the top three.

Configuration	FPS	Seg (ms)	Det (ms)	BEV (ms)
No detection, predictor on	25.3	20.3	—	11.0
GPU detection, predictor on	20.1	17.0	12.2	13.4
CPU detection, predictor on	12.2	18.8	39.0	14.6
No detection, predictor off	8.9	75.3	—	29.5

Table 10: System-level runtime under different configurations (4,407-frame profiled run,

512{\times}288

input).

Figure 9: Per-module runtime comparison. The image-space architecture eliminates BEV overhead and reduces planner cost by 173 $\times$ , achieving 59 FPS vs. 2.4 FPS.

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 557 papers

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation
Jul 23, 2026Hakan Calim, Anamaria Dumitrescu, Adarsh Bhandary Panambur +2Visual Navigation Visually Impaired Navigation Assistance
Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany · Department of Telecommunications, National University of Science and Technology “Politehnica” Bucharest, Romania
TraversRL: Traversable Pedestrian Pathway Generation With Reinforcement Learning
Jul 20, 2026Bin Han, Robert Wolfe, Bill HoweRemote Sensing Image Understanding Reinforcement Learning
University of Washington · Rutgers University
UrbanVGGT: Scalable Sidewalk Width Estimation from Street View Images
Mar 23, 2026Kaizhen Tan, Fan ZhangUrban Planning
Heinz College of Information Systems and Public Policy, Carnegie Mellon University, USA · Institute of Remote Sensing and Geographical Information System, Peking University, China

Jul 23, 2026J/K move · Enter open · S save

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation

Hakan Calim, Anamaria Dumitrescu, Adarsh Bhandary Panambur, Huzaifa Asif+1

Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany · Department of Telecommunications, National University of Science and Technology “Politehnica” Bucharest, Romania

Visual Navigation Visually Impaired Navigation Assistance Semantic Segmentation

Open paper AskPDF arXiv

Independent sidewalk mobility is essential for blind and visually impaired pedestrians (BVIPs), yet smartphone-based assistive navigation requires perception models that distinguish walkable sidewalks from adjacent unsafe regions. This study presents a safety-oriented semantic segmentation framework for future mobile guidance. We introduce SENSATION-DS, a chest-height pedestrian-view dataset with 2,752 image-mask pairs and nine-class navigation-relevant taxonomy. External urban and sidewalk datasets were harmonized to this label space, and five segmentation architectures were evaluated using staged target-domain adaptation with mask-conditioned synthetic images and Segment Anything Model 2 (SAM2) pseudo-labels. Models were assessed using mean Intersection over Union (mIoU), road- and sidewalk-specific metrics, Road-as-Sidewalk Error Rate as a proxy false-safe measure, and Android Open Neural Network Exchange benchmarking. Synthetic augmentation generally improved segmentation accuracy, whereas SAM2 pseudo-labels more consistently reduced Road-as-Sidewalk errors. UPerNet-MobileNetV3 achieved the highest offline mIoU (0.715 +/- 0.006), while DeepLabV3Plus-MobileNetV3 achieved the lowest Road-as-Sidewalk Error Rate (0.079) and highest Android runtime at 512x384 (7.383 FPS). These results show that assistive sidewalk perception should be evaluated jointly by segmentation accuracy, proxy false-safe behavior, and smartphone deployment feasibility, while real-world benefit requires validation with BVIP users. This evaluation supports selecting models that balance accurate perception, conservative error behavior, and practical runtime.

Abstract

Robot Navigation

Explore similar work

Safety-oriented sidewalk and road segmentation for smartphone-based assistive navigation

TraversRL: Traversable Pedestrian Pathway Generation With Reinforcement Learning

UrbanVGGT: Scalable Sidewalk Width Estimation from Street View Images