Authors: Lkhanaajav Mijiddorj, Yang Yan, Tyler Beringer, Bilguunzaya Mijiddorj, Alex N. Ho, Bin Xu, Binbin Weng
Organizations: School of Electrical and Computer Engineering, University of Oklahoma, Norman, OK 73019, USA · School of Aerospace and Mechanical Engineering, University of Oklahoma, Norman, OK 73019, USA
Sidewalk-scale path extraction demands perception and planning that run reliably on compact, low-power hardware in cluttered, map-sparse environments. We present a monocular vision pipeline for sidewalk path extraction in micromobility systems that progresses through three design iterations, from a skeleton-graph baseline through distance-transform corridor planning to a lightweight image-space architecture, and provides a systematic comparison of five path-planning methods across both bird's-eye-view (BEV) and image-space domains. A compact SegFormer-B0 student model, trained with a semi-supervised teacher-student framework using OneFormer Swin-L pseudo-labels, achieves a hand-annotated IoU of 0.946 at 11.7 ms per frame, improving over the baseline checkpoint (IoU 0.758, 18.9 ms). In a controlled planner comparison on 32 hand-labeled frames, image-space midpoint planning achieves the lowest lateral center error (14.3 px) at 2.2 ms, a 421x speedup over BEV distance-transform planning (926.8 ms, 65.0 px center error), while maintaining comparable mask-path alignment (98.5% versus 98.6%). A full-video replay across six campus sequences (22,679 frames) confirms that the improved segmentation reduces temporal instability from 1.46% to 0.33% and increases template-path availability from 73.7% to 79.3%. We further show that BEV-only path extraction is fragile in monocular settings: in one profiled run, 99.3% of frames produced no valid BEV path. The final recommended architecture, image-space midpoint primary, image-space distance-transform fallback, and BEV reserved for visualization, runs the full perception-to-path stack in under 50 ms per frame on CPU, making it suitable for embedded pedestrian-speed micromobility systems.
Figures & tables
Figure 1: Hardware context for this study. (a) Electric scooter platform used for data collection, with a forward-facing monocular camera rigidly mounted to the handlebar stem. (b) Sample sidewalk scene as seen by the camera; all experiments are conducted offline on video recorded from this rig.
Figure 2: Overview of the sidewalk navigation pipeline. Hand-annotated and pseudo-labeled data are merged to train a student segmentation model, followed by geometric processing to extract traversable paths. The system supports both BEV-domain and image-space planning; our experiments compare five methods across both domains.
Setting
Value
Student model
SegFormer-B0
Teacher model
OneFormer Swin-L pretrained on ADE20K
Input resolution
640×360
Pseudo-label confidence threshold
0.60
Optimizer
AdamW
Initial learning rate
5×10−5
Table 1: Known training settings of the SegFormer-B0 student model.
Iteration
Teacher
Train Pairs
Val IoU
Baseline
SegFormer-B2
300 + 2,300
0.758 †
OneFormer (4 vid)
OneFormer Swin-L
400
0.944
OneFormer (mixed)
OneFormer Swin-L
1,419
0.960
Table 2: Segmentation model training progression. All students are SegFormer-B0; validation IoU is on held-out splits.
Figure 3: Resolution sweep for SegFormer-B0. Each column shows input overlay (left) and resulting segmentation mask (right). Higher resolutions improve edge detail but increase latency.
Resolution
Latency (ms)
FPS
160×90
19.84
50.40
320×180
25.35
39.45
480×270
42.05
23.78
640×360
46.00
21.74
768×432
93.55
10.69
native 540×540
72.80
13.74
Table 3: SegFormer-B0 inference latency and throughput at various input resolutions (CPU-only).
Figure 4: Segmentation pipeline. (a) RGB input, (b) raw sidewalk mask in image plane, (c) BEV projection of cleaned mask. Morphological filtering and connected-component analysis suppress spurs and disconnected blobs.
Model
IoU
Prec.
Recall
F1
ms
Baseline
0.758
0.910
0.835
0.851
18.9
Candidate
0.946
0.983
0.962
0.972
11.7
Cand.+confhold
0.903
0.987
0.914
0.947
13.1
Table 4: Segmentation quality on 32 hand-annotated frames. “Baseline” uses the original SegFormer-B2 teacher; “Candidate” uses the OneFormer Swin-L teacher with optimized threshold.
Figure 5: Segmentation quality comparison on 32 hand-annotated frames. The OneFormer-trained candidate improves all metrics while reducing inference time.
Metric
Baseline
Candidate
Δ
Mean Seg IoU
0.909
0.925
+0.016
Unstable Rate [%]
1.46
0.33
− 1.12 pp
Has-Path Rate [%]
100.0
100.0
—
Template Success [%]
73.7
79.3
+5.6 pp
Fallback Rate [%]
19.0
14.3
− 4.7 pp
Mean Heading Δ [deg]
0.209
0.201
− 0.008
Table 5: Full-video replay metrics across six campus sequences ( 22 679 frames).
Planner
Path [%]
Inside-GT
Center Err. [px]
ms
BEV-domain methods
Skeleton-Graph
100
0.971
76.6
380.3
DT Ridge (full)
100
0.986
65.0
926.8
DT Ridge (near)
100
0.986
79.0
1603.0
Image-space methods
Midpoint
100
0.985
14.3
2.2
Table 6: Planner comparison on 32 hand-annotated frames (candidate mask). Runtime is mean per-frame wall time.
Figure 6: Planner comparison on 32 hand-annotated frames. Left: lateral center error (px) with runtime annotated. Right: inside-GT ratio. Image-space methods (green) dominate BEV methods (blue) in both accuracy and speed.
Figure 7: BEV skeleton-graph pipeline (baseline method). (a) Skeletonized BEV mask after Guo–Hall thinning, (b) candidate paths from Dijkstra search, (c) smoothed path reprojected to camera view. While topologically informative, this method is the slowest ( 380.3ms ) and least accurate of the five compared planners.
Metric
Value
Frames with valid path ( dt_ridge )
10 (0.2%)
Frames with held path ( dt_ridge_hold )
20 (0.5%)
Frames with no path ( none )
4,377 (99.3%)
Mean BEV mask occupancy ratio
0.0002
Table 7: BEV path extraction reliability over 4,407 profiled frames.
Figure 8: BEV path extraction outcome over 4,407 frames. 99.3% of frames produced no valid BEV path, demonstrating the fragility of monocular BEV planning.
Module
BEV Skeleton
Image-Space
SegFormer Inference
11.7 ms
11.7 ms
Mask Refinement
8.5 ms
3.0 ms †
BEV Projection
0.9 ms
—
BEV Cleanup
14.6 ms
—
Planner
380.3 ms
2.2 ms
Total
416.0 ms
16.9 ms
Table 8: Per-module runtime comparison at 640×360 (CPU-only).
Component
Cost (ms)
Recommendation
BEV DT Planner
926.8
Replace
BEV Graph Planner
380.3
Replace
YOLO Detection (CPU)
39.0
Gate or GPU
BEV Warp + Cleanup
15.5
Skip in img mode
Seg ( 512×288 )
75.3
Use 640×360
Predictor disabled
+73.0
Always enable
Table 9: Runtime offenders ranked by cost. The image-space architecture eliminates the top three.
Configuration
FPS
Seg (ms)
Det (ms)
BEV (ms)
No detection, predictor on
25.3
20.3
—
11.0
GPU detection, predictor on
20.1
17.0
12.2
13.4
CPU detection, predictor on
12.2
18.8
39.0
14.6
No detection, predictor off
8.9
75.3
—
29.5
Table 10: System-level runtime under different configurations (4,407-frame profiled run, 512×288 input).
Figure 9: Per-module runtime comparison. The image-space architecture eliminates BEV overhead and reduces planner cost by 173 × , achieving 59 FPS vs. 2.4 FPS.
Independent sidewalk mobility is essential for blind and visually impaired pedestrians (BVIPs), yet smartphone-based assistive navigation requires perception models that distinguish walkable sidewalks from adjacent unsafe regions. This study presents a safety-oriented semantic segmentation framework for future mobile guidance. We introduce SENSATION-DS, a chest-height pedestrian-view dataset with 2,752 image-mask pairs and nine-class navigation-relevant taxonomy. External urban and sidewalk datasets were harmonized to this label space, and five segmentation architectures were evaluated using staged target-domain adaptation with mask-conditioned synthetic images and Segment Anything Model 2 (SAM2) pseudo-labels. Models were assessed using mean Intersection over Union (mIoU), road- and sidewalk-specific metrics, Road-as-Sidewalk Error Rate as a proxy false-safe measure, and Android Open Neural Network Exchange benchmarking. Synthetic augmentation generally improved segmentation accuracy, whereas SAM2 pseudo-labels more consistently reduced Road-as-Sidewalk errors. UPerNet-MobileNetV3 achieved the highest offline mIoU (0.715 +/- 0.006), while DeepLabV3Plus-MobileNetV3 achieved the lowest Road-as-Sidewalk Error Rate (0.079) and highest Android runtime at 512x384 (7.383 FPS). These results show that assistive sidewalk perception should be evaluated jointly by segmentation accuracy, proxy false-safe behavior, and smartphone deployment feasibility, while real-world benefit requires validation with BVIP users. This evaluation supports selecting models that balance accurate perception, conservative error behavior, and practical runtime.
Pattern Recognition Lab, Friedrich-Alexander-University Erlangen-Nuremberg, Erlangen, Germany · Department of Telecommunications, National University of Science and Technology “Politehnica” Bucharest, Romania
Automatically generating pedestrian pathways from aerial images requires producing a connected network suitable for routing, not just detecting where sidewalks appear. Sidewalks and crossings, in contrast to roads, may be partially occluded, implicitly defined, and exhibit complex connectivity patterns. Existing segmentation-based approaches focus on labeling pixels to infer segments, but often produce disconnected or fragmentary graphs that are unreliable for navigation. We introduce TraversRL, a vision-conditioned model that iteratively grows a pathway network from an aerial image, simulating a traveler navigating the built environment. TraversRL uses an action space of short and long direction-distance segments designed to adapt to complex patterns and span occlusions, and uses a combination of graph-level and step-wise rewards to balance overall connectivity with precise edge placement. Across three visual backbones and three intersection datasets, TraversRL substantially improves buffered IoU with the ground-truth graph relative to a state-of-the-art segmentation baseline, and more than doubles metrics of connectivity. Moreover, combining global and local rewards produces cleaner graphs with fewer spurious branches while further improving overall performance. These results demonstrate that modeling pathway extraction as a sequential decision process from the perspective of a traveler, while optimizing for final graph quality with reinforcement learning, produces significantly more reliable pedestrian networks.
Sidewalk width is an important indicator of pedestrian accessibility, comfort, and network quality, yet large-scale width data remain scarce in most cities. Existing approaches typically rely on costly field surveys, high-resolution overhead imagery, or simplified geometric assumptions that limit scalability or introduce systematic error. To address this gap, we present UrbanVGGT, a measurement pipeline for estimating metric sidewalk width from a single street-view image. The method combines semantic segmentation, feed-forward 3D reconstruction, adaptive ground-plane fitting, camera-height-based scale calibration, and directional width measurement on the recovered plane. On a ground-truth benchmark from Washington, D.C., UrbanVGGT achieves a mean absolute error of 0.252 m, with 95.5% of estimates within 0.50 m of the reference width. Ablation experiments show that metric scale calibration is the most critical component, and controlled comparisons with alternative geometry backbones support the effectiveness of the overall design. As a feasibility demonstration, we further apply the pipeline to three cities and generate SV-SideWidth, a prototype sidewalk-width dataset covering 527 OpenStreetMap street segments. The results indicate that street-view imagery can support scalable generation of candidate sidewalk-width attributes, while broader cross-city validation and local ground-truth auditing remain necessary before deployment as authoritative planning data.
Kaizhen Tan, Fan Zhang
Heinz College of Information Systems and Public Policy, Carnegie Mellon University, USA · Institute of Remote Sensing and Geographical Information System, Peking University, China