From Cloud to Edge: A Fully Local Pipeline for Pothole Risk Assessment and Driver Warning Generation
Organizations: Department of Electrical and Computer Engineering, University of Michigan–Dearborn, Dearborn, MI 48128, USA
Abstract
Vision-language models can turn road images into clear, geometry-aware pothole warnings, but current systems depend on commercial cloud services. Our earlier five-stage pipeline needed two cloud calls per scene, taking about 20 s and USD 0.105, and could not run offline. We rebuild every stage to run on a single local GPU and evaluate it with the same data, prompts, and protocol. Segmentation keeps the original Segment Anything weights but runs them in half precision with graph compilation, which makes it 2.7x faster without changing any downstream risk class; five lightweight variants could not match its masks on small road defects. Severity is graded by six ConvNeXt students distilled from 3092 cloud labels. They reach Cohen's kappa = 0.268 against human consensus at 28 ms per pothole, close to the cloud model's 0.321, within the camera setup they were trained for. A local 8-billion-parameter vision-language model writes the warnings under a grammar that admits only in-lane maneuvers. Three blind, independent raters scored these warnings safer than the cloud warnings (4.18 vs. 3.65 of 5; p = 0.001; 50 scenes) and equally clear, whereas an automated LLM judge ranked them the other way. The complete pipeline runs at 1.27 s per scene, 16x faster than the cloud system, offline and at no per-scene cost.
Figures & tables
| Family | Input | Locates potholes | Grades severity | Writes warning | Local | Human-rated output |
| Vibration sensing Eriksson et al. (2008) ; Mednis et al. (2011) | accelerometer | – | – | – | ||
| Classical image processing Koch and Brilakis (2011) ; Nienaber et al. (2015) ; Ouma and Hahn (2017) | RGB image | – | – | – | ||
| Deep detectors Maeda et al. (2018) ; Arya et al. (2021) ; Ahmed (2021) ; Abu-raddaha et al. (2024) | RGB image | – | – | – | ||
| 3D measurement Jog et al. (2012) ; Moazzam et al. (2013) ; Fan et al. (2020) ; Fan et al. (2022) ; Yurdakul and Tasdemir (2025) | stereo, RGB-D | – | – | |||
| VLM road-safety rating Jongwiriyanurak et al. (2025) | RGB image | – | – | – | – | |
| Driving VLMs Sima et al. (2024) ; Xu et al. (2024) ; Tian et al. (2024) ; Marcu et al. (2024) | camera video | – | – | partly | partly |
| Maneuver type | Meaning |
| maintain_lane_and_speed | no action needed |
| reduce_speed | slow down and stay centred in the lane |
| shift_within_lane_left | small offset to the left, staying inside the lane |
| shift_within_lane_right | small offset to the right, staying inside the lane |
| shift_within_lane_and_reduce_speed | small in-lane offset combined with slowing down |
| brake_firmly | severe hazard with no room to move sideways |
| Model | s/scene | Mean IoU | 0.9 | Area error | Risk change |
| ViT-H (published, fp32) | 0.369 | 1.000 | 100% | 0% | 0% |
| MobileSAM Zhang et al. (2023) | 0.082 | 0.791 | 16% | 19.7% | 5.9% |
| TinySAM Shu et al. (2025) | 0.078 | 0.807 | 20% | 17.0% | 5.9% |
| EfficientViT-SAM-L2 Zhang et al. (2024) | 0.077 | 0.752 | 8% | 26.6% | 6.7% |
| EfficientViT-SAM-XL1 Zhang et al. (2024) | 0.090 | 0.797 | 16% | 21.3% | 7.6% |
| SAM 2.1-small Ravi et al. (2025) | 0.203 | 0.706 | 5% | 45.9% | 12.6% |
| Grader | Cohen’s | Agreement | Time per pothole |
| Mean pairwise human | 0.453 | – | – |
| Cloud VLM (Gemini 2.5 Pro, released labels) | 0.321 | 55.3% | 4.8 s (API) |
| Qwen3-VL-8B, published prompt | 0.007 | 39.5% | 12 s |
| + input resizing | 0.093 | 44.7% | 12 s |
| + resizing, reasoning first | 0.152 | 44.7% | 14 s |
| Qwen3-VL-30B-A3B (FP8) | 0.103 | 39.5% | 13 s |
| Cloud (prior work) | Local, unconstrained | Local, constrained | |
| Rubric consistency | – | 95.5% [94.3, 96.7] | 94.8% [93.5, 96.1] |
| Lane-departure advice | 0.4% | 48.4% | 6.1% |
| Judge clarity (1–5) | 4.60 | 3.29 | 3.78 |
| Judge safety (1–5) | 4.20 | 2.43 | 2.97 |
| Judge preference (cloud : local) | – | 110 : 14 | 103 : 17 |
| Cloud | Local | Difference | 95% CI | ||
| Safety appropriateness | 3.65 0.86 | 4.18 0.81 | [0.23, 0.81] | 0.001 | |
| Clarity | 4.02 0.76 | 4.18 0.75 | [ , 0.37] | 0.14 | |
| Preferred (of 150) | 35 | 68 | 47 ties | – | 0.0015 |
| Cloud pipeline Aburaddaha et al. (2026) | This work (local) | |
| Stage 2: segmentation | 0.369 s | 0.137 s |
| Stage 3: features | 0.05 s | 0.009 s |
| Stage 4: severity and risk | 13.3 s (API) | 0.030 s |
| Stage 5: warning | 3.5 s (API) | 1.080 s |
| Total per scene | 20.1 s | 1.27 s |
| Marginal cost per scene | $0.105 | $0 |
VLMs for Autonomous Driving
Vision-language models for autonomous driving connect camera and sensor data with language to interpret scenes, predict hazards, and plan actions.
See all 220 papers