Are We Ready for Snow-tonomous Driving? Challenges Toward Robust Automated Driving in Extreme Weather
Authors: Eren Erdal Aksoy, Abu Mohammed Raisuddin, Thien-Thu Ngo, Eren Aydemir, Osman Tahir Ekşi, Paula Litkey, Petri Manninen, Heikki Hyyti, Jyri Maanpää, Tarmo Kivioja, Tuomas Herranen, Väisänen Sofie, Niemi Kalle, Isabelle Morvan, Pierre Duthon, Ian Marsh, Martin Sanfridson, Henrik Toss, Thanh Bui, Yuri Poledna, Maikol Funk Drechsler, Mikko Tarkiainen, Matti Kutila, Topi Miekkala, Stefan Parzefall, Valentina Donzella, Pak Hung Chan, Sogol Kharrazi, Mario Ceccarelli
Organizations: Halmstad University, School of Information Technology, Sweden · Lund University, Department of Computer Science, Sweden. · FORD Otosan, Turkey. · Finnish Geospatial Research Institute, Finland. · Lapland University of Applied Sciences, Finland. · CRF, France. · CEREMA, France. · RISE, Sweden. · CARISSMA Institute of Automated Driving, TU Ingolstadt, Germany. · VTT, Finland. · AVL, Germany. · Queen Mary University of London, United Kingdom. · VTI, Swedish National Road and Transport Research Institute, Linkoping, Sweden. · accelopment Schweiz AG, Switzerland.
Automated driving has gained remarkable momentum over the past decade, with both industry and academia investing substantial resources in developing perception and control solutions to advance vehicle autonomy. However, these efforts have largely focused on clear-weather scenarios, leaving harsh conditions, such as snow, rain, and fog, comparatively understudied. As a result, inclement weather remains one of the most significant hurdles for autonomous vehicles, often leading to substantial performance degradation in downstream perception and control tasks. In this paper, we present insights from the EU-funded Horizon Europe project ROADVIEW, which aims to develop robust, weather-aware perception and control modules for connected and automated vehicles operating under severe conditions, including snow, rain, and fog. Drawing on four years of large-scale development and real-world demonstrations across multiple vehicle platforms in different locations, we identify ten key challenges that hinder the development of weather-resilient autonomy, including data scarcity, domain shifts, limitations in multimodal perception, control robustness, and validation gaps, among others. We summarize these challenges and lessons learned toward enabling reliable snow-autonomous driving, highlighting critical directions for future research and standardization.
Figures & tables
Modality
3D Annotation
Dataset
Year
LiDAR
RGB
Radar
Thermal
GNSS/IMU
BB
SL
SF
#Frames
#Classes
Coverage
Snow
KITTI [ 54 ]
2012
64
90 ∘
-
-
✓/✓
✓
-
-
15K
8
39.2 km
-
KAIST [ 55 ]
2018
32
26 ∘
-
25 ∘
✓/✓
✓
-
-
-
3
44.8 km
-
nuScenes [ 56 ]
2019
32
360 ∘
✓
-
✓/✓
✓
✓
-
40K
23
242 km
-
Waymo [ 57 ]
2020
64
360 ∘
-
-
✓/✓
✓
✓
-
230K
4
1750 km
-
A2D2 [ 58 ]
2020
16
360 ∘
-
-
✓/✓
✓
-
-
12K
14
-
-
TABLE I: Comparison of SoA outdoor public autonomous vehicle datasets. BB, SL, and SF refer to 3D annotations for Bounding Boxes, Semantic Labels, and Snow Flakes, respectively. - represents unsupported or no statistics.
Fig. 0: Sample images from the ROADVIEW Snowy Scenes [ 63 ] (left) and M3 Scenes [ 64 ] (right) datasets. Both datasets have different multimodal sensor setups and provide dense point-wise annotations, including falling snowflakes, and 3D bounding boxes to benchmark 3D semantic segmentation, object detection, and denoising under harsh weather conditions.
Fig. 1: Sample snowy scenes from [ 64 ] logged at different time steps. The time difference between the top and bottom rows is 8.5 seconds. The left column shows LiDAR point clouds in gray and RADAR data points in red. This difference is also observable in the front-facing camera.
Fig. 2: Illustration of the characteristics of LiDAR, RADAR, RGB camera, and thermal sensors. Contextually, LiDAR-RADAR-RGB-Thermal fusion has the highest complementary properties.
Fig. 3: 3D point cloud denoising in two ROADVIEW demonstrations: a passenger vehicle operating under snowfall in Finland (left) and a truck driving on a rainy highway in Turkey (right). The top row shows the point clouds before and after denoising. Corresponding RGB images are provided for visualization purposes only.
Fig. 4: Visualization of weight maps learned in the adaptive gated fusion mechanism in LRC-WeatherNet [ 79 ] for two sample scenes: clear daytime (left) and snowy (right) scenes.
Fig. 5: Multimodal uncertainty-aware slipperiness estimation [ 80 ] . ROADVIEW vehicle platform equipped with a 360-degree, high-resolution 128-beam LiDAR, a front-facing RGB camera, three thermal cameras, GNSS/IMU, and a Vaisala MD30 sensor (left). All thermal, LiDAR, and Vaisala data are projected to the RGB image space (middle). A sample road surface grip prediction (top: RGB, middle: predicted grip value, bottom: predicted grip uncertainty) (right). The predicted grip value is the mean of the predicted grip probability distribution, and lighter colors indicate more slippery surfaces. The predicted grip uncertainty is the distance between the mean of the grip distribution and the 5th percentile. Likewise, lighter colors represent higher prediction certainty.
Fig. 6: Accumulated semantic LiDAR point clouds located using the GNSS/INS measurements (a) and GLO-SAM-corrected trajectory (b). The corrected reference trajectory results in substantially improved global consistency of the accumulated point cloud.
Fig. 7: Visualization of the EA-NDT HD Map representation from Muonio centre, northern Finland. The data were recorded with the FGI ARVO autonomous research platform visible in Figs. 0 and 3 . The entire mapped area of Muonio is described in more detail in [ 85 ] .
Fig. 8: A box-and-whisker plot of absolute position (a) and absolute orientation (b) errors along an 11-km-long trajectory in Muonio, northern Finland.
Fig. 9: The visual difference of the environment demonstrated with a front camera view of the same scene in the centre of Muonio in (a) during the EA-NDT map data collection in April 2025 with the FGI ARVO vehicle, and (b) during the EA-NDT positioning demonstration in January 2026 with the VTT Martti vehicle.
Fig. 10: A box-and-whisker plot of RPE for the odometry benchmark using CT-ICP, F-LOAM, and KISS-ICP. The methods are evaluated using the original KITTI and revised FGI-KITTI scans against both the original KITTI and revised FGI-KITTI reference trajectories. The results demonstrate the effect of reference-trajectory accuracy on LiDAR odometry benchmarking.
Fig. 11: Camera-based roadside weather detection: weather type detected as snow, ground conditions as wet and snowy.
Fig. 12: Multimodal sensing in snowy conditions (a) Camera-based detection system correctly detecting the black car; (b) LiDAR-based detection system correctly detecting the black car; (c) Camera-based detection system partially detecting the white car due to snow spray; (d) LiDAR-based detection system correctly detecting the white car.
Fig. 13: Recall of roadside camera-based (left) and LiDAR-based (right) vehicle detection systems measured under different weather conditions and vehicle trajectories. The x-axis shows the recall score, while the y-axis represents the vehicle trajectory (path). Results show that perception performance is dependent on both weather conditions and vehicle trajectories, highlighting the need for confidence indicators and quality metadata in collective perception messages. Path labels are abbreviated as B=bottom, M=middle, and T=top.
Fig. 14: ROADVIEW look-ahead road-condition estimation for weather-aware decision making. The grip predictor (top-left) estimates local road-surface grip and uncertainty from forward-looking sensor data, while the direction-aware visibility estimator (bottom-left) assesses the operational perception range under reduced visibility [ 85 ] . In the ROADVIEW demonstration (top-middle), projected road-friction estimates were converted into braking-distance and safety-margin constraints. The vehicle initiated braking earlier on icy surfaces than on snow-covered surfaces (bottom-middle) and increased the following safety margin when reduced friction was estimated (right) [ 85 ] . Click here to see the demonstration video.
Fig. 15: XiL testing of the same scene under different weather conditions: (from left to right) sunny, snowy, foggy, and rainy.
Fig. 16: ROADVIEW Snowy Scenes dataset [ 63 ] coverage analysis: 3D voxelized distribution of object annotations as a function of sensor distance, yaw angle, and object category.
Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.
Autonomous driving under adverse weather remains a critical challenge, yet existing vision-language benchmarks mainly evaluate under standard conditions, synthetic corruptions, or single modality. As a result, it remains unclear how vision-language models behave under real-world adverse weather with multi-modal inputs. We argue that a key difficulty lies in degraded environmental observability: under fog, rain, snow, and low illumination, multi-modal observations become unreliable and cross-modally inconsistent, posing challenges to scene understanding, and subsequent decision-making. To study this, we introduce ObsDriveBench, a real-world multi-modal benchmark for adverse-weather autonomous driving. Our benchmark is designed with three capability dimensions: observability awareness, spatial reliability, and risk-aware decision-making, enabling fine-grained diagnosis of model behavior under degraded observations. We construct the benchmark through observability meta-annotation, scene description, and capability oriented multiple-choice tasks over synchronized camera, LiDAR, and radar inputs, forming a benchmark with over 14k training and 13k test questions. Experiments reveal consistent performance degradation of existing vision-language models. We further introduce ObsDrive model with normal-weather supervised fine-tuning and adverse-weather reinforcement learning, improving robustness across all three capabilities. The dataset and evaluation code will be released at ObsDriveBench.
Qiao Yan, Yihan Wang, Zhenghao Xing +2
Department of Computer Science and Engineering, The Chinese University of Hong Kong · Institute of Medical Intelligence and XR, The Chinese University of Hong Kong
LiDAR sensors provide high-resolution 3D perception and long-range detection, making them indispensable for autonomous driving and robotics. However, their performance significantly degrades under adverse weather conditions such as snow, rain, and fog, where spurious noise points dominate the point cloud and lead to false perception. To address this problem, various approaches have been proposed: distance-based filters exploiting spatial sparsity, intensity-based filters leveraging reflectance distributions, and learning-based methods that adapt to complex environments. Nevertheless, distance-based methods struggle to distinguish valid object points from noise, intensity-based methods often rely on fixed thresholds that lack adaptability to changing conditions, and learning-based methods suffer from the high cost of annotation, limited generalization, and computational overhead. In this study, we propose LIORNet, which eliminates these drawbacks and integrates the strengths of all three paradigms. LIORNet is built upon a U-Net++ backbone and employs a self-supervised learning strategy guided by pseudo-labels generated from multiple physical and statistical cues, including range-dependent intensity thresholds, snow reflectivity, point sparsity, and sensing range constraints. This design enables LIORNet to distinguish noise points from environmental structures without requiring manual annotations, thereby overcoming the difficulty of snow labeling and the limitations of single-principle approaches. Extensive experiments on the WADS and CADC datasets demonstrate that LIORNet outperforms state-of-the-art filtering algorithms in both accuracy and runtime while preserving critical environmental features. These results highlight LIORNet as a practical and robust solution for LiDAR perception in extreme weather, with strong potential for real-time deployment in autonomous driving systems.
Ji-il Park, Inwook Shim
Ministry of National Defense, 22, Itaewon-ro, Yongsan-gu, Seoul, Republic of Korea. · Department of Smart Mobility Engineering, Inha University, Republic of Korea.
Adverse conditions such as rain, snow, fog, and dust remain challenging for camera-based perception in autonomous driving. We study multi-class weather recognition from street-view images under domain shift, where most available training data come from non-street-view sources that differ markedly from real driving scenes. We propose Weather-Aware Adversarial Discriminative Domain Adaptation (WA-ADDA), which conditions the domain discriminator on predicted weather to promote features that are both domain-invariant and weather-sensitive. We also assemble a multi-dataset benchmark by unifying diverse non-street-view weather collections as sources and real street-view images as targets, and define a standardized evaluation protocol with macro accuracy as the primary metric. Across backbones (ResNet-50, EfficientNet, VGG, DenseNet), WA-ADDA consistently improves street-view performance and yields strong per-class recalls in challenging conditions while preserving clear-weather accuracy. These findings highlight the feasibility of domain-adapted weather recognition and the value of our benchmark for advancing robust, on-board perception.
Hossein Maghsoumi, George Atia, Yaser P. Fallah
Department of Electrical and Computer Engineering University of Central Florida, Orlando, USA · Department of Computer Science University of Central Florida, Orlando, USA