Weather Adaptive Autonomous Driving

Nicholas Taylor; Alexander Picon; Sudip Dhakal

cs.CVNov 3, 2026

Weather Adaptive Autonomous Driving

Authors: Nicholas Taylor, Alexander Picon, Sudip Dhakal

Organizations: Dept. of Computing and Software Engineering, Florida Gulf Coast University, Fort Myers, FL, USA

Abstract

Camera object detectors lose recall in rain, fog, snow and sand, and the confidence threshold at which a detector is deployed decides how many objects it misses and how many false alarms it raises. We study how a frozen detector should make this decision in adverse weather. We first reproduce SeqCRC, a conformal object detector whose first step calibrates the confidence threshold by conformal risk control (CRC), and use that step, calibrated separately in each weather condition, as our baseline. With YOLOv8x it keeps the held-out miss fraction within a 20% budget in every DAWN weather bin, where a single global threshold fails in fog, but the false alarms it needs are an output rather than a choice: from 1.6 to 117 per image across DAWN and BDD100K conditions. We then propose a calibrated trade-off dial. One setting, $λ$ , the number of false alarms accepted to avoid one missed object, selects the expected-cost-optimal decision rule: either one score threshold, or a per-detection rule that keeps a box when its calibrated probability of being a real object, estimated by gradient-boosted trees from the detector's own per-box outputs, is at least $1/(1+λ)$ . F-score preferences map onto the same dial, and CRC applied to $λ$ itself is planned to restore a per-condition miss guarantee. The dial is fitted once on BDD100K train and graded only on BDD100K val and DAWN.

Figures & tables

Paper	Method	Sensor / input	Dataset	Key result	Limitation
Weather-robust detection and benchmarks
Sakaridis [ 1 ] , IJCV’18	Physics-based synthetic fog; fog fine-tuning	RGB camera	Foggy Cityscapes; Foggy Driving (101 real)	Car AP 30.5 $\to$ 35.3 on 33 finely annotated real images	Homogeneous daytime fog; tiny real test set
Hassaballah [ 8 ] , T-ITS’21	Visibility enhancement + multi-scale CNN + GM-PHD tracker; introduces DAWN	RGB camera	DAWN (fog, rain, snow, sand), KITTI, COCO	Reported to outperform 21 compared vehicle detectors	Small real-image set (1,027 released images); no official splits
Chen [ 2 ] , CVPR’18	Adversarial image + instance alignment (DA Faster R-CNN)	RGB camera	Cityscapes $\to$ Foggy Cityscapes; SIM10k; KITTI	Foggy [email protected] 18.8 $\to$ 27.6	Retrains per shift; synthetic fog; AP/mAP only, no operating point
Li [ 3 ] , WACV’23	Hardness-weighted GRL + rainy-domain triplet loss (DA-Detect)	RGB camera	Cityscapes $\to$ Foggy Cityscapes; KITTI	Foggy [email protected] 23.41 $\to$ 42.34 (oracle 45.6)	Synthetic fog only; 3.5-pt hyperparameter swing; no operating point
Liu [ 4 ] , AAAI’22	CNN-predicted differentiable filters before YOLOv3	RGB camera	VOC-Foggy/Dark; RTTS; ExDark	RTTS mAP 28.82 $\to$ 37.08 (30.80 for same-data YOLOv3)	Retrains detector; mAP only

TABLE I: Related work at a glance (one row per paper; AP and mAP as defined in each paper).

	Baseline (SeqCRC step 1 per weather bin)	Ours (trade-off dial)
Detector	YOLOv8x, frozen COCO weights, 1280 px, score floor 0.001	unchanged
Decision rule	one score threshold per weather bin, $s\geq\hat{\tau}(b)$	per detection, $p(d)\geq 1/(1+\lambda)$ ; simple option: one threshold $s\geq\tau(\lambda)$
Trade-off	fixed miss budget $\alpha$ ; false alarms unpriced	explicit input $\lambda$ (or $\beta$ ); $\alpha$ planned as a floor via CRC over $\lambda$
Run-time inputs	score and the weather bin	score, box context and the detector’s per-box outputs (no weather label)
Fitting	calibration half of each bin	once, on 21,831 labelled BDD100K train frames
Added	—	per-box feature tap; calibrated model; $\beta\to\lambda$ map; label-free estimate of false alarms

TABLE II: What our method keeps, replaces and adds relative to the Phase-1 baseline.

Fig. 1: Architecture, following one BDD100K val frame (rain at night) through the online path. New components are orange, unchanged ones grey; the dashed grey lane is the Phase-1 baseline, kept for comparison, and the dashed blue box is planned for Phase 4. Fitting (green band) uses only BDD100K train; grading uses BDD100K val and DAWN.

Choice	Why	Alternatives considered
Keep the detector frozen; decide after it	calibration stays valid; one detector run serves every setting; no retraining per condition	domain adaptation (DA-Detect: 20.41 vs 21.44 AP@50 on real DAWN fog in Phase 1); fine-tuning (invalidates every calibration)
Trade-off as $\lambda$ , false alarms per avoided miss	use case sets the balance; the optimal rule is explicit (Eq. 4 ); $\beta$ and $\alpha$ map onto it	fixed $\alpha$ only (baseline, false alarms unpriced); maximise $F_{1}$ (one fixed balance); FPPI cap
Decide per detection	87–91% of a per-image oracle’s advantage on BDD100K is chance; the learnable signal is in individual boxes	per-image thresholds from pixels, from the confidence profile, or from YOLOv8x image features (all failed pre-registered tests); box-combination rules; summed-confidence maps
One threshold kept as the simple option	one auditable number per $\lambda$ ; per-weather thresholds give no gain (0.98–1.03 $\times$ the cost)	per-weather thresholds (the baseline’s form)
Gradient-boosted trees on detector outputs	tabular inputs; trains in minutes on CPU; calibrated without post-hoc scaling (expected calibration error 0.11 points on BDD100K val)	recalibrating the score alone (this is Option A); a neural head; retraining YOLOv8x’s head
YOLOv8x’s own per-box outputs as features	pre-registered gain: cost 0.972 $\times$ (0.969–0.975) of the context-only model on BDD100K, not worse on DAWN (0.995)	context features only (model 1); a separate CNN on box crops (more compute)

TABLE III: Design choices, the reasons for them, and the alternatives considered (with the evidence).

	Matching / conf. loss / loc. loss-set / class set	Source	Confidence		Localization		Classification		Global
			size	risk	size	risk	size	risk	risk
F1	Hausdorff / bc-recall / pixel-add / APS	Paper	11.71	0.019	3.87	0.049	41.73	0.043	0.082
		Ours, full	11.53	0.020	1.30	0.053	40.64	0.042	0.085
		Ours, 300/300	11.11	0.025	1.30	0.040	42.67	0.033	0.066
F2	LAC / bc-thresh / pixel-add / LAC	Paper	18.86	0.012	20.64	0.047	0.72	0.049	0.086
		Ours, full	18.02	0.015	17.69	0.047	0.75	0.046	0.083

TABLE IV: SeqCRC with YOLOv8x on COCO val2017,

\alpha_{\text{tot}}=0.1

: set size and test risk per task (confidence: boxes per image, budget 0.02; localization: box stretch, budget 0.05; classification: classes per set, budget 0.05) and global risk (budget 0.1). Paper: its appendix Table VII. Full: the same 2,500/2,500 split as the paper’s code. 300/300: first 300 images for calibration, next 300 for test.

Boxes scored	AP	AP50	AP75
Ultralytics model table (paper’s detector)	53.9	–	–
Ours: Ultralytics predict, NMS IoU 0.7 per class	53.2	69.8	58.0
Ours: Ultralytics predict, class-agnostic NMS 0.5	51.6	68.8	55.6
Ours: cods wrapper, batch 1 (SeqCRC input)	45.1	65.9	48.2
Ours: cods wrapper, batch 8	10.7	32.1	1.4

TABLE V: Detector check: COCO box AP of YOLOv8x on all 5,000 val2017 images (pycocotools).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 7 — Fig. 2: Evidence: excerpts of the saved logs (lines selected and wrapped, text unchanged; the first line names the command). Top: a full-size SeqCRC run, F3 ( logs/10_full_F3.log ). Bottom: COCO evaluation of YOLOv8x through Ultralytics ( logs/11_detector_sanity_native.log ).

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 552 papers

CADENet: Condition-Adaptive Asynchronous Dual-Stream Enhancement Network for Adverse Weather Perception in Autonomous Driving
May 19, 2026Sherif Khairy, Catherine M. EliasImage Enhancement Autonomous Driving Perception
Computer Science & Engineering Department, German University in Cairo (GUC), Egypt · C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems, Cairo, Egypt
XWOD: A Real-World Benchmark for Object Detection under Extreme Weather Conditions
May 12, 2026Chih-Hsin Chen, Yu-Tung Liu, Amar Fadillah +2Autonomous Driving Datasets Autonomous Driving Benchmarks
Department of Electronic Engineering, National Taipei University of Technology · Adobe Inc.
Real-World Perception for Autonomous Driving in Adverse Weather: Enhancing Standard Detectors via Foundation-Guided Auto-Annotation
Sep 22, 2026Sepideh Gohari, Goodarz Mehr, Azim EskandarianAutonomous Driving Perception Object Detection
Autonomous Robots and Vehicles Laboratory (ARVL), College of Engineering, Virginia Commonwealth University (VCU), Richmond, Virginia, United States

May 19, 2026J/K move · Enter open · S save

CADENet: Condition-Adaptive Asynchronous Dual-Stream Enhancement Network for Adverse Weather Perception in Autonomous Driving

Sherif Khairy, Catherine M. Elias

Computer Science & Engineering Department, German University in Cairo (GUC), Egypt · C-DRiVeS Lab: Cognitive Driving Research in Vehicular Systems, Cairo, Egypt

Image Enhancement Autonomous Driving Perception Object Detection Real-Time Object Detection

Open paper AskPDF arXiv

Adverse weather (rain, fog, sand, and snow) degrades camera-based object detection in autonomous vehicles. Existing enhancement-then-detect approaches stall the safety-critical perception loop, violating hard real-time requirements. Progress on this problem is also constrained by an under-recognized evaluation ceiling: ground truth annotated on degraded images cannot credit a detector that recovers objects the annotators themselves could not see, so a genuinely useful enhancement can register as a near-flat F1 gain. This paper presents CADENet (Condition-Adaptive Asynchronous Dual-stream Enhancement Network), a training-free three-thread system: Thread S (YOLOv11n) delivers detections at full frame rate with zero added latency; Thread Q applies condition-adaptive enhancement (CAPE) and fuses results via entropy-guided NMS (EG-NMS) without blocking Thread S; Thread E provides CLIP zero-shot weather classification, so new weather categories require only a new text prompt, with no labeled data and no retraining. Evaluated on 1327 DAWN images (YOLOv11m, IoU = 0.5, confidence = 0.25), CADENet achieves Recall = 0.0103 (micro), F1 = 0.0230 on snow, and F1 = 0.0038 on rain. We formalize the annotation completeness bias on DAWN-class data, so the reported F1 values are lower bounds on the true gain; recall is the annotation-gap-immune headline metric. Thread S sustains approximately 44 FPS regardless of enhancement load. No model retraining or additional sensor hardware is required.

Abstract

Autonomous Driving

Explore similar work

CADENet: Condition-Adaptive Asynchronous Dual-Stream Enhancement Network for Adverse Weather Perception in Autonomous Driving

XWOD: A Real-World Benchmark for Object Detection under Extreme Weather Conditions

Real-World Perception for Autonomous Driving in Adverse Weather: Enhancing Standard Detectors via Foundation-Guided Auto-Annotation