LiDSeg: Linear DehazeFormer Guidance for Efficient Semantic Segmentation in Adverse Weather Conditions
Authors: Yobsan Bayisa Leta, Christos Kyrkou, Maria K. Michael, Theocharis Theocharides
Organizations: KIOS Research and Innovation Center of Excellence, University of Cyprus, Nicosia, Cyprus · Department of Electrical and Computer Engineering, University of Cyprus, Nicosia, Cyprus
Robust semantic segmentation in adverse weather is essential for safety-critical autonomous systems. Existing restore-then-segment pipelines introduce latency and prioritize pixel-level restoration over semantic discriminability, while domain adaptation methods often face representation bottlenecks and computational complexity. We introduce LiDSeg, a framework that uses reconstruction-informed guidance to improve segmentation under adverse weather. During training, a lightweight guidance encoder is jointly optimized with the segmentation branch using segmentation and auxiliary reconstruction objectives. At inference, the reconstruction decoder is discarded, while the guidance encoder is retained to extract weather-robust features. A lightweight gated fusion module integrates these features into the segmentation branch, preserving clear-weather accuracy while improving adverse-weather robustness. On Foggy Cityscapes, LiDSeg limits the clear-to-dense-fog mIoU degradation to 2.5-2.8 percentage points, reducing the baseline performance drop by more than fivefold. On the official ACDC test server, LiDSeg-PIDNet achieves 75.07% pooled all-condition mIoU, comparable to supervised HRNet-W48 at 75.0%, while remaining substantially smaller. The gains generalize zero-shot to Foggy Driving and Foggy Zurich without training on real-fog images. LiDSeg adds only 1.9-3.1M parameters across several models, with total model sizes of 5.2-39.5M parameters and no inference-time reconstruction overhead. It delivers real-time inference on the NVIDIA Jetson AGX Orin. Code: https://github.com/yobsanb/LiDSeg
Figures & tables
Figure 1: LiDSeg architecture. The degraded image I is processed by a guidance branch (Linear DehazeFormer encoder with training-only decoder) and a segmentation branch.
Figure 2: Feature stability on Foggy Cityscapes val (BiSeNetV2) ( β=0.02 ): (a) L2 feature drift (LiDSeg μ=0.186 vs. baseline μ=0.343 ). (b) Per-scene drift (LiDSeg lower on all scenes).
Figure 3: Reconstruction and segmentation training dynamics on Foggy Cityscapes validation set (four guided models; fog curriculum β=0.005→0.01→0.02 ). Validation mIoU ( solid , left axis) and guidance-encoder reconstruction PSNR ( dashed , right axis).
Method
Backbone
Params
GFLOPs
Clear (mIoU)
Light (mIoU)
Medium (mIoU)
Dense (mIoU)
Δ
Real-time semantic segmentation models
BiSeNetV2 [ Yu et al.(2021)Yu, Gao, Wang, Yu, Shen, and Sang ]
–
3.36M
99.7
69.1
60.9
58.8
54.2
↓ 14.9
DDRNet-23s [ Hong et al.(2021)Hong, Pan, Sun, and Jia ]
–
5.70M
36.7
77.8
66.1
62.2
56.1
↓ 21.7
STDC1-Seg [ Fan et al.(2021)Fan, Lai, Huang, Wei, Chai, Luo, and Wei ]
–
8.43M
166.3
71.8
67.5
62.5
56.1
↓ 15.7
PIDNet-L [ Xu et al.(2023)Xu, Xiong, and Bhatt ]
–
37.31M
276.2
74.0
68.4
63.0
58.9
↓ 15.1
Unguided models with incremental (matched budget, same curriculum)
Table 1: Fog density robustness on Foggy Cityscapes val (mIoU %). Clear is clear-weather Cityscapes mIoU; Δ is the clear → dense ( β=0.02 ) mIoU drop (smaller is better). GFLOPs at 1024×2048 . † : denotes incremental widening to match the guided parameter/FLOP budget and trained on an identical fog curriculum without the guidance branch.
Method
Protocol
Params
GFLOPs
Fog
Night
Rain
Snow
Pooled All
Mean
Supervised ACDC labels
RefineNet [ Lin et al.(2017)Lin, Milan, Shen, and Reid ]
Sup.
118M
–
65.7
55.5
68.7
65.9
65.3
64.0
DeepLabV2 [ Sakaridis et al.(2021a)Sakaridis, Dai, and Van Gool ]
Sup.
44M
–
54.5
45.3
59.3
57.1
55.3
54.1
DeepLabV3+ [ Chen et al.(2018)Chen, Zhu, Papandreou, Schroff, and Adam ]
Sup.
62.7M
2032
69.1
60.9
74.1
69.6
70.0
68.4
HRNet-W48 [ Wang et al.(2020)Wang, Sun, Cheng, Jiang, Deng, Zhao, Liu, Ding, Guo, Xu, et al. ]
Sup.
65.9M
696
74.7
65.3
77.7
76.3
75.0
73.5
Mask2Former [ Cheng et al.(2022)Cheng, Misra, Schwing, Kirillov, and Garg ]
Sup.
107M
466
–
–
–
–
77.3
–
Table 2: ACDC test mIoU (%). Pooled All is the benchmark pooled all-condition evaluation; Mean is the arithmetic mean of Fog, Night, Rain, and Snow where per-condition scores are available. Sup.: supervised ACDC labels; Sup.-FT: LiDSeg fine-tuned with ACDC segmentation labels after synthetic paired pretraining; UDA: adaptation with unlabeled target images. (–): not reported.
Method
Road
Side.
Buil.
Wall
Fenc.
Pole
Ligh.
Sign
Vege.
Terr.
Sky
Pers.
Ride.
Car
Truc.
Bus
Trai.
Moto.
Bicy.
mIoU
ADVENT [ Vu et al.(2019)Vu, Jain, Bucher, Cord, and Pérez ]
72.9
14.3
40.5
16.6
21.2
9.3
17.4
21.2
63.8
23.8
18.3
32.6
19.5
69.5
36.2
34.5
46.2
26.9
36.1
32.7
MGCDA † [ Sakaridis et al.(2020)Sakaridis, Dai, and Van Gool ]
73.4
28.7
69.9
19.3
26.3
36.8
53.0
53.3
75.4
32.0
84.6
51.0
26.1
77.6
43.2
45.9
53.9
32.7
41.5
48.7
DANNet † [ Wu et al.(2021)Wu, Han, Lin, Uzunbas, Goldstein, Lim, and Metaxas ]
84.3
54.2
77.6
38.0
30.0
18.9
41.6
35.2
71.3
39.4
86.6
48.7
29.2
76.2
41.6
43.0
58.6
32.6
43.9
50.0
DAFormer [ Hoyer et al.(2022a)Hoyer, Dai, and Van Gool ]
58.4
51.3
84.0
42.7
35.1
50.7
30.0
57.0
74.8
52.8
51.3
58.3
32.6
82.7
58.3
54.9
82.4
44.1
50.7
55.4
SSAC † [ Li et al.(2020)Li, Chen, Yuan, Liu, Zhang, Xing, Wang, and Liu ]
76.5
59.8
82.7
52.6
34.5
51.2
58.1
61.2
63.9
58.1
67.2
61.5
40.0
81.8
59.2
61.7
83.4
47.5
55.6
59.6
HRDA [ Hoyer et al.(2022b)Hoyer, Dai, and Van Gool ]
88.3
57.9
88.1
55.2
36.7
56.3
62.9
65.3
74.2
57.7
85.9
68.8
45.7
88.5
76.4
82.4
87.7
52.7
60.4
68.0
Table 3: All-conditions ACDC test per-class IoU (%). Comparison rows are reported by WA 2 Net [ Pan et al.(2025) ] . LiDSeg (PIDNet-L) is Uber training on ACDC. Bold and underlined values denote the best and second-best results, respectively.
Cond.
Setup
Model
road
sidew.
build.
wall
fence
pole
light
sign
veget.
terrain
sky
person
rider
car
truck
bus
train
motorc.
bicycle
mIoU
Fog
Uber
BiSeNetV2
97.1
85.9
88.8
69.9
44.8
68.0
78.0
75.0
90.6
61.1
98.3
68.2
77.5
85.5
64.3
96.2
91.7
53.9
62.1
76.7
DDRNet
97.2
85.5
88.5
70.3
44.9
68.1
77.3
74.2
90.5
60.7
98.3
66.9
75.1
84.5
62.2
95.7
90.5
52.1
65.1
76.2
STDC
97.0
85.4
88.5
70.5
45.0
68.3
76.6
74.3
90.4
61.1
98.3
67.7
76.4
85.2
66.5
95.7
90.8
57.8
65.6
76.9
PIDNet
97.2
86.1
88.7
69.9
44.3
68.6
78.3
75.1
90.6
61.8
98.3
69.6
74.9
85.2
63.2
95.8
91.6
52.3
65.6
76.7
Expert
BiSeNetV2
97.2
86.6
87.5
68.0
41.6
64.6
72.4
67.9
90.2
61.4
98.3
61.0
67.2
85.0
54.4
86.2
87.6
31.2
57.3
71.9
DDRNet
96.5
84.7
86.2
66.3
40.0
61.3
70.0
65.6
89.8
61.5
98.2
61.8
68.3
83.8
53.2
88.3
83.6
30.6
58.6
71.0
Table 4: LiDSeg guided ACDC val per-condition IoU (%). Gray rows = Uber; white rows = Expert.
Model
Backbone
Method
Unlabeled FZ
FZ
FD
RefineNet [ Lin et al.(2017)Lin, Milan, Shen, and Reid ]
ResNet-101
SFT
×
34.6
44.3
RefineNet-lw [ Nekrasov et al.(2018)Nekrasov, Dharmasiri, Spek, Drummond, Shen, and Reid ]
ResNet-101
SFT
×
28.5
43.6
AdaptSegNet [ Tsai et al.(2018) ]
ResNet-101
UDA
✓
25.0
29.7
AdvEnt [ Vu et al.(2019)Vu, Jain, Bucher, Cord, and Pérez ]
ResNet-101
UDA
✓
39.7
46.9
FDA [ Yang and Soatto(2020) ]
ResNet-101
STL
✓
22.2
21.8
DANN [ Ganin et al.(2016) ]
ResNet-101
UDA
✓
43.1
46.0
Table 5: Zero-shot mIoU (%) on Foggy Zurich v2 (FZ) and Foggy Driving (FD). SFT: supervised fine-tuning; UDA: unsupervised domain adaptation; STL: style transfer learning; CDA: curriculum domain adaptation. ✓: uses unlabeled real-fog images (Foggy Zurich) during training. No method uses FD during training.
Model
Variant
Params (M)
GFLOPs
FPS
Mem (MB)
Clear
Dense
Δ
BiSeNetV2
Baseline
3.36
24.92
94.2
51
69.1
54.2
↓ 14.9
DDRNet
Baseline
5.70
9.17
350.4
42
77.8
56.1
↓ 21.7
STDC
Baseline
8.43
41.57
267.0
47
71.8
56.1
↓ 15.7
PIDNet-L
Baseline
37.31
69.04
116.9
111
74.0
58.9
↓ 15.1
TopFormer-B
Baseline
7.06
9.69
271.7
76
75.8
59.0
↓ 16.8
SeaFormer-B
Baseline
8.58
3.53
166.1
58
77.7
64.6
↓ 13.0
Table 6: Jetson AGX Orin efficiency at 512×1024 . Guided = LiDSeg with the reconstruction decoder discarded at inference. Clear/Dense mIoU : Cityscapes clear-weather mIoU and Foggy Cityscapes dense-fog mIoU ( β=0.02 ); Δ is the clear → dense drop.
Setting
Rec.
Fusion
Params (M)
FLOPs (G)
Mean
ACDC
(a) Seg. only
No
–
3.36
99.7
58.0
56.0
(b) Recon.+Concat
Yes
Concat
5.70
110.8
66.1
65.0
(c) LGF, no recon.
No
LGF
5.22
108.8
66.7
67.0
(d) LiDSeg full
Yes
LGF
5.22
108.8
75.4
73.6
Table 7: Reconstruction and fusion control using BiSeNetV2. Mean denotes Foggy Cityscapes val mIoU averaged over β∈{0.005,0.01,0.02} ; ACDC denotes validation pooled-All mIoU. Params and FLOPs correspond to the deployed networks.
Guidance
Attn.
Struct.
Fusion
Rec.
Params (M)
FLOPs (G)
Mean
Dehazing + Segment
Softmax
Seq.
–
Yes
6.7
631.4
67.8
Heavy guidance
Softmax
Asy.
Concat
Yes
5.9
446.2
67.1
Heavy guidance
Softmax
Asy.
LGF
Yes
5.9
446.0
75.4
LiDSeg
Linear
Seq.
–
Yes
6.3
143.0
66.3
LiDSeg
Linear
Asy.
Concat
Yes
5.7
110.8
66.1
LiDSeg
Linear
Asy.
Spatial
Yes
6.3
126.5
75.6
Table 8: Architectural design control on Foggy Cityscapes val using BiSeNetV2. Mean denotes mIoU averaged over β∈{0.005,0.01,0.02} . Reconstruction supervision is retained for all guided variants. Params and FLOPs correspond to the deployed networks.
Backbone
Setting
Params (M)
GFLOPs
FPS
Fog
Night
Rain
Snow
Mean
TopFormer-B [ Zhang et al.(2022)Zhang, Huang, Luo, Chen, Wang, Liu, Yu, and Shen ]
Baseline
7.06
9.69
271.7
67.5
45.0
61.5
63.5
59.4
Expert
9.01
12.02
126.4
75.8
52.4
67.9
69.6
66.4
Uber
76.4
55.3
69.5
71.4
68.2
SeaFormer-B [ Wan et al.(2023)Wan, Huang, Lu, Yu, and Zhang ]
Baseline
8.58
3.53
166.1
67.0
44.5
61.0
63.0
58.9
Expert
10.98
6.05
130.3
75.4
52.0
67.5
69.2
66.0
Uber
76.0
54.7
69.1
70.8
67.7
Table 9: Model-scaling: Transformer based semantic segmentation under the identical training protocol. Baseline is the unguided. Efficiency is reported at 512×1024 (Jetson AGX Orin).
Semantic segmentation model
0.2
0.3
0.4 (default)
0.5
BiSeNetV2
74.1
74.3
75.4
75.2
DDRNet-23s
73.0
74.2
75.3
74.1
STDC1-Seg
75.1
75.2
75.3
75.0
PIDNet-L
74.0
74.9
75.3
74.6
Table S1: Sensitivity to the auxiliary-loss weight λaux across semantic segmentation models (Foggy Cityscapes val , mean fog mIoU, %); the λaux=0.4 default consistently achieves optimal performance ( bold ).
Setting
BiSeNetV2
DDRNet-23s
STDC1-Seg
PIDNet-L
(a) Multi-scale fusion
none (baseline)
58.0
61.5
62.0
63.4
s8
69.5
70.6
70.8
71.3
s16
70.4
71.3
71.5
71.9
s32
67.8
69.3
69.5
70.1
s8+s16
73.6
73.9
73.9
74.1
Table S2: Design ablations across all four real-time models (Foggy Cityscapes val , mIoU % over the three fog densities). Each block varies one design choice; Fusion “none” is the unguided baseline.
Method
Road
Side.
Buil.
Wall
Fenc.
Pole
Ligh.
Sign
Vege.
Terr.
Sky
Pers.
Ride.
Car
Truc.
Bus
Trai.
Moto.
Bicy.
mIoU
LiDSeg (BiSeNetV2)
98
83
90
58
55
62
66
74
92
66
95
76
60
93
66
76
70
56
74
74.2
LiDSeg (DDRNet-23s)
99
82
90
57
54
62
66
73
92
67
95
75
60
93
69
75
71
55
73
74.1
LiDSeg (STDC1-Seg)
98
84
89
58
55
62
64
73
92
67
95
75
61
94
68
74
71
57
75
74.3
LiDSeg (PIDNet-L)
99
84
89
57
53
62
66
74
92
66
96
78
58
94
67
75
70
57
73
74.2
LiDSeg (TopFormer-B)
97
82
89
57
54
62
66
75
92
65
95
74
59
92
66
76
69
54
74
73.6
LiDSeg (SeaFormer-B)
97
82
89
56
53
61
66
74
91
66
95
73
59
92
65
76
70
54
74
73.3
Table S3: Per-class IoU (%) on Foggy Cityscapes val at dense fog ( β=0.02 ) for all LiDSeg variants, including the transformer extensions. Bold : best per class.
β
PSNR
SSIM
mIoU
0.005
34.2
0.946
76.3
0.01
33.3
0.929
75.7
0.02
31.0
0.902
74.2
Table S4: Reconstruction quality vs. fog density (BiSeNetV2). Input PSNR is foggy vs. clear.
Pipeline
Params
GFLOPs
FPS
mIoU
DehazeFormer → BiSeNetV2
0 5.9M
61.4
0 54
71.5
Restormer → BiSeNetV2
29.4M
213.0
0 18
72.3
LiDSeg (BiSeNetV2)
5.22M
27.2
83.4
74.2
Table S5: LiDSeg vs. restore-then-segment with the same BiSeNetV2 head. GFLOPs/FPS at 512×1024 (Jetson AGX Orin); mIoU at dense fog ( β=0.02 ).
Semantic segmentation model
FZ (unguided)
FZ (LiDSeg)
FD (unguided)
FD (LiDSeg)
BiSeNetV2
38.2
53.5
41.0
50.4
DDRNet-23s
40.1
52.8
42.5
51.0
PIDNet-L
39.5
52.6
41.8
50.8
STDC1-Seg
39.0
53.4
42.0
52.4
Table S6: Zero-shot mIoU (%) on Foggy Zurich v2 (FZ) and Foggy Driving (FD): unguided Semantic segmentation model vs. LiDSeg (no target labels).
Model
Variant
Params (M)
GFLOPs
FPS
Mem
BiSeNetV2
Baseline
3.36
6.23
161.5
18
DDRNet
Baseline
5.70
2.29
707.3
18
STDC
Baseline
8.43
10.39
724.2
23
SCTNet-B
Baseline
17.44
7.04
509.3
44
PIDNet-L
Baseline
37.31
17.26
308.0
80
TopFormer-B
Baseline
7.06
2.42
715.2
24
Table S7: Jetson AGX Orin efficiency at 256×512 (Benchmarking more efficiency in addition to 512×1024 Tab. 6 ). Guided: LiDSeg with the reconstruction decoder discarded at inference.
Model
Expert
Fog
Night
Rain
Snow
BiSeNetV2 (5.22M, 108.8 GFLOPs)
BiSeNetV2
Fog
71.9
41.7
54.4
51.5
BiSeNetV2
Night
50.4
53.1
43.4
42.1
BiSeNetV2
Rain
53.3
42.8
68.4
54.2
BiSeNetV2
Snow
51.8
40.7
57.1
67.1
DDRNet-23s (7.65M, 46.0 GFLOPs)
Table S8: ACDC val mIoU (%) for condition-specific Expert cross-evaluation. Rows: training condition; columns: test condition. Diagonal : same-condition performance. Bold and underline indicate the best and second-best off-diagonal values, respectively, for each model.
Figure S1: Foggy Cityscapes val : reconstruction and segmentation. Four Semantic segmentation models at medium ( β=0.01 ) and dense ( β=0.02 ) fog. Columns: foggy input, LiDSeg reconstruction (training only), clear reference, ground truth, unguided baseline, LiDSeg.
Figure S2: Real ACDC val : segmentation, all conditions for all Semantic segmentation models. Sixteen distinct scenes (Fog, Night, Rain, Snow and four Semantic segmentation models). Columns: adverse input, ground truth, unguided baseline, LiDSeg.
This paper describes our approach for the 8th UG2+ Workshop (CVPR 2026) Track~2, which targets semantic segmentation of outdoor scenes degraded by five weather conditions: blur, darkness, snow, haze, and glare. A central challenge we observe is a severe generalization gap -- models that perform well on the validation set often collapse on the test set. For instance, SegFormer-B5 drops 16.1 mIoU points from validation to test, suggesting that model capacity alone is insufficient for robustness. We investigate whether a carefully designed training recipe, rather than architectural complexity, can address this gap. Starting from a pre-trained SegMAN-S backbone, we systematically study the effects of domain-adaptive fine-tuning, multi-source data mixing, scene-balanced sampling, and synthetic degradation augmentation. Our final system achieves 59.9% mIoU on the official test set while maintaining a validation-test gap of only 6.5 points -- less than half that of larger models. We analyze negative results from architectural modifications, loss function variants, and model scaling to provide practical insights for weather-robust segmentation under limited data.
WeatherSeg, an advanced semi-supervised segmentation framework, addresses autonomous driving's environmental perception challenges in adverse weather while reducing annotation costs. This framework integrates a Dual Teacher-Student Weight-Sharing Model (DTSWSM) that enables knowledge distillation from weather-affected images, and a Classifier Weight Updating Attention Mechanism (CWUAM) that dynamically adjusts classifier weights based on environmental attributes. Comprehensive evaluations demonstrate that WeatherSeg significantly outperforms baseline models in both accuracy and robustness across various weather conditions, including clear, rainy, cloudy, and foggy scenarios, establishing it as an effective solution for all-weather semantic segmentation in autonomous driving and related applications.
Zhang Zhang, Yifeng Zeng, Houshi Jiang +1
School of Artificial Intelligence and Information Engineering, Zhejiang University of Science and Technology, Hangzhou, China · Department of Computer and Information Sciences, Northumbria University, Newcastle, United Kingdom · Beijing Quinovare Medical Technology Co., Ltd, Beijing, China +1
This report presents our solution for the WeatherProof Dataset Challenge, namely CVPR 2026 8th UG2+ Challenge Track 2: Semantic Segmentation in Adverse Weather. For the semantic segmentation task under adverse weather conditions, we propose a semi-supervised segmentation pipeline. Our method is trained exclusively on the WeatherProof dataset, without using any additional external data. Specifically, we adopt UniMatch V2 as the baseline model and treat all degraded-weather images as unlabeled data for semi-supervised training, thereby fully exploiting the data distribution provided by the challenge. During inference, we further apply test-time augmentation to improve the robustness and segmentation accuracy of the final predictions. The code is publicly available at: https://github.com/ylb888/weatherproof-challenge-unimatchv2.
Jinming Chai, Libo Yan, Licheng Jiao +1
School of Artificial Intelligence, Xidian University, Xi’an 710071, China