VRU detection in low-light conditions is a key component in ensuring VRU safety during the most collision prone time of the day. Currently, advances in low-light detection are limited due to the lack of large scale datasets and automated labeling methods. Most autolabeling methods use RGB detections as their labels source, which cannot be replicated in low-light conditions. To fill this gap in this paper we present an automated pipeline for low-light VRU labeling. We use LWIR-RGB image pairs and source our labels from LWIR as VRUs differentiate from the background due to their body temperature. Unlike our previous work that relied on a complex hardware setup to align the two modalities, we perform this experiment using a LWIR-RGB stereo setup, lowering hardware complexity and cost, making the pipeline easier to reproduce. We evaluate method end-to-end on a dataset collected in the capital region of Finland. Models trained on our generated labels beat both off-the-shelf baselines and models fine tuned on a hand labeled subset on F1, mAP@50 and LAMR metrics when tested on a manually labeled dataset, and the result holds for two different LWIR detectors. This shows that low-light VRU autolabeling can be made both accessible and effective without specialized alignment hardware.
Figures & tables
Figure 1: The proposed LWIR-RGB autolabeling pipeline.
Figure 2: LWIR-RGB stereo camera setup on the research vehicle.
Figure 3: LWIR and RGB chessboard images captured during calibration, with the chessboard being illuminated with a flood light. Top row corresponds to original images, and bottom row shows images undistorted with the calibrated parameters.
Figure 4: Visualizations of corresponding RGB and LWIR images, with the rectified images overlaid on top of each other to show the quality of the calibration.
Figure 5: Training data samples collected from various locations during data collection.
Base Model
Training Data
P
R
mAP@50
mAP@50-95
RF-DETR
FLIR-ADAS
0.682
0.5653
0.801
0.452
YOLOv11l
FLIR-ADAS
0.827
0.750
0.827
0.501
SAM3
None
0.560
0.315
0.339
0.110
Table I : Evaluation of the LWIR detection models on the FLIR-ADAS dataset. Bolded values indicate the best result in each column.
LWIR-YOLO labels
LWIR-DETR labels
ht
Frames
YOLO
DETR
Frames
YOLO
DETR
0.10
6164
0.208
0.161
5163
0.165
0.161
0.08
9489
0.226
0.187
8123
0.195
0.190
0.06
16455
0.252
0.210
13914
0.238
0.207
0.04
27930
0.243
0.225
24617
0.250
0.228
0.02
51483
0.282
0.219
53684
0.252
0.215
Table II : RGB detector model accuracies reported as mAP 50 on the test sequence, with different height thresholds used in the autolabel filtering. Confidence threshold selected as ct=0.25 . Best result per column in bold, second best underlined. Number of frames corresponds to the amount of image samples remaining in the training data after filtering.
Training set
Test
Labels
ct
ht
Images
Inst.
P
R
F1
mAP@50
mAP@50-95
LAMR ↓
Base Model YOLOv11l
COCO (zero-shot)
–
–
–
–
0.596
0.154
0.244
0.199
0.118
0.815
Manual (val seq.)
–
–
441
363
0.342
0.162
0.220
0.188
0.097
0.832
LWIR-YOLO
0.50
0.04
29513
75161
0.584
0.228
0.328
0.231
0.080
0.785
LWIR-YOLO
0.50
0.02
47914
174370
0.625
0.217
0.323
0.239
0.085
0.778
Table III : Low-light RGB detection on the test sequence (1440 images, 1173 annotated instances). Training-set size is reported per row, varying based on the filtering parameters. Green marks results better than the 441-frame manual reference within the same base model, red worse; color is applied to F1, mAP and LAMR only, since P and R are not informative as standalone values. P, R, F1, and LAMR are computed with an IoU-threshold of 0.5. LAMR is log-average miss rate over FPPI [10−2,100] , lower is better.
Training set
Test
Labels
ct
ht
Images
Inst.
P
R
F1
mAP@50
mAP@50-95
LAMR ↓
YOLOv11l
COCO (zero-shot)
–
–
–
–
0.596
0.154
0.244
0.199
0.118
0.815
Manual (val seq.)
–
–
441
363
0.342
0.162
0.220
0.188
0.097
0.832
LWIR-YOLO
0.25
0.02
51483
216466
0.613
0.208
0.311
0.282
0.124
0.750
LWIR-DETR
0.25
0.02
53684
212098
0.522
0.213
0.303
0.252
0.089
0.769
Table IV : Low-light RGB detection on the test sequence (1440 images, 1173 annotated instances). All autolabeled models use the same filtration setting, ct=0.25 and ht=0.02 . Resulting training set size is reported per row. Green marks results better than the 441-frame manual reference with the same base model, red worse; color is applied to F1, mAP and LAMR only, since P and R are not informative as standalone values. P, R, F1, and LAMR are computed with an IoU-threshold of 0.5. LAMR is log-average miss rate over FPPI [10−2,100] , lower is better.
Figure 6: Precision–Recall and log-average miss rate (MR) curves for YOLOv11l (top) and RF-DETR-Base (bottom).
Figure 7: Qualitative detection results on test set for YOLOv11l across four scenes (columns) for ground truth and selected models (rows). The LWIR-YOLO and LWIR-DETR labels were generated with ct=0.25 and ht=0.02 .