Robust 3D lane detection requires accurate metric road geometry recovery from visual inputs, yet conventional RGB-based approaches suffer from fragile lane evidence under adverse illumination, motion blur, and long-range perspective compression. Event cameras offer complementary high-temporal-resolution, high dynamic-range structural cues that are robust to these challenges, but their sparse, motion-dependent responses cannot be directly fused into consistent 3D geometry. In this paper, we present the first attempt to introduce event cameras into 3D lane detection. To enable systematic research, we first build two multimodal benchmarks with metric 3D lane annotations: DSEC-3DLD (real-world sequences) and Ev-OpenLane (large-scale simulated sequences). We further propose RETR, a structure preserving transformer that converts complementary RGB and event observations into coherent 3D lane geometry. RETR first aligns modality consistent evidence through Reciprocal Context Flow Fusion, then preserves thin and uncertain lane structures with Uncertainty Aware Structural Consolidation, and finally decodes ordered lane hypotheses using a Geometry State Decoder with proposal conditioned initialization, reference conditioned query evolution, and diversified geometric inquiry. Extensive experiments show that RETR achieves state-of-the-art performance on both benchmarks, especially showing strong improvements in challenging lighting and complex road geometry scenarios. Code and dataset will be available at https://github.com/duskswell/RETR.
Figures & tables
Figure 1: Overview of our DSEC-3DLD dataset. Each column presents one synchronized driving sample, including the RGB image, event frame, 2D lane annotation, and metric 3D lane annotation. The selected examples cover challenging real-world conditions such as tunnels, low illumination and strong exposure changes.
Figure 2: Overall architecture of RETR. RETR converts synchronized RGB and event observations into metric 3D lane geometry through a structure-preserving RGB-event pipeline. Modality-specific encoders first extract multi-scale RGB and event features {Frl}l=1L and {Fel}l=1L . RCF aligns both modalities at each scale by using a co-activation response Ml to emphasize jointly supported lane evidence and bidirectional Cross Attention Blocks (CAB) to exchange complementary context, producing fused features {Zl}l=1L . UASC aggregates the fused hierarchy into Fc while preserving thin and uncertain lane structures. It combines context and structural branches and gates detail refinement using high-frequency response A , learned detail cue D , and entropy-based uncertainty E . The query generator and geometry decoder finally predict ordered 3D lane hypotheses, including lane existence, 3D coordinates, and visibility.
Method
Overall
Scenario-wise F1-Score (%)
F1 ↑
P ↑
R ↑
CD ↓
Str. ↑
Cur. ↑
Int. ↑
M&S ↑
Nor. ↑
Ove. ↑
Und. ↑
U/D ↑
PersFormer [ 7 ]
50.1
48.7
51.5
0.127
51.5
48.3
48.8
58.9
53.3
31.3
41.8
34.5
SparsePers [ 62 ]
50.2
48.1
52.3
0.134
51.6
48.8
46.1
59.2
52.2
35.2
45.4
45.6
Anchor3DLane [ 19 ]
54.7
57.9
51.8
0.127
56.6
53.8
51.2
50.7
57.3
28.3
49.0
45.7
3DLaneDT [ 56 ]
53.7
55.2
52.2
0.130
54.1
54.2
51.2
53.3
56.2
24.9
48.8
43.5
Anchor3DLanePP [ 20 ]
61.8
67.1
57.4
0.121
63.8
59.9
59.3
65.6
64.4
39.7
56.2
48.6
Table 1: Comparison with state-of-the-art 3D lane detection methods on DSEC-3DLD. We report overall F1-score (F1), precision (P), recall (R), centerline distance error (CD), and scenario-wise F1-score . Scenario columns denote Straight (Str.), Curved (Cur.), Intersection (Int.), Merge/Split (M&S), Normal (Nor.), Overexposed (Ove.), Underexposed (Und.), and Up/Down road geometry (U/D). Best and second-best results are shown in bold and underlined , respectively.
Figure 3: Qualitative comparison on DSEC-3DLD and Ev-OpenLane. We compare RETR with the RGB-only LATR baseline under challenging illumination and road-geometry conditions. Each example shows image-plane lane predictions on the left and corresponding 3D geometry and top-view projections on the right. Red, green, and yellow denote ground truth, RETR, and LATR, respectively. Zoom in for details.
Component
RCF
UASC
GSD
F1-score (%) ↑
DSEC
Ev-Open
RGB only
✗
✗
✗
69.2
61.9
RCF
✓
✗
✗
72.9
64.5
RCF + UASC
✓
✓
✗
74.3
65.7
Ours
✓
✓
✓
75.6
66.7
Table 3: Comparison of component variants in RETR. Results are reported as F1-score (%) on DSEC-3DLD and Ev-OpenLane.
Method
All
Up & Down
Curve
Extreme Weather
Night
Intersection
Merge & Split
BEVLaneDet [ 55 ]
58.4
48.7
63.1
53.4
53.4
50.3
53.7
LATR [ 35 ]
61.9
55.2
68.2
57.1
55.4
52.3
61.5
HeightLane [ 45 ]
62.7
53.6
69.3
55.4
54.6
54.1
61.1
PVALane [ 63 ]
62.7
54.1
67.3
62.0
57.2
53.4
60.0
SC-Lane [ 44 ]
64.3
54.6
70.7
58.1
56.5
56.1
63.0
RETR (Ours)
66.7
60.0
73.1
59.8
60.5
58.9
65.9
Table 5: Scenario-wise quantitative comparison on the Ev-OpenLane validation set using F1-score. The best result in each column is highlighted in bold, and the second-best result is underlined.
Table 7
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: Annotation pipeline of DSEC-3DLD. We first prepare synchronized RGB images, event frames, and LiDAR points through cross-sensor alignment and calibration-based projection. Then, we construct a multi-source annotation pipeline with an image-based annotation branch and a geometry-cue branch. The image-based branch provides refined 2D lane annotations, while the geometry branch provides sparse geometric supervision and depth cues. These complementary sources are jointly refined to generate metric 3D lane ground-truth.
Parameter
Setting
Description
Original frame rate
10 fps
Frame rate of the original OpenLane key frames.
Interpolated frame rate
30 fps
Frame rate after temporal interpolation.
Interpolated frame interval
33.33 ms
Temporal interval between adjacent interpolated frames.
Image resolution
640×480
Resolution used for ESIM simulation and event representation.
Input timestamp interval
33.33 ms
Timestamp interval for ESIM input frames.
Data source
Image folder
ESIM reads interpolated frames and timestamps from an image sequence.
Appendix
Table 10: ESIM simulation settings for generating Ev-OpenLane. OpenLane images are resized to 640×480 and temporally upsampled to 30 fps before event simulation.
Table 11: Dataset statistics and sequence-level splits of DSEC-3DLD.
Table 12: Default architectural settings of RETR, including UASC and GSD.
Split Sequences Samples Lane instances Training 33 19,048 52,422 Validation 6 4,749 12,503 Test 13 6,359 17,821 Total 52 30,156 82,746
Symbol Description Value Lh High-resolution structural stages in UASC 2 N Initial lane proposals 12 Ks Activation maps per proposal 4 A Ordered anchors per proposal 20 C Hidden dimension 256 K Decoder layers 6 M Inquiry branches per decoder layer 2
Appendix
Table 11: Dataset statistics and sequence-level splits of DSEC-3DLD.
Table 13: Expert assessment of DSEC-3DLD annotations, reporting mean scores (1–5) and quality-control pass rates.
Table 14: Comparison of RETR input variants on DSEC-3DLD, including RGB-only, event-only, RGB-event, and noisy-event configurations.
Table 15: Event window ablation on DSEC-3DLD with other settings fixed.
Criterion Score Pass (%) 2D annotation 4.62 96.4 3D lane 4.41 93.7 Scenario label 4.70 97.8 Overall 4.57 95.9
Accum. window F1 (%) 25 ms 74.9 50 ms 75.6 100 ms 75.0
Appendix
Table 13: Expert assessment of DSEC-3DLD annotations, reporting mean scores (1–5) and quality-control pass rates.
Method
Params (M)
MACs (G)
Preproc. (ms)
Network (ms)
F1-score (%)
RETR-RGB
83.56
117.6
69.9
65.03
72.5
RETR-Event
84.93
119.2
78.5
64.92
64.8
RETR
95.47
148.6
80.9
81.26
75.6
Appendix
Table 16: Accuracy and computational cost of RETR and its single-modality variants on DSEC-3DLD. Preprocessing time and network latency are reported separately.
Figure 5: Additional qualitative results on DSEC-3DLD. Red dashed lines denote ground truth, green lines denote our predictions, and yellow lines denote LATR predictions. Our method shows better agreement with the ground truth in both image-space and 3D visualizations, demonstrating robust performance under real-world illumination and road-geometry variations.
Figure 6: Additional qualitative results on Ev-OpenLane. Red dashed lines denote ground truth, green lines denote our predictions, and yellow lines denote LATR predictions. Our method produces more coherent image-space and 3D lane predictions than LATR across diverse large-scale scenes, showing strong generalization on simulated RGB-event data.
Dept. of Electrical and Computer Engineering, Seoul National University · University of Michigan · Dept. of Future Automotive Mobility, Seoul National University +1