Tri-Perspective View (TPV) representations describe 3D scene structure through top, side, and front feature planes, but existing TPV lifting is primarily camera-based, leaving the depth of sampled image evidence ambiguous along projected camera rays. We propose RLG-TPV, a multimodal TPV framework for camera-radar 3D object detection in which radar and training-time LiDAR provide complementary geometric guidance during representation construction. A ray-guided deformable-attention lift weights sampled image features using LiDAR-supervised camera depth probabilities and radar frustum occupancy, while radar additionally refines the depth distribution before lifting. Because conventional radar provides limited elevation information, LiDAR-derived class-occupancy targets supervise the side and front planes during training; the corresponding heads are removed at inference, so deployment requires only cameras and radar. For temporal aggregation, Doppler-guided temporal fusion aligns past features using a motion field anchored by measured radar radial velocity, with gating that limits warping in regions without supported motion. An RCS-aware radar scatter further allows radar evidence to spread over spatial neighborhoods conditioned on radar cross section. On the nuScenes validation set, RLG-TPV achieves 0.4981 mAP and 0.5959 NDS, reducing orientation and velocity error by 31.9% and 30.7% relative to the published CRN baseline. Ablation studies show that ray-level geometric guidance is a major contributor to the final performance.
Figures & tables
Figure 1: Overview of RLG-TPV. Camera features are lifted into TPV using depth- and radar-guided attention, fused with RCS-aware radar features, temporally aligned using Doppler, and aggregated into BEV for 3D detection. Dashed paths denote training-only LiDAR supervision.
Figure 2: Radar-refined depth and ray-guided lifting. Radar features bias the camera depth logits via bR and provide occupancy OR for per-ray sample weighting before deformable-attention aggregation into the three TPV planes; D in the figure denotes Dbins .
Method
Input
Backbone
mAP
NDS
mATE
mASE
mAOE
mAVE
mAAE
CenterFusion [ 17 ]
C+R
DLA-34
0.332
0.453
0.649
0.263
0.535
0.540
0.142
BEVDepth [ 12 ]
C
ResNet-50
0.351
0.475
0.639
0.267
0.479
0.428
0.198
CRAFT [ 8 ]
C+R
DLA-34
0.411
0.517
0.494
0.276
0.454
0.486
0.176
CRN [ 9 ]
C+R
ResNet-50
0.490
0.560
0.487
0.277
0.542
0.344
0.197
RCBEVDet [ 14 ]
C+R
ResNet-50
0.453
0.568
0.486
0.285
0.404
0.220
0.192
CRT-Fusion [ 7 ]
C+R
ResNet-50
0.500
0.572
0.499
0.277
0.531
0.261
0.192
Table 1: Comparison with camera (C) and camera-radar (C+R) methods on nuScenes val at 256×704 . Published CRN is shown here; its released implementation is used for ablations. Higher is better for mAP/NDS and lower for the error metrics.
Lift variant
mAP
NDS
LSS + RVT (warm-start)
0.4886
0.5842
Unguided attentive
0.4266
0.5321
Per-cell radar gate
0.4311
0.5354
Per-ray P(d)× radar, xy only
0.4804
0.5782
XYONLY (keyframe-only)
0.4407
0.5422
Table 2: Camera-lift ablation on nuScenes val. Radar and MFA are fixed. Unguided variants use three TPV planes with a 3D-conv head, while per-ray variants use xy-only CenterPoint.
Configuration
mAP
NDS
CRN baseline (released code)
0.4725
0.5617
Single-plane warm start (LSS + RVT)
0.4886
0.5842
Ray-guided TPV lift, xy only
0.4804
0.5782
+ height planes, plane sup., RCS scatter
0.4920
0.5882
target: GT boxes → LiDAR occ.
0.4925
0.5872
+ radar-refined depth (RRD)
0.4932
0.5893
Table 3: Incremental construction of RLG-TPV on nuScenes val. From the ray-guided lift onward, each row is warm-started from the previous configuration and retrained for 24 epochs; changes within ±0.005 run-to-run variation are discussed in the text. The warm start is the LSS + RVT row of Tab. 2 .
Figure 3: Qualitative comparison on nuScenes val. Columns: ground truth, CRN [ 9 ] , RLG-TPV. Boxes are green (GT), orange (CRN) and blue (ours); map and LiDAR points are shown for visualisation only, as inference uses cameras and radar. Top: daytime crossing. Bottom: night.
Configuration
Δ mAP
Δ NDS
− radar entirely (camera-only)
−35.5%
−30.2%
− temporal context (1 keyframe)
−11.5%
−18.8%
− ray guidance (unguided attention)
−7.3%
−7.2%
− sampling density ( K=4 )
−2.7%
−0.7%
− RCS-aware scatter
−2.1%
−4.7%
− radar-refined depth (RRD)
−0.1%
−1.4%
Table 4: Removal ablations on nuScenes val. Each row removes one component from the full model and reports the relative change after retraining.