AevaScenes: An FMCW LiDAR Dataset and Benchmark for Long-Range Perception

Gautham Narayan Narasimhan; Heethesh Vhavle; Kumar Bhargav Viswanatha; James Reuther; Deva Ramanan

cs.CVSep 27, 2026

AevaScenes: An FMCW LiDAR Dataset and Benchmark for Long-Range Perception

Authors: Gautham Narayan Narasimhan, Heethesh Vhavle, Kumar Bhargav Viswanatha, James Reuther, Deva Ramanan

Organizations: Aeva Inc., Mountain View, CA, USA. · Carnegie Mellon University, Pittsburgh, PA, USA.

Abstract

FMCW LiDAR measures per-point radial Doppler velocity alongside range, providing a motion cue unavailable in conventional time-of-flight sensors. Exploiting this signal at long range remains understudied. We present an FMCW LiDAR dataset of 575 sequences (57.5K frames) with over 8 million annotated 3D boxes across 16 detection classes and per-point labels across 24 semantic classes, captured by six commercial FMCW LiDAR sensors and six paired 4K cameras across eight Bay Area cities, including 237 nighttime sequences, with annotations extending to 400m. We define a benchmark with three tasks: 3D object detection, scene flow estimation, and semantic segmentation. Detection and scene flow are evaluated across three range bins to 400m, with a public evaluation server. We explore the impact of Doppler measurements on flagship recognition tasks, and find significant improvements up to 2X in detection AP of far-away vehicles and pedestrians, particularly in low-latency single-frame settings. We similarly find scene flow accuracy is significantly improved with Doppler measurements across all ranges. Our dataset and benchmark have been publicly released at https://scenes.aeva.com.

Figures & tables

Panel of Figure 1 — Fig. 2 : Key advantages of FMCW LiDAR.

Panel of Figure 2 — Fig. 3 : FMCW LiDAR under challenging sensing conditions. Top: Direct sunlight. ( a ) Even under severe sun glare in the RGB image, ( b ) FMCW LiDAR still produces accurate range and velocity measurements, which ( c ) align well when overlaid. Bottom: Multi-LiDAR interference. ( d ) Here, the ego-vehicle is surrounded by other active LiDAR sensors (the Waymo vehicle). ( e ) FMCW LiDAR is immune to such interference and can still produce accurate range, velocity, and ( f ) reflectivity.

Year	Dataset	Scenes	Frames	Boxes/Fr	Pts/Fr	Objects	LiDARs	Type	Cameras	Range	Night (#)	Classes
		Scale					Sensor Setup			Coverage
2012	KITTI	22	15K	13	120K	0.2M	1	ToF	4	91 m	$\times$	8
2019	nuScenes	1000	40K	35	34K	1.4M	1	ToF	6	70 m	$\checkmark$	23
2020	Waymo Open	1150	230K	55	177K	12.6M	5	ToF	5	80 m	102	4
2023	Argoverse V2	1000	150K	33	107K	5M	2	ToF	9	214 m	$\times$	30
2023	HeLiPR	6	–	–	–	–	4	ToF+FMCW	–	200 m	$\checkmark$	–

TABLE I : Comparison with existing autonomous driving datasets. Our FMCW LiDAR dataset is the first large-scale dataset captured entirely with FMCW LiDAR, offering the highest point density (508K pts/fr), greatest annotation density (140 boxes/fr), longest effective range (400 m), and the second largest number of night sequences (237) among all compared datasets.

Panel of Figure 4 — Fig. 5 : Long-range annotation statistics of our FMCW LiDAR dataset. ( Top: ) distribution of 3D bounding boxes vs. distance from the ego vehicle; annotations extend up to 400 m (Waymo 80 m, nuScenes 141 m, Argoverse V2 214 m). ( Bottom: ) average LiDAR points per box vs. distance, remaining $\sim$ 13 points per box to 400 m.

Panel of Figure 5 — Fig. 6 : Class distribution of our annotated FMCW LiDAR dataset.

Panel of Figure 6 — Fig. 7 : Ego-motion compensation. ( Top: ) raw radial velocity measured by the LiDAR sensors moving on a highway towards the right. ( Bottom: ) after ego-motion compensation, static surfaces (e.g. ground) have zero velocity.

Panel of Figure 7 — Fig. 8 : Sensor setup and coordinate system

Panel of Figure 8 — Fig. 9 : Doppler improves single-frame and multi-frame long-range detection. We report AP ( $\uparrow$ ) by range for dynamic and all (static and dynamic) vehicles and pedestrians. At 200–400 m, adding Doppler to single-frame detectors consistently raises dynamic-vehicle AP from $0.33{\rightarrow}0.67$ (CenterPoint), $0.31{\rightarrow}0.63$ (DSVT), $0.39{\rightarrow}0.68$ (VoxelNeXt), and $0.50{\rightarrow}0.72$ (TransFusion). These velocity-aware single-frame detectors match or outperform their five-frame XYZ-only counterparts ( $0.52$ , $0.62$ , $0.56$ , and $0.60$ AP, respectively), using a 100 ms rather than 500 ms observation window. At the same range, all four single-frame XYZ-only models obtain $0.00$ dynamic-pedestrian AP, while Doppler raises VoxelNeXt to $0.12$ and TransFusion to $0.42$ . With five frames, Doppler further raises all-pedestrian AP from $0.01{\rightarrow}0.13$ for CenterPoint and $0.13{\rightarrow}0.33$ for TransFusion, showing that instantaneous motion and accumulated geometry provide complementary cues.

Fig. 10 : Scene-flow EPE ( $\downarrow$ ) by input configuration and range. Adding radial velocity reduces EPE for every plotted method, range, and point subset. At 200–400 m, all-point EPE decreases from $9.47{\rightarrow}5.02$ cm for DeFlow, $17.87{\rightarrow}5.44$ cm for Flow4D, and $10.34{\rightarrow}5.73$ cm for SSF.

Table 10

Panel of Figure 11 — Fig. 11 : Bird’s-eye-view visualization of FMCW LiDAR point cloud.

Range	Min pts	Max resid	Hits	Skip	Assoc dist
(m)	(cluster)	(m/s)	(mature)	(max)	(m)
0–50	8	5.0	3	5	8
50–100	3	5.0	2	5	8
100–200	3	8.0	2	5	12
200–300	3	8.0	2	5	15
300–400	4	6.0	3	3	15

TABLE IV : Range-tier overrides. Tiers are checked in order; the first tier whose max range covers the object is used.

Table 13

		Vehicle								Pedestrian
		AP				SpdE (m/s)				AP				SpdE (m/s)
Method	Input	0–100 m	100–200 m	200–400 m	Full Range	0–100 m	100–200 m	200–400 m	Full Range	0–100 m	100–200 m	200–400 m	Full Range	0–100 m	100–200 m	200–400 m	Full Range
Classical	1-frame	$0.61$	$0.59$	$0.41$	$0.58$	0.84	0.98	1.64	0.98	–	–	–	–	–	–	–	–
CenterPoint	1-frame	$0.64$	$0.65$	$0.33$	$0.62$	$8.99$	$10.54$	$12.99$	$9.91$	$0.03$	$0.02$	$0.00$	$0.03$	$1.27$	$2.32$	–	$1.32$
	+vel	$0.79$	$0.81$	$0.67$	$0.80$	$3.99$	$3.28$	$3.41$	$3.68$	$0.04$	$0.04$	$0.00$	$0.04$	$0.99$	$0.54$	–	$0.94$
	5-frame	$0.79$	$0.75$	$0.52$	$0.75$	$1.65$	$1.88$	$3.37$	$1.92$	$0.36$	$0.22$	$0.04$	$0.35$	$0.28$	$0.32$	$2.12$	$0.28$

TABLE VI : Dynamic object detection. Best AP (

\uparrow

) and SpdE (

\downarrow

) values are bold within each frame setting; one-frame configurations are green and five-frame configurations are orange.

		Vehicle								Pedestrian
		AP				SpdE (m/s)				AP				SpdE (m/s)
Method	Input	0–100 m	100–200 m	200–400 m	Full Range	0–100 m	100–200 m	200–400 m	Full Range	0–100 m	100–200 m	200–400 m	Full Range	0–100 m	100–200 m	200–400 m	Full Range
Classical	1-frame	$0.23$	$0.39$	$0.33$	$0.27$	$0.90$	1.07	2.61	$1.15$	–	–	–	–	–	–	–	–
CenterPoint	1-frame	$0.77$	$0.60$	$0.34$	$0.71$	$3.86$	$8.89$	$12.41$	$5.35$	$0.53$	$0.12$	$0.00$	$0.50$	$0.73$	$0.99$	–	$0.74$
	+vel	$0.76$	$0.68$	$0.58$	$0.74$	$1.77$	$2.85$	$4.34$	$2.18$	$0.65$	$0.37$	$0.01$	$0.63$	$0.76$	$0.99$	–	$0.77$
	5-frame	$0.79$	$0.63$	$0.39$	$0.74$	$0.78$	$1.60$	$3.81$	$1.15$	$0.75$	$0.46$	$0.01$	$0.72$	$0.41$	$0.56$	$2.73$	$0.42$

TABLE VII : All-object detection (static + dynamic). Best AP (

\uparrow

) and SpdE (

\downarrow

) values are bold within each frame setting; one-frame configurations are green and five-frame configurations are orange.

Table 16

Scene Flow Estimation

See all 12 papers

4DLidarOpen: An Open 4D FMCW Lidar Dataset for Motion-Aware Autonomous Driving
May 18, 2026Kane Qian, Xin Zhao, Yining Shi +7Autonomous Driving Datasets Autonomous Driving
1Tsinghua University. · 2Hesai Technology Co., Ltd.
FMCW-LIO: A Doppler LiDAR-Inertial Odometry
Sep 24, 2026Mingle Zhao, Jiahao Wang, Tianxiao Gao +2Inertial Odometry LiDAR Odometry
The authors are with the University of Macau.
UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles
Nov 23, 2025Siyi Li, Qingwen Zhang, Ishan Khatri +4Point Cloud Learning Autonomous Driving Perception
University of Pennsylvania · KTH Royal Institute of Technology · StackAV +1

May 18, 2026J/K move · Enter open · S save

4DLidarOpen: An Open 4D FMCW Lidar Dataset for Motion-Aware Autonomous Driving

Kane Qian, Xin Zhao, Yining Shi, Rujun Yan+6

1Tsinghua University. · 2Hesai Technology Co., Ltd.

Autonomous Driving Datasets Autonomous Driving Autonomous Driving Perception 3D Object Detection Bird's-Eye-View Segmentation Spatiotemporal Forecasting

Open paper AskPDF arXiv

We present 4DLidarOpen, a large-scale open multi-modal dataset for autonomous driving, centered on 4D frequency-modulated continuous-wave (FMCW) Lidar sensing. Unlike conventional time-of-flight Lidar datasets that mainly provide geometric measurements, 4DLidarOpen includes point-wise radial velocity measurements from a forward-facing 4D FMCW Lidar, together with multiple Lidars of different types, including rotating, solid-state, and blind-spot variants, surround-view cameras, and 6-DOF ego-vehicle poses. The dataset was collected in complex urban environments in Beijing and covers dense pedestrian interactions, congested traffic, high-speed driving, and unprotected maneuvers. 4DLidarOpen provides synchronized multi-sensor data and 3D bounding-box annotations with persistent track IDs across five object categories. A hybrid annotation strategy is adopted, where large-scale auto-labeled data support scalable training and human experts refine annotations for the human-annotated training and validation sets. Based on this dataset, we establish benchmarks for 3D object detection, birds-eye view (BEV) segmentation and flow prediction, and motion forecasting with planning. Extensive experiments show that direct velocity measurements from 4D FMCW Lidar provide complementary motion cues for dynamic-scene understanding. Compared with geometric-only sensing, the velocity-aware representation improves motion-related perception and downstream forecasting and planning, especially in scenarios involving vulnerable road users and fast-moving objects. These results indicate that 4D FMCW Lidar is a promising sensing modality for motion-aware autonomous driving. The dataset and evaluation toolkit are publicly released to support research on 4D scene understanding, multi-Lidar fusion, and velocity-aware perception and planning.

Abstract

Scene Flow Estimation

Explore similar work

4DLidarOpen: An Open 4D FMCW Lidar Dataset for Motion-Aware Autonomous Driving

FMCW-LIO: A Doppler LiDAR-Inertial Odometry

UniFlow: Zero-Shot LiDAR Scene Flow for Autonomous Vehicles