EagleVision: A Multi-Task Benchmark for Cross-Domain Perception in High-Speed Autonomous Racing

Zakhar Yagudin; Murad Mebrahtu; Ren Jin; Jiaqi Huang; Yujia Yue; Dzmitry Tsetserukou; Jorge Dias; Majid Khonji

cs.ROApr 13, 2026

EagleVision: A Multi-Task Benchmark for Cross-Domain Perception in High-Speed Autonomous Racing

Authors: Zakhar Yagudin, Murad Mebrahtu, Ren Jin, Jiaqi Huang, Yujia Yue, Dzmitry Tsetserukou, Jorge Dias, Majid Khonji

Organizations: Skolkovo Institute of Science, Intelligent Space Robotics Laboratory, Center for Engineering Systems and Sciences · Khalifa University, KUCARS-KU Center for Autonomous Robotic Systems, Department of Computer Science · Beijing Institute of Technology, Beijing Key Laboratory of UAV Autonomous Control

Abstract

High-speed autonomous racing presents extreme perception challenges, including large relative velocities and substantial domain shifts from conventional urban-driving datasets. Existing benchmarks do not adequately capture these high-dynamic conditions. We introduce EagleVision, a unified LiDAR-based multi-task benchmark for 3D detection and trajectory prediction in high-speed racing, providing newly annotated 3D bounding boxes for the Indy Autonomous Challenge dataset (14,893 frames) and the A2RL Real competition dataset (1,163 frames), together with 12,000 simulator-generated annotated frames, all standardized under a common evaluation protocol. Using a dataset-centric transfer framework, we quantify cross-domain generalization across urban, simulator, and real racing domains. Urban pretraining improves detection over scratch training (NDS 0.72 vs. 0.69), while intermediate pretraining on real racing data achieves the best transfer to A2RL (NDS 0.726), outperforming simulator-only adaptation. For trajectory prediction, Indy-trained models surpass in-domain A2RL training on A2RL test sequences (FDE 0.947 vs. 1.250), highlighting the role of motion-distribution coverage in cross-domain forecasting. EagleVision enables systematic study of perception generalization under extreme high-speed dynamics. The dataset and benchmark are publicly available at https://avlab.io/EagleVision

Figures & tables

Fig. 1 : Autonomous racing platform operating under real A2RL competition.

	IAC	A2RL Sim	A2RL Real
LiDAR sensor	3 $\times$ Luminar Iris	Virtual LiDAR	3 $\times$ Seyond Falcon
Frequency (Hz)	20	10	10
Annotated frames	14,893	12,000	1,163
Avg. objects/frame	1	4	1
Points per scan (approx.)	$\sim$ 67k	$\sim$ 144k	$\sim$ 100k

TABLE I : Summary of racing datasets used for the detection benchmark.

Panel of Figure 3 — Fig. 2 : Overview of the proposed benchmark. (a) 3D detection across urban, simulator, and real racing domains. (b) Trajectory prediction settings, illustrating straight, mild turn, and sharp turn scenarios based on ground-truth annotations.

Setup	Train	Validation	$\mathcal{I}$ Test	$\mathcal{R}$ Test
$\mathcal{I}$	$21{,}848$	$4{,}611$	$4{,}529$	$328$
$\mathcal{I}^{v3}$	$12{,}259$	$2{,}959$	$15{,}828$	$328$
$\mathcal{I}^{v5}$	$12{,}668$	$3{,}130$	$15{,}247$	$328$
$\mathcal{R}$ (in-domain)	$212$	$29$	$-$	$29$

TABLE II : Trajectory counts for cross-domain and

\mathcal{R}

in-domain splits.

Setup	AP $\uparrow$	ATE $\downarrow$	ASE $\downarrow$	AOE $\downarrow$	NDS $\uparrow$
$\mathcal{R}$	0.843	0.1796	0.0716	0.0372	0.69266
$\mathcal{W}\rightarrow\mathcal{R}$	0.890	0.1702	0.0333	0.0231	0.72234
$\mathcal{W}\rightarrow(\mathcal{R}+\mathcal{S})_{1:1}$	0.847	0.1918	0.0348	0.0254	0.69830
$\mathcal{W}\rightarrow\mathcal{S}_{10}\rightarrow\mathcal{R}$	0.879	0.1580	0.0294	0.0208	0.71868
$\mathcal{W}\rightarrow(\mathcal{R}+0.1\mathcal{S})$	0.873	0.1589	0.0310	0.0209	0.71542
$\mathcal{W}\rightarrow(\mathcal{R}+0.1\mathcal{I})$	0.883	0.1666	0.0413	0.0213	0.71858

TABLE III : Detection performance on A2RL Real.

Dataset	Train		Validation		Test		$\mathcal{R}$
Dataset	ADE $\downarrow$	FDE $\downarrow$	ADE $\downarrow$	FDE $\downarrow$	ADE $\downarrow$	FDE $\downarrow$	ADE $\downarrow$	FDE $\downarrow$
$\mathcal{I}^{v5}$	0.502	0.905	1.152	2.326	0.774	1.511	0.478	0.947
$\mathcal{I}^{v3}$	0.672	1.175	1.624	3.254	0.862	1.611	1.567	4.205
$\mathcal{I}$	0.378	0.743	0.621	1.276	1.611	3.177	0.852	1.389

TABLE IV : Trajectory prediction performance in terms of ADE and FDE (lower is better).

Dataset	Train		Validation		Test
Dataset	ADE $\downarrow$	FDE $\downarrow$	ADE $\downarrow$	FDE $\downarrow$	ADE $\downarrow$	FDE $\downarrow$
$\mathcal{R}$	0.266	0.414	0.214	0.302	0.484	1.24
$\mathcal{I}$	0.502	0.905	1.152	2.326	0.478	0.947

TABLE V : Cross-dataset trajectory prediction results on

\mathcal{R}

. Models are trained on

\mathcal{R}

and

\mathcal{I}

respectively and evaluated on

\mathcal{R}

using ADE and FDE (

\downarrow

Dataset	Split	#Traj	Accel $p95$	Turn $p95$ (rad)	Best ADE / FDE
$\mathcal{I}$	Train	21848	0.337	0.013	0.288 / 0.594
	Validation	4611	0.335	0.015	0.565 / 1.166
	Test	4529	2.906	2.051	1.658 / 3.328
$\mathcal{I}^{v3}$	Train	12259	0.349	0.013	0.512 / 0.888
	Validation	2959	3.031	2.135	1.556 / 2.988
	Test ∗	15828	0.334	0.014	0.724 / 1.364

TABLE VI :

\mathcal{I}

split difficulty (p95) and best trajectory prediction performance (lower is better).

Fig. 3 : LiDAR point cloud visualization of 3D detection result on A2RL Real.

Panel of Figure 10 — Fig. 4 : Representative trajectory prediction examples from the $\mathcal{I}^{v5}$ training setup, showing observed history and predicted future trajectories.

Explore similar work

A2RL V\textsubscript{max}: The A2RL autonomous racing dataset for long-range, high-speed perception and multi-vehicle interaction
Jul 20, 2026Marvin Klemp, Dominic Ebner, Cornelius Schröder +15Autonomous Driving Perception Autonomous Racing
Institute of Measurement and Control Systems, Karlsruhe Institute of Technology · Code 19 Racing · Institute of Automotive Technology, Technical University of Munich +8
3D Lane Detection with Odometry for High-Speed Vehicle Racing
Jul 15, 2026Omoruyi Atekha, John Subosits, Marcus GreiffAutonomous Driving Datasets Autonomous Driving Perception
Massachusetts Institute of Technology (MIT), Cambridge, MA, USA. · Toyota Research Institute (TRI), 94022 Los Altos, CA, USA.
Cross-View Urban Traffic Dataset: Drone-Supervised Ground Truth for Monocular Bird's-Eye View Localization
Jun 5, 2026Prakhar Bhardwaj, Simone Weikl, Kilian Mang +1Intelligent Transportation Systems Autonomous Driving Perception
Faculty of Computer Science and Mathematics, OTH Regensburg

Jul 20, 2026J/K move · Enter open · S save

A2RL V\textsubscript{max}: The A2RL autonomous racing dataset for long-range, high-speed perception and multi-vehicle interaction

Marvin Klemp, Dominic Ebner, Cornelius Schröder, Davide Malvezzi+14

Institute of Measurement and Control Systems, Karlsruhe Institute of Technology · Code 19 Racing · Institute of Automotive Technology, Technical University of Munich+8

Autonomous Driving Perception Autonomous Racing 3D Object Detection Multi-Object Tracking

Open paper AskPDF arXiv

In autonomous driving development, a perception dataset is crucial, as it provides fundamental data for training, testing, and validating algorithms for an autonomous vehicle's multimodal perception systems. So far, most research has concentrated on providing datasets for well-structured urban environments. This work introduces the A2RL V\textsubscript{max} open-source dataset, specifically designed for perception tasks in high-speed autonomous driving and multi-vehicle interaction. The dataset was captured during the 2024 Abu Dhabi Autonomous Racing League (A2RL), held at the Yas Marina F1 Circuit, with participation from all competing teams. It contains diverse scenarios, including single-vehicle data at varying speeds, multi-vehicle sessions, and the full final four-vehicle race. The dataset contains almost 30,000 professionally annotated LiDAR point clouds, along with RADAR point clouds. In particular, it is the first large-scale dataset in autonomous racing to feature professionally annotated LiDAR point clouds, enabling deep learning-based perception research. The data is provided in a developer-friendly format, enabling easy implementation and evaluation in future research. We provide implementation and evaluation for off-the-shelf 3D detection and tracking methods. Although baseline methods show promising results for both 3D detection and tracking, specialized methods are required to address the unique challenges of high-speed autonomous driving. For a detailed description of the dataset, please visit the A2RL Vmax Dataset Website

Abstract

Explore similar work

A2RL V\textsubscript{max}: The A2RL autonomous racing dataset for long-range, high-speed perception and multi-vehicle interaction

3D Lane Detection with Odometry for High-Speed Vehicle Racing

Cross-View Urban Traffic Dataset: Drone-Supervised Ground Truth for Monocular Bird's-Eye View Localization