Structured Representation Probing for Risk Anticipation in End-to-End Autonomous Driving

Jiawei Liang; Tianrui Lou; Le Yang; Xianghao Jiao; Wei Wang; Xiaochun Cao

cs.LGOct 27, 2026

Structured Representation Probing for Risk Anticipation in End-to-End Autonomous Driving

Authors: Jiawei Liang, Tianrui Lou, Le Yang, Xianghao Jiao, Wei Wang, Xiaochun Cao

Organizations: School of Cyber Science and Technology, Shenzhen Campus of Sun Yat-sen University, Shenzhen 518107, China · School of Mathematics, Sun Yat-sen University, Guangzhou 510275, China

Abstract

End-to-end driving policies form structured internal representations of traffic, ego behavior, and road constraints. These representations offer a monitoring interface for anticipating failures, complementing risk signals from trajectories, controls, and manually designed features. We introduce the Structured Representation Probe (SRP), a lightweight monitor that reads actor, ego, and map representations from a frozen policy. SRP learns a separate linear risk direction for each source and aggregates element scores through cardinality-normalized log-mean-exp pooling, retaining sensitivity to salient evidence while reducing dependence on token count. Event-window supervision links the readout to upcoming hard failures, and auxiliary event type heads jointly train the source directions across failure modes. Online alerting uses a single generic score without modifying the driving policy. Bench2Drive experiments on DriveTransformer, UniAD, and VAD show higher frame-level PR-AUC and ROC-AUC than the evaluated runtime baselines. On DriveTransformer at a horizon of 2 seconds, ROC-AUC reaches 0.833 versus 0.715 for temporal consistency, with 2.67% readout runtime overhead. The trained probes also retain the ability to distinguish impending failures on new routes.

Figures & tables

Fig. 1: Our key insight is that an existing end-to-end driving policy already carries early safety cues in its internal representations, making those representations a natural substrate for risk anticipation.

Fig. 2: SRP framework. Actor, ego, and map representations from a frozen policy are projected into element logits. Both pooling branches use cardinality-normalized log-mean-exp (LME). Event-window labels supervise the generic readout and auxiliary event heads; $\mathcal{L}_{\mathrm{type}}$ averages weighted BCE across the three heads. Flames mark trainable risk directions. Online alerting uses only $r$ with a threshold selected on validation routes.

Model	Method	PR-AUC	ROC-AUC	$R_{F}$ @5%FPR	$R_{E}$ @5%FPR	pLead@5%FPR (s)
DriveTransformer	Temporal Consistency	0.127	0.715	0.145	0.505	0.633
DriveTransformer	Task-Aware Risk	0.088	0.570	0.060	0.431	0.532
DriveTransformer	TRFD	0.080	0.496	0.046	0.320	0.337
DriveTransformer	Ours	0.261	0.833	0.300	0.497	0.701
UniAD	Temporal Consistency	0.167	0.751	0.175	0.573	0.768
UniAD	Task-Aware Risk	0.070	0.506	0.042	0.339	0.455

TABLE I: Comparison with runtime monitors on the Bench2Drive test routes using training and evaluation windows of 2 seconds.

Fig. 3: Distributions of risk logits on the Bench2Drive test routes. Solid and dashed curves show positive and negative event-window labels, respectively. Dotted lines indicate mean policy-specific thresholds under the 5% frame-level FPR protocol.

Fig. 4: Representative qualitative warning cases. SRP raises its generic risk score before three distinct hard failures: a red light violation, a collision with another vehicle, and a collision with a roadside guardrail.

Model	Eval. $W$	PR-AUC	ROC-AUC	$R_{F}$ @5%FPR	$R_{E}$ @5%FPR	pLead@5%FPR (s)
DriveTransformer	20	0.176	0.856	0.320	0.437	0.376
DriveTransformer	40	0.261	0.833	0.300	0.497	0.701
DriveTransformer	60	0.315	0.803	0.265	0.556	0.984
UniAD	20	0.226	0.874	0.439	0.635	0.541
UniAD	40	0.339	0.868	0.439	0.725	1.124
UniAD	60	0.408	0.853	0.429	0.804	1.711

TABLE II: Effect of the evaluation horizon on SRP across three driving policies on the Bench2Drive test routes. Probes are trained with a window of 2 seconds and evaluated with windows of 1, 2, and 3 seconds.

Fig. 5: Sensitivity to the training horizon across three driving policies. Rows identify policies, columns show PR-AUC, ROC-AUC, event recall, and penalized lead time, and colors identify evaluation horizons. Shaded bands indicate performance variability.

Model	Type	PR-AUC	ROC-AUC	$R_{F}$ @5%	$R_{E}$ @5%	pLead @5%
DriveTrans.	dynamic	0.234	0.810	0.293	0.550	0.775
DriveTrans.	static	0.119	0.808	0.283	0.675	1.153
DriveTrans.	rule	0.130	0.832	0.388	0.857	1.329
UniAD	dynamic	0.166	0.798	0.313	0.609	1.004
UniAD	static	0.128	0.831	0.452	0.750	1.188
UniAD	rule	0.044	0.815	0.440	0.500	0.875

TABLE III: Diagnostic performance of the auxiliary heads by event type on the Bench2Drive test routes with an evaluation window of 2 seconds. Each head is evaluated against its own event labels; lead times are in seconds. DriveTrans. abbreviates DriveTransformer.

Fig. 6: Temporal warning profiles aligned to selected target events on Bench2Drive test routes, grouped by observed failure type. Solid and dashed curves compare generic scores and scores by event type; the bottom row shows source scores averaged across policies. Shading marks the evaluation window of 2 seconds.

Fig. 7: Generic warning lead time for groups of Bench2Drive test routes containing dynamic collision, static collision, or rule violation. Bars show mean pLead under the 5% frame-level FPR protocol.

Panel of Figure 11 — Fig. 8: Qualitative warning cases grouped by driving policy: a vehicle collision for DriveTransformer and static collisions for UniAD and VAD. N1 and R1 link the scene snapshots to the generic score traces.

Variant	Actor	Ego	Map	Aux. Sup.	PR-AUC	ROC-AUC	$R_{F}$ @5%FPR	$R_{E}$ @5%FPR	pLead@5%FPR (s)
No actor	–	✓	✓	✓	0.250	0.813	0.291	0.506	0.712
No ego	✓	–	✓	✓	0.229	0.784	0.261	0.310	0.504
No map	✓	✓	–	✓	0.251	0.821	0.289	0.485	0.680
Actor only	✓	–	–	✓	0.239	0.812	0.286	0.394	0.561
Ego only	–	✓	–	✓	0.245	0.795	0.285	0.495	0.704
Map only	–	–	✓	✓	0.196	0.784	0.204	0.372	0.516

TABLE IV: Effects of representation sources and auxiliary event supervision on the retrained DriveTransformer probe. Checkmarks identify the sources and supervision used; all variants use training and evaluation windows of 2 seconds.

Setting	PR-AUC	ROC-AUC	$R_{F}$ @5%	$R_{E}$ @5%	pLead@5%
Full	0.363	0.807	0.360	0.681	1.029
$\times$ actor	0.336	0.785	0.326	0.631	0.964
$\times$ ego	0.217	0.716	0.205	0.411	0.606
$\times$ map	0.366	0.804	0.366	0.674	1.022

TABLE V: Source deletion at inference from the fixed DriveTransformer probe on the new Bench2Drive evaluation set. A red cross marks the removed source; lead times are in seconds. Ego deletion has the largest effect on warning performance.

Fig. 9: Readout and pooling comparisons on DriveTransformer. Paired markers show mean PR-AUC and pLead; connecting lines identify the same setting. Red markers highlight SRP in (a) and its default LME pooling in (b).

Fig. 10: Sensitivity to pooling temperature on DriveTransformer. Red circles mark the default $\tau=1$ ; shaded bands indicate performance variability.

Target data	Model	PR-AUC	ROC-AUC	$R_{F}$ @5%FPR	$R_{E}$ @5%FPR	pLead@5%FPR (s)
Bench2Drive (new)	DriveTransformer	0.363	0.807	0.360	0.681	1.029
Bench2Drive (new)	UniAD	0.324	0.864	0.526	0.705	1.138
Bench2Drive (new)	VAD	0.314	0.923	0.604	0.793	1.361
LB2	DriveTransformer	0.312	0.769	0.344	0.737	1.142
LB2	UniAD	0.380	0.829	0.456	0.444	0.683
LB2	VAD	0.371	0.878	0.513	0.765	1.406

TABLE VI: Transfer of probes trained on the original routes to the new Bench2Drive evaluation set with 220 routes and 20 CARLA Leaderboard 2.0 validation routes. Policy and probe parameters remain fixed.

Evaluation set	Policy	FPR (%)	$R_{E}$	pLead (s)
B2D new	DT	6.36	0.723	1.157
B2D new	UniAD	5.53	0.719	1.181
B2D new	VAD	2.66	0.690	1.073
LB2	DT	2.88	0.684	1.079
LB2	UniAD	5.56	0.500	0.728
LB2	VAD	2.78	0.647	1.159

TABLE VII: Warning performance on new Bench2Drive and LB2 routes with source-validation thresholds held fixed. FPR is the observed false-positive rate.

Model	Params	FLOPs	Overhead
DriveTransformer	2.3K	0.220M	6.47 ms (2.67%)
UniAD	0.8K	0.328M	1.89 ms (0.20%)
VAD	0.8K	0.412M	3.37 ms (1.58%)

TABLE VIII: Parameter counts for the probe alone, FLOPs for the generic score, and measured readout latency across driving policies. Percentages are relative to policy runtime.

Autonomous Driving

Autonomous driving develops vehicles that perceive road scenes, predict interactions, and plan and control safe movement with limited or no human input.

See all 536 papers

Data-Driven Risk Fields for Safer End-to-End Autonomous Driving
Sep 9, 2026Yuanxin Tian, Zhiyuan Liu, Jinhao Li +9End-to-End Autonomous Driving Safe Motion Planning
Tsinghua University · Yinwang · KAIST
Risk-Aware Occupancy for Safety-Oriented End-to-End Autonomous Driving
Sep 18, 2026Jiaxing Chen, Hengduo Zou, Yiren Zhao +1End-to-End Autonomous Driving 3D Occupancy Prediction
School of Vehicle and Mobility, Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou)
Driving on Registers, Reasoning on Risk: Risk-Aware Occupancy for Register-Based End-to-End Autonomous Driving
Sep 18, 2026Jiaxing Chen, Hengduo Zou, YuKai Qin +3End-to-End Autonomous Driving Autonomous Driving Planning
School of Vehicle and Mobility, Tsinghua University · The Hong Kong University of Science and Technology (Guangzhou) · Neolix, Beijing, China

Sep 9, 2026J/K move · Enter open · S save

Data-Driven Risk Fields for Safer End-to-End Autonomous Driving

Yuanxin Tian, Zhiyuan Liu, Jinhao Li, Liangfan Zhu+8

Tsinghua University · Yinwang · KAIST

End-to-End Autonomous Driving Safe Motion Planning Autonomous Driving Planning

Open paper AskPDF arXiv

Safety is a fundamental requirement for autonomous driving, yet existing end-to-end driving models still lack explicit risk-aware learning capacities. Existing rule-based risk models provide interpretable safety priors, yet their absolute risk scores depend on handcrafted functions, coefficients, and thresholds. Learning-based risk representations reduce part of this manual design, but their supervision often relies on occupancy-derived labels or heuristic cost values, which may not capture ego-conditioned planning risk. In this paper, we propose DRiF, a data-driven risk-field framework for safer end-to-end autonomous driving. DRiF learns a shared BEV feature with static map segmentation, dynamic risk prediction, and vehicle planning. For dynamic risk learning, DRiF converts rule-based safety priors into pairwise risk labels, and trains the risk field to preserve relative risk ordering instead of regressing handcrafted absolute scores. Experiments on Bench2Drive show that DRiF achieves competitive overall performance, with consistent improvements in driving score, success rate, and collision-related metrics. These results establish relative risk supervision as an effective way to connect explicit safety structure with end-to-end planning. The data and code will be publicly available.

Abstract

Autonomous Driving

Explore similar work

Data-Driven Risk Fields for Safer End-to-End Autonomous Driving

Risk-Aware Occupancy for Safety-Oriented End-to-End Autonomous Driving

Driving on Registers, Reasoning on Risk: Risk-Aware Occupancy for Register-Based End-to-End Autonomous Driving