Not All Agents Matter: From Global Attention Dilution to Risk-Prioritized Game Planning

Kang Ding; Hongsong Wang; Jie Gui; Lei He

cs.CVApr 7, 2026

Not All Agents Matter: From Global Attention Dilution to Risk-Prioritized Game Planning

Authors: Kang Ding, Hongsong Wang, Jie Gui, Lei He

Organizations: School of Cyberspace Security, Southeast University · School of Computer Science and Engineering, Southeast University · Vehicle and Mobility School, Tsinghua University

Abstract

End-to-end autonomous driving resides not in the integration of perception and planning, but rather in the dynamic multi-agent game within a unified representation space. Most existing end-to-end models treat all agents equally, hindering the decoupling of real collision threats from complex backgrounds. To address this issue, We introduce the concept of Risk-Prioritized Game Planning, and propose GameAD, a novel framework that models end-to-end autonomous driving as a risk-aware game problem. The GameAD integrates Risk-Aware Topology Anchoring, Strategic Payload Adapter, Minimax Risk-Aware Sparse Attention, and Risk Consistent Equilibrium Stabilization to enable game theoretic decision making with risk prioritized interactions. We also present the Planning Risk Exposure metric, which quantifies the cumulative risk intensity of planned trajectories over a long horizon for safe autonomous driving. Extensive experiments on the nuScenes and Bench2Drive datasets show that our approach significantly outperforms state-of-the-art methods, especially in terms of trajectory safety.

Figures & tables

Panel of Figure 1 — Figure 1: Comparison between our proposed end-to-end autonomous driving paradigm and previous approaches. (a) Previous approaches apply uniform interactions between each plan query and all agents during planning, implicitly assuming that all agents are equally important. (b) Our method computes a collision-risk matrix between each planning mode and agents, which directs attention toward potential conflict agents instead of being diluted by globally irrelevant targets.

Figure 2: The overall architecture of GameAD. After multi-scale feature extraction from multi-view images, the framework integrates four core components. Risk-Aware Topology Anchoring transfers road topology information unidirectionally into detection anchors, providing perception features enriched with risk-aware semantics. Strategic Payload Adapter subsequently combines candidate trajectories with perceived scene embeddings through trajectory-scene cross-attention, enabling risk-aware trajectory representation. During decision making, Minimax Risk-aware Sparse Attention(MRSA) computes the worst-case collision risk for each pair of ego planning modes and surrounding agents, guiding attention toward potential conflict participants. Finally, Risk-Consistent Equilibrium Stabilization connects historical and current planning spaces to produce temporally consistent multimodal trajectories.

Figure 3: The architecture of the Minimax Risk-aware Sparse Attention (MRSA) module. In the first stage, Risk Matrix Computation, the framework evaluates the worst-case geometric collision risk by analyzing the motion relationship between the ego planning trajectories and surrounding agents, thereby generating an ego-agent risk matrix. The second stage, Sparse Game Graph Construction, filters the risk matrix to identify a small subset of high-risk adversaries for each planning mode, which establishes a sparse game interaction structure. In the third stage, Risk-Aware Sparse Attention, the normalized risk values are injected into the attention mechanism as an adversarial prior. This process ensures that the planning queries concentrate on safety-critical agents and facilitates risk-aware interactive reasoning.

Method	Reference	Backbone	$\mathrm{L2}$ (m) $\downarrow$				Col. Rate (%) $\downarrow$				PRE (%) $\downarrow$				FPS $\uparrow$
Method	Reference	Backbone	1s	2s	3s	Avg.	1s	2s	3s	Avg.	1s	2s	3s	Avg.	FPS $\uparrow$
UniAD † [ 8 ]	CVPR2023	ResNet101	0.45	0.70	1.04	0.73	0.62	0.58	0.63	0.61	-	-	-	-	1.8 (A100)
VAD † [ 14 ]	ICCV2023	ResNet50	0.41	0.70	1.05	0.72	0.03	0.19	0.43	0.21	-	-	-	-	4.5 (RTX3090)
GenAD [ 42 ]	ECCV2024	ResNet50	0.36	0.83	1.55	0.91	0.06	0.23	1.00	0.43	-	-	-	-	6.7 (RTX3090)
SparseDrive † [ 33 ]	ICRA2025	ResNet50	0.30	0.58	0.96	0.61	0.01	0.05	0.23	0.10	4.35	4.50	4.61	4.49	7.4 (A100)
MomAD [ 31 ]	CVPR2025	ResNet50	0.31	0.57	0.91	0.60	0.01	0.05	0.22	0.09	4.33	4.41	4.52	4.42	6.3 (A100)

Table 1: Open-loop planning results on the nuScenes validation dataset. † denotes evaluation with the official checkpoint. PRE denotes Planning Risk Exposure metric. ∗ refers the re-implementation. As Ref. [ 18 ] states, we do not use the ego status information for a fair comparison.

Method	Open-loop Metric	Closed-loop Metric
Method	Avg. L2 $\downarrow$	DS $\uparrow$	SR(%) $\uparrow$	Effi $\uparrow$	Comf $\uparrow$
VAD [ 14 ]	0.91	42.35	15.00	157.94	46.01
VAD ${}_{\text{mmt}}$ [ 14 ]	0.89	42.87	15.91	158.12	47.22
GenAD [ 42 ]	-	44.81	15.90	-	-
SparseDrive [ 33 ]	0.87	44.54	16.71	170.21	48.63
MomAD [ 31 ]	0.82	47.91	18.11	174.91	51.20

Table 2: Open-loop and Closed-loop results on Bench2Drive(V0.0.3) under base training set. ‘mmt’ refers multi-modal trajectory variant of VAD.

Method	3D Object Detection		Multi-Object Tracking				Online Mapping				Motion Prediction
Method	mAP $\uparrow$	NDS $\uparrow$	AMOTA $\uparrow$	AMOTP $\downarrow$	Recall $\uparrow$	IDS $\downarrow$	mAP $\uparrow$	AP ${}_{\text{ped}}$ $\uparrow$	AP d $\uparrow$	AP b $\uparrow$	mADE $\downarrow$	mFDE $\downarrow$	MR $\downarrow$	EPA $\uparrow$
UniAD [ 8 ]	0.380	0.498	0.359	1.320	0.467	906	-	-	-	-	0.71	1.02	0.151	0.456
VAD † [ 14 ]	0.312	0.435	-	-	-	-	47.6	40.6	51.5	50.6	-	-	-	-
SparseDrive [ 33 ]	0.418	0.525	0.386	1.254	0.499	886	55.1	49.9	57.0	58.4	0.62	0.99	0.136	0.482
MomAD [ 31 ]	0.423	0.531	0.391	1.243	0.509	853	55.9	50.7	58.1	58.9	0.61	0.98	0.137	0.499
BridgeAD [ 39 ]	0.423	0.534	0.398	1.232	0.501	639	-	-	-	-	0.62	0.98	0.134	0.504

Table 3: Perception and motion results on the nuScenes validation dataset. † denotes evaluation with the official checkpoint. AP d denotes AP divider . AP b denotes AP boundary . mADE denotes minADE. mFDE denotes minFDE.

ID	Distance	$risk_{\beta}$	$t$	$\mathrm{L2}$ (m) $\downarrow$				Col. Rate (%) $\downarrow$				TPC (m) $\downarrow$
ID	Distance	$risk_{\beta}$	$t$	1s	2s	3s	Avg.	1s	2s	3s	Avg.	1s	2s	3s	Avg.
0	Euclidean	1.0	1	0.350	0.672	1.091	0.704	0.053	0.102	0.256	0.137	0.302	0.524	0.769	0.532
1	Hausdorff	1.0	1	0.341	0.661	1.082	0.695	0.049	0.098	0.251	0.133	0.296	0.519	0.763	0.526
2	Hausdorff	2.0	1	0.326	0.622	1.000	0.649	0.010	0.059	0.195	0.088	0.287	0.501	0.736	0.508
3	Hausdorff	4.0	1	0.331	0.654	1.082	0.689	0.059	0.088	0.247	0.131	0.287	0.507	0.753	0.516
5	Hausdorff	2.0	2	0.321	0.614	0.999	0.645	0.020	0.054	0.234	0.103	0.298	0.519	0.759	0.525

Table 4: Ablation studies of the risk game planning on the nuScenes validation split. Distance denotes the metric used to measure trajectory consistency.

risk_{\beta}

represents the factor that controls risk intensity, and

t

denotes the number of fused historical frames. TPC denotes Trajectory Prediction Consistency metric in [ 31 ] .

ID	RTA	MRSA	RCES	SPA	PRE (%) $\downarrow$				Coll. Rate (%) $\downarrow$				TPC (m) $\downarrow$
ID	RTA	MRSA	RCES	SPA	1s	2s	3s	Avg.	1s	2s	3s	Avg.	1s	2s	3s	Avg.
0	✓	✓	✓	✓	3.89	3.93	4.01	3.94	0.01	0.04	0.18	0.08	0.28	0.50	0.74	0.51
1		✓	✓	✓	3.92	3.97	4.05	3.98	0.01	0.07	0.22	0.10	0.28	0.51	0.75	0.51
2	✓		✓	✓	3.95	4.01	4.10	4.02	0.02	0.10	0.33	0.15	0.30	0.52	0.76	0.53
3	✓	✓		✓	3.90	3.95	4.02	3.96	0.02	0.08	0.26	0.12	0.31	0.54	0.77	0.54
4	✓	✓	✓		3.91	3.99	4.04	3.98	0.01	0.08	0.29	0.13	0.29	0.51	0.75	0.52

Table 5: Ablation studies of the core modules in GameAD on the nuScenes validation split. RTA represents Risk-Aware Topology Anchoring. SPA represents Strategic Payload Adapter. MRSA represents Minimax Risk-Aware Sparse Attention. RCES represents Risk-Consistent Equilibrium Stabilization.

ID	Method	3D Object Detection		Multi-Object Tracking				Online Mapping
ID	Method	mAP $\uparrow$	NDS $\uparrow$	AMOTA $\uparrow$	AMOTP $\downarrow$	Recall $\uparrow$	IDS $\downarrow$	mAP $\uparrow$	AP ${}_{\text{ped}}$ $\uparrow$	AP d $\uparrow$	AP b $\uparrow$
0	perception	0.418	0.525	0.386	1.254	0.499	886	55.1	49.9	57.0	58.4
1	perception $+$ instance robust	0.420	0.527	0.390	1.248	0.505	864	55.2	50.1	57.3	58.2
2	perception $+$ motion2det	0.421	0.537	0.396	1.234	0.502	574	54.2	52.1	54.3	56.2
3	perception $+$ uni. calibration	0.425	0.536	0.399	1.224	0.504	566	57.1	53.6	59.1	58.7
4	perception $+$ bid. calibration	0.411	0.516	0.371	1.273	0.453	1022	51.4	49.1	52.9	52.3

Table 6: Ablation studies of perception. AP d denotes AP divider . AP b denotes AP boundary . ‘perception’ follows the setting of SparseDrive [ 33 ] . ‘instance robust’ is re-implemented based on MomAD [ 31 ] . ‘motion2det’ is re-implemented based on BridgeAD [ 39 ] . ‘uni. calibration’ indicates the flow from the map branch to the detection branch, while ‘bid. calibration’ indicates mutual influence between the two branches.

Figure 4: Visualization of attention heatmaps for the comparison between GameAD and SparseDrive in multi-agent interaction. GameAD exhibits sparse and mode-differentiated interaction structures compared to the dispersed distribution in SparseDrive.

Figure 5: Qualitative comparison of GameAD with SparseDrive and BridgeAD. Our GameAD achieves better performance in both perception and planning.

End-to-End Autonomous Driving

End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.

See all 225 papers

Can Attribution Predict Risk? From Multi-View Attribution to Planning Risk Signals in End-to-End Autonomous Driving
May 7, 2026Le Yang, Ruoyu Chen, Haijun Liu +3Feature Attribution End-to-End Autonomous Driving
1Sun Yat-sen University · University of Chinese Academy of Sciences · 3Nanyang Technological University
Data-Driven Risk Fields for Safer End-to-End Autonomous Driving
Sep 9, 2026Yuanxin Tian, Zhiyuan Liu, Jinhao Li +9End-to-End Autonomous Driving Safe Motion Planning
Tsinghua University · Yinwang · KAIST
Driving is a Game: Combining Planning and Prediction with Bayesian Iterative Best Response
Dec 3, 2025Aron Distelzweig, Yiwei Wang, Faris Janjoš +5Autonomous Driving Planning Multi-Agent Planning
University of Freiburg, Germany · Bosch Center for Artificial Intelligence, Germany · Technical University of Munich, Germany +2

May 7, 2026J/K move · Enter open · S save

Can Attribution Predict Risk? From Multi-View Attribution to Planning Risk Signals in End-to-End Autonomous Driving

Le Yang, Ruoyu Chen, Haijun Liu, Jiawei Liang+2

1Sun Yat-sen University · University of Chinese Academy of Sciences · 3Nanyang Technological University

Feature Attribution End-to-End Autonomous Driving Autonomous Driving Planning

Open paper AskPDF arXiv

End-to-end autonomous driving models generate future trajectories from multi-view inputs, improving system integration but introducing opaque decisions and hard-to-localize risks. Existing methods either rely on auxiliary monitoring models or generate textual explanations, but are decoupled from the planning process and fail to reveal the visual evidence underlying trajectory generation. While attribution offers a direct alternative, planning differs from image classification by taking six-view camera images as input and predicting continuous multi-step trajectories, requiring attribution to capture both critical views and regions and their influence on outputs. Moreover, whether attribution maps can support risk identification remains underexplored. To address this, we propose a hierarchical attribution framework for end-to-end planning. Specifically, using L2 consistency with the original trajectory as the objective, we design a coarse-to-fine region attribution strategy that searches candidate regions across the full six-view input and refines attribution within them. We further extract three attribution statistics as predictive signals for planning risk, including attribution entropy to measure how concentrated the planner's reliance is over the joint visual space, within-camera spatial variance to characterize how spread out the attribution is within each view, and cross-camera Gini coefficient to quantify how unevenly attribution is distributed across the six cameras. Experiments on BridgeAD, UniAD, and GenAD show that these statistics correlate with planning risk, achieving Spearman correlations of

0.30 \pm 0.07

with trajectory error and AUROC of

0.77 \pm 0.04

for collision detection. The signal generalizes to held-out scenes with negligible degradation and remains stable under an alternative attribution baseline.

Abstract

End-to-End Autonomous Driving

Explore similar work

Can Attribution Predict Risk? From Multi-View Attribution to Planning Risk Signals in End-to-End Autonomous Driving

Data-Driven Risk Fields for Safer End-to-End Autonomous Driving

Driving is a Game: Combining Planning and Prediction with Bayesian Iterative Best Response