GRID-Planner: Graph-Structured Relational Reward Shaping for Diffusion Planning

Zhao Yang; Zezhong Qian; Yinan Shi; Longjun Liu; Yawei Jueluo

cs.ROOct 15, 2026

GRID-Planner: Graph-Structured Relational Reward Shaping for Diffusion Planning

Authors: Zhao Yang, Zezhong Qian, Yinan Shi, Longjun Liu, Yawei Jueluo

Organizations: Xi’an Jiaotong University, Xi’an, China · Cytoderm Intelligent Technology, Beijing, China

Abstract

Autonomous driving planning requires selecting trajectories that remain safe, rule-compliant, and comfortable under complex multi-agent interactions, yet robust closed-loop decision making in planning-critical scenes remains challenging. Existing diffusion- and RL-based planners often rely on generic rewards and use relational structure mainly for representation, limiting how planning-critical rollouts shape policy improvement. We present GRID-Planner, a graph--reward integrated diffusion planning framework that aligns generative planning with structured relational signals. Specifically, GRID-Planner combines graph-aware importance weighting, a time-indexed relational reward surrogate, and an advantage-weighted diffusion objective with on-policy self-distillation, forming a rollout-return-advantage-update loop without explicit likelihoods. Experiments on nuPlan and interPlan show that, compared with the diffusion-only baseline, GRID-Planner reduces collision rate, lane deviation, and jerk by 35.8%, 60.8%, and 27.4%, respectively, and improves the planning score by 3.4%. Project website: https://github.com/yangzhaojason/GRID-Planner.

Figures & tables

Figure 1. Overview of GRID-Planner. Traditional diffusion planning may miss planning-critical relational semantics, whereas GRID-Planner combines Graph-Aware Importance Weighting and RRS to prioritize high-risk trajectories and evaluate safety, lane/route consistency, and comfort.

Figure 2. Overall framework of GRID-Planner. We address the mismatch between semantically flat imitation signals and rare planning-critical scenarios through two components, graph-aware importance weighting (GA-IW) and relational reward surrogate (RRS), which are unified in the final graph-reward integrated diffusion objective.

Figure 3. Illustration of the graph-aware weighting mechanism.

\mathrm{CentFeat}

is computed from a snapshot interaction graph at

t=0

and map-element counts, while

\mathrm{CollProb}

is estimated from future proximity and TTC. They are combined to form the importance weight

w(\tau)

in Eq. ( 5 ).

Figure 4. Illustration of the relational reward surrogate (RRS). Along the candidate trajectory

\tau

, a time-indexed scene graph

G_{t}

is queried at each step to evaluate safety, rule-compliance, and comfort terms. Selected steps are shown for illustration, and the per-step scores are aggregated over the horizon to form the dense trajectory-level reward

\mathcal{R}_{\mathrm{RRS}}(\tau)

Planner	Val14		Test14-hard		Test14
Planner	NR	R	NR	R	NR	R
PDM-Open* ( Dauner et al., 2023 )	53.53	54.24	33.51	35.83	52.81	57.23
UrbanDriver ( Scheel et al., 2022 )	68.57	64.11	50.40	49.95	51.83	67.15
GameFormer w/o refine. ( Huang et al., 2023b )	13.32	8.69	7.08	6.69	11.36	9.31
PlanTF ( Cheng et al., 2023 )	84.27	76.95	69.70	61.61	85.62	79.58
PLUTO w/o refine.* ( Cheng et al., 2024 )	88.89	78.11	70.03	59.74	89.90	78.62

Table 1. Closed-loop planning results of learning-based planners on the nuPlan benchmark under non-reactive (NR) and reactive (R) evaluation protocols. Flow Planner ( Tan et al., 2025 ) is reproduced locally using its official repository. The best learning-based result in each column is highlighted in bold.

*

denotes methods that directly leverage the road centerline extracted from the map.

Planner	Overall Score	Nudge Around	High Traffic Density	Jaywalk
PlanTF ( Cheng et al., 2023 )	47.70	49.40	58.85	33.94
Diffusion Planner ( Zheng et al., 2025 )	52.90	60.48	49.71	26.20
Flow Planner ( Tan et al., 2025 )	54.83	58.02	56.41	41.64
GRID-Planner ( Ours )	57.25	58.77	65.37	43.20

Table 2. Closed-loop planning results on the interPlan benchmark ( Hallgarten et al., 2024 ) . We report the overall score together with representative scenario-specific scores. The best scores in each column are highlighted in bold.

Figure 5. Baseline collides while ours remains safe. We visualize closed-loop rollouts in several challenging intersection scenarios. The baseline (top) results in collisions, while our method (bottom) navigates the same scenes. The ego vehicle is highlighted in yellow and collisions are shown in red.

Variant	Score $\uparrow$	Coll. $\downarrow$	Lane dev. $\downarrow$	Jerk $\downarrow$	Latency $\downarrow$
Diffusion only ( $\mathcal{L}_{\mathrm{diff}}$ )	79.90	0.0783	0.0268	0.1606	353.98
+ GAIW ( $w(\tau)$ )	80.92	0.0699	0.0201	0.1302	347.56
+ $R_{\mathrm{traj}}$ ( $R_{\mathrm{traj}}(\tau)=w(\tau)\mathcal{R}_{\mathrm{RRS}}(\tau)$ )	81.31	0.0604	0.0154	0.1153	343.72
+ On-policy actor–critic	81.92	0.0503	0.0147	0.1266	345.28
+ On-policy self-distillation	82.10	0.0501	0.0123	0.1191	341.59
Full model	82.59	0.0503	0.0105	0.1166	340.53

Table 3. Main ablation on the full benchmark suite. Score (

\uparrow

; nuPlan Val14 Benchmark), safety violations (

\downarrow

), comfort (jerk,

\downarrow

), and planning latency (

\downarrow

). All variants use the same training budget and evaluation protocol.

Variant	Score $\uparrow$	Coll. $\downarrow$	Lane $\downarrow$	Jerk $\downarrow$	Latency $\downarrow$
No weighting ( $w{=}1$ )	81.61	0.0568	0.0163	0.1440	332.14
Centrality only ( $w{=}\exp(\alpha_{1}\mathrm{CentFeat})$ )	81.74	0.0559	0.0161	0.1368	332.41
Risk only ( $w{=}\exp(\alpha_{2}\mathrm{CollProb})$ )	81.82	0.0546	0.0159	0.1384	332.53
Full w/o $w$ -clipping	81.71	0.0551	0.0159	0.1349	332.22
Full w/o mean-norm	81.65	0.0547	0.0158	0.1315	332.07
Full + stabilization	82.07	0.0519	0.0154	0.1252	332.41

Table 4. GAIW ablation. Effects of centrality, collision-risk, and stabilization in

w(\tau)

. We report Val14 score (

\uparrow

), collision rate (

\downarrow

), lane deviation (

\downarrow

), jerk (

\downarrow

), and latency (

\downarrow

). All variants share the same backbone and training budget; only the weighting strategy differs.

Figure 6. Training dynamics of GAIW. The full GAIW design with clipping and mean-normalization achieves higher closed-loop score (left) and lower collision rate (right) than variants without clipping or mean-normalization, indicating improved training stability in planning-critical scenarios.

Variant	Score $\uparrow$	Coll. $\downarrow$	Lane dev. $\downarrow$	Jerk $\downarrow$	Latency $\downarrow$
Full RRS ( $R_{\mathrm{safety}}+R_{\mathrm{rule}}+R_{\mathrm{comfort}}$ )	82.59	0.0503	0.0105	0.1166	340.53
w/o rule term ( $R_{\mathrm{safety}}+R_{\mathrm{comfort}}$ )	80.12	0.0568	0.0150	0.1325	344.87
w/o comfort term ( $R_{\mathrm{safety}}+R_{\mathrm{rule}}$ )	81.05	0.0527	0.0116	0.1538	345.92
w/o safety term ( $R_{\mathrm{rule}}+R_{\mathrm{comfort}}$ )	74.85	0.0708	0.0287	0.1786	348.21
Safety only ( $R_{\mathrm{safety}}$ )	79.62	0.0555	0.0248	0.1451	342.96
Rule only ( $R_{\mathrm{rule}}$ )	78.91	0.0613	0.0158	0.1546	340.82

Table 5. RRS ablation. Removing individual components of

\mathcal{R}_{\mathrm{RRS}}(\tau)

reveals their targeted impact on safety, rule consistency, and comfort. Score (

\uparrow

; nuPlan Val14 Benchmark), safety violations (

\downarrow

), comfort (jerk,

\downarrow

), and planning latency (

\downarrow

) are reported. All variants use the full training pipeline and differ only in the reward surrogate.

Variant	Score $\uparrow$	Coll. $\downarrow$	Lane dev. $\downarrow$	Jerk $\downarrow$	Latency $\downarrow$
Offline diffusion only ( $\mathcal{L}_{\mathrm{diff}}$ )	79.90	0.0783	0.0268	0.1606	353.98
Unified objective (Eq. 25 )	82.59	0.0503	0.0105	0.1166	340.53
w/o critic baseline ( $V_{\phi}(G)=0$ )	80.03	0.0754	0.0264	0.1558	345.26
w/o advantage reweighting	80.76	0.0564	0.0220	0.1315	344.41
w/o OPSD term	80.29	0.0699	0.0235	0.1321	344.46
w/o advantage gate ( $\omega(G)=1$ )	80.89	0.0559	0.0176	0.1272	340.53

Table 6. Unified on-policy objective ablation. Each row removes one ingredient from Eq. 25 while keeping the graph-aware return

R(\tau,G)

, rollout budget, and evaluation protocol fixed. We report Score (

\uparrow

; nuPlan Val14 Benchmark), together with collision rate (

\downarrow

), lane deviation (

\downarrow

), comfort measured by jerk (

\downarrow

), and latency (

\downarrow

). The last two rows ablate OPSD-specific choices (advantage gating and teacher selection).

Figure 7. Training dynamics of the unified on-policy objective. The full graph-reward integrated objective shows faster convergence and lower collision rate than offline diffusion and partial ablations, demonstrating better sample efficiency and more stable closed-loop adaptation.

Autonomous Driving Planning

Autonomous driving planning selects safe, feasible vehicle actions and trajectories amid traffic, uncertainty, road rules, and changing goals.

See all 384 papers

G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance
Jun 24, 2026Hang Yu, Ye Jin, Alessandro Canevaro +7Diffusion-Based Planning Motion Planning
Mercedes-Benz AG, Germany · TU Munich, Germany · University of Tübingen, Germany +1
RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework
Apr 16, 2026Hao Gao, Shaoyu Chen, Yifan Zhu +4Diffusion-Based Planning RL for Diffusion Models
Huazhong University of Science & Technology · Horizon Robotics
CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving
Mar 16, 2026Yihong Guo, Dongqiangzi Ye, Sijia Chen +2Autonomous Driving Planning Safe Motion Planning
Department of Computer Science, Johns Hopkins University, Baltimore, USA. · XPENG Motors.

Jun 24, 2026J/K move · Enter open · S save

G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance

Hang Yu, Ye Jin, Alessandro Canevaro, Julian Schmidt+6

Mercedes-Benz AG, Germany · TU Munich, Germany · University of Tübingen, Germany+1

Diffusion-Based Planning Motion Planning Autonomous Driving Planning Inference-Time Guidance

Open paper AskPDF arXiv

In autonomous driving, diffusion-based planners have emerged as a promising paradigm for robust motion planning in dense and interactive traffic, as they can effectively model diverse driving behaviors. However, their inherent stochasticity often requires explicit guidance during denoising to ensure safety and route adherence for robust closed-loop execution. Existing guidance typically relies on sparse, entity-centric geometric queries or post-hoc refinement, yielding limited situational awareness and fragile performance in interactive scenes. To address this issue, we propose G2DP (Grid-Guided Diffusion Planning), a diffusion-based planner that directly enforces dense environmental constraints through inference-time guidance. Specifically, G2DP constructs a differentiable spatio-temporal cost volume by fusing probabilistic future occupancy distributions with a route-progress map. By formulating this volume as a continuous safety energy functional, it injects dense gradients directly into the denoising loop, actively steering trajectory generation toward collision-free and progress-optimal regions. Extensive closed-loop evaluations show that G2DP achieves state-of-the-art performance on nuPlan, outperforming the strongest imitation-learning baseline by +7.2 points in reactive score. It further maintains top scores in zero-shot transfers to interPlan and DeepScenario benchmarks, with collision avoidance improving by +10.15 over the unguided approach on interPlan. These results demonstrate that spatio-temporal cost grids serve as an effective representation for robust guidance in diffusion-based planning.

Abstract

Autonomous Driving Planning

Explore similar work

G2DP: Diffusion Planning with Spatio-Temporal Grid Guidance

RAD-2: Scaling Reinforcement Learning in a Generator-Discriminator Framework

CorrectionPlanner: Self-Correction Planner with Reinforcement Learning in Autonomous Driving