Gradient-Based Data Valuation Improves Curriculum Learning for Game-Theoretic Motion Planning

Shihao Li; Jiachen Li; Dongmei Chen

cs.LGApr 1, 2026

Gradient-Based Data Valuation Improves Curriculum Learning for Game-Theoretic Motion Planning

Authors: Shihao Li, Jiachen Li, Dongmei Chen

Organizations: The University of Texas at Austin

Abstract

We demonstrate that gradient-based data valuation produces curriculum orderings that significantly outperform metadata-based heuristics for training game-theoretic motion planners. Specifically, we apply TracIn gradient-similarity scoring to GameFormer on the nuPlan benchmark and construct a curriculum that weights training scenarios by their estimated contribution to validation loss reduction. Across three random seeds, the TracIn-weighted curriculum achieves a mean planning ADE of $1.704\pm0.029$ ,m, significantly outperforming the metadata-based interaction-difficulty curriculum ( $1.822\pm0.014$ ,m; paired $t$ -test $p=0.021$ , Cohen's $d_z=3.88$ ) while exhibiting lower variance than the uniform baseline ( $1.772\pm0.134$ ,m). Our analysis reveals that TracIn scores and scenario metadata are nearly orthogonal (Spearman $ρ=-0.014$ ), indicating that gradient-based valuation captures training dynamics invisible to hand-crafted features. We further show that gradient-based curriculum weighting succeeds where hard data selection fails: TracIn-curated 20% subsets degrade performance by $2\times$ , whereas full-data curriculum weighting with the same scores yields the best results. These findings establish gradient-based data valuation as a practical tool for improving sample efficiency in game-theoretic planning.

Figures & tables

Fig. 1 : Method overview. Phase 1: Three scoring methods assign per-sample importance. Phase 2: A three-phase curriculum converts scores to weights; weighted training preserves coverage while hard selection fails. Phase 3: GameFormer trained with gradient-based curriculum achieves lower ADE ( $p{=}0.021$ ) and reduced variance.

Fig. 2 : Score correlation analysis. Left: Spearman rank correlation heatmap showing near-orthogonality of TracIn and metadata scores ( $\rho=-0.014$ ). Right: Scatter plot of TracIn vs. metadata scores for each training scenario, colored by scoring tier. The two scoring methods capture fundamentally different aspects of data utility.

Fig. 3 : Curriculum weight schedule. (a) Weight trajectories over training epochs for samples at different score levels. All samples start at $w=1$ during warm-up (epochs 1–3), ramp up during epochs 3–8, and stabilize at maximum differentiation. (b) Distribution of final weights ( $e=20$ ) across all 5,148 training scenarios, showing smooth differentiation centered around $w\approx 2$ .

Fig. 4 : Representative scenarios from four quadrants of the TracIn $\times$ metadata scoring space. Top-left: low metadata but high TracIn—few nearby agents, yet the ego executes a turn that strongly aligns with validation gradient. Bottom-right: high metadata but low TracIn—many close agents in parallel lanes, yet the gradient opposes validation improvement. The off-diagonal quadrants illustrate the orthogonality of the two scoring methods ( $\rho=-0.014$ ). Blue/red lines: ego past/future; gray/orange: agent past/future.

	TracIn	Meta	Loss	Hybrid
TracIn	1.000	$-$ 0.014	0.155	0.695
Meta	—	1.000	$-$ 0.033	0.689
Loss	—	—	1.000	0.087
Hybrid	—	—	—	1.000

TABLE I : Spearman rank correlations between scoring methods (

n=5{,}148

). TracIn and metadata are nearly orthogonal (

\rho=-0.014

Fig. 5 : Multi-seed planning ADE comparison ( $n{=}3$ seeds per method). Horizontal bars: mean; whiskers: $\pm 1$ s.d.; shaped markers: individual seeds. Y-axis is broken to accommodate the Loss SPL outlier (seed 2024, ADE $=2.555$ ). The TracIn curriculum achieves the lowest mean ADE with the tightest seed clustering. ${}^{*}p{=}0.021$ (paired $t$ -test, TracIn vs. Metadata).

Method	planADE $\downarrow$	planFDE $\downarrow$	planAHE $\downarrow$	predADE $\downarrow$	CV
Baseline	$1.772\pm.134$	$3.837\pm.218$	$.146\pm.021$	$1.700\pm.205$	7.6%
Meta cur.	$1.822\pm.014$	$3.996\pm.216$	$.142\pm.010$	$1.714\pm.016$	0.7%
TracIn cur.	$\mathbf{1.704\pm.029}$	$\mathbf{3.731\pm.394}$	$\mathbf{.133\pm.019}$	$1.731\pm.095$	1.7%
Loss SPL	$2.003\pm.391$	$3.678\pm.200$	$.180\pm.041$	$\mathbf{1.633\pm.069}$	19.5%
Hybrid cur.	$1.766\pm.069$	$3.999\pm.185$	$.134\pm.016$	$1.707\pm.047$	3.9%

TABLE II : Multi-seed results (mean

\pm

std,

n=3

seeds). Best mean values in bold .

\downarrow

= lower is better. CV = coefficient of variation.

Seed	Baseline	Meta cur.	TracIn cur.	Loss SPL	Hybrid cur.
3407	1.917	1.832	1.687	1.728	1.772
42	1.593	1.803	1.680	1.726	1.848
2024	1.807	1.831	1.746	2.555	1.680
Mean	1.772	1.822	1.704	2.003	1.766
Std	0.134	0.014	0.029	0.391	0.069

TABLE III : Per-seed planning ADE (m,

\downarrow

). TracIn outperforms Meta in all 3 seeds. Best per-seed in bold .

Comparison	$\Delta$ ADE (m)	$p$ -value	Cohen’s $d_{z}$
TracIn vs. Meta	$-0.117$	$0.021^{*}$	$3.88$
TracIn vs. Base	$-0.068$	$0.540$	$0.63$
TracIn vs. SPL	$-0.299$	$0.296$	$1.20$
TracIn vs. Hyb	$-0.062$	$0.186$	$1.71$
Hybrid vs. Meta	$-0.056$	$0.291$	$1.23$
Meta vs. Base	$+0.050$	$0.622$	$0.33$

TABLE IV : Paired

t

-tests (planning ADE,

n=3

seeds). Significant at

\alpha=0.05

marked with *.

Fig. 6 : Validation planning ADE over training epochs for the best seed of each method. The TracIn curriculum (blue) achieves the lowest final ADE, converging below all other methods by epoch 15.

Fig. 7 : Dot-and-whisker comparison across five curriculum methods and three planning metrics. Diamonds: mean; shaped markers: individual seeds ( $n{=}3$ ). Methods sorted by mean Planning ADE (lower is better). The TracIn curriculum achieves the best ADE with minimal seed-to-seed variation, while the loss-based SPL shows a large outlier.

Curriculum Learning

Curriculum learning organizes training examples, tasks, or environments to match how a model learns.

See all 173 papers

Predictive Bandit Curriculum Reinforcement Learning via Gradient-Based Sample Group Discovery
Oct 19, 2026Liang Xu, Shufan Shen, Zhongni Hou +5Curriculum RL Curriculum Learning
State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan
Training-Free Diffusion Planning with Analytical Local Scores
Oct 1, 2026Michael Y. Fatemi, Jinhao Liang, Ferdinando FiorettoMulti-Robot Motion Planning Constrained Motion Planning
University of Virginia
PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models
May 20, 2026Ziliang Zhao, Zenan Xu, Shuting Wang +7LLM Evaluation LLM Planning
Gaoling School of Artificial Intelligence, Renmin University of China · LLM Department, Hunyuan Team, Tencent · Beijing Academy of Artificial Intelligence +1

Oct 19, 2026J/K move · Enter open · S save

Predictive Bandit Curriculum Reinforcement Learning via Gradient-Based Sample Group Discovery

Liang Xu, Shufan Shen, Zhongni Hou, Chao Bi+4

State Key Lab. of AI Safety, Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Meituan

Curriculum RL Curriculum Learning RL Post-Training

Open paper AskPDF arXiv

Curriculum learning, which organizes training samples according to their learning priority, has been widely adopted in reinforcement learning to improve training efficiency. However, existing methods commonly construct curriculum units using predefined data properties and estimate their utility by simply aggregating a single proxy signal from recent training history.Such designs may group samples with different optimization effects and fail to anticipate how their utility evolves in relation to the policy. We propose Predictive Gradient-based Bandit (PGB) Curriculum, which first discovers optimization-aware learning units by clustering projected sample-level gradients. PGB then employs a lightweight temporal predictor to forecast the future utility of each unit and converts the predictions into a Boltzmann sampling distribution for adaptive data allocation. Experiments across multiple model scales and reasoning benchmarks show that PGB consistently outperforms standard GRPO, difficulty-based curricula, and history-based scheduling strategies, demonstrating improved data utilization and training efficiency.

Abstract

Curriculum Learning

Explore similar work

Predictive Bandit Curriculum Reinforcement Learning via Gradient-Based Sample Group Discovery

Training-Free Diffusion Planning with Analytical Local Scores

PlanningBench: Generating Scalable and Verifiable Planning Data for Evaluating and Training Large Language Models