GAPL: Grounded Action-effect Policy Learning for LLM-Based Trajectory Planning
Authors: Zhihong Cui, Hengyu Liu, Zhangkai Wu, Yushuai Li, Tianyi Li, Peiyuan Guan, Amir Taherkordi, Tor Skeie
Organizations: Department of Informatics, University of Oslo, Oslo, Norway · Department of Computer Science, Aalborg University, Aalborg, Denmark · University of Sydney, Sydney, Australia · School of Software, Nanjing University of Information Science and Technology, Nanjing, China · Simula Research Laboratory, Oslo, Norway
Trajectory planning for autonomous driving requires both high-level reasoning and precise low-level control. Large Language Models (LLMs) offer semantic-rich planning capabilities, however, their application is limited by hallucinated reasoning, poor grounding in environment dynamics, and limited numerical precision in control. We propose GAPL (Grounded Action-effect Policy Learning), a unified framework that integrates LLM-based effect estimation, simulation-based effect grounding, and policy optimization into a closed-loop system. GAPL consists of three modules: (1) an LLM-based Effect Evaluator for structured multi-dimensional action-effect estimation; (2) a Simulation-based Effect Grounder that predicts dynamics-consistent effects from simulator rollouts; and (3) an Effect-Aware Decision Maker that grounds LLM effect estimates against simulation via a distiller to guide Proximal Policy Optimization (PPO)-based policy learning. Experiments on four Highway-env scenarios demonstrate that GAPL consistently outperforms baselines, achieving average reductions of {0.76, 0.86, 2.00} in collision rate, average displacement error (ADE), and final displacement error (FDE), and an average reward gain of 1.44.
Figures & tables
Fig. 1: Limitations of LLM-based Trajectory Planning
Fig. 2: The overview of GAPL framework
Scenario
Metrics
SayCan
Reflexion
CaDet
FUSION
Plan-R1
Cartographer
LLMCWM
GAPL
p-value
Hig.
CR ( ↓ ,%)
9.36 ± 1.24
8.89 ± 1.11
6.72 ± 1.05
4.18 ± 0.21
3.94 ± 0.29
3.62 ± 0.26
3.11 ± 0.32
2.18 ± 0.11
1.13 e -3
ADE ( ↓ )
10.52 ± 2.31
9.63 ± 2.07
10.26 ± 2.42
8.83 ± 2.51
8.57 ± 1.93
7.23 ± 1.64
7.04 ± 1.51
6.04 ± 0.89
1.47 e -3
FDE ( ↓ )
15.23 ± 3.02
13.45 ± 2.73
14.15 ± 3.12
12.08 ± 3.23
11.52 ± 2.85
10.86 ± 2.6
9.61 ± 2.33
6.45 ± 1.04
1.22 e -4
Reward ( ↑ )
6.28 ± 1.81
6.93 ± 2.07
8.45 ± 2.45
10.83 ± 2.11
11.57 ± 2.09
12.59 ± 1.83
13.61 ± 2.32
16.78 ± 1.61
1.15 e -4
Mer.
CR ( ↓ ,%)
9.52 ± 1.21
9.13 ± 1.17
7.45 ± 0.95
4.97 ± 0.67
4.57 ± 0.59
4.12 ± 0.53
3.58 ± 0.52
2.72 ± 0.38
3.16 e -3
ADE ( ↓ )
9.18 ± 1.82
8.79 ± 1.50
9.33 ± 2.07
7.91 ± 2.16
8.08 ± 1.45
7.46 ± 1.33
7.22 ± 1.13
6.65 ± 1.01
5.53 e -3
TABLE I: Performance Comparison on Four Driving Scenarios Compared With Baselines (mean ± std).
Metric
w/o LLM
w/o SIM
w/o Distiller
All
Hig.
ADE ( ↓ )
7.63 ± 2.11
9.10 ± 2.58
6.42 ± 1.93
6.04 ± 0.89
FDE ( ↓ )
8.14 ± 2.30
9.98 ± 2.79
6.81 ± 2.15
6.45 ± 1.04
Reward ( ↑ )
13.99 ± 1.73
10.84 ± 1.99
15.80 ± 1.69
16.78 ± 1.61
Mer.
ADE ( ↓ )
8.24 ± 1.51
9.23 ± 1.70
7.22 ± 1.29
6.65 ± 1.01
FDE ( ↓ )
10.33 ± 2.37
12.08 ± 2.59
9.01 ± 2.21
7.86 ± 1.10
Reward ( ↑ )
5.61 ± 0.72
4.56 ± 0.78
6.22 ± 0.70
7.55 ± 0.68
TABLE II: Ablation Study Across Four Driving Scenarios.
Fig. 3: Planning performance under different rollout length
Data size
ADE ( ↓ )
FDE ( ↓ )
Reward ( ↑ )
0
13.88 ± 1.47
14.85 ± 2.31
3.11 ± 0.41
8k
12.41 ± 1.39
14.31 ± 2.22
4.03 ± 0.36
12k
11.01 ± 1.33
13.92 ± 2.14
4.54 ± 0.31
18k
10.85 ± 1.27
13.45 ± 2.10
5.67 ± 0.29
24k
9.52 ± 0.98
11.23 ± 1.01
6.33 ± 0.46
TABLE III: Performance Under Different Data Sizes for Grounder Warm-Up in the Intersection Scenario.
Fig. 4: Case study of trajectory planning compared with baselines in intersection and roundabout scenarios
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Environment Configuration
Frequency
30
Lanes count
{2, 3, 4}
Vehicles density
{2, 3}
LLM Module
Model
GPT-4.1-nano / GPT-3.5-turbo
Action Space
DiscreteMetaAction (5 actions)
Appendix
TABLE IV: Implementation Details.
Fig. 5: The prompt of structural multi-dimensional effects.
SayCan
Reflexion
Plan-R1
Cartographer
LLMCWM
GAPL
Rou.
Time
756
1124
612
934
823
125
Calls
6.2
9.4
5.1
7.8
6.9
1.0
Int.
Time
893
1387
743
1089
967
138
Calls
7.4
11.6
6.2
9.1
8.1
1.0
Appendix
TABLE V: Computational Overhead Comparison. “Time” denotes system inference time per decision step (ms). “Calls” denotes the number of LLM API calls per step.
Fig. 6: Training curves of reward under varying weighting configurations across environments.
Model
ADE ( ↓ )
FDE ( ↓ )
Reward ( ↑ )
Rou.
GPT-3.5-turbo
7.78 ± 1.11
9.46 ± 1.19
6.29 ± 0.37
GPT-4.1-nano
7.22 ± 1.04
9.13 ± 1.06
6.84 ± 0.41
Int.
GPT-3.5-turbo
10.38 ± 1.22
12.36 ± 1.25
5.69 ± 0.77
GPT-4.1-nano
9.52 ± 0.98
11.23 ± 1.01
6.33 ± 0.46
Appendix
TABLE VI: Trajectory Planning Performance With Different LLM Backbones in the Roundabout and Intersection Scenarios.
Department of Information Management, Peking University, Beijing 100871, China · School of Intelligence Science and Technology, Peking University, Beijing 100871, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing 100080, China +3
the Department of Informatics, University of Oslo, Norway · the Department of Computing, Hong Kong Polytechnic University, Hong Kong, China · the Department of Computer Science, Aalborg University, Denmark
State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130000, China · Center of Research for Cyber Security and Network (CSNET), Faculty of Computer Science and Information Technology, Universiti Malaya, 50603 Kuala Lumpur, Malaysia · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg +3