GAPL: Grounded Action-effect Policy Learning for LLM-Based Trajectory Planning
Authors: Zhihong Cui, Hengyu Liu, Zhangkai Wu, Yushuai Li, Tianyi Li, Peiyuan Guan, Amir Taherkordi, Tor Skeie
Organizations: Department of Informatics, University of Oslo, Oslo, Norway · Department of Computer Science, Aalborg University, Aalborg, Denmark · University of Sydney, Sydney, Australia · School of Software, Nanjing University of Information Science and Technology, Nanjing, China · Simula Research Laboratory, Oslo, Norway
Trajectory planning for autonomous driving requires both high-level reasoning and precise low-level control. Large Language Models (LLMs) offer semantic-rich planning capabilities, however, their application is limited by hallucinated reasoning, poor grounding in environment dynamics, and limited numerical precision in control. We propose GAPL (Grounded Action-effect Policy Learning), a unified framework that integrates LLM-based effect estimation, simulation-based effect grounding, and policy optimization into a closed-loop system. GAPL consists of three modules: (1) an LLM-based Effect Evaluator for structured multi-dimensional action-effect estimation; (2) a Simulation-based Effect Grounder that predicts dynamics-consistent effects from simulator rollouts; and (3) an Effect-Aware Decision Maker that grounds LLM effect estimates against simulation via a distiller to guide Proximal Policy Optimization (PPO)-based policy learning. Experiments on four Highway-env scenarios demonstrate that GAPL consistently outperforms baselines, achieving average reductions of {0.76, 0.86, 2.00} in collision rate, average displacement error (ADE), and final displacement error (FDE), and an average reward gain of 1.44.
Figures & tables
Fig. 1: Limitations of LLM-based Trajectory Planning
Fig. 2: The overview of GAPL framework
Scenario
Metrics
SayCan
Reflexion
CaDet
FUSION
Plan-R1
Cartographer
LLMCWM
GAPL
p-value
Hig.
CR ( ↓ ,%)
9.36 ± 1.24
8.89 ± 1.11
6.72 ± 1.05
4.18 ± 0.21
3.94 ± 0.29
3.62 ± 0.26
3.11 ± 0.32
2.18 ± 0.11
1.13 e -3
ADE ( ↓ )
10.52 ± 2.31
9.63 ± 2.07
10.26 ± 2.42
8.83 ± 2.51
8.57 ± 1.93
7.23 ± 1.64
7.04 ± 1.51
6.04 ± 0.89
1.47 e -3
FDE ( ↓ )
15.23 ± 3.02
13.45 ± 2.73
14.15 ± 3.12
12.08 ± 3.23
11.52 ± 2.85
10.86 ± 2.6
9.61 ± 2.33
6.45 ± 1.04
1.22 e -4
Reward ( ↑ )
6.28 ± 1.81
6.93 ± 2.07
8.45 ± 2.45
10.83 ± 2.11
11.57 ± 2.09
12.59 ± 1.83
13.61 ± 2.32
16.78 ± 1.61
1.15 e -4
Mer.
CR ( ↓ ,%)
9.52 ± 1.21
9.13 ± 1.17
7.45 ± 0.95
4.97 ± 0.67
4.57 ± 0.59
4.12 ± 0.53
3.58 ± 0.52
2.72 ± 0.38
3.16 e -3
ADE ( ↓ )
9.18 ± 1.82
8.79 ± 1.50
9.33 ± 2.07
7.91 ± 2.16
8.08 ± 1.45
7.46 ± 1.33
7.22 ± 1.13
6.65 ± 1.01
5.53 e -3
TABLE I: Performance Comparison on Four Driving Scenarios Compared With Baselines (mean ± std).
Metric
w/o LLM
w/o SIM
w/o Distiller
All
Hig.
ADE ( ↓ )
7.63 ± 2.11
9.10 ± 2.58
6.42 ± 1.93
6.04 ± 0.89
FDE ( ↓ )
8.14 ± 2.30
9.98 ± 2.79
6.81 ± 2.15
6.45 ± 1.04
Reward ( ↑ )
13.99 ± 1.73
10.84 ± 1.99
15.80 ± 1.69
16.78 ± 1.61
Mer.
ADE ( ↓ )
8.24 ± 1.51
9.23 ± 1.70
7.22 ± 1.29
6.65 ± 1.01
FDE ( ↓ )
10.33 ± 2.37
12.08 ± 2.59
9.01 ± 2.21
7.86 ± 1.10
Reward ( ↑ )
5.61 ± 0.72
4.56 ± 0.78
6.22 ± 0.70
7.55 ± 0.68
TABLE II: Ablation Study Across Four Driving Scenarios.
Fig. 3: Planning performance under different rollout length
Data size
ADE ( ↓ )
FDE ( ↓ )
Reward ( ↑ )
0
13.88 ± 1.47
14.85 ± 2.31
3.11 ± 0.41
8k
12.41 ± 1.39
14.31 ± 2.22
4.03 ± 0.36
12k
11.01 ± 1.33
13.92 ± 2.14
4.54 ± 0.31
18k
10.85 ± 1.27
13.45 ± 2.10
5.67 ± 0.29
24k
9.52 ± 0.98
11.23 ± 1.01
6.33 ± 0.46
TABLE III: Performance Under Different Data Sizes for Grounder Warm-Up in the Intersection Scenario.
Fig. 4: Case study of trajectory planning compared with baselines in intersection and roundabout scenarios
Appendix figures & tables5 assets
Supplementary material from the paper’s appendix.
Appendix
Environment Configuration
Frequency
30
Lanes count
{2, 3, 4}
Vehicles density
{2, 3}
LLM Module
Model
GPT-4.1-nano / GPT-3.5-turbo
Action Space
DiscreteMetaAction (5 actions)
Appendix
TABLE IV: Implementation Details.
Fig. 5: The prompt of structural multi-dimensional effects.
SayCan
Reflexion
Plan-R1
Cartographer
LLMCWM
GAPL
Rou.
Time
756
1124
612
934
823
125
Calls
6.2
9.4
5.1
7.8
6.9
1.0
Int.
Time
893
1387
743
1089
967
138
Calls
7.4
11.6
6.2
9.1
8.1
1.0
Appendix
TABLE V: Computational Overhead Comparison. “Time” denotes system inference time per decision step (ms). “Calls” denotes the number of LLM API calls per step.
Fig. 6: Training curves of reward under varying weighting configurations across environments.
Model
ADE ( ↓ )
FDE ( ↓ )
Reward ( ↑ )
Rou.
GPT-3.5-turbo
7.78 ± 1.11
9.46 ± 1.19
6.29 ± 0.37
GPT-4.1-nano
7.22 ± 1.04
9.13 ± 1.06
6.84 ± 0.41
Int.
GPT-3.5-turbo
10.38 ± 1.22
12.36 ± 1.25
5.69 ± 0.77
GPT-4.1-nano
9.52 ± 0.98
11.23 ± 1.01
6.33 ± 0.46
Appendix
TABLE VI: Trajectory Planning Performance With Different LLM Backbones in the Roundabout and Intersection Scenarios.
Large language models (LLMs) are promising for autonomous driving, but semantics-only decision policies can yield physically unsafe behavior in dynamic traffic. Existing methods either perform online language reasoning without explicit dynamics verification or use world models mainly in offline pipelines, leaving a gap between semantic intent and physical feasibility at decision time. We propose Reason--Imagine--Act (RIA), a closed-loop framework that couples an LLM reasoner with an action-conditioned world model for online safety verification. At each step, the LLM proposes an action template and candidate sub-actions, the world model performs short-horizon rollouts, and a safety scorer selects the safest executable action with feedback to the next reasoning step. Under a unified CARLA point-goal protocol (1000 episodes), RIA achieves 80.05% route completion, 51.10% arrival rate, and 0.20% collision rate. Under the same closed-loop interface, RIA consistently outperforms training-free baselines, including CARLA TM and MADA, on core closed-loop metrics. For reproducibility, code is available at https://github.com/pku-smart-city/source_code/tree/main/RIA.
Zhengqi Sun, Yiwen Sun, Boxuan Liu +3
Department of Information Management, Peking University, Beijing 100871, China · School of Intelligence Science and Technology, Peking University, Beijing 100871, China · State Key Laboratory of General Artificial Intelligence, BIGAI, Beijing 100080, China +3
Trajectory planning for autonomous driving increasingly leverages large language models (LLMs) for commonsense reasoning, yet LLM outputs are inherently unreliable, posing risks in safety-critical applications. We propose C-TRAIL, a framework built on a Commonsense World that couples LLM-derived commonsense with a trust mechanism to guide trajectory planning. C-TRAIL operates through a closed-loop Recall, Plan, and Update cycle: the Recall module queries an LLM for semantic relations and quantifies their reliability via a dual-trust mechanism; the Plan module injects trust-weighted commonsense into Monte Carlo Tree Search (MCTS) through a Dirichlet trust policy; and the Update module adaptively refines trust scores and policy parameters from environmental feedback. Experiments on four simulated scenarios in Highway-env and two real-world levelXData datasets (highD, rounD) show that C-TRAIL consistently outperforms state-of-the-art baselines, reducing ADE by 40.2%, FDE by 51.7%, and improving SR by 16.9 percentage points on average. The source code is available at https://github.com/ZhihongCui/CTRAIL.
Zhihong Cui, Haoran Tang, Tianyi Li +4
the Department of Informatics, University of Oslo, Norway · the Department of Computing, Hong Kong Polytechnic University, Hong Kong, China · the Department of Computer Science, Aalborg University, Denmark
Autonomous driving planning is a key component of IoT-enabled intelligent transportation systems, requiring vehicles to generate safe, efficient, and executable trajectories in complex urban environments from multi-source contextual information. While imitation learning (IL) has shown promise on large-scale datasets, IL-based planners still suffer from limited coverage of complex long-tail interactions, weak consistency with downstream constrained refinement, and insufficient use of high level scene semantics under real time constraints. To address these issues, this paper proposes a large language model (LLM) enhanced differentiable trajectory planning framework for IoT-enabled autonomous driving. Specifically, we introduce a surrounding agent centric data augmentation strategy to reorganize sur rounding agent trajectories as additional planning supervision, thereby improving the training distribution without collecting additional raw data. We further design a complexity-aware asyn chronous LLM-based semantic enhancement module to extract scene-related high-level semantic features with controlled online overhead. In addition, a differentiable optimization module is incorporated to refine generated trajectories with explicit residual penalties while backpropagating optimization gradients to the upstream planner. Experiments show that the proposed method achieves the best overall scores of 83.63 and 78.29 on the nuPlan closed-loop nonreactive and reactive Hard20 benchmarks, respectively, and CARLA-ROS tests further verify its online deployment and real time closed-loop execution capability.
Shihao Zhang, Jing Yang, Ziyu Song +7
State Key Laboratory of Automotive Simulation and Control, Jilin University, Changchun 130000, China · Center of Research for Cyber Security and Network (CSNET), Faculty of Computer Science and Information Technology, Universiti Malaya, 50603 Kuala Lumpur, Malaysia · Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg, Luxembourg +3