Learning to Tune Pure Pursuit in Autonomous Racing: Joint Lookahead and Steering-Gain Control with PPO

Mohamed Elgouhary; Amr S. El-Wakeel

cs.ROFeb 20, 2026

Learning to Tune Pure Pursuit in Autonomous Racing: Joint Lookahead and Steering-Gain Control with PPO

Authors: Mohamed Elgouhary, Amr S. El-Wakeel

Organizations: The authors are with the Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA

Abstract

Pure Pursuit (PP) is widely used in autonomous racing for real-time path tracking due to its efficiency and geometric clarity, yet performance is highly sensitive to how key parameters-lookahead distance and steering gain-are chosen. Standard velocity-based schedules adjust these only approximately and often fail to transfer across tracks and speed profiles. We propose a reinforcement-learning (RL) approach that jointly chooses the lookahead Ld and a steering gain g online using Proximal Policy Optimization (PPO). The policy observes compact state features (speed and curvature taps) and outputs (Ld, g) at each control step. Trained in F1TENTH Gym and deployed in a ROS 2 stack, the policy drives PP directly (with light smoothing) and requires no per-map retuning. Across simulation and real-car tests, the proposed RL-PP controller that jointly selects (Ld, g) consistently outperforms fixed-lookahead PP, velocity-scheduled adaptive PP, and an RL lookahead-only variant, and it also exceeds a kinematic MPC raceline tracker under our evaluated settings in lap time, path-tracking accuracy, and steering smoothness, demonstrating that policy-guided parameter tuning can reliably improve classical geometry-based control.

Figures & tables

Fig. 1: Pipeline: mapping and minimum-curvature raceline feed the simulator and a baseline PP controller. A PPO agent learns $(L_{d},g)$ and integrates with PP via ROS topics. We validate in simulation and deploy on a real F1TENTH car.

Term	Weight
$w_{v}$	1.8
$w_{L}$	3.0
$w_{G}$	0
$w_{jL}$	0.4
$w_{jG}$	0
$w_{k}$	1.5

TABLE I: Reward weights used for PPO training.

Panel of Figure 3 — Fig. 2: PPO diagnostics (1/2): update stability metrics. Approx. KL and clip fraction remain small after the initial transient, indicating conservative policy updates.

Panel of Figure 4 — Fig. 3: PPO diagnostics (2/2): exploration and critic learning. The policy action standard deviation decreases as exploration reduces, while the critic loss stabilizes as value estimates improve.

Panel of Figure 5 — Fig. 4: Classical Pure Pursuit failure modes with fixed lookahead.

Panel of Figure 6 — Fig. 5: Simulator maps used in our study: (a) Hockenheim for training, (b) Montreal, and (c) YasMarina for evaluation.

Panel of Figure 7 — Fig. 6: Qualitative comparison in highlighted regions on two unseen tracks. For each track, the full-layout plot (top) indicates the zoom window shown below. The zoom overlays the reference raceline and all five controllers. The learned RL–PP policies follow the reference closely while remaining within the track boundaries, whereas Fixed PP and Adaptive PP (linear $v\!\mapsto\!L_{d}$ ) show larger deviations in these regions. The kinematic MPC tracker provides a model-based comparison under the same reference and constraints. Colors follow the legend.

Fig. 7: Real-car tracking comparison using a shared raceline speed profile capped at

v_{\max}=6~\mathrm{m/s}

: overlaid trajectories for all five controllers against the same reference raceline. Using the same capped profile emphasizes differences in lateral tracking and stability while keeping longitudinal commands comparable across controllers.

Panel of Figure 9 — Fig. 8: Policy interpretability on Montreal. (a) Parameter schedule versus curvature and speed along the raceline. (b) Along-lap evolution of $L_{d}$ and $g$ plotted against waypoint index, with normalized $\kappa_{\max}$ (dashed) highlighting high-curvature segments.

Controller	Mean	Std	Min	Max
RL–PP (joint $L_{d},g$ , $+16\%$ )	32.85	$0.24$	$32.18$	$32.88$
RL–PP ( $L_{d}$ only, $+13\%$ )	$33.23$	$0.13$	$33.05$	$33.49$
Adaptive PP (linear $v\!\mapsto\!L_{d}$ , $+10\%$ )	$34.25$	$0.04$	$34.18$	$34.34$
Fixed PP ( $L_{d}$ fixed, $-10\%$ )	$41.35$	$0.09$	$41.24$	$41.57$
MPC raceline tracker ( $-15\%$ )	$48.06$	$0.08$	$47.96$	$48.25$

TABLE II: Montreal F1TENTH racetrack: lap times over 10 consecutive laps. For the 10/10 completion objective, each controller uses the best speed multiplier that yields 10/10 laps in our setup. Adaptive PP (linear) applies a linear

v\!\mapsto\!L_{d}

rule with bounds

L_{d}\in[1.0,2.5]~\mathrm{m}

Controller	Mean	Std	Min	Max
RL–PP (joint $L_{d},g$ , $+17.2\%$ )	45.10	$0.07$	$44.96$	$45.20$
RL–PP ( $L_{d}$ only, $+15\%$ )	$46.04$	$0.10$	$45.93$	$46.21$
Adaptive PP (linear $v\!\mapsto\!L_{d}$ , [+15]%)	$46.41$	$0.12$	$46.24$	$46.57$
Fixed PP ( $L_{d}$ fixed, orig.)	$52.97$	$0.21$	$52.79$	$53.44$
MPC raceline tracker ([-10]%)	$64.01$	$0.25$	$63.60$	$64.51$

TABLE III: Yas Marina F1TENTH racetrack: lap times over 10 consecutive laps. For the 10/10 completion objective, each controller uses the best speed multiplier that yields 10/10 laps in our setup. Adaptive PP (linear) applies a linear

v\!\mapsto\!L_{d}

rule with bounds

L_{d}\in[1.0,2.5]~\mathrm{m}

Controller	Mean	Std	Min	Max
RL–PP (joint $L_{d},g$ )	9.46	0.23	9.09	9.82
RL–PP ( $L_{d}$ only)	9.61	0.58	8.94	10.51
Adaptive PP (linear $v\!\mapsto\!L_{d}$ )	9.72	0.27	9.34	10.40
Fixed PP ( $L_{d}$ fixed)	9.85	0.43	9.32	10.55
MPC raceline tracker	15.42	0.47	14.48	16.17

TABLE IV: Real-car lap times using a raceline speed profile uniformly capped at

v_{\max}=6~\mathrm{m/s}

Explore similar work

Dynamic Lookahead Distance via Reinforcement Learning-Based Pure Pursuit for Autonomous Racing
Mar 30, 2026Mohamed Elgouhary, Amr S. El-WakeelAutonomous Racing RL Control
Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA.
MAP2: Model- and Acceleration-Based Pursuit with MPC and Gaussian Process Residual Learning for Autonomous Racing
Oct 8, 2026Weizhan Huang, Cheng Hu, Edoardo Ghignone +5Autonomous Racing Vehicle Dynamics
Zhejiang University · Center for Project-Based Learning, D-ITET, ETH Zurich
Efficient Real-World Autonomous Racing via Attenuated Residual Policy Optimization
Mar 13, 2026Raphael Trumpp, Denis Hoornaert, Mirco Theile +1Residual Policy Learning Autonomous Racing
Technical University of Munich, Munich, Germany

Mar 30, 2026J/K move · Enter open · S save

Dynamic Lookahead Distance via Reinforcement Learning-Based Pure Pursuit for Autonomous Racing

Mohamed Elgouhary, Amr S. El-Wakeel

Lane Department of Computer Science and Electrical Engineering, West Virginia University, Morgantown, WV, USA.

Autonomous Racing RL Control Feedback Control

Open paper AskPDF arXiv

Pure Pursuit (PP) is a widely used path-tracking algorithm in autonomous vehicles due to its simplicity and real-time performance. However, its effectiveness is sensitive to the choice of lookahead distance: shorter values improve cornering but can cause instability on straights, while longer values improve smoothness but reduce accuracy in curves. We propose a hybrid control framework that integrates Proximal Policy Optimization (PPO) with the classical Pure Pursuit controller to adjust the lookahead distance dynamically during racing. The PPO agent maps vehicle speed and multi-horizon curvature features to an online lookahead command. It is trained using Stable-Baselines3 in the F1TENTH Gym simulator with a KL penalty and learning-rate decay for stability, then deployed in a ROS2 environment to guide the controller. Experiments in simulation compare the proposed method against both fixed-lookahead Pure Pursuit and an adaptive Pure Pursuit baseline. Additional real-car experiments compare the learned controller against a fixed-lookahead Pure Pursuit controller. Results show that the learned policy improves lap-time performance and repeated lap completion on unseen tracks, while also transferring zero-shot to hardware. The learned controller adapts the lookahead by increasing it on straights and reducing it in curves, demonstrating effectiveness in augmenting a classical controller by online adaptation of a single interpretable parameter. On unseen tracks, the proposed method achieved 33.16 s on Montreal and 46.05 s on Yas Marina, while tolerating more aggressive speed-profile scaling than the baselines and achieving the best lap times among the tested settings. Initial real-car experiments further support sim-to-real transfer on a 1:10-scale autonomous racing platform

Abstract

Explore similar work

Dynamic Lookahead Distance via Reinforcement Learning-Based Pure Pursuit for Autonomous Racing

MAP2: Model- and Acceleration-Based Pursuit with MPC and Gaussian Process Residual Learning for Autonomous Racing

Efficient Real-World Autonomous Racing via Attenuated Residual Policy Optimization