Pure Pursuit (PP) is a widely used path-tracking algorithm in autonomous vehicles due to its simplicity and real-time performance. However, its effectiveness is sensitive to the choice of lookahead distance: shorter values improve cornering but can cause instability on straights, while longer values improve smoothness but reduce accuracy in curves. We propose a hybrid control framework that integrates Proximal Policy Optimization (PPO) with the classical Pure Pursuit controller to adjust the lookahead distance dynamically during racing. The PPO agent maps vehicle speed and multi-horizon curvature features to an online lookahead command. It is trained using Stable-Baselines3 in the F1TENTH Gym simulator with a KL penalty and learning-rate decay for stability, then deployed in a ROS2 environment to guide the controller. Experiments in simulation compare the proposed method against both fixed-lookahead Pure Pursuit and an adaptive Pure Pursuit baseline. Additional real-car experiments compare the learned controller against a fixed-lookahead Pure Pursuit controller. Results show that the learned policy improves lap-time performance and repeated lap completion on unseen tracks, while also transferring zero-shot to hardware. The learned controller adapts the lookahead by increasing it on straights and reducing it in curves, demonstrating effectiveness in augmenting a classical controller by online adaptation of a single interpretable parameter. On unseen tracks, the proposed method achieved 33.16 s on Montreal and 46.05 s on Yas Marina, while tolerating more aggressive speed-profile scaling than the baselines and achieving the best lap times among the tested settings. Initial real-car experiments further support sim-to-real transfer on a 1:10-scale autonomous racing platform
Figures & tables
Fig. 1: Training-and-deployment workflow. A map and minimum-curvature raceline are generated offline and used by the simulator environment and baseline Pure Pursuit controller during training. The PPO agent receives observations and reward from the environment and outputs a dynamic lookahead policy, which is then integrated with Pure Pursuit for simulation validation and real-world deployment on a 1:10-scale autonomous racing platform.
Fig. 2: Training diagnostics for PPO. Reward and episode length are computed on a fixed evaluation environment; explained variance and entropy are logged during training.
Controller
Mean
Std
Min
Max
RL PP ( +13% speed)
33.16
0.069
33.06
33.28
Adaptive PP ( +10% speed)
34.15
0.061
34.02
34.23
Fixed- L PP ( −10% speed)
41.36
0.067
41.25
41.45
TABLE I: Montreal track lap times over 10 consecutive laps (s).
Controller
Mean
Std
Min
Max
RL PP ( +15% speed)
46.05
0.035
46.00
46.10
Adaptive PP ( +14% speed)
46.79
0.095
46.67
46.79
Fixed- L PP (orig. speed)
52.81
0.18
52.52
53.10
TABLE II: Yas Marina track lap times over 10 consecutive laps (s).
Controller
Mean
Std
Min
Max
RL PP
11.13
0.26
10.79
11.66
Fixed- L PP
DNF / failed to complete repeated laps
TABLE III: Real-car lap times over 10 consecutive laps with a raceline speed profile capped at 5m/s (s).
Fig. 3: Qualitative real-car trajectory overlay. The learned Dynamic- L controller tracks the reference raceline and completes repeated laps on the mapped track under the tested speed profile.