cs.ROAug 25, 2026
SaveRoG-DAgger: Rollout-Guided Post-Training for End-to-End Driving
Organizations: CARIAD SE, Volkswagen Group
Abstract
Recent end-to-end driving systems demonstrate strong performance on closed-loop benchmarks, yet are still predominantly trained on fixed expert-collected data using open-loop imitation learning. This training-inference mismatch leaves the policy vulnerable in policy-induced states, where accumulated errors can lead to safety-critical failures. A promising post-training approach to overcome this issue is Dataset Aggregation (DAgger), which gathers expert demonstrations in policy-induced states and subsequently fine-tunes the policy on the resulting aggregated dataset. Existing driving DAgger pipelines, however, face three challenges: i) the expert is restricted to a limited trajectory-and-speed solution space, ii) takeover may occur too early or too late relative to impending failures, and iii) privileged expert decisions may rely on information unavailable to the student. To address this, we introduce RoG-DAgger, a post-training framework that uses short-horizon kinematic rollouts to construct high-quality expert demonstrations in safety-critical states. Specifically, RoG-DAgger expands the expert's trajectory-and-speed solution space and evaluates candidate plans through rollout to construct preventive supervision. Moreover, it uses rollout solvability to time the takeover near the estimated point of no return. Lastly, it aligns the expert's field of view with that of the student to provide student-compatible supervision. Across in-distribution (including long-horizon) and out-of-distribution evaluations, RoG-DAgger improves the end-to-end model SimLingo by 5.3 driving-score points and 6.2 percentage points in success rate on Bench2Drive, doubles its driving score from 22 to 44 on Longest6 v2, and improves out-of-distribution success rate from 55% to 66% on Fail2Drive.
Figures & tables
Figure 1: SimLingo pre-training data (orange) lacks safety-critical interactions as measured by time-to-collision (TTC), limiting the model’s ability to learn how to prevent or recover from such situations. As interactions, however, occur frequently in closed-loop Bench2Drive inference (blue), this constitutes a substantial distribution shift. Samples with TTC > 2s are omitted for clarity.
Figure 2: RoG-DAgger leverages kinematic rollout and introduces three improvements to existing DAgger methods. Panel A shows how rollouts are used to construct trajectory-and-speed supervision and to evaluate candidate expert plans. Panels B1-B3 highlight three modifications: (1) an expanded trajectory-and-speed solution space (red: collision-prone nominal plan, green: improved plan), (2) solvability-aware takeover near the estimated point of no return (yellow star: PONR, red region: no feasible corrective plan remains), and (3) expert-student field-of-view alignment (yellow triangle) for student-learnable supervision.
Figure 3: Overview of the RoG-DAgger training pipeline. Starting from a pretrained E2E policy, we collect post-training data in two independent closed-loop runs (see zoom-in panel) and fine-tune the policy using a mixture of pre- and post-training data. The infraction-based run captures policy-induced failures and expert recovery demonstrations, while the solvability-aware run intervenes before failures become unavoidable to collect corrective behavior (gray hatched bar indicates avoided infraction). A restart trigger is used in both runs but omitted for clarity.
Figure 4: Comparison between experts with fixed and expanded sets of geometric trajectories. In (a), the expert follows a fixed pre-computed route and attempts to avoid the predicted collision through emergency braking. However, due to vehicle dynamics, the ego vehicle cannot decelerate sufficiently before reaching the collision point. In (b), the expert additionally considers alternative geometric trajectories, which are evaluated through rollouts for collision-freeness before selection. Geometric trajectories (red dots in the images) correspond to the routes executed in simulation. TTC and distance are estimated using our rollout model.
| Method | Bench2Drive | Longest6 v2 | ||
| DS | SR | DS | RC | |
| Privileged methods | ||||
| PDM-Lite ( Zimmerlin et al. 2024 ) | 97.02 | 92.27 | 73 | 100 |
| RoG-Expert (ours) | 93.41 | 81.81 | 55 | 100 |
| Pre-trained-only methods | ||||
| UniAD ( Hu et al. 2023 ) | 45.81 | 16.36 | – | – |
Table 1: Closed-loop performance on Bench2Drive and Longest6 v2. DS, SR, and RS denote Driving Score, Success Rate, and Route Completion, respectively. RoG-DAgger achieves strong performance across both benchmarks. Dash indicates an unreported result.
| Method | Base | Generalization | ||||
| DS | SR | HM | DS | SR | HM | |
| UniAD | 47.5 | 36.3 | 41.2 | 44.0 | 27.6 | 33.9 |
| TF | 83.3 | 78.5 | 80.8 | 75.4 | 61.1 | 67.5 |
| ORION | 53.0 | 52.0 | 52.5 | 51.2 | 46.0 | 48.5 |
| HiP-AD | 74.1 | 70.7 | 72.4 | 67.1 | 56.7 | 61.5 |
| SimLingo | 82.6 | 79.3 | 80.9 | 71.7 | 55.0 | 62.2 |
Table 2: Closed-loop performance on Fail2Drive. Base denotes in-distribution evaluation, while Generalization refers to distribution-shifted scenarios. DS, SR, and HM denote Driving Score, Success Rate, and their harmonic mean, respectively. RoG-DAgger remains competitive on base scenarios while showing improved OOD generalization over SimLingo.
| Ablation | Bench2Drive | ||||
| Traj.- speed | Solv.- aware | FoV alignment | DS | SR | |
| Component | 88.71 | 68.94 | |||
| 88.65 | 69.86 | ||||
| 88.96 | 69.86 | ||||
| Takeover data | Post-takeover only | 88.05 | 68.04 | ||
| PDM-Lite DAgger baseline | 87.85 | 66.66 | |||
Table 3: Ablation study on Bench2Drive. We evaluate the effects of trajectory-and-speed space expansion, solvability-aware triggering, FoV alignment, takeover data selection and expert model.
Figure 5: Qualitative comparison of RoG-DAgger with SimLingo on a Bench2Drive scene. At , both policies approach a junction while a cross-traffic fire truck is already visible. Both policies continue to accelerate, likely because the pre-training data contains primarily successful junction traversals and lacks corrective supervision for such hazardous interactions. As a result, SimLingo keeps accelerating at and collides with the fire truck at . In contrast, after RoG-DAgger post-training, the policy slows down at and shifts to the left, avoiding the collision at . Red dots denote the predicted geometric trajectory, while green dots denote the predicted temporal trajectory. “thr”: throttle, “brk”: brake.
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | DS | SR (%) |
| SimLingo ( Renz et al. 2025 ) | 85.07 | 67.27 |
| SimLingo-Qwen3-VL-2B ( ours ) | 86.59 | 66.36 |
Table 4: Comparison between the original SimLingo and our Qwen3-VL-2B-based version. Both variants achieve comparable driving performance despite the change in VLM backbone. DS and SR denote driving score and success rate, respectively.
| Failure Category | Count | Proportion (%) |
| Total recorded failures | 73 | – |
| Ignored cases | 5 | – |
| Analyzed failures | 68 | 100.0 |
| Collision-related failures | 53 | 77.9 |
| Collision with vehicle | 49 | 72.1 |
| Collision with static | 4 | 5.9 |
Table 5: Breakdown of the recorded failures. After excluding five Yield To Emergency Vehicle cases, we analyze 68 failures, of which 53 are collision-related. The collision subcategories are not mutually exclusive, as one failure involves both a vehicle and a static object.
| Split | Window | Samples | Collision | Rate |
| Pre-takeover | 20 | 5568 | 2404 | 43.18% |
| Pre-takeover | 40 | 9565 | 2829 | 29.58% |
| Post-takeover | 40 | 20845 | 99 | 0.47% |
Table 6: Collision statistics for pre- and post-takeover samples. For pre-takeover samples, we compare temporal windows of 20 and 40 frames preceding the trigger, while the post-takeover window is 40. Collision rate denotes the fraction of retained samples for which our rollout model predicts a collision within the corresponding window when executing the trajectory-and-speed plan provided by PDM-Lite.
| Trigger | Trigger Condition |
| Infraction | A standard CARLA infraction, including a collision, red-light violation, or out-of-lane violation. |
| Solvability-Aware Takeover | Replan Success: replanning succeeds. Repeated Replan Success: replanning succeeds in two of four consecutive frames. Replan Failure: replanning fails. |
| Restart | A persistent discrepancy is detected between the ego vehicle and the expert model: the ego speed remains below for all of the previous 200 frames, while both the mean expert target speed over these frames and the current expert target speed exceed . |
Table 7: Details on the three types of trigger conditions used for post-training data collection. R-TTC denotes the time-to-collision estimated by our rollout model .
Figure 6: Expert without expanded trajectory-and-speed solution space fails to prevent collision. An expert such as PDM-Lite follows a precomputed route and primarily responds to predicted collisions with full braking along the nominal path. However, in safety-critical states, especially those induced by the policy, braking alone does not necessarily prevent an impending collision. At , the ego vehicle controlled by expert follows the nominal route and prepares for lane change. At , the expert predicts a collision and initiates full braking, but the intervention is insufficient due to the vehicle dynamics, and a collision occurs at . The geometric trajectories shown as red dots in the lower row correspond to the trajectories illustrated in the upper row.
Figure 8: SimLingo with solvability-aware takeover. At , the policy controls the ego vehicle while preparing for a left turn. As oncoming traffic approaches, our kinematic rollout model predicts at a time-to-collision (TTC) of , indicating an imminent collision. This activates the solvability-aware takeover, which shifts the geometric trajectory to the right while setting the target speed to 50% of the current speed. At , the RoG-Expert applies a second, larger correction of to the right, again setting the target speed to 50% of the current speed, successfully avoiding the collision. At , the ego vehicle continues safely under expert control during the post-takeover period.
| (a) Takeover Phase | |
| Pre-takeover | 27.3% |
| Post-takeover | 72.7% |
| (b) Collection Trigger / Event Type | |
| Infraction | 55.3% |
| Collision | 47.7% |
| Red-Light Infraction | 2.9% |
Table 8: Composition of the post-training dataset from two different perspectives: (a) takeover phase and (b) collection trigger and event type. Percentages are normalized separately within each panel. Bold rows in (b) denote high-level trigger classes, while indented rows show their event-type breakdown .
| Scenario Class | SimLingo | RoG-DAgger | |
| Visual-longitudinal | 71.1 | 81.2 | +10.1 |
| Visual-lateral | 45.9 | 51.3 | +5.4 |
| Behavior | 31.2 | 62.0 | +30.8 |
| Robustness | 86.8 | 80.7 | -6.1 |
| Mean | 58.8 | 68.8 | +10.0 |
Table 9: Per-class harmonic mean of DS and SR on generalization scenarios of Fail2Drive. RoG-DAgger improves over SimLingo on three out of four scenario classes, with the largest gain on behavior generalization (+30.8 HM), while SimLingo performs better on robustness scenarios.
| Method | Efficiency | Comfort | Multi-Capability (%) | |||||
| Merging | Overtaking | Emergency Brake | Give Way | Traffic Sign | Mean | |||
| Privileged methods | ||||||||
| PDM-Lite ( Zimmerlin et al. 2024 ) | 213.47 | 22.33 | 82.05 | 91.11 | 81.67 | 80.00 | 70.97 | 81.16 |
| RoG-Expert (ours) | 196.89 | 14.13 | 75.00 | 84.44 | 93.33 | 40.00 | 88.95 | 76.35 |
| Pre-trained-only methods | ||||||||
| UniAD ( Hu et al. 2023 ) | 129.21 | 43.58 | 12.16 | 20.00 | 23.64 | 10.00 | 13.89 | 15.89 |
Table 10: Detailed Bench2Drive evaluation of efficiency, comfort, and multi-ability performance. RoG-DAgger improves over SimLingo in merging, overtaking, and traffic-sign handling, and achieves a higher overall mean. It performs comparably to TakeVLA overall, with stronger merging but lower emergency-braking performance. Beyond these scenario-specific capabilities, we further assess whether the observed gains preserve desirable overall driving behavior in terms of efficiency and comfort. RoG-DAgger maintains competitive efficiency while improving comfort over both SimLingo and TakeVLA, suggesting that its closed-loop gains do not come at the cost of less comfortable driving.
Figure 9: Head-to-head comparison of RoG-DAgger (bottom) and SimLingo (top) in the Construction Pedestrian scenario of Fail2Drive. SimLingo tends to steer left despite oncoming traffic approaching from the left while the right side of the construction site is completely free. This behavior is consistent with a training bias in which construction zones predominantly appear on the right side of the road. As a result, SimLingo may resort to a late evasive maneuver and collide with the traffic warning sign while attempting to avoid the oncoming vehicles. In contrast, RoG-DAgger learns to steer right in the same situation. This behavior can be likely attributed to the RoG-Expert, whose expanded trajectory-and-speed solution space exposes the student to multiple feasible recovery strategies and thereby encourages greater trajectory multimodality.
Figure 11: Qualitative examples of RoG-DAgger on Bench2Drive. In row (A), RoG-DAgger successfully merges into traffic while executing a left turn, using emergency braking to avoid a collision with the lead vehicle. In row (B), the predicted geometric trajectory adapts laterally from left to right as the pedestrian moves across the route. This contrasts with PDM-Lite/SimLingo-style behavior, where the geometric path remains largely fixed and collision avoidance is handled primarily through speed adjustment. Row (C) shows a failure case: RoG-DAgger slows down and attempts a lane change to pass through the construction area, however, due to the limited field of view, a vehicle approaching from the rear-left remains unobserved and causes a side collision .
More in this topic
End-to-End Autonomous Driving
End-to-end autonomous driving learns to turn sensor observations and driving context into vehicle actions or trajectories.
See all 227 papers