cs.CVJan 16, 2026

Generative Scenario Rollouts for End-to-End Autonomous Driving

Authors: Rajeev Yasarla, Deepti Hegde, Shizhong Han, Hsin-Pai Cheng, Yunxiao Shi, Meysam Sadeghigooghari, Shweta Mahajan, Apratim Bhattacharyya, 

Organizations: Qualcomm AI Research · Qualcomm Technologies, Inc.

Abstract

Vision-Language-Action (VLA) models are emerging as highly effective planning models for end-to-end autonomous driving systems. However, current works mostly rely on imitation learning from sparse trajectory annotations and underutilize their potential as generative models. We propose Generative Scenario Rollouts (GeRo), a plug-and-play framework for VLA models that jointly performs planning and generation of language-grounded future traffic scenes through an autoregressive rollout strategy. First, a VLA model is trained to encode ego vehicle and agent dynamics into latent tokens under supervision from planning, motion, and language tasks, facilitating text-aligned generation. Next, GeRo performs language-conditioned autoregressive generation. Given multi-view images, a scenario description, and ego-action questions, it generates future latent tokens and textual responses to guide long-horizon rollouts. A rollout-consistency loss stabilizes predictions using ground truth or pseudo-labels, mitigating drift and preserving text-action alignment. This design enables GeRo to perform temporally consistent, language-grounded rollouts that support long-horizon reasoning and multi-agent planning. On Bench2Drive, GeRo improves driving score and success rate by +15.7 and +26.2, respectively. By integrating reinforcement learning with generative rollouts, GeRo achieves state-of-the-art closed-loop and open-loop performance, demonstrating strong zero-shot robustness. These results highlight the promise of generative, language-conditioned reasoning as a foundation for safer and more interpretable end-to-end autonomous driving.

Figures & tables

Appendix figures & tables11 assets

Supplementary material from the paper’s appendix.

Appendix

Explore similar work

CardsList
  1. Learning Vision-Language-Action World Models for Autonomous Driving

    Apr 10, 2026Guoqing Wang, Pin Tang, Xiangxuan Ren +3World Models for Autonomous DrivingVision-Language-Action Models

  2. LMGenDrive: Bridging Multimodal Understanding and Generative World Modeling for End-to-End Driving

    Apr 9, 2026Hao Shao, Letian Wang, Yang Zhou +5Autonomous DrivingEnd-to-End Autonomous Driving

  3. DriveWLA: Unifying Future Dynamics and Scene Semantics through World-Language-Action Modeling for Autonomous Driving

    Oct 13, 2026Shuzheng Wang, Haoxuan Li, Shengzhen Li +5Mixture-of-TransformersAutonomous Driving Planning