cs.CVNov 16, 2026

GridDriver: On-Board BEV-Based Language Planning and Reasoning

Authors: Katharina Winter, William Engel, Zain A. Alishammat, Fabian B. Flohr

Organizations: Munich University of Applied Sciences, Intelligent Vehicles Lab (IVL), 80335 Munich, Germany

Abstract

Bird's-Eye-View (BEV)-conditioned large language models (LLMs) enable language-interactive driving, but onboard deployment requires efficient perception and continuous planning alongside autoregressive language generation. We present GridDriver, an efficient BEV-conditioned LLM framework combining trajectory prediction, structured driving indicators, natural-language navigation, driving explanations, and scene question answering. Lightweight spatio-temporal BEV perception and spatially structured tokens provide scene context to a shared LLM, while frame-wise scheduling interleaves trajectory prediction with asynchronous language generation. GridDriver achieves competitive performance on public benchmarks, reaching 87.9% PDMS on NAVSIM and 57.6% accuracy on nuScenes-QA with a shared planning and QA checkpoint. Spatial representation analyses demonstrate improved geometric information retention and trajectory prediction compared with Q-Former compression. The system achieves 32.7 ms standalone planning latency and 9.4 plans/s in an onboard ROS 2 pipeline. We further evaluate planning on our own data recordings and demonstrate open-loop onboard operation on a research vehicle. Across public benchmarks and real-world evaluations, we identify strengths and limitations of the BEV-based approach.

Figures & tables

Explore similar work

CardsList
  1. OneDrive: Unified Multi-Paradigm Driving with Vision-Language-Action Models

    Apr 20, 2026Yiwei Zhang, Xuesong Chen, Jin Gao +5VLMs for Autonomous DrivingEnd-to-End Autonomous Driving

  2. Qwen-Drive-1.0: An Initial Step towards a Vision-Language Foundation Model for Autonomous Driving

    Aug 31, 2026Xin Zhou, Zongchuang Zhao, Zhibo Yang +13VLMs for Autonomous DrivingAutonomous Driving Perception

  3. NaviDriveVLM: Decoupling High-Level Reasoning and Motion Planning for Autonomous Driving

    Mar 9, 2026Ximeng Tao, Pardis Taghavi, Dimitar Filev +2VLMs for Autonomous DrivingAutonomous Driving Planning