VLMs for Autonomous Driving

VLM: Vision-Language Model

Follow this topic Ask about this topic

Momentum

16 papers in the last four weeks, up 45% on the four weeks before. 0.2% of all new papers.

Jul 13Week of Sep 28

Latest papers 122

All topics

GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving
Nov 14, 2025Fabian Schmidt, Markus Enzweiler, Abhinav ValadaVision-Language Models VLMs for Autonomous Driving
Esslingen University of Applied Sciences · University of Freiburg
LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving
May 1, 2025Zhijie Qiao, Haowei Li, Zhong Cao +1VLM Evaluation Autonomous Driving Benchmarks
Department of Civil and Environmental Engineering, University of Michigan, Ann Arbor, MI 48109, USA · University of Michigan Transportation Research Institute, Ann Arbor, MI 48109, USA

Nov 14, 2025, cs.CVJ/K move · Enter open · S save

GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving

Fabian Schmidt, Markus Enzweiler, Abhinav Valada

Esslingen University of Applied Sciences · University of Freiburg

Vision-Language Models VLMs for Autonomous Driving Language Model-Based Control Autonomous Driving Autonomous Driving Planning Graph Reasoning

Open paper AskPDF arXiv

Vision-language models have recently emerged as promising planners for autonomous driving, where success hinges on topology-aware reasoning over spatial structure and dynamic interactions from multimodal input. However, existing models are typically trained without supervision that explicitly encodes these relational dependencies, limiting their ability to infer how agents and other traffic entities influence one another from raw sensor data. In this work, we bridge this gap with a novel model-agnostic method that conditions language-based driving models on structured relational context in the form of traffic scene graphs. We serialize scene graphs at various abstraction levels and formats, and incorporate them into models via structured prompt templates, enabling systematic analysis of when and how relational supervision is most beneficial and computationally efficient. Extensive evaluations on the LangAuto and Bench2Drive benchmarks show that scene graph conditioning yields large and persistent improvements. We observe a substantial performance increase in the Driving Score of our proposed approach versus competitive LMDrive, BEVDriver, and SimLingo baselines. These results indicate that diverse architectures can effectively internalize and ground relational priors through scene graph-conditioned training, even without requiring scene graph input at test-time. Code, fine-tuned models, and our scene graph dataset are publicly available at https://github.com/iis-esslingen/GraphPilot.

Momentum

Related topics

Latest papers 122

GraphPilot: Grounded Scene Graph Conditioning for Language-Based Autonomous Driving

LightEMMA: A Longitudinal Evaluation of Vision-Language Models for Autonomous Driving