cs.CVMar 10, 2026
SaveProbing the Reliability of Driving VLMs: From Inconsistent Responses to Grounded Temporal Reasoning
Organizations: DFKI Augmented Vision · TU Delft
Abstract
A reliable driving assistant should provide consistent responses based on temporally grounded reasoning derived from observed information. In this work, we investigate whether Vision-Language Models (VLMs), when applied as driving assistants, can response consistantly and understand how present observations shape future outcomes, or whether their outputs merely reflect patterns memorized during training without temporally grounded reasoning. While recent efforts have integrated VLMs into autonomous driving, prior studies typically emphasize scene understanding and instruction generation, implicitly assuming that strong visual interpretation naturally enables consistant future reasoning and thus ensures reliable decision-making, a claim we critically examine. We focus on two major challenges limiting VLM reliability in this setting: response inconsistency, where minor input perturbations yield different answers or, in some cases, responses degenerate toward near-random guessing, and limited temporal reasoning, in which models fail to reason and align sequential events from current observations, often resulting in incorrect or even contradictory responses. Moreover, we find that models with strong visual understanding do not necessarily perform best on tasks requiring temporal reasoning, indicating a tendency to over-rely on pretrained patterns rather than modeling temporal dynamics. To address these issues, we adopt existing evaluation methods and introduce FutureVQA, a human-annotated benchmark dataset specifically designed to assess future scene reasoning. In addition, we propose a simple yet effective self-supervised tuning approach with chain-of-thought reasoning that improves both consistency and temporal reasoning without requiring temporal labels.
Figures & tables
Figure 1: Reliability failures in VLMs. The figure illustrates three issues: (i) response inconsistency—identical or very similar prompts yield different answers; (ii) contradiction—correct local interpretation but inconsistent future description; and (iii) temporal misalignment—events predicted at incoherent times despite accurate per-frame cues.
Figure 2: Overview of our framework for evaluating reliable temporal reasoning in VLM driving assistants. Left: The agent consumes past frames and a prompt to generate temporally aligned predictions over a variable future horizon. Right (FutureVQA): Benchmark construction combines human and AI contributions: human experts create natural Q/A pairs, while AI performs quality control to ensure answerability and consistency. Bottom (Evaluation): To thoroughly analyze model reliability, we adopt a self-aligned future description setup, where a model’s predicted description is compared to a reference response generated by the same model when the actual future frames are provided. An AI checker is further applied to validate that predictions remain coherent and meaningful. Beyond this, we evaluate consistency under repeated queries and option shuffling, and analyze temporal performance decay to quantify how model reliability changes as the prediction horizon increases.
Figure 3: Example of the FutureVQA task. The VLM is asked to answer questions about future scenes based on predictions, without access to the corresponding future frames.
Figure 4: Proposed self-supervised approach to align temporal events and minimize incorrect or contradictory reasoning. Given a video sequence , we generate detailed descriptions using a pretrained VLM as pseudo reference labels . We then fine-tune the model , initialized from , using only past frames as input and training it to predict descriptions of unseen future frames . A weighting function adjusts the contribution of each loss term based on the temporal distance .
| VLM | Visual Label | Accuracy | |||
|---|---|---|---|---|---|
| Single-Trial | Multi-Trial | ||||
| GPT-4o [ 26 ] | Image | 76.2% | 66.1% | 11.1% | 86.7% |
| GPT-4o-mini [ 26 ] | Image | 66.9% | 54.5% | 12.4% | 81.5% |
| LLV-v1.5-7b [ 39 ] | Image | 55.1% | 33.8% | 21.3% | 61.3% |
| LLV-v1.5-13b [ 39 ] | Image | 61.0% | 42.3% | 18.7% | 69.3% |
| LLV-Next-13b [ 38 ] | Image | 41.8% | 18.7% | 23.1% | 44.7% |
Table 1: In this evaluation we examine the ability of different VLMs on our evaluation dataset, where multiple answer options are shuffled across several rounds of answering by the VLMs. The accuracy change reflects the difference in performance between single-trial approach and multiple-trial answering, where the LLM must consistently identify the correct option in every round. This method minimizes the influence of random guessing by ensuring that only consistently correct answers are counted. denotes the performance drop from single-trial to multi-trial. The ratio represents the remaining performance.
| Model | Accuracy | NDR | mRAR | ||||
|---|---|---|---|---|---|---|---|
| Acc@1s | Acc@4s | Acc@12s | |||||
| GPT-4o | 59.1% | 41.1% | 31.6% | -27.5% | 42.2% | 0.42 | 0.64 |
| GPT-4o-mini | 47.7% | 36.0% | 32.0% | -15.7% | 37.7% | 0.29 | 0.69 |
| LLV-v1.5-7b | 24.0% | 18.1% | 16.0% | -8.0% | 18.6% | 0.24 | 0.55 |
| LLV-v1.5-13b | 37.8% | 30.9% | 26.3% | -11.5% | 30.7% | 0.27 | 0.73 |
| LLV-Next-13b | 15.4% | 9.3% | 4.2% | -11.2% | 7.3% | 0.60 | 0.39 |
Table 2: Accuracy (Acc) of models on our VQA benchmark at different future time frames. All accuracy values are evaluated across multiple trials to minimize the influence of random chance. The result suggest that models like GPT-4o, while showing strong ability in visual understdaning, fail to maintain consistent future scene reasoning across different time interval. *Our model is not trained with explicit temporal (video) label.
Figure 5: Temporal performance decay analysis on the FutureVQA dataset. (a) Accuracy decay across horizons, where solid lines denote four trials and shaded regions indicate fewer trials (1–3). (b) Relationship between regular VQA performance (y-axis) and relative long-horizon preservation (x-axis: Acc@12 divided by regular VQA accuracy). (c) Relationship between regular VQA performance (y-axis) and relative mean preservation (x-axis: divided by regular VQA accuracy). Together, these plots show how well models retain their performance when extending from immediate perception to future prediction.
| Model | |||||
|---|---|---|---|---|---|
| Baseline | 10.7 | 6.0 | 22.8 | 2.3 | 25.4 |
| FutureAgent | 20.3 | 19.8 | 35.2 | 11.3 | 34.6 |
| Baseline | 12.3 | 7.1 | 25.2 | 3.6 | 28.5 |
| FutureAgent | 28.8 | 22.7 | 37.3 | 12.3 | 39.2 |
| w/o CoT | 25.9 | 20.4 | 35.9 | 11.1 | 38.3 |
Table 3: We compare how well our proposed model describes future scenes as if it ”sees” them. A higher value indicates greater similarity between the reference description and the predicted description. The mean score, , is computed over discrete time steps seconds. B3: BLEU-3, B4: BLEU-4, R-L: ROUGE-L, C: CIDEr, M: METEOR.
| Model | Score Over Time | ||||
|---|---|---|---|---|---|
| S@1s | S@2s | S@4s | S@8s | S@12s | |
| LLV-v1.5-7b | 2.59 | 2.67 | 2.07 | 2.52 | 2.25 |
| LLV-v1.5-13b | 2.13 | 1.92 | 1.94 | 2.49 | 2.40 |
| LLV-Next-13b | 2.11 | 2.87 | 2.26 | 2.57 | 2.15 |
| Baseline | 4.88 | 4.01 | 2.96 | 2.34 | 2.41 |
| FutureAgent | 5.31 | 5.01 | 3.98 | 3.44 | .2.46 |
Table 4: Model-based evaluation of predicted caption quality across various time frames using GPT-4o, with a specific focus on objective descriptions, such as the accuracy of object appearance and location within the image.
Appendix figures & tables14 assets
Supplementary material from the paper’s appendix.
Appendix
| Dataset | N. Ques. | N. Uniq. Ques. | Vocab. Size | TTR |
|---|---|---|---|---|
| DriveLM(Pred.) | 123k | 15 | 69 | |
| DriveLM(Pred.&Percep.) | 285k | 234 | 150 | |
| Ours | 2.7k | 969 | 433 |
Table 5: Comparison of question diversity between our dataset and DriveLM. N. Ques. denotes the total number of questions; N. Uniq. Ques. represents the number of unique questions after de-duplication; Vocab. Size is the number of distinct words used in the questions; and TTR (Type-Token Ratio) measures lexical diversity, computed as the ratio of unique words to total words. The results highlight its greater linguistic diversity and reduced reliance on fixed templates.
| Benchmark | Task | T. Size | Cust. Q | Ans. Type | Mul. Trl. | Mul. C. | Pred. | T-Pred. |
|---|---|---|---|---|---|---|---|---|
| nuScenes-QA [ 47 ] | Drive VQA | 83.3k** | ✗ | Mixed | ✗ | ✓ | ✗ | ✗ |
| BDD-X [ 33 ] | Drive Action | 2.6k | - | Sentence | ✗ | ✓ | ✗ | ✗ |
| DRAMA [ 42 ] | Drive VQA | 11.6k | ✗ | Mixed | ✗ | ✗ | ✗ | ✗ |
| Rank2Tell [ 50 ] | Drive VQA | - | ✗ | Mixed | ✗ | ✓ | ✗ | ✗ |
| OmniDrive [ 55 ] | Drive VQA | 24k† | ✓ | Sentence | ✗ | ✓ | ✓ | ✗ |
| DriveLM-nS [ 52 ] | Drive VQA | 73k* | ✗ | Sentence | ✗ | ✓ | ✓ | ✗ |
Table 6: Comparison of existing VLM benchmarks. Key aspects of dataset creation include test size (T. Size), whether questions are customized for different scenarios and video clips (Cust. Q), answer type (Ans. Type), multi-trial evaluation for each question (Mul. Tri), inclusion of multiple cities (Mul. C.), presence of perception tasks (Perc.), inclusion of prediction tasks (Pred.), and whether the dataset challenges VLMs with time-specific prediction (T-Pred.). Our benchmark dataset consists of fully human-annotated QA pairs tailored to different scenes, rather than relying on rule-based methods. Furthermore, our dataset challenges VLMs to predict future scenes at specific time intervals, requiring precise temporal reasoning to differentiate between near-future and far-future events. †: The QA pairs are fully generated by GPT-4. ** Fully rule-based (no human annotators), * Semi-rule-based labeling (with human annotators for certain tasks).
Figure 6: The prompt used to instruct VLMs to predict the future scene and answer the corresponding question.
Figure 7: Examples of visual question answering (VQA) tasks categorized into different types: Hallucination , General , Traffic Understanding , Absolute Location , and Relative Position . Each question is categorized based on the type of reasoning it requires; however, a single question can belong to multiple categories simultaneously, depending on its context and the type of information needed.
Figure 8: Radar plots comparing the performance of various models across five VQA categories: Hallucination, General, Traffic Understanding, Absolute Location, and Relative Position. In this experiment, models perform regular VQA on images, with the actual images provided as input. The plots illustrate the strengths and weaknesses of each model in handling different reasoning tasks, providing a comparative baseline for understanding the capabilities of existing VLMs before extending to future image QA tasks.
Figure 9: Radar plots comparing the performance of different models across various VQA categories (Hallucination, General, Traffic Understanding, Absolute Location, and Relative Position) at different future time steps: (a) , (b) , (c) , and (d) . The results highlight that while most models maintain robustness in hallucination detection, their performance in other categories, particularly traffic understanding and spatial reasoning, declines as the time offset increases.
Figure 10: Overview of our visual encoding pipeline. The goal is to minimize the number of tokens while maintaining similar performance. In the context of autonomous driving videos, recent frames typically have greater influence on upcoming events. To reflect this, the Memory Decay Sampler assigns fewer queries to older frames, while the Adaptive Token Sampler adjusts the number of tokens based on the similarity between adjacent frames. The Prior Frame Memory Encoder is a transformer-based module that integrates temporal information from preceding frames.
Figure 11: Visualization of frame similarity evaluation using Gaussian smoothing followed by the Sobel operator with different kernel sizes. The input images have a resolution of 1280 × 720 pixels. The difference between two consecutive frames, , is computed, smoothed using Gaussian filters with kernel sizes of 5, 9, 11, 13, 15, and 17, and then processed with the Sobel operator, , to highlight changes. For better readability, the colors of the similarity maps are inverted.
| Time | Model | Scores | ||||
|---|---|---|---|---|---|---|
| B-3 | B-4 | R-L | C | M | ||
| +1s | Baseline | 13.4 | 6.1 | 25.0 | 2.4 | 25.9 |
| Ours | 32.2 | 26.2 | 40.2 | 17.7 | 41.5 | |
| w/o CoT | 28.1 | 22.7 | 38.2 | 15.1 | 40.2 | |
| w/o self-sup. | 12.1 | 7.2 | 24.9 | 2.7 | 26.2 | |
| 32.5 | 26.5 | 40.0 | 17.6 | 41.3 | ||
Table 7: In this comparison the reference captions are from regular image captioning, while the compared captions are generated by our fine-tuned model which perform future scenes captioning with only previous frames are given. B-3: BLEU-3, B-4: BLEU-4, R-L: ROUGE-L, C: CIDEr, M: METEOR.
Figure 12: Accuracy over time in the FutureVQA task with different visual input encoding strategies, showing that our sampling approach reduces the number of required tokens while maintaining higher performance.
Figure 13: Comparison of captions generated by LLaVA-V1.5-7B [ 39 ] and LLaVA-V1.6-34B [ 38 ] . While LLaVA-V1.5-7B produces shorter and repetitive captions with occasional hallucination, LLaVA-V1.6-34B generates significantly longer and more detailed descriptions. Additionally, LLaVA-V1.6-34B exhibits a varied response pattern, providing distinct levels of detail and focus when presented with different images.
Figure 14: Prompts used for three tasks: GPT-4o as a judge in captioning evaluation and Regular VQA on our annotated evaluation dataset. Each prompt is tailored to the specific requirements of its respective task.
Figure 15: Quantitative result.
Figure 16: Quantitative result.