cs.CVNov 16, 2025

FSDAM: Few-Shot Driving Attention Modeling via Vision-Language Coupling

Authors: Kaiser Hamid, Can Cui, Khandakar Ashrafi Akbar, Ziran Wang, Nade Liang

Organizations: Texas Tech University, Lubbock, TX, USA · Purdue University, West Lafayette, IN, USA · Towson University, Towson, MD, USA

Abstract

Understanding not only where drivers look but also why their attention shifts is essential for interpretable human-AI collaboration in autonomous driving. Driver attention is not purely perceptual but semantically structured. Thus attention shifts can be learned through minimal semantic supervision rather than dense large-scale annotation. We present FSDAM (Few-Shot Driver Attention Modeling), a framework that achieves joint spatial attention prediction and structured explanation generation using 90 annotated examples. Our key insight is to decompose attention into an explicit reasoning representation, including scene context, current focus, anticipated next focus, and causal explanation, and to learn next-focus anticipation through minimal-pair supervision. To address task conflict and large sample requirements of existing models, and to mitigate task interference under limited data, we introduce a novel dual-pathway architecture in which separate modules handle spatial prediction and caption generation. In addition, we use a training-only vision-language alignment mechanism that injects semantic priors into spatial learning without increasing inference complexity, mitigating task interference under few-shot training. Despite extreme data scarcity, FSDAM achieves competitive performance in gaze prediction, and generates coherent, context-aware structural reasoning for improved interpretability. The model further demonstrates strong zero-shot generalization across multiple driving benchmarks. These results suggest that semantically grounded attention modeling enables data-efficient learning and provides a scalable path toward explainable driver attention systems in data-constrained environments.

Figures & tables

Explore similar work

CardsList
  1. DiffAttn: Diffusion-Based Drivers' Visual Attention Prediction with LLM-Enhanced Semantic Reasoning

    Mar 30, 2026Weimin Liu, Qingkun Li, Jiyuan Qiu +2Visual AttentionAutonomous Driving Perception

  2. From Scene to Object: Text-Guided Dual-Gaze Prediction

    Apr 22, 2026Zehong Ke, Yanbo Jiang, Jinhao Li +5Gaze EstimationAutonomous Driving Perception