cs.CVNov 17, 2026

Multimodal Masked Diffusion Autoencoder for Modeling Impaired Driving

Authors: John Gideon, Anshul Gupta, Megan Applegate-Kenton, Kayli Battel, Todd J Rowell, Patricio Reyes Gomez, Laporsha Dees, Emily Sumner, 

Organizations: Toyota Research Institute †

Abstract

Human cognitive state (e.g., distraction, stress), gaze, and behavior interact with the driving environment in subtle ways that AI could use to assist drivers, but ground-truth labels for human cognitive state levels are costly to obtain at scale. We propose a multimodal masked-diffusion autoencoder, ID-MAE, that adds a self-supervised objective to impairment prediction: alongside predicting the impairment label as a masked token, it reconstructs masked gaze and trajectory. To probe the effects of different modalities and types of impairments on model inference, we introduce DIVERT, a driving-simulator dataset containing 103 hours of driving data (n = 109) with ten impairment inductions spanning cognitive, visual, and emotional states, recorded with gaze, vehicle state, and scene video. We demonstrate how ID-MAE affords significant improvements in impairment inference results on both the DIVERT dataset and existing datasets, and enables strong results in additional tasks, such as trajectory and gaze prediction. For location-matched pairs, our model achieves a paired AUC of 0.706 for impairment detection on DIVERT, compared to 0.590 without reconstruction loss, improving over existing baselines. We probe the design space in terms of input modalities, masking scheme, positional encoding, attention mechanism, and decoding. We also investigate the interaction between various impairments in training and inference, demonstrating that visually demanding distractions are separable from cognitive distractions. Code and data will be available upon acceptance.

Figures & tables

Explore similar work

CardsList
  1. EyeCue: Driver Cognitive Distraction Detection via Gaze-Empowered Egocentric Video Understanding

    May 8, 2026Lang Zhang, JinYi Yoon, Matthew Corbett +2Egocentric Video UnderstandingMultimodal Fusion

  2. Towards Robust Vision-Language Reasoning: A Multi-Stage Optimization Pipeline with Masked Diffusion Model

    Nov 11, 2026Shunming Liu, Pei Huang, Yajing Sun +1VLMs for Autonomous DrivingAutonomous Driving Planning

  3. Cognitive-Causal Multi-Task Learning with Psychological State Conditioning for Assistive Driving Perception

    Apr 8, 2026Keito Inoshita, Nobuhiro Hayashida, Akira ImanishiMulti-Task LearningAutonomous Driving Perception