Turn-by-turn (TBT) navigation systems are integral to modern driving experiences, providing real-time audio instructions to guide drivers safely to destinations. However, existing audio instruction policy often relies on rule-based approaches that struggle to balance informational content with cognitive load, potentially leading to driver confusion or missed turns in complex environments. To overcome these difficulties, we first model the generation of navigation instructions as a multi-task learning problem by decomposing the audio content into combinations of modular elements. Then, we propose a novel deep learning framework that leverages the powerful spatiotemporal information processing capabilities of Transformers and the strong multi-task learning abilities of Mixture of Experts (MoE) to generate real-time, context-aware audio instructions for TBT driving navigation. A cloud-edge collaborative architecture is implemented to handle the computational demands of the model, ensuring scalability and real-time performance for practical applications. Experimental results in the real world demonstrate that the proposed method significantly reduces the yaw rate (the proportion of vehicles deviating from navigation routes) compared to traditional methods, delivering clearer and more effective audio instructions. This is the first large-scale application of deep learning in driving audio navigation, marking a substantial advancement in intelligent transportation and driving assistance technologies.
Figures & tables
Fig. 1: TBT driving navigation. The figure displays the essential cue content and key terminology used in the TBT driving navigation, along with the paradigm of the audio instruction while the car moves through the path.
Fig. 2: Overview of TBT navigation instruction model. In the dashed box is the proposed navigation instruction model. The dashed red box represents the model component, the solid blue box represents the edge-side data embedding, and the solid green box represents the cloud-side data embedding. To the left of the dashed box is the engineering design for model real-time playing and data preparation. Abbreviations: MoE means Mixture of Experts, MLP means Multi-Layer Perceptron, MHA means Multi-Head Attention.
Fig. 3: Cloud-edge collaboration framework. The left side shows the user’s driving behavior, and the edge device in the middle determines whether it needs to generate audio instruction based on the driving progress, as well as the start of each segment requesting cloud feature embedding from the cloud server on the right side.
Fig. 4: Real-world A/B test results. Green represents the sequence model and orange represents the deployed HMM policy. (a) Yaw rate, lower yaw rates represent more effective audio instruction; (b) Words density, lower word density means more streamlined audio; (c) Main action element play rate, higher means more frequent TBT message alerts.
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China
Joint Research Laboratory for Embodied Intelligence, Xinjiang University · Joint International Research Laboratory of Silk Road Multilingual Cognitive Computing, Xinjiang University · School of Computer Science and Technology, Xinjiang University, Urumqi 830017, China