PanguMotion: Continuous Driving Motion Forecasting with Pangu Transformers

Quanhao Ren; Yicheng Li; Nan Song

cs.ROMar 17, 2026

PanguMotion: Continuous Driving Motion Forecasting with Pangu Transformers

Authors: Quanhao Ren, Yicheng Li, Nan Song

Organizations: School of Data Science, Fudan University

Abstract

Motion forecasting is a core task in autonomous driving systems, aiming to accurately predict the future trajectories of surrounding agents to ensure driving safety. Existing methods typically process discrete driving scenes independently, neglecting the temporal continuity and historical context correlations inherent in real-world driving environments. This paper proposes PanguMotion, a motion forecasting framework for continuous driving scenarios that integrates Transformer blocks from the Pangu-1B large language model as feature enhancement modules into autonomous driving motion prediction architectures. We conduct experiments on the Argoverse 2 datasets processed by the RealMotion data reorganization strategy, transforming each independent scene into a continuous sequence to mimic real-world driving scenarios.

Figures & tables

Figure 1: Overview of our PanguMotion architecture. PanguMotion adopts an encoder-decoder structure with a frozen Pangu Block designed to capture interactive relationships within each scene and across the continuous scenes.

Model	minADE 1	minFDE 1	minADE 6	minFDE 6	b-minFDE 6
RealMotion-I	1.808	4.510	0.728	1.420	2.043
RealMotion	1.646	4.100	0.669	1.303	1.935
PanguMotion-I	1.777	4.435	0.717	1.398	2.033
PanguMotion	1.620	4.047	0.672	1.314	1.958

Table 1: Performance Comparison on Argoverse 2 Validation Set

Model	minADE 1	minFDE 1	minADE 6	minFDE 6	b-minFDE 6
LlamaMotion	1.718	4.265	0.690	1.339	1.983
PanguMotion	1.620	4.047	0.672	1.314	1.958

Table 2: Comparison between Pangu and LLaMA Models

Layer Selection	minADE 1	minFDE 1	minADE 6	minFDE 6	b-minFDE 6
Last layer (layer 25)	1.620	4.047	0.672	1.314	1.958
Last three layers (layers 23-25)	1.693	4.204	0.694	1.359	2.003
Last six layers (layers 20-25)	1.702	4.217	0.697	1.357	2.002

Table 3: Comparison of Different Pangu Layer Counts

Insertion Position	minADE 1	minFDE 1	minADE 6	minFDE 6	b-minFDE 6
Layer 0	1.650	4.107	0.676	1.313	1.955
Layer 8	1.680	4.169	0.680	1.326	1.969
Layer 16	1.677	4.182	0.682	1.329	1.972
Layer 25 (last)	1.620	4.047	0.672	1.314	1.958

Table 4: Comparison of Different Insertion Positions

ID	SC	AT	PanGu	$\textit{minADE}_{1}$	$\textit{minFDE}_{1}$	$\textit{minADE}_{6}$	$\textit{minFDE}_{6}$	$\textit{b-minFDE}_{6}$
ID	Strm	Strm		$\textit{minADE}_{1}$	$\textit{minFDE}_{1}$	$\textit{minADE}_{6}$	$\textit{minFDE}_{6}$	$\textit{b-minFDE}_{6}$
1				1.793	4.499	0.721	1.423	2.054
2			$\checkmark$	1.777	4.435	0.717	1.398	2.033
3	$\checkmark$	$\checkmark$		1.646	4.100	0.669	1.303	1.935
4	$\checkmark$	$\checkmark$	$\checkmark$	1.664	4.141	0.677	1.319	1.964
5	$\checkmark$		$\checkmark$	1.620	4.047	0.672	1.314	1.958

Table 5: Ablation Study on Agent Trajectory Stream,“SC Strm” and “AT Strm” indicate our proposed scene context stream and agent trajectory stream, respectively.“PanGu“ indicate the last transformer layer of PanGu Large Language Model.

Trajectory Prediction

Trajectory prediction forecasts how people, vehicles, robots, and other dynamic systems will move from observed histories and context.

See all 187 papers

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting
Sep 8, 2026Yuhang Wang, Chuheng Wei, Jingxin Yang +2Autonomous Driving Datasets Autonomous Driving Benchmarks
University of South Florida · Purdue University · Stanford University +1
Class-Incremental Motion Forecasting
Mar 10, 2026Nicolas Schischka, Nikhil Gosala, B Ravi Kiran +2Class-Incremental Learning Autonomous Driving Perception
Department of Computer Science, University of Freiburg, Germany. · Qualcomm SARL France. · Automated Driving, Qualcomm Technologies, Inc.
FutureLift: Lifting Trajectory Proposals into Bidirectional Future Graphs for Motion Forecasting
Oct 15, 2026Daniel Grimm, J. Marius ZöllnerMultimodal Trajectory Prediction Multi-Agent Trajectory Prediction
FZI Research Center for Information Technology

Sep 8, 2026J/K move · Enter open · S save

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting

Yuhang Wang, Chuheng Wei, Jingxin Yang, Xishun Liao+1

University of South Florida · Purdue University · Stanford University+1

Autonomous Driving Datasets Autonomous Driving Benchmarks Autonomous Driving Human Motion Prediction

Open paper AskPDF arXiv

Driver motion can provide cues to ongoing behavior, attention, and near-term driving intent. However, most existing driver-centric datasets focus on recognizing predefined driver behaviors from short video clips, while human motion forecasting benchmarks largely target motion outside the vehicle. We introduce DriveMotion, a multi-source benchmark for continuous driver motion forecasting. DriveMotion contains 393 hours of 133-keypoint motion sequences at 10 Hz from 360 drivers, integrating naturalistic driving data, curated public in-cabin videos, and the AIDE dataset into a unified representation with per-joint validity masks and synchronized driving context. Naturalistic driving contains long periods of limited body movement, making uniformly sampled evaluation dominated by persistence and less sensitive to brief but behaviorally meaningful motion. To address this, we use dynamics-anchored evaluation, placing forecasting windows around vehicle maneuvers identified offline from CAN signals without providing CAN to the model at inference. Arm motion in pre-maneuver windows is 3.4x greater than in route-matched stable-driving controls. On these anchored windows, learned models reduce forecasting error over persistence by up to 15%, while maneuver-enriched training improves forecast-derived Part-State F1 by 44% over the zero-motion reference. Training on the full multi-source corpus further reduces forecasting error on held-out web drivers by 38% compared with BATON-only training. DriveMotion provides identity-disjoint splits, fixed evaluation subsets, and reference implementations for reproducible evaluation of continuous driver motion forecasting. The dataset and benchmark are available at https://huggingface.co/datasets/HenryYHW/DriveMotion

Abstract

Trajectory Prediction

Explore similar work

DriveMotion: A Large-Scale Multi-Source Benchmark for Driver Motion Sequence Modeling and Forecasting

Class-Incremental Motion Forecasting

FutureLift: Lifting Trajectory Proposals into Bidirectional Future Graphs for Motion Forecasting