TrajMamba: An Ego-Motion-Guided Mamba Model for Pedestrian Trajectory Prediction from an Egocentric Perspective

Yusheng Peng; Gaofeng Zhang; Liping Zheng

cs.CVMar 16, 2026

TrajMamba: An Ego-Motion-Guided Mamba Model for Pedestrian Trajectory Prediction from an Egocentric Perspective

Authors: Yusheng Peng, Gaofeng Zhang, Liping Zheng

Organizations: School of Computer Science and Information Engineering, Hefei University of Technology, Hefei 230601, China · School of Software, Hefei University of Technology, Hefei 230601, China

Abstract

Future trajectory prediction of a tracked pedestrian from an egocentric perspective is a key task in areas such as autonomous driving and robot navigation. The challenge of this task lies in the complex dynamic relative motion between the ego-camera and the tracked pedestrian. To address this challenge, we propose an ego-motion-guided trajectory prediction network based on the Mamba model. Firstly, two Mamba models are used as encoders to extract pedestrian motion and ego-motion features from pedestrian movement and ego-vehicle movement, respectively. Then, an ego-motion guided Mamba decoder that explicitly models the relative motion between the pedestrian and the vehicle by integrating pedestrian motion features as historical context with ego-motion features as guiding cues to capture decoded features. Finally, the future trajectory is generated from the decoded features corresponding to the future timestamps. Extensive experiments demonstrate the effectiveness of the proposed model, which achieves state-of-the-art performance on the PIE and JAAD datasets.

Figures & tables

Fig. 1 : Overview of the proposed TrajMamba.

Models	Publications	Years	PIE				JAAD
			ADE $\downarrow$	FDE $\downarrow$	ARB $\downarrow$	FRB $\downarrow$	ADE $\downarrow$	FDE $\downarrow$	ARB $\downarrow$	FRB $\downarrow$
			Observation frames: 15 (0.5s) Prediction frames: 30 (1.0s)
FOL [ 40 ]	ICRA	2019	73.87	164.53	78.16	143.49	61.39	126.97	70.12	129.17
FPL [ 39 ]	CVPR	2018	56.66	132.23	/	/	42.24	86.13	/	/
B-LSTM [ 17 ]	CVPR	2018	27.09	66.74	37.41	75.87	28.36	70.22	39.14	79.66
PIE_traj [ 43 ]	ICCV	2019	21.82	53.63	27.16	55.39	23.49	50.18	30.40	57.17

TABLE I : Performance of the proposed TrajMamba and other existing models on the PIE and JAAD datasets.

Input	Output	ADE	FDE	ARB	FRB
Input	Output	PIE
LT-BR	LT-BR	8.72	18.69	24.68	27.88
xywh	xywh	13.64	24.84	51.50	78.76
LT-BR + v	LT-BR	8.58	18.20	13.15	24.16
xywh+v+s	CV-CS-Offset	7.72	16.99	11.60	22.00
		JAAD

TABLE II : Ablation results on input and output representation.

	Model	ADE	FDE	ARB	FRB	Flops	Params
PIE
PFD	LSTM	7.98	17.82	12.14	23.29	128.27G	3.16M
	GRU	7.97	17.98	12.25	23.72	100.68G	2.51M
	Transformer	7.94	17.86	12.25	23.52	87.48G	2.31M
	Mamba	7.91	17.87	11.99	23.19	30.11G	1.32M
EMGD	LSTM	8.14	18.26	12.34	23.57	63.64G	1.78M

TABLE III : Ablation Results on Ego-motion-guided decoder. (PFD: post-fusion decoding mechanism, EMGD: ego-motion-guided decoding mechanism)

Encoder	Decoder	ADE	FDE	ARB	FRB
Encoder	Decoder	PIE
LSTM	Mamba	8.13	18.23	12.28	23.58
GRU	Mamba	7.87	17.41	11.89	22.67
Transformer	Mamba	7.81	17.46	11.84	22.67
Mamba	Mamba	7.72	16.99	11.60	22.00
		JAAD

TABLE IV : Ablation Results on Encoder.

Encoder	Decoder	ADE	FDE	ARB	FRB
Encoder	Decoder	PIE
Mamba	LSTM	8.05	18.01	12.56	24.03
Mamba	GRU	7.98	17.90	12.33	23.59
Mamba	Transformer	7.72	17.33	11.90	22.87
Mamba	Mamba	7.72	16.99	11.60	22.00
		JAAD

TABLE V : Ablation Results on Decoder.

Panel of Figure 7 — Fig. 2 : Pedestrian trajectory prediction qualitative samples on the PIE and JAAD datasets. Ground truth positions in red, predictions in green. Best viewed in colour.

Pedestrian Trajectory Prediction

Pedestrian trajectory prediction forecasts where people will move from their observed motion and surrounding context.

See all 36 papers

Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos
Jun 17, 2026Yuxuan Xie, Nicolas Pugeault, Chongfeng Wei +2Pedestrian Intention Prediction Pedestrian Behavior Modeling
School of Computing Science, University of Glasgow, Glasgow, United Kingdom · James Watt School of Engineering, University of Glasgow, Glasgow, United Kingdom · Department of Computer Science, Durham University, Durham, United Kingdom
MUSCLE-NET: Predicted-Multiscale-Aware Network for Pedestrian Trajectory Forecasting
May 30, 2026Yu Liu, Ming Huang, Xiao Ren +3Pedestrian Behavior Modeling Pedestrian Trajectory Prediction
Guangdong Provincial Key Laboratory of Fully Actuated System Control Theory and Technology, School of Automation and Intelligent Manufacturing, Southern University of Science and Technology (SUSTech), Shenzhen 518055, China · Department of Mechanical Engineering, City University of Hong Kong, Hong Kong SAR, China
MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States
Sep 7, 2026Juncen Long, Xiaofeng Jin, Gianluca Bardaro +2Pedestrian Behavior Modeling Pedestrian Trajectory Prediction
Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

Jun 17, 2026J/K move · Enter open · S save

Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos

Yuxuan Xie, Nicolas Pugeault, Chongfeng Wei, Hubert P. H. Shum+1

School of Computing Science, University of Glasgow, Glasgow, United Kingdom · James Watt School of Engineering, University of Glasgow, Glasgow, United Kingdom · Department of Computer Science, Durham University, Durham, United Kingdom

Pedestrian Intention Prediction Pedestrian Behavior Modeling Pedestrian Trajectory Prediction Autonomous Driving Perception Multimodal Trajectory Prediction Trajectory Prediction

Open paper AskPDF arXiv

Pedestrian trajectory prediction from an ego-centric camera is challenging since it depends on complex interactions with vehicles and scene context, as well as the intention of the pedestrian. By modelling correlation and intent from the historical and future trajectories of the pedestrian, it will usually result in a multimodal (i.e. multiple modes) distribution. Existing stochastic predictors often sample multiple futures from a single unimodal distribution, which can yield sub-optimal 'mixed-mode' trajectories that lie between distinct motion patterns and become implausible in real scenes. In this paper, we propose MMPM, a mode-aware framework that separately models future trajectory distributions into semantically meaningful modes based on the pedestrian's crossing behavior. MMPM consists of two modules: behavior-aware Pedestrian Interaction Module (PIM) that jointly captures pedestrian-vehicle and pedestrian-environment interactions by introducing gaze, head and hand gesture, and a CVAE-based Mode-aware Trajectory Predictor (MTP) module to model the future trajectory distributions on two modes, crossing and non-crossing the road, separately. A query-based decoder further enforces mode consistency during decoding. Experiments on PIE and JAAD datasets show that our method surpasses state-of-the-art baselines. Our proposed MTP is model-agnostic, which can be integrated into existing frameworks such as BiTrap-NP and SGNet-ED to further improve future trajectory prediction performance. We additionally introduce a data-driven validation protocol that matches predictions to spatio-temporally consistent ground-truth trajectories, demonstrating improved frame-wise displacement errors over previous work.

Abstract

Pedestrian Trajectory Prediction

Explore similar work

Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos

MUSCLE-NET: Predicted-Multiscale-Aware Network for Pedestrian Trajectory Forecasting

MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States