EgoNeMo: Transferable Map of Pedestrian Dynamics via Egocentric LiDAR Scan

Azusa Sawada; Allan Wang; Hideo Saito; Aaron Steinfeld

cs.LGSep 5, 2026

EgoNeMo: Transferable Map of Pedestrian Dynamics via Egocentric LiDAR Scan

Authors: Azusa Sawada, Allan Wang, Hideo Saito, Aaron Steinfeld

Organizations: NEC Corporation · Miraikan - The National Museum of Emerging Science and Innovation · Keio University · Carnegie Mellon University

Abstract

This paper proposes a transferable Map of Dynamics (MoD) framework that generalizes to unknown environments using only egocentric 3D LiDAR point clouds to overcome the long-standing limitation of traditional MoD methods. While MoDs are essential for encoding human motion characteristics to enable accurate pedestrian trajectory prediction or safe robot navigation, traditional approaches suffer from site-specificity, requiring exhaustive trajectory accumulation at every new location. Extending recent advances in neural implicit modeling, our framework trains a continuous, LiDAR-based MoD estimator across diverse environments. To mitigate the inherent sparsity and temporal bias of real-world trajectory data, we introduce a position-balanced sampling strategy and a multi-task learning architecture that jointly predicts motion distributions and a spatial frequency score map. The latter is further augmented by visibility-aware losses to compensate for incomplete observation data. Comprehensive experiments demonstrate that our method effectively reconstructs underlying motion maps even in unknown locations from a single instantaneous LiDAR scan, despite highly sparse training data. Finally, we show that our improvements enhance the reliability of downstream trajectory prediction.

Figures & tables

Figure 1: Concept of our framework compared to NeMo-map [ 34 ] . We train model across $L$ multiple locations to build transferable map of dynamics.

Figure 2: Overview of transferable MoD framework EgoNeMo. A neural network with a BEV encoder and two task heads is trained to predict Semi-wrapped Gaussian Mixture Model (SWGMM) parameters and a frequency score from LiDAR input. SWGMM image is from paper [ 34 ]

Figure 3: Spatial sparsity of ATC, KTH and JRDB. Red color shows position counts from all trajectories in a sequence.

Model	JRDB-Known*	JRDB-Unknown	SiT
NeMo	12.0 (1.16)	N/A	N/A
BFF*	1.01 (0.06)	1.53 (0.05)	1.34 (0.12)
Ours	1.29 (0.16)	1.42 (0.05)	1.24 (0.11)

Table 1: NLL comparison with existing MoD methods. JRDB-Known* refer to three sequences with clear spatial overlaps with train sequences, gates-ai-lab-2019-04-17_0, huang-2-2019-01-25_1, and tressider-2019-03-16_2 from known location sequences.

	JRDB-Known		JRDB-Unknown		SiT
Method	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)
Motion	1.22 (0.08)	4.12 (0.61)	1.79 (0.29)	4.66 (0.35)	1.63 (0.04)	4.68 (0.86)
Motion-Freq	1.14 (0.03)	3.68 (0.12)	1.67 (0.09)	4.03 (0.39)	1.45 (0.07)	3.82 (0.74)
Motion(GS)-Freq	1.14 (0.01)	2.82 (0.55)	1.47 (0.08)	3.34 (0.48)	1.22 (0.10)	2.80 (0.46)
Motion(GS)-FreqPU	0.95 (0.03)	3.08 (0.69)	1.33 (0.03)	3.88 (0.66)	1.22 (0.01)	3.43 (1.35)
Motion(GS)-FreqPNi	1.12 (0.07)	2.54 (0.27)	1.42 (0.05)	2.98 (0.51)	1.24 (0.11)	2.82 (0.74)

Table 2: NLL comparison in JRDB and SiT dataset. Motion corresponds to the baseline that simply extends NeMo-map [ 34 ] . Each value represents the mean of three trials, with the standard deviation shown in parentheses. The lowest mean value per column is in bold.

Panel of Figure 4 — Figure 4: Outputs of Motion and Motion-Freq on JRDB unknown locations. In the motion maps, arrows visualize the velocity modes; opacity represents weight, arrow length represents speed, and hue represents direction using a cyclic color mapping. The same hue corresponds to the same direction across all maps. The reference column shows the sequence names and stitched images, horizontally compressed for visualization. The input column shows LiDAR data as BEV with height-based color from navy to yellow.

Panel of Figure 5 — Figure 5: Frequency score and motion outputs of Motion(GS)-Freq , Motion(GS)-FreqPU , and Motion(GS)-FreqPNi . For each method, the left panel shows the predicted frequency score (brighter color means higher score), and the right panel shows the motion map. Samples and arrow encoding of motion maps are the top two rows of Fig. 4 , the rest are in the Appendix Fig. 4.

	Fewer locations				Shorter durations
	JRDB-Known		JRDB-Unknown		JRDB-Known		JRDB-Unknown
Method	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)
M only	1.24 (0.08)	4.70 (0.54)	1.81 (0.09)	5.09 (0.16)	1.50 (0.14)	4.37 (0.58)	2.26 (0.54)	5.31 (0.74)
M-F	1.23 (0.06)	4.20 (0.48)	1.81 (0.03)	5.40 (0.54)	1.43 (0.10)	4.34 (0.23)	1.96 (0.05)	5.17 (0.43)
M(G)-F	1.51 (0.01)	3.40 (0.43)	1.87 (0.05)	4.11 (0.34)	1.30 (0.02)	2.89 (0.14)	1.75 (0.06)	3.78 (0.05)
M(G)-FPU	0.97 (0.04)	3.22 (0.45)	1.40 (0.02)	4.03 (0.64)	1.01 (0.03)	3.20 (0.35)	1.37 (0.06)	4.22 (0.76)

Table 3: NLL comparison in JRDB dataset for reduced train data. Each value represents the mean of three trials, with the standard deviation shown in parentheses. The lowest mean value in each column is highlighted in bold. The method column shows short-hand names (M, F, and G) instead of Motion, Freq, and GS, respectively.

		T=12		T=18		T=24
Method	K	ADE	FDE	ADE	FDE	ADE	FDE
ST [ 24 ]	1	0.54	0.99	0.79	1.57	1.05	2.14
ViTE [ 14 ]	3	0.36	0.57	0.52	0.84	0.67	1.11
NeMo [ 34 ]	1	0.44	0.82	0.63	1.23	0.81	1.60
NeMo [ 34 ]	3	0.42	0.80	0.62	1.20	0.79	1.57
Ours-base	3	0.42	0.79	0.61	1.19	0.78	1.55

Table 4: Trajectory prediction errors on JRDB.

K

is the number of modes. NeMo uses CLiFF-LHMP algorithm [ 32 , 34 ] on the same MoD as Ours . All results are averaged over 3 training runs.

Parameter	Value
Time step size $\Delta t$	$0.33$ [s]
Kernel time scale $\sigma_{t}$	$1.5$ [s]
Mode weight threshold $w_{\text{th}}$	$0.05$
Velocity blend parameter $\beta$	$5.0$ [ $\text{rad}^{-2}$ ]
Prior speed variance $\sigma_{\rho}^{2}$	$(0.01\text{ [m/s]})^{2}$
Prior angle variance $\sigma_{\theta}^{2}$	$(0.4\text{ [rad/s]})^{2}$

Table S1 : Hyperparameter configurations utilized in the trajectory prediction evaluation.

	JRDB-Known				JRDB-Unknown
Method	[0.0, 2.5]	[2.5, 5.0]	[5.0, 7.5]	[7.5, 10.0]	[0.0, 2.5]	[2.5, 5.0]	[5.0, 7.5]	[7.5, 10.0]
M only	1.16 (0.07)	1.19 (0.09)	1.53 (0.07)	1.68 (0.02)	1.71 (0.19)	1.76 (0.30)	1.98 (0.37)	1.92 (0.17)
M-F	1.11 (0.07)	1.11 (0.03)	1.44 (0.08)	1.54 (0.04)	1.61 (0.15)	1.67 (0.08)	1.83 (0.09)	1.87 (0.10)
M(G)-F	1.05 (0.04)	1.12 (0.05)	1.26 (0.03)	1.39 (0.06)	1.36 (0.04)	1.47 (0.09)	1.69 (0.13)	1.78 (0.19)
M(G)-FPU	0.87 (0.06)	0.95 (0.03)	1.08 (0.02)	1.20 (0.02)	1.28 (0.03)	1.32 (0.04)	1.47 (0.07)	1.62 (0.10)
M(G)-FPNi	1.06 (0.05)	1.07 (0.08)	1.29 (0.03)	1.44 (0.02)	1.43 (0.06)	1.40 (0.05)	1.61 (0.08)	1.70 (0.07)

Table S2 : Spatial range analysis across regions from center to boundary. Negative Log-Likelihood (NLL) is evaluated across non-overlapping region bins centered at the sensor origin. Ranges indicate the lower and upper bounding thresholds for both

x

and

y

absolute coordinates in meters. Each value represents the mean of three trials, with the standard deviation shown in parentheses. The lowest mean value in each column is highlighted in bold.

	JRDB-Known				JRDB-Unknown
Method	[0.0, 2.5]	[2.5, 5.0]	[5.0, 7.5]	[7.5, 10.0]	[0.0, 2.5]	[2.5, 5.0]	[5.0, 7.5]	[7.5, 10.0]
M only	3.93 (1.04)	4.13 (0.57)	4.92 (0.60)	5.37 (0.51)	4.61 (0.90)	4.55 (0.32)	5.30 (0.30)	5.55 (0.67)
M-F	3.35 (0.55)	3.70 (0.10)	4.54 (0.50)	4.73 (0.52)	3.97 (0.25)	3.99 (0.32)	4.46 (0.53)	4.76 (0.24)
M(G)-F	2.32 (0.42)	2.87 (0.63)	3.21 (0.57)	3.53 (0.55)	2.73 (0.28)	3.43 (0.54)	4.03 (0.58)	4.25 (0.71)
M(G)-FPU	2.84 (1.20)	3.12 (0.66)	3.33 (0.21)	3.53 (0.15)	3.51 (1.03)	3.89 (0.61)	4.27 (0.71)	4.49 (0.40)
M(G)-FPNi	2.23 (0.35)	2.50 (0.28)	3.05 (0.16)	3.30 (0.03)	2.89 (0.55)	2.96 (0.51)	3.49 (0.63)	3.72 (0.27)

Table S3 : Spatial range analysis across regions from center to boundary. Negative Log-Likelihood (NLL) with a fixed angular variance

20^{\circ}

is evaluated across non-overlapping region bins centered at the sensor origin. Ranges indicate the lower and upper bounding thresholds for both

x

and

y

absolute coordinates in meters. Each value represents the mean of three trials, with the standard deviation shown in parentheses. The lowest mean value in each column is highlighted in bold.

Figure S1 : Occlusion sensitivity analysis. These plot show NLL for each frame in static-robot sequences from JRDB.

Figure S2 : Indoor vs outdoor performance. These plots show NLL for each frame in test sequences from JRDB.

		JRDB-Known		JRDB-Unknown		SiT
Method	Input type	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)
BFF*	2D occ	1.17 (0.07)	4.80 (0.62)	1.53 (0.05)	4.82 (0.14)	1.34 (0.12)	4.07 (1.64)
Motion	3D pts	1.22 (0.08)	4.12 (0.62)	1.79 (0.29)	4.66 (0.35)	1.63 (0.04)	4.68 (0.86)
M(G)-FPNi	2D occ	1.05 (0.01)	2.58 (0.25)	1.43 (0.01)	3.60 (0.17)	1.35 (0.17)	3.21 (0.52)
M(G)-FPNi	3D pts	1.12 (0.07)	2.53 (0.27)	1.42 (0.05)	2.98 (0.51)	1.24 (0.11)	2.82 (0.74)

Table S4 : Input type comparison. 2D occ refers to models that take occupancy images as input following BFF [ 27 ] . Motion corresponds to the baseline that simply extends NeMo-map [ 34 ] . Each value represents the mean of three trials, with the standard deviation shown in parentheses. The lowest mean value per column is in bold.

Panel of Figure 16 — Figure S3 : Known location evaluation for NeMo-map. Gray dots are LiDAR points that correspond to obstacle classes in panoptic segmentation labels.

		gates-ai-lab-2019-04-17_0		huang-2-2019-01-25_1		tressider-2019-03-16_2
Model(/Train method)	Input	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)	NLL	( $20^{\circ}$ var)
NeMo-map	None	16.0 (0.91)	16.9 (0.90)	4.74 (0.92)	7.62 (0.14)	11.0 (1.65)	12.3 (1.22)
BFF*/Motion	2D occ	1.03 (0.08)	4.93 (1.08)	0.98 (0.04)	4.07 (1.23)	0.99 (0.04)	4.14 (1.62)
Ours-base/Motion	3D pts	1.18 (0.03)	3.86 (0.10)	0.83 (0.19)	2.94 (1.33)	1.06 (0.04)	2.81 (0.75)
BFF*/M(G)-FPNi	2D occ	1.05 (0.02)	2.81 (0.02)	0.75 (0.14)	1.64 (0.31)	0.85 (0.10)	2.49 (1.09)
Ours/M(G)-FPNi	3D pts	1.80 (0.10)	3.36 (0.26)	0.66 (0.21)	1.57 (0.77)	0.88 (0.19)	2.27 (1.20)

Table S5 : Comparison with NeMo-map [ 34 ] and extended BFF [ 27 ] on Known sequences in JRDB. 2D occ refers to models that take occupancy images as input following BFF [ 27 ] .

Panel of Figure 18 — Figure S4 : Frequency score and motion outputs of Motion(GS)-Freq , Motion(GS)-FreqPU , and Motion(GS)-FreqPNi . For each method, the left panel shows the predicted frequency score, with brighter colors indicating higher scores, and the right panel shows the motion map. Arrow opacity, length, and hue represent mixture weight, speed, and heading direction, respectively.

		T=12		T=18		T=24		T=48		T=96
Split	Method	ADE	FDE	ADE	FDE	ADE	FDE	ADE	FDE	ADE	FDE
JRDB-Known	ViTE	0.28	0.46	0.41	0.67	0.53	0.88	0.91	1.59	1.29	2.32
JRDB-Known	Ours	0.31	0.58	0.45	0.84	0.57	1.09	0.98	1.83	1.63	2.70
JRDB-Unknown	ViTE	0.42	0.65	0.61	0.98	0.79	1.30	1.47	2.64	2.46	4.56
JRDB-Unknown	Ours	0.47	0.89	0.69	1.35	0.89	1.77	1.53	3.10	2.36	4.52

Table S6 : Trajectory prediction errors on JRDB. All results are averaged over 3 training runs.

Pedestrian Behavior Modeling

Pedestrian behavior modeling predicts and simulates how people move, interact, choose routes, and respond to vehicles, robots, hazards, and surroundings.

See all 40 papers

EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations
Feb 26, 2026Iacopo Catalano, David Morilla-Cabello, Jorge Pena-Queralta +1Robot Navigation World Models for Robotics
I. Catalano is with the University of Turku, Finland. · E. Montijano and D. Morilla-Cabello are with the Instituto de Investigación en Ingeniería de Aragón, Universidad de Zaragoza, Spain. · J. Peña-Queralta is with the Centre for Artificial Intelligence, Zürich University of Applied Sciences, Winterthur, Switzerland.
JEPLO: Joint-Embedding Predictive Learning for LiDAR-Based Legged Locomotion
Sep 14, 2026Qihao Yuan, Yixuan Qiu, Ziyu Cao +2Joint-Embedding Predictive Architecture Terrain-Aware Robot Locomotion
Faculty of Science and Engineering, University of Groningen, The Netherlands · Department of Electrical Engineering, Linköping University, Sweden
MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States
Sep 7, 2026Juncen Long, Xiaofeng Jin, Gianluca Bardaro +2Pedestrian Behavior Modeling Pedestrian Trajectory Prediction
Department of Electronics, Information and Bioengineering, Politecnico di Milano, Milan, Italy

Feb 26, 2026J/K move · Enter open · S save

EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations

Iacopo Catalano, David Morilla-Cabello, Jorge Pena-Queralta, Eduardo Montijano

I. Catalano is with the University of Turku, Finland. · E. Montijano and D. Morilla-Cabello are with the Instituto de Investigación en Ingeniería de Aragón, Universidad de Zaragoza, Spain. · J. Peña-Queralta is with the Centre for Artificial Intelligence, Zürich University of Applied Sciences, Winterthur, Switzerland.

Robot Navigation World Models for Robotics

Open paper AskPDF arXiv

Efficient navigation in dynamic environments requires anticipating how motion patterns evolve beyond the robot's immediate perceptual range, enabling preemptive rather than purely reactive planning in crowded scenes. Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning, but constructing them traditionally requires global environment observations over extended periods of time. We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short egocentric video clips collected during robot operation. Our method learns to infer environment-wide motion tendencies from local dynamic cues using a video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision, allowing local observations to serve as predictive signals of global motion structure. Thanks to this, we offer the capacity to forecast future motion dynamics over the whole environment rather than merely extend past patterns in the robot's field of view. As a site-specific dynamic prior, EgoMoD replaces the external global sensing infrastructure required by prior MoD methods at inference time with standard onboard sensors. Experiments in large simulated environments show that EgoMoD predicts future MoDs under limited observability, while evaluation with real images showcases its zero-shot transferability to real systems.

Abstract

Pedestrian Behavior Modeling

Explore similar work

EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations

JEPLO: Joint-Embedding Predictive Learning for LiDAR-Based Legged Locomotion

MamMA: A Mamba-Based Pedestrian Trajectory Prediction Algorithm Considering Occupancy Map and Pedestrian Awareness States