EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations

Iacopo Catalano; David Morilla-Cabello; Jorge Pena-Queralta; Eduardo Montijano

cs.ROFeb 26, 2026

EgoMoD: Predicting Global Maps of Dynamics from Local Egocentric Observations

Authors: Iacopo Catalano, David Morilla-Cabello, Jorge Pena-Queralta, Eduardo Montijano

Organizations: I. Catalano is with the University of Turku, Finland. · E. Montijano and D. Morilla-Cabello are with the Instituto de Investigación en Ingeniería de Aragón, Universidad de Zaragoza, Spain. · J. Peña-Queralta is with the Centre for Artificial Intelligence, Zürich University of Applied Sciences, Winterthur, Switzerland.

Abstract

Efficient navigation in dynamic environments requires anticipating how motion patterns evolve beyond the robot's immediate perceptual range, enabling preemptive rather than purely reactive planning in crowded scenes. Maps of Dynamics (MoDs) offer a structured representation of motion tendencies in space useful for long-term global planning, but constructing them traditionally requires global environment observations over extended periods of time. We introduce EgoMoD, the first approach that learns to predict future MoDs directly from short egocentric video clips collected during robot operation. Our method learns to infer environment-wide motion tendencies from local dynamic cues using a video- and pose-conditioned architecture trained with MoDs computed from external observations as privileged supervision, allowing local observations to serve as predictive signals of global motion structure. Thanks to this, we offer the capacity to forecast future motion dynamics over the whole environment rather than merely extend past patterns in the robot's field of view. As a site-specific dynamic prior, EgoMoD replaces the external global sensing infrastructure required by prior MoD methods at inference time with standard onboard sensors. Experiments in large simulated environments show that EgoMoD predicts future MoDs under limited observability, while evaluation with real images showcases its zero-shot transferability to real systems.

Figures & tables

Fig. 1 : EgoMoD learns to predict global maps of dynamics (bottom) from local video observations (top) from the egocentric vision of the robot, leveraging priors from a privileged expert.

Fig. 2 : EgoMoD architecture overview. A short egocentric video sequence is processed by a frozen foundation video encoder to extract spatio-temporal features. These features are combined with a learned pose embedding via a Transformer encoder, reshaped to a spatial grid and decoded through convolutional and upsampling layers to produce allocentric Maps of Dynamics.

Panel of Figure 3 — Fig. 3 : Example of construction of our Maps of Dynamics

Parameter	Value	Parameter	Value
Map of Dynamics		Regularization
Direction bins $B$	8	Input (patch) dropout	0.2
Temporal horizon $T$	10 / 20 s
Grid cell size	0.30 m
Loss Weights		Optimization
Valid region weight $w_{\text{valid}}$	5.0	Learning rate	$1\times 10^{-4}$

TABLE I : Model, training, and loss hyperparameters.

Panel of Figure 5 — Fig. 4 : Overview of the hospital simulation environment shown from a top-down view (left) and a perspective view (right).

Horizon	Observability	Method	Flow			Entropy			Direction		ECR
Horizon	Observability	Method	$\downarrow$ MSE	$\downarrow$ MAE	$\uparrow$ SSIM	$\downarrow$ MSE	$\downarrow$ MAE	$\uparrow$ SSIM	$\uparrow$ Acc.	$\uparrow$ IoU	$\uparrow$ ECR
10 s	Local	Mean-MoD	59.26	5.38	0.52	19.26	4.27	0.23	0.70	0.23	0.00
		Local-StefMap [ 5 ]	204.86	8.83	0.31	30.66	5.33	0.19	0.29	0.04	$-$
		EgoMoD (ours)	48.43	4.55	0.66	13.00	2.86	0.30	0.44	0.11	0.23
	Global	Mean-MoD	63.94	5.65	0.84	19.26	4.27	0.73	0.69	0.26	0.00
		Local-StefMap [ 5 ]	185.24	8.35	0.81	33.3	5.74	0.73	0.24	0.02	$-$

TABLE II : Quantitative comparison on the hospital simulation environment. EgoMoD outperforms both baselines on flow metrics and achieves consistent improvements across evaluation scopes and horizons. Local-StefMap stores only within-FOV predictions, making global ECR unavailable (

-

Ablation	Flow	Entropy	Direction	ECR
Ablation	$\downarrow$ MAE	$\downarrow$ MAE	$\uparrow$ Acc.	$\uparrow$ ECR
baseline	4.78	2.92	0.42	0.21
vision: no_motion	4.87	3.15	0.41	0.19
vision: motion	4.94	2.67	0.36	0.21
vision: static	5.32	2.28	0.36	0.22
vision: OOD	5.75	3.91	0.24	0.06

TABLE III : Ablation study on the model input.

Fig. 5 : Qualitative results showing predicted flow, entropy, and direction maps from our model on test scenes, compared to ground truth.

Fig. 6 : Photorealistic office simulated environment

Panel of Figure 10 — Fig. 7 : Qualitative results for the MoD predicted by our network on a real video clip.

Panel of Figure 11 — Fig. 8 : ECR-based A* planning on EgoMoD’s predicted direction map. Direction- aware costs route A $\rightarrow$ B with the dominant flow (left) and redirect B $\rightarrow$ A through an alternative path (right) to avoid moving against it.

Robot Navigation

Robot navigation studies how robots perceive, localize, plan, and move safely toward goals in static or changing environments.

See all 554 papers

EgoNeMo: Transferable Map of Pedestrian Dynamics via Egocentric LiDAR Scan
Sep 5, 2026Azusa Sawada, Allan Wang, Hideo Saito +1Pedestrian Behavior Modeling Domain Generalization
NEC Corporation · Miraikan - The National Museum of Emerging Science and Innovation · Keio University +1
NeMo-map: Neural Implicit Flow Fields for Spatio-Temporal Motion Mapping
Oct 16, 2025Yufei Zhu, Shih-Min Yang, Andrey Rudenko +3Implicit Neural Representations Human Trajectory Prediction
Orebro University, Sweden · Technical University of Munich, Germany · Aalto University, Finland
EgoCogNav: Cognition-aware Human Egocentric Navigation
Nov 15, 2025Zhiwen Qiu, Ziang Liu, Wenqian Niu +2Robot Navigation Egocentric Video Understanding
Cornell University, Ithaca, NY, USA · Georgia Institute of Technology, Atlanta, GA, USA

Sep 5, 2026J/K move · Enter open · S save

EgoNeMo: Transferable Map of Pedestrian Dynamics via Egocentric LiDAR Scan

Azusa Sawada, Allan Wang, Hideo Saito, Aaron Steinfeld

NEC Corporation · Miraikan - The National Museum of Emerging Science and Innovation · Keio University+1

Pedestrian Behavior Modeling Domain Generalization Pedestrian Trajectory Prediction

Open paper AskPDF arXiv

This paper proposes a transferable Map of Dynamics (MoD) framework that generalizes to unknown environments using only egocentric 3D LiDAR point clouds to overcome the long-standing limitation of traditional MoD methods. While MoDs are essential for encoding human motion characteristics to enable accurate pedestrian trajectory prediction or safe robot navigation, traditional approaches suffer from site-specificity, requiring exhaustive trajectory accumulation at every new location. Extending recent advances in neural implicit modeling, our framework trains a continuous, LiDAR-based MoD estimator across diverse environments. To mitigate the inherent sparsity and temporal bias of real-world trajectory data, we introduce a position-balanced sampling strategy and a multi-task learning architecture that jointly predicts motion distributions and a spatial frequency score map. The latter is further augmented by visibility-aware losses to compensate for incomplete observation data. Comprehensive experiments demonstrate that our method effectively reconstructs underlying motion maps even in unknown locations from a single instantaneous LiDAR scan, despite highly sparse training data. Finally, we show that our improvements enhance the reliability of downstream trajectory prediction.

Abstract

Robot Navigation

Explore similar work

EgoNeMo: Transferable Map of Pedestrian Dynamics via Egocentric LiDAR Scan

NeMo-map: Neural Implicit Flow Fields for Spatio-Temporal Motion Mapping

EgoCogNav: Cognition-aware Human Egocentric Navigation