HiMAP: History-aware Map-occupancy Prediction with Fallback

Yiming Xu; Yi Yang; Hao Cheng; Monika Sester

cs.CVFeb 19, 2026

HiMAP: History-aware Map-occupancy Prediction with Fallback

Authors: Yiming Xu, Yi Yang, Hao Cheng, Monika Sester

Organizations: Institute of Cartography and Geoinformatics, Leibniz University Hannover, Appelstr. 9a, 30167 Hannover, Germany · Institute of Information Processing, Leibniz University Hannover, Schneiderberg 32, 30167 Hannover, Germany · Faculty of Geo-Information Science and Earth Observation, University of Twente, 7522 NH Enschede, The Netherlands

Abstract

Accurate motion forecasting is critical for autonomous driving, yet most predictors rely on multi-object tracking (MOT) with identity association, assuming that objects are correctly and continuously tracked. When tracking fails due to, e.g., occlusion, identity switches, or missed detections, prediction quality degrades and safety risks increase. We present HiMAP, a tracking-free, trajectory prediction framework that remains reliable under MOT failures. HiMAP converts past detections into spatiotemporally invariant historical occupancy maps and introduces a historical query module that conditions on the current agent state to iteratively retrieve agent-specific history from unlabeled occupancy representations. The retrieved history is summarized by a temporal map embedding and, together with the final query and map context, drives a DETR-style decoder to produce multi-modal future trajectories. This design lifts identity reliance, supports streaming inference via reusable encodings, and serves as a robust fallback when tracking is unavailable. On Argoverse~2, HiMAP achieves performance comparable to tracking-based methods while operating without IDs, and it substantially outperforms strong baselines in the no-tracking setting, yielding relative gains of 11% in FDE, 12% in ADE, and a 4% reduction in MR over a fine-tuned QCNet. Beyond aggregate metrics, HiMAP delivers stable forecasts for all agents simultaneously without waiting for tracking to recover, highlighting its practical value for safety-critical autonomy. The code is available under: https://github.com/XuYiMing83/HiMAP.

Figures & tables

Fig. 1 : Comparison between tracking-based prediction and our tracking-free fallback. Tracking-based methods rely on stable identity association, but fail when tracking breaks. Our tracking-free module provides a complementary safety mechanism by reconstructing history from historical occupancy maps, ensuring reliable prediction even under tracking failures. Our method matches the current agent state to historical occupancy maps, implicitly recovering its past states without explicit IDs for robust trajectory prediction.

Fig. 2 : Overview of our HiMAP pipeline. The framework consists of four main components: Agent and Map Encoding , which embeds agent states and HD map elements in spatiotemporally invariant local frames; Historical Occupancy Map Encoder , which aggregates per-frame agent–lane interactions into occupancy representations without relying on tracking IDs; Historical Query Module , which initializes a history-aware query from the current agent state and iteratively attends to past occupancy maps to reconstruct agent-specific trajectories; Future Trajectory Decoder , a DETR-style query decoder that generates multi-modal predictions from the reconstructed history, final query, and map context. This design provides a robust fallback when tracking fails, enabling reliable forecasting directly from historical detections.

Method	w/o Tracking	minFDE 1 $\downarrow$	minADE 1 $\downarrow$	minFDE 6 $\downarrow$	minADE 6 $\downarrow$	MR 6 $\downarrow$	b-minFDE 6 $\downarrow$
FRM [ 34 ]		5.93	2.37	1.81	0.89	0.29	2.47
HDGT [ 35 ]		5.37	2.08	1.60	0.84	0.21	2.24
MTR [ 13 ]		4.39	1.74	1.44	0.73	0.15	1.98
HPTR [ 36 ]		4.61	1.84	1.43	0.73	0.19	2.03
HeteroGCN [ 14 ]		4.40	1.72	1.34	0.69	0.18	1.90
ProphNet [ 15 ]		4.74	1.80	1.33	0.68	0.18	1.88

TABLE I : Results on the Argoverse 2 motion forecasting benchmark. All leaderboard methods assume perfect tracking, whereas HiMAP is evaluated under the no-tracking setting (historical detections only). HiMAP achieves competitive results and clearly outperforms QCNet baselines when tracking fails.

Method	minFDE 6 $\downarrow$	minADE 6 $\downarrow$	MR 6 $\downarrow$
QCNet [ 3 ]	0.86	0.63	0.08
QCNet w/o tracking	3.05	1.99	0.37
QCNet w/ fine-tuning	1.24	0.79	0.12
HiMAP (Ours)	0.94	0.66	0.09

TABLE II : Results on the Argoverse 1 motion forecasting validation set.

Index	Historical Query Module				Metrics
Index	recurrent query	hi. occ. map	hi. query init.	hi. temp. map	reconstruct hi. ADE $\downarrow$	minFDE 6 $\downarrow$	minADE 6 $\downarrow$	MR 6 $\downarrow$	b-minFDE 6 $\downarrow$
1					—–	1.46	0.85	0.21	2.06
2	✓				0.27	1.40	0.83	0.19	2.01
3	✓	✓			0.21	1.38	0.79	0.18	2.00
4	✓	✓	✓		0.14	1.37	0.79	0.17	1.99
5	✓	✓	✓	✓	0.12	1.33	0.76	0.17	1.96

TABLE III : Ablation studies on Historical Query Module components. Experimental results are based on the Argoverse 2 validation set.

Index	Decoder				Metrics
Index	current agent status & map	hi. temporal map	updated query	recurrent	minFDE 6 $\downarrow$	minADE 6 $\downarrow$	MR 6 $\downarrow$	b-minFDE 6 $\downarrow$
1	✓			✓	1.46	0.85	0.21	2.06
2	✓	✓		✓	1.39	0.79	0.18	2.01
3	✓	✓	✓		1.41	0.83	0.19	2.03
4	✓	✓	✓	✓	1.33	0.76	0.17	1.96

TABLE IV : Ablation studies on decoder inputs. Results are reported on the Argoverse 2 validation set.

Fig. 3 : Comparison between HiMAP and QCNet on the Argoverse 2 validation set under different numbers of available tracking steps. HiMAP maintains fixed performance without requiring tracking, shown as horizontal dashed lines. QCNet gradually improves as more tracked history is available, surpassing HiMAP after 13–14 steps. The green curve indicates the average distance traveled per timestep.

Fig. 4 : Qualitative comparison of trajectory prediction results on the Argoverse 2 validation set. The blue box denotes the target agent. Blue lines show the agent’s past trajectory (not used during prediction), green lines represent the ground-truth future trajectory, and orange lines indicate predicted trajectories, with darker colors corresponding to higher predicted probabilities, and the colorbar is on the right.

reconstructed historical steps $T_{h}$	minFDE 6 $\downarrow$	minADE 6 $\downarrow$
10	1.37	0.78
20	1.34	0.76
30	1.33	0.76
40	1.35	0.77
50	1.36	0.77

TABLE V : Ablation studies on reconstructed historical steps

T_{h}

. Results are reported on the Argoverse 2 validation set.

Trajectory Prediction

Trajectory prediction forecasts how people, vehicles, robots, and other dynamic systems will move from observed histories and context.

See all 186 papers

SIPTraj: Map-Free End-to-End Trajectory Prediction via Physics-Guided Scene Interaction
Aug 1, 2026Feifei Liu, Zejun Wei, Haozhe Wang +6Multi-Agent Trajectory Prediction Autonomous Driving
School of Data Science and Engineering, Xingzhi College, South China Normal University, Shanwei, 516600, China · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China · University of Macau, Macau, China +2
FORTE: Forecasting Occupancy for Spatiotemporal Risk-Aware Planning in Dynamic Environments
Sep 30, 2026Hahjin Lee, Young J. KimRobot Navigation Safe Motion Planning
Department of Computer Science and Engineering at Ewha Womans University in Korea
MC-DeTra: Motion-Consistent Joint Object Detection and Socially-Aware Trajectory Forecasting in Bird's-Eye-View Images
Sep 10, 2026Vladislav Diuzhev, Dmitry YudinMulti-Agent Trajectory Prediction Autonomous Driving
Moscow Institute of Physics and Technology, Moscow, Russia · Artificial Intelligence Research Institute, Moscow, Russia

Aug 1, 2026J/K move · Enter open · S save

SIPTraj: Map-Free End-to-End Trajectory Prediction via Physics-Guided Scene Interaction

Feifei Liu, Zejun Wei, Haozhe Wang, Yazhi Ye+5

School of Data Science and Engineering, Xingzhi College, South China Normal University, Shanwei, 516600, China · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China · University of Macau, Macau, China+2

Multi-Agent Trajectory Prediction Autonomous Driving Agent Trajectory Analysis Trajectory Prediction

Open paper AskPDF arXiv

Trajectory prediction of surrounding agents is a prerequisite for safe planning and decision making in autonomous driving. Without high-definition (HD) maps, sensor-derived bird's-eye-view (BEV) features provide no explicit lane topology or drivable-area priors, making it inherently difficult to ground each agent in its surrounding scene context. Moreover, physical feasibility remains difficult to capture through data-driven learning alone, as kinematic constraints on agent motion cannot be explicitly encoded without structured supervision. Existing map-free predictors extract scene context in an agent-agnostic manner through a single fusion step and treat physical constraints only as output-level penalties, leaving both challenges unaddressed. We propose SIPTraj, a map-free trajectory prediction framework that jointly addresses scene grounding and physical feasibility. SIPTraj introduces a Hierarchical Agent-Scene Encoder (HASE) progressively grounding each agent in agent-guided scene evidence and refining inter-agent relations within the scene-grounded space. To tackle physical infeasibility in predicted trajectories, we develop a Physics-Guided Iterative Decoder (PGID). It conditions decoding on instantaneous kinematic states, propagating physical supervision into internal representations rather than output trajectories alone. Extensive experiments on nuScenes and Argoverse 2 Sensor show that SIPTraj surpasses prior map-free predictors and strong map-based baselines without any HD map at inference. Our code will be released as open-source.

Abstract

Trajectory Prediction

Explore similar work

SIPTraj: Map-Free End-to-End Trajectory Prediction via Physics-Guided Scene Interaction

FORTE: Forecasting Occupancy for Spatiotemporal Risk-Aware Planning in Dynamic Environments

MC-DeTra: Motion-Consistent Joint Object Detection and Socially-Aware Trajectory Forecasting in Bird's-Eye-View Images