HiMAP: History-aware Map-occupancy Prediction with Fallback
Authors: Yiming Xu, Yi Yang, Hao Cheng, Monika Sester
Organizations: Institute of Cartography and Geoinformatics, Leibniz University Hannover, Appelstr. 9a, 30167 Hannover, Germany · Institute of Information Processing, Leibniz University Hannover, Schneiderberg 32, 30167 Hannover, Germany · Faculty of Geo-Information Science and Earth Observation, University of Twente, 7522 NH Enschede, The Netherlands
Accurate motion forecasting is critical for autonomous driving, yet most predictors rely on multi-object tracking (MOT) with identity association, assuming that objects are correctly and continuously tracked. When tracking fails due to, e.g., occlusion, identity switches, or missed detections, prediction quality degrades and safety risks increase. We present HiMAP, a tracking-free, trajectory prediction framework that remains reliable under MOT failures. HiMAP converts past detections into spatiotemporally invariant historical occupancy maps and introduces a historical query module that conditions on the current agent state to iteratively retrieve agent-specific history from unlabeled occupancy representations. The retrieved history is summarized by a temporal map embedding and, together with the final query and map context, drives a DETR-style decoder to produce multi-modal future trajectories. This design lifts identity reliance, supports streaming inference via reusable encodings, and serves as a robust fallback when tracking is unavailable. On Argoverse~2, HiMAP achieves performance comparable to tracking-based methods while operating without IDs, and it substantially outperforms strong baselines in the no-tracking setting, yielding relative gains of 11% in FDE, 12% in ADE, and a 4% reduction in MR over a fine-tuned QCNet. Beyond aggregate metrics, HiMAP delivers stable forecasts for all agents simultaneously without waiting for tracking to recover, highlighting its practical value for safety-critical autonomy. The code is available under: https://github.com/XuYiMing83/HiMAP.
Figures & tables
Fig. 1 : Comparison between tracking-based prediction and our tracking-free fallback. Tracking-based methods rely on stable identity association, but fail when tracking breaks. Our tracking-free module provides a complementary safety mechanism by reconstructing history from historical occupancy maps, ensuring reliable prediction even under tracking failures. Our method matches the current agent state to historical occupancy maps, implicitly recovering its past states without explicit IDs for robust trajectory prediction.
Fig. 2 : Overview of our HiMAP pipeline. The framework consists of four main components: Agent and Map Encoding , which embeds agent states and HD map elements in spatiotemporally invariant local frames; Historical Occupancy Map Encoder , which aggregates per-frame agent–lane interactions into occupancy representations without relying on tracking IDs; Historical Query Module , which initializes a history-aware query from the current agent state and iteratively attends to past occupancy maps to reconstruct agent-specific trajectories; Future Trajectory Decoder , a DETR-style query decoder that generates multi-modal predictions from the reconstructed history, final query, and map context. This design provides a robust fallback when tracking fails, enabling reliable forecasting directly from historical detections.
Method
w/o Tracking
minFDE 1 ↓
minADE 1 ↓
minFDE 6 ↓
minADE 6 ↓
MR 6 ↓
b-minFDE 6 ↓
FRM [ 34 ]
5.93
2.37
1.81
0.89
0.29
2.47
HDGT [ 35 ]
5.37
2.08
1.60
0.84
0.21
2.24
MTR [ 13 ]
4.39
1.74
1.44
0.73
0.15
1.98
HPTR [ 36 ]
4.61
1.84
1.43
0.73
0.19
2.03
HeteroGCN [ 14 ]
4.40
1.72
1.34
0.69
0.18
1.90
ProphNet [ 15 ]
4.74
1.80
1.33
0.68
0.18
1.88
TABLE I : Results on the Argoverse 2 motion forecasting benchmark. All leaderboard methods assume perfect tracking, whereas HiMAP is evaluated under the no-tracking setting (historical detections only). HiMAP achieves competitive results and clearly outperforms QCNet baselines when tracking fails.
Method
minFDE 6 ↓
minADE 6 ↓
MR 6 ↓
QCNet [ 3 ]
0.86
0.63
0.08
QCNet w/o tracking
3.05
1.99
0.37
QCNet w/ fine-tuning
1.24
0.79
0.12
HiMAP (Ours)
0.94
0.66
0.09
TABLE II : Results on the Argoverse 1 motion forecasting validation set.
Index
Historical Query Module
Metrics
recurrent query
hi. occ. map
hi. query init.
hi. temp. map
reconstruct hi. ADE ↓
minFDE 6 ↓
minADE 6 ↓
MR 6 ↓
b-minFDE 6 ↓
1
—–
1.46
0.85
0.21
2.06
2
✓
0.27
1.40
0.83
0.19
2.01
3
✓
✓
0.21
1.38
0.79
0.18
2.00
4
✓
✓
✓
0.14
1.37
0.79
0.17
1.99
5
✓
✓
✓
✓
0.12
1.33
0.76
0.17
1.96
TABLE III : Ablation studies on Historical Query Module components. Experimental results are based on the Argoverse 2 validation set.
Index
Decoder
Metrics
current agent status & map
hi. temporal map
updated query
recurrent
minFDE 6 ↓
minADE 6 ↓
MR 6 ↓
b-minFDE 6 ↓
1
✓
✓
1.46
0.85
0.21
2.06
2
✓
✓
✓
1.39
0.79
0.18
2.01
3
✓
✓
✓
1.41
0.83
0.19
2.03
4
✓
✓
✓
✓
1.33
0.76
0.17
1.96
TABLE IV : Ablation studies on decoder inputs. Results are reported on the Argoverse 2 validation set.
Fig. 3 : Comparison between HiMAP and QCNet on the Argoverse 2 validation set under different numbers of available tracking steps. HiMAP maintains fixed performance without requiring tracking, shown as horizontal dashed lines. QCNet gradually improves as more tracked history is available, surpassing HiMAP after 13–14 steps. The green curve indicates the average distance traveled per timestep.
Fig. 4 : Qualitative comparison of trajectory prediction results on the Argoverse 2 validation set. The blue box denotes the target agent. Blue lines show the agent’s past trajectory (not used during prediction), green lines represent the ground-truth future trajectory, and orange lines indicate predicted trajectories, with darker colors corresponding to higher predicted probabilities, and the colorbar is on the right.
reconstructed historical steps Th
minFDE 6 ↓
minADE 6 ↓
10
1.37
0.78
20
1.34
0.76
30
1.33
0.76
40
1.35
0.77
50
1.36
0.77
TABLE V : Ablation studies on reconstructed historical steps Th . Results are reported on the Argoverse 2 validation set.
Trajectory prediction of surrounding agents is a prerequisite for safe planning and decision making in autonomous driving. Without high-definition (HD) maps, sensor-derived bird's-eye-view (BEV) features provide no explicit lane topology or drivable-area priors, making it inherently difficult to ground each agent in its surrounding scene context. Moreover, physical feasibility remains difficult to capture through data-driven learning alone, as kinematic constraints on agent motion cannot be explicitly encoded without structured supervision. Existing map-free predictors extract scene context in an agent-agnostic manner through a single fusion step and treat physical constraints only as output-level penalties, leaving both challenges unaddressed. We propose SIPTraj, a map-free trajectory prediction framework that jointly addresses scene grounding and physical feasibility. SIPTraj introduces a Hierarchical Agent-Scene Encoder (HASE) progressively grounding each agent in agent-guided scene evidence and refining inter-agent relations within the scene-grounded space. To tackle physical infeasibility in predicted trajectories, we develop a Physics-Guided Iterative Decoder (PGID). It conditions decoding on instantaneous kinematic states, propagating physical supervision into internal representations rather than output trajectories alone. Extensive experiments on nuScenes and Argoverse 2 Sensor show that SIPTraj surpasses prior map-free predictors and strong map-based baselines without any HD map at inference. Our code will be released as open-source.
Feifei Liu, Zejun Wei, Haozhe Wang +6
School of Data Science and Engineering, Xingzhi College, South China Normal University, Shanwei, 516600, China · Department of Electronic and Computer Engineering, Hong Kong University of Science and Technology, Hong Kong, China · University of Macau, Macau, China +2
Safe navigation in dynamic environments requires anticipating future environmental states to account for spatiotemporal risks, specifically when and where collisions may occur. To this end, occupancy grid map (OGM) prediction has been widely adopted as an effective approach. However, existing OGM-based navigation methods often struggle to achieve accurate and efficient forecasting and fail to fully exploit the temporal information in predicted OGMs during planning. To address these challenges, we propose FORTE, a navigation framework that directly exploits the spatiotemporal evolution of predicted occupancy from the perspectives of spatiotemporal occupancy overlap and occupancy directivity. Based on these properties, FORTE evaluates multiple topology-distinct paths and selects the suitable one without explicit object detection or tracking. To support online planning, we formulate a latent diffusion model-based OGM predictor that generates the entire forecast horizon in a non-autoregressive manner while maintaining temporal consistency through temporal shift modules. Extensive evaluations demonstrate that FORTE outperforms state-of-the-art baselines. For prediction, FORTE achieves up to 215.3% higher IoU and 5.24x faster inference; for navigation, it yields up to a 3.5x higher success rate.
Hahjin Lee, Young J. Kim
Department of Computer Science and Engineering at Ewha Womans University in Korea
Unified models for object detection and trajectory forecasting aim to merge perception and prediction for autonomous driving, refining actor trajectories directly over shared bird's-eye-view (BEV) images rasterized from LiDAR and high-definition maps. Their accuracy on dynamic, moving actors, however, remains the hardest part of the task, and the strongest such model, DeTra, has no public implementation. We contribute an openly released DeTra reimplementation with documented approximations, and on top of it MC-DeTra: a family of motion-consistency mechanisms that add supervision through two annotation-derived auxiliary signals -- each actor's observed past motion and the occupancy of the surrounding traffic that forms its social context -- and one inter-output consistency constraint that aligns an actor's predicted heading with its predicted direction of motion. Every proposed loss is train-only and inference-safe: it shapes the shared BEV representation during training and is removed at test time, adding no inference latency. On the Waymo Open Dataset, evaluated under a strict, detection-conditioned forecasting protocol, MC-DeTra improves dynamic, socially-situated trajectory forecasting while preserving or improving detection accuracy; a gradient-based loss-calibration analysis exposes how the auxiliary objectives compete at the shared backbone, and our ablation identifies which signals contribute most. We release code, configurations, and evaluation tooling at https://github.com/diuzhevVlad/MC-DeTra.
Vladislav Diuzhev, Dmitry Yudin
Moscow Institute of Physics and Technology, Moscow, Russia · Artificial Intelligence Research Institute, Moscow, Russia