Destination Support Restoration for Finite-Set Multimodal Trajectory Prediction
Authors: Fengrui Liu, Jiajun Peng, Duo Peng, Feng Liu
Organizations: School of Computer Science and Technology, East China Normal University, Shanghai, China · School of Data Science, University of Science and Technology of China, Hefei, China · School of Computer Science and Technology, Tongji University, Shanghai, China · School of Psychology, Shanghai Jiao Tong University, Shanghai, China
Robots operating around pedestrians often reason over a finite set of predicted human futures. Repeated online updates can concentrate this limited prediction budget on dominant destinations and leave plausible alternatives underrepresented or absent, removing those alternatives from the finite representation available to downstream decision making. We introduce Destination Support Restoration (DSR), a causal post-selection operator that repairs destination support without retraining the host predictor or increasing the maintained set size. At a repair step, DSR evaluates a temporary destination-stratified candidate bank from the observed prefix, converts candidate evidence into integer target counts, protects representatives of active modes, and reallocates redundant surplus hypotheses to deficient modes. The maintained and returned sets retain exactly N hypotheses, and DSR replaces at most ⌈ρN⌉ entries. Protected representatives preserve current categorical support; lineage-aware particle filters also preserve surviving resampling ancestors. Each replacement reduces the allocation mismatch to the evidence-driven target by one. On the complete 3,719-trajectory Edinburgh protocol over three seeds, DSR reduces MIF weighted ADE and FDE by 13.36% and 13.30% at N=64. Paired integrations with CLiFF, PPT, causal GDTS, Social Informer, and PECNet improve both metrics in every evaluated pair. These results show that finite-set support allocation is a useful prediction-side control point when a fixed hypothesis set serves as the interface to downstream systems.
Figures & tables
Fig. 1: Overview of Destination Support Restoration (DSR). (a) An observed prefix remains compatible with multiple destination modes. (b) A fixed-size maintained set concentrates on dominant modes. (c) Resampling may eliminate a still-plausible destination from the finite representation. (d) DSR protects surviving support and reallocates surplus slots toward evidence-supported deficits while returning the same number of hypotheses. (e) The figure illustrates the lineage-aware case; set-based hosts use protected mode representatives instead.
Fig. 2: Host update and DSR pipeline. The host first propagates and weights its maintained hypotheses and then checks the effective sample size. When ESS<ηN , DSR maps evidence from a temporary destination-stratified candidate bank to target counts, protects active-mode representatives, replaces only unprotected surplus hypotheses, and returns N equally weighted hypotheses to the host.
Predictor
Configuration
ADE/AOE ↓
FDE/FOE ↓
NLL ↓
DSR Reduction
Social Force [ 16 , 26 ] †
-
3.1240
3.9090
-
-
LSTM [ 16 , 27 ] †
-
2.1320
3.0050
-
-
Social LSTM [ 1 , 16 ] †
-
1.5240
2.5100
-
-
Attention LSTM [ 16 , 28 ] †
-
0.9860
1.3110
-
-
Social GAN [ 2 , 16 ] †
340 outputs
1.0420
2.0880
-
-
MIF [ 16 ]
N=64
0.6594
1.2420
5.1041
-
TABLE I: Results on Edinburgh. Lower is better. We include historical AOE/FOE values marked with † as context because they follow a different protocol. NLL comparisons are paired within host; for CLiFF, we measure the DSR reduction relative to GoalBridge.
Host
Scene
wADE ↓
wFDE ↓
MIF
ETH
1.2456→1.2255
2.6604→2.5983
HOTEL
0.4133→0.4066
0.9036→0.8825
UNIV
0.7292→0.7277
1.5781→1.5699
ZARA1
0.5258→0.5185
1.1513→1.1306
ZARA2
0.4222→0.4171
0.9280→0.9134
CLiFF
ETH
1.1324→1.1205
2.2120→2.1076
TABLE II: Paired ETH/UCY results over three seeds. Each cell reports Host → DSR; lower is better. UNIV denotes stu03 for MIF and CLiFF.
Variant
wADE ↓
wFDE ↓
Full DSR
0.5713
1.0768
Zero repair ( ρ=0 )
0.69391
1.30667
w/o lineage + surplus protection
0.58088
1.09413
Random donor
0.59654
1.09956
Random candidate
0.58372
1.09875
TABLE III: Component ablation on MIF with N=64 . Lower is better.
Mechanism
Random Reset
DSR
Gain (pp)
Immediate revival
2.94%
24.11%
21.17
Historical recovery
3.15%
22.21%
19.07
Pre-extinction increase
1.98%
22.72%
20.74
TABLE IV: Support-repair rates on MIF with N=64 . Gain is the cluster-paired difference between method-specific rates in percentage points (pp).
Many high-stakes decisions depend on forecasts made before outcomes are known. In this future prediction setting, the central challenge is that public evidence evolves over time, while the main supervision signal arrives only after resolution: the realized outcome mainly assesses final correctness, offering only coarse guidance on what to track, what to verify, and which judgments to leave uncertain along the way. Our key observation is that revisiting the same unresolved question over time creates informative temporal contrasts across evolving evidence and repeated forecasts, exposing what earlier attempts missed before resolution and yielding a diagnostic signal we call the pre-resolution signal. We instantiate this idea in Milkyway, a future prediction agent with a persistent future prediction harness, an editable external state that stores reusable procedural guidance across revisits to the same unresolved question. As the same unresolved question is revisited, Milkyway extracts pre-resolution signals from evolving evidence and repeated forecasts, uses them to update the harness, and improves later forecasts on that question before resolution. After resolution, the realized outcome serves as a post-resolution check of provisional updates. On the FutureX and FutureWorld benchmarks, Milkyway achieves strong performance against competitive baselines, and a mechanism study suggests that the gains stem from harness evolution driven by pre-resolution signals rather than repeated prediction alone.
Chuyang Wei, Maohang Gao, Zhixin Han +12
University of Science and Technology of China · Zhongguancun Academy, Beijing, China · IIIS, Tsinghua University
Can a trace explaining model execution also compute the changes needed for a specified alternative prediction? We propose trace-guided intervention search, which uses executable reasoning traces as an intermediate representation for intervention synthesis. A Specified-Foil Counterfactual edits past events so that a frozen temporal predictor selects a designated foil. Our method constructs facts and replacement values from completed original and foil executions and recovered unmet conditions. Proposal generation constructs edits and selects candidates within a fixed cap; exact replay verifies foil top-1 outcomes among retained edits and compositions. Implemented in LiFTER for continuous-time dynamic graphs (CTDGs) and TLogic for temporal knowledge graphs (TKGs), the method improves success over coordinate-based proposal generation by 13.7-34.7 percentage points on four CTDG datasets and 60.0-83.3 points on two TKG datasets under matched downstream search and a proposal cap of 32. Separate shared-candidate comparisons retain 85.7-93.6% of black-box greedy's CTDG success rate with 75.0-80.0% fewer predictor evaluations. A Pulse case study confirms simulator-level survival for five of six interventions. Executable traces thus provide both explanatory evidence and a reusable computational representation for constructing and testing specified alternatives.
Minwoo Yu, Young-guk Ha
Smart Computing Laboratory, Department of Computer Science & Engineering, Konkuk University, Seoul 05029, Republic of Korea
Trajectory prediction methods have demonstrated remarkable capabilities in capturing complex motion patterns. However, existing methods rely on global state assumptions, suffer from insufficient belief inference under partial observability, and lack cognitive behavioral constraints in prediction. These limitations severely compromise both deployment feasibility and physical plausibility in real-world settings. In this work, we propose FEP-Diff, an agent-centric trajectory prediction framework grounded in the Free Energy Principle, aimed at achieving cognitively plausible predictions under realistic constraints. Specifically, a dual-branch spatiotemporal encoder extracts ego-motion dynamics and social interaction cues from local observations. Building upon this, a goal-conditioned belief learner infers multimodal latent belief distributions optimized via a free-energy objective, with a social consistency constraint on the local neighborhood graph to promote cognitive alignment among neighboring agents. Finally, a residual diffusion trajectory generator is conditioned on the learned belief representations with token-level proxy conditioning, producing precise and diverse future predictions. Extensive experiments on five public benchmarks demonstrate that FEP-Diff consistently outperforms state-of-the-art methods under restricted observability. Code: https://anonymous.4open.science/r/FEP-Diff-8876.
Yanping Wu, Ji Zhang, Hao Chen +2
University of Glasgow Glasgow, UK · Southwest Jiaotong University Chengdu, China