SpecFlow: Flow Matching in the Acceleration Spectrum for Multi-Agent Trajectory Forecasting
Organizations: National Space Science Center, Chinese Academy of Sciences, Beijing 100190, China · University of Chinese Academy of Sciences, Beijing 101408, China
Abstract
Multimodal trajectory forecasting must cover diverse future behaviors without sacrificing dynamical coherence. Most generative forecasters evolve frame-wise positions directly, leaving the relationships among position, velocity, and acceleration implicit. We present SpecFlow, a conditional flow-matching framework that treats the Discrete Cosine Transform (DCT) coefficients of future accelerations as its generative state. SpecFlow learns a transport from Gaussian noise to acceleration spectra and reconstructs velocity and position through fixed analytical integration initialized by the last observed state. A joint phase-space objective supervises all three kinematic levels. To represent destination uncertainty without future information, an endpoint predictor produces candidate destinations and confidence scores solely from observed context, and the highest-confidence candidate conditions the spectral denoiser. Alternating attention over trajectory hypotheses and agents captures multimodality and interaction. Under best-of-20 evaluation, SpecFlow achieves average minADE/minFDE of 0.15/0.22 on ETH/UCY, 0.69/0.86 at four seconds on NBA, and 7.38/11.57 on SDD. It establishes state-of-the-art results across all five ETH/UCY scenes and achieves the lowest four-second ADE on NBA with a tied-best FDE, while remaining competitive on SDD. Together, these results demonstrate that acceleration-spectrum flow matching provides an effective structured generative state across smooth pedestrian motion, coordinated multi-agent sports dynamics, and heterogeneous outdoor trajectories. Code will be made publicly available.
Figures & tables
| Method | ETH | HOTEL | UNIV | ZARA1 | ZARA2 | AVG |
| Social-GAN | 0.87/1.62 | 0.67/1.37 | 0.76/1.52 | 0.35/0.68 | 0.42/0.84 | 0.61/1.21 |
| Social-STGCNN | 0.64/1.11 | 0.49/0.85 | 0.44/0.79 | 0.34/0.53 | 0.30/0.48 | 0.44/0.75 |
| Social-VAE | 0.41/0.58 | 0.13/0.19 | 0.21/0.36 | 0.17/0.29 | 0.13/0.22 | 0.21/0.33 |
| MID | 0.39/0.66 | 0.13/0.22 | 0.22/0.45 | 0.17/0.30 | 0.13/0.27 | 0.21/0.38 |
| LED | 0.39/0.58 | 0.11 /0.17 | 0.26/0.43 | 0.18/0.26 | 0.13/0.22 | 0.21/0.33 |
| EigenTrajectory | 0.36/0.53 | 0.12/0.19 | 0.24/0.43 | 0.19/0.33 | 0.14/0.24 | 0.21/0.34 |
| 1 s | 2 s | 3 s | Total (4 s) | |||||
|---|---|---|---|---|---|---|---|---|
| Method | ADE | FDE | ADE | FDE | ADE | FDE | ADE | FDE |
| Social-GAN | 0.41 | 0.62 | 0.81 | 1.32 | 1.19 | 1.94 | 1.59 | 2.41 |
| Social-STGCNN | 0.34 | 0.48 | 0.71 | 0.94 | 1.09 | 1.77 | 1.53 | 2.26 |
| PECNet | 0.40 | 0.71 | 0.83 | 1.61 | 1.27 | 2.44 | 1.69 | 2.95 |
| NPSN | 0.35 | 0.58 | 0.68 | 1.23 | 1.01 | 1.76 | 1.31 | 1.79 |
| GroupNet | 0.26 | 0.34 | 0.49 | 0.70 | 0.73 | 1.02 | 0.96 | 1.30 |
| Method | Year | minADE | minFDE |
|---|---|---|---|
| Social-GAN | 2018 | 27.23 | 41.44 |
| SoPhie | 2019 | 16.27 | 29.38 |
| PECNet | 2020 | 9.96 | 15.88 |
| PCCSNet | 2021 | 8.62 | 16.16 |
| Social-VAE | 2022 | 8.10 | 11.72 |
| MID | 2022 | 7.61 | 14.30 |
| Variant | ETH/UCY | NBA | SDD |
|---|---|---|---|
| Representation | |||
| Frame-wise position | 0.18/0.28 | 0.7310/0.9180 | 7.92/12.45 |
| Position DCT spectrum | 0.16/0.25 | 0.7160/0.8980 | 7.66/12.03 |
| Time-domain acceleration | 0.16/0.24 | 0.7110/0.8910 | 7.61/11.94 |
| Phase-space supervision | |||
| Position only | 0.17/0.25 | 0.7160/0.8970 | 7.71/12.06 |
Multi-Agent Trajectory Prediction
Multi-Agent Trajectory Prediction forecasts the possible future paths of interacting people, vehicles, robots, or other agents.
See all 84 papers