Predicting the future motion of multiple people is inherently stochastic: the same observed history admits many plausible continuations depending on intent, coordination, and social context. Yet the dominant paradigm in multi-person 3D motion prediction remains deterministic, with existing methods producing a fixed number of outputs per agent. In contrast, we present a conditional flow matching framework that jointly generates diverse future motions for multiple agents, modeling both global trajectories and root-relative poses. A B-spline reparameterization compresses the generative state space while enforcing temporal smoothness, and a dual-stream transformer conditions each agent's predictions on observed neighbor motion for interaction-aware forecasting. Our method achieves state-of-the-art performance on CMU-Mocap (UMPM) and 3DPW, outperforming deterministic baselines on joint position error, pose error, and final displacement error despite producing a full predictive distribution. We additionally establish the first motion prediction baseline on WorldPose, a large-scale professional soccer dataset, where social conditioning demonstrably concentrates predictive uncertainty along interaction-constrained directions.
Figures & tables
Figure 1: Motion forecasts projected onto broadcast video for two players. The gray skeleton and trails mark the recent motion history. ( Left ) T2P deterministic predictions (red). ( Right ) Our multi-modal predictions are colored by log-likelihood (red: high, blue: low).
Figure 2: Overview of the proposed method. (a) Past trajectory and pose observations are encoded into a shared context X~ . Noisy future control points are encoded and concatenated into Y~t , conditioned on flow timestep t via C . (b) L dual-stream transformer blocks process past and future tokens separately, with future tokens attending to [X~;Y~] . (c) Velocity heads decode trajectory and pose control points, lifted to Tout frames by B-spline evaluation.
Symbol
Shape
Meaning
Xq
Tin×dq
Observed sequence in frame space
Yq,Y^q
Tout×dq
Ground-truth and predicted future in frame space
Y~q0,Y~q1,Y~qt
Cq×dq
Source noise, target, and interpolant in control-point space
Bq,Bq†
Tout×Cq , Cq×Tout
Spline evaluation and projection matrices
vq∗,v^q
Tout×dq
Target and predicted velocities after projection to frame space
Table 1: Notation. Subscripts q∈{traj,pose} identify the branches, with dtraj=3 and dpose=3J .
CMU-Mocap (UMPM)
3DPW
JPE ( ↓ )
APE ( ↓ )
FDE ( ↓ )
JPE ( ↓ )
APE ( ↓ )
FDE ( ↓ )
Model
Venue
Gen.?
1.0s
2.0s
1.0s
2.0s
1.0s
2.0s
0.8s
1.6s
0.8s
1.6s
0.8s
1.6s
MRT
NeurIPS'21
✗
164.7
280.1
127.0
164.4
99.6
204.7
159.1
251.2
117.9
153.2
102.7
185.3
DuMMF*
ICLR'23
✓
162.9
275.0
125.3
161.1
97.6
199.2
158.5
248.7
115.5
151.9
102.8
183.1
TBIFormer
CVPR'23
✗
170.0
290.9
125.1
160.8
112.1
228.5
153.9
265.8
115.4
152.7
106.7
215.9
JRT
ICCV'23
✗
168.5
316.9
121.2
181.6
117.7
250.8
181.9
287.3
133.4
178.0
133.7
235.4
Table 2: Comparison of best-of-6 JPE, APE, and FDE metrics (in mm ) at different time horizons on CMU-Mocap (UMPM) and 3DPW datasets. Lower values are better (↓) , with the best results shown in bold . An asterisk (∗) denotes models we recomputed for this setup.
Figure 3: Qualitative comparison on CMU-Mocap (UMPM). Each column shows the root trajectory and final-frame skeleton for a single test agent: the observed past, baselines, and our model. Dashed black lines denote ground truth future trajectory; colored lines show the trajectories of predictions.
JPE ( ↓ )
APE ( ↓ )
FDE ( ↓ )
Model
1.0s
2.0s
3.0s
1.0s
2.0s
3.0s
1.0s
2.0s
3.0s
T2P ∗
0.91
1.70
1.99
0.19
0.22
0.26
0.84
1.50
1.77
EMPMP ∗
0.92
1.79
1.96
0.15
0.16
0.19
0.86
1.64
1.93
Ours w. N=0
0.87
1.41
1.87
0.08
0.09
0.10
0.80
1.34
1.79
Ours w. N=3
0.83
1.37
1.84
0.07
0.09
0.10
0.78
1.32
1.77
Ours w. N=10
0.80
1.36
1.82
0.06
0.09
0.10
0.77
1.31
1.75
Table 3: Comparison of best-of-6 JPE, APE, and FDE metrics (in m ) at different time horizons on WorldPose soccer data. Lower values are better (↓) , with the best results shown in bold . An asterisk (∗) denotes models we recomputed for this setup.
Figure 4: Predicted hip trajectory distributions under varying social contexts for two players. Each panel shows 50 predicted root trajectories with a KDE density overlay, the observed past trajectory (gray), the ground-truth future path (dashed black), and predicted endpoints (scatter).
Representation
Config.
Traj. Basis
Pose Basis
JPE ( ↓ )
APE ( ↓ )
Smoothness ( ↑ )
Fwd. Pass Latency ( ↓ )
Frame-based
Raw
50
50
245.8
133.3
0.62
7.86 ± 0.22
Freq.-based
DCT
50
50
249.7
133.7
0.61
12.30 ± 0.49
Spline-based
Linear
4
12
238.8
129.2
0.67
6.41 ± 0.31
8
24
239.4
129.2
0.67
6.59 ± 0.32
Cubic
4
12
237.5
129.0
0.68
6.46 ± 0.16
8
24
237.4
128.6
0.67
6.61 ± 0.40
Table 4: Ablation study of temporal representations. JPE and APE are reported in mm at 2.0s horizon. Smoothness is measured via NPSS ∈[0,1] . Forward Pass Latency is the GPU wall-clock time for a single forward call with batch size 64, mean ± std. of 50 runs in ms . Best results are shown in bold .
Component change
Δ JPE
Δ APE
Input encoding
w/o per-joint encoding
+6.4
+8.4
sep. → shared traj/pose tokens
+18.2
+11.9
Attention
dual → single-stream
+12.8
+5.3
fixed → iterative past context
+19.9
+7.8
Table 5: Ablation study of architecture components. Each row removes or replaces a single component independently. Δ reported at 2.0s in mm .
ID
Logit-normal t
Frame-space loss
Ljpe
Context drop
Data aug.
JPE ( ↓ )
APE ( ↓ )
(1)
256.5
142.2
(2)
✓
255.0
141.3
(3)
✓
✓
249.2
138.8
(4)
✓
✓
✓
242.9
134.7
(5)
✓
✓
✓
✓
238.1
130.0
(6)
✓
✓
✓
✓
✓
237.4
128.6
Table 6: Ablation study of training components. JPE and APE are reported in mm at 2.0s horizon.
Appendix figures & tables10 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Formula
Notes
JPE (mm)
minkJ1∑j∥Y^T,j(k)−YT,j∗∥2
Absolute joint error at final frame
APE (mm)
minkJ1∑j∥Y~T,j(k)−Y~T,j∗∥2
Root-relative; Y~f,j=Yf,j−Yf,0
FDE (mm)
mink∥Y^T,0(k)−YT,0∗∥2
Root joint displacement at final frame
NPSS
F1∑fcos(∥Pf(Y^)∥Pf(Y^),∥Pf(Y∗)∥Pf(Y∗))
Pf : power at freq. f via rFFT [ Gopalakrishnan et al., 2019 ]
Appendix
Table 7: Evaluation metrics. Y^ denotes the prediction, T=Tout , j indexes joints, f indexes frames.
Figure 5: Sensitivity analysis of key hyperparameters on CMU-Mocap (UMPM), reporting JPE at 2.0s (lower is better). Each plot varies one hyperparameter while holding all others fixed at their selected values (marked ⋆ ). (Left) transformer embedding dimension D . (Middle) number of proposals K for best-of- K′ selection. (Right) number of Euler integration steps S at inference.
Model
CMU-Mocap (UMPM) Latency (s) ↓
3DPW Latency (s) ↓
WorldPose Latency (s) ↓
T2P ∗
8.7 ± 0.3
4.3 ± 0.9
30.1 ± 2.2
EMPMP ∗
6.5 ± 0.1
3.8 ± 0.2
21.9 ± 1.0
Ours
304.3 ± 14.8
221.6 ± 4.7
1767.0 ± 31.3
Appendix
Table 8: Total inference latency comparison. Mean ± std. calculated over 10 runs. All models are run on an NVIDIA L40S GPU with a batch size of 64. Lower values are better (↓) . A (∗) denotes models we recomputed for this experiment.
Setting
1.0s
2.0s
3.0s
Ground truth
0.14
0.13
0.13
Ours N=0
2.69
3.43
4.27
Ours N=3
2.61
3.34
4.31
Ours N=10
1.64
2.09
2.65
Appendix
Table 9: Inter-agent collision rate (%) on WorldPose at different prediction horizons. Lower is better (↓) .
K=10
K=100
Model
<1.0s
1.0-2.0s
Total
<1.0s
1.0-2.0s
Total
DuMMF
110.3
312.4
211.3
121.3
314.2
222.8
Ours
129.8
242.0
185.9
140.7
256.9
198.4
Appendix
Table 10: APD in mm on CMU-Mocap (UMPM) measured across K samples per agent.
Figure 6: Generative diversity evaluation on CMU-Mocap (UMPM). (Left) Best-of- K JPE versus proposal budget K (lower is better). (Right) Coverage recall CDF. The plot displays P(mink′JPE<d) for k′∈K′ predictions, plotted as a function of threshold d . Higher and further left is better.
Model
Energy score (mm) ↓
CRPS (mm) ↓
Spread/error
DuMMF
5491
96.2
0.62
Ours
4218
61.4
0.48
Appendix
Table 11: Distributional quality and calibration on CMU-Mocap (UMPM) using K′=6 samples. Lower is better (↓) for energy score and CRPS. A spread/error ratio of 0.5 is ideal.
Figure 7: Qualitative comparison on 3DPW. Each column shows the root trajectory and final-frame skeleton for a single test agent: the observed past, baselines, and our model. Dashed black lines denote ground truth future trajectory; colored lines show the trajectories of predictions.
Figure 8: Qualitative comparison on WorldPose. The gray skeleton and trails mark the recent motion history. Each row shows the root trajectory and final-frame skeleton prediction for a single test agent: the ground truth (black), baselines (red), and our model (red: high, blue: low).
Figure 9: Failure case on WorldPose. Two players (gray trails: observed history) change direction to receive an incoming ball. Ground-truth future paths are shown as dashed black lines, and our model’s predicted trajectories are colored by likelihood.
3D multi-person motion prediction requires modeling both individual kinematics and inter-person interactions. While Flow Matching is effective for multi-hypothesis generation to improve prediction accuracy, directly predicting skeletal sequences from pure noise often compromises structural consistency and introduces unreliable cross-agent interactions during early noise-dominated integration steps. To address this, we propose a Prior-Guided Residual Flow Matching framework. First, a Deterministic Coarse Prior (DCP) establishes a kinematic anchor, formulating the generative process as a conditional flow over motion residuals to simplify the generative objective and preserve structural stability. Second, a Dynamic Cross-Interaction (DCI) mechanism temporally synchronizes inter-agent message-passing with the integration progress, ensuring the extraction of reliable social contexts and improving multi-person motion fidelity. Finally, a decoupled joint-motion architecture with bidirectional fusion effectively preserves fine-grained kinematic coherence. Extensive experiments demonstrate that our approach achieves state-of-the-art prediction accuracy across multiple datasets. Code is available at https://github.com/Wei-Wei-a/Residual-Flow-Matching-with-Dynamic-Cross-Interaction-for-3D-Multi-Person-Motion-Prediction.
Stochastic human motion prediction aims to forecast future motion distributions. Although recent studies have achieved strong performance in terms of accuracy and diversity, they often overlook plausibility (e.g., resulting in physically unrealistic predictions) and uncertainty quantification, both of which are essential for real-world applications and downstream tasks. To address these issues, we propose a latent flow-based model equipped with a data-driven Gaussian mixture prior that more effectively disentangles diverse human behaviors than conventional single-modal priors. This prior is derived from patterns in the training data without requiring additional annotations. Furthermore, the fully invertible nature of our model enables natural uncertainty quantification through tractable likelihood computation. Experiments on the Human3.6M and AMASS datasets demonstrate that our approach achieves state-of-the-art performance in both accuracy and plausibility.
Yue Ma, Frederick W. B. Li, Xiaohui Liang
Beihang University · Durham University · Zhongguancun Laboratory
Accurately forecasting the movement of people in complex scenes requires reasoning over the past and present state of the entire environment. In this context, effectively incorporating object information and social interactions into a unified framework remains particularly challenging. To address this, we propose Object-Conditioned Social Diffusion (OCSD), a conditional diffusion model that integrates motion history, multi-person interactions, and object cues into a single framework. OCSD uses an object-conditioning mechanism that modulates denoising at every timestep, enabling fine-grained human-object reasoning, and a social encoder that models the interactions between all humans in the scene. As a result, our model naturally handles varying group sizes, complex social interactions, and supports sampling multiple plausible futures. Extensive experiments show that OCSD achieves state-of-the-art results on the Humans in Kitchens (HiK) and HOI-M3 benchmarks. It reduces the two-second path error by 121.5 mm (31.3%) on HiK and 130.5 mm (33.2%) on HOI-M3 compared to prior work, and produces more realistic long-term forecasts.
Serdar Ozsoy, Lars Doorenbos, Juergen Gall
University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany