FlowMoP: Stochastic Multi-Person Motion Prediction

Aadya Agrawal; Ho Kei Cheng; Alexander Schwing

cs.CVNov 6, 2026

FlowMoP: Stochastic Multi-Person Motion Prediction

Authors: Aadya Agrawal, Ho Kei Cheng, Alexander Schwing

Organizations: University of Illinois Urbana-Champaign

Abstract

Predicting the future motion of multiple people is inherently stochastic: the same observed history admits many plausible continuations depending on intent, coordination, and social context. Yet the dominant paradigm in multi-person 3D motion prediction remains deterministic, with existing methods producing a fixed number of outputs per agent. In contrast, we present a conditional flow matching framework that jointly generates diverse future motions for multiple agents, modeling both global trajectories and root-relative poses. A B-spline reparameterization compresses the generative state space while enforcing temporal smoothness, and a dual-stream transformer conditions each agent's predictions on observed neighbor motion for interaction-aware forecasting. Our method achieves state-of-the-art performance on CMU-Mocap (UMPM) and 3DPW, outperforming deterministic baselines on joint position error, pose error, and final displacement error despite producing a full predictive distribution. We additionally establish the first motion prediction baseline on WorldPose, a large-scale professional soccer dataset, where social conditioning demonstrably concentrates predictive uncertainty along interaction-constrained directions.

Figures & tables

Figure 1: Motion forecasts projected onto broadcast video for two players. The gray skeleton and trails mark the recent motion history. ( Left ) T2P deterministic predictions (red). ( Right ) Our multi-modal predictions are colored by log-likelihood (red: high, blue: low).

Figure 2: Overview of the proposed method. (a) Past trajectory and pose observations are encoded into a shared context $\tilde{\mathbf{X}}$ . Noisy future control points are encoded and concatenated into $\tilde{\mathbf{Y}}^{t}$ , conditioned on flow timestep $t$ via $\mathbf{C}$ . (b) $L$ dual-stream transformer blocks process past and future tokens separately, with future tokens attending to $[\tilde{\mathbf{X}};\tilde{\mathbf{Y}}]$ . (c) Velocity heads decode trajectory and pose control points, lifted to $T_{\text{out}}$ frames by B-spline evaluation.

Symbol	Shape	Meaning
$\mathbf{X}_{q}$	$T_{\mathrm{in}}\times d_{q}$	Observed sequence in frame space
$\mathbf{Y}_{q},\hat{\mathbf{Y}}_{q}$	$T_{\mathrm{out}}\times d_{q}$	Ground-truth and predicted future in frame space
$\tilde{\mathbf{Y}}^{0}_{q},\tilde{\mathbf{Y}}^{1}_{q},\tilde{\mathbf{Y}}^{t}_{q}$	$C_{q}\times d_{q}$	Source noise, target, and interpolant in control-point space
$\mathbf{B}_{q},\mathbf{B}_{q}^{\dagger}$	$T_{\mathrm{out}}\times C_{q}$ , $C_{q}\times T_{\mathrm{out}}$	Spline evaluation and projection matrices
$v^{*}_{q},\hat{v}_{q}$	$T_{\mathrm{out}}\times d_{q}$	Target and predicted velocities after projection to frame space

Table 1: Notation. Subscripts

q\in\{\mathrm{traj},\mathrm{pose}\}

identify the branches, with

d_{\mathrm{traj}}=3

and

d_{\mathrm{pose}}=3J

			CMU-Mocap (UMPM)						3DPW
			JPE ( $\downarrow$ )		APE ( $\downarrow$ )		FDE ( $\downarrow$ )		JPE ( $\downarrow$ )		APE ( $\downarrow$ )		FDE ( $\downarrow$ )
Model	Venue	Gen.?	1.0s	2.0s	1.0s	2.0s	1.0s	2.0s	0.8s	1.6s	0.8s	1.6s	0.8s	1.6s
MRT	NeurIPS'21	✗	164.7	280.1	127.0	164.4	99.6	204.7	159.1	251.2	117.9	153.2	102.7	185.3
DuMMF*	ICLR'23	✓	162.9	275.0	125.3	161.1	97.6	199.2	158.5	248.7	115.5	151.9	102.8	183.1
TBIFormer	CVPR'23	✗	170.0	290.9	125.1	160.8	112.1	228.5	153.9	265.8	115.4	152.7	106.7	215.9
JRT	ICCV'23	✗	168.5	316.9	121.2	181.6	117.7	250.8	181.9	287.3	133.4	178.0	133.7	235.4

Table 2: Comparison of best-of-6 JPE, APE, and FDE metrics (in mm ) at different time horizons on CMU-Mocap (UMPM) and 3DPW datasets. Lower values are better

(\downarrow)

, with the best results shown in bold . An asterisk

(^{*})

denotes models we recomputed for this setup.

Figure 3: Qualitative comparison on CMU-Mocap (UMPM). Each column shows the root trajectory and final-frame skeleton for a single test agent: the observed past, baselines, and our model. Dashed black lines denote ground truth future trajectory; colored lines show the trajectories of predictions.

	JPE ( $\downarrow$ )			APE ( $\downarrow$ )			FDE ( $\downarrow$ )
Model	1.0s	2.0s	3.0s	1.0s	2.0s	3.0s	1.0s	2.0s	3.0s
T2P ∗	0.91	1.70	1.99	0.19	0.22	0.26	0.84	1.50	1.77
EMPMP ∗	0.92	1.79	1.96	0.15	0.16	0.19	0.86	1.64	1.93
Ours w. $N=0$	0.87	1.41	1.87	0.08	0.09	0.10	0.80	1.34	1.79
Ours w. $N=3$	0.83	1.37	1.84	0.07	0.09	0.10	0.78	1.32	1.77
Ours w. $N=10$	0.80	1.36	1.82	0.06	0.09	0.10	0.77	1.31	1.75

Table 3: Comparison of best-of-6 JPE, APE, and FDE metrics (in m ) at different time horizons on WorldPose soccer data. Lower values are better

(\downarrow)

, with the best results shown in bold . An asterisk

(^{*})

denotes models we recomputed for this setup.

Figure 4: Predicted hip trajectory distributions under varying social contexts for two players. Each panel shows 50 predicted root trajectories with a KDE density overlay, the observed past trajectory (gray), the ground-truth future path (dashed black), and predicted endpoints (scatter).

Representation	Config.	Traj. Basis	Pose Basis	JPE ( $\downarrow$ )	APE ( $\downarrow$ )	Smoothness ( $\uparrow$ )	Fwd. Pass Latency ( $\downarrow$ )
Frame-based	Raw	50	50	245.8	133.3	0.62	7.86 $\pm$ 0.22
Freq.-based	DCT	50	50	249.7	133.7	0.61	12.30 $\pm$ 0.49
Spline-based	Linear	4	12	238.8	129.2	0.67	6.41 $\pm$ 0.31
	Linear	8	24	239.4	129.2	0.67	6.59 $\pm$ 0.32
	Cubic	4	12	237.5	129.0	0.68	6.46 $\pm$ 0.16
		8	24	237.4	128.6	0.67	6.61 $\pm$ 0.40

Table 4: Ablation study of temporal representations. JPE and APE are reported in mm at 2.0s horizon. Smoothness is measured via NPSS

\in[0,1]

. Forward Pass Latency is the GPU wall-clock time for a single forward call with batch size 64, mean

\pm

std. of 50 runs in ms . Best results are shown in bold .

Component change	$\boldsymbol{\Delta}$ JPE	$\boldsymbol{\Delta}$ APE
Input encoding
w/o per-joint encoding	+6.4	+8.4
sep. $\rightarrow$ shared traj/pose tokens	+18.2	+11.9
Attention
dual $\rightarrow$ single-stream	+12.8	+5.3
fixed $\rightarrow$ iterative past context	+19.9	+7.8

Table 5: Ablation study of architecture components. Each row removes or replaces a single component independently.

\Delta

reported at 2.0s in mm .

ID	Logit-normal $t$	Frame-space loss	$\mathcal{L}_{\text{jpe}}$	Context drop	Data aug.	JPE ( $\downarrow$ )	APE ( $\downarrow$ )
(1)						256.5	142.2
(2)	✓					255.0	141.3
(3)	✓	✓				249.2	138.8
(4)	✓	✓	✓			242.9	134.7
(5)	✓	✓	✓	✓		238.1	130.0
(6)	✓	✓	✓	✓	✓	237.4	128.6

Table 6: Ablation study of training components. JPE and APE are reported in mm at 2.0s horizon.

Appendix figures & tables10 assets

Supplementary material from the paper’s appendix.

Appendix

Metric	Formula	Notes
JPE (mm)	$\min_{k}\frac{1}{J}\sum_{j}\\|\hat{\mathbf{Y}}^{(k)}_{T,j}-\mathbf{Y}^{*}_{T,j}\\|_{2}$	Absolute joint error at final frame
APE (mm)	$\min_{k}\frac{1}{J}\sum_{j}\\|\tilde{\mathbf{Y}}^{(k)}_{T,j}-\tilde{\mathbf{Y}}^{*}_{T,j}\\|_{2}$	Root-relative; $\tilde{\mathbf{Y}}_{f,j}=\mathbf{Y}_{f,j}-\mathbf{Y}_{f,0}$
FDE (mm)	$\min_{k}\\|\hat{\mathbf{Y}}^{(k)}_{T,0}-\mathbf{Y}^{*}_{T,0}\\|_{2}$	Root joint displacement at final frame
NPSS	$\frac{1}{F}\sum_{f}\cos\!\left(\frac{P_{f}(\hat{\mathbf{Y}})}{\\|P_{f}(\hat{\mathbf{Y}})\\|},\,\frac{P_{f}(\mathbf{Y}^{})}{\\|P_{f}(\mathbf{Y}^{})\\|}\right)$	$P_{f}$ : power at freq. $f$ via rFFT [ Gopalakrishnan et al., 2019 ]

Appendix

Table 7: Evaluation metrics.

\hat{\mathbf{Y}}

denotes the prediction,

T=T_{\text{out}}

j

indexes joints,

f

indexes frames.

Figure 5: Sensitivity analysis of key hyperparameters on CMU-Mocap (UMPM), reporting JPE at 2.0s (lower is better). Each plot varies one hyperparameter while holding all others fixed at their selected values (marked

\star

). (Left) transformer embedding dimension

D

. (Middle) number of proposals

K

for best-of-

K^{\prime}

selection. (Right) number of Euler integration steps

S

at inference.

Model	CMU-Mocap (UMPM) Latency (s) $\downarrow$	3DPW Latency (s) $\downarrow$	WorldPose Latency (s) $\downarrow$
T2P ∗	8.7 $\pm$ 0.3	4.3 $\pm$ 0.9	30.1 $\pm$ 2.2
EMPMP ∗	6.5 $\pm$ 0.1	3.8 $\pm$ 0.2	21.9 $\pm$ 1.0
Ours	304.3 $\pm$ 14.8	221.6 $\pm$ 4.7	1767.0 $\pm$ 31.3

Appendix

Table 8: Total inference latency comparison. Mean

\pm

std. calculated over 10 runs. All models are run on an NVIDIA L40S GPU with a batch size of 64. Lower values are better

(\downarrow)

. A

(^{*})

denotes models we recomputed for this experiment.

Setting	1.0s	2.0s	3.0s
Ground truth	0.14	0.13	0.13
Ours $N=0$	2.69	3.43	4.27
Ours $N=3$	2.61	3.34	4.31
Ours $N=10$	1.64	2.09	2.65

Appendix

Table 9: Inter-agent collision rate (%) on WorldPose at different prediction horizons. Lower is better

(\downarrow)

	$K=10$			$K=100$
Model	<1.0s	1.0-2.0s	Total	<1.0s	1.0-2.0s	Total
DuMMF	$110.3$	$312.4$	$211.3$	$121.3$	$314.2$	$222.8$
Ours	129.8	$242.0$	$185.9$	140.7	$256.9$	$198.4$

Appendix

Table 10: APD in mm on CMU-Mocap (UMPM) measured across

K

samples per agent.

Figure 6: Generative diversity evaluation on CMU-Mocap (UMPM). (Left) Best-of-

K

JPE versus proposal budget

K

(lower is better). (Right) Coverage recall CDF. The plot displays

P(\min_{k^{\prime}}\text{JPE}<d)

for

k^{\prime}\in K^{\prime}

predictions, plotted as a function of threshold

d

. Higher and further left is better.

Model	Energy score (mm) $\downarrow$	CRPS (mm) $\downarrow$	Spread/error
DuMMF	5491	96.2	0.62
Ours	4218	61.4	0.48

Appendix

Table 11: Distributional quality and calibration on CMU-Mocap (UMPM) using

K^{\prime}=6

samples. Lower is better

(\downarrow)

for energy score and CRPS. A spread/error ratio of 0.5 is ideal.

Figure 7: Qualitative comparison on 3DPW. Each column shows the root trajectory and final-frame skeleton for a single test agent: the observed past, baselines, and our model. Dashed black lines denote ground truth future trajectory; colored lines show the trajectories of predictions.

Figure 8: Qualitative comparison on WorldPose. The gray skeleton and trails mark the recent motion history. Each row shows the root trajectory and final-frame skeleton prediction for a single test agent: the ground truth (black), baselines (red), and our model (red: high, blue: low).

Figure 9: Failure case on WorldPose. Two players (gray trails: observed history) change direction to receive an incoming ball. Ground-truth future paths are shown as dashed black lines, and our model’s predicted trajectories are colored by likelihood.

Human Motion Prediction

Human motion prediction forecasts how people will move, supporting applications from animation and activity understanding to safe robot interaction.

See all 57 papers

Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction
Aug 4, 2026Wei Wei, Yinyuan Zhao, Ruixuan YuFlow Matching Human Motion Prediction
Shandong University
Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion Prediction
Aug 21, 2026Yue Ma, Frederick W. B. Li, Xiaohui LiangHuman Motion Prediction Flow-Based Generative Modeling
Beihang University · Durham University · Zhongguancun Laboratory
Multi-Person Human Motion Forecasting in Complex Scenes
Aug 27, 2026Serdar Ozsoy, Lars Doorenbos, Juergen GallHuman Motion Prediction Diffusion Models for Motion Generation
University of Bonn, Germany · Lamarr Institute for Machine Learning and Artificial Intelligence, Germany

Aug 4, 2026J/K move · Enter open · S save

Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction

Wei Wei, Yinyuan Zhao, Ruixuan Yu

Shandong University

Flow Matching Human Motion Prediction

Open paper AskPDF arXiv

3D multi-person motion prediction requires modeling both individual kinematics and inter-person interactions. While Flow Matching is effective for multi-hypothesis generation to improve prediction accuracy, directly predicting skeletal sequences from pure noise often compromises structural consistency and introduces unreliable cross-agent interactions during early noise-dominated integration steps. To address this, we propose a Prior-Guided Residual Flow Matching framework. First, a Deterministic Coarse Prior (DCP) establishes a kinematic anchor, formulating the generative process as a conditional flow over motion residuals to simplify the generative objective and preserve structural stability. Second, a Dynamic Cross-Interaction (DCI) mechanism temporally synchronizes inter-agent message-passing with the integration progress, ensuring the extraction of reliable social contexts and improving multi-person motion fidelity. Finally, a decoupled joint-motion architecture with bidirectional fusion effectively preserves fine-grained kinematic coherence. Extensive experiments demonstrate that our approach achieves state-of-the-art prediction accuracy across multiple datasets. Code is available at https://github.com/Wei-Wei-a/Residual-Flow-Matching-with-Dynamic-Cross-Interaction-for-3D-Multi-Person-Motion-Prediction.

Abstract

Human Motion Prediction

Explore similar work

Residual Flow Matching with Dynamic Cross-Interaction for 3D Multi-Person Motion Prediction

Gaussian-Mixture Latent Flow for Stochastic 3D Human Motion Prediction

Multi-Person Human Motion Forecasting in Complex Scenes