Robots that share space with people need human motion predictions with reliable uncertainty, not just accurate means. This paper presents a structured multitask variational Gaussian process (GP) for probabilistic full-body human motion prediction. The model integrates one-shot prediction of the full horizon, multitask outputs that couple future time steps, a factorization into one sparse variational GP per joint-dimension pair of the rotation representation, and a continuous 6D rotation representation with forward kinematics that preserves bone lengths. Its predictive distribution is available in closed form: a Gaussian with full temporal covariance for each rotation coordinate, which downstream planners operating in rotation coordinates can use without sampling, density estimation, or forward kinematics. On Human3.6M, the model achieves lower kernel-density-estimated negative log-likelihood (KDE NLL) than Motron and DLow at every prediction step, 22-52 nats below Motron. Empirical coverage is conservative at the 50% and 80% levels and close to nominal at 95% over a 2 s horizon. The probabilistic model uses 0.24 M parameters, about seven times fewer than Motron, and a separately trained 0.35 M-parameter deterministic variant has a mean angle error 3-22% higher than Motron's. Ablations confirm the benefit of the 6D representation and show that, at a fixed training budget, factorized joint outputs outperform coupled right-arm alternatives in NLL and CRPS while using less memory. These results show that compact, structured GPs provide analytic, temporally structured uncertainty with conservative empirical coverage for robotic systems.
Figures & tables
Fig. 1: Model architecture illustrated for a single joint with D dimensions. Each joint–dimension pair is modeled by a GP that maps H past time steps to F future time steps and produces a Gaussian predictive distribution at each future time step. This structure is replicated for all joints, resulting in 96 parallel GPs after preprocessing. The factorization keeps full-body inference tractable while each GP retains a full temporal covariance over the prediction horizon.
Parameter
Probabilistic/Stochastic
Deterministic
Input H
12 (0.5 s)
50 (2 s)
Output F
50 (2 s)
25 (1 s)
Inducing M
25
25
Latent L
3
3
Validation data
Train-subject tails
S5 subset
Batch (K-means/train/val)
4096/2048/512
TABLE I: GP implementation parameters for Probabilistic/Stochastic and Deterministic evaluations. Input and output horizons are reported as number of frames, with the equivalent duration in seconds in parentheses. Horizons follow the Motron benchmark protocols [ 3 ] , and both configurations use the same compact model size ( M=25 , L=3 ).
Factor
Options Tested
Pose representation
exponential map, quaternion, 6D rotations
Covariance kernel
RBF, Matérn 3/2, Matérn 5/2 (+ Linear)
Inducing M
10, 25, 50, 100, 200
Latent L
1, 2, 3, 4, 6
Batch size
256, 512, 1024, 2048, 4096
TABLE II: Explored settings for representation and hyperparameter selection. The final configuration (6D rotations, Matérn 3/2 + Linear, M=25 , L=3 ) combines the best-performing representation with a compact model size (Section V-D ).
Fig. 2: KDE NLL (lower is better) of our final model, Motron, and DLow. Our model is lowest at every time step and 22–52 nats below Motron, placing substantially higher density on the observed future motion across the full 2 s horizon.
Fig. 3: Empirical coverage of predicted joint positions for the 50%, 80%, and 95% central intervals across prediction horizons. Gray dashed lines mark the nominal levels. The 50% and 80% intervals stay above nominal throughout, and the 95% interval stays near nominal, falling slightly below it (to about 93%) beyond about 0.5 s.
Fig. 4: Visualization of 50 sampled skeleton predictions drawn from the predicted distributions at different horizons. The solid skeleton denotes the ground-truth motion, while the translucent skeletons represent sampled predictions. Example shown for Subject S9 performing the walking action (first sequence). Colors indicate body sides (blue: right, orange: left). The sample spread widens with horizon while continuing to enclose the ground truth, illustrating uncertainty that grows as future motion becomes less predictable.
ms
80
160
320
400
560
720
880
1000
Zero Vel [ 3 ]
0.40
0.70
1.11
1.25
1.46
1.63
1.76
1.84
GRU sup [ 11 ]
0.43
0.74
1.15
1.30
-
-
-
-
QuaterNet [ 21 ]
0.37
0.62
1.00
1.14
-
-
-
-
HisRepItself [ 22 ]
0.28
0.52
0.88
1.02
1.23
1.40
1.55
1.64
Motron [ 3 ]
0.28
0.51
0.87
1.01
1.22
1.40
1.54
1.63
Ours
0.33
0.62
1.02
1.16
1.36
1.50
1.61
1.68
TABLE III: MAE (rad, lower is better) of our model compared to SOTA. The gap to Motron narrows to 3% at 1000 ms, so the predictive mean remains competitive at longer horizons. GRU sup and QuaterNet report results only up to 400 ms.
Fig. 5: Visualization of mean skeleton predictions at different horizons. The solid skeleton denotes the ground-truth motion, while the translucent skeleton represents the mean prediction. Example shown for Subject S5 performing the walking action (first sequence). Colors indicate body sides (blue: right, orange: left). The mean prediction closely follows the ground truth at short horizons, with deviations growing over time, consistent with the MAE trend in Table III .
1 s
2 s
APD ↑
ADE ↓
FDE ↓
APD ↑
ADE ↓
FDE ↓
DSF [ 25 ]
-
-
-
9.330
0.493
0.592
MT-VAE [ 26 ]
-
-
-
0.403
0.457
0.595
MotionDiff [ 23 ]
-
-
-
15.353
0.411
0.509
HumanMAC [ 20 ]
-
-
-
6.301
0.369
0.480
ProbHMI [ 15 ]
-
-
-
6.682
0.364
0.493
TABLE IV: APD, best-of-50 ADE, and best-of-50 FDE (m) of our model compared to SOTA. Our samples are less diverse than most baselines, with higher best-of-50 ADE than all baselines and higher FDE than most. These metrics reward diverse sample sets, whereas our model is trained for predictive likelihood, on which it outperforms Motron and DLow (Fig. 2 ).
Type
Method
Params (M)
siMLPe [ 19 ]
Deterministic
DCT, MLP
0.14
HisRepItself [ 22 ]
Deterministic
Attention, GCN
3.24
DLow [ 10 ]
Stochastic
Latent, CVAE
7.30
MotionDiff [ 23 ]
Stochastic
Diffusion
29.93
HumanMAC [ 20 ]
Stochastic
DCT, Diffusion
28.40
Motron [ 3 ]
Probabilistic
Latent, GNN
1.67
TABLE V: Parameter counts of our model and SOTA methods. Bold marks the fewest parameters among probabilistic and stochastic methods. Our probabilistic model is about seven times smaller than Motron while achieving lower KDE NLL, showing that high-quality predictive distributions can be obtained with a compact parameterization.
Fig. 6: KDE NLL by horizon for seed-0 absolute-rotation models with H=12 , F=50 , a Matérn 3/2 kernel, and M=100 . Curves use the reported adaptive estimator with 2000 posterior samples. The 6D representation has the lowest NLL at every horizon, supporting its selection for the final model. Its advantage narrows at longer horizons as the adaptive KDE bandwidth grows with predictive spread.
A
B
C
KDE NLL
-11.237
-4.814
-2.613
CRPS (m)
0.0473
0.0899
0.1087
MAE (rad)
0.637
0.776
1.021
Coverage 50%
70.6%
58.6%
49.0%
Coverage 80%
91.0%
84.2%
79.4%
Coverage 95%
97.1%
96.0%
95.4%
TABLE VI: Right-arm coupling at a shared training budget for independent (A), within-joint (B), and full-chain (C) GPs. Memory is peak allocated GPU memory, and epoch time covers ten batches. The factorized variant A achieves the best KDE NLL, CRPS, and MAE with the lowest memory, supporting joint–dimension factorization in the main model.
This paper compares three human motion prediction methods - GMM-based clustering, TransFusion, and Graph-Mixer - evaluating their effectiveness in human-robot collaboration. These predictions are valuable for improving robot motion planning by anticipating human movements and enhancing safety and efficiency in dynamic environments.
Placido Falqueto, Elena Basei, Edoardo Lamon +4
Dept. of Information Engineering and Computer Science (DISI), University of Trento, Trento, Italy · Department of Industrial Engineering (DII), University of Trento, Trento, Italy
Human motion prediction combines the tasks of trajectory forecasting and human pose prediction. For each of the two tasks, specialized models have been developed. Combining these models for holistic human motion prediction is non-trivial, and recent methods have struggled to compete on established benchmarks for individual tasks. To address this, we propose a simple yet effective transformer-based model for human motion prediction. The model employs a stack of self-attention modules to effectively capture both spatial dependencies within a pose and temporal relationships across a motion sequence. This simple, streamlined, end-to-end model is sufficiently versatile to handle pose-only, trajectory-only, and combined prediction tasks without task-specific modifications. We demonstrate that this approach achieves state-of-the-art results across all tasks through extensive experiments on a wide range of benchmark datasets, including Human3.6M, AMASS, ETH-UCY, and 3DPW.
Stochastic human motion prediction aims to forecast future motion distributions. Although recent studies have achieved strong performance in terms of accuracy and diversity, they often overlook plausibility (e.g., resulting in physically unrealistic predictions) and uncertainty quantification, both of which are essential for real-world applications and downstream tasks. To address these issues, we propose a latent flow-based model equipped with a data-driven Gaussian mixture prior that more effectively disentangles diverse human behaviors than conventional single-modal priors. This prior is derived from patterns in the training data without requiring additional annotations. Furthermore, the fully invertible nature of our model enables natural uncertainty quantification through tractable likelihood computation. Experiments on the Human3.6M and AMASS datasets demonstrate that our approach achieves state-of-the-art performance in both accuracy and plausibility.
Yue Ma, Frederick W. B. Li, Xiaohui Liang
Beihang University · Durham University · Zhongguancun Laboratory