cs.CVMar 4, 2026
SaveSimpliHuMoN: Simplifying Human Motion Prediction
Organizations: University of Illinois Urbana-Champaign, USA
Abstract
Human motion prediction combines the tasks of trajectory forecasting and human pose prediction. For each of the two tasks, specialized models have been developed. Combining these models for holistic human motion prediction is non-trivial, and recent methods have struggled to compete on established benchmarks for individual tasks. To address this, we propose a simple yet effective transformer-based model for human motion prediction. The model employs a stack of self-attention modules to effectively capture both spatial dependencies within a pose and temporal relationships across a motion sequence. This simple, streamlined, end-to-end model is sufficiently versatile to handle pose-only, trajectory-only, and combined prediction tasks without task-specific modifications. We demonstrate that this approach achieves state-of-the-art results across all tasks through extensive experiments on a wide range of benchmark datasets, including Human3.6M, AMASS, ETH-UCY, and 3DPW.
Figures & tables
Figure 1: An overview of our architecture. Past observations of 3D poses ( ) and trajectories ( ) are jointly processed by an encoder. Learnable input queries ( ), representing potential future states, interact with the encoded past motion within a decoder to produce distinct future motion proposals ( ) for all agents over a specified horizon.
| Model | Training Throughput (samples/sec) | Test Throughput (samples/sec) |
|---|---|---|
| T2P* | 187 22 | 401 64 |
| EMPMP* | 812 58 | 2041 129 |
| Ours (wide) | 862 43 | 2251 140 |
| Ours (deep) | 928 45 | 3673 161 |
Table 2: Throughput mean std calculated over 10 runs on MOCAP-UMPM data. All models are run on a NVIDIA RTX A6000 GPU with a batch size of 64. Higher values are better . An asterisk(*) denotes models we recomputed for this experiment.
Figure 2: Visualization of predictions on a MOCAP-UMPM scene. Model predictions are in color, and ground truth future poses are black dashes. The last-known input positions are colored dashes.
Figure 3: Visualization of motion proposals ( ) of our (wide) model on MOCAP-UMPM data. All model predictions are in color. Ground-truth future poses are represented by black dashes, and the last-known input positions are colored dashes.
| Depth | Embed dim | Total Params | APE | JPE |
| 8 | 192 | 5.2M | 126.05 | 212.84 |
| 6 | 192 | 4.0M | 125.70 | 212.72 |
| 4 | 192 | 2.8M | 126.22 | 213.40 |
| 12 | 96 | 1.9M | 128.47 | 212.08 |
| 6 | 96 | 1.0M | 128.52 | 212.45 |
| 12 | 64 | 860K | 130.72 | 212.30 |
Table 3: Comparison of our model’s performance with different hyperparameter configurations.
| Model | APE | JPE | APE | JPE |
| T2P | 154.4 | 366.4 | 151.7 | 262.7 |
| EMPMP | 147.2 | 283.1 | 146.5 | 250.4 |
| Ours (wide) | 145.84 | 280.8 | 125.70 | 212.72 |
| Ours (deep) | 149.35 | 286.96 | 131.41 | 211.76 |
Table 4: Model performance with two different modes on MOCAP-UMPM data. Lower values are better . Best results are in bold .
Appendix figures & tables11 assets
Supplementary material from the paper’s appendix.
Appendix
| Model | ETH | HOTEL | UNIV | ZARA1 | ZARA2 | AVG |
| MID | 0.39/0.66 | 0.13/0.22 | 0.22/0.45 | 0.17/0.30 | 0.13/0.27 | 0.21/0.38 |
| GP-Graph | 0.43/0.63 | 0.18/0.30 | 0.24/0.42 | 0.17/0.31 | 0.15/0.29 | 0.23/0.39 |
| TUTR | 0.40/0.61 | 0.11/0.18 | 0.23/0.42 | 0.18/0.34 | 0.13/0.25 | 0.21/0.36 |
| SingularTraj. | 0.35/0.42 | 0.13/0.19 | 0.25/0.44 | 0.19/0.32 | 0.15/0.25 | 0.22/0.34 |
| TrajCLIP ∧ | 0.36/0.57 | 0.10/0.17 | 0.19/0.41 | 0.16/0.28 | 0.11/0.20 | 0.18 /0.33 |
| NMRF | 0.26/0.37 | 0.11/0.17 | 0.28/0.49 | 0.17/0.30 | 0.14/0.25 | 0.19/ 0.32 |
Table 5: Trajectory prediction performance (ADE/FDE) on ETH-UCY. Lower values are better, with the best results shown in bold . A dagger ( ) marks models adapted for the specific task, while a ( ) notes models that use external training data.
| MOCAP-UMPM | 3DPW | |||||||||
| In/Out Length (s) | 0.4s | 0.8s | 1.2s | 1.6s | 2.0s | 0.4s | 0.8s | 1.2s | 1.6s | |
| APE | T2P* | 71.7 | 107.8 | 120.4 | 137.1 | 151.7 | 98.2 | 114.6 | 135.3 | 150.0 |
| EMPMP* | 60.1 | 96.0 | 116.9 | 131.6 | 146.5 | 96.3 | 111.9 | 134.4 | 150.6 | |
| Ours (wide) | 57.3 | 87.7 | 104.5 | 115.3 | 125.7 | 92.8 | 107.1 | 130.0 | 142.9 | |
| Ours (deep) | 62.3 | 89.5 | 107.3 | 119.0 | 128.5 | 93.2 | 108.4 | 131.5 | 148.9 | |
| JPE | T2P* | 70.2 | 139.2 | 160.1 | 226.4 | 262.7 | 107.7 | 142.6 | 181.0 | 236.2 |
Table 6: Comparison of APE/JPE metrics across models and datasets. Lower values are better , with the best results shown in bold . An asterisk (*) denotes models we recomputed for this setup.
| # | AMASS | Human3.6M | MOCAP-UMPM | 3DPW |
| 1 | Pelvis | – | Hips | Pelvis |
| 2 | L_Hip | LeftUpLeg | LHip | LHip |
| 3 | R_Hip | RightUpLeg | RHip | RHip |
| 4 | Spine1 | Spine | Spine | – |
| 5 | L_Knee | LeftLeg | LKnee | LKnee |
| 6 | R_Knee | RightLeg | RKnee | RKnee |
Table 7: Mapping from dataset-specific skeletons to our 22-joint canonical representation. AMASS serves as the canonical skeleton itself. Dashes (–) indicate that a direct mapping for that specific canonical joint is unavailable in the source dataset.
Figure 4: Distribution of winning mode indices (best-of-6) on pose + trajectory prediction task across the training and validation sets of MOCAP-UMPM. The dashed line ( - - - ) indicates equal distribution. Both distributions demonstrate balanced mode utilization without mode collapse.
| Pose Prediction | Trajectory Prediction | |||
| Training Method Metric | Only Pose ADE /FDE | Pose + Traj ADE /FDE | Only Traj ADE /FDE | Pose + Traj ADE /FDE |
| T2P † | - | 0.61/0.95 | - | 0.20/0.29 |
| EMPMP † | - | 0.53/0.65 | - | 0.14/0.20 |
| Ours (wide) | 0.46/0.58 | 0.41 / 0.51 | 0.09/0.18 | 0.08/0.16 |
| Ours (deep) | 0.46/0.59 | 0.42/ 0.51 | 0.09/0.18 | 0.08 /0.17 |
| Improvement (%) | -11.4/-12.2 | -13.7/-11.9 | ||
Table 10: Comparison of the influence of combined pose and trajectory information vs. individual performance on MOCAP-UMPM data. Lower values are better , with the best results shown in bold . A dagger ( ) marks models adapted for the specific task.
| Wide | Deep | |||
| Attention Mechanism | APE | JPE | APE | JPE |
| Self-Attn over [ ] (Ours) | 125.70 | 212.72 | 131.41 | 211.76 |
| Self-Attn ( ) + Cross ( ) | 134.61 | 229.05 | 140.32 | 227.89 |
Table 11: Ablation study on attention mechanisms measuring metrics on MOCAP-UMPM. Lower values are better , with the best results shown in bold .
Figure 5: Attention patterns in the first transformer block at epoch 100. Brighter colors indicate stronger attention weights. Dashed lines ( - - - ) separate past context from future query tokens.
| Components | Ours (wide) | Ours (deep) | ||||
| RMSNorm | LayerNorm | Type Emb. | APE | JPE | APE | JPE |
| ✓ | ✓ | 126.24 | 213.85 | 132.09 | 212.13 | |
| ✓ | 127.37 | 213.06 | 132.80 | 214.85 | ||
| ✓ | ✓ | 125.70 | 212.72 | 131.41 | 211.76 | |
Table 12: Extended ablation study of architecture choices on MOCAP-UMPM. We explicitly compare RMSNorm vs. LayerNorm and the effect of Type Embeddings. Lower values are better ( ), with the best results shown in bold .
Figure 6: Visualization of trajectory predictions ( ) of our (wide) model on ETH-UCY data. X-Y coordinate values in the plots are in meters. All model predictions are in color, ground truth future trajectories are black dashes, and input trajectories are in gray.
Figure 7: Visualization of predictions of our (wide) model on MOCAP-UMPM data. All model predictions are in color. Ground truth future poses are black dashes, and the last-known input positions are colored dashes.
| Model | APE | JPE |
| T2P* | 362.7 | 913.6 |
| EMPMP* | 443.4 | 981.5 |
| Ours (wide) | 156.8 | 746.3 |
Table 13: Comparison of APE/JPE metrics on WorldPose data. Lower values are better ( ), with the best results shown in bold . An asterisk (*) denotes models we recomputed for this setup.