Forecasting future human motion requires jointly reasoning about an agent's global trajectory and 3D body pose, which are tightly coupled, particularly in team sports where multiple interacting players introduce complex social dynamics. We exploit a socio-temporal encoder that fuses trajectory and pose at the input level and models them jointly through shared factorized attention. The model is further enhanced by a per-joint loss reweighting scheme that emphasizes dynamic joints and prevents degenerate solutions, and a frequency-domain trajectory smoothing term that improves trajectory prediction while indirectly benefiting pose forecasting. To ensure fair comparison with prior joint forecasting methods, we identify and correct evaluation biases arising from overlapping sequence construction, adopting instead a non-overlapping protocol and a deterministic, oracle-free strategy for multi-future prediction. Our approach improves over prior joint forecasting methods on both WorldPose and 3DPW, with the largest gains on pose accuracy, demonstrating that the proposed joint formulation generalizes beyond the densely interactive scenarios that motivate it.
Figures & tables
Figure 1: Overview. Prior trajectories and poses are first projected independently into a shared latent space. The fused representation is processed by a socio-temporal encoder whose output is fed into two task-specific heads implemented as independent MLPs.
Input Modalities
APE ↓
PPE ↓
Pose only
123.0
90.7
Pose + Trajectory
121.8
88.9
Table 1: Effect of trajectory and pose as an auxiliary input on pose/trajectory-only forecasting , evaluated on WorldPose [ 32 ] . The model is trained to predict poses (left) / trajectories (right) exclusively; only the input modalities are varied.
Fusion
Params
ADE ↓
FDE ↓
JPE ↓
APE ↓
PPE ↓
Sequential
2.58M
1074.6
2186.7
1131.9
116.3
85.3
Cross Att
1.45M
951.4
1962.8
1012.7
115.1
84.1
Late Halved
0.74M
850.9
1875.2
907.3
115.2
84.4
Late
2.58M
810.7
1804.9
863.6
114.8
84.2
Early Sum
1.31M
681.2
1663.4
736.8
113.0
83.9
Early Concat
1.32M
670.8
1642.0
723.5
112.6
83.7
Table 2: Fusion topology on WorldPose. All lines share backbone, losses, protocol and training recipe, only the fusion point changes. Rebalancing/smoothing are disabled to isolate fusion.
Joints
Smoothing
ADE ↓
FDE ↓
JPE ↓
APE ↓
PPE ↓
0
✗
670.8
1642.0
723.5
112.6
83.7
2.5
✗
676.3
1651.1
720.2
105.9
81.4
5
✗
683.3
1667.2
719.8
101.3
79.1
7.5
✗
714.6
1694.0
745.6
99.8
78.7
10
✗
848.5
1832.5
874.7
98.7
78.5
0
✓
611.7
1528.1
659.1
106.7
82.6
Table 3: Ablation studies evaluated on WorldPose [ 32 ] . Joint rebalancing factor in the top rows show that larger factors steadily improve pose accuracy at the expense of trajectory accuracy. Trajectory smoothing enabled for the last two rows improves trajectory accuracy directly, and pose accuracy indirectly, by removing unrealistic high-frequency jitter in the predictions
Figure 2: Effect of trajectory smoothing on predicted player paths. Left and central part: In each crop, the solid opaque line shows the observed prior trajectory, the solid semi-transparent line shows the ground-truth future trajectory, and the dashed opaque line shows the predicted path. Left and central representations are without and with the smoothing term enabled, respectively. Right part: Qualitative pose forecasting on WorldPose [ 32 ] test set. Ground-truth poses are shown in blue and predicted poses in red. Each row shows a single player action at five temporally equidistant frames within the 2-second forecasting horizon. Best viewed with zooming.
Method
Futures
WorldPose [ 32 ]
3DPW [ 52 ]
ADE ↓
FDE ↓
JPE ↓
APE ↓
PPE ↓
ADE ↓
FDE ↓
JPE ↓
APE ↓
PPE ↓
StaticBaseline
–
–
–
–
169.8
124.3
–
–
–
108.5
79.5
SimpliHuMoN [ 2 ]
1
757.0
1920.3
803.4
157.3
109.2
613.0
1155.5
693.4
191.6
139.4
SimpliHuMoN [ 2 ]
6
763.0
1988.3
804.4
149.8
106.7
615.6
1161.5
702.2
202.2
174.8
T2P [ 30 ]
1
584.6
1668.0
631.6
142.2
94.4
636.8
1156.9
667.5
109.7
80.5
T2P [ 30 ]
6
564.1
1625.4
613.0
142.1
94.1
650.1
1165.5
682.4
109.1
80.2
Table 4: Quantitative comparison against T2P [ 30 ] , SimpliHuMoN [ 2 ] , and a static baseline on WorldPose [ 32 ] and 3DPW [ 52 ] , with 1 second of observed context and a 2 second forecasting horizon. Futures denotes the number of sampled continuations used at inference for generative baselines, averaged when greater than one. Our model achieves competitive results on WorldPose [ 32 ] and, despite the limited social context available, the best results on 3DPW [ 52 ] .
Figure A1 : Per-joint error under uniform loss weighting (left) and with joint rebalancing (right). Each joint is colored by its mean absolute error on the test set, on a shared scale from 0 (light, low error) to 10 (dark, high error) across both skeletons, so the two panels are directly comparable. The targeted distal joints (elbows, wrists, ankles, and feet), outlined in cyan, are those with consistently above-average error under uniform weighting, and rebalancing visibly reduces their error.