Human mesh recovery (HMR) models 3D human body from monocular videos, with recent works extending it to world-coordinate human trajectory and motion reconstruction. However, most existing methods remain offline, relying on future frames or global optimization, which limits their applicability in interactive feedback and perception-action loop scenarios such as AR/VR and telepresence. To address this, we propose OnlineHMR, a fully online framework that jointly satisfies four essential criteria of online processing, including system-level causality, faithfulness, temporal consistency, and efficiency. Built upon a two-branch architecture, OnlineHMR enables streaming inference via a causal key-value cache design and a curated sliding-window learning strategy. Meanwhile, a human-centric incremental SLAM provides online world-grounded alignment under physically plausible trajectory correction. Experimental results show that our method achieves performance comparable to existing chunk-based approaches on the standard EMDB benchmark and highly dynamic custom videos, while uniquely supporting online processing. Page and code are available at https://tsukasane.github.io/Video-OnlineHMR/.
Figures & tables
Figure 2: The online processing workflow of our method. Given a streaming video input, we estimate the world coordinate 3D human body of the most recent frame. Ti here is the homogeneous transformation matirx composed by qic and tic in Sec. 3.1 .
Figure 3: Sliding window learning pipeline. The input sequence is sliced to overlapping windows, learning spatial and temporal information fusion inside each window, and alleviate jitter effect through velocity regularization.
Table 3
Figure 4: Quantitative comparison of OnlineHMR and Human3R on the same EMDB-2 video with ground truth after world coordinate alignments.
Figure 5: Quantitative comparison of OnlineHMR and Human3R on the same EMDB-2 video with ground truth after world coordinate alignments.
Models
Online
FPS ↑
Avg. Delay (s) ↓
WA-MPJPE 100 ↓
TRACE [ 37 ]
✓
24.6
0.04
529.0
GLAMR [ 53 ]
✗
17.2
14.16
280.8
SLAHMR [ 51 ]
✗
0.1
2435
326.9
WHAM (w/ DPVO) [ 34 ]
✗
9.3
26.18
135.6
TRAM [ 47 ]
✗
2.1
115.95
76.4
GVHMR (w/ DPVO) [ 33 ]
✗
12.1
20.12
111.0
Table 3: Efficiency comparison of world coordinate HMR methods. The average delay time is presented in seconds.
Models
3DPW
EMDB-1
Accel ↓
Jitter ↓
Accel ↓
Jitter ↓
w/o Velocity Regularization
8.9
32.3
15.7
70.1
w/ Velocity Regularization
6.4
19.5
9.0
33.7
Table 4: The comparison of jittering effect w/ or w/o the velocity regularization.
Figure 6: Visualization of multi-individual, diverse scene cases. More examples are in Suppl.
Method
Vanilla
Hard Mask
Soft Mask
DROID-SLAM
2.52
1.55
1.07
MAST3R-SLAM
1.22
0.96
0.83
Table 5: Comparison of ATE using different masking strategies for DROID-SLAM and MAST3R-SLAM. Lower is better.
Figure 7: Visualization of estimated camera trajectory compared with GT.
Figure 8: Difference spectrograms computed as GT-Pred, visualizing discrepancies in the time–frequency domain. The left figure corresponds to a model trained for 1K iterations, and the right for 52K iterations. Lower values (darker/closer to zero) indicate better alignment with the ground truth. The converged model (52K) exhibits substantially reduced differences.
Dataset
GVHMR
TRAM
Ours
3DPW
3.59 / 0.02
17.01 / 0.64
19.91 / 0.82
EMDB-1
75.52 / 0.12
25.82 / 1.24
24.98 / 1.41
Table 6: RMSE ↓ / Corr ↓ results.
3DPW
EMDB-1
SWS
PA-MPJPE ↓
MPJPE ↓
PVE ↓
ACCEL ↓
PA-MPJPE ↓
MPJPE ↓
PVE ↓
ACCEL ↓
3
43.7
69.9
83.7
6.4
46.0
74.0
86.1
9.0
4
41.7
64.9
78.6
6.4
46.8
73.8
85.3
9.1
5
43.2
70.8
85.7
6.6
46.7
74.5
86.5
9.4
6
42.5
65.5
80.1
6.6
46.9
74.3
84.4
9.6
Table 7: Ablation study on sliding window size (SWS).
Strategy
WA-MPJPE
W-MPJPE
RTE
ERVE
Vanilla
119.6
412.9
4.1
14.4
Hard Mask
112.6
386.8
3.2
13.3
Soft Mask
93.5
310.4
2.2
12.4
Table 8: Comparison of different masking strategies on world coordinate human reconstruction upon MASt3R-SLAM [ 25 ]
Figure 9: Using the same video input with Fig. 12 , we show the global human mesh, camera trajectory, and the final updated scene.
Figure 10: Qualitative results w/ and w/o EMA correction on custom videos.
Figure 11: Example of failure cases.
Figure 12: Quantitative comparison of OnlineHMR and Human3R on a custom video of a famous dancer. OnlineHMR produces a faithful reconstruction of the human trajectory, whereas Human3R yields trajectories that appear compressed and crowded together.
Figure 13: More examples with multiple individuals and a diverse scene. Dynamic results can be found on the demo page.
Figure 14: Quantitative comparison of OnlineHMR and Human3R on a custom video of running up stairs.