KHMP: Frequency-Domain Kalman Refinement for High-Fidelity Human Motion Prediction
Authors: Wenhan Wu, Zhishuai Guo, Chen Chen, Srijan Das, Hongfei Xue, Pu Wang, Aidong Lu
Organizations: School of Engineering, Yunnan University, Kunming, Yunnan, China · Department of Computer Science, University of North Carolina at Charlotte, Charlotte, NC, USA · Department of Computer Science, Northern Illinois University, DeKalb, IL, USA · Institute of Artificial Intelligence, University of Central Florida, Orlando, FL, USA
Stochastic human motion prediction aims to generate diverse, plausible futures from observed sequences. Despite advances in generative modeling, existing methods often produce predictions corrupted by high-frequency jitter and temporal discontinuities. To address these challenges, we introduce KHMP, a novel framework featuring an adaptiveKalman filter applied in the DCT domain to generate high-fidelity human motion predictions. By treating high-frequency DCT coefficients as a frequency-indexed noisy signal, the Kalman filter recursively suppresses noise while preserving motion details. Notably, its noise parameters are dynamically adjusted based on estimated Signal-to-Noise Ratio (SNR), enabling aggressive denoising for jittery predictions and conservative filtering for clean motions. This refinement is complemented by training-time physical constraints (temporal smoothness and joint angle limits) that encode biomechanical principles into the generative model. Together, these innovations establish a new paradigm integrating adaptive signal processing with physics-informed learning. Experiments on the Human3.6M and HumanEva-I datasets demonstrate that KHMP achieves state-of-the-art accuracy, effectively mitigating jitter artifacts to produce smooth and physically plausible motions.
Figures & tables
Figure 1: Comparison illustrating motion prediction for the jogging action. Previous frameworks (top) often generate predictions exhibiting physical implausibility (highlighted by red circles) and temporal discontinuity (abrupt pose transitions). KHMP (bottom) integrates physical constraints during training and adaptive frequency-Kalman refinement during inference to produce more plausible and temporally smoother motion sequences.
Figure 2: Overview of the proposed KHMP framework. During training (left), a VAE backbone model learns with standard losses augmented by structured Physical Constraints (Sec. 4.2 ) for enhanced realism. During inference (right), raw predictions are processed by the Adaptive Frequency-Kalman Refinement module (Sec. 4.3 ). This module uses SNR -based parameter adjustments and recursive Kalman filtering to adaptively smooth high-frequency noise, yielding refined predictions.
Method
Venue
HumanEva-I
Human3.6M
APD ↑
ADE ↓
FDE ↓
MMADE ↓
MMFDE ↓
APD ↑
ADE ↓
FDE ↓
MMADE ↓
MMFDE ↓
ERD [ 8 ]
ICCV 2015
0
0.382
0.461
0.521
0.595
0
0.722
0.969
0.776
0.995
DeLiGAN [ 9 ]
CVPR 2017
2.177
0.306
0.322
0.385
0.371
6.509
0.483
0.534
0.520
0.545
BoM [ 3 ]
CVPR 2018
2.846
0.271
0.279
0.373
0.351
6.265
0.448
0.533
0.514
0.544
DLow [ 34 ]
ECCV 2020
4.855
0.251
0.268
0.362
0.339
11.741
0.425
0.518
0.495
0.531
DSF [ 33 ]
ICLR 2020
4.538
0.273
0.290
0.364
0.340
9.330
0.493
0.592
0.550
0.599
Table 1: Quantitative comparison between KHMP and published state-of-the-art methods. The bold and underlined values represent the best and second-best results, respectively. * indicates our re-implementation of the baseline [ 30 ] , trained with KHMP’s hyperparameters but without physical constraints (Sec. 4.2 ) and Frequency-Kalman refinement (Sec. 4.3 ).
Table 2: Comprehensive ablation studies and quantitative analysis of the KHMP framework. (a) Analysis of the full framework components. (b) Ablation on structured physical losses. (c) Comparison of Frequency-Kalman refinement strategies. (d) Fixed Frequency Suppression [ 29 ] vs. our Adaptive Frequency-Kalman Refinement on HumanEva-I. (e) Quantitative evaluation of jitter reduction.
Figure 3: Sensitivity analysis of hyperparameters: k0 , λQ , λR , λangle , and λtemporal .
Figure 4: Visual comparison of joint trajectories demonstrating KHMP’s refinement over the baseline across two distinct actions. The top two rows ( a - d ) present examples from the Gesturing action (Ankle, Upper Arm, Wrist, and Shin). The bottom two rows ( e - h ) present examples from the Walking action (Ankle, Shin, Thigh, and Wrist). Within each subfigure, the first column displays the 3D trajectory, while the subsequent columns show the X, Y, and Z coordinates over time. Across all examples, the baseline prediction (red line) exhibits noticeable high-frequency jitter, whereas KHMP (blue line) significantly smooths the trajectory, closely adhering to the ground truth (green dashed line) and demonstrating effective refinement. Best viewed by zooming in.
Figure 5: Visual comparison highlighting KHMP’s enhanced prediction quality over the baseline across various actions ( Boxing , Jogging , Walking , and Gesturing ). The figure showcases KHMP’s ability to correct physical implausibility often present in raw generative model outputs. In the examples shown, red arrows indicate issues in baseline predictions, while blue boxes highlight the corresponding smoother and more coherent KHMP results. Best viewed by zooming in. We also provide video demos in the SupMat .
Appendix figures & tables4 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 6: Behavior of adaptive Kalman filter parameters versus estimated SNR ( SNRest ). As signal quality increases (higher SNR), observation noise R decreases sharply, while process noise Q decreases moderately. Consequently, the Kalman gain Kk increases, shifting the filter’s behavior from strong smoothing (low SNR, noisy signals) towards detail preservation (high SNR, clean signals).
Loss Term
SLD-HMP
Ours
λrecon
8.0
11.0
λh
10.0
16.0
λmm
4.0
0.1
λd
16.0
0.63
λlimb
50.0
0.5
λnf
0.002
0.002
Appendix
Table 3: Loss weights configuration.
Method
HumanEva-I
Human3.6M
APD ↑
ADE ↓
FDE ↓
MMADE ↓
MMFDE ↓
APD ↑
ADE ↓
FDE ↓
MMADE ↓
MMFDE ↓
MotionWavelet [ 7 ]
4.171
0.235
0.213
0.304
0.280
6.506
0.376
0.408
0.466
0.443
KHMP (Ours)
7.481
0.188
0.204
0.301
0.291
9.235
0.349
0.441
0.436
0.468
Appendix
Table 4: Quantitative comparison between KHMP and MotionWavelet on HumanEva-I and Human3.6M datasets. Best results are highlighted in bold.
Figure 7: Additional qualitative comparison between Baseline and KHMP predictions for selected frames from Jogging and Gesturing actions. Red arrows highlight implausible poses in the baseline, while blue boxes show the refined and more realistic results from KHMP.
Human motion prediction is a crucial capability for advanced robotic systems that interact with humans. In facilities with dynamic human-robot collaboration settings, robots must anticipate human movements to ensure safety, prevent collisions, and optimize cooperative tasks. Traditionally, motion forecasting is treated as a sequential modeling problem using historical pose data, but achieving long-term accuracy and physical realism remains challenging. We present Adversarial Motion Transformer (AdvMT), a novel approach that integrates a Transformer-based motion encoder with a temporal continuity discriminator to address these challenges. The Transformer captures rich spatio-temporal dependencies across human joints, while adversarial training with a continuity discriminator enforces smooth, natural motion trajectories that adhere to biomechanical constraints. Our training scheme includes a bone-length consistency term and adversarial loss to reduce common artifacts like pose freezing or unnatural transitions. In experiments on the Human3.6M motion dataset, AdvMT achieves state-of-the-art long-horizon prediction accuracy while also delivering robust short-term predictions. These improvements strengthen the prediction foundation for physical AI in manufacturing and human-robot collaboration, where anticipating human motion is a prerequisite for safe and efficient robot coordination.
Sarmad Idrees, Seokman Sohn, Jongeun Choi
School of Mechanical Engineering, Yonsei University, Seoul 03722, Korea · Power Generation Lab, Korea Power Research Institute, 105, Munji-Ro, Yuseong-Gu, Daejeon, 34056, South Korea
Human motion prediction combines the tasks of trajectory forecasting and human pose prediction. For each of the two tasks, specialized models have been developed. Combining these models for holistic human motion prediction is non-trivial, and recent methods have struggled to compete on established benchmarks for individual tasks. To address this, we propose a simple yet effective transformer-based model for human motion prediction. The model employs a stack of self-attention modules to effectively capture both spatial dependencies within a pose and temporal relationships across a motion sequence. This simple, streamlined, end-to-end model is sufficiently versatile to handle pose-only, trajectory-only, and combined prediction tasks without task-specific modifications. We demonstrate that this approach achieves state-of-the-art results across all tasks through extensive experiments on a wide range of benchmark datasets, including Human3.6M, AMASS, ETH-UCY, and 3DPW.
Stochastic human motion prediction aims to forecast future motion distributions. Although recent studies have achieved strong performance in terms of accuracy and diversity, they often overlook plausibility (e.g., resulting in physically unrealistic predictions) and uncertainty quantification, both of which are essential for real-world applications and downstream tasks. To address these issues, we propose a latent flow-based model equipped with a data-driven Gaussian mixture prior that more effectively disentangles diverse human behaviors than conventional single-modal priors. This prior is derived from patterns in the training data without requiring additional annotations. Furthermore, the fully invertible nature of our model enables natural uncertainty quantification through tractable likelihood computation. Experiments on the Human3.6M and AMASS datasets demonstrate that our approach achieves state-of-the-art performance in both accuracy and plausibility.
Yue Ma, Frederick W. B. Li, Xiaohui Liang
Beihang University · Durham University · Zhongguancun Laboratory