Learning Context-Adaptive Motion Priors for Masked Motion Diffusion Models with Efficient Kinematic Attention Aggregation

Junkun Jiang; Jie Chen; Ho Yin Au; Jingyu Xiang

cs.CVMar 8, 2026

Learning Context-Adaptive Motion Priors for Masked Motion Diffusion Models with Efficient Kinematic Attention Aggregation

Authors: Junkun Jiang, Jie Chen, Ho Yin Au, Jingyu Xiang

Organizations: Department of Computer Science, Hong Kong Baptist University, Hong Kong SAR, China

Abstract

Vision-based motion capture solutions often struggle with occlusions, which result in the loss of critical joint information and hinder accurate 3D motion reconstruction. Other wearable alternatives also suffer from noisy or unstable data, often requiring extensive manual cleaning and correction to achieve reliable results. To address these challenges, we introduce the Masked Motion Diffusion Model (MMDM), a diffusion-based generative reconstruction framework that enhances incomplete or low-confidence motion data using partially available high-quality reconstructions within a Masked Autoencoder architecture. Central to our design is the Kinematic Attention Aggregation (KAA) mechanism, which enables efficient, deep, and iterative encoding of both joint-level and pose-level features, capturing structural and temporal motion patterns essential for task-specific reconstruction. We focus on learning context-adaptive motion priors, specialized structural and temporal features extracted by the same reusable architecture, where each learned prior emphasizes different aspects of motion dynamics and is specifically efficient for its corresponding task. This enables the architecture to adaptively specialize without altering its structure. Such versatility allows MMDM to efficiently learn motion priors tailored to scenarios such as motion refinement, completion, and in-betweening. Extensive evaluations on public benchmarks demonstrate that MMDM achieves strong performance across diverse masking strategies and task settings. The source code is available at https://github.com/jjkislele/MMDM.

Figures & tables

Fig. 1: Architecture comparison of the proposed Masked Motion Diffusion Model (MMDM) against other methods. (a) Masked Autoencoders (MAEs) [ 4 , 29 , 30 ] reconstruct masked (low-confidence) joints from unmasked (visible/high-confidence) joints, but they are not designed for noisy input. (b) Motion diffusion models [ 27 , 31 ] denoise pose sequences to generate high-quality motions, which typically require complete input tokens. (c) Our MMDM combines both paradigms, taking partial, noisy inputs and fusing joint- and pose-level representations via the proposed Kinematic Attention Aggregation (KAA) to output complete, high-quality motions.

Method	Pub. Venue	Joint lvl.	Pose lvl.	Objective
Method	Pub. Venue	Joint lvl.	Pose lvl.	HPE	MIB	MG
D-MAE [ 4 ]	ACMMM’22	$\checkmark$		$\checkmark$
StridedTran. [ 28 ]	TMM’22	$\checkmark$		$\checkmark$
DiffPose [ 27 ]	CVPR’23	$\checkmark$		$\checkmark$
MDM [ 17 ]	ICLR’23		$\checkmark$		$\checkmark$	$\checkmark$
GMD [ 18 ]	ICCV’23		$\checkmark$		$\checkmark$	$\checkmark$

TABLE I: Comparisons of motion representation in state-of-the-art studies, with their objectives: human pose estimation (HPE), motion in-betweening (MIB), and motion generation (MG).

Fig. 2: Illustration of the reverse diffusion process in the proposed Masked Motion Diffusion Model (MMDM). It begins at iteration $k=K$ and proceeds sequentially to $k=0$ , reconstructing the masked motion sequence $\mathbf{d}^{m}_{k}$ from Gaussian noise into high‐quality data, conditioned on the unmasked motion sequence $\mathbf{d}^{\overline{m}}$ . Specifically, at each iteration $k$ , (a) the Kinematic Encoder encodes the unmasked joints to latent tokens $\mathbf{h}^{\overline{m}}$ and yields the kinematic condition $c$ , and (b) the Motion Decoder decodes the concatenated tokens $[\mathbf{h}^{\overline{m}};\mathbf{z}_{k}^{m}]$ conditioned on $c$ . To preserve motion context, the unmasked output $\mathbf{d}^{\overline{m}}_{k}$ is replaced by the input $\mathbf{d}^{\overline{m}}$ at every step. Positional encoding involves joint indices, frame numbers, and diffusion step indices are incorporated into the hidden features before each encoding and decoding stage.

Fig. 3: Qualitative comparisons of motion capture performance. Both 2D projections and new perspectives of 3D renderings are shown. In other methods, green and blue skeletons represent the ground truth and motion capture results, respectively. For our approach, applied in the motion completion setting, green, blue, and red skeletons differentiate the ground truth, motion capture results, and completion results. Red dotted boxes highlight the failure cases, indicating that our method yields more accurate results.

Method	Shelf				Campus
Method	A-1	A-2	A-3	AVG	A-1	A-2	A-3	AVG
3DPS [ 32 ]	75.3 ${}^{\;\;\;\;}$	69.7 ${}^{\;\;\;\;}$	87.6 ${}^{\;\;\;\;}$	77.5 ${}^{\;\;\;\;}$	93.5 ${}^{\;\;\;\;}$	75.7 ${}^{\;\;\;\;}$	84.4 ${}^{\;\;\;\;}$	84.5 ${}^{\;\;\;\;}$
4DAG [ 1 ]	99.0 ${}^{\;\;\;\;}$	96.2 ${}^{\;\;\;\;}$	97.6 ${}^{\;\;\;\;}$	97.6 ${}^{\;\;\;\;}$	64.8 ${}^{\;\;\;\;}$	82.0 ${}^{\;\;\;\;}$	96.6 ${}^{\;\;\;\;}$	81.1 ${}^{\;\;\;\;}$
MVPose [ 3 ]	98.8 ${}^{\;\;\;\;}$	94.1 ${}^{\;\;\;\;}$	97.8 ${}^{\;\;\;\;}$	96.9 ${}^{\;\;\;\;}$	97.6 ${}^{\;\;\;\;}$	93.3 ${}^{\;\;\;\;}$	98.0 ${}^{\;\;\;\;}$	96.3 ${}^{\;\;\;\;}$
TesseTrack [ 2 ]	99.1 ${}^{\;\;\;\;}$	96.3 ${}^{\;\;\;\;}$	98.3 ${}^{\;\;\;\;}$	97.9 $\dagger$ ${}^{\;\;}$	97.9 ${}^{\;\;\;\;}$	95.2 ${}^{\;\;\;\;}$	99.1 ${}^{\;\;\;\;}$	97.4 ${}^{\;\;\;\;}$
PlaneSweep [ 33 ]	99.3 ${}^{\;\;\;\;}$	96.5 ${}^{\;\;\;\;}$	98.0 ${}^{\;\;\;\;}$	97.9 ${}^{\;\;\;\;}$	98.4 ${}^{\;\;\;\;}$	93.7 ${}^{\;\;\;\;}$	99.0 ${}^{\;\;\;\;}$	97.0 ${}^{\;\;\;\;}$

TABLE II: Motion capture performance on Shelf and Campus [ 51 ] datasets using the PCP (%) metric. “A-n” corresponds to the

n

-th actor, while ‘AVG’ denotes the average PCP. ‘

\dagger

’ indicates the corrected value. Bold indicates the best performance; Underlined indicates the second-best.

Fig. 4: Qualitative comparisons for motion in-betweening task. Motion sequences are sampled into key poses at a fixed ratio. Grey segments illustrate preceding and succeeding parts, while the transitioning part is color-coded from yellow to purple in a rainbow gradient, indicating the chronological order. We emphasize the joint trajectories of the pelvis, elbows, shoulders, knees, and four end-effectors. Our model generates trajectories that are closest to the ground truth, whereas other methods suffer from issues such as over-smoothing and jitter.

Dataset	Method	PCP $\uparrow$	Precision $\uparrow$	Recall $\uparrow$	MPJPE $\downarrow$
BU	4DAG [ 1 ]	82.1 ${}^{\;\;\;\;}$	95.1 ${}^{\;\;\;\;}$	94.4 ${}^{\;\;\;\;}$	70.8 ${}^{\;\;\;\;}$
	D-MAE [ 4 ]	94.3 ${}^{\;\;\;\;}$	97.5 ${}^{\;\;\;\;}$	97.5 ${}^{\;\;\;\;}$	56.3 ${}^{\;\;\;\;}$
	TEMPO [ 5 ]	88.7 ${}^{\;\;\;\;}$	86.4 ${}^{\;\;\;\;}$	86.4 ${}^{\;\;\;\;}$	96.5 ${}^{\;\;\;\;}$
	JCSAT [ 6 ]	95.5 ${}^{\;\;\;\;}$	97.9 ${}^{\;\;\;\;}$	97.9 ${}^{\;\;\;\;}$	52.2 ${}^{\;\;\;\;}$
	MMDM (Ours)	96.2 ±.52	98.1 ±.14	98.1 ±.14	52.2 ±.20
BU-X	4DAG [ 1 ]	73.1 ${}^{\;\;\;\;}$	95.1 ${}^{\;\;\;\;}$	95.1 ${}^{\;\;\;\;}$	56.5 ${}^{\;\;\;\;}$

TABLE III: Motion capture performance on the BUMocap [ 4 ] (BU) and BUMocap-X [ 6 ] (BU-X) datasets. Average PCP (%), Precision (%), Recall (%), and MPJPE (mm) are reported.

Input	Method	PCP			MPJPE			Accel
Input	Method	Before	After	$\Delta\uparrow$	Before	After	$\Delta\uparrow$	Before	After	$\Delta\uparrow$
Noise (5 cm)	SmoothNet [ 58 ]	95.4	95.6	0.2	79.3	67.4	15.0	305.8	131.3	57.0
	VPoser-t [ 35 ]		97.1	1.7		70.9	10.5		165.9	45.7
	HuMoR [ 35 ]		97.8	2.5		63.4	20.0		134.4	56.0
	MMDM (Ours)		98.3 ±.21	3.0		60.2 ±.31	24.0		132.1 ±.28	56.8
Noise (10 cm)	SmoothNet [ 58 ]	86.4	88.2	2.0	107.5	94.2	12.3	418.4	183.8	56.0

TABLE IV: Quantitative comparisons for the motion refinement task on the Shelf dataset [ 51 ] . We report the results before and after refinement, with the incremental change

\Delta

(%).

Method	L2-P $\downarrow$	L2-Q $\downarrow$	NPSS $\downarrow$
Interp.	0.0764 ${}^{\;\;\;\;}$	0.0497 ${}^{\;\;\;\;}$	0.5501 ${}^{\;\;\;\;}$
RMIB [ 46 ]	0.3830 ${}^{\;\;\;\;}$	0.2345 ${}^{\;\;\;\;}$	2.2503 ${}^{\;\;\;\;}$
CMIB [ 47 ]	0.0963 ${}^{\;\;\;\;}$	0.0490 ${}^{\;\;\;\;}$	0.4687 ${}^{\;\;\;\;}$
MDM [ 17 ]	0.2223 ±.17	0.1419 ±.09	1.1515 ±.10
GMD [ 18 ]	0.2499 ±.21	0.1109 ±.18	1.0820 ±.16
MDM ⋆	0.2051 ±.20	0.1361 ±.18	1.2511 ±.08

TABLE V: Motion in-betweening results on the BABEL-TEACH [ 55 ] dataset. The performance of the 30-frame transition is reported. MDM ⋆ and MDM ⋆⋆ represent two MDM variants.

10\times

50\times

100\times

denote results under DDIM sampling [ 42 ] .

Fig. 5: Ablation study for the motion refinement task. We report the average PCP on the Shelf dataset [ 51 ] . The line graphs depict: (a) horizontal axis for the fine-tune masking patterns, with lines showing pre-train masking patterns; (b) horizontal axis for fine-tune masking ratios, with lines showing pre-train masking ratios; (c) horizontal axis for diffusion steps.

Method		PCP $\uparrow$	MPJPE $\downarrow$	Accel $\downarrow$	FPS
SmoothNet [ 58 ]		97.4 ${}^{\;\;\;\;}$	48.7 ${}^{\;\;\;\;}$	33.3	1257.2
HuMoR [ 35 ]		98.8 ${}^{\;\;\;\;}$	43.0 ${}^{\;\;\;\;}$	35.0	31.9
MMDM (Ours)	S	98.2 ±.08	49.2 ±.11	35.6 ±.15	154.1
	T	98.3 ±.11	49.0 ±.13	35.1 ±.14	148.2
	S&T	98.8 ±.13	45.6 ±.18	33.6 ±.18	50.8
	KAA $\dagger$	98.6 ±.25	44.9 ±.19	33.1 ±.23	107.9

TABLE VI: Motion refinements results on the Shelf dataset [ 51 ] using “Mocap” data.

5\times

and

10\times

denote the speed-up ratios under DDIM sampling [ 42 ] .

Fig. 6: Computational complexity and motion in-betweening performance of each model are evaluated on the BABEL-TEACH dataset [ 55 ] using the L2P metric. The size of the circle indicates the number of parameters, with a larger area representing a greater number.

Normalization		Augmentation		Avg. PCP $\uparrow$
Hip	Centroid	Rot.	Flip	Avg. PCP $\uparrow$
$\checkmark$				94.9
$\checkmark$		$\checkmark$		95.7
$\checkmark$			$\checkmark$	96.1
$\checkmark$		$\checkmark$	$\checkmark$	97.8
	$\checkmark$			96.2

TABLE I: Impact of combinations of the data preparation processes. “Rot.” denotes the random rotation augmentation. We provide the average PCP (Avg. PCP) (%) for motion completion conducted on the Shelf dataset.

Encoder	Num.	Decoder	Num.
In Dim.	3	In Dim.	512
Feat. Dim.	512	Feat. Dim.	512
Depth	9	Depth	3
Head	4	Head	4
Head Dim.	32	Head Dim.	32
FFN Dim.	512	FFN Dim.	512

TABLE II: The detailed structure of the Masked Motion Diffusion Model, for motion completion and motion refinement tasks.

Fig. 1: Demonstration for the reserve diffusion process at $k$ time step. Green and blue skeletons denote the ground truth and the prediction, respectively. The masked joints are first sampled from a normal distribution and iteratively denoised.

Objective		Version	Avg. PCP $\uparrow$
$\epsilon$	$\mathbf{d}_{0}$	Version	Avg. PCP $\uparrow$
✓		Normal	98.47
	✓	Normal	98.48
	✓	Lite	95.41
	✓	Small	96.90
	✓	Large	98.52

TABLE III: Motion completion performance on the Shelf dataset, using the average PCP metric. We evaluate the MMDM under different objective and network structure settings.

Estimator	Avg. PCP $\uparrow$	MPJPE $\downarrow$	Pub. Venue
AlphaPose [ 61 ]	98.5	44.7	TPAMI’22
SimCC [ 62 ]	98.6	44.8	ECCV’22
OpenPose [ 52 ]	98.5	45.1	TPAMI’19

TABLE IV: Motion completion performance on the Shelf dataset, using the average PCP and MPJPE metrics. We evaluate the MMDM under different 2D pose inputs.

Encoder	Num.
In Dim.	12
Feat. Dim.	512
Depth	8
Head	8
Head Dim.	64
FFN Dim.	1024

TABLE V: The detailed structure of the proposed Masked Motion Diffusion Model, for motion in-betweening.

Method	Pre-process		L2-P $\downarrow$	L2-Q $\downarrow$	NPSS $\downarrow$
Method	Rand. Proj.	Scale	L2-P $\downarrow$	L2-Q $\downarrow$	NPSS $\downarrow$
Interp.	-	-	0.0764	0.0497	0.5501
MDM [ 17 ]	✗	-	0.2236	0.1422	1.1508
MDM [ 17 ]	$\checkmark$	-	0.1952	0.1264	1.1509
GMD [ 18 ]	$\checkmark$		0.3615	0.1410	1.4928
GMD [ 18 ]	$\checkmark$	0.2	0.2917	0.1240	1.2498

TABLE VI: Impact of GMD [ 18 ] ’s pre-processing in the motion in-betweening task. The performance of the 30-frame transition on the BABEL-TEACH dataset is reported.

3D Human Motion Reconstruction

See all 23 papers

DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction
Mar 3, 2026Yufu Wang, Evonne Ng, Soyong Shin +83D Human Motion Reconstruction 3D Human Reconstruction
Meta Reality Labs · University of Pennsylvania · Carnegie Mellon University
Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors
Mar 31, 2026Pengfei Zhou, Xiangyue Zhang, Xukun Shen +1Human Motion Generation Text-to-Motion Generation
Wuhan University, Wuhan, China
CleanMDM: Clean Motion Diffusion Model for Multimodal Motion Cleanup
Oct 4, 2026Zhe Li, Shicheng Wang, Bowen Cai +1Masked Diffusion Models Motion Diffusion Models
Peking University · Light Origins · Alibaba

Mar 3, 2026J/K move · Enter open · S save

DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction

Yufu Wang, Evonne Ng, Soyong Shin, Rawal Khirodkar+7

Meta Reality Labs · University of Pennsylvania · Carnegie Mellon University

3D Human Motion Reconstruction 3D Human Reconstruction Markerless Motion Capture 3D Reconstruction Motion Diffusion Models

Open paper AskPDF arXiv

We present DuoMo, a generative method that recovers human motion in world-space coordinates from unconstrained videos with noisy or incomplete observations. Reconstructing such motion requires solving a fundamental trade-off: generalizing from diverse and noisy video inputs while maintaining global motion consistency. Our approach addresses this problem by factorizing motion learning into two diffusion models. The camera-space model first estimates motion from videos in camera coordinates. The world-space model then lifts this initial estimate into world coordinates and refines it to be globally consistent. Together, the two models can reconstruct motion across diverse scenes and trajectories, even from highly noisy or incomplete observations. Moreover, our formulation is general, generating the motion of mesh vertices directly and bypassing parametric models. DuoMo achieves state-of-the-art performance. On EMDB, our method obtains a 16% reduction in world-space reconstruction error while maintaining low foot skating. On RICH, it obtains a 30% reduction in world-space error. Project page: https://yufu-wang.github.io/duomo/

Abstract

3D Human Motion Reconstruction

Explore similar work

DuoMo: Dual Motion Diffusion for World-Space Human Reconstruction

Not All Frames Are Equal: Complexity-Aware Masked Motion Generation via Motion Spectral Descriptors

CleanMDM: Clean Motion Diffusion Model for Multimodal Motion Cleanup