Motion-Adaptive Multi-Scale Temporal Modelling with Skeleton-Constrained Spatial Graphs for Efficient 3D Human Pose Estimation

Ruochen Li; Shuang Chen; Wenke E; Farshad Arvin; Amir Atapour-Abarghouei

cs.CVApr 4, 2026

Motion-Adaptive Multi-Scale Temporal Modelling with Skeleton-Constrained Spatial Graphs for Efficient 3D Human Pose Estimation

Authors: Ruochen Li, Shuang Chen, Wenke E, Farshad Arvin, Amir Atapour-Abarghouei

Organizations: Department of Computer Science, Durham University, UK

Abstract

Accurate 3D human pose estimation from monocular videos requires effective modelling of complex spatial and temporal dependencies. However, existing methods often face challenges in efficiency and adaptability when modelling spatial and temporal dependencies, particularly under dense attention or fixed modelling schemes. In this work, we propose MASC-Pose, a Motion-Adaptive multi-scale temporal modelling framework with Skeleton-Constrained spatial graphs for efficient 3D human pose estimation. Specifically, it introduces an Adaptive Multi-scale Temporal Modelling (AMTM) module to adaptively capture heterogeneous motion dynamics at different temporal scales, together with a Skeleton-constrained Adaptive GCN (SAGCN) for joint-specific spatial interaction modelling. By jointly enabling adaptive temporal reasoning and efficient spatial aggregation, our method achieves strong accuracy with high computational efficiency. Extensive experiments on Human3.6M and MPI-INF-3DHP datasets demonstrate the effectiveness of our approach.

Figures & tables

Fig. 1: Comparison of temporal dependency modelling strategies and efficiency–accuracy trade-offs. Left : (a) full-sequence, (b) fixed-scale, and (c) the proposed multi-scale temporal modelling. Right : Comparison of recent methods on Human3.6M in terms of MPJPE (lower is better) versus MACs/frame, showing that our method achieves state-of-the-art performance with low computational cost.

Fig. 2: Overview of the proposed framework. The model integrates (a) a Skeleton-constrained Adaptive GCN (SAGCN) for spatial modelling and (b) an Adaptive Multi-scale Temporal Modelling (AMTM) module for temporal modelling. STGC denotes sparse temporal graph convolution operation.

Method	Venue	CE	$T$	Parameter	MACs	MACs/frames	MPJPE $\downarrow$	P-MPJPE $\downarrow$	MPJPE ${}^{{\dagger}}\downarrow$
MixSTE [ 13 ]	CVPR’22	✗	243	33.6M	139.0G	572M	40.9	32.6	21.6
P-STMO [ 28 ]	ECCV’22	✓	243	6.2M	0.7G	740M	42.8	34.4	29.3
PoseFormerV2 [ 12 ]	CVPR’23	✓	243	14.3M	0.5G	528M	45.2	35.6	–
GLA-GCN [ 26 ]	ICCV’23	✓	243	1.3M	1.5G	1556M	44.4	34.8	21.0
MotionBERT [ 20 ]	ICCV’23	✗	243	42.3M	174.8G	719M	39.2	32.9	17.8
KTPFormer [ 17 ]	CVPR’24	✗	243	33.7M	69.5G	286M	40.1	31.9	19.0

TABLE I: Quantitative comparisons on the Human3.6M dataset. CE indicates whether the model estimates the center frame only.

T

denotes the number of input frames. MACs/frames represents the number of multiply-accumulate operations per output frame. MPJPE † indicates that ground-truth 2D poses are used as input. The best results are shown in bold and the second-best results are underlined.

Methods	Venue	$T$	PCK $\uparrow$	AUC $\uparrow$	MPJPE $\downarrow$
MixSTE [ 13 ]	CVPR’22	27	94.4	66.5	54.9
P-STMO [ 28 ]	ECCV’22	81	97.9	75.8	32.2
PoseFormerV2 [ 12 ]	CVPR’23	81	97.9	78.8	27.8
GLA-GCN [ 26 ]	ICCV’23	81	98.5	79.1	27.7
KTPFormer [ 17 ]	CVPR’24	81	98.9	85.9	16.7
MotionAGFormer-L [ 18 ]	WACV’24	81	98.2	85.3	16.2

TABLE II: Quantitative comparison on MPI-INF-3DHP dataset.

T

denotes the number of input frames. PCK and AUC are reported in percentage (%). The best results are shown in bold and the second-best results are underlined.

Variant	MPJPE $\downarrow$
Baseline (w/o AMTM & SAGCN)	41.1
Only AMTM	38.5
Only SAGCN	39.9
AMTM + SAGCN (w/o adaptive aggregation)	38.3
AMTM (w/o multi-scale fusion) + SAGCN	38.6
AMTM + SAGCN ( Ours )	37.8

TABLE III: Ablation study of different model variants on Human3.6M.

Table 6

Fig. 3: Qualitative comparisons of our method with MotionAGFormer and TCPFormer on in-the-wild videos. We highlight inaccurate or ambiguous 2D detections with light-yellow arrows and indicate the corresponding deviations in the reconstructed 3D poses using orange arrows.

Fig. 4: Qualitative comparisons between our method and TCPFormer on the Human3.6M dataset for the Sitting and Walk actions. Black dashed circles indicate highlighted regions.

Fig. 5: Visualisation of the scale weight distribution for the Walk and SittingDown actions on the Human3.6M dataset (243 frames). Body joints are grouped into lower body (hip, knee, ankle), torso (root, spine, thorax, neck, head), and upper body (shoulder, elbow, wrist). Bars represent the average selection weights assigned to three temporal scales: short (9 frames), medium (27 frames), and long (81 frames).

Explore similar work

HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation
Aug 12, 2026Ruochen Li, Shuang Chen, Wenke E +23D Reconstruction 3D Human Pose Estimation
Department of Computer Science Durham University Durham, UK
DSR-Net: Learning Dynamic Skeletal Relations for 3D Human Pose Estimation
Oct 10, 2026Fan Li, Jian Xiang, LinLin Xue +13D Human Pose Estimation Dynamic Graph Learning
School of Computer Science and Technology, Zhejiang University of Science and Technology, Liuxia, Hangzhou 310023, China
PRGCN: A Graph Memory Network for Cross-Sequence Pattern Reuse in 3D Human Pose Estimation
Oct 22, 2025Zhuoyang Xie, Yibo Zhao, Hui Huang +2Memory-Augmented Neural Networks Graph Neural Networks
College of Computer Science and Artificial Intelligence, Wenzhou University, Wenzhou, 325035, P.R China. · the Key Laboratory of Computer Vision and System, Ministry of Education, Tianjin University of Technology, Tianjin, 300384, China. · School of Data Science and Artificial Intelligence, Wenzhou University of Technology, Wenzhou, 325000, P.R China. +1

Aug 12, 2026J/K move · Enter open · S save

HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation

Ruochen Li, Shuang Chen, Wenke E, Farshad Arvin+1

Department of Computer Science Durham University Durham, UK

3D Reconstruction 3D Human Pose Estimation Spatiotemporal GNNs

Open paper AskPDF arXiv

Transformer-based methods have achieved strong performance in monocular 3D human pose estimation, but most existing approaches organise spatial and temporal reasoning as separate stages, which may weaken unified spatial-temporal interdependencies inherent in human motion and compress frame-level structural information before temporal modelling. In this paper, we propose HSTGFormer, a graph-enhanced Transformer framework that reformulates spatial-temporal reasoning as localised coupled graph aggregation over joint-time nodes. Specifically, HSTGFormer introduces a Hyper Spatial-Temporal Graph (HSTG), which decomposes global spatial-temporal reasoning into local spatial-temporal receptive fields around individual joint-time nodes by extending per-frame skeleton graphs into temporal neighbourhoods, thereby enabling structure-aware coupled reasoning while preserving local structural motion information. It further incorporates an Adaptive Dual-Scale Temporal Graph (ADSTG) to capture joint-specific temporal dependencies over complementary short- and long-range windows. A lightweight node-wise fusion module further adaptively integrates the two graph representations for each joint-time node. Experiments on Human3.6M and MPI-INF-3DHP show that HSTGFormer achieves strong accuracy with high computational efficiency.

Abstract

Explore similar work

HSTGFormer: Hyper Spatial-Temporal Graph Transformer for 3D Human Pose Estimation

DSR-Net: Learning Dynamic Skeletal Relations for 3D Human Pose Estimation

PRGCN: A Graph Memory Network for Cross-Sequence Pattern Reuse in 3D Human Pose Estimation