Video-based human pose estimation remains challenged by motion blur, occlusion, and complex spatiotemporal dynamics. Existing methods often rely on heatmaps or implicit spatio-temporal feature aggregation, which limits joint topology expressiveness and weakens cross-frame consistency. To address these problems, we propose a novel node-centric framework that explicitly integrates visual, temporal, and structural reasoning for accurate pose estimation. First, we design a visuo-temporal velocity-based joint embedding that fuses sub-pixel joint cues and inter-frame motion to build appearance- and motion-aware representations. Then, we introduce an attention-driven pose-query encoder, which applies attention over joint-wise heatmaps and frame-wise features to map the joint representations into a pose-aware node space, generating image-conditioned joint-aware node embeddings. Building upon these node embeddings, we propose a dual-branch decoupled spatio-temporal attention graph that models temporal propagation and spatial constraint reasoning in specialized local and global branches. Finally, a node-space expert fusion module is proposed to adaptively fuse the complementary outputs from both branches, integrating local and global cues for final joint predictions. Extensive experiments on three widely used video pose benchmarks demonstrate that our method outperforms state-of-the-art methods. The results highlight the value of explicit node-centric reasoning, offering a new perspective for advancing video-based human pose estimation.
Figures & tables
Figure 1: Overview of the proposed node-centric decoupled spatio-temporal attention graph for VHPE. Given three frames, initial pose estimates and backbone features are extracted. A VTVJE encodes sub-pixel joint locations, velocity, peak score, and visibility into a joint descriptor, which the PQE refines into node embeddings. The embeddings are processed by the DSTAG: a local branch enforcing fine-grained structure and a global branch capturing long-range context. The NSEF module then fuses both branches to produce node predictions. Finally, a fixed GNC-Decoding module renders heatmaps from node predictions, supervised by a visibility-aware node loss.
Figure 2: Illustration of the Local and Global masks for one joint. The yellow box denotes the local window, while the blue box indicates the larger global window. Left: past frame t−2 with the heatmap peak and local patch. Middle: past frame t−1 with the peak shifted by the estimated motion between t−2 and t−1 . Right: current frame t , where the velocity-guided node window is centered at the extrapolated position.
Figure 3: Adjacency matrices used in the spatial GAT branches. Each row and column corresponds to a body joint, and nonzero entries indicate neighboring joints. (a) The 1-hop skeleton adjacency matrix used in the local spatial GAT branch. (b) The 2-hop skeleton adjacency matrix used in the global spatial GAT branch.
Method
Frames used
PoseWarper [ 3 ]
(t−3,…,t+3)
DCPose [ 18 ]
(t−1,t,t+1)
SLT-Pose [ 10 ]
(t−2,…,t+2)
FAMI-Pose [ 19 ]
(t−2,…,t+2)
DGNN [ 28 ]
(t−3,t−2,t−1,t)
DSTA [ 11 ]
(t−2,…,t+2)
Table 1: Temporal window configuration for each method. Unlike the majority of offline methods that rely on future frames, our approach is fully online and processes only past and current frames.
Method
Head
Shoulder
Elbow
Wrist
Hip
Knee
Ankle
Mean
DCPose [ 18 ]
83.2
84.7
82.3
78.1
80.3
79.2
73.5
80.5
FAMI-Pose [ 19 ]
83.3
85.4
82.9
78.6
81.3
80.5
75.3
81.2
DiffPose [ 9 ]
84.7
85.6
83.6
80.8
81.4
83.5
80.0
82.9
DSTA [ 11 ]
87.5
87.0
84.2
81.4
82.3
82.5
77.7
83.5
TDMI-ST [ 8 ]
86.8
87.4
85.1
81.4
83.8
82.7
78.0
83.8
JM-Pose [ 26 ]
85.8
88.1
85.7
82.5
84.1
83.1
78.5
84.0
Table 2: PoseTrack21 comparison with per-joint AP and mean.
Method
Head
Shoulder
Elbow
Wrist
Hip
Knee
Ankle
Mean
PoseWarper [ 3 ]
79.9
86.3
82.4
77.5
79.8
78.8
73.2
79.7
DCPose [ 18 ]
84.0
86.6
82.7
78.0
80.4
79.3
73.8
80.9
SLT-Pose [ 10 ]
84.3
87.5
83.5
78.5
80.9
80.2
74.4
81.5
FAMI-Pose [ 19 ]
85.5
87.7
84.2
79.2
81.4
81.1
74.9
82.2
DGNN [ 28 ]
85.1
87.7
85.3
80.0
81.1
81.6
77.2
82.7
DiffPose [ 9 ]
85.0
87.7
84.3
81.5
81.4
82.9
77.6
83.0
Table 3: PoseTrack18 comparison with per-joint AP and mean.
Method
Head
Shoulder
Elbow
Wrist
Hip
Knee
Ankle
Mean
PoseWarper [ 3 ]
81.4
88.3
83.9
78.0
82.4
80.5
73.6
81.2
DCPose [ 18 ]
88.0
88.7
84.1
78.4
83.0
81.4
74.2
82.8
SLT-Pose [ 10 ]
88.9
89.7
85.6
79.5
84.2
83.1
75.8
84.2
FAMI-Pose [ 19 ]
89.6
90.1
86.3
80.0
84.6
83.4
77.0
84.8
DGNN [ 28 ]
90.9
90.7
86.0
79.2
83.8
72.7
78.0
84.9
DSTA [ 11 ]
89.3
90.6
87.3
82.6
84.5
85.1
77.8
85.6
Table 4: PoseTrack17 comparison with per-joint AP and mean.
Method
Params (M)
GFLOPs
GMACs
Ours (ViTPose-H)
643.7
573.3
285.6
Ours (ViTPose-L)
314.9
283.2
141.7
Ours (ViTPose-B)
96.2
94.0
47.1
Ours (ViTPose-S)
30.7
38.6
19.2
TIPose (ViTPose-H) [ 14 ]
666.8
585.0
–
DSTA (ViTPose-H) [ 11 ]
631.0
123.9
–
Table 5: Model complexity comparison.
Figure 4: Comparisons on challenging scenes on PoseTrack17 including heavy occlusion, rapid motion, and atypical pose configuration. (a) Input frames; prediction from (b) Ours, (c) DSTA, and (d) DCPose. False predictions are highlighted with red circles.
Components
Head
Shoulder
Elbow
Wrist
Hip
Knee
Ankle
Mean
Δ Mean
Full Model
90.0
90.7
88.6
83.5
84.3
85.9
80.1
86.4
- 0.0
LOCAL ONLY
87.8
90.2
87.5
82.7
83.6
85.3
79.2
85.4
−1.0
GLOBAL ONLY
88.5
90.2
87.0
82.0
84.1
85.1
77.7
85.2
−1.2
SPATIAL ONLY
87.9
89.7
87.1
82.1
83.2
84.6
78.2
84.9
−1.5
TEMPORAL ONLY
87.4
89.7
86.9
81.7
83.3
84.2
78.1
84.7
−1.7
NO CENTER PREDICT
88.4
90.12
87.4
82.7
84.74
85.1
79.0
85.6
−0.8
Table 6: PoseTrack17 ablation by component (per-joint AP and mean AP). The last column reports the change in mean AP relative to the Full Model (negative means worse).