Skeleton-based rehabilitation assessment faces fundamental limitations of fixed topologies which cannot model compensatory patterns, uniform joint treatments overlook clinically important deviations, and lack of geometric understand- ing. We propose Concave-Convex Decomposition Sparse Topological Graph Convolutional Network (CC2-STG), a multi-stream model that combines geo- metric decomposition, sparse feature learning, and adaptive topology. Our framework decomposes joint relationships into complementary convex-concave distance functions to model both local and global movement characteristics, with sparse regularization with proximal optimization selecting clinically relevant geo- metric features, while multi-scale topological analysis captures non-anatomical joint dependencies in pathological movements. Experiments on the KiMoRe and UIPRMD datasets validate the effectiveness of our approach, achieving state-of-the-art performance on clinical pathological movement assessment.
Figures & tables
Figure 1: Overview of the proposed CC 2 -STG pipeline through three parallel streams.
Figure 2: The proposed CC 2 -STG architecture. (a) Overall architecture with three parallel streams-skeleton-geometry with SK attention, Convex-Concave geometric decomposition, and sparse topological streams-fused and processed through GRU layers for clinical score regression. (b) Shift-GCN block with shift-in and shift-out operations. (c) SK attention module architecture with multi-scale convolution branches.
Figure 3: Convex-Concave geometric feature extraction in which skeletal joint positions are used to compute pairwise distances, being decomposed into convex and concave components to capture both local and global movement characteristics.
Figure 4: Spare topological feature extraction pipeline. Pairwise Euclidean distances are computed between joints across temporal frames. For each joint, local descriptors are extracted at multiple scales and combined via learned softmax-normalized weights for non-anatomical joint dependencies in compensatory movements.
Model
Ex1
Ex2
Ex3
Ex4
Ex5
Ex6
Ex7
Ex8
Ex9
Ex10
Li et al. ( Li et al., 2018 )
0.011
0.029
0.036
0.014
0.017
0.019
0.027
0.025
0.027
0.047
Song et al. ( Song et al., 2020 )
0.011
0.006
0.010
0.014
0.013
0.009
0.017
0.017
0.008
0.038
Zhang et al. ( Zhang et al., 2020 )
0.022
0.003
0.016
0.016
0.003
0.003
0.021
0.025
0.027
0.066
Liao et al. ( Liao et al., 2020a )
0.011
0.028
0.029
0.012
0.019
0.012
0.018
0.023
0.023
0.042
S. Deb et al. ( Deb et al., 2022 )
0.009
0.006
0.013
0.006
0.008
0.006
0.011
0.016
0.008
0.031
Mourchid et al. ( Mourchid and Slama, 2023 )
0.011
0.009
0.013
0.009
0.009
0.013
0.022
0.020
0.013
0.014
Table 1: Performance comparison (MAD ↓ ) on UIPRMD Exercises 1–10. Best results in red and second-best in blue .
Model
MAD ↓
RMSE ↓
MAPE ↓
Ex1
Ex2
Ex3
Ex4
Ex5
Ex1
Ex2
Ex3
Ex4
Ex5
Ex1
Ex2
Ex3
Ex4
Ex5
Li et al. ( Li et al., 2018 )
1.378
1.877
1.452
0.675
1.662
2.344
2.823
2.034
1.078
2.575
3.491
5.298
4.188
2.130
3.752
Yan et al. ( Yan et al., 2018a )
0.889
2.096
0.604
0.842
1.218
2.017
3.262
0.799
1.331
1.951
2.339
6.136
1.727
2.325
3.802
Liao et al. ( Liao et al., 2020a )
1.141
1.528
0.845
0.468
0.847
2.534
3.738
1.640
0.792
1.941
2.589
3.976
2.023
1.354
2.312
Song et al. ( Song et al., 2020 )
0.977
1.282
1.033
0.715
1.536
2.165
3.345
1.679
2.018
3.198
2.605
5.202
2.968
3.599
4.959
Zhang et al. ( Zhang et al., 2020 )
1.757
3.139
1.737
1.202
1.853
2.916
4.140
2.615
1.836
2.916
5.034
10.456
5.774
3.797
6.531
Table 2: Performance comparison on the KiMoRe dataset across Exercises 1-5. All metrics report lower values as better performance. Best results per column are highlighted in red and second-best in blue .
Figure 5: Training convergence curves for CC 2 -STG on the KiMoRe dataset across Exercises 1-5 where left for MAD, enter for RMSE, and right for MAPE, and star markers indicating the epoch at which each exercise achieving optimal validation performance.
S1
S2
S3
MAD ↓
RMSE ↓
MAPE ↓
✓
✗
✗
0.360
0.794
0.830
✗
✓
✗
2.977
3.334
7.368
✗
✗
✓
1.322
2.292
3.302
✓
✓
✗
3.642
4.654
8.947
✓
✗
✓
0.439
0.733
1.035
✗
✓
✓
3.889
5.105
9.892
Table 3: Ablation study of different stream configurations.
Function pair
MAD ↓
RMSE ↓
MAPE ↓
Linear only ( d )
0.182
0.323
0.399
Squared only ( d2 )
0.218
0.300
0.481
Log only ( log(d+1) )
0.140
0.235
0.312
Polynomial pair ( d2,d3 )
0.130
0.231
0.285
RBF pair ( e−d,e−d2 )
0.114
0.171
0.238
Sqrt + log ( d,log(d+1) )
0.155
0.276
0.335
Table 4: Ablation on the choice of distance-function pair.
Figure 6: t-SNE visualization of feature representations from CC 2 -STG model. (a) Ground truth clinical score distribution. (b) Learned feature space from CC 2 -STG.
Figure 7: Learned joint importance scores from CC 2 -STG across KiMoRe Exercises 1-5 (top to bottom), where joints with highlighted borders indicating the highest importance weights.
This paper studies the problem of recovering motion from monocular video, even when the target skeleton topology differs from the subject observed in the video. Although previous methods have made efforts to resolve this problem, they remain vulnerable to joint-rotation errors that are coupled across the skeleton and can propagate along its kinematic hierarchy. To mitigate these errors, we present CAST, a topology-aware motion capture framework that combines learned local rotations with geometric correction guided by predicted global bone directions. Our observability analysis characterizes which rotational components can be constrained by outgoing bone directions. Guided by this analysis, GRACE uses reliability-gated alignment to correct composed orientations in parent-to-child order, leaving unconstrained components to the learned rotation predictor. The motion predictor and correction module are trained jointly with motion-derived supervision, using a frozen image encoder. Experiments under the TopoCap evaluation protocol show that the proposed framework improves motion recovery across different target topologies, including unseen structures. Reference-based cross-skeleton evaluation on a curated Mixamo benchmark and a user preference study further assess transfer to different target structures. A causal CAST-B variant achieves over 51 FPS on a single NVIDIA A100 GPU, with timing covering image encoding and motion prediction. Code repository: https://github.com/fankewen/CAST.
Pu Pang, Zhongfei Qing, Ling-Hao Chen +3
Xi’an Jiaotong University, China · Zhongguancun Academy, China · Tencent, China +2
Modeling motion for articulated objects of arbitrary skeleton topology remains difficult: existing motion generators target a fixed human skeleton, and prior adaptations either fail to share a vocabulary across rigs or discard motion detail through global pooling. Our key observation is that while joint-level motion does not correspond cleanly across species, motion of functional joint groups does: a human arm, a wolf foreleg, and a bird wing share motion structure despite differing joint counts and connectivity, a correspondence that joint names (e.g., "forearm", "wing_L1") partially expose even when topology does not. We introduce SAMoR (Skeleton-Aware Motion Representation for Articulated Objects), a cross-topology motion representation that encodes each motion segment as a small fixed number (K=8) of part tokens shared across arbitrary skeletons. A graph-transformer encoder consumes per-joint motion features, kinematic graph structure, and joint-name embeddings, then compresses them into part-level tokens via cross-attention pooling and residual vector quantization, yielding a discrete motion codebook shared across rigs. To keep the part queries from collapsing into redundant global representations, we introduce a topology-agnostic attention supervision loss, with joint-name dropout to reduce over-reliance on text labels. We curate a heterogeneous corpus from HumanML3D, Truebones Zoo, and animated Objaverse-XL assets, and evaluate SAMoR on held-out characters with unseen skeletons. It supports accurate reconstruction and cross-topology transfer, and enables text-conditioned generation and part-wise editing via a MaskGIT token generator. SAMoR reaches 2.75×10−2 normalized MPJPE on cross-topology reconstruction, 5.8× below the strongest adapted variable-J tokenizer baseline, while remaining competitive with fixed-skeleton specialists on HumanML3D.
Yuhao Zhang, Gerard Pons-Moll, Tolga Birdal
Imperial College London · University of Tübingen, Tübingen AI Center
Skeleton-based action recognition is widely applied in sensor-based systems, including human-computer interaction and intelligent surveillance. However, typical sensors produce sparse and discrete joint coordinates, often leading to the loss of fine-grained spatiotemporal information during dynamic movements. Furthermore, predefined physical topologies restrict modeling potential long-range dependencies. To address these challenges, we propose KGS-GCN, which integrates kinematics-driven Gaussian splatting and probabilistic topology within a graph convolutional network. A Gaussian splatting module constructs anisotropic covariance matrices by extracting instantaneous joint velocity vectors, rendering sparse skeleton sequences into multi-view continuous heatmaps rich in spatiotemporal semantics. Additionally, a probabilistic topology construction strategy transcends physical connectivity limitations by utilizing the Bhattacharyya distance to quantify statistical correlations between joint Gaussian distributions, generating an adaptive prior adjacency matrix. Finally, the lightweight multi-view rendering branch and topological GCN backbone are unified through a visual context gating mechanism, enabling seamless fusion of continuous dynamic cues with structural priors while maintaining high computational efficiency, requiring only 1.4M parameters and 1.3 GFLOPs. Extensive experiments on multiple benchmark datasets demonstrate that KGS-GCN significantly enhances the modeling of complex spatiotemporal dynamics and achieves competitive performance at low computational cost, establishing an efficient paradigm for improving the perceptual robustness of low-fidelity sensor data.
Yuhan Chen, Yicui Shi, Guofa Li +4
College of Mechanical and Vehicle Engineering, Chongqing University, Chongqing 400044, China · Department of Mathematical Sciences, Tsinghua University, Beijing 100084, China · School of Vehicle and Mobility, Tsinghua University, Beijing 100084, China +1