QCLR: A Plug-in Robustification Module via Quantization for Contrastive Learning in Self-Supervised Skeleton-Based Human Action Recognition

Nanzhu Chen; Feixiang He; Yunfeng Diao; Duygu Sarikaya; He Wang

cs.CVOct 9, 2026

QCLR: A Plug-in Robustification Module via Quantization for Contrastive Learning in Self-Supervised Skeleton-Based Human Action Recognition

Authors: Nanzhu Chen, Feixiang He, Yunfeng Diao, Duygu Sarikaya, He Wang

Organizations: University of Leeds, Leeds, United Kingdom · Central South University, Changsha, China · Hefei University of Technology, Hefei, China · University College London, London, United Kingdom

Abstract

Skeleton-based human action recognition has gained increasing attention due to the stability of skeleton data across diverse environments and advances in motion capture technologies. Following the success of supervised learning approaches, self-supervised learning approaches, particularly contrastive learning ones, have achieved impressive performance. However, they remain vulnerable to adversarial attacks. In this work, we propose a plug-in robustification module via quantization for contrastive learning in self-supervised skeleton-based human action recognition (QCLR), which enhances the robustness of representations against adversarial attacks while preserving their representational capability. Specifically, we employ a dual-level quantization scheme across the encoding pipeline, which breaks the gradient propagation and absorbs small adversarial perturbations within the quantization bins. To mitigate the impact of inevitable quantization error, we further design an attention-guided multi-faceted quantization strategy to maintain the representation quality. Experiments on several benchmark datasets demonstrate that integrating QCLR into a progressive lineage of mainstream contrastive learning models substantially improves robustness against both gradient-based white-box and gradient-free black-box adversarial attacks, while preserving highly competitive classification accuracy.

Figures & tables

Figure 1: An overview of QCLR. Note that the three parallel branches extract spatial (top), spatio-temporal (middle), and temporal (bottom) features, respectively.

Model	NTU-60							NTU-120							PKU-MMD
	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	BASAR			Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	BASAR			Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	BASAR
	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$
SkeletonCLR	68.3	99.98	100	99.42	0.09	0.03	8.5	56.8	100	100	99.93	0.08	0.02	5.0	80.9	99.82	100	98.67	0.09	0.03	8.2
TRADES-SkeletonCLR	63.83	93.96	99.99	83.94	0.15	0.05	17.4	49.66	99.67	99.91	98.48	0.12	0.03	10.9	76.07	93.73	100	80.85	0.17	0.05	16.6
BEAT-SkeletonCLR	69.56	97.96	94.03	99.37	0.09	0.03	9.1	57.04	99.30	97.66	99.94	0.07	0.02	4.9	79.84	93.84	85.83	98.75	0.09	0.03	8.0
Q-SkeletonCLR (Ours)	71.02	5.99	4.13	54.86	0.36	0.04	15.4	60.63	5.19	3.24	63.74	0.32	0.04	14.5	83.21	3.29	2.04	30.93	0.37	0.05	16.9

Table 1: Results of adversarial robustness evaluation across three datasets. Evaluation metrics include the classification accuracy (%,

\uparrow

), the ASR (%,

\downarrow

) of PGD, SMART, and APGD-DLR, and the three metrics (

l

(

\uparrow

\Delta a

(

\uparrow

\Delta B/B

(%,

\uparrow

)) of BASAR. The best results for each model are indicated in bold .

Model	Classification Accuracy	Classification Accuracy with the First Implementation	Classification Accuracy with the Second Implementation	Classification Accuracy with Feature Quantization (Ours)
SkeletonCLR	68.3	1.67	42.55	72.95
AimCLR	74.3	1.67	48.65	76.07
ActCLR	80.9	1.67	64.55	80.85

Table 2: Results of different vector quantization strategies on NTU-60. The best result for each model is indicated in bold .

Defense	SkeletonCLR							AimCLR							ActCLR
	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	BASAR			Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	BASAR			Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	BASAR
	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$	Classification Accuracy	PGD ASR	SMART ASR	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$
N/A	68.3	99.98	100	99.42	0.09	0.03	8.5	74.3	99.97	100	97.64	0.10	0.04	11.7	80.9	99.96	99.71	100	0.12	0.03	10.9
Input Quantization	70.64	84.46	100	59.93	0.35	0.05	17.1	73.24	81.66	97.42	54.46	0.43	0.06	20.6	76.27	72.71	99.57	41.27	0.41	0.06	19.7
Feature Quantization	72.95	5.94	3.28	100	0.06	0.01	4.5	76.07	9.74	5.31	97.49	0.17	0.04	12.7	80.85	1.06	1.09	100	0.11	0.03	8.9
QCLR	71.02	5.99	4.13	54.86	0.36	0.04	15.4	73.77	8.07	4.79	58.93	0.41	0.06	18.6	77.47	1.30	0.84	41.40	0.43	0.06	20.4

Table 3: Results of different quantization levels of QCLR on NTU-60. The best results for each model are indicated in bold .

Step Size	SkeletonCLR					AimCLR					ActCLR
	Classification Accuracy	APGD-DLR ASR	BASAR			Classification Accuracy	APGD-DLR ASR	BASAR			Classification Accuracy	APGD-DLR ASR	BASAR
	Classification Accuracy	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$	Classification Accuracy	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$	Classification Accuracy	APGD-DLR ASR	$l$	$\Delta a$	$\Delta B/B$
N/A	68.3	99.42	0.09	0.03	8.5	74.3	97.64	0.10	0.04	11.7	80.9	100	0.12	0.03	10.9
0.01	71.86	89.09	0.32	0.05	16.0	75.85	93.52	0.28	0.05	17.7	78.72	93.29	0.29	0.05	16.9
0.05	70.84	67.62	0.36	0.05	17.0	74.57	75.43	0.39	0.06	18.6	77.87	63.88	0.41	0.06	19.3
0.1	71.02	54.86	0.36	0.04	15.4	73.77	58.93	0.41	0.06	18.6	77.47	41.40	0.43	0.06	20.4

Table 4: Results of different step sizes of input quantization on NTU-60. The best results for each model are indicated in bold .

Explore similar work

M3GCLR: Multi-View Mini-Max Infinite Skeleton-Data Game Contrastive Learning For Skeleton-Based Action Recognition
Mar 10, 2026Yanshan Li, Ke Ma, Miaomiao Wei +1Skeleton-Based Action Recognition Contrastive Learning
Shenzhen University, the Institute of Intelligence Information Processing, and the Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen 518000, China
Beyond Binary Contrast: Modeling Continuous Skeleton Action Spaces with Transitional Anchors
Apr 20, 2026Yingjie Feng, Yi Wang, Jiaze Wang +2Skeleton-Based Action Recognition Contrastive Learning
Harbin Institute of Technology, Shenzhen · Central South University · FitX Technology (Hong Kong) Limited
Quality-Preserving Imperceptible Adversarial Attack on Skeleton-based Human Action Recognition
Jun 11, 2026Ziyi Chang, Kanglei Zhou, Xiaohui Liang +1Skeleton-Based Action Recognition Adversarial Attacks
Department of Computer Science, Durham University, Durham, DH1 4FL, UK · Department of Psychology and Cognitive Science, Tsinghua University, Beijing 100084, China · State Key Laboratory of Virtual Reality Technology and Systems, Beihang University, Beijing 100191, China +1

Mar 10, 2026J/K move · Enter open · S save

M3GCLR: Multi-View Mini-Max Infinite Skeleton-Data Game Contrastive Learning For Skeleton-Based Action Recognition

Yanshan Li, Ke Ma, Miaomiao Wei, Linhui Dai

Shenzhen University, the Institute of Intelligence Information Processing, and the Guangdong Key Laboratory of Intelligent Information Processing, Shenzhen 518000, China

Skeleton-Based Action Recognition Contrastive Learning Motion Representation Learning Multi-View Learning

Open paper AskPDF arXiv

In recent years, contrastive learning has drawn significant attention as an effective approach to reducing reliance on labeled data. However, existing methods for self-supervised skeleton-based action recognition still face three major limitations: insufficient modeling of view discrepancies, lack of effective adversarial mechanisms, and uncontrollable augmentation perturbations. To tackle these issues, we propose the Multi-view Mini-Max infinite skeleton-data Game Contrastive Learning for skeleton-based action Recognition (M3GCLR), a game-theoretic contrastive framework. First, we establish the Infinite Skeleton-data Game (ISG) model and the ISG equilibrium theorem, and further provide a rigorous proof, enabling mini-max optimization based on multi-view mutual information. Then, we generate normal-extreme data pairs through multi-view rotation augmentation and adopt temporally averaged input as a neutral anchor to achieve structural alignment, thereby explicitly characterizing perturbation strength. Next, leveraging the proposed equilibrium theorem, we construct a strongly adversarial mini-max skeleton-data game to encourage the model to mine richer action-discriminative information. Finally, we introduce the dual-loss equilibrium optimizer to optimize the game equilibrium, allowing the learning process to maximize action-relevant information while minimizing encoding redundancy, and we prove the equivalence between the proposed optimizer and the ISG model. Extensive Experiments show that M3GCLR achieves three-stream 82.1%, 85.8% accuracy on NTU RGB+D 60 (X-Sub, X-View) and 72.3%, 75.0% accuracy on NTU RGB+D 120 (X-Sub, X-Set). On PKU-MMD Part I and II, it attains 89.1%, 45.2% in three-stream respectively, all results matching or outperforming state-of-the-art performance. Ablation studies confirm the effectiveness of each component.

Abstract

Explore similar work

M3GCLR: Multi-View Mini-Max Infinite Skeleton-Data Game Contrastive Learning For Skeleton-Based Action Recognition

Beyond Binary Contrast: Modeling Continuous Skeleton Action Spaces with Transitional Anchors

Quality-Preserving Imperceptible Adversarial Attack on Skeleton-based Human Action Recognition