Vocabulary-Guided Gait Recognition

Panjian Huang; Saihui Hou; Chunshui Cao; Xu Liu; Yongzhen Huang

cs.CVSep 16, 2026

Vocabulary-Guided Gait Recognition

Authors: Panjian Huang, Saihui Hou, Chunshui Cao, Xu Liu, Yongzhen Huang

Organizations: School of Artificial Intelligence, Beijing Normal University

Abstract

What is a gait? Appearance-based gait networks consider a gait as the human shape and motion information from images. Model-based gait networks treat a gait as the human inherent structure from points. However, the considerations remain vague for humans to comprehend truly. In this work, we introduce a novel paradigm Vocabulary-Guided Gait Recognition, dubbed Gait-World, which attempts to explore gait concepts through human vocabularies with Vision-Language Models (VLMs). Although VLMs have achieved the remarkable progress in various vision tasks, the cognitive capability regarding gait modalities remains limited. The success element in Gait-World is the proper vocabulary prompt where this paradigm carefully selects gait cycle actions as Vocabulary Base, bridging the gait and vocabulary feature spaces and further promoting human understanding for the gait. How to extract gait features? Although previous gait networks have made significant progress, learning solely from gait modalities on limited gait databases makes it difficult to learn universal gait features for practicality. Therefore, we propose the first Gait-World model, dubbed α-Gait, which guides the gait network learning with vocabulary knowledge from VLMs. However, due to the heterogeneity of the modalities, directly integrating vocabulary and gait features is highly challenging as they reside in different embedding spaces. To address the issues, α-Gait designs Vocabulary Relation Mapper and Gait Fine grained Detector to map and establish vocabulary relations in the gait space for detecting corresponding gait features. Extensive experiments on CASIA-B, CCPG, SUSTech1K, Gait3D and GREW reveal the potential value and research directions of vocabulary information from VLMs in the gait field.

Figures & tables

Figure 1: Vocabulary-guided gait recognition aims to explore gait concepts through human vocabularies with VLMs where the vocabulary features enable to guide the gait network learning. Specifically, the universal vocabulary space ( e.g. , initial contact) can guide the gait network to derive the corresponding semantic gait features, thereby yielding a more universal gait space for practicality.

Figure 2: Up : In the silhouette modality, VLMs struggle to identify fine-grained details ( e.g. , localizing the right leg), yet remain sensitive to basic walking patterns, which rely on overall structure. Down : The success element of Gait-World lies in leveraging gait cycle actions as the vocabularies to bridge the gap between VLMs and gait modality.

Figure 3: $\alpha$ -Gait employs the vocabulary features from Text Encoder to guide the gait feature learning. HP denotes Horizontal Partition. Gait Encoder consists of several convolution blocks, and Gait Head includes Separate FCs and BNNeck. After the gait sequence passes through Gait Encoder and HP, the gait part features are fed into the Vocabulary Relation Mapper and Gait Fine-grained Detector, which guide them with vocabulary information for identification.

Paradigm	Method	Venue	Gait Evaluation Protocol					ReID Evaluation Protocol
Paradigm	Method	Venue	CL	UP	DN	BG	Mean	CL	UP	DN	BG	Mean
Model	GaitGraph2 Teepe et al. (2021)	CVPRW22	5.0	5.3	5.8	6.2	5.1	5.0	5.7	7.3	8.8	6.7
	Gait-TR Zhang et al. (2023)	ES23	15.7	18.3	18.5	17.5	17.5	24.3	28.7	31.1	28.1	28.1
	MSGG Peng et al. (2024b)	MTA23	29.0	34.5	37.1	33.3	33.5	43.1	52.9	57.4	49.9	50.8
	SkeletonGait Fan et al. (2024)	AAAI24	40.4	48.5	53.0	61.7	50.9	52.4	65.4	72.8	80.9	67.9
Appearance	GaitSet Chao et al. (2019)	AAAI19	60.2	65.2	65.1	68.5	64.8	77.5	85.0	82.9	87.5	83.2

Table 1: The evaluation on CCPG with clothing-changing conditions.

Paradigm	Method	Venue	Probe Sequence								Overall
Paradigm	Method	Venue	NM	BG	CL	CRY	UMB	UNI	OCC	NT	Rank-1	Rank-5
Model	GaitGraph2 Teepe et al. (2021)	CVPRW22	22.2	18.2	6.8	18.6	13.4	19.2	27.3	16.4	18.6	40.2
	Gait-TR Zhang et al. (2023)	ES23	33.3	31.5	21.0	30.4	22.7	34.6	44.9	23.5	30.8	56.0
	MSGG Peng et al. (2024b)	MTA23	67.1	66.2	35.9	63.3	61.6	58.1	66.6	17.9	33.8	-
	SkeletonGait Fan et al. (2024)	AAAI24	67.9	63.5	36.5	61.6	58.1	67.2	79.1	50.1	63.0	83.5
Appearance	GaitSet Chao et al. (2019)	AAAI19	69.1	68.2	37.4	65.0	63.1	61.0	67.2	23.0	65.0	84.8

Table 2: The evaluation on SUSTech1K with different attributes (abbrev.: NM=Normal, BG=Bag, CL=Clothing, CRY=Carrying, UMB=Umbrella, UNI=Uniform, OCC=Occlusion, NT=Night).

Paradigm	Method	Venue	NM	BG	CL	Mean
Model	GaitGraph2 Teepe et al. (2022)	CVPRW22	80.3	71.4	63.8	71.8
	GaitTR Zhang et al. (2023)	ES23	94.7	89.3	86.7	90.2
	GPGait Fu et al. (2023)	ICCV23	93.6	80.2	69.3	81.0
Appearance	GaitSet Chao et al. (2019)	AAAI19	95.0	87.2	70.4	84.2
	GaitPart Fan et al. (2020)	CVPR20	96.2	91.5	78.7	88.8
	GLN Hou et al. (2020)	ECCV20	96.9	94.0	77.5	89.5

Table 3: The evaluation on CASIA-B under different conditions with Rank-1 accuracy (%).

Paradigm	Method	Venue	Gait3D			GREW
Paradigm	Method	Venue	Rank-1	Rank-5	mAP	Rank-1	Rank-5	Rank-10
Model	GaitGraph2 Teepe et al. (2022)	CVPRW22	11.2	-	-	64.8	-	-
	GaitTR Zhang et al. (2023)	ES23	7.2	-	-	48.6	-	-
	GPGait Fu et al. (2023)	ICCV23	22.4	-	-	57.0	-	-
Appearance	GaitSet Chao et al. (2019)	AAAI19	36.7	58.3	30.0	46.3	63.6	70.3
	GaitPart Fan et al. (2020)	CVPR20	28.2	47.6	47.6	44.0	60.7	67.3

Table 4: The evaluation on Gait3D and GREW.

Method	Gait3D		CCPG
Method	Rank-1	mAP	CL	UP	DN	BG	Mean
$\alpha$ -Gait	76.2	67.8	82.8	89.0	84.6	92.7	87.3
Gait Encoder	73.6	65.1	80.1	86.4	83.9	91.4	85.5
The analysis on Text Encoder
Initial Random	71.8	63.3	77.7	84.2	77.6	86.2	81.4
Learnable Query	74.1	66.8	82.3	89.0	83.9	92.5	86.9

Table 5: The ablation study on Gait3D and CCPG.

Paradigm	Variant	Vocabulary Base	Rank-1	mAP
Appearance	Gait Encoder	—	73.6	65.1
Appearance	Gait Encoder w/ $\alpha$ -Gait	[8 phases]	76.2	67.8
Model	SkeletonGait	—	37.9	29.6
Model	SkeletonGait w/ $\alpha$ -Gait	[8 phases]	40.3	31.3
Multimodel	SkeletonGait++	—	76.0	69.2
Multimodel	SkeletonGait++ w/ $\alpha$ -Gait	[8 phases]	78.1	71.5

Table 6: Ablations on modality and vocabulary expansions on Gait3D

Figure 4: The gray box represents the complete gait sequence. Give the eight gait cycle vocabularies, GFD detects the eight silhouettes with the highest Softmax response in the attention mechanism, shown in the dash boxes.

Figure 5: Model comparison on Gait3D: Rank-1 (%) vs. parameters (M) and FLOPs (G).

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Environment	Dataset	Train		Test		Condition	Stage	Channels	Strides
Environment	Dataset	Id.	Seq.	Id.	Seq.	Condition	[ $\mathcal{D}_{1}$ , $\mathcal{D}_{2}$ , $\mathcal{D}_{3}$ , $\mathcal{D}_{4}$ ]	[ $\mathcal{C}_{1}$ , $\mathcal{C}_{2}$ , $\mathcal{C}_{3}$ , $\mathcal{C}_{4}$ ]	Strides
Constrained	CASIA-B Yu et al. (2006)	74	8,140	50	5,500	CV, BG, CL	[1, 1, 1, -]	[64, 128, 256, -]	[1, 2, 1, -]
Constrained	CCPG Li et al. (2023)	100	8,187	100	8,095	CV, BG, CL	[1, 1, 1, 1]	[64, 128, 256, 512]	[1, 2, 2, 1]
In-the-wild	SUSTech1K Shen et al. (2023)	200	5988	850	19,228	Real-world	[1, 1, 1, 1]	[64, 128, 256, 512]	[1, 2, 2, 1]
	Gait3D Zheng et al. (2022b)	3,000	18,940	1,000	6,369	Real-world	[1, 4, 4, 1]	[64, 128, 256, 512]	[1, 2, 2, 1]
	GREW Zhu et al. (2021)	20,000	102,887	6,000	24,000	Real-world	[2, 4, 4, 2]	[64, 128, 256, 512]	[1, 2, 2, 1]

Appendix

Table 7: Id. and Seq. denote the number of identities and sequences. CV, BG and CL refer to cross-view and carrying bags and cross-clothing conditions.

\mathcal{D}

and

\mathcal{C}

denote the number of conv blocks and the channels in each visual stage.

Explore similar work

Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding
Oct 10, 2026Saihui Hou, Chenye Wang, Wenpeng Lang +5Gait Recognition Vision-Language Alignment
School of Artificial Intelligence, Beijing Normal University · The Hong Kong University of Science and Technology · Zhejiang University +3
Who Remains, What Changes: Identity Anchored Composed Gait Retrieval
Aug 27, 2026Jingchen Fei, Zengbin Wang, Yukun Liu +3Cross-Modal Retrieval Gait Recognition
Beijing University of Posts and Telecommunications · Huazhong University of Science and Technology
BioGait-VLM: A Tri-Modal Vision-Language-Biomechanics Framework for Interpretable Clinical Gait Assessment
Mar 9, 2026Erdong Chen, Yuyang Ji, Jacob K. Greenberg +5Gait Analysis Medical VLMs
Department of Computer Science, Drexel University · Department of Neurological Surgery, Washington University · University of California, Berkeley

Oct 10, 2026J/K move · Enter open · S save

Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding

Saihui Hou, Chenye Wang, Wenpeng Lang, Qiannan Guo+4

School of Artificial Intelligence, Beijing Normal University · The Hong Kong University of Science and Technology · Zhejiang University+3

Gait Recognition Vision-Language Alignment Motion Representation Learning

Open paper AskPDF arXiv

Walking sequences reveal human-interpretable attributes related to body structure, posture, and motion. Gait attribute recognition provides a concrete route to fine-grained gait understanding by predicting these attributes. However, existing benchmarks provide limited fine-grained annotations, while classifier-based gait attribute recognition methods typically represent candidate classes through learned output weights rather than directly matching visual evidence with textual class representations. To address the annotation gap, we present RA-GAR, a large-scale, richly annotated multimodal benchmark for gait attribute recognition, containing 1,001 subjects, 242,729 walking sequences, and annotations for 15 static and motion gait attributes. We further propose Gait-VL, an attribute-centric vision-language alignment framework that recognizes gait attributes by matching attribute-specific multi-frame visual evidence with textual class prototypes. On the visual side, attribute-specific learnable queries aggregate RGB features across spatial locations and frames into per-attribute visual embeddings. On the language side, diverse descriptions expanded by a large language model are combined with context about relevant body regions to construct textual class prototypes. Beyond attribute recognition, we evaluate the transferability of the attribute-pretrained visual encoder by using it to replace generic upstream large vision models in identityoriented gait recognition pipelines; attribute pre-training uses no identity classification or metric-learning objectives. Experiments demonstrate state-of-the-art attribute recognition performance on RA-GAR and consistent improvements in downstream identity recognition across multiple pipelines and benchmarks, supporting the effectiveness and transferability of attribute-centric gait representation learning.

Abstract

Explore similar work

Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding

Who Remains, What Changes: Identity Anchored Composed Gait Retrieval

BioGait-VLM: A Tri-Modal Vision-Language-Biomechanics Framework for Interpretable Clinical Gait Assessment