cs.CVOct 10, 2026
SaveAttribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding
Organizations: School of Artificial Intelligence, Beijing Normal University · The Hong Kong University of Science and Technology · Zhejiang University · School of Artificial Intelligence, Shenzhen University · Department of Computer Science and Engineering at the Southern University of Science and Technology · Watrix Technology Limited Co. Ltd.
Abstract
Walking sequences reveal human-interpretable attributes related to body structure, posture, and motion. Gait attribute recognition provides a concrete route to fine-grained gait understanding by predicting these attributes. However, existing benchmarks provide limited fine-grained annotations, while classifier-based gait attribute recognition methods typically represent candidate classes through learned output weights rather than directly matching visual evidence with textual class representations. To address the annotation gap, we present RA-GAR, a large-scale, richly annotated multimodal benchmark for gait attribute recognition, containing 1,001 subjects, 242,729 walking sequences, and annotations for 15 static and motion gait attributes. We further propose Gait-VL, an attribute-centric vision-language alignment framework that recognizes gait attributes by matching attribute-specific multi-frame visual evidence with textual class prototypes. On the visual side, attribute-specific learnable queries aggregate RGB features across spatial locations and frames into per-attribute visual embeddings. On the language side, diverse descriptions expanded by a large language model are combined with context about relevant body regions to construct textual class prototypes. Beyond attribute recognition, we evaluate the transferability of the attribute-pretrained visual encoder by using it to replace generic upstream large vision models in identityoriented gait recognition pipelines; attribute pre-training uses no identity classification or metric-learning objectives. Experiments demonstrate state-of-the-art attribute recognition performance on RA-GAR and consistent improvements in downstream identity recognition across multiple pipelines and benchmarks, supporting the effectiveness and transferability of attribute-centric gait representation learning.
Figures & tables
Fig. 1: Illustration of the gait-attribute design, emphasizing semantic interpretability, gait relevance, and coverage of static and motion attributes. Green labels show representative attributes included in the taxonomy, whereas red labels show clothing color and garment-type cues excluded from it.
| Type | Region | Attr. ID | Attribute | Class Labels (Index: Label) |
| Static | Full body | A1 | Gender | 0: male; 1: female |
| A2 | Age | 0: ; 1: 18–60; 2: | ||
| A3 | Height | 0: cm; 1: 150–159 cm; 2: 160–169 cm; 3: 170–179 cm; 4: cm | ||
| A4 | Body mass | 0: underweight; 1: normal; 2: overweight; 3: obesity | ||
| Head | A5 | Hair | 0: short hair; 1: long hair; 2: bald; 3: hat | |
| A6 | Forward head position | 0: no; 1: yes |
TABLE I: Attribute taxonomy and class-level annotation labels of RA-GAR. The 15 attributes include 7 binary and 8 multi-class attributes, with 45 candidate labels in total.
Fig. 2: (a) Camera configurations and walking routes of the two RA-GAR scenarios. (b) Examples of RGB images, silhouettes, 2D poses, and 3D poses under normal walking, bag carrying, and clothing changes.
Fig. 3: RA-GAR statistics: (a) distribution of sequence lengths measured in frames; (b) sequence counts for the classes of each attribute. A1–A15 correspond to the attributes in Table I ; Classes 1–5 follow the label order in that table, corresponding to indices 0–4.
| Dataset | Year | #Subj. | #Seq. | #Cam. | Cloth- Changing | Gait-Oriented Attributes | Modalities |
| CASIA-B [ 4 ] | 2006 | 124 | 13,640 | 11 | ✓ | Gender, Height | RGB, Silhouette |
| OU-MVLP [ 2 ] | 2018 | 10,307 | 288,596 | 7 | ✗ | – | Silhouette |
| GREW [ 3 ] | 2021 | 26,345 | 128,671 | 882 | ✗ | Age, Gender | Silhouette, Optical Flow, 2D/3D Pose |
| Gait3D [ 5 ] | 2022 | 4,000 | 25,309 | 39 | ✗ | – | Silhouette, 2D/3D Pose, 3D Mesh |
| CASIA-E [ 6 ] | 2023 | 1,014 | 778,752 | 8 | ✓ | Age, Gender, Height, Weight | Silhouette, Thermal IR |
| OU-ISIR LP-Age [ 9 ] | 2017 | 63,846 | 63,846 | 1 | ✗ | Age, Gender | Silhouette |
TABLE II: Comparison between RA-GAR and representative gait datasets. #Subj., #Seq., and #Cam. denote the numbers of subjects, sequences, and physical cameras, respectively. The rows above and below the horizontal separator list identity-oriented and gait-semantic benchmarks, respectively. A checkmark and a cross indicate the presence and absence of a dedicated within-subject clothing-change protocol, respectively. For MA-Gait, only the 12 non-emotional gait-oriented attributes are counted. The effective view counts of OU-MVLP, CASIA-E, MA-Gait, and the two RA-GAR scenarios are 14, 26, 12, and 10/8, respectively. Slash-separated RA-GAR values refer to Scenario 1 and Scenario 2, respectively.
Fig. 4: Overview of Gait-VL. SigLIP2 encodes RGB frames into spatial tokens. Attribute-specific temporal queries aggregate multi-frame evidence, after which attribute-specific spatial queries produce per-attribute visual embeddings through attention pooling. Here, and denote the query banks and , respectively; and denote the keys and values in the corresponding temporal and spatial stages. On the text side, LLM-expanded descriptions are combined with context about relevant body regions and encoded into cached embedding banks. During training, sampled descriptions form stochastic textual prototypes that supervise sequence-text matching through class-wise sigmoid and attribute-wise softmax objectives; during inference, each complete embedding bank is averaged into a deterministic textual prototype.
| ID | Class | Template Seed | LLM Expansion |
|---|---|---|---|
| A1 | Gender–male | The gender of this pedestrian is male. | Prioritize full body cues related to gait. The pedestrian has a male-presenting appearance while walking. |
| A5 | Hair–long | The hair of this pedestrian is long. | Pay attention to head details during walking. This walker has hair extending below the head or neck. |
| A12 | Upper arm movement–yes | The upper arms of this pedestrian move. | Emphasize upper body appearance and motion patterns. The upper arms change position clearly over the gait sequence. |
| A15 | Limping–yes | The legs of this pedestrian are limping. | Focus more on lower body characteristics in walking. The pedestrian walks with a visible limp. |
TABLE III: Template seeds and one representative LLM-expanded description for each selected class. The expansion process jointly introduces linguistic diversity and attribute-relevant gait context.
| Method | Type | Scenario 1 | Scenario 2 | Full Set | ||||||
|---|---|---|---|---|---|---|---|---|---|---|
| F1 | Acc. | mA | F1 | Acc. | mA | F1 | Acc. | mA | ||
| GaitGraph [ 25 ] | Pose | 69.38 | 82.31 | 59.75 | 74.63 | 85.15 | 63.26 | 71.76 | 83.60 | 62.26 |
| GaitGraph2 [ 70 ] | 72.62 | 83.81 | 63.06 | 78.29 | 86.89 | 67.79 | 75.24 | 85.24 | 66.13 | |
| GaitTR [ 71 ] | 74.19 | 83.97 | 65.22 | 79.71 | 87.29 | 70.05 | 76.72 | 85.49 | 68.33 | |
| GPGait [ 26 ] | 75.62 | 84.10 | 66.76 | 80.50 | 87.25 | 70.94 | 77.86 | 85.55 | 69.68 | |
| GPGait++ [ 72 ] | 72.54 | 82.49 | 62.15 | 75.63 | 84.48 | 65.80 | 73.96 | 83.40 | 64.75 | |
TABLE IV: Gait attribute recognition performance on the two scenario-defined subsets (Scenario 1 and Scenario 2) and the full RA-GAR test set (Full Set). “Type” denotes the input modality. Gait-VL (Zero-Shot) is evaluated without RA-GAR optimization. All results are reported in percentages. The best and second-best results in each column are in bold and underline , respectively.
| Region | Attribute | GAR-Net [ 7 ] | CLIP-GAR [ 13 ] | Gait-VL (Ours) |
|---|---|---|---|---|
| Full body | A1 Gender | 92.76 | 94.20 | 96.73 |
| A2 Age | 79.25 | 80.04 | 81.90 | |
| A3 Height | 61.04 | 60.10 | 61.47 | |
| A4 Body mass | 52.76 | 54.59 | 53.05 | |
| Head | A5 Hair | 68.71 | 73.14 | 77.48 |
| A6 Forward head position | 70.33 | 73.81 | 76.81 |
TABLE V: Per-attribute mA of GAR-Net, CLIP-GAR, and Gait-VL on the full RA-GAR test set. Each method uses the same checkpoint and inference settings as in Table IV . For each attribute, mA (%) is averaged over its candidate labels. Attributes are grouped by body region. The best and second-best results in each row are in bold and underline .
| Pipeline | Upstream | Input Resolution | CCPG [ 14 ] | SUSTech1K [ 15 ] | CCGR-MINI [ 16 ] | |||||
|---|---|---|---|---|---|---|---|---|---|---|
| CL | UP | DN | BG | Cloth. | R-1 | R-1 | R-5 | |||
| BigGait [ 11 ] | DINOv2-S [ 65 ] | 82.6 | 85.9 | 87.1 | 93.1 | 43.7 | 56.4 | 7.4 | 16.3 | |
| Gait-VL (Zero-Shot) | 90.9 | 91.9 | 93.2 | 95.4 | 47.9 | 63.1 | 10.5 | 20.9 | ||
| Gait-VL (Ours) | 92.2 | 92.6 | 93.5 | 95.8 | 53.7 | 65.0 | 11.1 | 22.3 | ||
| BiggerGait [ 12 ] | SAM-S [ 66 ] | 86.9 | 89.4 | 92.3 | 95.8 | 60.8 | 74.4 | 9.1 | 17.2 | |
| CLIP-S [ 43 ] | 78.9 | 83.8 | 87.9 | 96.1 | 63.1 | 81.5 | 13.9 | 24.2 | ||
TABLE VI: Evaluation of the proposed Gait-VL visual encoders in existing gait recognition pipelines. Gait-VL (Zero-Shot) and Gait-VL denote the SigLIP2-based upstream representations without and with RA-GAR attribute training, respectively. Within each pipeline, the downstream recognition architecture, training objectives, and evaluation protocols are kept unchanged. Input resolution is reported as height width. CL, UP, DN, and BG on CCPG denote clothes-changing, upper-body, lower-body, and bag-carrying protocols, respectively. Cloth. denotes the clothes-changing condition on SUSTech1K. All columns report Rank-1 accuracy (%) except R-5, which reports Rank-5 accuracy (%). The best and second-best results in each column within each pipeline are in bold and underline , respectively.
| Setting | F1 | Acc. | mA |
|---|---|---|---|
| (a) Input Resolution: Gait-VL (Zero-Shot) | |||
| (square) | 39.91 | 59.94 | 53.51 |
| (portrait) | 41.60 | 61.07 | 53.78 |
| (square) | 42.41 | 61.61 | 53.71 |
| (portrait, main setting) | 44.43 | 62.96 | 54.67 |
| (portrait) | 43.76 | 62.50 | 54.55 |
TABLE VII: Ablation on input resolution and model scale on the full RA-GAR test set. (a, b) Different input resolutions (height width) with SigLIP2-Base-Patch16-NaFlex for the evaluation-only Gait-VL (Zero-Shot) configuration and the RA-GAR-trained Gait-VL, respectively. (c) Different backbone scales for Gait-VL at the main setting. All results are reported in percentages. The best results in each column within each block, including ties at the reported precision, are in bold .
| Agg Mode | Spatial Aggregation | Temporal Aggregation | F1 | Acc. | mA |
|---|---|---|---|---|---|
| Temporal Before Spatial | Uniform Mean | Uniform Mean | 80.47 | 86.81 | 70.15 |
| Shared Query | Uniform Mean | 81.15 | 87.10 | 72.06 | |
| Per-Attr Query | Uniform Mean | 81.43 | 87.13 | 72.60 | |
| Per-Attr Query | Shared Query | 81.45 | 87.20 | 72.51 | |
| Per-Attr Query | Per-Attr Query | 81.94 | 87.59 | 73.28 | |
| Temporal After Spatial | Per-Attr Query | Per-Attr Query | 81.13 | 87.41 | 72.53 |
TABLE VIII: Ablation on attribute-aware sequence encoding on the full RA-GAR test set. “Spatial Aggregation” and “Temporal Aggregation” use uniform mean, a query shared across attributes, or per-attribute queries; “Agg Mode” denotes whether temporal aggregation is applied before or after spatial aggregation. All results are reported in percentages. The best results in each column are in bold .
| Sigmoid Loss | Softmax Loss | F1 | Acc. | mA |
|---|---|---|---|---|
| ✓ | – | 81.21 | 87.39 | 71.96 |
| – | ✓ | 18.05 | 67.68 | 52.76 |
| ✓ | ✓ | 81.94 | 87.59 | 73.28 |
TABLE IX: Ablation on training objectives on the full RA-GAR test set. “Sigmoid” denotes class-wise sigmoid loss; “Softmax” denotes attribute-wise cross-entropy. All results are reported in percentages. The best results in each column are in bold .
| Text Construction | F1 | Acc. | mA |
|---|---|---|---|
| Attribute–Label Only | 80.37 | 86.28 | 70.54 |
| Template Only | 81.21 | 86.90 | 71.77 |
| LLM Expansion | 81.94 | 87.59 | 73.28 |
TABLE X: Ablation on attribute text construction on the full RA-GAR test set. Attribute–Label Only uses concise attribute–label strings, while Template Only uses sentence-level template seeds. LLM Expansion applies the complete expansion process, including semantic diversification and attribute-relevant gait context. All results are reported in percentages. The best results in each column are in bold .