Two-Stage Multi-View Gait Recognition with a Re-Embedding Network

Long Hoang Le; Trung Thanh Ngo

cs.CVSep 26, 2026

Two-Stage Multi-View Gait Recognition with a Re-Embedding Network

Authors: Long Hoang Le, Trung Thanh Ngo

Organizations: Hanoi University of Science and Technology, Dai Co Viet, Hanoi, 100000, Vietnam

Abstract

Gait recognition always remains challenging due to severe overfitting and the rigid view constraints common in single-stage approaches. We propose a two-stage framework, termed Translate-First-Then-Reason (TFTR), to address these issues. In the first stage, a shallow Siamese convolutional network with triplet loss maps Gait Energy Images (GEIs) into a 128-dimensional view-specific embedding space. In the second stage, these per-view embeddings are treated as tokens and processed by a 12-layer Transformer encoder, which re-projects them into a new space with improved cosine separability. This design enables flexible fusion of an arbitrary number of views at inference, overcoming the fixed-input limitations of prior methods. Trained on the OU-MVLP dataset (6,000 subjects) and evaluated on unseen CASIA-B across normal, bag-carrying, and coat-wearing conditions, our pipeline achieves 96.91% single-view and 99.49% three-view accuracy on OU-MVLP, and attains 100% accuracy on CASIA-B with three views.

Figures & tables

Figure 1: Phase 1 sequential cross-view sweep. Starting from 90 ∘ (dark marker), training moves through nearby views before crossing to the opposite side, then continues around to the best-performing 225 ∘ view (highlighted marker). The dashed arrow shows the wrap-around back to 90 ∘ for the second cycle.

Figure 2: Phase 2 opposite-view expansion. Training alternates between the two poles (90 ∘ and 270 ∘ ), stepping 15 ∘ inward from each pole on alternating turns.

View	Stage 1 ( cnn \xspace )	Stage 2 (Transformer)
$000^{\circ}$	82.10	83.49
$015^{\circ}$	88.84	90.57
$030^{\circ}$	93.62	95.04
$045^{\circ}$	94.60	95.25
$060^{\circ}$	93.70	94.61
$075^{\circ}$	94.56	95.44

Table 1: Single-view rank-1 accuracy (%) on OU-MVLP: Stage 1 ( cnn \xspace ) vs. Stage 2 (Transformer).

	000	015	030	045	060
000	83.54	97.32	98.03	98.34	97.16
015	97.32	90.52	96.83	97.45	97.69
030	98.09	96.83	95.04	96.63	97.18
045	98.34	97.45	96.58	95.25	96.82
060	97.16	97.69	97.18	96.82	94.61
075	97.84	97.36	97.09	97.47	96.88

Table 2: Two-view fusion rank-1 accuracy (%) on OU-MVLP.

Figure 3: OU-MVLP two-view fusion rank-1 accuracy (%). Diagonal cells are single-view accuracy (from Table 1 ); off-diagonal cells show the accuracy obtained by fusing the corresponding pair of views through the Stage 2 Transformer encoder. Fusion consistently and substantially improves on single-view accuracy, especially for the weakest single views ( $0^{\circ}$ , $180^{\circ}$ ).

View combination	Accuracy
000-090-180	99.00
045-195-225	99.49
210-060-030	99.07
255-075-015	98.98
270-180-090	99.15

Table 3: Three-view fusion rank-1 accuracy (%) on OU-MVLP.

Condition	View	Stage 1	Stage 2
nm	$0^{\circ}$	80.64	95.16
nm	$90^{\circ}$	83.87	85.48
nm	$180^{\circ}$	83.06	91.93
bg	$0^{\circ}$	87.90	95.96
bg	$90^{\circ}$	89.51	88.70
bg	$180^{\circ}$	87.09	93.54

Table 4: CASIA-B single-view rank-1 accuracy (%): Stage 1 ( cnn \xspace ) vs. Stage 2 (Transformer).

Condition	View combination	Accuracy
nm	$0^{\circ}-90^{\circ}-180^{\circ}$	100.00
bg	$0^{\circ}-90^{\circ}-180^{\circ}$	100.00
cl	$0^{\circ}-90^{\circ}-180^{\circ}$	100.00

Table 5: CASIA-B three-view fusion rank-1 accuracy (%):

Multi-View Feature Fusion

See all 25 papers

Gait Recognition via Deep Residual Networks and Multi-Branch Feature Fusion
Apr 30, 2026Yabo Luo, Xiaoyun Wang, Cunrong LiGait Analysis Gait Recognition
School of Mechanical and Electrical Engineering, Osh State University
Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding
Oct 10, 2026Saihui Hou, Chenye Wang, Wenpeng Lang +5Gait Recognition Vision-Language Alignment
School of Artificial Intelligence, Beijing Normal University · The Hong Kong University of Science and Technology · Zhejiang University +3
MHRGait: Gait Recognition from Momentum Human Rig Pose
Jul 31, 2026Huiran Duan, Qian Zhou, Xianda Guo +4Gait Recognition
City University of New York · School of Computer Science, Wuhan University · ELLIS Institute Finland +1

Apr 30, 2026J/K move · Enter open · S save

Gait Recognition via Deep Residual Networks and Multi-Branch Feature Fusion

Yabo Luo, Xiaoyun Wang, Cunrong Li

School of Mechanical and Electrical Engineering, Osh State University

Gait Analysis Gait Recognition

Open paper AskPDF arXiv

Gait recognition has emerged as a compelling biometric modality for surveillance and security applications, offering inherent advantages such as non-intrusiveness, resistance to disguise, and long-range identification capability. However, prevailing approaches struggle to comprehensively capture and exploit the rich biometric cues embedded in human locomotion, particularly under covariate interference including viewpoint variation, clothing change, and carrying conditions. In this paper, we present a high-precision gait recognition framework that deeply extracts and synergistically fuses gait dynamics with body shape characteristics through a multi-branch architecture grounded in deep residual learning. Specifically, we first employ the High-Resolution Network (HRNet) to perform robust skeletal keypoint estimation, preserving fine-grained spatial information even under low-resolution inputs. We then construct three complementary feature branches -- body proportion, gait velocity, and skeletal motion -- from the extracted pose sequences. A 50-layer Residual Network (ResNet-50) backbone is leveraged within a deep feature extraction module to capture hierarchically rich and discriminative representations. To effectively integrate heterogeneous feature streams, we design a Multi-Branch Feature Fusion (MFF) module inspired by channel-wise attention mechanisms, which dynamically allocates contribution weights across branches through learned activation parameters. Extensive experiments on the cross-view multi-condition CASIA-B benchmark demonstrate that our method achieves a Rank-1 accuracy of 94.52% under normal walking, with the best recognition performance among skeleton-based methods for the coat-wearing condition.

Abstract

Multi-View Feature Fusion

Explore similar work

Gait Recognition via Deep Residual Networks and Multi-Branch Feature Fusion

Attribute-Centric Vision-Language Alignment for Fine-Grained Gait Understanding

MHRGait: Gait Recognition from Momentum Human Rig Pose