Sign language recognition (SLR) systems typically require large labeled corpora for each language, yet the majority of the world's 300+ sign languages lack sufficient annotated data. Cross-lingual few-shot transfer, pretraining on a data-rich source language and adapting with only a handful of target-language examples, offers a scalable alternative, but conventional coordinate-based keypoint representations are susceptible to domain shift arising from differences in camera viewpoint, hand scale, and recording conditions. This shift is particularly detrimental in the few-shot regime, where class prototypes estimated from only K examples are highly sensitive to extrinsic variance. We propose a geometry-aware metric-learning framework centered on a compact 20-dimensional inter-joint angle descriptor derived from MediaPipe static hand keypoints. These angles are invariant to SO(3) rotation, translation, and isotropic scaling, eliminating the dominant sources of cross-dataset shift and yielding tighter, more stable class prototypes. Evaluated on four fingerspelling alphabets spanning typologically diverse sign languages, ASL, LIBRAS, Arabic Sign Language, and Thai Sign Language, the proposed angle features improve over normalized-coordinate baselines by up to 25 percentage points within-domain and enable frozen cross-lingual transfer that frequently exceeds within-domain accuracy, using a lightweight MLP encoder with about 10^5 parameters. These findings demonstrate that invariant hand-geometry descriptors provide a portable and effective foundation for cross-lingual few-shot SLR in low-resource settings.
Figures & tables
Figure 1: Pipeline overview. A hand image is processed by MediaPipe Hands into 21 keypoints (63-D when flattened), converted to one of three representations— raw (63-D), angle (20-D), or raw_angle (83-D)—and encoded into a 128-D embedding for Prototypical Network classification. In the cross-lingual setting, the encoder is either frozen or undergoes target-supervised adaptation (last-layer fine-tuning on the target train split).
Figure 2: MediaPipe Hands keypoint topology ( i∈{0,…,20} ). Keypoint 0 is the wrist (root); each finger forms a four-joint kinematic chain following the official ordering [ 29 ] : Thumb (1–4), Index (5–8), Middle (9–12), Ring (13–16), Pinky (17–20).
Encoder
Repr
Input Dim
Parameters
MLP
raw
63
116,096
MLP
angle
20
105,088
MLP
raw_angle
83
121,216
Transformer
raw
63
282,240
Transformer
angle
20
284,416
Transformer
raw_angle
83
292,480
Table 1: Encoder parameter counts by representation. All encoders output 128-D embeddings.
Dataset
Classes
Train
Test
Min nctest
Elig. ( K=5 )
ASL Alphabet [ 20 ]
29
44 498
19 093
1
28/29
LIBRAS Alphabet [ 11 ]
21
23 993
10 291
447
21/21
Arabic SL [ 1 ]
31
4 952
2 141
56
31/31
Thai Finger. [ 14 ]
42
1 998
863
12
27/42
Table 2: Dataset statistics after MediaPipe extraction and 70/30 splitting (seed 42). Eligible = classes with ≥K+Q test samples at K=5 , Q=15 .
MLP Encoder
Transformer Encoder
Dataset
Repr.
1-shot
3-shot
5-shot
1-shot
3-shot
5-shot
ASL
raw
88.7 ± 0.7
93.8 ± 0.5
94.9 ± 0.4
74.2 ± 1.0
80.3 ± 0.9
82.1 ± 0.9
angle
81.5 ± 0.9
86.8 ± 0.7
88.4 ± 0.6
80.8 ± 0.9
86.2 ± 0.7
87.7 ± 0.7
raw_angle
90.4 ± 0.7
94.5 ± 0.5
95.4 ± 0.4
90.8 ± 0.7
94.7 ± 0.4
95.4 ± 0.4
LIBRAS
raw
69.7 ± 1.0
78.9 ± 0.8
81.2 ± 0.8
61.1 ± 0.9
65.9 ± 0.9
67.9 ± 0.9
angle
89.2 ± 0.7
92.9 ± 0.5
94.1 ± 0.5
86.9 ± 0.8
91.1 ± 0.6
92.5 ± 0.5
Table 3: Within-domain few-shot accuracy (%) on the test split . 5-way K -shot, Q=15 , ProtoNet (Euclidean), 600 episodes, seed 42. Bold = best accuracy per dataset across all representations and encoders.
MLP Encoder
Transformer Encoder
Target
Mode
raw
angle
raw_angle
raw
angle
raw_angle
ASL
Frozen
97.0 ± 0.4
91.7 ± 0.5
97.1 ± 0.4
92.7 ± 0.6
87.5 ± 0.7
97.1 ± 0.3
Target-sup.
97.0 ± 0.4
91.7 ± 0.5
97.1 ± 0.4
92.7 ± 0.6
87.5 ± 0.7
97.1 ± 0.3
LIBRAS
Frozen
86.5 ± 0.8
95.0 ± 0.4
87.8 ± 0.7
81.3 ± 0.8
93.2 ± 0.5
88.3 ± 0.7
Target-sup.
94.2 ± 0.5
96.0 ± 0.4
95.2 ± 0.5
81.4 ± 0.8
93.2 ± 0.5
88.4 ± 0.7
Arabic
Frozen
74.2 ± 0.8
91.3 ± 0.5
76.6 ± 0.9
66.1 ± 0.9
87.4 ± 0.7
75.9 ± 0.9
Table 4: Effect of ASL pretraining on few-shot accuracy (%): encoder pretrained on ASL (SupCon), evaluated on each target’s test split . 5-way 5-shot, Q=15 , 600 episodes, seed 42. Bold = best accuracy within each (target, mode) pair across all representations and encoders. ASL → ASL rows are in-domain references (source=target), not cross-lingual transfer. †
Direction
Repr
Frozen
Target-sup.
LIBRAS → Arabic
raw
89.8 ± 0.6
91.8 ± 0.6
angle
91.2 ± 0.6
92.2 ± 0.5
raw_angle
91.7 ± 0.6
93.8 ± 0.5
Arabic → LIBRAS
raw
95.9 ± 0.5
96.5 ± 0.4
angle
94.9 ± 0.5
95.9 ± 0.4
raw_angle
97.1 ± 0.4
97.4 ± 0.4
Table 5: Multi-source transfer: LIBRAS ↔ Arabic per-representation breakdown (MLP, 5-way 5-shot, Q=15 , 600 episodes, seed 42). Bold = best representation within each (direction, mode) pair.
Target
ASL
LIBRAS
Arabic
Thai
ASL
95.4 ± 0.4 † ( ra )
93.7 ± 0.5 ( r )
94.1 ± 0.4 ( r )
95.1 ± 0.4 ( ra )
LIBRAS
95.0 ± 0.4 ( a )
94.1 ± 0.5 † ( a )
97.1 ± 0.4 ( ra )
94.6 ± 0.5 ( a )
Arabic
91.3 ± 0.5 ( a )
91.7 ± 0.6 ( ra )
89.8 ± 0.6 † ( a )
90.2 ± 0.5 ( a )
Thai
53.2 ± 0.8 ( a )
52.2 ± 0.8 ( ra )
54.6 ± 0.9 ( r )
52.7 ± 0.8 † ( a )
† Within-domain (no pretraining); all other cells use frozen cross-lingual transfer.
Table 6: Best accuracy (%) by source language (MLP, 5-way 5-shot, Q=15 , 600 episodes, seed 42). Off-diagonal: frozen-encoder transfer; each cell reports the highest-scoring representation (parenthesised: r = raw , a = angle , ra = raw_angle ). Diagonal † : within-domain baseline (no pretraining; from Tab. 3 ). Bold = best cross-lingual source for each target row.
Dataset
Repr
No Norm
Normalised
Diff. (p.p.)
LIBRAS
raw
76.4
81.2
4.8
LIBRAS
angle
94.4
94.1
− 0.3 †
Arabic
raw
59.1
64.5
5.4
Arabic
angle
90.1
89.8
− 0.3 †
Table 7: Effect of removing wrist-centring and scale normalisation. 5-way 5-shot, Q=15 , MLP encoder, 600 episodes, seed 42. “No norm” = MediaPipe raw output.
Setting
1-shot
3-shot
5-shot
No normalisation
41.8
56.3
59.1
+ Wrist-centring & scale
51.3
60.6
64.5
+ Geometry-aware (angle)
81.1
88.2
89.8
Table 8: Cumulative normalisation ablation on Arabic (MLP / raw). 5-way K -shot, Q=15 , 600 episodes, seed 42.
Sign language processing advances rapidly for high-resource languages such as American Sign Language (ASL), yet most of the world's sign languages lack the phonological annotations new methods require. We present the first zero-shot cross-lingual framework for handshape recognition, transferring from ASL to Catalan Sign Language (LSC). Our approach leverages the decomposition of handshapes into five phonological features -- selected fingers, flexion, spread, thumb position, and thumb contact -- shared across both languages, to decode LSC handshapes from predicted features via a composite phonological distance metric. We evaluate three architectures (MLP, SL-GCN, SHuBERT) trained on two ASL corpora (PopSign, Sem-Lex) against a 37-handshape, single-signer LSC benchmark. Zero-shot transfer proves viable once recording-format disparities are harmonized, reaching 80.0% phonological feature accuracy and 54.5% expected handshape accuracy. Phonological decomposition thus offers a bridge for extending sign language technologies to low-resource languages without any target-language video training labels.
Marcel Granero-Moya, Carolina del Corral Farrarós, Gloria Haro +2
We present the first experiments on isolated sign language recognition (ISLR) for Icelandic Sign Language (ÍTM). We use ÍTM SignWiki, a dataset derived from a bilingual Icelandic--ÍTM online dictionary. It is genuinely low-resource: 1,845 videos cover 849 classes, 86% of which have only two examples, making the full task effectively one-shot recognition across signers. We compare two open-source ISLR frameworks, OpenHands and SPOTER, on three tasks of increasing vocabulary size (22, 117 and 849 classes), and evaluate three pose estimators and two forms of cross-lingual transfer. With ÍTM data alone, SPOTER outperforms OpenHands on all three tasks, and MediaPipe poses give better results than AlphaPose or SDPose. Cross-lingual transfer brings the largest gains: pretraining SPOTER on American Sign Language data before finetuning on ÍTM raises accuracy by 14--24 percentage points, to 72.7%, 47.9% and 22.6% on the three tasks, and multilingual training with data from six other sign languages lifts OpenHands from 1.41% to 28.86% on the full task. Although far from practical use, the results suggest that transfer from better-resourced sign languages is promising for very low-resource ones. We release our adapted versions of both frameworks.
Finnur Ágúst Ingimundarson, Guðný Björk Þorvaldsdóttir, Mathias Müller +1
Department of Computational Linguistics, University of Zurich · Communication Centre for the Deaf and Hard of Hearing in Iceland
Automated sign language recognition for underrepresented languages remains a largely unsolved problem. Central African Sign Language (CASL) exemplifies this challenge: the CASL-W60 benchmark contains 60 isolated signs, and the best published result is 69.93%. We present TransSLR, a lightweight Temporal Transformer Encoder trained from scratch on normalized 64-frame pose sequences. By operating on geometric keypoint representations rather than raw RGB, TransSLR achieves 80.39% top-1 accuracy and 91.07% top-5 accuracy on a signer-independent evaluation split. This result is 10.46 percentage points higher than the published 69.93% result. TransSLR has 8.67 million trainable parameters. We also evaluate zero-shot transfer from a high-resource sign-language model, which achieves 0.00% exact-match accuracy under our manual gloss-matching protocol. Our experiments compare pose-based, RGB-based, and multimodal approaches to sign-language recognition for a low-resource language.
Lucia Yen Wanchi, Samuel Johnny, Victor Tolulope Olufemi +2