Bukva: Russian Sign Language Alphabet

Petr Surovtsev; Alexander Nagaev; Karina Kvanchiani; Elizaveta Petrova; Alexander Kapitanov

cs.CVOct 11, 2024

Bukva: Russian Sign Language Alphabet

Authors: Petr Surovtsev, Alexander Nagaev, Karina Kvanchiani, Elizaveta Petrova, Alexander Kapitanov

Organizations: SberDevices, Moscow, Russia

Abstract

This paper investigates the recognition of the Russian fingerspelling alphabet, also known as the Russian Sign Language (RSL) dactyl. Dactyl is a component of sign languages where distinct hand movements represent individual letters of a written language. This method is used to spell words without specific signs, such as proper nouns or technical terms. The alphabet learning simulator is an essential isolated dactyl recognition application. There is a notable issue of data shortage in isolated dactyl recognition: existing Russian dactyl datasets lack subject heterogeneity, contain insufficient samples, or cover only static signs. We provide Bukva, the first full-fledged open-source video dataset for RSL dactyl recognition. It contains 3,757 videos with more than 101 samples for each RSL alphabet sign, including dynamic ones. We utilized crowdsourcing platforms to increase the subject's heterogeneity, resulting in the participation of 155 deaf and hard-of-hearing experts in the dataset creation. We use a TSM (Temporal Shift Module) block to handle static and dynamic signs effectively, achieving 83.6% top-1 accuracy with a real-time inference with CPU only. The dataset, demo code, and pre-trained models are publicly available.

Figures & tables

Figure 1: Each of the RSL alphabet’s 33 signs corresponds to a letter of the Russian language alphabet. Arrows indicate the direction of movement for 8 dynamic signs.

Dataset	Letters	Samples	Signers	Resolution	Language
Images
MU HandImages ASL, 2011 ( Barczak et al., 2011 )	26	2,425	5	crop	American
ASL Fingerspelling, 2015 ( Kang et al., 2015 )	24	24,000	5	320 × 240	American
RSL dactyl, 2019 ( Makarov et al., 2019 )	26	520	20	crop	Russian
BdSL alphabet, 2021 ( Podder et al., 2020 )	37	35,149	350	crop	Bangla
PSL Alphabet 100, 2021 ( Szyszka, 2021 )	36	3,707	–	crop	Polish

Table 1: The main parameters of widespread SL alphabets in different languages. The datasets are divided into two groups, containing predominantly images or videos. Relevant Russian SL datasets are highlighted in blue. In the Resolution column, “crop” denotes images cropped to the hand region, and “signals” denotes data glove sensor recordings rather than RGB samples. The WLASL row refers only to the alphabet-letter subset of this word-level dataset.

Figure 2: Video resolution, brightness, signers, classes, and splitting analysis. a) class-based video distribution: number of videos for each RSL alphabet letter in the train and test sets; b) brightness distribution: the central image of each video sign part is extracted, converted to grayscale, and then the average pixel brightness is calculated; c) distance distribution: the distance is approximately estimated in meters by computing the length between the signers’ left and right shoulders using MediaPipe ( Zhang et al., 2020 ) ; d) signers’ devices; e) video resolution; f) signers’ gender; g) signers’ age distribution: the age is determined by the MiVOLO model ( Kuprashevich and Tolstykh, 2023 ) .

Model	Model Size (MB)	Parameters (M)	Inference Time (ms)	top-1 acc.
ResNet-18	44.8	11.2	191	72.7
ResNet-50	94.6	23.6	333	79.2
MobileNetV2	10.3	2.3	145	83.6
MobileOne S0	4.4	1.1	94	71.3
MobileOne S1	14.1	3.5	165	75.7
MobileOne S2	23.4	5.8	232	77.2

Table 2: The models’ training results on the Bukva. We evaluate models with top-1 accuracy as the standard metric in the action recognition task. All models are tested on a MacBook M1 Pro using a single core. We measured the model’s inference 100 times with further averaging.

Figure 3: The confusion matrix for test samples using the MobileNetV2 model. Off-diagonal cells indicate misclassifications. The model confuses the classes ‘‘м’’ (“m”) and ‘‘л’’ (“l”), as their signs are nearly identical.

Figure 4: Russian sign language demo stand for education.

Appendix figures & tables3 assets

Supplementary material from the paper’s appendix.

Appendix

Panel of Figure 5 — Figure 5: Example of a web page with an exam on crowdsourcing platforms translated into English.

Panel of Figure 8 — Figure 8: All crowdworkers are required to pass an exam before getting access to all crowd stages. During data collection, signers recorded the sign from the video template with further filtering to remove low-quality videos and duplicates. Only the videos remaining after the filtering stage were verified by at least three crowdworkers to ensure their correctness compared to the template. The annotation stage was also applied only to the samples remaining after the previous stage. Three annotators are asked to mark the video’s sign start and end marks. Received marks are aggregated by the algorithm proposed in Kapitanov et al. (2023) .

Letters	Sign length			Brightness			Gender		Devices		Age			Total
	Min	Max	Median	Min	Max	Median	Male	Female	Smartphone	PC	Min	Max	Median
А (A)	23	94	45.0	59	185	131	26	92	101	17	11.1	71.3	30.7	118
Б (B)	21	112	50.0	62	191	132	17	92	94	15	13.5	76.6	31.7	109
В (V)	22	105	44.0	57	192	129	21	95	95	21	10.4	71.7	32.4	116
Г (G)	25	100	43.0	69	190	125	19	92	94	18	12.7	63.9	31.4	112
Д (D)	32	105	50.0	65	190	130	20	82	84	18	11.4	66.1	31.1	102

Appendix

Table 3: Dataset statistics. “Sign length” in frames, “Brightness” in grayscale levels, “Gender”, “Devices” by records, “Age” of crowdworkers and “Total” number of videos. The Latin transliteration of each Cyrillic letter is given in brackets.

Fingerspelling Recognition

See all 8 papers

Znaki: A Russian Fingerspelling Recognition Dataset
Nov 2, 2026Petr Surovtsev, Pavel Korotaev, Alexander Nagaev +1Fingerspelling Recognition Sign Language Datasets
Department of Artificial Intelligence and Machine Learning Development, DAIMLD
JSL-DC: A Word-Level Japanese Sign Language Dataset with Linguist-Derived Descriptions for Distinguishing Confusable Signs
Aug 19, 2026Ken Takaki, Asuka Ando, Misa Suzuki +12Sign Language Datasets Sign Language Recognition
The University of Tokyo · Independent · Kwansei Gakuin University +3
Isolated Sign Language Recognition for Icelandic Sign Language: Experiments in a Low-resource Setting
Sep 22, 2026Finnur Ágúst Ingimundarson, Guðný Björk Þorvaldsdóttir, Mathias Müller +1Few-Shot Learning Sign Language Recognition
Department of Computational Linguistics, University of Zurich · Communication Centre for the Deaf and Hard of Hearing in Iceland

Nov 2, 2026J/K move · Enter open · S save

Znaki: A Russian Fingerspelling Recognition Dataset

Petr Surovtsev, Pavel Korotaev, Alexander Nagaev, Alexander Kapitanov

Department of Artificial Intelligence and Machine Learning Development, DAIMLD

Fingerspelling Recognition Sign Language Datasets Sign Language Recognition

Open paper AskPDF arXiv

Fingerspelling - the sequential signing of letters that compose proper names, technical terms, and out-of-vocabulary words - is an integral component of any sign language. Despite Russian Sign Language (RSL) being used by more than 150,000 native signers, no public fingerspelling dataset for RSL exists, in contrast to American, British, German, and several other sign languages. We introduce Znaki the first open RSL fingerspelling dataset: 37,252 video recordings (35.4 hours) of 1,593 unique sequences - proper names sampled from 16 domain categories - recorded by 68 native RSL signers (37 Deaf, 29 hard-of-hearing, and 2 certified interpreters) through a crowdsourcing pipeline with a two-stage signer qualification procedure and expert validation. The dataset covers all 33 Cyrillic handshapes, and its letter-frequency distribution matches the Russian National Corpus. As reference points for future research, we establish two baselines adapted from American Sign Language fingerspelling recognition: a Squeezeformer-based encoder-decoder, reaching a character-level accuracy (

1-\mathrm{CER}

) of 0.932 and a sequence accuracy of 0.596, and Fingerspelling-PoseNet, reaching 0.886 and 0.545, respectively.

Abstract

Fingerspelling Recognition

Explore similar work

Znaki: A Russian Fingerspelling Recognition Dataset

JSL-DC: A Word-Level Japanese Sign Language Dataset with Linguist-Derived Descriptions for Distinguishing Confusable Signs

Isolated Sign Language Recognition for Icelandic Sign Language: Experiments in a Low-resource Setting