Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field

Ozge Mercanoglu Sincan; Jian He Low; Sobhan Asasi; Richard Bowden

cs.CVFeb 18, 2026

Gloss-Free Sign Language Translation: An Unbiased Evaluation of Progress in the Field

Authors: Ozge Mercanoglu Sincan, Jian He Low, Sobhan Asasi, Richard Bowden

Organizations: Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, GU2 7XH, Surrey, UK

Abstract

Sign Language Translation (SLT) aims to automatically convert visual sign language videos into spoken language text and vice versa. While recent years have seen rapid progress, the true sources of performance improvements often remain unclear. Do reported performance gains come from methodological novelty, or from the choice of a different backbone, training optimizations, hyperparameter tuning, or even differences in the calculation of evaluation metrics? This paper presents a comprehensive study of recent gloss-free SLT models by re-implementing key contributions in a unified codebase. We ensure fair comparison by standardizing preprocessing, video encoders, and training setups across all methods. Our analysis shows that many of the performance gains reported in the literature often diminish when models are evaluated under consistent conditions, suggesting that implementation details and evaluation setups play a significant role in determining results. We make the codebase publicly available here (https://github.com/ozgemercanoglu/sltbaselines) to support transparency and reproducibility in SLT research.

Figures & tables

Figure 1: Reported vs reproduced results.

Figure 2: Comparison of sign language translation datasets in terms of gloss/vocabulary size, BLEU-4 score, dataset duration, and citation-based popularity. The x-axis represents the gloss size for datasets with gloss annotations (shown with solid borders), or vocabulary size from spoken language sentences for datasets without gloss annotations (shown with dashed borders). The y-axis shows the maximum BLEU-4 translation performance as reported in the literature. Circle sizes reflect the total number of hours. Color opacity encodes dataset popularity based on citation counts, with darker circles indicating more widely used datasets.

Dataset	Year	Source	Domain	Signers	Hours	Text Vocab.	Glosses	Max BLEU4	Popularity†
BSL Corpus [ 40 , 41 ]	2008-present*	Lab	Conversational	249	125	-	$\sim$ 1800	-	187
Phoenix-2014T [ 2 ]	2018	TV	Weather	9	11	3K	1066	26.75 [ 6 ]	841
MeineDGS [ 25 ]	2008-20*	Lab	Conversational	330	50	18K	$\sim$ 10000	1.08 [ 44 ]	16
CSL-Daily [ 57 ]	2021	Lab	Daily	10	23	2K	2000	21.61 [ 6 ]	256
How2Sign [ 11 ]	2021	Lab	Instructional	11	79	16K	N/A	15.5 [ 38 ]	262
BOBSL [ 1 ]	2021	TV	Diverse	39	1467	77K	N/A	7.3 [ 20 ]	82

Table 1: Summary statistics of sign language datasets. N/A demonstrates that gloss annotations are not available for the dataset, ‘-’ indicates that the value was not reported. *The BSL Corpus, initiated in 2008, and its data annotation is still in progress. MeineDGS was initiated in 2008. †Popularity is the number of citations for the associated publication(s) as of May 13, 2025.

Figure 3: Architectural overview and training objectives of the compared sign language translation methods. The right side summarizes which objectives are used by each method.

Pretraining stage
	GFSLT-VLP	Sign2GPT	SignCL	FLa-LLM	C 2 RL
Key idea	Visual-language pretraining	Pseudo-gloss pretraining	Contrastive learning on frames	Light-T in pretraining	Combine CiCO and Light-T
Code availability	Yes	Yes	Partially	No	No
number of epochs	80	100	80 *	100	200
visual backbone	ResNet18†	DinoV2	Resnet18†	ResNet18†	ResNet18†
-TempBlock	Conv1D-BN-ReLU-MaxPool	–	Conv1D-BN-ReLU-MaxPool	Conv1D-BN-ReLU	Conv1D-BN-ReLU

Table 2: Architectural design and hyperparameters of methods on the Phoenix-2014T [ 2 ] . Empty cells indicate that the information is not reported in the original papers. VLP : Visual-Language Pretraining, Light-T : Lightweight translation model, CiCO : Cross-Ingual COntrastive learning [ 8 ] . † A temporal module (Temporal Block + Temporal Encoder) follows ResNet18 in all cases. These temporal components differ across methods and are detailed in the sub-rows. * These details are not reported explicitly in SignCL, but the authors state that all training settings follow those used in GFSLT.

Figure 4: Overview of our two-stage architecture. Pretraining stage: Video frames are encoded using a ResNet-18 backbone, followed by temporal modeling with 1D convolutions and max pooling. A [CLS] token is appended and passed through a Transformer encoder to obtain a global video representation. Text is encoded similarly and summarized into a [LANG] token. Contrastive learning is then employed to match CLS-LANG token pairs. Meanwhile, a masked sentence modeling objective is used to train the text decoder. Fine-tuning stage: The pretrained visual encoder, text encoder, and decoder are loaded from pretraining and fine-tuned for sign translation using standard sequence generation.

Pretraining stage
	Phoenix-2014T			CSL-Daily
	GFSLT, SignCL	FLa-LLM	C 2 RL	GFSLT, SignCL	FLa-LLM	C 2 RL
number of epochs	80	100	200	80	100	100
learning rate	$5\times 10^{-3}$	$10^{-2}$	$10^{-2}$	$5\times 10^{-3}$ , $10^{-3}$	$\underline{5\times 10^{-3}},10^{-3}$	$\underline{5\times 10^{-3}},10^{-3}$
Translation stage
number of epochs	200	75	80	80	75	80

Table 3: Hyperparameters of different datasets across methods. Underlined values indicate the best-performing learning rate.

Panel of Figure 5 — Figure 5: Validation loss and BLEU-4 score graphs for the GFSLT-VLP baseline [ 55 ] . For CSL-Daily, validation loss remained stable or began to rise early in training when using the same learning rate as for Phoenix-2014T, suggesting that the learning rate was too high.

Reported
Method	BLEU1	BLEU2	BLEU3	BLEU4	ROUGE-L
GFSLT-VLP [ 55 ]	43.71	33.18	26.11	21.44	42.49
Sign2GPT [ 50 ]	49.54	35.96	28.83	22.52	48.94
SignCL [ 51 ]	49.76	36.85	29.97	22.74	49.04
FLa-LLM [ 7 ]	46.29	35.33	28.03	23.09	45.27
C 2 RL (ICL only) [ 6 ]	-	-	-	24.83

Table 4: Reported and reproduced results on the Phoenix-2014T.

Reported
Method	BLEU1	BLEU2	BLEU3	BLEU4	ROUGE-L
GFSLT-VLP [ 55 ]	39.37	24.93	16.26	11.00	36.44
Sign2GPT [ 50 ]	41.75	28.73	20.60	15.40	42.36
SignCL [ 51 ]	47.47	32.53	22.62	16.16	48.92
FLa-LLM [ 7 ]	37.13	25.12	18.38	14.20	37.25
C 2 RL (ICL + ECL) [ 6 ]	49.32	36.28	27.54	21.61	48.21

Table 5: Reported and reroduced results on the CSL-Daily.

Panel of Figure 6 — Figure 6: BLEU-4 progression on the Phoenix-2014T dev set for FLa-LLM [ 7 ] and C 2 RL [ 6 ] during training. Each model is first pretrained and then fine-tuned starting from the best pretrained checkpoint. Fine-tuning exhibits rapid improvement, but then saturates. This behavior causes the gap between our reproduced results and those reported in the original papers.

Stage	Optimizer	Learning rate	BLEU1	BLEU2	BLEU3	BLEU4	ROUGE-L
Pretraining	SGD	$10^{-2}$	44.63	31.72	24.15	19.50	43.69
Fine-tuning	SGD	$5\times 10^{-3}$	45.29 ± 0.26	32.08 ± 0.29	24.53 ± 0.33	19.84 ± 0.28	44.43 ± 0.15
	Adam	$5\times 10^{-6}$	44.44	32.13	24.89	20.24	44.47
	Adam	LLM: $5\times 10^{-6}$ , LLM Adapter: $5\times 10^{-4}$	x	x	x	x	x
	Adam	$5\times 10^{-7}$	46.55	33.76	26.16	21.09	44.86
	Adam	LLM: $5\times 10^{-7}$ , LLM Adapter: $5\times 10^{-5}$	45.37	32.75	25.14	20.23	43.51

Table 6: Performance of FLa-LLM [ 7 ] on Phoenix-2014T using different learning rate variations with Adam optimizer. The experiment marked with ‘x’ was aborted early due to unstable training caused by a high learning rate.

Sign Language Translation

Sign language translation maps continuous signing into written or spoken-language text using visual, pose, and linguistic cues.

See all 50 papers

Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translation
Apr 16, 2026Yiyang Jiang, Li Zhang, Xiao-Yong Wei +1Sign Language Translation Video-Language Models
The Hong Kong Polytechnic University · Sichuan University
Selective Contrastive Learning For Gloss Free Sign Language Translation
Apr 24, 2026Changhao Lai, Rui Zhao, Xuewen Zhong +2Contrastive Learning Sign Language Translation
School of Informatics, Xiamen University, China · 2Key Lab of Digital Protection and Intelligent Processing of Intangible Cultural Heritage of Fujian-Taiwan (XMU), Ministry of Culture and Tourism, China · 3National Language Resources Monitoring and Research Center for Education and Teaching Media, Xiamen University, China
Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation
Sep 7, 2026Keren Artiaga, Sabyasachi Kamila, Haithem Afli +2Sign Language Translation Sign Language Datasets
ADAPT Centre, Munster Technological University, Cork, Ireland · Manipal Institute of Technology Bengaluru, MAHE, Manipal, India · Nimbus Research Centre, Munster Technological University, Cork, Ireland +1

Apr 16, 2026J/K move · Enter open · S save

Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translation

Yiyang Jiang, Li Zhang, Xiao-Yong Wei, Li Qing

The Hong Kong Polytechnic University · Sichuan University

Sign Language Translation Video-Language Models Latent CoT Reasoning

Open paper AskPDF arXiv

Many SLT systems quietly assume that brief chunks of signing map directly to spoken-language words. That assumption breaks down because signers often create meaning on the fly using context, space, and movement. We revisit SLT and argue that it is mainly a cross-modal reasoning task, not just a straightforward video-to-text conversion. We thus introduce a reasoning-driven SLT framework that uses an ordered sequence of latent thoughts as an explicit middle layer between the video and the generated text. These latent thoughts gradually extract and organize meaning over time. On top of this, we use a plan-then-ground decoding method: the model first decides what it wants to say, and then looks back at the video to find the evidence. This separation improves coherence and faithfulness. We also built and released a new large-scale gloss-free SLT dataset with stronger context dependencies and more realistic meanings. Experiments across several benchmarks show consistent gains over existing gloss-free methods. Our code and data are available at https://github.com/fletcherjiang/SignThought.

Abstract

Sign Language Translation

Explore similar work

Think in Latent Thoughts: A New Paradigm for Gloss-Free Sign Language Translation

Selective Contrastive Learning For Gloss Free Sign Language Translation

Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation