Sign Language Translation

Ask about this topic

Overview

Sign language translation turns continuous video of signing into text, requiring models to capture hand motion, facial expression, timing, and sign-language grammar. Systems may predict intermediate glosses or translate directly from video or pose, using methods such as CTC pretraining, temporal alignment, multimodal fusion, and language-model decoding. Researchers evaluate translation on benchmarks such as PHOENIX14T, CSL-Daily, How2Sign, and OpenASL, while also exploring low-resource languages and real-time streaming. A key tension is that standard text-overlap scores can miss meaning errors, and results may be inflated by signer overlap or inconsistent evaluation; reliable progress also depends on broader data and community-informed testing.

Latest papers

CardsList

50 papers7 papers in the last four weeks, up 17% on the four weeks before. 0.0% of all new papers.

  1. Isharah-Selfie: Continuous Sign Language Recognition Dataset for One-handed Signing

    Oct 23, 2026Ahmed Abul Hasanaath, Murtadha Aljubran, Sarah Alyami +2Sign Language DatasetsSign Language Recognition

    King Fahd University of Petroleum & Minerals · Mohamed bin Zayed University of Artificial Intelligence · Imam Abdulrahman Bin Faisal University

  2. SignRAG: Unified Retrieval-Augmented Gloss-Free Sign Language Translation

    Oct 8, 2026Zhi Rao, Yucheng Zhou, Qianran Sun +8Sign Language TranslationRetrieval-Augmented Generation

    Faculty of Innovation Engineering, Macau University of Science and Technology, Macau, China · SKL-IOTSC, CIS, University of Macau, Macau, China · MAIS, Institute of Automation, Chinese Academy of Sciences, Beijing, China +2

  3. Not What a Child Expressed: Auditing the Sign-to-Text Safety Interface in Child-Facing AI

    Oct 5, 2026Muhammad Rafiullah Memon, Viet Vo, Wanlun Ma +1Sign Language TranslationAI Safety Evaluation

    School of Science, Computing and Emerging Technologies Swinburne University of Technology, Hawthorn, VIC, Australia

  4. ReSCUE: Re-translation with Sentence Commitment for Unsegmented Long-Form Simultaneous Sign Language Translation

    Oct 2, 2026Sihan Ren, Gaozheng Li, Yuanshang Quan +5Sign Language TranslationMachine Translation

    ShanghaiTech University · DGene · University of Derby

  5. Machine Translation for Sign Languages

    Oct 1, 2026Ozge Mercanoglu Sincan, Anton Pelykh, Edward Fish +7Sign Language TranslationSign Language Recognition

    Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey, Guildford, UK, GU2 7XH.

  6. Simultaneous Translation between Sign Languages

    Sep 28, 2026Zetian Wu, Bowen Xie, Stefan Lee +1Sign Language Translation

    Oregon State University

  7. SignFLIP: A Unified Model for Sign Language Translation and Generation via Stage-wise Alignment at Scale

    Sep 28, 2026Zhaoyi An, Sihan Tan, Youngbae Hwang +2Sign Language TranslationRepresentation Alignment

    Institute of Science Tokyo · Chungbuk National University

  8. OneSign: Unifying Sign Language Understanding Tasks with One Model

    Sep 27, 2026Shiwei Gan, Yafeng Yin, Xiao Liu +3Sign Language RecognitionUnified Multimodal Models

    State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing 210023, China

  9. SignGPT: Toward LLM-Mediated Sign Language Interaction through Gloss-Free Translation and Generation

    Sep 18, 2026Ronghui Li, Jun Dong, Zhongyuan Hu +8Sign Language TranslationSign Language Generation

    Tsinghua University, China · Tencent AI Lab, China

  10. Investigating Temporal Motion Features for Pose-to-Text Indian Sign Language Translation

    Sep 11, 2026Manav Dhamecha, Praveen Kumar Chandaliya, Pruthwik MishraSign Language TranslationTransformer

    Sardar Vallabhbhai National Institute of Technology, Surat, India

  11. Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation

    Sep 7, 2026Keren Artiaga, Sabyasachi Kamila, Haithem Afli +2Sign Language TranslationSign Language Datasets

    ADAPT Centre, Munster Technological University, Cork, Ireland · Manipal Institute of Technology Bengaluru, MAHE, Manipal, India · Nimbus Research Centre, Munster Technological University, Cork, Ireland +1

  12. DINO Eats Geometry: Self-Supervised Sign Representations that Absorb Articulated Hand Geometry

    Sep 5, 2026Junyi Hu, Zhewen He, Haomian Huang +4Self-Supervised LearningSign Language Recognition

    New York University Abu Dhabi

  13. Beyond BLEU: A Case for Redefining Sign Language Translation Benchmarks

    Sep 3, 2026Oline Ranum, Edward Fish, Simon Hadfield +1Sign Language TranslationMT Evaluation

    Centre for Vision, Speech and Signal Processing (CVSSP), University of Surrey

  14. ODE-Based Transformer Decoders for Iterative Sign Language Translation

    Aug 11, 2026Tuğçe Kızıltepe, Hacer Yalim KelesSign Language TranslationIterative Refinement

    ASELSAN, Ankara, Türkiye · Department of Computer Engineering, Hacettepe University, Ankara, Türkiye

  15. Bridging the Gap Between Semantics and Reconstruction:Unifying Sign Language Translation and Production

    Aug 10, 2026Xiao Liu, Shiwei Gan, Yafeng Yin +5Sign Language TranslationRepresentation Learning

    State Key Laboratory of Novel Software Technology, Nanjing University, Suzhou, Jiangsu, China · State Key Laboratory of Novel Software Technology, Nanjing University, Nanjing, Jiangsu, China

  16. SignLlama: Enhancing Gloss-free Sign Language Translation by Prioritizing Visual Features for LLMs

    Aug 10, 2026Shiwei Gan, Xiao Liu, Yafeng Yin +5Sign Language TranslationVLM Adaptation

    University of Warwick Coventry, United Kingdom

  17. Attention-Steered Vision-Language Models for Sign Language Translation

    Jul 31, 2026Meibo Hu, Guohao Sun, Annemarie D. Ross +2Sign Language TranslationVision-Language Models

    Rochester Institute of Technology · National Technical Institute for the Deaf / Rochester Institute of Technology · University of Virginia

  18. DualAnchor: Preserving Language Priors and Improving Lexical Fidelity in Gloss-Free Sign Language Translation

    Jul 30, 2026Hongbin Zhang, Junhao Liu, Xuefeng Bai +3Sign Language TranslationVision-Language Alignment

    Institute of Computing and Intelligence, Harbin Institute of Technology, Shenzhen, China · Peng Cheng Laboratory, Shenzhen, China

  19. Q-BridgeNet: A Quantization Network for Cross-Lingual Sign Language Translation

    Jul 13, 2026Liqian Feng, Lintao Wang, Xiaochen Liu +5Sign Language TranslationCross-Lingual Transfer

    The University of Sydney, Darlington NSW, Australia · Beijing University of Technology, Beijing, China · Edith Cowan University, Joondalup WA, Australia

  20. Toward Real-Time Sentence-Level Sign Language Translation

    Jul 10, 2026Thanh-Hoang Nguyen DoanSign Language TranslationStreaming Inference

    The University of Danang - University of Science and Technology · The University of Danang – University of Science and Technology

  21. VTaMo: Video-Text Alignment Model for Sign Language Translation

    Jul 10, 2026Junyi Hu, Zhewen He, Haomian Huang +2Sign Language TranslationCross-Modal Alignment

    New York University Abu Dhabi, UAE · ChatSign Technology

  22. ViPo-MLLM: Visual-Pose Multimodal LLM for Gloss-Free Sign Language Translation

    Jul 4, 2026Ahmed Abul Hasanaath, Bicheng Xu, Mir Rayat Imtiaz Hossain +2Sign Language TranslationMultimodal Large Language Models

    King Fahd University of Petroleum and Minerals, Dhahran, KSA · University of British Columbia, Vancouver, Canada

  23. BackTranslation2.0 -- A Linguistically Motivated Metric to Assess Sign Language Production

    Jun 27, 2026Oliver Cory, Maksym Ivashechkin, Karahan Sahin +6Sign Language TranslationAutomated Evaluation

    Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, UK

  24. SIGNET: Motion-Level Knowledge Transfer for Cross-Language Sign Language Translation

    Jun 26, 2026Sobhan Asasi, Ozge Mercanoglu Sincan, Richard BowdenSign Language TranslationCross-Lingual Transfer

    CVSSP, University of Surrey, United Kingdom

  25. SignNet-1M: Large-Scale Multilingual Sign Language Video Dataset with Downstream Benchmarks

    Jun 23, 2026Zhewen He, Junyi Hu, Haomian Huang +3Distribution Shift RobustnessSynthetic Data Augmentation

    New York University Abu Dhabi, UAE · ChatSign Technology · Tsinghua University, China

  26. Deep Learning-Based Sign Language Recognition from Videos and Cross-Lingual Translation to Indian Vernaculars

    Jun 21, 2026Ramesh Nandipalli, Chandranath AdakSign Language TranslationSign Language Recognition

    Department of Computer Science and Engineering, Indian Institute of Technology Patna, Bihar, India

  27. SignVLA: Real-Time Sign Language-Guided Robotic Manipulation via Attention LSTM and Vision-Language-Action Models

    Jun 18, 2026Ningwei Bai, Xinyu Tan, Harry Gardner +6Sign Language TranslationLanguage-Conditioned Robot Manipulation

    Department of Computer Science, University College London.

  28. Corpus Augmentation for Sign Language Translation via LLM-Guided Video Stitching

    Jun 10, 2026Zsolt Robotka, Ádám Rák, Jalal Al-Afandi +2Synthetic Data AugmentationSign Language Translation

    Peter Pazmany Catholic University, Faculty of Information Technology and Bionics, Budapest, Hungary · DeepSign Technologies Ltd., Budapest, Hungary

  29. SLU-2K: A Question-Based Benchmark for Semantic Evaluation of Sign Language Translation

    Jun 2, 2026Zeno Testa, Antonino Furnari, Lorenzo Baraldi +1Sign Language TranslationVideo QA

    University of Modena and Reggio Emilia Modena, Italy · University of Catania Catania, Italy · University of Granada, CITIC & DaSCI Institute, Spain

  30. Conditional Collapse in Sign Language Production: A Diagnostic and a Scaling Argument

    Jun 1, 2026Rui Hong, Jana KošeckáSign Language TranslationHuman Motion Generation

    George Mason University