Not What a Child Expressed: Auditing the Sign-to-Text Safety Interface in Child-Facing AI

Muhammad Rafiullah Memon; Viet Vo; Wanlun Ma; Yang Xiang

cs.CLOct 5, 2026

Not What a Child Expressed: Auditing the Sign-to-Text Safety Interface in Child-Facing AI

Authors: Muhammad Rafiullah Memon, Viet Vo, Wanlun Ma, Yang Xiang

Organizations: School of Science, Computing and Emerging Technologies Swinburne University of Technology, Hawthorn, VIC, Australia

Abstract

Automatic sign language translation (SLT) has entered consumer products, turning American Sign Language into English text for dictation, messaging, and queries put to a conversational assistant. Child-facing AI and platform trust-and-safety tooling decide on text, using filters on minor accounts and grooming classifiers that score chat messages. A signing child who uses SLT therefore reaches these safeguards through a translation. We found no publicly documented system in which the two have been jointly evaluated, and the leading deployed SLT model was neither trained nor formally evaluated on signers under 18. Errors that alter negation, participant roles, secrecy, urgency or help-seeking could change a safety decision without disturbing fluency. This paper proposes a Deaf-informed pre-deployment audit of that boundary, with a failure taxonomy, a sanitised scenario schema, four comparison conditions, and four outcome measures. Auslan is the planned first case study.

Figures & tables

Panel of Figure 1 — Figure 1: Proposed audit of the sign-to-text boundary. One sanitised scenario is rendered under four translation conditions and passed to the same mechanism, so variation in the decision is attributable to translation.

Explore similar work

Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation
Sep 7, 2026Keren Artiaga, Sabyasachi Kamila, Haithem Afli +2Gloss-Free Sign Language Translation Sign Language Translation
ADAPT Centre, Munster Technological University, Cork, Ireland · Manipal Institute of Technology Bengaluru, MAHE, Manipal, India · Nimbus Research Centre, Munster Technological University, Cork, Ireland +1
BackTranslation2.0 -- A Linguistically Motivated Metric to Assess Sign Language Production
Jun 27, 2026Oliver Cory, Maksym Ivashechkin, Karahan Sahin +6Sign Language Translation Multilingual Benchmark
Centre for Vision, Speech and Signal Processing, University of Surrey, Guildford, UK
CAREBench: A Child-Safety Risk Benchmark for Language Models
Jun 29, 2026Kaavya Krishna-Kumar, Elaine Lau, Vaughn Robinson +6Artificial Intelligence Safety Frontier Models
Handshake AI · University of California, Los Angeles · McGill University

Sep 7, 2026, cs.CLJ/K move · Enter open · S save

Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation

Keren Artiaga, Sabyasachi Kamila, Haithem Afli, Conor Lynch+1

ADAPT Centre, Munster Technological University, Cork, Ireland · Manipal Institute of Technology Bengaluru, MAHE, Manipal, India · Nimbus Research Centre, Munster Technological University, Cork, Ireland+1

Gloss-Free Sign Language Translation Sign Language Translation Online Signature Verification Model Evaluation Cross Validation Impact Analysis

Open paper AskPDF arXiv

Sign Language Translation has advanced with deep learning, yet evaluations remain largely signer-dependent, with overlapping signers across train/dev/test. This raises concerns about whether models truly generalise or instead rely on signer-specific regularities. We conduct signer-fold cross-validation on GFSLT-VLP, GASLT, and SignCL, three leading, publicly available, gloss-free SLT models, on CSL-Daily and PHOENIX14T. Under signer-independent evaluation, performance drops sharply: on PHOENIX14T, GFSLT-VLP falls from BLEU-4 21.44 to 3.59 and ROUGE-L 42.49 to 11.89; GASLT from 15.74 to 8.26; and SignCL from 22.74 to 3.66. We also observe that in CSL-Daily many target sentences are performed by multiple signers, so common splits can place identical sentences in both training and test, inflating absolute scores by rewarding recall of recurring sentences rather than genuine generalisation. These findings indicate that signer-dependent evaluation can substantially overestimate SLT capability. We recommend: (1) adopting signer-independent protocols to ensure generalisation to unseen signers; (2) restructuring datasets to include explicit signer-independent, sentence-disjoint splits for consistent benchmarking; and (3) reporting both signer-dependent and signer-independent results together with train-test sentence overlap to improve transparency and comparability.

Abstract

Explore similar work

Rethinking Sign Language Translation: The Impact of Signer Dependence on Model Evaluation

BackTranslation2.0 -- A Linguistically Motivated Metric to Assess Sign Language Production

CAREBench: A Child-Safety Risk Benchmark for Language Models