cs.CVNov 18, 2026

SIGNTOK: A Text-Aligned Continuous Pose Tokenizer for Sign Language Generation

Authors: Nabeela Khan, Gaurav Rai, Bowen Wu, Sihan Tan, Benjamin Yen, Takeshi Ashizawa, Kazuhiro Nakadai

Organizations: Institute of Science Tokyo, Japan · IIIT Delhi, India · RIKEN, Japan

Abstract

Text-conditioned sign language generation (SLG) depends not only on the generator but also on the space in which signing is generated. Existing approaches generate directly in pose space, in quantized code spaces, or in learned latent spaces trained for reconstruction. We find that these spaces are organized largely by signer identity rather than linguistic content: on CSL-Daily, a test clip's nearest neighbor in raw pose space is the same sentence signed by another person in only 2% of cases. To address this, we propose SIGNTOK, a continuous pose tokenizer whose latent space is shaped by three training-only heads: a kinematics head that predicts motion dynamics, a sentence-level head that aligns signing with the overall text, and a word-level head that aligns short motion windows with content words. Together, these objectives raise the same-sentence cross-signer nearest-neighbor rate to 82%. On three sign language benchmarks, the same generator trained in the SIGNTOK latent instead of pose space raises back-translation BLEU-4 in five of six benchmark-scorer settings at comparable joint accuracy, and word-level alignment gives higher back-translation scores than sentence-level alignment on every benchmark.

Figures & tables

Explore similar work

CardsList
  1. SIGNER: Temporally Grounded Sign Language Generation via Time-Resolved Conditioning

    Jun 9, 2025Taeryung Lee, Hyeongjin Nam, Gyeongsik Moon +1Conditional Generative ModelingSign Language Generation

  2. Semantic Residual Vector Quantization for Sign Language Generation

    Nov 4, 2026Chang Liu, Ke Han, Niccolò Biondi +4Sign Language GenerationHuman Motion Generation

  3. SignFLIP: A Unified Model for Sign Language Translation and Generation via Stage-wise Alignment at Scale

    Sep 28, 2026Zhaoyi An, Sihan Tan, Youngbae Hwang +2Sign Language TranslationRepresentation Alignment