cs.CVOct 31, 2026

Beyond Temporal Overlap: Consumer-Aware Evaluation of Sign-Language Segmentation across Three Corpora

Authors: Guohou Zhang, Kalin Stefanov

Organizations: Monash University, Australia · Japan Advanced Institute of Science and Technology, Japan

Abstract

Sign-language segmentation divides continuous signing into clips. In this study, the downstream consumer is a frozen recognizer that assigns one sign label to each predicted clip. Existing metrics instead compare predicted clips with manually annotated boundaries. They do not directly measure whether a clip preserves the visual evidence needed by the recognizer or whether the segmenter produces too many clips. We therefore ask whether overlap metrics and this consumer prefer the same segmenter. We evaluate 27 outputs from three segmentation architectures, three random seeds, and three corpora. Controlled boundary edits separate the effect of removing sign evidence from that of adding surrounding context. Overlap predicts recognition on average, but it does not always determine model choice. At the same intersection-over-union, different recognizers can prefer the shorter or the longer clip. The real outputs reveal two operating regimes. On DGS, overlap, recognition, and sequence error select the same model. On BSL, recognition-based selection recovers and recognizes more reference signs but increases word error rate (WER). Dicta-Sign shows the reverse trade-off: WER selection reduces in-vocabulary WER by 31.17 points but misses more recognizable signs. Segmenters should therefore be evaluated on four axes: temporal agreement, reference recovery, clip recognition, and sequence error, rather than on overlap alone.

Figures & tables

Explore similar work

CardsList
  1. BEACON: Distilled Begin/End-Aware Structured Decoding for Continuous Sign Language Segmentation

    Oct 29, 2026Guohou Zhang, Kalin StefanovSign Language RecognitionBoundary-Aware Segmentation

  2. SignAligner: Dual-Granularity Span Generation for Sign Language Alignment

    Nov 17, 2026Jongbhin Woo, Youngjoon Jang, Joon Son Chung

  3. Seeing Semantic Shift: Difference-Aware Sentence-Level Temporal Segmentation of Sign Language Videos

    Sep 25, 2026Bowen Guo, Shiwei Gan, Yafeng Yin +4Boundary-Aware SegmentationTemporal Action Segmentation