Sign-language segmentation divides continuous signing into clips. In this study, the downstream consumer is a frozen recognizer that assigns one sign label to each predicted clip. Existing metrics instead compare predicted clips with manually annotated boundaries. They do not directly measure whether a clip preserves the visual evidence needed by the recognizer or whether the segmenter produces too many clips. We therefore ask whether overlap metrics and this consumer prefer the same segmenter. We evaluate 27 outputs from three segmentation architectures, three random seeds, and three corpora. Controlled boundary edits separate the effect of removing sign evidence from that of adding surrounding context. Overlap predicts recognition on average, but it does not always determine model choice. At the same intersection-over-union, different recognizers can prefer the shorter or the longer clip. The real outputs reveal two operating regimes. On DGS, overlap, recognition, and sequence error select the same model. On BSL, recognition-based selection recovers and recognizes more reference signs but increases word error rate (WER). Dicta-Sign shows the reverse trade-off: WER selection reduces in-vocabulary WER by 31.17 points but misses more recognizable signs. Segmenters should therefore be evaluated on four axes: temporal agreement, reference recovery, clip recognition, and sequence error, rather than on overlap alone.
Figures & tables
Fig. 1: Consumer-aware evaluation of sign segmentation. A Public DGS Corpus sequence [ 8 ] connects real model outputs and controlled boundary edits to localization, recognition, and sequence-level evaluation. The protocol fixes features, vocabularies, matching, and consumers before comparing segmenters.
Consumer objective
Select with
Report alongside
Recover references
AUC-RU
coverage, C
Transcribe sequence
WERall
coverage, S/D/I
TABLE I: The selection metric follows the intended use. The remaining measures show what the selected model gains and loses. S/D/I denotes substitution/deletion/insertion.
Corpus
Train/val/test
Eligible test tokens
Classes
Segmenter input
Frozen consumer
Split / clusters
DGS
583/12/17 videos
3,080/5,713 (53.9%)
500
MediaPipe 225-D
Logos RGB
signer-disj. / 14
Dicta-Sign
58/16/20 recordings
2,018/4,654 (43.4%)
183
MediaPipe 225-D
Logos RGB
session-disj. / 6
BSL
5,413/763/703 clips
4,167/4,751 (87.7%)
968
HaMeR 288-D
BOBSL RGB
signer-disj. / 21
TABLE II: Three held-out corpus protocols. Eligible tokens are test references inside the train-only consumer vocabulary. Absolute recognition values are compared only within corpus. The final column gives split control and the number of signer clusters used for test inference.
Fig. 2: Controlled mechanism and replication. (a) Dilation-minus-erosion recognition difference at matched IoU; whiskers are paired video-bootstrap 95% intervals. (b) Mean per-seed mF1S ranking-reversal rates before and after a one-point practical recognition margin.
Corpus
Family
mF1S
Coverage
Accuracy
AUC-RU
WERall
DGS
TCN
36.5 / 38.3 / 37.9
95.9 / 96.0 / 96.3
37.0 / 38.2 / 37.7
0.259 / 0.273 / 0.268
272.18 / 272.73 / 271.36
BiLSTM
43.8 / 44.2 / 42.7
95.8 / 96.0 / 96.4
39.5 / 39.8 / 39.8
0.306 / 0.305 / 0.297
248.47 / 241.23 / 246.53
Transformer
39.5 / 39.9 / 40.2
95.7 / 96.3 / 95.6
39.0 / 39.7 / 40.0
0.285 / 0.297 / 0.294
274.58 / 274.45 / 269.45
Dicta-Sign
TCN
32.7 / 28.6 / 30.3
90.3 / 84.1 / 89.2
42.1 / 30.6 / 39.3
0.259 / 0.152 / 0.227
302.28 / 253.22 / 287.96
BiLSTM
35.6 / 32.4 / 30.8
87.9 / 92.9 / 89.4
45.1 / 42.8 / 38.4
0.298 / 0.251 / 0.211
305.85 / 335.93 / 290.63
Transformer
21.3 / 22.3 / 20.9
82.3 / 87.8 / 80.2
30.1 / 35.9 / 28.4
0.135 / 0.175 / 0.119
273.49 / 348.61 / 240.54
TABLE III: Held-out results for all 27 real-output conditions. Each cell lists seeds 7/42/2027 ; bold marks within-family validation selection for mF1S , AUC-RU, and the primary WERall objective. Coverage and accuracy preserve every seed for inspection.
Corpus
Validation objective
Selected
Δ Cov.
Δ Acc.
Δ AUC-RU
ΔWERall
DGS
AUC-RU
BiLSTM/42
+0.00
+0.00
+0.0000
+0.00
WERall
BiLSTM/42
+0.00
+0.00
+0.0000
+0.00
Dicta-Sign
AUC-RU
BiLSTM/7
+0.00
+0.00
+0.0000
+0.00
WERall
TCN/42
-3.77
-14.52
-0.1453
-52.63
BSL
AUC-RU
TCN/42
+5.62
+1.97
+0.0112
+9.31
WERall
BiLSTM/2027
+0.00
+0.00
+0.0000
+0.00
TABLE IV: Held-out change relative to the validation- mF1S selection. Every model is selected on validation; positive coverage, accuracy, and AUC-RU are benefits, while negative WERall is a benefit.
Consumer
Gold acc.
ρ
Rev.
Val. AUC-RU sel.
RGB Logos
46.88
.967
0/19
BiLSTM/42
Statistical MLP
35.75
.983
0/15
BiLSTM/7
Sequence TCN
39.84
.933
1/20
BiLSTM/42
TABLE V: The same nine DGS outputs evaluated by three frozen consumers. ρ compares mF1S and AUC-RU; Rev. applies a one-point accuracy margin.
Fig. 3: Held-out localization and reference utility for all 27 real-output conditions. Color denotes architecture and marker denotes seed. Black rings identify the validation- mF1S seed within each family. DGS and Dicta-Sign form aligned frontiers; BSL separates overlap from reference utility.
Consumer
Gold Top-1
Rev.
Δ Acc.
Δ AUC-RU
VideoSwin
40.82
16/17
+1.97
+.0112
BSL-1K I3D–TCN
36.72
20/20
+2.14
+.0114
TABLE VI: BSL conflict replication across independently trained RGB consumers. Rev. counts mF1S –accuracy ranking reversals among candidate pairs separated by at least one accuracy point. Δ compares the validation-frozen TCN/42 and BiLSTM/2027 outputs.
Fig. 4: Held-out changes from switching the validation objective on Dicta-Sign and BSL. Whiskers are 95% signer-clustered intervals; green and red indicate benefit and cost after orienting each metric.
Corpus
OOV pred.
WERall
WERIV
Selector
DGS
31–33%
241–275
150–175
BiLSTM/42
Dicta-Sign
39–42%
241–349
134–201
TCN/42
BSL
.4–.7%
89–118
88–117
BiLSTM/2027
TABLE VII: Vocabulary and sequence burden on held-out test. OOV pred. is the fraction assigned to annotated OOV signs. The primary WERall selector and the WERIV sensitivity analysis select the same candidate in every corpus.
Corpus
No rejection
90% retention
Selected
DGS
150.26/96.0
130.68/89.2
BiLSTM/42
Dicta-Sign
137.96/84.1
124.73/77.4
TCN/42
BSL
88.12/74.7
84.74/70.1
BiLSTM/2027
TABLE VIII: Validation-frozen rejection sensitivity. The 90% operating point sets one corpus-level confidence threshold from validation. Values are held-out WERIV /coverage (%).