End-to-end speech large language models (LLMs) often answer the same question less accurately from speech than from text, even when the spoken content matches the text prompt. Existing analyses study this modality gap through the geometry of representations and measure how closely speech representations align with text. Geometric alignment gives the gap a measurable form, but it does not show whether the model uses aligned representations in the same way. This work connects representation geometry with answer formation and asks whether geometric alignment implies functional equivalence. We compare cross-layer Centered Kernel Alignment (CKA) with layer-wise answer margins for four open-weight speech LLMs on SpeechMMLU and VoiceBench BBH. In Qwen2.5-Omni and MiniCPM-o, speech and text representations align in middle layers while their late-layer answer margins still differ, so mid-layer alignment does not imply functional equivalence. Qwen2-Audio instead shows weak alignment, which indicates a different failure mode. An attention intervention at fixed input and token count identifies a causal effect of late attention allocation on answer-code separation, a quantity distinct from answer correctness. Compressing the speech signal or its token sequence repairs a subset of the errors that occur only with speech.
Figures & tables
Figure 1: Geometric alignment without functional equivalence. The same question enters a speech LLM as text and as speech. Speech–text CKA is high in middle layers, yet the answer margins of the two inputs separate in late layers. Curves use Qwen2.5-Omni on BBH, with CKA from Section A.2 and margins averaged over the 233 items answered correctly only from text.
Figure 2: The speech–text modality gap. In the benchmark overview, speech-to-text (S2T) accuracy is below matched text-to-text (T2T) accuracy for all four speech LLMs on both SpeechMMLU and VoiceBench BBH.
Figure 3: Speech and text align in a broad middle-layer band. Each heatmap shows cross-layer CKA between S2T layers on the y-axis and T2T layers on the x-axis after dynamic time warping (DTW) token alignment, with brighter colors denoting higher CKA. The red path marks the best-matching text layer for each speech layer, and the dashed diagonal marks equal layer indices.
Setting
SpMMLU
Δ
BBH
Δ
Gold T2T
73.49
+3.25
72.7
+12.8
Direct S2T
70.24
base
59.9
base
ASR-to-LLM
59.23
−11.0
67.4
+7.5
Table 1: Behavioral gap for Qwen2.5-Omni. Subtable (a) gives benchmark-overview accuracy with deltas in points relative to direct S2T, where SpMMLU denotes SpeechMMLU. Subtable (b) gives the BBH categories Hyperbaton, Navigate, Sports, and Web-of-lies in the 1,024-token diagnostic, where the gap is T2T minus S2T.
Figure 4: Answer margins separate in late layers. Curves show the mean correct-minus-competing candidate probability for Qwen2.5-Omni on 1,000 BBH items, grouped by free-generation correctness, and shading marks the late window of layers 23–28. In this window, the text margin is larger for text-only-correct items and the speech margin is larger for speech-only-correct items.
Model
Benchmark
ΔA
ΔW.95
Qwen2.5-Omni
SpeechMMLU
.0952 [.0867, .1034]
7.56 [6.88, 8.22]
Qwen2.5-Omni
BBH
.0981 [.0925, .1038]
5.64 [5.07, 6.21]
MiniCPM-o
SpeechMMLU
.1343 [.1250, .1438]
5.17 [4.52, 5.80]
MiniCPM-o
BBH
.0916 [.0840, .0993]
5.47 [4.98, 5.95]
Table 2: Controls for the two central findings. Subtable (a) gives the middle-layer CKA lift A and band width W.95 without pooling, each minus its position-permutation baseline, with 95% bootstrap intervals ( Section A.2 ). Subtable (b) gives the mean late T2T-minus-S2T margin of Qwen2.5-Omni on BBH for the two discordant groups in three margin coordinates, with the crossover contrast and its 95% interval.
Figure 5: Speech attention is more dispersed than text attention. Each column shows a selected decision-position attention distribution for Qwen2.5-Omni on SpeechMMLU, labeled by subject, item, layer (L), and head (H). The speech row uses a ten-times smaller vertical scale, and dashed lines mark the sorted-token fraction that covers 90% of the mass.
Figure 6: Sharpening late speech attention increases answer-code separation. Points show the mean change in ∣zX−zY∣ for Qwen2.5-Omni on 1,000 BBH items with 95% bootstrap intervals, under two answer-to-code mappings and two rule orders. The input and token count are fixed, and the gap measures separation irrespective of which code is correct.
Model
SpeechMMLU gap
BBH gap
Mid-band
Mode
Qwen2.5-Omni
+3.25
+12.8
clear
Decision Instability
MiniCPM-o
+7.02
+7.7
clear
Decision Instability
LLaMA-Omni
≈ +58.8
≈ +12.1
broad
Intermediate
Qwen2-Audio
+43.07
+2.2
absent
Representation Failure
Table 3: Two failure modes across models. Gaps are T2T-minus-S2T accuracy points in the benchmark overview, approximate for LLaMA-Omni. Mid-band describes the middle-layer CKA band.
Overview protocol
Shared-instruction protocol
Intervention
Target
Gap repaired
Δ Acc.
Gap repaired
Δ Acc.
Tempo 1.2×
audio duration
61/178 (34.3%)
+2.2
113/233 (48.5%)
+2.2
Silence trim
pauses
68/178 (38.2%)
+2.0
104/233 (44.6%)
+1.3
Token merging
speech tokens
26/178 (14.6%)
+0.5
99/233 (42.5%)
−1.5
Table 4: Compression repairs a subset of speech-only errors on BBH. Gap repaired counts corrected answers among the text-correct, speech-wrong items of Qwen2.5-Omni under each protocol, and Δ Acc. is the aggregate accuracy change in points.
Appendix figures & tables34 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Cross-layer best-match sensitivity to DTW settings. Qwen2.5-Omni, 100 items per benchmark. The top row reports the R2 of a linear fit to matched text-layer indices across speech layers, and the bottom row the fraction of adjacent speech-layer pairs mapped to the same text layer. Open rings mark the width-64 band and coincide with the unbanded values. Pooling-free, null-adjusted results appear in Section A.2 .
Model
Benchmark
ΔA
ΔW.95
Cross-item (ΔA,ΔW.95)
Qwen2.5-Omni
SpeechMMLU
.0952 [.0867, .1034]
7.56 [6.88, 8.22]
(.0299, − 1.37)
Qwen2.5-Omni
BBH
.0981 [.0925, .1038]
5.64 [5.07, 6.21]
(.0081, −0.72)
MiniCPM-o
SpeechMMLU
.1343 [.1250, .1438]
5.17 [4.52, 5.80]
(.0521, −0.46)
MiniCPM-o
BBH
.0916 [.0840, .0993]
5.47 [4.98, 5.95]
(−.0031, −3.30)
Appendix
Table 5: Native-length CKA controls. The first two endpoints subtract the within-item mean position-permutation endpoint and report item means with 95% bootstrap intervals. The last column gives the alignment-free matched-minus-wrong-item differences in A and W.95 , respectively, without an inferential interval.
Figure 8: Best-match paths and phase windows from the CKA heatmaps. Black lines connect each speech layer to its best-matching text layer, point colors give the peak CKA in that speech-layer row, and the dashed diagonal marks equal layer indices. Background shading marks the three phase windows, whose speech-layer ranges appear above each panel and are listed in Section A.2 . The late plateau describes repeated best-matching text layers, with functional behavior tested separately by answer readouts.
Figure 9: Signed-logit counterpart of the probability replication. The items, readout and groups match Figure 4 . Curves show mean correct-minus-competing logits, with the fixed late window shaded.
Model
Benchmark
S2T acc.
T2T acc.
Unparsed S2T / T2T
Group sizes
Qwen2.5-Omni
SpeechMMLU
66.7
72.3
0 / 4
183 / 34 / 17 / 66
Qwen2.5-Omni
BBH
62.0
69.7
–
147 / 62 / 39 / 52
MiniCPM-o
SpeechMMLU
50.0
57.3
64 / 73
101 / 71 / 49 / 79
MiniCPM-o
BBH
56.7
50.0
51 / 98
96 / 54 / 74 / 76
Qwen2-Audio
SpeechMMLU
33.3
55.3
10 / 8
70 / 96 / 30 / 104
Qwen2-Audio
BBH
49.0
42.3
26 / 81
78 / 49 / 69 / 104
Appendix
Table 6: Free-generation correctness groups of the 300 items per benchmark. Accuracy counts unparsed answers as wrong. Group sizes are both correct, text only correct, speech only correct, and both wrong. Qwen2.5-Omni on BBH uses the groups of Section D.3 , whose unparsed counts are reported there.
Figure 10: Qwen2.5-Omni logit margin by correctness group. Rows are SpeechMMLU and BBH, and columns are the free-generation groups with their sizes. Curves show the mean candidate-readout margin of the group. The BBH groups are those of the 1,000-item analysis in Figure 4 restricted to 300 items. In the text-only-correct group the late-layer margin (mean of the last six coordinates) is +0.63 for text and −1.40 for speech on SpeechMMLU, and +1.58 and −0.46 on BBH.
Figure 11: Layer-wise projected-logit entropy of Qwen2.5-Omni. Curves show the mean over 300 items per benchmark. Speech and text have similar entropy trajectories, with mean absolute differences of 0.67 nats on SpeechMMLU and 0.76 nats on BBH.
Figure 12: Layer-wise subject probe of Qwen2.5-Omni. Logistic-regression (linear) and one-hidden-layer MLP probes on the mean hidden state of the evidence tokens predict the SpeechMMLU subject of the 300 items (34 subjects, chance 2.9%), with 5-fold cross-validated accuracy. Linear speech accuracy rises from 0.13 at the embedding output to 0.71 at layer 10 and text accuracy from 0.48 to 0.70, and both stay between 0.6 and 0.7 afterwards; fold standard deviations reach 0.10. These curves describe probe performance rather than generated-answer correctness.
Figure 13: Cosine similarity between speech and text layer-wise updates in Qwen2.5-Omni. An update is the difference between consecutive hidden states after the fixed base-layer-0 token alignment, averaged over 100 items per benchmark. For every speech layer the largest cosine lies at the text layer with the same index. The mean diagonal cosine is 0.33 on SpeechMMLU and 0.30 on BBH, against 0.03 off the diagonal.
Layers
Items
Median cov. ratio
Median Δ Cov .90
Q90 Δ Cov .90
Pairs Δ Cov > 0
Q90 Δ Ent.
7–21
100
2.20×
+16.0
+83.5
78.4%
+0.037
23–28
100
3.74×
+48.0
+102.5
86.1%
+0.073
Appendix
Table 7: All-head attention coverage audit for Qwen2.5-Omni on SpeechMMLU. We summarize all head–layer pairs within each item, then report the across-item medians of the within-item medians and 90th percentiles (Q90). The positive-pair fraction is averaged over items. Δ denotes S2T minus T2T, and Ent. is attention entropy divided by the log of valid prompt length. Attention is normalized over all valid prompt keys at the final prompt query. These windows differ from the CKA phase windows. Speech usually needs more tokens to cover 90% of the attention mass, while normalized entropy differences vary across heads.
Metric
T2T
S2T
Δ
Entropy norm
0.361
0.657
+0.296
pmax
0.638
0.105
−0.533
Top-10 mass
0.933
0.483
−0.450
Cov .90 count
8
101
+93
Appendix
Table 8: Selected-head attention summary for Qwen2.5-Omni on SpeechMMLU. Δ is S2T minus T2T, computed from the displayed values. Entropy norm is normalized attention entropy, pmax is the largest attention coefficient, top-10 mass sums the ten largest coefficients, and Cov .90 counts the tokens needed to cover 90% of attention.
Condition
Order
Identity mapping
Swapped mapping
S2T late sharpening
01
+.0291 [.0237, .0346]
+.0076 [.0026, .0127]
10
+.0332 [.0274, .0389]
+.0129 [.0076, .0184]
S2T middle sharpening
01
−.0039 [−.0091, .0012]
+.0056 [.0019, .0093]
10
+.0006 [−.0048, .0059]
−.0029 [−.0069, .0013]
S2T late permutation
01
−.0314 [−.0410, −.0219]
−.0004 [−.0084, .0077]
10
−.0344 [−.0436, −.0253]
+.0054 [−.0032, .0138]
Appendix
Table 9: Change in absolute answer-code separation. Each entry is the mean treatment-minus-baseline change with its 95% interval on the same 1,000 items. Positive values indicate greater separation regardless of which semantic answer is favored. Order 01 states the class-0 rule first, and order 10 states the class-1 rule first.
Setting
T2T
S2T
Use in the paper
Benchmark overview
72.7
59.9
Aggregate model comparison and overview repair changes
1,024-token diagnostic
71.9
59.7
Category breakdown, duration groups, 178-item gap subset
Calibration
—
62.3
Input- and last-layer moment matching
Word repetition
62.4
—
r = 1 reference for the text stress test
Shared instructions, 256 tokens
74.1
61.5
233-item gap subset, paired readouts, complete behavioral transitions
Appendix
Table 10: Behavioral settings and their baseline accuracies. All values are percentages for Qwen2.5-Omni on BBH. A dash denotes a route not used for that comparison. Baseline differences identify separate settings, so their accuracy changes and correctness groups are interpreted within each row.
Condition
Acc. (%)
Unparsed
Repairs
Harms
Retained C/W
Gap repairs
Δ Acc. [95% CI]
Tempo 1.2×
63.7
20
151
129
486/234
113/233
+2.2 [−1.1, 5.4]
Silence trimming
62.8
22
139
126
489/246
104/233
+1.3 [−1.9, 4.5]
Token merging
60.0
21
139
154
461/246
99/233
−1.5 [−4.8, 1.8]
ASR cascade
71.5
14
219
119
496/166
181/233
+10.0 [6.7, 13.4]
Appendix
Table 11: Complete paired transitions relative to direct speech. Repairs are wrong-to-correct transitions, harms are correct-to-wrong transitions, and retained C/W denotes unchanged correct/wrong outcomes across all 1,000 items. Gap repairs use the 233 items correct only under gold text as their denominator. Accuracy changes are percentage points with category-stratified paired-bootstrap 95% intervals.
Dataset / slice
N
WER
Med. WER
Marker
Sub.
Ins.
Del.
SpeechMMLU
200
3.28%
0.00%
98.4%
1.89%
0.66%
0.72%
BBH overall
200
4.65%
3.45%
99.5%
4.09%
0.26%
0.29%
Hyperbaton
45
2.55%
0.00%
100.0%
1.82%
0.60%
0.13%
Navigate
51
3.09%
0.00%
100.0%
2.76%
0.06%
0.28%
Sports
54
3.22%
2.44%
100.0%
2.48%
0.31%
0.43%
Web-of-lies
50
9.66%
9.84%
98.0%
9.24%
0.11%
0.31%
Appendix
Table 12: ASR transcript audit. We transcribe 200 randomly sampled items per benchmark with Whisper-large-v2. Word error rate (WER), substitution (Sub.), insertion (Ins.), and deletion (Del.) rates are averaged over items. Med. WER is the median item WER. Marker recall measures recognition of “the answer is” on BBH and that phrase plus “question” and “option A/B/C/D” on SpeechMMLU.
Group
Count
Duration (s)
Seconds per word
Both correct
541
22.41 [22.04, 22.77]
0.44 [0.43, 0.44]
Only T2T correct
178
23.58 [22.97, 24.18]
0.45 [0.44, 0.46]
Only S2T correct
56
21.23 [20.32, 22.17]
0.45 [0.43, 0.46]
Both wrong
225
21.54 [20.97, 22.11]
0.42 [0.42, 0.43]
Appendix
Table 13: BBH temporal-load metadata by correctness group. Durations use the 1,000-item, 1,024-token diagnostic setting in Table 10 . Confidence intervals are 95% bootstrap intervals over items, pooled across categories.
r
SpeechMMLU
BBH
1
73.9
62.4
2
67.4
56.2
4
57.3
55.4
8
51.2
53.5
Appendix
Table 14: Text-redundancy stress test for Qwen2.5-Omni. Each word of the text prompt is repeated r times, and entries are T2T accuracy in percent.
Task
N
Qwen2.5-Omni
MiniCPM-o
S2T
Text
Δ
S2T
Text
Δ
Continuation writing
110
100.0
22.7
+77.3
99.1
22.7
+76.4
Homophone reasoning
101
97.0
26.7
+70.3
95.0
35.6
+59.4
Intent detection
115
88.7
4.3
+84.3
87.0
25.2
+61.7
Polysemy reasoning
98
99.0
43.9
+55.1
96.9
38.8
+58.2
Deixis resolution
103
94.2
24.3
+69.9
83.5
51.5
+32.0
Appendix
Table 15: MMSU results on selected speech-native tasks. Acoustic cues can be answer-relevant on MMSU. Scores use deterministic parsing, counting unparsed answers as wrong. These selected tasks have large speech advantages, with Δ denoting S2T minus text accuracy in percentage points.
Figure 14: MiniCPM-o layer-wise projected-logit entropy. Curves show the mean over 300 items per benchmark. Speech and text both show a late-layer decrease, with larger differences between the routes than in Qwen2.5-Omni.
Figure 15: MiniCPM-o layer-wise logit margin by correctness group. Rows are SpeechMMLU and BBH. In the text-only-correct group the text margin exceeds the speech margin in late layers, and speech margins are smaller than text margins when both routes are correct.
Figure 16: MiniCPM-o pre-LN and post-LN norms. Left is SpeechMMLU and right is BBH. Dashed curves are the block input before the input layer norm and solid curves after it. Post-LN norms are comparatively stable across modalities despite late pre-LN divergence.
Figure 17: LLaMA-Omni cross-layer CKA heatmaps. Left is SpeechMMLU with DTW base layer 10 and right is BBH with base layer 20. Red lines mark the best-matching text layer of each speech layer and dashed lines the diagonal. Similarity is high across most layer pairs and lowest at the first two text layers.
Figure 18: LLaMA-Omni layer-wise projected-logit entropy. Curves show the mean over 300 items per benchmark. Entropy curves are broadly comparable across modalities.
Figure 19: LLaMA-Omni layer-wise logit margin by correctness group. Rows are SpeechMMLU and BBH. The text-only-correct group exhibits negative speech margins and positive text margins in late layers.
Figure 20: LLaMA-Omni pre-LN and post-LN norms. Left is SpeechMMLU and right is BBH. Post-LN norms are comparatively stable across modalities while pre-LN norms grow with depth.
Figure 21: Qwen2-Audio cross-layer CKA heatmaps. Left is SpeechMMLU with DTW base layer 10 and right is BBH with base layer 20. Red lines mark the best-matching text layer of each speech layer and dashed lines the diagonal. Unlike Qwen2.5-Omni and MiniCPM-o, Qwen2-Audio lacks a clear high-similarity middle-layer band.
Figure 22: Qwen2-Audio candidate-only readout. Per-layer probability of the correct answer option among candidates for S2T and T2T at the decision position after an assistant answer prefix, averaged over 300 items per benchmark. Thin lines mark chance, 0.25 on SpeechMMLU and 0.5 on BBH. Speech does not raise the correct option, especially on SpeechMMLU.
Figure 23: Qwen2-Audio layer-wise projected-logit entropy. Curves show the mean over 300 items per benchmark. Speech entropy is not uniformly higher than text entropy across layers.
Figure 24: Qwen2-Audio layer-wise logit margin by correctness group. Rows are SpeechMMLU and BBH. Margins favor the correct option less strongly under speech than under text on SpeechMMLU.
Figure 25: Qwen2-Audio pre-LN and post-LN norms. Left is SpeechMMLU and right is BBH. Post-LN norms remain comparatively stable across modalities.
Dataset
Item
Question summary
SpeechMMLU
anatomy_1
Which craniofacial condition is associated with a “dished face” profile?
SpeechMMLU
anatomy_11
Which signs follow damage to the third cranial nerve after an orbital injury?
BBH
bbh_hyperbaton_105
Choose the correct ordering of adjectives describing a repulsive, normal-size, green, old-fashioned cardboard surfboard.
BBH
bbh_hyperbaton_111
Choose between “little black driving ship” and “driving little black ship.”
BBH
bbh_web_of_lies_74
Jamey tells the truth. Raymond says Jamey lies. Audrie says Raymond lies. Michaela says Audrie lies. Rashida says Michaela lies. Does Rashida tell the truth?
Appendix
Table 16: Qualitative cases (text correct, speech wrong). We summarize selected questions where T2T answered correctly but S2T failed in the earlier comparison that selected them. The accompanying figure plots the per-layer probability of the correct option, normalized over answer candidates.
Figure 26: Candidate-probability traces for four instances selected as text-correct and speech-wrong. Curves show, for each fixed item, the probability of the correct option after normalization over answer candidates in Qwen2.5-Omni. Only hyperbaton_105 has a higher text probability at the last layer.
Parameter
Value or Method
Datasets and Audio
SpeechMMLU
Dataset-provided paired question_text , answer options, question_audio , and speaker identifier from MiMo-Audio-Eval
VoiceBench BBH
Dataset-provided paired prompt , reference answer, and audio clip for hyperbaton , navigate , sports_understanding , and web_of_lies
T2T Prompt
The paired text field corresponding to the audio item. No ASR transcript is used for gold-text T2T
Hidden State Extraction
Extraction Window
Last user segment, i.e., question turn only
Appendix
Table 17: Summary of implementation hyperparameters.
Asset and source
Stated license or condition
Qwen2.5-Omni-7B
Apache-2.0
MiniCPM-o 2.6
Apache-2.0 for code and weights
Qwen2-Audio-7B-Instruct
Apache-2.0
LLaMA-Omni
Code: Apache-2.0. Model: academic, noncommercial use with citation
Whisper-large-v2
MIT in the OpenAI release, with differing checkpoint-card metadata
MiMo-Audio-Eval toolkit
Apache-2.0
Appendix
Table 18: Provider-stated licenses of existing assets. Links point to the official model cards, dataset cards, or repositories. These are source declarations rather than a complete audit of every underlying asset.
Speech large language models (SLMs) are typically built from text large language model (TLM) checkpoints, yet they still suffer from a substantial modality gap. Prior work has mainly attempted to reduce this gap from the output side by making speech generation more text-like, but the gap remains. We argue that the key remaining bottleneck lies on the input side. We propose TextPro-SLM, an SLM that makes spoken input more closely resemble that of a prosody-aware text LLM. TextPro-SLM combines WhisperPro, a unified speech encoder that produces synchronized text tokens and prosody embeddings, with an LLM backbone trained to preserve the semantic capabilities of the original TLM while learning paralinguistic understanding. Experiments show that TextPro-SLM achieves the lowest modality gap among leading SLMs at both 3B and 7B scales, while also delivering strong overall performance on paralinguistic understanding tasks. These gains are achieved with only roughly 1,000 hours of LLM training audio, suggesting that reducing the modality gap from the input side is both effective and data-efficient.
Wenqian Cui, Xiao-Hui Li, Daxin Tan +2
The Chinese University of Hong Kong · Huawei Technologies
Spoken dialogue models typically start from text LLM backbones, yet reasoning often degrades when conditioning on speech instead of text. We attribute part of this modality gap to a temporal-granularity mismatch: speech tokens are temporally redundant and far longer than text under matched semantics, diluting per-token semantic density and weakening text-native reasoning dynamics. We study speech token design as a representation selection problem and sweep frame rates under a frozen LLM backbone with a fixed information rate. To make low frame rates feasible, we introduce factorized FSQ and a lightweight non-autoregressive audio LM head, scaling capacity to nearly 300,bits/frame without sacrificing efficient prediction. With the bottleneck removed, we sweep frame rates (50→2.08,Hz) and alignment depth, and observe a consistent best regime for speech QA at 4.17,Hz with intermediate-layer representation alignment.
Zhen Ye, Xu Tan, Yiming Li +10
Hong Kong University of Science and Technology, Hong Kong SAR · Tencent, China · Independent Researcher +4
Pure speech language models often lag behind text and speech-text language models in generating coherent content, but this gap is difficult to quantify because speech and text systems are typically evaluated with different metrics and trained on different data. We study the speech-text modality gap in a family of spoken language models, based on flow matching for continuous acoustic feature generation. We construct a unified generation-based evaluation suite that compares speech-only, text-only, and speech-text language models trained on matched data distributions and evaluated in matched generation settings. We evaluate generated continuations along multiple dimensions: semantic coherence, measured by transcribing generated speech and scoring it with a reference language model; local phonetic structure, measured by phone n-gram distributional statistics; speaker consistency and acoustic quality; and emotion-based distributional metrics. Across datasets, we find that joint speech-text modeling substantially improves semantic coherence. However, the improvement is not uniform across metrics: phone-level metrics change only modestly, speaker similarity and predicted quality are lower for speech-text continuations, while emotion-based distributional metrics improve. Compared with larger-scale speech-only models, our speech-text model closes much of the scaling gap in transcript-based semantic coherence, suggesting that text provides an efficient semantic training signal for spoken language modeling.