SpeechLLMs typically combine ASR-trained encoders with text-based LLM backbones, leading them to inherit written-style output patterns unsuitable for text-to-speech synthesis. This mismatch is particularly pronounced in Japanese, where spoken and written registers differ substantially in politeness markers, sentence-final particles, and syntactic complexity. We propose a preference-based alignment approach to adapt Japanese SpeechLLMs for speech-worthy outputs: text that is concise, conversational, and readily synthesized as natural speech. To rigorously evaluate this task, we introduce SpokenElyza, a benchmark for Japanese speech-worthiness derived from ELYZA-tasks-100 with auditory verification by native experts. Experiments show that our approach achieves substantial improvement on SpokenElyza while largely preserving performance on the original written-style evaluation. We will release SpokenElyza to support future research on Japanese spoken dialog systems.
Figures & tables
Figure 1: Overview of our speech-worthy alignment. SpeechLLM responses are aligned from written style (verbose, symbol-heavy) to spoken style (concise, conversational) for auditory comprehension and TTS synthesis. English translations at right; non-speech-worthy elements are highlighted in red.
Score
Description
1
Fatal Failure. The response contains fundamental factual errors, hallucinations, or completely fails to follow the instruction.
2
Low Quality. The response contains partial errors, incoherent Japanese, or excessive safety refusals (e.g., declining to answer harmless queries).
3
Written Style. The content is factually accurate, but the format is unsuitable for speech, containing markdown, bullet points, URLs, or visual references (e.g., ”as shown below”).
4
Spoken Style. The response is accurate and composed in natural spoken Japanese. It is free of non-verbalizable artifacts and structurally ready for synthesis.
5
Auditory Ideal. In addition to meeting the Score 4 criteria, the response is stellar in its listenability. It uses short sentences to reduce cognitive load and incorporates natural conversational markers to enhance the tone.
Table 1: Scoring rubric used for the LLM-as-Judge evaluation in SpokenElyza. The response to each instruction is scored from 1 to 5; we report averaged score among the examples.
Elyza
SpokenElyza
PT
3.97
2.91
PT + Prpt
-
2.94
DPO + SFT
3.78
2.97
DPO + SFT + Prpt
-
3.44
Table 2: LLM-as-Judge scores on Elyza and SpokenElyza. PT: Pretrained model. Prpt: Spoken system prompt. Higher is better; maximum score is 5. Best results per benchmark in bold .
Model
Word Count ↓
DD ↓
NV % ↓
PT
325.91
6.38
13.46%
DPO + SFT
302.15
6.42
12.65%
PT + Prpt
65.53
5.06
3.69%
DPO + SFT + Prpt
77.79
4.97
3.24%
Table 3: Surface-form evaluation results on SpokenElyza. PT: Pretrained model, Prpt: spoken system prompt. DD: dependency depth. NV%: percentage of non-verbalizable contents.
Elyza
SpokenElyza
Pretrained
3.97
2.91
TopLayers
3.61
2.91
TopLayers + Prpt
-
3.38
KQ-LN
3.78
2.97
KQ-LN + Prpt
-
3.44
Table 4: Comparison of parameter selection strategies. KQ-LN (key-query and LayerNorm) outperforms TopLayers tuning on both benchmarks.
Figure 2: Effect of DPO loss weight on LLM-as-Judge scores. Higher DPO weights improve SpokenElyza performance across all datasets. ALL: combination of all preference training datasets.