Recent advancements in Large Audio Language Models (LALMs) have demonstrated exceptional performance in speech recognition and translation. However, existing models often suffer from a disconnect between perception and expression, resulting in a robotic "read-speech" style that lacks the spontaneity and emotional resonance of real human interaction. In this report, we introduce Hello-Chat, an end-to-end audio language model designed for realistic social scenarios. By leveraging a massive dataset of real-life conversations and employing a modality-interleaved training strategy, Hello-Chat achieves a breakthrough in anthropomorphic generation. Experimental results show that our model not only reaches state-of-the-art (SOTA) performance on specific audio understanding tasks but also significantly outperforms existing baselines in prosodic naturalness and emotional alignment, paving the way for the next generation of empathetic AI agents.
Figures & tables
Figure 1: Architecture of Hello-Chat.
Primary Category
Attribute
Tag Examples
Speaker Profile
Gender & Age
Young male, Middle-aged female, Elderly male, Child, Infant…
Accent
Standard Mandarin, Beijing Accent, Cantonese, Wu/Shanghainese, English…
Prosody & Expression
Emotion
Neutral, Happy, Sad, Angry, Fearful, Surprised, Disgusted
Table 1: The Detailed Captioning Taxonomy for fine-grained acoustic perception.
Figure 2: Token organization strategies for the Talker module. We employ distinct token construction patterns—Dialogue Mode, Long-text Mode, and Standard Sentence Mode—to model different prosodic features and context dependencies.
Model
ASR ↓
NLP Question ↑
Translation ↑
MMAU ↑
Gemini3-Preview
4.06
8.85
8.87
0.75
GPT-4o-Audio
6.45
8.50
8.09
0.64
Qwen3-Omni-32B
3.51
8.66
8.07
0.74
Step-Audio 2 Mini
3.21
7.32
8.34
0.66
MiDashengLM
4.50
3.82
8.43
0.65
Kimi-Audio
3.36
7.41
8.26
0.59
Table 2: Audio Understanding Evaluation on Audio-to-Text Tasks. Note: Bold indicates the best result, underline indicates the second-best result.
Model
SER ↑
AED ↑
Gemini3-Preview
0.791
0.861
GPT-4o-Audio
0.586
0.489
Qwen3-Omni-32B
0.856
0.644
Step-Audio 2 Mini
0.680
0.533
MiDashengLM
0.561
0.441
Kimi-Audio
0.625
0.392
Table 3: Paralinguistic Perception Evaluation on Audio-to-Text Tasks. Note: Bold indicates the best result, underline indicates the second-best result.
Model
Only-Yes Accuracy (%) ↑
Gemini3-Preview
88
GPT-4o-Audio
23
Qwen3-Omni-32B
100
Step-Audio 2 mini
87
MiDashengLM
0
Kimi-Audio
22
Table 4: Instruction Following Evaluation on Audio-to-Text Tasks. Note: Bold indicates the best result, underline indicates the second-best result.
Model
CMOS ↑
CER (%) ↓
SS ↑
F5-TTS
3.48
1.56
0.741
CosyVoice 2
3.66
1.45
0.748
CosyVoice 3-0.5B
3.59
1.16
0.780
Qwen2.5-Omni-7B
-
1.70
0.752
Qwen3-TTS-12Hz-0.6B-Base
4.12
0.92
0.763
FireRedTTS-2
3.68
1.14
0.736
Table 5: Zero-Shot Performance Comparison of Various Systems on the SEED Chinese subset. Note: Bold indicates the best result, underline indicates the second-best result.
Appendix figures & tables2 assets
Supplementary material from the paper’s appendix.
Appendix
Metric
Baseline
w/o Caption
Single Template
ASR ↓
4.299
4.606
4.926
NLP Question ↑
7.671
7.471
7.497
Translation ↑
8.918
8.582
8.672
MMAU ↑
0.641
0.557
0.602
SER ↑
0.671
0.494
0.566
AED ↑
0.796
0.674
0.785
Appendix
Table 6: Ablation Results at Pretraining Stage 2 (Step 10k). Note: Bold indicates the best result.
Model
A2T
A2A
Difference
Baseline
0.594 | 24.1%
0.549 | 12.9%
-0.045 | -22.2%
Baseline + Non-Interleaved
0.813 | 70.0%
0.763 | 61.5%
-0.053 | -9.5%
Baseline + Interleaved
0.816 | 70.8%
0.783 | 63.3%
-0.033 | -7.3%
Appendix
Table 7: Ablation Results on Modality-Interleaved Training. Values denote semantic similarity ∣ semantic consistency. Note: Bold indicates the best result.