Supervised fine tuning (SFT) is commonly monitored through training and evaluation loss, but loss does not fully characterize how model functionality evolves along the same trajectory. We study this gap through dialect adaptation of Qwen3-ASR, MiMo-Audio, and FireRed-LLM across learning rates from 10−5 to 10−3. Character error rate (CER) reveals pronounced transient collapse and recovery whose severity and temporal structure are only weakly reflected by loss. Bidirectional layer patching reveals strong cross checkpoint functional incompatibility during recoverable collapse, with model dependent depth profiles that reorganize during recovery. Pairwise gradient similarity alone does not reliably indicate optimization stability, whereas raw gradient spectral analysis provides a clearer structural distinction between stable and unstable optimization: successfully converged runs approach a broad and temporally stable spectral energy distribution, while failed runs remain spectrally concentrated or exhibit persistent fluctuations. Finally, AdamW update transfer bypasses pronounced early collapse, yet substantial Mandarin degradation still emerges later in training. These results motivate three separable SFT dynamics: collapse, repair, and drift, and show that transient collapse is not necessary for persistent source domain degradation.
Figures & tables
Figure 1: Evaluation loss and recognition CER can exhibit qualitatively different trajectories during SFT.
Figure 2: Evaluation loss trajectories during SFT across three models and learning rates.
Figure 3: Test CER curves during supervised fine-tuning across three models and learning rates on Mandarin (top) and dialect (bottom) test sets. The piecewise-linear x-axis expands steps 0–20.
Figure 4: Bidirectional layer patching across models. Qwen3-ASR and FireRed-LLM use a learning rate of 10−4 , while MiMo-Audio uses 10−5 . The top and bottom rows show forward and reverse patching, respectively. Colored curves highlight selected checkpoints; gray curves show the remaining checkpoints. The dashed Base line in each column marks the corresponding pretrained model’s decoding CER.
Figure 5: Gradient cosine similarity across models and learning rates. Top: within-step similarity across dialects. Bottom: similarity between adjacent steps within each dialect. Colored curves show the median, shaded bands indicate the interquartile range, and gray curves show individual series.
Figure 6: Gradient effective dimensionality across models and learning rates, measured by Deff . Values near 1 indicate concentrated spectral energy; values near 5 indicate approximately equal energy shares.
Figure 7: CER trajectories of teacher and student models during SFT for Qwen3-ASR at 10−4 and 10−5 and FireRed-LLM at 10−4 . The top row shows Mandarin speech and the bottom row shows dialect speech. The horizontal axis uses zero-indexed global batch steps: the student remains at the pretrained parameters during steps 0–9 and receives AdamW update transfer from step 10 to step 70.
Appendix figures & tables7 assets
Supplementary material from the paper’s appendix.
Appendix
Training
Validation
Test
Dialect / Language
Utterances
Hours
Utterances
Hours
Utterances
Hours
Changsha
50,000
61.52
5,000
5.90
1,000
1.17
Hakka
50,000
69.82
5,000
7.03
1,000
1.40
Minnan
50,000
81.47
5,000
8.16
1,000
1.65
Nanchang
50,000
38.36
5,000
3.88
1,000
0.78
Ningbo
50,000
136.02
5,000
13.66
1,000
2.67
Appendix
Table 1: Dataset statistics. Duration is reported in hours. Mandarin data are from SpeechIO-3 and used only for evaluation.
Model
Input and assistant prefix
Supervised tokens
Qwen3-ASR
Audio and the fixed prefix language Chinese<asr_text>
Transcript and EOS token
FireRed-LLM
<speech> followed by 请转写音频为文字
Plain-text transcript in the assistant response
MiMo-Audio
Audio and the fixed instruction 请将这段语音转换为文字; the assistant prefix includes an empty thinking block
Plain-text transcript and <|im_end|>
Appendix
Table 2: Input templates and supervised tokens for the three ASR models. Fixed input-side tokens and assistant prefixes are excluded from the loss.
Figure 8: Bidirectional layer patching at learning rates 10−5 (top), 10−4 (middle), and 10−3 (bottom). Red stars beside model names indicate runs with effective adaptation. Within each plot, the upper row shows forward patching and the lower row shows reverse patching. Colored curves highlight steps 1, 10, and tpeak ; gray curves show the remaining early checkpoints.
Figure 9: Evolution of gradient spectral energy during SFT across three models and learning rates. Stacked areas show the energy share of each singular component, pk=σk2/∑jσj2 , ordered by descending singular value at each training step.
Figure 10: Test CER with and without learning-rate warm-up (warm-up ratio: 0.02). Rows show FireRed-LLM at 10−4 , Qwen3-ASR at 10−4 , and Qwen3-ASR at 10−5 , respectively. Columns show Mandarin (left) and dialect speech (right). Orange and purple curves denote runs without and with warm-up, respectively. The vertical axis is logarithmic; the horizontal axis uses different linear scales before and after step 20 to highlight early training.
Table 3: Representative dialect ASR predictions during early recognition collapse in Qwen3-ASR at step 3 with a learning rate of 10−4 . This run subsequently recovers and continues effective training.