Reasoning-oriented large language models (RLMs) achieve strong gains on tasks such as mathematics and coding by generating explicit intermediate reasoning. However, their impact on machine translation (MT) remains underexplored. We systematically evaluate several open- and closed-weights RLMs on the WMT24++ benchmark and find that enabling explicit reasoning consistently degrades translation quality across languages and models. Analysis reveals that MT reasoning traces are highly linear, lacking revision, self-correction and exploration of alternative translations, which limits their usefulness. Furthermore, injecting higher-quality reasoning traces from stronger models does not reliably improve weaker models' performance. To address this mismatch, we propose a structured reasoning framework tailored to translation, based on multi-step drafting, adequacy refinement, fluency improvement, and selective iterative revision. We curate a synthetic dataset of dynamic structured reasoning traces and post-train a large reasoning model on this data. Experiments show significant improvements over standard translation fine-tuning and injected generic reasoning baselines. Our findings demonstrate that reasoning must be task-structured to benefit MT.
Figures & tables
Model
en-ar
en-cs
en-fa
en-fr
en-hi
en-ja
en-ko
en-ru
en-zh
Avg.
Command-A-Reasoning (w)
70.4
83.0
78.6
82.5
59.8
80.6
81.4
82.0
77.2
77.3
Command-A-Reasoning (w/o)
72.6
83.5
81.2
83.0
60.4
80.7
81.7
82.6
78.6
78.3
Claude-4-Opus (w)
74.9
84.9
81.5
83.2
64.4
83.9
83.5
84.2
80.7
80.1
Claude-4-Opus (w/o)
75.1
85.2
81.0
83.4
64.6
84.0
83.8
85.1
81.6
80.4
DeepSeek-R1 (w)
69.5
82.5
76.8
82.0
61.0
74.8
74.2
83.0
70.4
74.9
DeepSeek-R1 (w/o)
73.0
83.7
79.2
82.7
63.1
81.3
81.8
83.8
79.3
78.7
Table 1: Performance evaluation of reasoning language models on the WMT24++ benchmark using XCOMET-XL score. We observe that having reasoning traces hurts the translation quality across almost all languages and models.
Model
Avg. # Reasoning Paths
Std.
Command-A-Reasoning
1.56
0.23
Claude-4-Opus
0.006
0.0
DeepSeek-R1
0.107
0.16
Gemini-2.5-Flash
0.012
0.0
Table 2: Average number of reasoning paths in a generated reasoning trace per model, aggregated over the selected WMT24++ language pairs. All the RLMs show a high degree of linearity in their reasoning process.
Figure 1: The performance of open-weights RLMs when their reasoning traces are generated and injected from other models. The dash-line represents the baseline performance of the receiving models, and the injecting models are presented at the top. As can be seen, the strength of the injecting model used to generate reasoning traces does not correlate with final translation quality across different injecting and receiving models.
Figure 2: An example of the dynamic template used to generate structured reasoning traces. Curly-braced fields denote placeholders. Steps 2 and 3 are applied only to selected challenging sentences, those sentences that benefited the most from steps 2 and 3 with the defined MetricX margin; all other segments are discarded from the initial draft.
Setup
en-ar
en-cs
en-fa
en-fr
en-hi
en-ja
en-ko
en-ru
en-zh
Avg.
Baselines
Base model
70.4
83.0
78.6
82.5
59.8
80.6
81.4
82.0
77.2
77.3
Direct Translation
76.7
86.3
82.7
85.7
66.2
83.7
84.5
85.5
81.9
81.5
Structured Reasoning ours
77.8
87.1
83.6
86.4
66.3
83.8
84.7
86.2
82.1
82.0
Injected Reasoning
Command-A-Reasoning
76.9
86.7
82.9
85.7
66.1
83.4
84.5
85.8
82.3
81.6
Table 3: The systematic performance evaluation of different reasoning-based translation systems. At the top, we have the Command-A-Reasoning as the backbone model and Direct Translations, where the base model is trained on MT without reasoning. The structured reasoning is the base model trained on our proposed structured reasoning. Injected Reasoning represents a comparison between the performance of the base model trained on injected reasoning from different reasoning models. At the bottom, a comparison between different training sizes of structured reasoning
Setup
en-ar
en-cs
en-fa
en-fr
en-hi
en-ja
en-ko
en-ru
en-zh
Avg.
R-Src: Command-A-Reasoning
Command-A-Reasoning
73.4
83.4
79.5
83.4
60.3
80.4
81.0
82.6
78.6
78.1
Command-A-Translate
76.9
86.7
82.9
85.7
66.1
83.4
84.5
85.8
82.3
81.6
R-Src: DeepSeek-R1
DeepSeek-R1
72.0
82.3
77.9
81.9
59.9
80.2
81.0
81.9
76.9
77.1
Command-A-Translate
76.5
86.5
83.1
85.7
66.1
83.7
84.5
85.6
82.3
81.6
Table 4: Comparing the role of reasoning quality and final translation quality on the performance. R-Src represents the injecting model of reasoning traces. Each row in a block represents the model that generated the final translations. As shown, both the quality of reasoning traces and final translations contribute positively to performance across languages, indicating that reasoning traces from stronger models improve performance and that higher-quality final translations in the training data lead to better performance.
Large Reasoning Models increasingly use intermediate traces for machine translation, but it remains unclear when such reasoning helps or hurts. We analyze reasoning traces across models, languages, domains, and datasets, focusing on reasoning language, length, and structure. We find that the best reasoning language is model-specific, reasoning length has a non-monotonic relationship with quality, and traces exhibit recurring functional patterns. To uncover these patterns, we introduce Hierarchical Meta-Summarization (HMS), a scalable framework that induces coarse- and fine-grained reasoning structures without predefined taxonomies. HMS reveals a shared organization--understanding/planning, translating/drafting, and refining/verifying--alongside domain-specific variation. Our results suggest that MT reasoning should be controlled in a model-aware, length-aware, and pattern-aware manner rather than uniformly encouraged.
Yuxiang Liu, Jiaming Luo, Eleftheria Briakou +1
University of Illinois at Urbana-Champaign · Google DeepMind
Large Language Models (LLMs) can be trained to perform chain-of-thoughts reasoning in order to improve the reliability of their responses. In this work, we investigate how explicit reasoning can be leveraged for LLM-Based Machine Translation (MT) with in-context samples. We introduce a novel fragment-based reasoning framework in which the model first extracts parallel source-target fragments from retrieved similar exemplars, and uses these fragments as intermediate reasoning traces to produce the final translation. To train our model, we distill silver fragments and drafts from a large teacher model. Our experiments with the Qwen3 model family, over 6 languages, including up to 5 domains per language, demonstrate that fragment-based MT significantly outperforms alternative methods like standard k-shot or basic drafting.
Maxime Bouthors, Josep Crego, François Yvon
SYSTRAN by ChapsVision, 5 rue Feydeau, F-75002 Paris, France · Sorbonne Université, CNRS, ISIR, F-75005 Paris, France
Across multiple language pairings (English → {Spanish, French, German, Mandarin, Japanese, Urdu, Cantonese}), we find reasoning errors in translation. To quantify how often these reasoning errors occur, we leverage an automated annotation protocol for reasoning evaluation wherein the goal is to detect if a reasoning step is any of three error categories: (1) source sentence-misaligned, (2) model hypothesis-misaligned, or (3) reasoning trace-misaligned. We probe the reasoning model with perturbed traces correcting for these identified reasoning errors using an array of weak-to-strong interventions: hedging, removal, re-reasoning after removal, hindsight, and oracle interventions. Experimenting with interventions on the reasoning traces suggests that small corrections to the reasoning have little impact on translation quality, but stronger interventions yield the highest resolution rates, despite translation quality gains being mixed. We find ultimately that reasoning errors in MT can be identified with high precision in Urdu but lower precision in Spanish, but that removing these reasoning errors does not resolve the initial errors significantly, suggesting limited reasoning faithfulness for machine translation.
Calvin Bao, Marine Carpuat
Department of Computer Science, University of Maryland, College Park, MD 20742, USA