When Does Reasoning Help in Machine Translation? A Hierarchical Analysis of LRM Reasoning Traces

Yuxiang Liu; Jiaming Luo; Eleftheria Briakou; Colin Cherry

cs.CLSep 18, 2026

When Does Reasoning Help in Machine Translation? A Hierarchical Analysis of LRM Reasoning Traces

Authors: Yuxiang Liu, Jiaming Luo, Eleftheria Briakou, Colin Cherry

Organizations: University of Illinois at Urbana-Champaign · Google DeepMind

Abstract

Large Reasoning Models increasingly use intermediate traces for machine translation, but it remains unclear when such reasoning helps or hurts. We analyze reasoning traces across models, languages, domains, and datasets, focusing on reasoning language, length, and structure. We find that the best reasoning language is model-specific, reasoning length has a non-monotonic relationship with quality, and traces exhibit recurring functional patterns. To uncover these patterns, we introduce Hierarchical Meta-Summarization (HMS), a scalable framework that induces coarse- and fine-grained reasoning structures without predefined taxonomies. HMS reveals a shared organization--understanding/planning, translating/drafting, and refining/verifying--alongside domain-specific variation. Our results suggest that MT reasoning should be controlled in a model-aware, length-aware, and pattern-aware manner rather than uniformly encouraged.

Figures & tables

Inst. $\rightarrow$ Reason	WMT					CMT			DRT	Mean
	en-de	en-es	en-ja	en-ru	en-zh	en-es	en-fr	en-zh	en-zh
en $\rightarrow$ en (%)	78.6	93.3	29.4	66.8	64.2	92.5	92.9	57.0	63.8	71.0
zh $\rightarrow$ zh (%)	97.2	98.3	92.1	94.8	99.2	97.5	97.8	99.8	99.0	97.3
tgt $\rightarrow$ tgt (%)	27.7	7.2	69.3	55.0	–	4.6	7.3	–	–	28.5

Table 1: Reasoning-Language Instruction-Adherence (RLIA) rates for Qwen-32B. The model follows Chinese reasoning-language instructions most reliably, while target-language instructions yield much lower adherence.

Metric	Inst.	WMT					CMT			DRT	Mean
		en-de	en-es	en-ja	en-ru	en-zh	en-es	en-fr	en-zh	en-zh
COMET $\uparrow$	en	0.7613	0.7774	0.8186	0.7631	0.8297	0.8063	0.7676	0.8361	0.7742	0.7927
COMET $\uparrow$	zh	0.7442	0.7976	0.8267	0.7662	0.8363	0.8135	0.7636	0.8404	0.7810	0.7966
Norm.MX $\uparrow$	en	0.8640	0.8293	0.7685	0.7728	0.8825	0.8456	0.8219	0.8634	0.8400	0.8320
Norm.MX $\uparrow$	zh	0.8710	0.8526	0.7842	0.7807	0.8894	0.8678	0.8296	0.8715	0.8502	0.8441
Norm.MX-QE $\uparrow$	en	0.8766	0.8358	0.7845	0.7977	0.8884	0.8605	0.8676	0.8769	0.8361	0.8471

Table 2: Translation quality for Qwen-32B by instructed reasoning language. Chinese reasoning yields higher mean scores across all three metrics, especially on Norm.MX and Norm.MX-QE.

Model	en $\rightarrow$ en (%)	zh $\rightarrow$ zh (%)
Qwen-14B	71.6	97.6
Qwen-32B	64.2	99.2
gpt-oss-20B	98.3	2.4
Llama-8B	77.8	99.9
Gemma-4-E4B	95.3	2.1

Table 3: RLIA on WMT (en-zh). Qwen and Llama favor Chinese reasoning instructions, whereas gpt-oss-20B and Gemma-4-E4B favor English.

Model	Inst.	COMET $\uparrow$	MX $\uparrow$	MX-QE $\uparrow$
Qwen-14B	en	0.8227	0.8644	0.8730
Qwen-14B	zh	0.8285	0.8749	0.8838
Qwen-32B	en	0.8297	0.8825	0.8884
Qwen-32B	zh	0.8363	0.8894	0.8960
gpt-oss-20B	en	0.8352	0.8837	0.8935
gpt-oss-20B	zh	0.8326	0.8791	0.8886

Table 4: Translation quality on WMT (en-zh). Chinese reasoning instructions perform better for Qwen and Llama, whereas English reasoning instructions perform better for gpt-oss-20B and Gemma-4-E4B.

Figure 1: Translation quality by ranked reasoning trace length for Qwen-32B on WMT.

Figure 2: Hierarchical Meta-Summarization (HMS). HMS is a data-driven framework for structural reasoning pattern analysis that abstracts raw reasoning traces into atomic steps and recursively applies meta-summarization to derive coarse Level-0 patterns and finer Level-1 sub-patterns.

Model	Induced HMS cluster	Function
Gemma-4-E4B	Analyzing Source Text	U/P
	Drafting Initial Translation	T/D
	Refining and Polishing Translation	R/V
Llama-8B	Initial Understanding and Planning	U/P
	Executing Translation	T/D
	Refining and Verifying	R/V

Table 5: Cross-model mapping of induced HMS clusters to shared functions. General-purpose LRMs largely recover the three-stage structure, whereas MT-specialized DRT-14B emphasizes critique and revision.

Panel of Figure 3 — Figure 3: Level-0 reasoning allocation on WMT24 across four domains. Reasoning ratios remain stable, while reasoning budgets vary modestly by domain.

Figure 4: Qwen-32B COMET scores on WMT24 when the reasoning pattern is set to its maximum versus minimum observed length. Longer U/P is the most stable positive signal, while T/D and R/V effects vary more by domain.

Appendix figures & tables21 assets

Supplementary material from the paper’s appendix.

Appendix

Inst. $\rightarrow$ Reason	WMT					CMT			DRT	Mean
	en-de	en-es	en-ja	en-ru	en-zh	en-es	en-fr	en-zh	en-zh
en $\rightarrow$ en (%)	87.7	97.3	43.5	72.8	71.6	95.9	96.7	64.0	66.8	77.4
zh $\rightarrow$ zh (%)	93.2	91.2	93.9	82.4	97.6	90.0	90.4	99.2	98.7	92.9

Appendix

Table 6: Reasoning-Language Instruction-Adherence (RLIA) rates for Qwen-14B. Chinese instructions yield higher overall adherence than English, though English is stronger on several English-to-European language pairs.

Metric	Inst.	WMT					CMT			DRT	Mean
		en-de	en-es	en-ja	en-ru	en-zh	en-es	en-fr	en-zh	en-zh
COMET $\uparrow$	en	0.7520	0.7872	0.8051	0.7259	0.8227	0.8205	0.7713	0.8267	0.7600	0.7857
COMET $\uparrow$	zh	0.7450	0.7940	0.8086	0.7192	0.8285	0.8176	0.7684	0.8329	0.7669	0.7868
Norm.MX $\uparrow$	en	0.8258	0.8071	0.7334	0.6925	0.8644	0.8572	0.8168	0.8420	0.8163	0.8062
Norm.MX $\uparrow$	zh	0.8346	0.8282	0.7555	0.7076	0.8749	0.8609	0.8175	0.8565	0.8272	0.8181
Norm.MX-QE $\uparrow$	en	0.8438	0.8138	0.7508	0.7204	0.8730	0.8749	0.8648	0.8579	0.8164	0.8240

Appendix

Table 7: Translation quality for Qwen-14B by instructed reasoning language. Chinese reasoning yields higher mean scores across all three metrics, with especially consistent gains on Norm.MX and Norm.MX-QE.

Model	Inst.	zh%	en%	COMET $\uparrow$	Norm.MX $\uparrow$	Norm.MX-QE $\uparrow$
Qwen-14B	English	28.4%	71.6%	0.8227	0.8644	0.8730
	Chinese	97.6%	2.4%	0.8285	0.8749	0.8838
	None	38.0%	62.0%	0.8235	0.8659	0.8745
Qwen-32B	English	35.8%	64.2%	0.8297	0.8825	0.8884
	Chinese	99.2%	0.8%	0.8363	0.8894	0.8960
	None	44.0%	56.0%	0.8306	0.8834	0.8894

Appendix

Table 8: Reasoning-language distributions and translation quality under English, Chinese, and no-language-control conditions on WMT24 en-zh.

Dataset	Model	Lang.	Ranked reasoning trace length																$r_{16}/r_{1}$
Dataset	Model	Lang.	$r_{1}$	$r_{2}$	$r_{3}$	$r_{4}$	$r_{5}$	$r_{6}$	$r_{7}$	$r_{8}$	$r_{9}$	$r_{10}$	$r_{11}$	$r_{12}$	$r_{13}$	$r_{14}$	$r_{15}$	$r_{16}$	$r_{16}/r_{1}$
WMT	Qwen-14B	en	236	274	300	322	342	361	380	399	419	443	467	495	530	577	648	825	$3.49\times$
	Qwen-14B	zh	339	404	444	476	502	527	550	573	596	619	643	671	705	744	800	934	$2.76\times$
	Qwen-32B	en	276	325	359	389	416	442	469	496	525	556	592	635	685	752	856	1092	$3.96\times$
	Qwen-32B	zh	406	479	524	559	589	618	644	670	696	724	753	785	826	874	946	1147	$2.83\times$
CMT	Qwen-14B	en	240	277	302	323	341	360	377	395	414	435	457	483	515	554	613	739	$3.07\times$

Appendix

Table 9: Reasoning trace length by rank across datasets. For each input, 16 traces are sorted from shortest (

r_{1}

) to longest (

r_{16}

); cells report the mean length at each rank.

Model	Lang.	Ranked reasoning trace length																$r_{16}/r_{1}$
Model	Lang.	$r_{1}$	$r_{2}$	$r_{3}$	$r_{4}$	$r_{5}$	$r_{6}$	$r_{7}$	$r_{8}$	$r_{9}$	$r_{10}$	$r_{11}$	$r_{12}$	$r_{13}$	$r_{14}$	$r_{15}$	$r_{16}$	$r_{16}/r_{1}$
Qwen-14B	en	195	226	249	268	284	301	319	335	351	377	396	416	442	473	516	599	$3.07\times$
Qwen-14B	zh	324	380	418	448	473	497	518	538	559	580	603	632	665	698	740	817	$2.52\times$
Qwen-32B	en	212	253	281	305	327	348	369	392	416	439	466	496	531	575	655	796	$3.76\times$
Qwen-32B	zh	369	432	478	513	543	572	598	625	649	675	702	730	765	807	866	996	$2.70\times$
gpt-oss-20B	en	158	204	240	269	296	318	340	364	385	408	433	460	490	530	587	704	$4.45\times$

Appendix

Table 10: Reasoning trace length by rank on WMT (en-zh). For each input, 16 traces are sorted from shortest (

r_{1}

) to longest (

r_{16}

); cells report the mean length at each rank.

Figure 5: Translation quality by ranked reasoning trace length for Qwen-32B on CMT.

Figure 6: Translation quality by ranked reasoning trace length for Qwen-32B on DRT.

Figure 7: Translation quality by ranked reasoning trace length for Qwen-14B on WMT.

Figure 8: Translation quality by ranked reasoning trace length for Qwen-14B on CMT.

Figure 9: Translation quality by ranked reasoning trace length for Qwen-14B on DRT.

Figure 10: Translation quality by ranked reasoning trace length on WMT (en-zh) across evaluated models. Adjacent length ranks are grouped into bins from shortest to longest traces.

Effort	COMET	Norm.MX	Norm.MX-QE	#words
Low	0.8665	0.8428	0.8501	93.3
Med	0.8696	0.8455	0.8530	143.6
High	0.8689	0.8469	0.8547	173.1

Appendix

Table 11: Controlled reasoning effort for gpt-oss-20B on WMT (en-zh). Higher effort increases trace length and slightly improves Norm.MX and Norm.MX-QE, while COMET peaks at medium effort.

HMS stage	API calls per trace	Input tokens per trace	Output tokens per trace
Step segmentation + abstraction	1	$\sim$ 900–1,000	$\sim$ 570–690
Meta-summarization	amortized, negligible	$<10$	$<10$
Step labeling	$\sim$ 7	$\sim$ 5,300–6,500	$\sim$ 15–30
Total, step-wise labeling	$\sim$ 8 calls	$\sim$ 6.2k–7.5k	$\sim$ 0.6k–0.7k

Appendix

Table 12: Approximate API call and token costs per trace for HMS step-wise labeling.

Evaluation	Yes rate	Fleiss’ $\kappa$
Summary faithfulness	92%	0.71
Level-0 label correctness	88%	0.68
Level-1 label correctness	79%	0.61
R/V corrective behavior	67%	0.65
R/V checking behavior	24%	0.60
Overall R/V validity	91%	0.74

Appendix

Table 13: Human evaluation of HMS summary and label quality with inter-annotator agreement.

Dataset	Stage	Level-0 Pattern	Level-1 Sub-patterns
WMT	U/P	Understanding & Planning	Understanding Task and Context
			Identifying Linguistic & Cultural Nuances
			Understanding User Intent and Context
			Formulating & Planning Translation Strategy
			Source Text Analysis and Deconstruction
	T/D	Translating & Drafting	Initial Translation and Drafting

Appendix

Table 14: HMS-derived hierarchical reasoning pattern labels for Qwen-32B across three translation datasets. For DRT, HMS separates initial interpretation from translation-strategy planning, reflecting the stronger semantic and stylistic demands of literary translation.

Panel of Figure 11 — Figure 11: Domain-specific allocation of WMT24 Level-1 reasoning patterns for Qwen-32B. Absolute budgets and relative ratios show how reasoning effort is distributed across fine-grained patterns within each functional stage.

Figure 12: Qwen-32B normalized reference-based MetricX scores under maximum versus minimum reasoning lengths on WMT24. Longer U/P is the most stable positive signal, while T/D and R/V effects vary more by domain.

Figure 13: Qwen-32B normalized MetricX-QE scores under maximum versus minimum reasoning lengths on WMT24. Longer U/P is generally favorable; longer T/D and R/V show more selective, domain-dependent patterns.

Figure 14: Llama-8B COMET scores under maximum versus minimum reasoning lengths on WMT24. The comparison broadly follows the Qwen-32B trend, with U/P more stable than T/D or R/V across domains.

Figure 15: Llama-8B normalized reference-based MetricX scores under maximum versus minimum reasoning lengths on WMT24. U/P shows the clearest positive pattern, whereas T/D and R/V depend more on domain.

Figure 16: Llama-8B normalized MetricX-QE scores under maximum versus minimum reasoning lengths on WMT24. Longer reasoning is not uniformly beneficial: U/P is comparatively robust, while T/D and R/V are more conditional.

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 259 papers

Unlocking Reasoning Capability on Machine Translation in Large Language Models
Feb 16, 2026Sara Rajaee, Sebastian Vincent, Alexandre Berard +3Structured Reasoning in Language Models Large Reasoning Models
University of Amsterdam · Cohere · Cohere Labs
Should We be Pedantic About Reasoning Errors in Machine Translation?
Apr 10, 2026Calvin Bao, Marine CarpuatMachine Translation MT Evaluation
Department of Computer Science, University of Maryland, College Park, MD 20742, USA
Reasoning about In-Context Samples for Machine-Translation
Aug 27, 2026Maxime Bouthors, Josep Crego, François YvonMachine Translation In-Context Learning
SYSTRAN by ChapsVision, 5 rue Feydeau, F-75002 Paris, France · Sorbonne Université, CNRS, ISIR, F-75005 Paris, France

Feb 16, 2026J/K move · Enter open · S save

Unlocking Reasoning Capability on Machine Translation in Large Language Models

Sara Rajaee, Sebastian Vincent, Alexandre Berard, Marzieh Fadaee+2

University of Amsterdam · Cohere · Cohere Labs

Structured Reasoning in Language Models Large Reasoning Models LLM Fine-Tuning Cross-Lingual Reasoning in Language Models

Open paper AskPDF arXiv

Reasoning-oriented large language models (RLMs) achieve strong gains on tasks such as mathematics and coding by generating explicit intermediate reasoning. However, their impact on machine translation (MT) remains underexplored. We systematically evaluate several open- and closed-weights RLMs on the WMT24++ benchmark and find that enabling explicit reasoning consistently degrades translation quality across languages and models. Analysis reveals that MT reasoning traces are highly linear, lacking revision, self-correction and exploration of alternative translations, which limits their usefulness. Furthermore, injecting higher-quality reasoning traces from stronger models does not reliably improve weaker models' performance. To address this mismatch, we propose a structured reasoning framework tailored to translation, based on multi-step drafting, adequacy refinement, fluency improvement, and selective iterative revision. We curate a synthetic dataset of dynamic structured reasoning traces and post-train a large reasoning model on this data. Experiments show significant improvements over standard translation fine-tuning and injected generic reasoning baselines. Our findings demonstrate that reasoning must be task-structured to benefit MT.

Abstract

Machine Translation

Explore similar work

Unlocking Reasoning Capability on Machine Translation in Large Language Models

Should We be Pedantic About Reasoning Errors in Machine Translation?

Reasoning about In-Context Samples for Machine-Translation