Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation

Shijun Li; Yang Liu; Najmeh Sadoughi; Dongmei Wang; Xinjie Li; Zhu Liu; Rohith Mysore Vijaya Kumar; Yuzong Liu

cs.CLOct 14, 2026

Retrieve What Helps, Only When It Helps: Agentic Reinforcement Learning for Selective Retrieval-Augmented Machine Translation

Authors: Shijun Li, Yang Liu, Najmeh Sadoughi, Dongmei Wang, Xinjie Li, Zhu Liu, Rohith Mysore Vijaya Kumar, Yuzong Liu

Organizations: The University of Texas at Austin · Amazon

Abstract

Retrieving external knowledge can improve large language model (LLM) translation by providing supplementary information such as idiom interpretations and terminology. However, existing retrieval-augmented machine translation systems typically apply predefined retrieval strategies uniformly across all inputs. We argue that translation operates in a partial-evidence regime: LLM's parametric knowledge suffices for many inputs, while external evidence benefits a subset and usually supports rather than directly provides the desired translation. This creates two challenges: deciding when to retrieve and what evidence to acquire. We introduce Selective Retrieval-Augmented Translation (SRAT), an agentic reinforcement learning framework that learns when to translate directly or retrieve, what to retrieve, and how to integrate these decisions by optimizing final translation quality. To align evidence retrieval with downstream translation quality rather than simple semantic similarity, we further develop joint Plackett--Luce optimization and translation-quality contrastive training for retriever-training. Experiments on idiom, knowledge-intensive, and document-level translation with offline or online retrieval show that SRAT consistently outperforms never- and always-retrieve baselines, and quality-aligned retriever training provides further gains when similarity-based retrieval is insufficient. The results demonstrate the benefit of treating retrieval as a learnable translation action rather than fixed preprocessing.

Figures & tables

Figure 1: SRAT’s workflow by integrating reasoning, retrieval, and translation for LLM-based translation.

Figure 2: Joint translator–retriever optimization. Plackett–Luce sampling exposes the retriever selection probability, allowing the trajectory advantage to update both components.

Figure 3: Translation-quality contrastive alignment. Candidate documents are evaluated by their effect on translation quality and converted into direct retriever supervision with the translator fixed.

	Zh $\rightarrow$ En						En $\rightarrow$ Zh
Model	BLEU	COMET	I5	I3	MQM	Ret. (%)	BLEU	COMET	I5	I3	MQM	Ret. (%)
Direct	21.50	0.805	0.652	0.668	-3.94	0.0	33.10	0.829	0.646	0.689	-3.88	0.0
Self-CoT	20.18	0.795	0.673	0.658	-5.10	0.0	33.02	0.838	0.650	0.708	-3.32	0.0
KB-CoT	19.44	0.792	0.658	0.644	-4.50	100.0	33.86	0.842	0.662	0.700	-3.84	100.0
Ladder	24.36	0.804	0.646	0.637	-3.60	0.0	47.33	0.857	0.750	0.814	-2.30	0.0
CSC	24.95	0.812	0.635	0.643	-4.01	100.0	48.47	0.860	0.763	0.829	-2.11	100.0

Table 1: Results on IdiomKB. I5 and I3 denote Idiom-Judge-5 and Idiom-Judge-3.

	En $\rightarrow$ Zh						En $\rightarrow$ De
Model	BLEU	COMET	GRF	GRB	MQM	Ret. (%)	BLEU	COMET	GRF	GRB	MQM	Ret. (%)
Direct	45.74	0.867	0.856	0.774	-11.27	0.0	29.40	0.806	0.743	0.654	-21.28	0.0
Self-CoT	49.67	0.872	0.870	0.781	-10.07	0.0	29.95	0.819	0.775	0.681	-19.27	0.0
KB-CoT	46.01	0.868	0.874	0.786	-11.60	100.0	29.89	0.808	0.747	0.658	-20.72	100.0
Ladder	54.05	0.875	0.913	0.831	-5.50	0.0	38.38	0.829	0.877	0.779	-7.93	0.0
CSC	55.42	0.880	0.920	0.840	-5.40	100.0	38.18	0.836	0.880	0.796	-7.54	100.0

Table 2: Results on RAGtrans. GRF and GRB are the reference-free and reference-based LLM judges.

Alignment	Dir.	BLEU	COMET	I5	I3	MQM	Dir.	BLEU	COMET	GRF	GRB	MQM
	IdiomKB						RAGtrans
Similarity	Zh $\rightarrow$ En	24.62	0.822	0.731	0.728	-2.74	En $\rightarrow$ Zh	52.37	0.888	0.939	0.860	-4.58
Joint	Zh $\rightarrow$ En	24.71	0.818	0.725	0.733	-2.80	En $\rightarrow$ Zh	53.49	0.891	0.941	0.863	-4.41
Contrastive	Zh $\rightarrow$ En	24.75	0.824	0.730	0.734	-2.70	En $\rightarrow$ Zh	53.99	0.898	0.947	0.871	-4.27
Similarity	En $\rightarrow$ Zh	48.25	0.891	0.792	0.871	-1.85	En $\rightarrow$ De	36.84	0.857	0.917	0.826	-5.98
Joint	En $\rightarrow$ Zh	48.73	0.886	0.790	0.877	-1.52	En $\rightarrow$ De	36.86	0.858	0.917	0.828	-5.89

Table 3: Retriever alignment on both datasets. Similarity uses the default retriever; Joint uses Plackett–Luce policy-gradient training; and Contrastive uses separate, contrastive training.

[IdiomKB]	Zh $\rightarrow$ En						En $\rightarrow$ Zh
Model	BLEU	COMET	I5	I3	MQM	Ret. (%)	BLEU	COMET	I5	I3	MQM	Ret. (%)
Claude	28.30	0.826	0.758	0.752	-0.92	0.0	54.63	0.903	0.812	0.878	-0.99	0.0
Claude-Ret	28.13	0.821	0.760	0.748	-0.99	3.4	57.21	0.902	0.814	0.882	-1.12	11.8
SRAT -Claude	28.39	0.829	0.763	0.755	-0.94	1.9	56.85	0.907	0.814	0.879	-0.97	10.2
[RAGtrans]	En $\rightarrow$ Zh						En $\rightarrow$ De
Model	BLEU	COMET	GRF	GRB	MQM	Ret. (%)	BLEU	COMET	GRF	GRB	MQM	Ret. (%)

Table 4: Composition with a black-box translator. Claude-Ret uses prompted retrieval decisions. SRAT -Claude keeps Claude frozen and uses the learned Qwen policy for retrieval and context construction.

Appendix figures & tables12 assets

Supplementary material from the paper’s appendix.

Appendix

Model	BLEU	COMET	GRF	GRB	MQM	Ret. (%)
Direct	17.01	0.690	0.601	0.544	-20.78	–
KB-CoT	18.60	0.708	0.622	0.554	-19.73	100.0
Self-CoT	19.29	0.746	0.715	0.640	-13.86	–
Ladder	24.96	0.796	0.831	0.740	-6.87	–
CSC	25.01	0.800	0.829	0.737	-7.06	100.0
SRAT-N	23.21	0.801	0.830	0.740	-7.09	0.0

Appendix

Table 5: Results on Document-MT-LLM for En

\rightarrow

Ru translation with online retrieval.

	En $\rightarrow$ Zh						En $\rightarrow$ De
Model	BLEU	COMET	GRF	GRB	MQM	Ret. (%)	BLEU	COMET	GRF	GRB	MQM	Ret. (%)
GLM 5	58.58	0.892	0.953	0.891	-1.666	0.0	46.35	0.869	0.956	0.882	-1.668	0.0
GLM-Ret	57.98	0.891	0.945	0.887	-2.076	68.1	46.41	0.868	0.952	0.880	-1.956	79.7
SRAT -GLM	58.82	0.893	0.953	0.892	-1.661	32.3	46.52	0.870	0.957	0.883	-1.662	24.0

Appendix

Table 6: Results with GLM 5 as the frozen black-box translator on RAGtrans. GLM-Ret uses prompted retrieval decisions, while SRAT -GLM trains a separate Qwen policy for retrieval and context construction using GLM’s translation quality as reward.

[IdiomKB]
Model	Dir.	BLEU	COMET	J-F	J-B	MQM	Ret.
SRAT	Zh $\rightarrow$ En	24.62	0.822	0.731	0.728	-2.74	30.6
Oracle	Zh $\rightarrow$ En	25.27	0.831	0.739	0.726	-2.67	38.5
SRAT	En $\rightarrow$ Zh	48.25	0.891	0.792	0.871	-1.85	77.0
Oracle	En $\rightarrow$ Zh	48.11	0.897	0.786	0.860	-1.74	57.2
[RAGtrans]

Appendix

Table 7: Comparison between SRAT and the Oracle retrieval policy. J-F/J-B correspond to I5/I3 on IdiomKB and GRF/GRB on RAGtrans and Document-MT; Ret. denotes retrieval rate (%). The Oracle selects between SRAT-N and SRAT-A for each example using the quality score in the RL reward.

Dataset	Dir.	Ret.	Multi-Ret.	Reason.
IdiomKB	Zh $\rightarrow$ En	30.6	0.6	0.0
	En $\rightarrow$ Zh	77.0	1.4	0.1
RAGtrans	En $\rightarrow$ Zh	52.2	2.1	89.2
	En $\rightarrow$ De	43.0	1.9	92.1
Document-MT	En $\rightarrow$ Ru	88.6	3.1	66.3

Appendix

Table 8: Learned retrieval and reasoning behaviors of SRAT . Ret., Multi-Ret., and Reason. denote retrieval, multi-retrieval, and reasoning rates (%), respectively.

	Opus 4.8			DeepSeek 3.2
Model	I5	I3	MQM	I5	I3	MQM
Direct	0.704	0.711	-3.52	0.464	0.379	-1.94
KB-CoT	0.736	0.756	-3.12	0.498	0.412	-1.64
Self-CoT	0.726	0.754	-3.10	0.481	0.402	-1.97
Ladder	0.829	0.857	-1.47	0.545	0.450	-1.25
CSC	0.840	0.874	-1.37	0.558	0.464	-1.33

Appendix

Table 9: IdiomKB En

\rightarrow

Zh results under alternative LLM evaluators.

	Opus 4.8			DeepSeek 3.2
Model	GRF	GRB	MQM	GRF	GRB	MQM
Direct	0.707	0.713	-13.95	0.831	0.804	-10.35
KB-CoT	0.711	0.715	-13.69	0.836	0.815	-10.37
Self-CoT	0.734	0.735	-12.44	0.848	0.825	-9.76
Ladder	0.874	0.855	-4.65	0.924	0.890	-3.01
CSC	0.872	0.850	-4.80	0.920	0.891	-3.10

Appendix

Table 10: RAGtrans En

\rightarrow

De results under alternative LLM evaluators.

Task	Model	BLEU	COMET	J-F	J-B	MQM
En $\rightarrow$ Zh	CSC	55.42	0.880	0.920	0.840	-5.40
	SRAT (SFT)	55.34	0.875	0.918	0.837	-5.45
	SRAT (SFT+RL)	52.37	0.888	0.939	0.860	-4.58
Zh $\rightarrow$ En	CSC	24.95	0.812	0.635	0.643	-4.01
	SRAT (SFT)	24.91	0.815	0.640	0.644	-3.95
	SRAT (SFT+RL)	24.62	0.822	0.731	0.728	-2.74

Appendix

Table 11: Contribution of SFT and RL. En

\rightarrow

Zh uses RAGtrans, where J-F/J-B denote GRF/GRB; Zh

\rightarrow

En uses IdiomKB, where they denote I5/I3.

	RAGtrans: En $\rightarrow$ De						IdiomKB: En $\rightarrow$ Zh
Model	BLEU	COMET	GRF	GRB	MQM	Ret.	BLEU	COMET	I5	I3	MQM	Ret.
Direct	34.82	0.8481	0.8714	0.7898	-9.37	–	45.99	0.8687	0.665	0.716	-2.96	–
KB-CoT	34.56	0.8453	0.8724	0.7856	-9.09	100.0	46.52	0.8728	0.691	0.743	-2.57	100.0
Self-CoT	33.51	0.8456	0.8692	0.7883	-8.89	–	46.03	0.8674	0.676	0.730	-2.88	–
Ladder	42.11	0.8592	0.9239	0.8384	-4.39	–	54.04	0.8890	0.754	0.822	-2.00	–
CSC	42.28	0.8611	0.9229	0.8406	-4.20	100.0	54.27	0.8902	0.767	0.829	-1.87	100.0

Appendix

Table 12: Results with Qwen3-8B. Ret. denotes retrieval rate (%).

	IdiomKB: Zh $\rightarrow$ En						RAGtrans: En $\rightarrow$ De
Top- $k$	BLEU	COMET	I5	I3	MQM	Ret.	BLEU	COMET	GRF	GRB	MQM	Ret.
$k=1$	24.62	0.822	0.731	0.728	-2.74	30.6	36.84	0.857	0.917	0.826	-5.98	43.3
$k=3$	24.68	0.817	0.729	0.721	-2.88	27.1	36.88	0.860	0.913	0.820	-6.08	40.1
$k=5$	23.11	0.804	0.713	0.708	-3.13	22.4	34.71	0.795	0.880	0.803	-9.44	31.7

Appendix

Table 13: Effect of the number of documents returned per retrieval action. Ret. denotes the percentage of examples with at least one retrieval call. Higher is better for all translation metrics, including MQM where values closer to zero are better.

Figure 4: Alternating translator and retriever optimization on RAGtrans. $\mathrm{T}_{1}/\mathrm{T}_{2}$ denote translator RL updates and $\mathrm{R}_{1}/\mathrm{R}_{2}$ denote contrastive retriever updates.

Method	Accuracy	Fluency	Style	Terminology	Non-translation	Total MQM
Direct	-1.73	-1.55	-0.92	-1.61	-0.01	-6.32
CSC	-1.30	-0.60	-0.86	-0.78	-0.02	-3.56
SRAT-A	-1.15	-0.58	-0.73	-0.31	0.00	-2.77
SRAT	-0.81	-0.39	-0.50	-0.24	0.00	-1.94

Appendix

Table 14: Human evaluation on 32 En

\rightarrow

Zh test examples from IdiomKB and RAGtrans. Scores are averaged across examples and annotators. Higher is better for all dimensions and Total MQM.

Retriever	BLEU	COMET	GRF	GRB	MQM	Ret.
DuckDuckGo	23.66	0.817	0.849	0.753	-6.64	88.6
Google	23.98	0.826	0.856	0.761	-6.53	91.3

Appendix

Table 15: Online retrieval backends on Document-MT-LLM En

\rightarrow

Ru. Ret. denotes retrieval rate (%).

Machine Translation

Machine translation develops computational methods for converting text, speech, and visual content between languages while preserving meaning and purpose.

See all 267 papers

Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation
Jun 6, 2026Boxuan Lyu, Haiyue Song, Zhi Qu +3RL for Language Models Machine Translation
Institute of Science Tokyo · Preferred Networks, Inc. · Nara Institute of Science and Technology
From prompting to evidence-based translation: A RAG+prompt system for Japanese-Chinese translation and its pedagogical potential
May 5, 2026Wenshi GuLLM Prompting Machine Translation
School of Foreign Languages, Beihang Univeristy.Beijing, China
Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning
Jun 5, 2026Pratik Jayarao, Chaitanya Dwivedi, Himanshu Gupta +5RL for Language Models LLM Tool Use
Amazon Stores Foundation AI

Jun 6, 2026J/K move · Enter open · S save

Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation

Boxuan Lyu, Haiyue Song, Zhi Qu, Hidetaka Kamigaito+2

Institute of Science Tokyo · Preferred Networks, Inc. · Nara Institute of Science and Technology

RL for Language Models Machine Translation

Open paper AskPDF arXiv

Rewriting source text with large language models (LLMs) before translation has been shown to improve machine translation (MT) quality. However, we find that prompt-based rewriting can degrade translation quality rather than improve it, particularly when smaller LLMs, such as 4B-parameter models, are used. We argue that this limitation stems from the difficulty of controlling rewriting behavior through natural-language prompts alone: a rewrite is useful only if it improves downstream translation, yet existing prompt-based methods do not explicitly optimize for this signal. To address this issue, we propose RLSR (Reinforcement Learning for Source Rewriting), a reinforcement learning framework that trains the rewriting model with a reward based on the downstream translation-quality improvement produced by each rewrite. Experiments across six MT systems and 16 language pairs show that our 4B RLSR-trained rewriting models significantly outperform both the no-rewriting baseline and prompt-based rewriting baselines at the same model scale, while remaining competitive with baselines that use a 235B LLM.

Abstract

Machine Translation

Explore similar work

Rewrite to Translate, Translate to Reward: Reinforcement Learning for Source Rewriting in Machine Translation

From prompting to evidence-based translation: A RAG+prompt system for Japanese-Chinese translation and its pedagogical potential

Translate-R1: Cost-Aware Translation Tool Use via Reinforcement Learning