Reasoning traces improve large language models (LLMs), but current models are trained to reason mostly in English. It has been shown that forcing a model to reason in another language degrades accuracy, even when the reasoning language matches the language of the prompt -- but only for a setting where the model reasons over a short prompt. Here, we ask whether the same holds for retrieval-augmented generation (RAG), where the model must read and integrate a large amount of retrieved evidence in the target language. To study this, we build a fully monolingual German RAG question-answering testbed over the fictional world of the tabletop role-playing game The Dark Eye, a domain that is richly documented in German but too niche for the model to answer from memory, so that it has to rely on retrieval. Varying the forced reasoning language of an agentic RAG system on this testbed, we find that aligning the reasoning language with the language of the query and the retrieved documents helps. Forced German reasoning outperforms forced French, although the model benchmarks higher in French, so the benefit comes from alignment and not from language proficiency. The advantage grows when the retrieved context is richer and structure-aware. However, forced German only reaches the level of the model's native, unconstrained English reasoning without surpassing it, showing that native multilingual reasoning is needed. We publicly release the testbed and QA benchmark.
Figures & tables
Figure 1: Our setting. An agentic RAG system answers German questions using retrieved German sourcebook passages; we control the language the model reasons in. In this real example, English reasoning drops the umlaut of the river Südask , while German reasoning keeps it.
Figure 3: Architecture of our agentic RAG system. Given a German query, the LLM reasons and then either issues a retrieval call or produces the final answer. Retrieval embeds the query, runs similarity search over the hierarchical German knowledge base, and reranks the hits to the top-3 chunks, which are appended to the context; the loop repeats until the model answers, always in German. The one control we vary is the language the model is forced to reason in (German, English, French, or none).
SectionChunks
SimpleChunks
Book
C
TC
C~
TC~
G01
738
499
736
501
G05
665
549
730
501
G08
843
466
784
501
Total
2246
501
2250
501
Table 1: Knowledge-base statistics. C is the number of chunks and TC the mean chunk size in tokens. SimpleChunks uses a fixed chunk size TC~ equal to the mean SectionChunks chunk size, so both hold the same 1.13M tokens in total at comparable granularity.
Reasoning
1
2
3
4
5
Avg.
NoRAG (no retrieval)
small model
478
49
40
5
13
1.335
large model
490
36
36
11
12
1.323
SimpleChunks (fixed-size)
German
115
36
130
103
201
3.409
English
111
45
118
82
229
3.467
Table 2: Single-passage set (585 questions), LLM-as-a-Judge ratings (1–5). Columns 1–5 give how many outputs received each rating; Avg. is the mean. All questions and answers are in German; Reasoning is the language the model is forced to reason in. Pairwise significance tests are in Appendix D .
Figure 4: Average ratings on the single-passage set. Retrieval (orange, green) is far above no-retrieval (blue). Among reasoning settings on the structure-aware SectionChunks base (green), forced German rises above forced French and reaches forced and unconstrained English, while disabling reasoning is highest.
Figure 5: Full rating distribution per configuration on the single-passage set (Table 2 as shares). Colour encodes the judge rating from 1 (orange, hallucinated) to 5 (blue, correct). Retrieval moves the mass from rating 1 toward rating 5; for the retrieval settings rating 2 is the rarest.
Reasoning
RA
RB
RC
Avg.
NoRAG (large)
1.20
1.13
1.23
1.189
no reasoning
2.87
2.90
2.57
2.778
French
2.87
2.70
2.77
2.778
German
2.87
3.03
2.77
2.889
English
3.30
2.83
2.87
3.000
unconstrained
3.13
3.17
2.73
3.011
Table 3: Multi-hop set (30 questions), human ratings (1–5) from three expert reviewers RA,RB,RC . All use the SectionChunks knowledge base.
Human rating
Judge
1
2
3
4
5
1
12
1
2
2
0
2
0
1
0
0
0
3
0
1
4
4
0
4
0
1
0
2
2
5
0
0
0
4
14
Table 4: LLM-as-a-Judge vs. human ratings on 50 sampled answers (five per configuration). Exact agreement ( diagonal ) is 66%; the larger disagreements sit in the top row, where the judge assigns 1 to an answer the human rates 3–4.
Figure 6: Mean answer length by judge rating, pooled over the eight retrieval configurations on the single-passage set (4,680 graded answers). Higher-rated answers are shorter (Spearman ρ=−0.39 , p<0.001 ); colour encodes the rating. This is direct evidence for the judge’s length bias.
Appendix figures & tables1 asset
Supplementary material from the paper’s appendix.
Appendix
Comparison
Δ
W/T/L
p
pHolm
no reason. > French
0.41
214/263/108
8.2⋅10−9
8.2⋅10−8 ***
no reason. > English
0.23
195/265/125
.0008
.007 **
unconstr. > French
0.23
193/254/138
.0013
.010 *
German > French
0.21
182/265/138
.0027
.019 *
no reason. > German
0.20
190/270/125
.0033
.020 *
no reason. > unconstr.
0.19
185/273/127
.0080
.040 *
Appendix
Table 5: Pairwise Wilcoxon signed-rank tests on the single-passage set (585 paired ratings), for the five SectionChunks reasoning settings. Δ is the difference in mean rating; W/T/L counts questions where the first setting wins, ties, or loses; pHolm is corrected across the ten pairs. *** p<.001 , ** p<.01 , * p<.05 .
Multilingual Retrieval-Augmented Generation (mRAG) leverages cross-lingual evidence to ground Large Language Models (LLMs) in global knowledge. However, we show that current mRAG systems suffer from a language bias during reranking, systematically favoring English and the query's native language. By introducing an estimated oracle evidence analysis, we quantify a substantial performance gap between existing rerankers and the achievable upper bound. Further analysis reveals a critical distributional mismatch: while optimal predictions require evidence scattered across multiple languages, current systems systematically suppress such ``answer-critical'' documents, thereby limiting downstream generation performance. To bridge this gap, we propose Language-Agnostic Utility-driven Reranker Alignment (LAURA), which aligns multilingual evidence ranking with downstream generative utility. Experiments across diverse languages and generation models show that LAURA effectively mitigates language bias and consistently improves mRAG performance.
Dan Wang, Guozhao Mo, Yafei Shi +9
Chinese Information Processing Laboratory, Institute of Software, Chinese Academy of Sciences · University of Chinese Academy of Sciences · MYbank, AntGroup
Retrieval-augmented generation (RAG) has become a standard mechanism for grounding language models in external knowledge, yet conventional retrieval based on lexical or semantic similarity is poorly suited for complex reasoning tasks: a semantically similar problem may demand an entirely different solution strategy, while a superficially different problem may share the same underlying reasoning pattern. We propose Retrieval-Augmented Reinforcement Fine-Tuning (RA-RFT), a post-training framework that teaches language models to reason by analogy. RA-RFT uses gold-relevance distillation to train a retriever that ranks contexts by expected reasoning benefit rather than semantic overlap, and then fine-tunes the policy model via reinforcement fine-tuning methods with retrieved analogous demonstrations, so the model learns to leverage reasoning traces under verifiable outcome rewards. We further analyze the diversity of retrieved contexts and find that reasoning-aware retrieval surfaces complementary solution strategies that provide distinct reasoning scaffolds for individual problems. Across challenging mathematical reasoning benchmarks, RA-RFT consistently outperforms standard reinforcement fine-tuning methods. For example, it improves AIME 2025 average@32 accuracy by 7.1 and 2.8 points over GRPO for Qwen3-1.7B and Qwen3-4B respectively -- suggesting that reasoning-aware retrieval is a complementary axis of improvement and orthogonal to advances in reward design or training curricula.
Zilin Xiao, Qi Ma, Chun-cheng Jason Chen +4
Meta Superintelligence Labs · Rice University · Work partially done during internship at Meta
Large reasoning models such as DeepSeek-R1 and OpenAI o1 generate extended chains of thought spanning thousands of tokens, yet their integration with retrieval-augmented generation (RAG) remains fundamentally misaligned. Current RAG systems optimize for providing context before reasoning begins, while reasoning models require evidence injection during multi-step inference chains. We introduce ReaLM-Retrieve, a reasoning-aware retrieval framework that addresses this mismatch through three key innovations: (1) a step-level uncertainty detector that identifies knowledge gaps at reasoning-step granularity rather than token or sentence level; (2) a retrieval intervention policy that learns when external evidence maximally benefits ongoing reasoning; and (3) an efficiency-optimized integration mechanism that reduces per-retrieval overhead by 3.2x compared to naive integration. Experiments on MuSiQue, HotpotQA, and 2WikiMultiHopQA demonstrate that ReaLM-Retrieve achieves on average 10.1% absolute improvement in answer F1 over standard RAG (range: 9.0-11.8% across the three benchmarks) while reducing retrieval calls by 47% compared to fixed-interval approaches like IRCoT (all improvements significant at p<0.01, paired bootstrap). On the challenging MuSiQue benchmark requiring 2-4 hop reasoning, our method achieves 71.2% F1 with an average of only 1.8 retrieval calls per question. Analysis shows that ReaLM-Retrieve also improves retrieval quality itself, achieving 81.3% Recall@5 with consistently higher precision and MRR than fixed-interval baselines on supporting evidence, establishing new state-of-the-art efficiency-accuracy trade-offs for reasoning-intensive retrieval tasks.
Dongxin Guo, Jikun Wu, Siu Ming Yiu
The University of Hong Kong Hong Kong, China · Brain Investing Limited Hong Kong, China · Stellaris AI Limited Hong Kong, China