Organizations: School of Computing, Kyung Hee University, Yongin-si, South Korea · School of Computer Science and Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu, China · School of Robotics and Advanced Manufacture, Harbin Institute of Technology, Shenzhen, China · School of Computer Science and Technology, Tongji University, Shanghai, China
Retrieval-Augmented Generation (RAG) mitigates hallucination in large language models (LLMs) by incorporating external knowledge during generation. However, the effectiveness of RAG depends not only on the design of the retriever and the capacity of the underlying model, but also on how retrieved evidence is structured and aligned with the query. Existing RAG approaches typically retrieve and concatenate unstructured text fragments as context, which often introduces redundant or weakly relevant information. This practice leads to excessive context accumulation, reduced semantic alignment, and fragmented reasoning chains, thereby degrading generation quality while increasing token consumption. To address these challenges, we propose Tri-RAG, a structured triplet-based retrieval framework that improves retrieval efficiency through reasoning-aligned context construction. Tri-RAG automatically transforms external knowledge from natural language into standardized structured triplets consisting of Condition, Proof, and Conclusion, explicitly capturing logical relations among knowledge fragments using lightweight prompt-based adaptation with frozen model parameters. Building on this representation, the triplet head Condition is treated as an explicit semantic anchor for retrieval and matching, enabling precise identification of query-relevant knowledge units without directly concatenating lengthy raw texts. As a result, Tri-RAG achieves a favorable balance between retrieval accuracy and context token efficiency. Experimental results across multiple benchmark datasets demonstrate that Tri-RAG significantly improves retrieval quality and reasoning efficiency, while producing more stable generation behavior and more efficient resource utilization in complex reasoning scenarios.
Figures & tables
Fig. 1: Triplet Matching for Efficient Reasoning. The input query is mapped to a condition representation that is matched against triplet heads in the knowledge base. The matched triplet (Condition, Proof, Conclusion) is returned to support structured reasoning with reduced unnecessary context expansion.
Fig. 2: Soft Prompt–Driven Triplet Structuring and Retrieval-Augmented Inference. The upper part shows how soft prompts guide frozen LLMs to extract structured triplets (Condition, Proof, Conclusion). The lower part uses triplet retrieval to enhance LLM inference via relevance ranking and head matching.
Model
Method
LongBench
2WikiMultihopQA
MuSiQue
NQ
SQuAD
AVG
GPT-4o-mini
Direct
58.1
48.9
40.1
36.4
51.2
46.9
ReAct
62.7
52.3
53.6
48.9
55.3
54.6
Self-Act
68.9
69.2
67.3
62.4
69.5
67.5
Aflow
61.0
65.7
71.6
67.9
73.7
68.0
Tri-RAG
70.9
73.3
74.6
69.8
75.3
72.8
GPT-4.1-nano
Direct
56.3
45.3
38.2
33.7
49.3
44.6
TABLE I: Performance of search strategies (Direct, ReAct, Self-Act, Aflow, and Tri-RAG) across five QA benchmarks. All metrics are reported as F1 scores (%).
Methods
Hotpot-Dist
Hotpot-Full
Shuffle-Hotpot-Dist
Shuffle-Hotpot-Full
F1
Precision
Recall
F1
Precision
Recall
F1
Precision
Recall
F1
Precision
Recall
Response Quality
LLM-only
23.7
25.9
23.4
23.7
25.9
23.4
15.8
17.5
15.8
15.8
17.5
15.8
Semantic RAG
61.7
64.6
64.3
52.8
55.8
53.5
50.8
53.3
52.4
42.2
44.9
43.3
+ Rerank
65.2
68.5
66.5
58.7
61.3
60.3
53.2
56.0
54.6
44.7
47.6
45.6
Hybrid RAG
65.3
67.6
65.5
55.1
58.2
55.8
52.0
54.8
53.4
44.3
47.3
44.6
TABLE II: Performance of different methods on HotpotQA datasets and their respective response and retrieval qualities.
Method
Knowledge construction
Inference
Input tokens
Output tokens
LLM calls
Time
Retrieval
Generation
LightRAG
1,269
381
1
3 s
40 ms
5,600 ms
GraphRAG
2,791
629
5
6 s
42 ms
5,500 ms
KG 2 RAG
561
22
1
1 s
25 ms
2,300 ms
Tri-RAG (Ours)
640
96
1
1 s
18 ms
1,900 ms
TABLE III: Comparison of system cost for structured knowledge construction and query-time inference.
Method
DeepSeek-7B
Llama-3-8B
Qwen3-8B
Avg.
Hotpot
2Wiki
MuSiQue
Hotpot
2Wiki
MuSiQue
Hotpot
2Wiki
MuSiQue
Pretrained
49.8
46.5
44.1
52.1
48.7
46.0
53.5
50.2
48.4
48.8
Prefix Tuning
56.7
52.8
50.3
58.9
55.1
52.2
60.1
58.4
55.6
55.6
LoRA
63.9
60.4
57.1
66.2
62.9
59.5
68.4
67.1
64.8
63.4
Tri-RAG (SP)
66.8
63.7
60.2
69.0
73.3
74.6
72.5
71.8
70.4
69.6
TABLE IV: End-to-end F1 (%) on multi-hop QA benchmarks for different extractor backbones under parameter-efficient tuning methods.
Metric
Prefix Tuning
LoRA
Tri-RAG
Quality Metrics
Logical Faithfulness
4.12 ± 0.35
4.41 ± 0.12
4.48 ± 0.18
Structural Integrity
3.75 ± 0.82
4.75 ± 0.10
4.82 ± 0.05
Condition Alignment
3.98 ± 0.44
4.51 ± 0.23
4.55 ± 0.21
Efficiency Metrics
Params
4.3 M
43.1 M
3.7 M
TABLE V: Triplet extraction quality (1–5, mean ± std) and efficiency (trainable parameters) for Prefix Tuning, LoRA, and Tri-RAG.
Fig. 3: Retrieval-key ablation comparing All retrieval using the concatenation [Tc;Tp;Tr] and single-field retrieval using Tc , Tp , or Tr . We report Hit@1 together with index-level retrieval latency and memory footprint.
Retrieval-Augmented Generation (RAG) has become a standard approach for enhancing large language models (LLMs) with external knowledge, mitigating hallucinations, and improving factuality. However, existing systems rely on generating natural language queries at each hop and maintaining a strict architectural separation between retriever and generator, preventing them from leveraging the full representational capacity of the LLM. We propose LAnR (Latent Abstraction for RAG), a unified framework in which a single LLM jointly performs encoding, retrieval, and generation entirely within its own latent space. Rather than generating textual queries, LAnR produces dense retrieval vectors from the hidden states of a designated [PRED] token and uses them to match against encoded document representations from the same model. Furthermore, LAnR adaptively decides when sufficient evidence has been retrieved using a lightweight MLP control head over those same hidden states, eliminating both the separate retriever and explicit token-level stopping reasoning. This design is motivated by our empirical observation that answer token entropy reliably signals retrieval sufficiency. Extensive experiments on six QA benchmarks spanning single-hop and multi-hop settings demonstrate that LAnR outperforms existing RAG methods, while achieving improved inference efficiency through reduced number of retrieval calls and tighter model integration.
Retrieval-Augmented Generation (RAG) significantly enhances Large Language Models (LLMs) by providing access to external knowledge. However, current research primarily focuses on retrieval quality, often overlooking the critical ''integration bottleneck'': even when relevant documents are retrieved, LLMs frequently fail to utilize them effectively due to conflicts with their internal parametric knowledge. In this paper, we argue that implicitly resolving this conflict in a single generation pass is suboptimal. We introduce GuarantRAG, a framework that explicitly decouples reasoning from evidence integration. First, we generate an ''Inner-Answer'' based solely on parametric knowledge to capture the model's reasoning flow. Second, to guarantee faithful evidence extraction, we generate a ''Refer-Answer'' using a novel Contrastive DPO objective. This objective treats the parametric Inner-Answer as a negative constraint and the retrieved documents as positive ground truth, forcing the model to suppress internal hallucinations in favor of external evidence during this phase. Finally, rather than naive concatenation or using the DPO trained model directly, we propose a joint decoding mechanism that dynamically fuses the logical coherence of the Inner-Answer with the factual precision of the Refer-Answer at the token level. Experiments on five QA benchmarks demonstrate that GuarantRAG improves accuracy by up to 12.1% and reduces hallucinations by 16.3% compared to standard and dynamic RAG baselines.
Zhengyi Zhao, Shubo Zhang, Zezhong Wang +7
The Chinese University of Hong Kong · Ministry of Education Key Laboratory of High Confidence Software Technologies, CUHK · University of International Relations +2
Retrieval-Augmented Generation (RAG) mitigates knowledge obsolescence and factual hallucination in large language models by introducing external context. However, when retrieved knowledge conflicts with the model's internal parametric knowledge, the model may either blindly follow misleading context or incorrectly rely on parametric knowledge, leading to unreliable responses. To address this issue, this paper proposes TRACE (Debate-TRace and Answer-Completeness rEgularized fine-tuning), a robust fine-tuning framework for RAG under knowledge conflicts. First, we propose a fine-tuning method that leverages multi-agent debate traces to extract correct candidates, incorrect candidates, and answer-shift patterns, providing fine-grained supervision for reliable knowledge-source selection. In addition, we design an answer completeness regularization mechanism to alleviate empty, overly short, and prematurely terminated responses via answer-tail token reinforcement and premature termination suppression. The fine-tuning objective combines correct-answer supervision, incorrect-candidate suppression, answer-tail token reinforcement, and premature termination suppression, enabling the model to use reliable external context, resist misleading or irrelevant retrieved content, and fall back to parametric knowledge when retrieved evidence is unreliable. Experiments across multiple knowledge-conflict scenarios and datasets show that TRACE improves robustness against misleading retrieved knowledge and reduces incomplete answers. These results demonstrate that multi-agent debate traces and answer completeness regularization jointly enhance knowledge-source selection, conflict robustness, and answer quality in RAG models. Our code is available at https://github.com/PHD-lanyu/TRACE.
Zhengchen Huang, Yundong Sun, Minrui Song +4
School of Information Science & Engineering, LinYi University, China · School of Computer Science and Technology, Harbin Institute of Technology, China