cs.CLApr 11, 2026
SaveFAITH: Factuality Alignment through Integrating Trustworthiness and Honestness
Organizations: Tsinghua University, Institute for Interdisciplinary Information Sciences · Shanghai Qi Zhi Institute · South China Normal University, School of Electronic Science and Engineering · Guangzhou Richstone Data Technologies Co., Ltd. ·
Abstract
Large Language Models (LLMs) can generate factually inaccurate content even if they have corresponding knowledge, which critically undermines their reliability. Existing approaches attempt to mitigate this by incorporating uncertainty in QA prompt during training, but these numerical scores lack the semantic richness for LLM to properly understand its internal states of trustworthiness and honestness, leading to insufficient factuality alignment. We introduce FAITH (Factuality Alignment through Integrating Trustworthiness and Honestness), a post-training framework for factuality alignment that integrates natural-language uncertainty signals with external knowledge. Specifically, we augment training datasets by computing confidence scores and semantic entropy from LLM outputs and mapping them into a knowledge state quadrant that describes the model's internal knowledge possession (trustworthiness) and answering behaviors (honestness) in natural language. Based on this enhanced data, we design a reward function that considers both correctness and uncertainty signals, and fine-tune the LLM using the Proximal Policy Optimization (PPO) algorithm. To further mitigate weakly grounded responses, we design a retrieval-augmented module that retrieves relevant external passages, improving the consistency between internal and external knowledge representations. Extensive experiments on four knowledge-intensive benchmarks demonstrate that FAITH enhances the factual accuracy and truthfulness of LLMs.
Figures & tables
| Abbr. | Explanation |
| KC | Known and answered correctly |
| KI | Known but answered incorrectly |
| KR | Known but refused to answer |
| UC | Unknown but answered correctly |
| UI | Unknown but answered incorrectly |
| UR | Unknown and refused to answer |
Table 1: Categories of model output.
Figure 1: Illustration of the FAITH framework. Panel (a) shows the procedure for augmenting the training datasets, Panel (b) depicts model training with the augmented datasets, and Panel (c) presents the inference pipeline.
| Method | TVQA (ID) | SciQ (ID) | NQ-Open (ID) | Average (ID) | WebQ-QA (OOD) | |||||
| Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | |
| Llama3-8B | ||||||||||
| ICL-CoT | 66.68 | 53.37 | 72.34 | 45.90 | 57.34 | 23.60 | 65.45 | 40.95 | 65.97 | 30.85 |
| SFT | 70.80 | 52.57 | 72.18 | 45.40 | 41.41 | 16.57 | 61.46 | 38.18 | 66.46 | 31.18 |
| SFT rag | 71.49 | 53.04 | 73.23 | 45.97 | 42.64 | 17.58 | 62.45 | 38.86 | 66.93 | 31.85 |
| RL-DPO | 72.08 | 53.96 | 71.23 | 44.20 | 49.65 | 19.18 | 64.32 | 39.11 | 65.99 | 32.41 |
Table 2: Precision and Truthfulness of FAITH (ours) vs. strong baselines on in-domain (ID) and out-of-domain (OOD) QA datasets. The Average (ID) column denotes the average performance on all three ID datasets. The subscript “sft” denotes ablation results with only supervised fine-tuning (SFT), excluding the PPO and RAG (if applicable) module. Similarly, “sft+ppo” denotes results with SFT and PPO, but excluding the RAG module. All results are reported in percentages.
Figure 2: Ratios of on four datasets.
| Method | TVQA (ID) | SciQ (ID) | NQ-Open (ID) | Average (ID) | WebQ-QA (OOD) | |||||
| Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | |
| Llama3-8B | ||||||||||
| Estimator | 82.95 | 59.80 | 80.29 | 49.70 | 57.99 | 26.52 | 73.79 | 45.69 | 67.31 | 33.75 |
| Sample-based | 83.99 | 59.86 | 83.20 | 51.50 | 58.23 | 26.93 | 75.14 | 46.10 | 67.85 | 34.07 |
| Mistral-7B-v0.1 | ||||||||||
| Estimator | 87.00 | 60.58 | 83.68 | 51.80 | 46.60 | 23.19 | 72.43 | 45.19 | 55.48 | 38.65 |
Table 3: Performance comparison between model-based and sampling-based knowledge state estimation ( ). Results are reported on precision and truthfulness across all datasets and models.
Figure 3: Training-time scaling with different numbers of sampled responses ( ) on Llama3-8B.
Appendix figures & tables6 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 4: All the prompt templates employed in FAITH (our framework).
| # Responses | TVQA (ID) | SciQ (ID) | NQ-Open (ID) | Average (ID) | WebQ-QA (OOD) | |||||
| Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | |
| Llama-3-8B | ||||||||||
| K=6 | 82.95 | 59.80 | 80.29 | 49.70 | 57.99 | 26.52 | 73.79 | 45.69 | 67.31 | 33.75 |
| K=8 | 84.36 | 63.55 | 81.52 | 54.70 | 60.95 | 25.51 | 75.61 | 47.92 | 67.42 | 37.76 |
| K=10 | 84.01 | 63.79 | 80.94 | 55.20 | 60.44 | 25.98 | 75.13 | 48.32 | 67.10 | 38.43 |
| K=12 | 84.36 | 64.65 | 80.76 | 55.40 | 60.35 | 26.40 | 75.16 | 48.82 | 66.15 | 38.28 |
Table 4: Training-time scaling with different numbers of sampled responses ( ) on Llama-3-8B.
| Method | TVQA (ID) | SciQ (ID) | NQ-Open (ID) | Average (ID) | WebQ-QA (OOD) | |||||
| Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | Prec. | Truth. | |
| Llama3-8B | ||||||||||
| FAITH sample-base | 85.02 | 60.98 | 81.43 | 50.33 | 59.25 | 28.02 | 75.23 | 46.44 | 68.67 | 34.89 |
| FAITH Estimator | 84.19 | 60.69 | 80.61 | 49.99 | 58.13 | 27.58 | 74.26 | 45.73 | 67.99 | 34.03 |
| Mistral-7B-v0.1 | ||||||||||
| FAITH sample-base | 88.12 | 61.43 | 82.53 | 52.23 | 48.97 | 24.59 | 73.21 | 46.08 | 58.87 | 41.23 |
Table 5: Performance Comparison of FAITH_sample-base and FAITH_Estimator.
| Type | Question & Answer | Retrieved Passages | Our Analysis |
| Implicitly Supported Correction | Q: Protists play critically important ecological roles as producers and, on the other end of food webs, as what? A: decomposers A1: consumers A2: decomposers | I. In real world ecosystems, there is more than one food chain for most organisms, since most organisms eat more than one kind of food or are eaten by more than one type of predator. A diagram that sets out the intricate network of intersecting and overlapping food chains for an ecosystem is called its food web. Decomposers are often left off food webs, but if included, they mark the end of a food chain. Thus, food chains start with primary producers and end with decay and decomposers. II. Food webs have trophic levels and positions. Basal species, such as plants, form the first level and are the resource-limited species that feed on no other living creature in the web. Basal species can be autotrophs or detritivores, including decomposing organic material and its associated microorganisms, which we defined as detritus, micro-inorganic material, and associated microorganisms (MIP), and vascular plant material. III. The microbial food web refers to the combined trophic interactions among microbes in aquatic environments. These microbes include viruses, bacteria, algae, and heterotrophic protists (such as ciliates and flagellates). In aquatic environments, microbes constitute the base of the food web. Single-celled photosynthetic organisms such as diatoms and cyanobacteria are generally the most important primary producers in the open ocean. Many of these cells, especially cyanobacteria, are too small to be captured and consumed by small crustaceans and planktonic larvae. Instead, these cells are consumed by phagotrophic protists, which are readily consumed by larger organisms. | Some protists do function as A1 (consumers), but the model fails to accurately address the specific context of “the other end of food webs” posed in the question. This suggests the model’s insufficiency in effectively utilizing its internal knowledge to answer the question, or a failure to correctly understand the question, particularly its key constraints. After introducing the retrieved information to our trained RAG model, it successfully corrected the answer to A2 (decomposers). The retrieved information, in paragraph I, provides the crucial background knowledge: “Thus food chains start with primary producers and end with decay and decomposers.” This information does not explicitly state that “protists are decomposers.” Instead, it requires the model to synthesize this information with “protists” and “the other end of food webs” to deduce the correct answer. We denote this process as Implicitly Supported Correction . |
Table 6: Case Study Analysis 1 of Model Responses. Specifically, A denotes Ground Truth, A1 denotes the policy model’s output, and A2 denotes the final output.
| Type | Question & Answer | Retrieved Passages | Our Analysis |
| Explicitly Supported Correction | Q: Rita Coolidge sang the title song for which Bond film? A: Octopussy A1: North by Northwest A2: Octopussy | I. Octopussy is the soundtrack for the eponymous thirteenth James Bond film. The score was composed by John Barry, the lyrics by Tim Rice. The opening theme, All Time High is sung by Rita Coolidge and is one of six Bond film title songs or songs that are not named after the film’s title. The original compact disc released in 1985 by A&M Records, was recalled because of a printing error and became a rarity. II. Another Way to Die is a song by American musicians Jack White and Alicia Keys. Written and produced by White as the theme song to the 2008 James Bond film Quantum of Solace, it was released as a single in the United States on September 30, 2008, and in Europe on October 20, 2008. III. Tomorrow Never Dies is the song, performed by Sheryl Crow, which served as the theme song to the James Bond film of the same name. The song was co-written by Crow and the song’s producer Mitchell Froom, and became her fifth UK Top 20 hit, peaking at No. 12 in 1997. Another song, Tomorrow Never Dies, written by the movie’s composer David Arnold and performed by k.d. lang, was originally produced as the official theme tune. When Crow’s song became the official theme, the k.d. lang song was relegated to the end credits, and renamed Surrender. | The model’s initial response, A1 (North by Northwest), is a significant factual error, as this film is not even part of the James Bond series. This indicates a substantial knowledge gap or a “hallucination” in the model’s internal knowledge base. After the RAG intervention, the model successfully corrected the answer to A2 (Octopussy). The retrieved information in paragraph I contains all the key details required to rectify the error. The passage explicitly states, “The opening theme, All Time High is sung by Rita Coolidge” and that the soundtrack was for the film Octopussy. The model simply needed to match the key entity from the question, “Rita Coolidge”, with the retrieved text to directly find the name of the film for which she sang the theme. The entire process involves direct information extraction and localization, requiring almost no complex reasoning. We denote this process as Explicitly Supported Correction . |
Table 7: Case Study Analysis 2 of Model Responses. Specifically, A denotes Ground Truth, A1 denotes the policy model’s output, and A2 denotes the final output.
| Type | Question & Answer | Retrieved Passages | Our Analysis |
| Misleading Override | Q: Which grand slam did Pete Sampras not win in the 20th century? A: French A1: French Open A2: Wimbledon | I. As the Swiss national anthem played Federer was overcome with emotion after finally capturing the elusive title at Roland Garros. This match was momentous in the history of tennis. After missing the chance to equal Pete Sampras’ then-record of fourteen Grand Slam championships of all time when he lost to Rafael Nadal in the final of the Australian Open earlier in the year, Federer finally did so by winning the French Open for the first time. Sampras himself commented on Federer following the victory saying, Regardless he [Federer] goes down as the greatest ever. II. In the eight Wimbledons inclusive between 1993 and 2000, 1996 was the only year that Sampras would fail to win the championship at Wimbledon. Sampras lost in the quarterfinals of Wimbledon to the eventual winner, Richard Krajicek, the tournament’s 17th-seed. The match lasted three long sets, with Krajicek winning 7-5, 7-6, 6-4. In the quarterfinals of the US Open, Sampras vomited on the court at 1–1 in the final set tiebreak (due to dehydration) while facing Àlex Corretja; nonetheless, Sampras would win that match. III. He beat former champion Michael Stich in the fourth round and met Sampras in the quarterfinals. By that time, he had managed to turn his notably weak slice backhand into an aggressive top-spin shot. Krajicek shocked the tennis world by defeating Sampras in straight sets, becoming the only player to beat Sampras in a Wimbledon singles match in the eight-year period from 1993 until Sampras’ fourth-round loss to Roger Federer in the 2001 tournament. | This is a failure case of a ’correct-to-incorrect’ reversal caused by the RAG. The model’s initial judgment, A1 (French Open), was correct, indicating that its internal knowledge base already contained the key fact about Sampras’s career. However, the intervention of RAG instead led to a degradation in performance. The core of the failure lies in the Retrieval stage. The retrieved information, though related to the key entities “Pete Sampras” and “Grand Slam“, did not align with the question’s specific requirement (“did not win” in the 20th century). The retrieved content, particularly in paragraphs II and III, repeatedly and in detail described a specific loss Sampras had at Wimbledon (in 1996 to Krajicek). Phrases like “fail to win the championship at Wimbledon” became a strong and irrelevant distracting signal. When generating the final answer, the model over-relied on this incorrectly retrieved and distracting content, thereby ignoring its own correct prior knowledge. It was misled into outputting the incorrect answer A2 (Wimbledon). We denite this process as Misleading Override . |
Table 8: Case Study Analysis 3 of Model Responses. Specifically, A denotes Ground Truth, A1 denotes the policy model’s output, and A2 denotes the final output.
More in this topic
LLM Alignment
Large language model alignment studies methods for making models follow human intentions, values, and safety constraints across contexts.
See all 611 papers