Contextual Faithfulness

Follow this topic Ask about this topic

Contextual faithfulness studies whether a model’s answers and reasoning are supported by the documents or other evidence it receives, especially when that evidence conflicts with its learned knowledge. Approaches include retrieval-augmented generation, preference or reinforcement-learning objectives, and inference-time decoding or attention adjustments that favor context-supported claims. Evaluation ranges from claim-level support and coverage to pairwise tests of whether answers change with context sufficiency, as well as step-level reasoning and phrase-level alignment benchmarks. A central challenge is balancing supported answers with appropriate abstention: precise systems may omit relevant facts, while retrieval quality and limited source exposure can constrain both coverage and faithfulness.

Latest papers

11 papers2 papers in the last four weeks, with none the four weeks before. 0.0% of all new papers.

Aligning Faithfulness for Theory-Scale Autoformalization
Oct 13, 2026Yuming Feng, Tadd Mao, Dhruva Arekar +4Contextual Faithfulness Autoformalization
Johns Hopkins University · University of Toronto · University of Pennsylvania
Rethinking Faithfulness in LLMs: A Pairwise Context-Sensitive Perspective
Oct 6, 2026Zizhuo Zhang, Xiong Peng, Jingwei Sun +3Contextual Faithfulness LLM Evaluation
TMLR Group, Department of Computer Science, Hong Kong Baptist University · Huawei Noah’s Ark Lab.
On Improving Faithfulness of Podcasts from Documents
Jul 24, 2026Soumya Dutta, Tejas Indulal Dhamecha, Pannaga ShivaswamyContextual Faithfulness LLM Evaluation
1Indian Institute of Science · 2Adobe Research
On-Device Deep Research at 4B: Exposure Bounds Faithfulness, Retrieval Bounds Coverage
Jul 14, 2026Vinay Kumar ChagantiContextual Faithfulness Citation Verification
Independent Researcher
GRACE: Step-Level Benchmark for Faithful Reasoning over Context
Jun 15, 2026Hoang Pham, Dong Le, Anh Tuan LuuCoT Faithfulness Contextual Faithfulness
Nanyang Technological University · VinUniversity
Precision Is Not Faithfulness: Coverage-Aware Evaluation of Grounded Generation with a Complete Oracle
Jun 8, 2026Juan S. SantillanaContextual Faithfulness Factual Consistency Evaluation
Globant
Resonant Context Anchoring: Decoupling Attention Routing and Signal Gain at Inference Time
Jun 1, 2026Mingkuan Zhao, Yide Gao, Wentao Hu +6Contextual Faithfulness Hallucination in Language Models
Xi’an Jiaotong University · University of Science and Technology of China · Tongji University +1
The Need for an External Observer Formalizing the Sufficiency Gap: A Mathematical Extension of Mixture Identifiability and Contextual Grounding in Sequence Models
May 26, 2026Francesco CorielliContextual Faithfulness LLM Grounding
Investigating the Interplay between Contextual and Parametric Chain-of-Thought Faithfulness under Optimization
May 24, 2026Jingyi Sun, Qianli Wang, Pepa Atanasova +2CoT Faithfulness Contextual Faithfulness
University of Copenhagen · 2Technische Universität Berlin · 3German Research Center for Artificial Intelligence (DFKI) +2
Faithfulness-QA: A Counterfactual Entity Substitution Dataset for Training Context-Faithful RAG Models
Apr 28, 2026Li Ju, Junzhe Wang, Qi ZhangContextual Faithfulness Retrieval-Augmented Generation
College of Computer Science and Artificial Intelligence, Fudan University
Context-Fidelity Boosting: Enhancing Faithful Generation through Watermark-Inspired Decoding
Apr 24, 2026Weixu Zhang, Fanghua Ye, Qiang Gao +7Contextual Faithfulness Language Model Decoding
Hunyuan AI Digital Human, Tencent · McGill University · Mila - Quebec AI Institute +4

Oct 13, 2026J/K move · Enter open · S save

Aligning Faithfulness for Theory-Scale Autoformalization

Yuming Feng, Tadd Mao, Dhruva Arekar, Tianjun Zhong+3

Johns Hopkins University · University of Toronto · University of Pennsylvania

Contextual Faithfulness Autoformalization

Open paper AskPDF arXiv

Autoformalization (AF) requires more than compilation: formal statements must preserve the meaning of their informal sources, a property known as faithfulness. An emerging frontier is the automated evaluation of this property, yet most existing methods operate at the statement level and produce whole-item verdicts, failing to reveal how individual expressions correspond or which contextual interpretations support the judgment. This omission is consequential because AF is inherently theory-scale: target statements derive their meaning from surrounding mathematical material and unstated conventions. We formulate faithfulness evaluation as a context-aware align-then-label problem: identifying phrase-level correspondences between informal and formal statements and assigning a faithfulness label to each under explicitly specified contextual interpretations and assumptions. We also introduce FaithAlign, the first theory-scale faithfulness benchmark comprising 460 items and 4,982 annotated phrase-level links, alongside FaithAlign-Gap, a companion set of 146 items with informal context and assumptions withheld. Each FaithAlign item includes the target statement pair, relevant context, formal dependencies, and, where needed, assumptions specifying one given interpretation. Across 8 models, GPT-6 Astra achieves the highest defect F1 at 47.8% and identifies all defects in only 4.5% of items with two or more defects. On FaithAlign-Gap, Astra identifies only 28.2% of undecidable items as such. These results show the difficulty of recovering context-dependent correspondences, establishing FaithAlign as a testbed for fine-grained, context-aware faithfulness evaluation.