Averaging Bias: Human Faithfulness Annotations are not Locally Faithful

Large language models are increasingly used to summarize clinical trial results for healthcare providers, patients, and payers, but their tendency to hallucinate poses significant risks in this high-stakes context. This study introduces a benchmark evaluation framework for measuring the faithfulness of LLM-generated clinical trial summaries across three stakeholder audiences. The framework consists of 200 stratified trials drawn from the Aggregate Analysis of ClinicalTrials.gov database, evaluated using audience-specific prompt templates and a six-dimension faithfulness annotation schema. Baseline measurements were established for GPT-4o, Claude Sonnet 4.6, and Gemini 2.5 Flash across 1,800 generated summaries scored using a cross-encoder natural language inference (NLI) model. Unsupported Claims was identified as the dominant failure mode across all three models, with a mean annotation score of 1.55 out of three. A knowledge-graph-augmented retrieval system was developed and evaluated against the baseline, producing statistically significant improvements in NLI-based faithfulness scores (entailment +0.0125, faithfulness +0.0130, p < 0.0001). Improvement pathways were model-dependent, with GPT-4o improving primarily through contradiction reduction while Claude Sonnet 4.6 and Gemini 2.5 Flash improved through increased entailment.

Chains of thought (CoTs) have become central in interpreting and auditing behaviors of large language models. Yet growing evidence suggests that these traces often fail to faithfully represent the computations behind a model's predictions. Several faithfulness metrics have been proposed, but whether they indeed measure faithfulness remains unknown. Answering this requires ground-truth labels, which are hard to obtain since internal computations are not directly observable. Consequently, most works proposing metrics report only absolute scores or comparisons to prior metrics, and the few existing benchmarks rely on proxies like plausibility or importance, properties orthogonal to faithfulness that can mislead about whether a CoT can be trusted. We address this challenge by constructing tasks whose outputs reveal which intermediate computations must have produced them, and developing an automated labeling pipeline that yields ground-truth faithfulness labels at both the step and CoT level. Building on this methodology, we present BonaFide, a benchmark of 3,066 labeled CoTs across 13 tasks and 10 models, and use it to conduct the first systematic evaluation of prominent faithfulness metrics. Our experiments show that most metrics perform near chance, exhibit strong prediction biases and degrade on longer CoTs. The best metric reaches only 0.70 AUROC at the CoT level while another reaches 0.59 at the step level, with neither transferring across settings, while entailing prohibitively high computational cost. Our results expose fundamental gaps in current faithfulness evaluation and call for the development of more reliable and efficient metrics.

Large language models produce fluent multi-document summaries, but their attributions are typically coarse---whole documents or passages---and generated post hoc, leaving each statement hard to verify. We argue that attribution should be a structural property of generation rather than a downstream prediction. We present CAMS, a Claim-Anchored Multi-document Summarization framework that decomposes every source document into atomic claims whose provenance is resolved deterministically from verbatim quotes to token spans, clusters equivalent claims across documents while flagging inter-source conflicts, selects a support-aware and salient subset, and rewrites it so that every summary sentence terminates in claim identifiers resolving back to source spans. This yields a separation we make explicit: provenance is an invariant holding for every emitted sentence independently of model accuracy, whereas faithfulness is an objective that selection, constrained rewriting, and verification only encourage---a distinction end-to-end and post-hoc systems conflate. We evaluate on MultiNews, DiverseSumm, and zero-shot on WCEP under a two-regime protocol separating reference-free citation quality from gold-aligned localization, audited by a support model never used for selection or verification. CAMSmatches strong end-to-end and span-attribution baselines on summary quality while improving faithfulness and citation precision, raising multi-source attribution accuracy from 38% to 64% without inflating the number of cited sources, and cutting human verification time per claim by $3.4\times$. We release code and ${\sim}320$K claim--quote--span annotations over MultiNews as a reusable fine-grained attribution resource.