Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing

Wenhao Yuan; Chenchen Lin; Jian Chen; Jinfeng Xu; Xuehe Wang; Edith Cheuk Han Ngai

cs.AIApr 9, 2026

Verify Before You Commit: Towards Faithful Reasoning in LLM Agents via Self-Auditing

Authors: Wenhao Yuan, Chenchen Lin, Jian Chen, Jinfeng Xu, Xuehe Wang, Edith Cheuk Han Ngai

Organizations: The University of Hong Kong · Sun Yat-sen University

Abstract

In large language model (LLM) agents, reasoning trajectories are treated as reliable internal beliefs for guiding actions and updating memory. However, coherent reasoning can still violate logical or evidential constraints, allowing unsupported beliefs repeatedly stored and propagated across decision steps, leading to systematic behavioral drift in long-horizon agentic systems. Most existing strategies rely on the consensus mechanism, conflating agreement with faithfulness. In this paper, inspired by the vulnerability of unfaithful intermediate reasoning trajectories, we propose Self-Audited Verified Reasoning (SAVeR), a novel framework that enforces verification over internal belief states within the agent before action commitment, achieving faithful reasoning. Concretely, we structurally generate persona-based diverse candidate beliefs for selection under a faithfulness-relevant structure space. To achieve reasoning faithfulness, we perform adversarial auditing to localize violations and repair through constraint-guided minimal interventions under verifiable acceptance criteria. Extensive experiments on six benchmark datasets demonstrate that our approach consistently improves reasoning faithfulness while preserving competitive end-task performance.

Figures & tables

Figure 1: Demonstration of unfaithful agent reasoning. The agent outputs the correct answer ‘ Animorphs ’, but its multi-step reasoning process is logically invalid, as an unverified intermediate assumption “ The phrase $\ldots$ reminds me of The Hork-Bajir Chronicles " is used to derive the conclusion that it already presupposes. This failure mode differs fundamentally from unfaithful CoT, where the reasoning is merely an explanatory artifact, while unfaithful reasoning in the agent determines the following behavior and final decision.

Figure 2: An overview of our proposed SAVeR framework. This diagram illustrates the overall closed-loop workflow, highlighting the end-to-end process from belief generation and selection to auditing and iterative repair.

Models	Methods	HotpotQA		2WikiMHQA		MuSiQue		NQ		Quoref		FEVER
		EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$	F1 $\uparrow$	EM $\uparrow$
LLaMA-3.1-8B	Vanilla LM	34.8	43.6	39.6	47.3	23.8	34.6	29.5	38.2	29.4	37.7	53.7
LLaMA-3.1-8B	CoT	38.3	47.5	44.2	50.7	27.1	36.8	32.7	43.4	33.2	42.3	55.9
	MAD	43.1	51.2	47.9	55.4	30.9	40.8	36.6	46.9	36.3	45.2	60.7
	Self-Refine	40.8	48.9	46.3	53.3	28.7	37.8	33.5	43.0	34.1	43.6	57.6
	B -2	42.9	51.3	46.7	54.4	31.0	40.6	35.9	44.9	36.7	46.2	60.6

Table 1: The overall evaluation results of SAVeR and other baseline methods on six benchmarks. The best-performed method is marked by bold and the runner-up performing method is marked by underline .

Methods	HotpotQA				2WikiMHQA				MuSiQue
Methods	Avg Viol $\downarrow$	VFR $\uparrow$	Post-Res $\downarrow$	USR $\downarrow$	Avg Viol $\downarrow$	VFR $\uparrow$	Post-Res $\downarrow$	USR $\downarrow$	Avg Viol $\downarrow$	VFR $\uparrow$	Post-Res $\downarrow$	USR $\downarrow$
Vanilla LM	2.65	7.43%	–	46.41%	2.83	6.58%	–	53.19%	3.25	5.34%	–	62.63%
CoT	1.98	24.89%	–	27.36%	2.21	17.41%	–	32.11%	2.91	13.26%	–	37.58%
MAD	1.33	36.74%	–	23.94%	1.81	32.78%	–	28.82%	2.16	26.17%	–	36.51%
SAVeR (Ours)	0.37	81.36%	0.05	9.12%	0.56	72.34%	0.08	13.84%	0.83	69.38%	0.11	19.73%

Table 2: Reasoning Faithfulness Evaluation on Multi-hop QA Benchmarks under LLaMA-3.1-8B.

Panel of Figure 3 — Figure 3: Audit-Repair (A-R) dynamics on HotpotQA. For SAVeR , iterations correspond to one audit-repair cycle. For MAD , iterations denote debates to reduce inconsistencies without verifiable acceptance criteria.

Figure 4: A case study on a multi-hop factual query to correct unjustified inference. The agent initially proposes plausible capacity values based on arena similarity and entity identification, but without explicit evidence . SAVeR audits belief candidates, flags unsupported numerical guesses and missing citable capacity statements , and repairs them by enforcing evidence-bound extraction of the exact seated-capacity integer from retrieved sources. After iterative re-auditing, only the verified capacity value is committed to the final answer and written to agent memory.

Methods	HotpotQA						2WikiMHQA
Methods	EM $\uparrow$	F1 $\uparrow$	Avg Viol $\downarrow$	VFR $\uparrow$	Post-Res $\downarrow$	USR $\downarrow$	EM $\uparrow$	F1 $\uparrow$	Avg Viol $\downarrow$	VFR $\uparrow$	Post-Res $\downarrow$	USR $\downarrow$
w/o Persona	43.2	52.4	0.49	74.55%	0.06	11.97%	47.6	55.3	0.78	65.98%	0.10	19.24%
w/o k-DPP	43.3	52.2	0.64	71.47%	0.08	15.86%	47.5	55.2	0.86	61.78%	0.14	20.52%
w/o Auditing	43.8	52.8	1.37	42.65%	–	26.74%	47.8	55.7	1.76	38.95%	–	29.17%
w/o Repair	44.0	52.9	1.56	33.68%	–	37.63%	48.1	55.6	1.83	29.17%	–	39.84%
SAVeR	43.7	52.6	0.37	81.36%	0.05	9.12%	47.7	55.5	0.56	72.34%	0.08	13.84%

Table 3: Ablation Study on the HotpotQA and 2WikiMHQA Benchmarks under LLaMA-3.1-8B.

Faithfulness of Language Model Reasoning

See all 24 papers

MAVEN: Multi-Agent Verification-Elaboration Network with In-Step Epistemic Auditing
May 8, 2026Yinsheng Yao, Jiehao Tang, Zhaozhen Yang +1LLM Auditing Multi-Agent Debate
Tongji University Shanghai, China
Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses
Jul 6, 2026Neeraj Karamchandani, Piyush Nagasubramaniam, Sencun Zhu +1LLM Agent Memory Agent Memory Poisoning
The Pennsylvania State University, University Park, PA, USA
AgentV-RL: Scaling Reward Modeling with Agentic Verifier
Apr 17, 2026Jiazheng Zhang, Ziche Fu, Zhiheng Xi +13Reward Modeling RL for Language Model Reasoning
College of Computer Science and Artificial Intelligence, Fudan University · Huazhong University of Science and Technology · The University of Hong Kong +3

May 8, 2026J/K move · Enter open · S save

MAVEN: Multi-Agent Verification-Elaboration Network with In-Step Epistemic Auditing

Yinsheng Yao, Jiehao Tang, Zhaozhen Yang, Dawei Cheng

Tongji University Shanghai, China

LLM Auditing Multi-Agent Debate Multi-Agent Reasoning LLM Reasoning

Open paper AskPDF arXiv

While explicit reasoning trajectories enhance model interpretability, existing paradigms often rely on monolithic chains that lack intermediate verification, allowing early errors to cascade unchecked. This lack of modularity impedes granular auditing and compromises the epistemic trust required for high-stakes applications. We propose MAVEN (Multi-Agent Verification-Elaboration Network with In-Step Epistemic Auditing), a blackboard-inspired framework designed to transform LLMs into deliberate reasoners through explicit role-decoupling. At its core, MAVEN operationalizes an adversarial Skeptic-Researcher-Judge loop, simulating expert deliberation by functionally separating logical defense from factual grounding. Experiments on OpenBookQA, TruthfulQA, HALUEVAL and StrategyQA benchmarks demonstrate that MAVEN delivers superior reasoning quality across four fine-grained metrics. Notably, MAVEN consistently outperforms latent reasoning models such as GEMINI-3.1-Pro and consensus-based baselines (e.g., ReConcile) by generating explicitly structured, modular, and verifiable deliberation trajectories, rather than relying on implicit internal states or post-hoc consensus. Moreover, comprehensive evaluations confirm that MAVEN is fully model-agnostic, serving as a strong and transferable reasoning booster that yields substantial performance improvements across diverse backbone models.

Abstract

Faithfulness of Language Model Reasoning

Explore similar work

MAVEN: Multi-Agent Verification-Elaboration Network with In-Step Epistemic Auditing

Your Agent's Memories Are Not Its Own: Forged Reasoning Attacks on LLM Agent Memory and Defenses

AgentV-RL: Scaling Reward Modeling with Agentic Verifier