Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution

Nithin Sivakumaran; Shoubin Yu; Hyunji Lee; Yue Zhang; Ali Payani; Mohit Bansal; Elias Stengel-Eskin

cs.CLFeb 18, 2026

Balancing Faithfulness and Performance in Reasoning via Multi-Listener Soft Execution

Authors: Nithin Sivakumaran, Shoubin Yu, Hyunji Lee, Yue Zhang, Ali Payani, Mohit Bansal, Elias Stengel-Eskin

Organizations: UNC Chapel Hill · Cisco · University of Texas at Austin

Abstract

Chain-of-thought (CoT) reasoning sometimes fails to faithfully reflect the true computation of a large language model (LLM), hampering its utility in explaining how LLMs arrive at their answers. Moreover, optimizing for faithfulness and interpretability in reasoning often degrades task performance. To address this tradeoff and improve CoT faithfulness, we propose Reasoning Execution by Multiple Listeners (REMUL), a multi-party reinforcement learning approach. REMUL builds on the hypothesis that reasoning traces which other parties can follow will be more faithful. A speaker model generates a reasoning trace, which is truncated and passed to a pool of listener models who "execute" the trace, continuing the trace to an answer. Speakers are rewarded for producing reasoning that is clear to listeners, with additional correctness regularization via masked supervised finetuning to counter the tradeoff between faithfulness and performance. On multiple reasoning benchmarks (BIG-Bench Extra Hard, MuSR, ZebraLogicBench, and FOLIO), REMUL consistently and substantially improves three measures of faithfulness -- hint attribution, early answering area over the curve (AOC), and mistake injection AOC -- while also improving accuracy. Our analysis finds that these gains are robust across training domains, translate to legibility gains, and are associated with shorter and more direct CoTs.

Figures & tables

Figure 1: Existing reasoning models are prone to producing reasoning that is unfaithful and hard to follow. Thus, when given a reasoning chain prefix (blue) from a base speaker model (A), listener models are often unable to reproduce the speaker’s final decision, i.e., their soft execution suffix (white) diverges. REMuL rewards speakers for inducing consistent reasoning in the listener pool. With REMuL , the speaker’s reasoning is more faithful and easier for the listeners to follow consistently, with all listeners reaching the same conclusion as the speaker. Note that reasoning here is elided, and the shown prefix and truncated suffix are not by necessity the same length.

Figure 2: REMuL consists of two components: (Top) A speaker-listener reasoning execution reward, where listeners execute reasoning prefixes from a speaker, who is rewarded for listener consensus. The speaker’s final answer is only used for reward computation and not seen by the listeners. (Bottom) A masked supervised finetuning step to maintain correctness via a LoRA adapter, with loss computed only on answer tokens.

Model	Method	BBEH		ZLB		MuSR		FOLIO
Model	Method	Acc	Hint	Acc	Hint	Acc	Hint	Acc	Hint
Qwen3-14b	Original	31.3	56.7	32.6	67.3	65.8	76.4	75.3	63.0
	MAT-Steer	32.6	58.1	31.8	68.9	66.2	78.3	77.1	63.4
	Faithfulness Only	28.4	61.8	28.9	70.6	60.5	80.1	74.5	67.1
	Correctness Only	34.5	53.8	38.3	65.7	65.4	72.8	77.8	62.1
	Balanced Rewards	28.9	52.4	33.5	66.3	62.2	74.5	74.5	61.5

Table 1: Accuracy and hint usage across different reasoning benchmarks. REMuL consistently improves hint usage over the correctness baseline while maintaining or improving accuracy, demonstrating an effective balance between task performance and faithful reasoning. We bold and italicize the first and second-best results, respectively. Hint Optimized (grayed) is excluded from ranking as it is directly optimized for the hint usage test metric.

Method	Truncated CoT Answering				Adding Mistake
Method	BBEH	ZLB	MuSR	FOLIO	BBEH	ZLB	MuSR	FOLIO
Original	0.580	0.520	0.332	0.284	0.621	0.793	0.708	0.667
MAT-Steer	0.644	0.527	0.336	0.320	0.649	0.808	0.722	0.674
Faithfulness Only	0.665	0.587	0.330	0.350	0.672	0.838	0.731	0.714
Correctness Only	0.518	0.474	0.317	0.247	0.598	0.778	0.694	0.648
Balanced Rewards	0.574	0.512	0.328	0.299	0.614	0.790	0.712	0.671

Table 2: Qwen3-14B AOC metrics for measuring faithfulness using truncated CoT and after adding mistakes to the CoT, following Lanham et al. (2023) .

Method	BBEH	ZLB	MuSR	FOLIO
Original	91.2	93.4	94.9	93.1
Faithfulness-only	92.8	95.3	96.0	94.5
Correctness-only	88.7	91.2	93.0	91.4
Balanced Rewards	92.6	93.5	94.4	92.3
REMuL	92.1	95.8	95.5	92.8

Table 3: Legibility scores obtained following Emmons et al. (2025) , from a Qwen3-14B speaker. The best and second-best scores are in bold and italics , respectively.

Method	BBEH		ZLB
Method	Acc	Hint	Acc	Hint
Original	31.3	56.7	32.6	67.3
REMuL	33.3	59.3	35.7	70.1
w/ 3x same listener type	33.6	57.3	35.4	68.0
w/ 1x random listener	29.2	56.1	32.0	68.5

Table 4: Ablation study on listener model configurations (Qwen3). The best and second-best scores are in bold and italics , respectively.

Figure 3: Qwen3-14B reward curves during training for a model trained with a balanced, joint reward (Faithfulness + Correctness) as compared to each feature trained separately.

Method	BBEH	ZLB	MuSR	FOLIO
$\%\Delta(\textsc{REMuL}{}-\textit{Qwen3-14b})$
Qwen3 (Speaker)	$-5.85$	$-0.83$	$+2.61$	$-7.96$
Phi4 (Listener)	$+5.02$	$-19.51$	$+14.22$	$-23.43$
Ministral 3 (Listener)	$-12.02$	$-13.67$	$-26.66$	$+25.03$
Qwen3 (Listener)	$-11.92$	$+1.26$	$-7.61$	$-27.14$

Table 5: Percent change in tokens after training with REMuL . For the Speaker models, the token count is the full generation length. For the Listener models, the token count is the generation length after a 50% truncated prefix CoT is passed in from the speaker (i.e., only counting the newly generated tokens).

Method	BBEH	ZLB	MuSR	FOLIO
Original (Qwen3)	4.37	4.76	4.02	3.79
REMuL (Qwen3)	3.70	4.57	3.78	3.18

Table 6: Average frequency of backtracking terms in speaker CoTs.

Method	FOLIO
Method	Acc	Hint	AOC
Original (Qwen3)	75.3	63.0	0.284
REMuL (BBH Training Data)	77.3	64.6	0.332
REMuL (FOLIO Training Data)	83.0	61.3	0.292

Table 7: Performance on FOLIO across Accuracy (Acc), hint attribution (Hint), and area over curve (AOC) for CoT Truncation answering for Qwen3-14b, REMuL using our default training data, and REMuL finetuned on the FOLIO training split. The best and second-best scores in bold and italics , respectively.

Appendix figures & tables6 assets

Supplementary material from the paper’s appendix.

Appendix

Model	Method	BBEH		ZLB		MuSR		FOLIO
Model	Method	Acc	Hint	Acc	Hint	Acc	Hint	Acc	Hint
Qwen3	Original	31.3	56.7	32.6	67.3	65.8	76.4	75.3	63.0
REMuL ( Qwen3 )	Split by newline	33.3	59.3	35.7	70.1	65.8	79.5	77.3	64.6
REMuL ( Qwen3 )	Split by sentence	33.6	58.8	36.2	70.4	65.8	78.2	77.9	65.0

Appendix

Table 8: Accuracy and hint usage across different reasoning benchmarks. We bold the best result for each metric.

Method	BBEH	ZLB	MuSR	FOLIO
Original	0.281	0.166	0.208	0.243
Faithfulness-only	0.275	0.160	0.194	0.238
Correctness-only	0.277	0.156	0.201	0.241
Balanced Rewards	0.284	0.164	0.211	0.248
REMuL	0.274	0.161	0.198	0.234

Appendix

Table 9: Expected calibration error (lower is better) for REMuL and other baselines for Qwen3-14b. We bold the best result.

Method	BBEH	ZLB	MuSR	FOLIO
Original	60	1,247	144	100
MAT-Steer	62	1,231	143	101
Faithfulness Only	55	1,275	151	85
Correctness Only	65	1,193	136	95
Balanced Rewards	63	1,213	141	96
Hint Optimized	59	1,423	169	118

Appendix

Table 10: Number of changed-answer examples per method and dataset for Qwen3-14B. Total dataset sizes for BBEH, ZLB, MuSR, and FOLIO are 120, 3,259, 250, and 202 respectively.

Method	BBEH	ZLB	MuSR	FOLIO
Original	0.71	0.59	0.67	0.75
Faithfulness-only	0.68	0.63	0.71	0.77
Correctness-only	0.73	0.59	0.66	0.76
Balanced Rewards	0.70	0.58	0.66	0.75
REMuL	0.72	0.60	0.69	0.77

Appendix

Table 11: Solvability estimation metrics for REMuL and other baselines for Qwen3-14b. We bold the best result.

Method	Sycophancy (%) $\downarrow$
Original (Qwen3)	38.6
REMuL (Qwen3)	41.8

Appendix

Table 12: Sycophancy rate (percentage of examples where the model changed its answer to match the hint) for Qwen3-14B, averaged across BBEH, ZLB, MuSR, and FOLIO.

Panel of Figure 4 — Figure 4: All prompt templates used in experiments.

CoT Faithfulness

Chain-of-thought faithfulness measures whether a model’s stated reasoning causally reflects and supports the answer it gives.

See all 73 papers

CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness
Jul 21, 2026Ziming Wang, Yinghua Yao, Changwu Huang +2CoT Faithfulness Faithfulness of Language Model Explanations
Department of Computer Science and Engineering of Southern University of Science and Technology, Shenzhen, China · Center for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, China +1
Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment
Sep 30, 2026Yihuai Hong, Shauli Ravfogel, Chen Zhao +1CoT Faithfulness Faithfulness of Language Model Explanations
New York University · NYU Shanghai
Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?
Mar 23, 2026Richard J. YoungCoT Faithfulness Faithfulness of Language Model Explanations
University of Nevada, Las Vegas, Department of Management, Entrepreneurship and Technology, Lee Business School, Las Vegas, NV, USA · DeepNeuro AI, Las Vegas, NV, USA

Jul 21, 2026J/K move · Enter open · S save

CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness

Ziming Wang, Yinghua Yao, Changwu Huang, Ke Tang+1

Department of Computer Science and Engineering of Southern University of Science and Technology, Shenzhen, China · Center for Frontier AI Research, Agency for Science, Technology and Research (A*STAR), Singapore · School of AI and Liberal Arts, Beijing Normal-Hong Kong Baptist University, Zhuhai, China+1

CoT Faithfulness Faithfulness of Language Model Explanations Causal Reasoning in Language Models Causal Interventions in Language Models LLM Reasoning

Open paper AskPDF arXiv

Chain-of-thought (CoT) reasoning is widely used to improve both the performance and interpretability of large language models (LLMs), yet the generated reasoning may not faithfully support the final answer. We study this problem from a causal perspective, where a faithful CoT process should follow the chain

Z\rightarrow X\rightarrow Y

, with

Z

X

, and

Y

denoting the instruction, reasoning chain, and final answer, respectively. In this process, the instruction should affect the answer only through the reasoning chain. However, conventional autoregressive LLMs condition answer generation on both the instruction and the CoT, which still allows a direct instruction-to-answer shortcut. To address this issue, we propose CASE, a framework that combines training-time causal alignment and inference-time structural enforcement. During training, CASE builds counterfactual-CoT, biased-instruction, and empty-instruction datasets, and applies selective-loss fine-tuning to strengthen CoT-to-answer dependence while suppressing instruction shortcuts. During inference, CASE masks direct attention from instruction tokens to answer tokens, preventing the model from bypassing the generated CoT. We provide an information-theoretic analysis showing how these components promote faithful chains. Experiments on three models and four benchmarks show that CASE achieves a 37% average per-setting relative improvement in overall CoT faithfulness over the strongest baselines, exhibits stronger cross-dataset faithfulness transfer, and maintains competitive average accuracy. Code is available at https://github.com/oddwang/CASE.

Abstract

CoT Faithfulness

Explore similar work

CASE: Causal Alignment and Structural Enforcement for Improving Chain-of-Thought Faithfulness

Making LLMs Say What They Think: Measuring and Improving CoT-Interpretability Alignment

Lie to Me: How Faithful Is Chain-of-Thought Reasoning in Reasoning Models?