Correct Chains, Wrong Answers: Dissociating Reasoning from Output in LLM Logic

Abinav Rao; Sujan Rachuri; Nikhil Vemuri

cs.CLMar 19, 2026

Correct Chains, Wrong Answers: Dissociating Reasoning from Output in LLM Logic

Authors: Abinav Rao, Sujan Rachuri, Nikhil Vemuri

Abstract

LLMs can execute every step of chain-of-thought reasoning correctly and still produce wrong final answers. We introduce the Novel Operator Test, a benchmark that separates operator logic from operator name, enabling rigorous distinction between genuine reasoning and pattern retrieval. By evaluating Boolean operators under unfamiliar names across depths 1-10 on five models (up to 8,100 problems each), we demonstrate a reasoning-output dissociation that existing benchmarks cannot detect. At Claude Sonnet 4's depth 7, all 31 errors have verifiably correct reasoning yet wrong declared answers; 17/19 errors in mixed-operator chains exhibit the same pattern. The benchmark reveals two failure types: strategy failures at depth 2, where models attempt terse retrieval (+62pp from scaffolding), and content failures at depth 7, where models reason fully but err systematically (+8-30pp, 0/300 errors post-intervention). A Trojan operator (XOR's truth table under a novel name) confirms name alone does not gate reasoning (p >= 0.49), while Llama's novelty gap widens to 28pp at depth 8-9 with the Trojan at 92-100%, isolating genuine difficulty with novel logic from name unfamiliarity.

Figures & tables

Group	Name	TT	TF	FT	FF	Sym	#T
A (Standard)	AND	T	F	F	F	Y	1
	OR	T	T	T	F	Y	3
	XOR	F	T	T	F	Y	2
	IF-THEN	T	F	T	T	N	3
B (Novel)	BLIF	F	T	F	F	N	1
	TARN	F	F	F	T	Y	1

Table 1: Truth tables for all nine operators. Group B operators use novel names. ZENT (Group C) is XOR’s truth table under a novel name, serving as the Trojan control.

		Accuracy (%)					Novelty Gap (A $-$ B, pp)
Model	Group	d=1	d=2	d=3	d=4	d=5	d=2	d=3	d=4	d=5
GPT-4o	A (Std)	100	81.0	98.0	97.5	99.5
	B (Novel)	100	69.0	99.0	99.5	99.5	12.0	$-$ 1.0	$-$ 2.0	0
	C (Trojan)	100	82.0	98.0	100	100
Claude Sonnet 4	A (Std)	100	95.0	92.5	96.5	97.5
	B (Novel)	100	93.0	94.0	94.0	98.5	2.0	$-$ 1.5	2.5	$-$ 1.0

Table 2: Accuracy (%) by operator group and chain depth. GPT-4o, Claude Sonnet 4, and QwQ-32B show no meaningful novelty gap at depth 5 (

|

gap

|\leq

6pp). Llama 3.1 70B shows an 11pp gap at depth 5, concentrated on structurally complex novel operators.

n=50

instances per cell.

Figure 1: Accuracy by operator group and chain depth (1–5) across five models. The depth-4 data point fills the recovery trajectory between the depth-2 dip and depth-5 ceiling. Llama shows a notable non-monotonic dip at depth 4 (91% Group A), driven by IF-THEN (68%). All conditions use

n=50

instances; shaded regions show 95% binomial CIs.

Model	XOR	ZENT	XOR	ZENT	XOR	ZENT	XOR	ZENT	XOR	ZENT	$p$
	d=1		d=2		d=3		d=4		d=5
GPT-4o	100	100	62	82	92	98	90	100	98 $\pm$ 4	100 $\pm$ 0	1.0
Claude Sonnet 4	100	100	98	100	86	94	100	94	100 $\pm$ 0	96 $\pm$ 5	0.49
Llama 3.1 70B	100	100	100	88	100	92	96	96	98 $\pm$ 4	96 $\pm$ 5	1.0
o3-mini	100	100	100	100	100	100	100	100	100 $\pm$ 0	98 $\pm$ 4	1.0
QwQ-32B	100	96	98	74	100	82	92	100	100 $\pm$ 0	100 $\pm$ 0	1.0

Table 3: XOR vs. ZENT accuracy (%) by depth (

n=50

per cell). No model shows a significant name effect at depth 5 (Fisher’s exact

p\geq 0.49

). 95% binomial CIs shown for depth 5; Fisher’s

p

in rightmost column.

Figure 2: GPT-4o response length and accuracy by chain depth for selected operators. (a) AND stays at 1 token across all depths (pure retrieval); TARN and DREM jump from

<

20 tokens at depth 2 to 300+ tokens at depth 3, marking the transition from retrieval to chain-of-thought reasoning. (b) Accuracy dips at depth 2 (the transition point) and recovers at depth 3+ once models adopt explicit reasoning.

	Group A (Std)		Group B (Novel)		Group C (Trojan)
Model	Tokens	Acc	Tokens	Acc	Tokens	Acc
GPT-4o	141 $\pm$ 181	99.5%	343 $\pm$ 67	99.5%	357 $\pm$ 35	100%
Claude Sonnet 4	199 $\pm$ 54	97.5%	272 $\pm$ 15	98.5%	279 $\pm$ 16	96%
Llama 3.1 70B	103 $\pm$ 180	98.5%	124 $\pm$ 168	87.5%	241 $\pm$ 117	96%
o3-mini	227 $\pm$ 176	100%	630 $\pm$ 266	100%	375 $\pm$ 118	98%

Table 4: Mean response length (completion tokens

\pm

std. dev.) at depth 5 by operator group (

n=200

for A/B,

n=50

for C). GPT-4o’s high Group A variance reflects the bimodal distribution: AND/OR produce 1 token while XOR/IF-THEN produce

\sim

300.

Figure 3: Mean completion tokens at depth 5 by operator group across four of five models (QwQ-32B excluded; response length data not collected). GPT-4o shows the starkest divergence between standard (141 tokens) and novel (343 tokens) groups. Error bars show 95% CIs.

Depth 2

Depth 3

Depth 5

Model

BLIF

TARN

QUEX

DREM

BLIF

TARN

QUEX

DREM

BLIF

TARN

QUEX

DREM

GPT-4o

+10

+62

+26

Claude Sonnet 4

+30

+14

+12

Llama 3.1 70B

+10

+22

+10

+12

+10

+16

+22

+16

Table 5: ETT improvement (pp) on Group B operators (

n=50

per cell). ETT has its largest effect at depth 2, where it bridges the reasoning mode transition (Section 4.3 ). At depths 3–5, ETT helps Llama and Claude but has no effect on GPT-4o (already at ceiling). The ETT experiment uses paired within-experiment baselines with separate variable assignments (seed 44) from the core experiment (seed 42).

	IF-THEN	ZENT	TARN	DREM	BLIF	XOR
Baseline (%)	70	82	88	92	100	100
ETT (%)	100	100	100	100	100	100
$\Delta$ (pp)	+30	+18	+12	+8	0	0
Fisher’s $p$	${<}0.0001$	$0.003$	$0.027$	$0.12$	—	—
Baseline errors	15	9	6	4	0	0
on GT=True	15	9	6	4	—	—

Table 6: ETT improvement at depth 7 for Claude Sonnet 4 (

n=50

per cell, seed 142). ETT achieves 100% on all operators. The four operators with baseline errors show +8 to +30pp improvement; controls (BLIF, XOR) are unaffected. All 34 baseline errors answer False on True-ground-truth instances. Note: the deep experiment (Section 4.7 , different seed) produces 31 errors with the same directional pattern.

Model	Group	d=6	d=7	d=8	d=9	d=10
GPT-4o	A (Std)	99.5	100	99.0	100	100
	B (Novel)	99.0	100	99.5	99.0	99.0
	C (Trojan)	100	100	98.0	100	100
Claude Sonnet 4	A (Std)	99.5	96.0	97.5	99.0	99.5
	B (Novel)	94.5	93.5	100	99.5	99.0
	C (Trojan)	96.0	80.0	100	100	100

Table 7: Group-level accuracy (%) at depths 6–10 (

n=200

per group cell,

n=50

for Trojan). GPT-4o and o3-mini maintain near-perfect accuracy. Claude shows a depth-7 dip with recovery. Llama’s novelty gap widens from 11pp at depth 5 to 28pp at depth 8–9. QwQ-32B tested at d=7 and d=10 only (—); its novelty gap vanishes by d=10.

Figure 4: Two failure modes in LLM logical reasoning. (a) Claude Sonnet 4 accuracy across depths 1–10 by operator group, showing the depth-7 dip with full recovery by depth 8. (b) Strategy failures (GPT-4o d=2, blue squares): low token counts, low accuracy; models attempt retrieval over chain-of-thought. Content failures (Claude d=7, orange diamonds): high token counts (

\sim

350), moderate accuracy; full reasoning produces wrong answers. (c) Error directionality at Claude’s depth 7: all 31 errors (red) occur on True-ground-truth instances; zero errors on False-ground-truth instances. BLIF and XOR (100% accuracy) serve as controls.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

	Depth 1			Depth 2
Model	Table	Code	NL	Table	Code	NL
GPT-4o	100	72	89.5	72.5	93.5	76
Claude Sonnet 4	100	100	100	92.5	100	90.5
Llama 3.1 70B	100	88	100	86.5	98	98.5

Appendix

Table 8: Accuracy (%) by representation format for Group B operators (

n=200

per cell, aggregated across 4 operators). Code format matches or exceeds truth tables at depth 2 for all models.

Grp	Operator	Sym	#T	GPT-4o	Claude	Llama	o3-mini	QwQ
A	AND	Y	1	100	100	100	100	100
	OR	Y	3	100	100	100	100	100
	XOR	Y	2	98	100	98	100	100
	IF-THEN	N	3	100	90	96	100	100
B	BLIF	N	1	100	100	94	100	100
	TARN	Y	1	98	98	92	100	98

Appendix

Table 9: Accuracy (%) per operator at depth 5 (

n=50

per cell). GPT-4o, Claude, and o3-mini achieve

\geq

90% on all operators. Llama shows difficulty on QUEX (78%) and DREM (86%); QwQ-32B shows difficulty specifically on DREM (78%).

Group	Operator	d=1	d=2	d=3	d=4	d=5	d=2 tokens
A	AND	100	100	100	100	100	1
	OR	100	100	100	100	100	1
	XOR	100	62	92	90	98	1
	IF-THEN	100	62	100	100	100	1
B	BLIF	100	96	100	100	100	1
	TARN	100	36	98	100	98	5

Appendix

Table 10: Full per-operator accuracy (%) across all depths for GPT-4o. Non-monotonic depth-2 dip and mean response tokens at depth 2 illustrate the reasoning mode transition. Depth 4 fills the recovery trajectory between depth 3 and 5.

		Accuracy (%)		Mean tokens
Operator	Depth	Standard	Show-work	Standard	Show-work
AND	2	100	100	1	361
AND	5	100	100	1	770
OR	2	100	100	1	343
OR	5	100	100	1	749
BLIF	2	87	100	1	475

Appendix

Table 11: Standard vs. “show your work” prompts for GPT-4o (

n=30

per cell). Show-work prompt forces verbose reasoning (342–770 tokens for AND/OR) and eliminates depth-2 strategy failures (TARN: 33%

\to

100%).

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 902 papers

ReasonOps: Operator Segmentation for LLM Reasoning Traces
May 28, 2026Daniel Lee, Owen Queen, James ZouLLM Reasoning Reasoning Trace Analysis
Stanford University
BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models
Mar 16, 2026Yuzhe TangLanguage Model Reasoning Evaluation LLM Evaluation
Georgia Institute of Technology
Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models
Jul 16, 2026Jungseob Lee, Seungyoon Lee, Suhyune Son +4CoT Distillation Language Model Distillation
1Korea University · 2Zoom Communications · 3Yonsei University

May 28, 2026J/K move · Enter open · S save

ReasonOps: Operator Segmentation for LLM Reasoning Traces

Daniel Lee, Owen Queen, James Zou

Stanford University

LLM Reasoning Reasoning Trace Analysis CoT Evaluation CoT Reasoning Structured Reasoning Large Reasoning Models+4

Open paper AskPDF arXiv

Chain-of-thought traces from large reasoning models can span tens of thousands of tokens, yet we lack a vocabulary for describing their internal structure. Previous methods developed to analyze chain-of-thought traces are either too rigid or not expressive enough, failing to capture features across domains and models. To remedy this, we develop ReasonOps, an unsupervised, expressive method for annotating chain-of-thought traces, providing succinct universal operators. Using ReasonOps, we analyze 44,662 traces from 12 thinking LLMs spanning 6 families across 8 reasoning benchmarks and discover that they share a common compositional structure: 7 recurring reasoning operators -- discourse-level moves such as backtracking, inferring, and hypothesizing -- that emerge from unsupervised clustering of sentence-initial 3-token pivots. These operators appear across every model family and benchmark domain, confirmed by three independent LLM judges who classify held-out samples at 70 -76% accuracy. We analyze the structure of operators on easy vs. hard problems, revealing that reflective operators are more helpful on hard problems and harm performance on easy problems. Operator sequences are highly model-identifying: a classifier trained on operator distributions alone recovers the source model with macro-AUC, revealing that each model family has a distinctive reasoning fingerprint. Structural operator features predict within-problem answer correctness well above baselines. Classifiers built on these operators reach WP-AUC and on AIME specifically. ReasonOps further enables early quality estimation well before the trace completes: we predict at WP-AUC for only 50% of the trace. The ReasonOps pipeline is unsupervised and annotation-free, enabling deep insights into LLM reasoning traces as well as strong downstream results on model identification and correctness prediction.

Abstract

LLM Reasoning

Explore similar work

ReasonOps: Operator Segmentation for LLM Reasoning Traces

BrainBench: Exposing the Commonsense Reasoning Gap in Large Language Models

Answer-Conditioned Chains of Thought Degrade Verifiable-Reasoning Distillation in Large Language Models