Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought?

Jiashu He; Jinxuan Fan; Xiao Xiao; Radu Marculescu; Alejandro Ribeiro

cs.AISep 27, 2026

Ceiling of a Task: When Can a Transformer Succeed Without Its Chain of Thought?

Authors: Jiashu He, Jinxuan Fan, Xiao Xiao, Radu Marculescu, Alejandro Ribeiro

Organizations: University of Pennsylvania · University of Texas at Austin · Yale University

Abstract

Reasoning models generate long chains of thought before they answer, yet it is debated whether the content of these chains does real computational work or is largely decorative. We study this question by viewing a transformer as a shallow circuit. One forward pass through a fixed number of layers has constant depth, so any procedure that runs the model a constant number of times is a shallow circuit. We call the best accuracy that a shallow circuit can reach on a task the ceiling of the task, and a task is serial if its ceiling lies below one. We prove three results on serial tasks that hold for every transformer, no matter how it was trained. Necessity: replacing the chain by anything that does not depend on its content, such as filler tokens or a restatement of the question, drives the accuracy down to the ceiling, and on a maximally serial task down to chance. Depth: no shallow computation can write the chain of a model whose accuracy exceeds the ceiling, not even approximately. Locality: the answer is one shallow pass away from the finished chain, so all of the serial reasoning happens in the chain. On word problems of finite groups, whose ceilings are known, small transformers trained from scratch, with or without reinforcement learning, attain the predicted numbers: chain-trained models solve every input length and fall to chance when the chain is erased, chainless models collapse to the ceiling as the input length grows, and open-weight reasoning models given the same problem in words return to the baseline without their chain. On MATH-500 and AIME, erasing the chain costs open reasoning models 0.52 to 0.82 accuracy, a sentence shuffle is harmless, and a token shuffle is as harmful as erasing; the same holds for checkpoints trained by GRPO with a correct or a random reward. The ceiling of a task therefore answers when a transformer can succeed without its chain of thought.

Figures & tables

Panel of Figure 1 — Figure 1: Left: a transformer with $L=3$ layers on three input tokens and two generated ones. Inside one forward pass a hidden state reads only earlier positions of the layer below, so information travels at most $L$ steps. A transformer is therefore shallow when its output length is constant and not when it grows with the input. Right: the three degrees of seriality of Definition 2.4 for a task: a collection of inputs $q_{i}$ with targets $y(q_{i})$ .

Figure 3: Small transformers on word problems (

L=4

unless stated;

n=8

128

). (a) Necessity, from the least to the most serial group: models trained with the chain solve every length (black); erasing their chain at test time drops them to the baseline

b

on every group (Theorem 3.1 ). (b) The same drop, for each of the three erasing interventions, after reinforcement learning with GRPO, RLOO, or PPO on

A_{5}

n=32

. (c) Depth: without a chain, accuracy stays high only while one pass can track the input, and the collapse length grows with

L

. (d) The tracked prefix

t^{\ast}

levels off at a value set by

L

. (e) Models trained to write a chain of

T

steps: only

T=n

solves the task. (f) Locality on

A_{5}

for three chain formats. After a token shuffle the answer head reads only the bag of chain tokens, so its accuracy (green) cannot exceed that of the best bag decoder trained on the bag (

\triangledown

) (Theorem 5.2 , Lemma F.1 ). The bag reveals the answer only when the chain repeats it

2n+1

times (Proposition F.2 ).

Figure 4: Open-weight reasoning models. (a) Qwen3 models on

A_{5}

(maximally serial): accuracy after a chain of at most the given number of tokens. (b) Mathematical benchmarks: accuracy after truncating the chain to a prefix (solid) against the fraction of problems whose kept prefix already states the answer (dashed). (c) Accuracy after each intervention: the four erasing interventions (red) drop the accuracy, a sentence shuffle keeps the accuracy of the full chain, and a token shuffle falls to the level of erasing although some answer is still present in the shuffled chains (hollow diamonds). (d) The four untrained models (filled) and

19

checkpoints of Qwen3 models trained by GRPO with a correct or a random reward (hollow), on MATH-500: accuracy after intervention against the accuracy with the full chain (Table 6 ).

Appendix figures & tables9 assets

Supplementary material from the paper’s appendix.

Appendix

Class	Gates	Depth	Contains	Does not contain
$\mathsf{AC}^{0}$	AND, OR, NOT with unbounded fan-in	$O(1)$	OR of $n$ bits, addition of two $n$ -bit numbers	parity and majority (proved)
$\mathsf{TC}^{0}$	the same gates and majority	$O(1)$	counting, sums of $n$ numbers, multiplication, division, sorting, parity, word problems of solvable groups	the word problem of $A_{5}$ if and only if $\mathsf{TC}^{0}\neq\mathsf{NC}^{1}$ (open)
$\mathsf{NC}^{1}$	AND, OR, NOT with fan-in two	$O(\log n)$	the word problem of every finite group, evaluation of Boolean formulas

Appendix

Table 1: The three circuit classes used in this paper. All classes have polynomial size. One forward pass of a transformer lies in

\mathsf{TC}^{0}

at log precision and in

\mathsf{AC}^{0}

at constant precision (Fact 2.1 ).

Figure 5: Three circuits that combine

n=8

inputs. The sequential chain has depth

n-1

and is what a chain of thought implements, one step per token. The balanced tree has depth

\log_{2}n

and is how a circuit in

\mathsf{NC}^{1}

multiplies

n

elements of any finite group. The single wide gate has depth

1

and exists only when the answer depends on counts, which is the case for abelian groups and is the reason their word problems lie in

\mathsf{TC}^{0}

Panel of Figure 6 — Figure 6: Test-time interventions. The model generates its chain $c$ as usual. The intervention $\Phi$ then replaces $c$ by a modified chain $\tilde{c}$ , and the answer head produces the answer from the input and $\tilde{c}$ . The load-bearing gap is the accuracy that the model loses in this way. The lower part shows examples of $\tilde{c}$ (orange) for a chain that consists of four running products $P_{1},\dots,P_{4}$ .

Group $G$	$\|G\|$	$[G,G]$	chance $1/\|G\|$	price of the abelian quotient $1/\|[G,G]\|$	best shallow success (ceiling)
$\mathbb{Z}_{60}$ (abelian)	$60$	$\{e\}$	$1/60$	$1$	$1$
$A_{4}\times\mathbb{Z}_{5}$ (solvable)	$60$	$V_{4}$	$1/60$	$1/4$	$1$
$S_{5}$	$120$	$A_{5}$	$1/120$	$1/60$	$1/60+o(1)$
$A_{5}$ (simple)	$60$	$A_{5}$	$1/60$	$1/60$	$1/60+o(1)$

Appendix

Table 2: What a shallow circuit can obtain on the word problem of each group. The price of the abelian quotient is the success of the circuit that computes the image of the product in

G/[G,G]

and guesses inside the coset (Proposition D.2 ). The last column is the ceiling

\mathrm{shallow}_{n}(d,s)

at every fixed depth and size, i.e., the best success of a shallow circuit, up to the error term

n^{-k}

. It equals

1

for the two solvable groups, because their word problems lie in

\mathsf{TC}^{0}

; these two groups are the controls of Section 6 . For

S_{5}

and

A_{5}

the last column holds under Assumption C.2 (Proposition D.1 and Lemma C.3 ). The ceiling equals the guessing baseline only when the group has no shallow quotient, which is why Section 6.1 instantiates Theorem 3.1 on

A_{5}

Intervention	The answer head still sees	If the accuracy stays high, we learn that	If the accuracy drops, we learn that
Erasing (filler, random tokens, template, empty chain)	nothing from the chain	the model can answer these inputs without its chain, so a shortcut exists	the content of the chain was needed. On a maximally serial task the drop is forced and goes all the way to the guessing baseline (Theorem 3.1 )
Truncation (keep a prefix)	the first part of the chain	the answer was already determined at that point of the chain	this answer head cannot finish the remaining work in one pass. We learn nothing about the chain itself
Shuffle (tokens or sentences)	the bag of tokens or of sentences	the answer is readable from the bag, so the chain states it in a way that does not depend on order	nothing about the chain. Two policies with identical chains can react in opposite ways (Proposition F.3 )
Re-encoding (relabeling, paraphrase)	all of the information, in a different form	the answer head relies on features that survive the re-encoding	nothing about the chain. The information is unchanged, so the drop only reflects the answer head

Appendix

Table 3: How to read an intervention experiment. By Theorem 5.2 , the accuracy after an intervention is bounded by what a shallow decoder can read from the part of the chain that the intervention keeps. High accuracy is therefore informative for every intervention. A drop in accuracy is informative about the chain only for erasing interventions.

Figure 7: What small transformers learn without a chain, and parity (

L=4

). Left: on

A_{4}\times\mathbb{Z}_{5}

n=128

, chainless models predict the coset of the running product modulo

V_{4}=[G,G]

at every position and are right about the product itself a quarter of the time (three seeds). Middle: every chainless seed on

S_{5}

; seeds that learn the sign of the product land at the ceiling

2b=1/60

(dashed), the others at

b=1/120

(dotted). Right: on parity, chain-trained models fall to

1/2

when their chain is erased, while chainless models with dense supervision (one triangle per seed) solve it in

15

18

seeds.

setting	$n$	with chain	chain erased	tokens shuffled	bag decoder
Hand-coded plurality head on the chain of Prop. F.2	16–64	1.000	0.000–0.018	1.000	1.000
Trained on that chain, 12 architectures	32	1.000	0.015–0.019	0.690–1.000	1.000
Trained on that chain, $L=4$	18–56	1.000	0.014–0.019	0.696–0.926	1.000
Twin with plurality head (Prop. F.3 )	16–64	1.000	0.000–0.021	1.000	–
Twin with gated head (Prop. F.3 )	16–64	1.000	0.015–0.018	0.015–0.018	–
Student cloned from a trained teacher, same architecture	16–64	1.000	0.009–0.019	0.032–0.077	0.034–0.108

Appendix

Table 4: Witnesses for Propositions F.2 and F.3 on the word problem of

A_{5}

. Ranges over input lengths, architectures, and the three erasing interventions (means over three seeds, or over three evaluation seeds for the hand-coded heads). The bag decoder is trained on the counts of the chain tokens; the hand-coded twins read identical chains, so the plurality head is itself a decoder with accuracy

1

on the shuffled chains of the gated head.

task	model	$n=4$	$n=6$	$n=8$	$n=16$
$A_{5}$	Qwen3-4B	0.12 / 0.01	0.03 / 0.01	0.03 / 0.01	0.02 / 0.03
	Qwen3-8B	0.29 / 0.01	0.09 / 0.03	0.07 / 0.01	0.03 / 0.01
	Qwen3-14B	0.45 / 0.02	0.26 / 0.01	0.17 / 0.00	0.02 / 0.01
	Qwen3-32B	0.62 / 0.01	0.37 / 0.02	0.29 / 0.01	0.04 / 0.02
$\mathbb{Z}_{60}$	Qwen3-4B	0.14 / 0.03	0.03 / 0.01	0.02 / 0.01	0.01 / 0.01
	Qwen3-8B	0.35 / 0.03	0.12 / 0.01	0.04 / 0.03	0.02 / 0.01

Appendix

Table 5: Open models on the word problem of

A_{5}

and on its controls,

200

problems per cell: accuracy with the full chain / with the chain replaced by filler of the same length. The baseline is

1/60

on the three groups and

1/2

on parity; the retrieval task counts marked items in a context that grows with

n

start model	reward	step	records	full chain	filler	random	template	empty	sent. shuffle	token shuffle
Qwen3-4B	correct	25	128	0.891	0.258	0.266	0.297	0.297	0.891	0.328
Qwen3-4B	correct	50	128	0.953	0.297	0.273	0.336	0.312	0.914	0.375
Qwen3-4B	correct	425	2000	0.573	0.205	0.206	0.242	0.218	0.566	0.197
Qwen3-8B	correct	45	2000	0.853	0.266	0.244	0.296	0.304	0.839	0.271
Qwen3-8B	correct	255	2000	0.782	0.188	0.157	0.268	0.274	0.690	0.079
Qwen3-14B	correct	110	2000	0.874	0.316	0.300	0.334	0.340	0.861	0.332

Appendix

Table 6: The

22

Qwen3 checkpoints trained by GRPO that we evaluated on MATH-500 (records are problems times generations): accuracy with the full chain and after each intervention, all with the forced answer. † Excluded from Figure 4 d: the forced answer of these checkpoints ignores the chain, so every intervention gives the same accuracy, and the answer is never present in the chain.

LLM Reasoning

Large language model reasoning studies how models solve multi-step problems, verify intermediate steps, and use computation or tools reliably.

See all 915 papers

Sep 29, 2026

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

Pretrained transformers use little of their depth to follow references in context. Thirteen base models reliably follow only 1.4-3.6 lines, and extra pretrained loops add little. A task-trained rank-8 LoRA at one early layer extends this computation with all model weights frozen. Qwen3-8B improves from 15.5% to 99% exact accuracy on 24-line chains; a longer-trained LoRA reaches 50 lines. Ouro-1.4B reaches 60 lines after four loops and at least 160 after eight. The LoRA starts a relay: program lines pass on their chain identity through a short range of middle layers. Frozen heads read progressively further up the chain, and removing parent-line attention stops the relay. A frozen-model measurement locates the last useful intervention layer within tolerance in three of four held-out models. Task-specific LoRAs also improve MuSiQue. Default answers therefore understate the computation accessible through a tiny edit. Code and an interactive demo are available at https://lunamos.github.io/stop-thinking-too-early/

Zehao Jin, Ruixuan Deng, Junran Wang

Georgia Institute of Technology

Transformer LLM Fine-Tuning Low-Rank Adaptation Causal Interventions in Language Models LLM Reasoning

Ask PDF

Mar 10, 2026

Quantifying the Necessity of Chain of Thought through Opaque Serial Depth

Large language models (LLMs) tend to externalize their reasoning in their chain of thought, making the chain of thought a good target for monitoring. This is partially an inherent feature of the Transformer architecture: sufficiently long serial cognition must pass through the chain of thought (Korbak et al., 2025). We formalize this argument through the notion of opaque serial depth, given by the length of the longest computation that can be done without the use of interpretable intermediate steps like chain of thought. Given this formalization, we compute numeric upper bounds on the opaque serial depth of Gemma 3 models, as well as asymptotic results for additional architectures beyond standard LLMs. We also open-source an automated method that can calculate upper bounds on the opaque serial depth of arbitrary neural networks, and use it to demonstrate that Mixture-of-Experts models likely have lower depth than dense models. Overall, our results suggest that opaque serial depth is a useful tool for understanding the potential for models to do significant reasoning that is not externalized.

Jonah Brown-Cohen, David Lindner, Rohin Shah

Google DeepMind

Implicit Reasoning in Language Models CoT Reasoning LLM Interpretability

Ask PDF

May 29, 2026

Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation

Chain-of-thought prompting and looped Transformers both give a fixed model more test-time computation, but they differ in what they remember. Chain-of-thought stores intermediate state in generated tokens that remain in the context, whereas a looped Transformer carries state through recurrent hidden activations. We argue that this persistent mutable memory is a central resource for test-time reasoning. We compare three memory regimes, the compressed latent loop, the full sequence-state loop, and the chain-of-thought scratchpad. Our main result shows that a compressed loop is limited by the size of its recurrent state. Running the loop longer adds computation but does not by itself create a growing scratchpad, so a loop with a small recurrent state remains a small-space reasoner even when run for many steps. Under a standard complexity assumption, such loops cannot decide problems that are P-complete under logspace reductions, whereas polynomial-length chain-of-thought can. The separation is specific to compressed loops, as full sequence-state loops carry state at every input position and live in a memory-rich regime closer to explicit scratchpads. Controlled pointer-chasing and associative-recall sweeps illustrate this memory-budget view, with performance sensitive to whether the persistent-state budget matches the task's working-memory demand.

Haozhou Zhang

Department of Mathematics and Statistics University of Idaho

CoT Reasoning Persistent Memory for Language Models Recurrent Transformers

Ask PDF

Abstract

LLM Reasoning

Explore similar work

Transformers Stop Thinking Too Early, and a Tiny LoRA Fixes It

Quantifying the Necessity of Chain of Thought through Opaque Serial Depth

Chain-of-Thought and Compressed Looped Transformers: A Memory-Budget Separation