LLMs can perform seemingly planning-intensive tasks, like writing coherent stories or functioning code, without explicitly verbalizing a plan; however, the extent to which they implicitly plan is unknown. In this paper, we define latent planning as occurring when LLMs possess internal planning representations that (1) cause the generation of a specific future token or concept, and (2) shape preceding context to license said future token or concept. We study the Qwen-3 family (0.6B-14B) on simple planning tasks, finding that latent planning ability increases with scale. Models that plan possess features that represent a planned-for word like "accountant", and cause them to output "an" rather than "a"; moreover, even the less-successful Qwen-3 4B-8B have nascent planning mechanisms. On the more complex task of completing rhyming couplets, we find that models often identify a rhyme ahead of time, but even large models seldom plan far ahead. However, we can elicit some planning that increases with scale when steering models towards planned words in prose. In sum, we offer a framework for measuring planning and mechanistic evidence of how models' planning abilities grow with scale.
Figures & tables
Figure 1: Feature circuit for the input Someone who studies living organisms is a biologist. Someone who handles financial records is , explaining Qwen-3 (14B)’s output, an . The model plans to say accountant , causing it to output the appropriate article, an . Labeled nodes are sets of active transcoder features with shared semantics; edges indicate that the source node increases the target node’s activation. We demonstrate node semantics by selecting a node’s feature and showing its top-activating inputs and the vocabulary items that it up-/down-weights.
Category
Example Input
Next
Planned
a / an
Someone who handles financial records is
an
accountant
is / are
There were 5 dogs but 4 left. Now there
is
1
el / la
El animal marino con ocho tentáculos es
el
pulpo
Table 1: Three simple planning tasks. Each task prompts the model to output a planned token, preceded by a next token with two possible forms; the planned token determines the correct form.
Figure 2: Left : Qwen-3 family models’ recall of correct article on the a/an task. All models can recall a , but models ≤8 B have lower recall on the less-common an . Right : The mean proportion of influence flowing through planning nodes in the a/an dataset, by model, article, and correctness. On an examples where the model correctly predicts the next token, more influence tends to flow through the planning nodes. This effect is reversed and weaker for the majority class a .
Figure 3: Left : Change in p(correct article) caused by zero and multiplying interventions on planning features. As expected, ablating these harms performance, while upweighting them improves it; however, both affect primarily an examples, the minority class. Right : Change in p(correct article) caused by direct-effect interventions. Effects are smaller, indicating that planning features act both directly (by upweighting the correct article) and indirectly (by activating e.g say “a/an” features.)
Figure 4: A feature circuit for the couplet Fury burns where calm once stayed,\n Hope flickers where the shadows laid , explaining Qwen-3 (14B)’s decision to output shadows laid,\n . Halfway through outputting the couplet’s second line, the model’s ”near end of a line of poetry” features activate. These cause it to attend back to the end of the first line, where “end of a line of poetry” features are active, and to move “rhymes with -ayed ” features into the second line. These influence the model’s outputs, eventually leading to laid . End of line features then cause it to output .\n .
Figure 5: Left : Qwen-3 rhyming accuracy. In the base case, models have moderate rhyming accuracy, reaching 0.6 at 14B parameters (solid blue); when we consider assonant (vowel-only) rhyme, Qwen-3 (14B) achieves 0.8 (dashed blue). When steered to predict a new rhyme, model accuracy is only moderate for perfect rhymes (solid orange), but improves with scale, and is better on assonant rhyme (dashed orange). Right : Model rhyming accuracy when trying to predict a token satisfying the couplet’s the steered rhyme (blue lines) or original rhyme (red lines), given the original (solid) or steered (dashed) context. The model predicts the steered-for rhyme with similar accuracy given the original or steered context. This suggests that the steered context does not better license the rhyme.
Figure 6: Adaptation metrics by model. As models grow, so does the proportion of (1) outputs containing X (blue), and (3) coherent and X -containing outputs that also adapt the context to license X (green). Few examples do all three.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: A diagram of a transcoder. The transcoder takes in the dense MLP inputs, computes a sparse representation thereof, and then reconstructs the MLP’s dense outputs.
Figure 8: A 2-layer transformer LM, and its corresponding local replacement model. We replace model’s MLPs with transcoders, as well as error terms unique to the given input. The attention patterns (from the QK matrix) have been frozen, detaching them from the computation graph. Despite this, the OV-matrix of each attention block is still attached. Thus, when we refer to e.g. the direct effect of a feature of the layer-0 transcoder on a vocabulary logit, this direct effect may pass through the residual stream alone, or additionally through the OV matrix of the attention, a linear transformation. The direct effect of any given feature on any other feature (or any vocabulary logit) is thus linear. See Elhage et al. (2021) for more on QK/OV matrices and the residual stream.
Figure 9: The interface used for circuit visualization / annotation, from circuit-tracer .
Figure 10: Left : Recall of is and are on the is/are dataset, by model. Models below 8B in size mostly fail to predict is , while larger models perform perfectly. All models can predict are . Right : The mean proportion of influence flowing through planning nodes in the is/are dataset, by model, verb, and correctness; recall that the only incorrect examples are small models failing to predict is . The most influence flows through the planning nodes in the is examples, where more planning nodes are present. Still, more influence flows through these nodes in correct than incorrect is examples.
Figure 11: Left : Change in p(correct verb) caused by zero and multiplying interventions on planning features. The former generally harm performance, while the latter improve it. Both affect only is examples, which have the most planning nodes, and also are the only examples models answer incorrectly. Right : Change in p(correct verb) caused by direct zero and multiplying interventions on planning features. As before, these are relatively ineffective, though less so than in the a/an case.
Figure 12: Left : Recall of el and la examples by Qwen-3 models. Unlike in prior examples, the majority class ( el ) is not perfectly captured by any model, though recall is generally high. Moreover, while performance on the minority class la improves with scale, recall is ultimately still middling. Right : Interventions performed with respect to el/la planning features fail primarily due to a lack of said planning features.
Figure 13: Left : Planned word accuracy, i.e. whether the model’s predicted word matches the intended word, when given the correct or incorrect article. Models above 4B in size are highly accurate when given the correct article ( >80% ), and even smaller model achieve moderate accuracies. Given the wrong article, accuracies are lower, but still non-zero, indicating that models may have a strong planning goal that prevails even when the word is at odds with the article. Right : Planned word accuracy given the wrong article, by correct article ( a or an ). Though accuracy is low, models succeed on both a and an examples, indicating that successes are not driven by one class.
Figure 14: Number accuracy, i.e. whether the model’s predicted number of animals matches the intended number, when given the correct or incorrect verb ( is / are ). Notably, small models produce the correct number regardless of whether they are given the correct or incorrect verb. In contrast, Qwen3-14B and 32B have starkly reduced accuracy when given the wrong verb form.
Figure 15: Effects of random features interventions on the a/an ( left ) and is/are ( right ) tasks. Neither intervention has a large effect on either dataset, indicating that our interventions do not succeed by random chance.
Figure 16: Macro F1 scores when probing models’ last-token representations for the animal that the model will output in the following sentence, by model layer. F1 scores are high for certain models—Llama-3 (8B) and Qwen-3 (8B/32B)—but notably lower for others.
Figure 17: Success rates of our patching intervention, where we patch one prompt ( p2 )’s last token activations onto another ( p1 )’s last token during generation. We report both exact match (True if the output animal is p2 ’s animal) and any change (True if the output animal differs from p1 ’s original animal). In general, exact match is low, below 10%, while any change is higher, but under 30%.
Figure 18: A : Effects of intervening on end-of-line (EOL) features. Upweighting them in the second line causes the line to end early, while downweighting them causes it to continue for longer than normal. B : Effect of upweighting near-end of line (NEOL) features in the second line. Upweighting these causes the model to emit a rhyme over 2 words earlier than normal. C : Effects of downweighting EOL features at the end of the first line, downweighting NEOL features in the second line, or ablating rhyming-relevant attention heads’ patterns. The first two interventions drastically decrease the model’s propensity to produce a rhyme, indicating that they help enable rhyming. The last is less effective, but still reduces accuracy far below the original, 100% accuracy. D : Rhyming accuracy when ablating original rhyming features, and upweighting those from another rhyme group. Larger models switch to the new rhyme group with 40% accuracy—lower than their original rhyming accuracy, but still relatively high.
Figure 19: Length of the shared prefix between the original generation, generations with temperature 1.0, and steered generations, both greedily sampled and with temperature 1.0. Error bars show SE.
Figure 20: A feature circuit for a couplet ending in in the night . Unlike prior circuits, this circuit involves a specific night feature that drives the production of the phrase in the night .
Figure 21: Steering metrics by model size, averaged across steering strength. Overall, as model size increases, the ability to elicit say X by steering increases. The model’s tendency to adapt their context also appears to increase with size. Coherence is mixed, and appears unrelated to model size. Ultimately, few examples fulfill all three conditions—10% in the best case.
Figure 22: Performance of base (dashed line) and instruction-tuned (solid line) models on the a/an (left) and is/are (right) tasks.
Figure 23: The effectiveness of replacement interventions, measured by the percent of cases where the intervention induced to model to output the expected the measure word. Left : the expected measure word is considered to be the measure word that the model output on the target example. Right : the expected measure word is considered to be the ground truth measure word from the list above—even if the model failed to output it originally. Models’ accuracies with respect to the original ground truth are shown in green.
Figure 24: Results of steering on planning features from the a/an task. For each model, we plot the proportion of examples where the model produces the planned word (blue) and the proportion where the model produces a or an , followed by the planned word. Models often produce the planned word when steered, but less often produce the article in front of it.
Figure 25: Planning feature counts on an examples, by model and correctness. The smallest models have few planning features overall, while the largest has relatively many in both the correct and incorrect cases. In the 4B and 8B parameter models (with nascent circuits), there is a large gap between the number of planning features active in correct and incorrect cases..
When LLMs exhibit uneven performance across planning tasks, these gaps are often attributed to task difficulty. We argue that this explanation is incomplete, as task-level variation may reflect distinct latent planning competencies rather than differences along a single ability spectrum. We study this question on ACPBench-Hard by evaluating multiple LLM families under varying test-time reasoning budgets and applying a multidimensional item response theory model to uncover the latent competency structure underlying LLM planning. The analysis reveals two principal dimensions that shape planning performance: operational reasoning, the ability to evaluate local action applicability and immediate state transitions, and structural enumeration, the ability to reason about goal reachability and landmark structure. Operational reasoning improving under model scaling and longer reasoning traces, while structural enumeration remains comparatively insensitive. Our findings motivate competency-level evaluation of LLM planning, shifting the focus from whether models improve overall to which planning competencies improve, under what conditions, and why.
Sukai Huang, Chenyuan Zhang, Fucai Ke +4
Faculty of Information Technology, Monash University
We study planning site formation in language models -- where internal representations of structurally-constrained future tokens form during the forward pass, and whether they causally drive generation. Using rhyming-couplet completion as a clean test of forward-looking constraint, we apply two lightweight methods (linear probing and activation patching) across Qwen3, Gemma-3, and Llama-3 at more than ten scales. Probing shows that future-rhyme information is linearly decodable at the line boundary, with signal that strengthens with scale in all three families. Activation patching reveals that only Gemma-3-27B causally relies on this encoding, exhibiting a handoff in which the causal driver migrates from the rhyme word to the line boundary around layer 30. Every other model we test conditions on the rhyme word throughout generation, with near-zero causal effect at the line boundary despite strong probe signal. We localize the Gemma-3-27B handoff to five attention heads through two-stage path patching that recover ~90% of the rhyme-routing capacity at the newline.
Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and early token commitments in discrete reasoning traces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, many such methods optimize latent states end to end without a trained interface for intermediate textual readout, and several representative configurations use a pre-defined number of latent steps during inference. In this work, we introduce PLaT (Planning with Latent Thoughts), a framework that decouples latent planning from verbalization. The Planner deterministically evolves latent planning states, while an independent Decoder provides textual readouts when needed. Answer-aware textual stopping allows the latent rollout to use a problem-dependent number of groups rather than a pre-specified chain length. PLaT achieves competitive coverage at larger k in several mathematical settings, with lower Pass@1: on Llama-1B GSM8K, it reaches 80.59% Pass@128 versus CODI's 72.37%. These results support PLaT as a candidate-generation interface supplying multiple textual readouts for downstream verification or reranking.