Textual Planning with Explicit Latent Transitions

Eliezer Shlomi; Ido Levy; Eilam Shapira; Michael Katz; Guy Uziel; Segev Shlomov; Nir Mashkif; Roi Reichart

cs.CLFeb 4, 2026

Textual Planning with Explicit Latent Transitions

Authors: Eliezer Shlomi, Ido Levy, Eilam Shapira, Michael Katz, Guy Uziel, Segev Shlomov, Nir Mashkif, Roi Reichart, +1 more

Organizations: Technion – Israel Institute of Technology, Haifa, Israel · IBM, Haifa, Israel.

Abstract

Planning requires a transition model that predicts how each action changes the current state. When a large language model (LLM) plays this role, every next state is generated token by token, which makes searching over many possible futures slow and expensive. Existing alternatives either still query an LLM at every step or require a symbolic model of the domain. We propose EmbedPlan, a transition model built on frozen text embeddings: it embeds natural language descriptions of the state and the action with a frozen LLM, predicts the embedding of the next state with a lightweight learned network, and returns the closest real state. Because this network can be trained on top of any encoder, EmbedPlan also provides a controlled way to compare text representations for learning transitions. We evaluate it on 9 classical planning domains, under six settings that hold out progressively more of the data, from transitions to entire domains, and against baselines ranging from predicting no change to learning symbolic action rules. On planning problems seen during training, EmbedPlan almost always ranks the true next state among its top five guesses, still does so for most queries even when every observed state is a candidate, and retains 92-99% of its single-step accuracy when predicting several steps ahead from its own outputs. Given the same candidate states as GPT-5.4, it picks the true next state more often while taking about 0.17 ms per transition with cached embeddings. Accuracy is lower on unseen problems and near chance on unseen domains, and the controlled comparison traces this limit to the state representation rather than to the learned transition.

Figures & tables

Panel of Figure 1 — Figure 1: EmbedPlan . A frozen LLM encoder $E$ embeds the state $s$ and the action $a$ (Blocksworld, pick-up(C) ), and learned heads $\pi_{s},\pi_{a}$ project them into a latent space where the transition is computed: a small network $T_{\theta}$ predicts the next-state embedding $\hat{\mathbf{h}}_{s^{\prime}}$ , and the nearest real state is retrieved as the successor $\hat{s}^{\prime}$ . Fed back as the next input, it supports multi-step rollout. Search itself is external. Latency is per transition with state embeddings cached (Appendix B.5 ).

Exposure	Protocol	Trained on $\rightarrow$ tested on	Distractors from
Observed problems	Interpolation	$80\%$ $\rightarrow$ $20\%$ of transitions	the whole domain
	Plan-Variant	some $\rightarrow$ other optimal plans	alternative-plan successors
Unseen problems	Extrapolation	${\sim}80\%$ $\rightarrow$ $20\%$ of problems	the query’s problem
	Multi-Domain	the same, all nine domains at once	the query’s problem
Unseen domains	Cross-Domain	one domain $\rightarrow$ another	the target domain
	Leave-One-Out	eight domains $\rightarrow$ the ninth	the target domain

Table 1: Evaluation protocols, grouped by exposure: what the test data share with training. Every query is ranked against 128 states, the true successor and 127 distractors (definitions in Appendix A ).

Table 3

Regime	Ferry	Logistics	Blocksworld
True state at each step	$95.3\pm 0.5$	$74.9\pm 2.2$	$80.3$
Own prediction, snapped	$94.2\pm 0.7$	$69.0\pm 2.1$	$78.4$
Own prediction, not snapped	$68.3\pm 2.0$	$40.0\pm 1.9$	$59.6$
Retention (snapped)	$99\%$	$92\%$	$98\%$

Table 4: Replacing each state prediction with the nearest real state before the next step ( snapped ) keeps multi-step rollout within

92

–

99\%

of the accuracy obtained with the true state at each step. Step-wise Hit@1 (%), Interpolation, 1,000 candidate states, mean

\pm

SE over seeds (Blocksworld: single run). Retention: snapped over true-state accuracy.

Table 6

Given the symbolic facts of each state
	Hit@5				Hit@1
Method	Ferry	Logistics	Goldminer	Mean	Mean
Lifted STRIPS induction (oracle)	$100.0\pm 0.0$	$100.0\pm 0.0$	$99.3\pm 0.3$	$99.8$	$99.8$
Bag-of-literals	$51.9\pm 5.7$	$81.4\pm 0.7$	$95.7\pm 2.0$	$76.3$	$37.1$
Sparse lexical features of the same text
Char 3–5 grams	$71.1\pm 1.7$	$69.8\pm 7.4$	$97.3\pm 0.7$	$\mathbf{79.4}$	$41.9$

Table 7: With the head, data, and pool fixed, the state representation decides transfer to unseen problems. Extrapolation, Hit@5 (%) per domain (mean

\pm

SE over 3 seeds) and three-domain means. The learned rows train the identical head on a different state representation (lifted STRIPS induction instead induces symbolic effects), and the first group requires the symbolic facts of each state. On these templated states an action edits only a few facts, which favors sparse and symbolic representations (Section 4.3 ). Bold: best without symbolic facts.

Appendix figures & tables36 assets

Supplementary material from the paper’s appendix.

Appendix

Protocol	Measures	Shared	Shared	Shared	Difficulty
		domain	problems	plans
Interpolation	Interpolation	Yes	Yes	Yes	Lowest
Plan-Variant	Plan generalization	Yes	Yes	No	Low
Extrapolation	Extrapolation	Yes	No	No	Medium
Multi-Domain	Capacity sharing	Yes	No	No	Medium
Cross-Domain	Zero-shot transfer	No	No	No	High

Appendix

Table 8: Evaluation protocol hierarchy by generalization difficulty.

Architecture
Hyperparameter	Values explored
Model type	MLP , hypernetwork
Hidden size	128 , 256
Number of layers	2 , 4
Dropout	0.0 , 0.5
Layer normalization	Yes

Appendix

Table 9: Hyperparameter search space. Bold values indicate the final configuration.

Domain	Metric	128	512	2,048	8,192	Full	Infl.
Ferry 46,205 states	Hit@1	$98.96\pm 0.09$	$97.21\pm 0.27$	$90.90\pm 0.55$	$76.58\pm 0.92$	$35.23\pm 1.14$	$2.81\times$
	Hit@5	$99.97\pm 0.03$	$99.85\pm 0.08$	$99.38\pm 0.09$	$97.97\pm 0.25$	$86.93\pm 1.36$	$1.15\times$
	Hit@10	$99.99\pm 0.01$	$99.94\pm 0.03$	$99.73\pm 0.08$	$99.03\pm 0.14$	$94.96\pm 0.63$	$1.05\times$
Logistics 13,373 states	Hit@1	$96.29\pm 0.23$	$88.21\pm 0.65$	$70.12\pm 1.83$	$47.20\pm 1.60$	$32.55\pm 1.69$	$2.96\times$
	Hit@5	$99.85\pm 0.08$	$99.37\pm 0.12$	$96.73\pm 0.31$	$83.13\pm 0.93$	$74.89\pm 1.71$	$1.33\times$
	Hit@10	$99.90\pm 0.05$	$99.72\pm 0.09$	$98.84\pm 0.11$	$92.99\pm 0.50$	$88.01\pm 0.92$	$1.14\times$

Appendix

Table 10: Effect of candidate pool size under Interpolation, Llama-3.3-70B, mean

\pm

SE over 3 seeds. “Full” retrieves against every observed state in the domain. The inflation column is the 128-way value divided by the full-pool value.

Metric	Full pipeline	Batched/cached	LLM API
	EmbedPlan
Latency	18.6 ms	0.168 ms	1,888.2 ms
Speedup vs. LLM	101 $\times$	11,215 $\times$	1 $\times$
Compute per transition	$\sim$ 286M FLOPs †	$\sim$ 286M FLOPs †	198 tokens

Appendix

Table 11: Wall-clock latency and computational cost comparison. The full pipeline includes text-to-vector encoding time with BGE-M3. EmbedPlan eliminates the sequential token bottleneck of autoregressive generation. † Projection heads, transition network, and scoring of 128 candidates with cached encoder outputs (Llama-3.3-70B dimensions). The full-pipeline latency additionally includes encoding the query texts.

Domain	Problems	States	Transitions	Actions
Blocksworld	5	43,551	43,065	4
Depot	7	5,795	13,256	5
Ferry	10	46,205	225,300	3
Floortile	6	33,608	166,565	6
Goldminer	7	12,237	52,023	7
Grid	5	8,671	664,346	5

Appendix

Table 12: Dataset statistics by domain.

Panel of Figure 5 — Figure 5: Representative transition from Blocksworld.

Panel of Figure 6 — Figure 6: Representative transition from Ferry.

Panel of Figure 7 — Figure 7: Representative transition from Logistics.

Panel of Figure 8 — Figure 8: Representative transition from Rovers.

	Qwen2.5-7B		Llama-3.3-70B
Domain	Interp.	Extrap.	Interp.	Extrap.
Blocksworld	100.0	41.6 $\pm$ 8.7	100.0	49.1 $\pm$ 10
Depot	98.2	24.8 $\pm$ 5.2	98.8	25.9 $\pm$ 6
Ferry	99.9	36.7 $\pm$ 0.6	100.0	40.6 $\pm$ 3
Floortile	99.4	55.2 $\pm$ 11	99.6	68.8 $\pm$ 16
Goldminer	99.9	74.4 $\pm$ 5.8	100.0	76.2 $\pm$ 5

Appendix

Table 13: Per-domain Hit@5 (%) for the Interpolation and Extrapolation splits.

	Next state prediction (%)			Action disambiguation (%)
Domain	Hit@1	Hit@5	Hit@10	Acc@1	Acc@5	Acc@10
Blocksworld	$94.6\pm 0.4$	$100.0\pm 0.0$	$100.0\pm 0.0$	$25.9\pm 0.9$	$88.6\pm 0.2$	$99.5\pm 0.1$
Depot	$76.9\pm 1.2$	$98.8\pm 0.1$	$99.4\pm 0.1$	$29.8\pm 2.3$	$82.2\pm 2.1$	$93.3\pm 1.4$
Ferry	$98.5\pm 0.2$	$100.0\pm 0.0$	$100.0\pm 0.0$	$43.1\pm 0.8$	$96.8\pm 0.1$	$99.8\pm 0.0$
Floortile	$97.9\pm 0.4$	$99.6\pm 0.0$	$99.8\pm 0.0$	$37.1\pm 2.8$	$86.5\pm 1.9$	$97.3\pm 0.8$
Goldminer	$93.8\pm 0.8$	$100.0\pm 0.0$	$100.0\pm 0.0$	$17.0\pm 0.5$	$73.3\pm 0.6$	$97.4\pm 0.2$

Appendix

Table 14: Single-step transition prediction (Interpolation split, Llama-3.3-70B). Next state prediction : whether the ground-truth next state is in the top-

k

retrieved candidates. Action disambiguation : whether, among all actions applied to

s

, the correct action

a

produces the prediction closest to ground-truth

s^{\prime}

. Mean Acc@5 is 85.3%.

	State prediction			Action accuracy
Domain	Hit@1	Hit@5	Hit@10	Acc@1	Acc@5	Acc@10
Blocksworld	17.6 $\pm$ 5.2	49.1 $\pm$ 9.8	64.6 $\pm$ 7.5	0.7 $\pm$ 0.1	7.9 $\pm$ 1.2	24.0 $\pm$ 2.3
Depot	4.7 $\pm$ 1.2	25.9 $\pm$ 6.4	41.2 $\pm$ 8.1	0.7 $\pm$ 0.2	7.8 $\pm$ 1.7	21.8 $\pm$ 2.3
Ferry	12.0 $\pm$ 1.7	40.6 $\pm$ 3.5	58.1 $\pm$ 4.0	1.1 $\pm$ 0.4	10.3 $\pm$ 1.2	23.7 $\pm$ 2.3
Floortile	37.6 $\pm$ 13	68.8 $\pm$ 16	78.9 $\pm$ 13	3.2 $\pm$ 0.6	28.3 $\pm$ 8.7	52.3 $\pm$ 14
Goldminer	35.8 $\pm$ 6.4	76.2 $\pm$ 5.2	88.0 $\pm$ 1.7	3.0 $\pm$ 0.2	16.5 $\pm$ 1.2	45.7 $\pm$ 3.5

Appendix

Table 15: Full metrics (Llama-3.3-70B, Extrapolation split). Action accuracy measures whether the correct action is identified given

(s,s^{\prime})

	Block	Depot	Ferry	Floor	Gold	Grid	Logis	Rover	Satel	Mean
Blocksworld		5.7	5.5	6.7	6.1	6.5	9.2	5.1	5.3	6.3
Depot	4.3		5.3	4.8	5.0	4.9	6.0	7.5	5.4	5.4
Ferry	5.3	8.7		9.3	5.2	9.0	22.3	6.6	5.6	9.0
Floortile	5.0	7.3	7.0		6.1	6.5	7.5	10.0	7.9	7.2
Goldminer	4.2	5.4	5.7	5.0		7.5	5.4	5.9	4.9	5.5
Grid	5.4	6.6	8.6	9.0	10.4		14.3	5.9	5.4	8.2

Appendix

Table 16: Cross-Domain transfer (Llama-3.3-70B). Hit@5 (%) training on the row domain and testing on the column domain. Chance: 3.9% (5 of 128). Untrained model: 4.2%.

Held-out domain	Hit@1	Hit@5	Hit@10
Logistics	3.3 $\pm$ 0.4	15.8 $\pm$ 2.2	27.6 $\pm$ 3.3
Grid	2.3 $\pm$ 0.2	12.3 $\pm$ 0.8	22.9 $\pm$ 1.4
Rovers	2.6 $\pm$ 0.3	12.6 $\pm$ 1.1	22.6 $\pm$ 1.6
Ferry	1.8 $\pm$ 0.1	9.0 $\pm$ 0.5	17.2 $\pm$ 0.8
Satellite	1.6 $\pm$ 0.1	8.4 $\pm$ 0.8	16.1 $\pm$ 1.5
Floortile	1.7 $\pm$ 0.3	7.9 $\pm$ 1.1	14.9 $\pm$ 2.0

Appendix

Table 17: Leave-One-Out results (Llama-3.3-70B). Train on eight domains, test on the held-out domain.

Domain	Hit@5	Domain	Hit@5
Floortile	52.0 $\pm$ 10	Blocksworld	36.9 $\pm$ 13
Rovers	51.9 $\pm$ 5	Satellite	34.3 $\pm$ 7
Goldminer	46.0 $\pm$ 8	Grid	32.6 $\pm$ 1
Ferry	37.7 $\pm$ 14	Logistics	24.8 $\pm$ 10
		Depot	18.9 $\pm$ 12
Mean : 37.2 $\pm$ 3.8 (vs. 54.6 single-domain)

Appendix

Table 18: Multi-Domain unified model (Llama-3.3-70B, Extrapolation split).

Domain	Hit@1	Hit@5	Hit@10
Blocksworld	0.8 $\pm$ 0.1	4.0 $\pm$ 0.2	8.0 $\pm$ 0.4
Depot	0.9 $\pm$ 0.2	3.9 $\pm$ 0.0	8.0 $\pm$ 0.4
Ferry	0.7 $\pm$ 0.1	4.3 $\pm$ 0.6	8.4 $\pm$ 0.9
Floortile	0.8 $\pm$ 0.0	4.0 $\pm$ 0.1	8.0 $\pm$ 0.1
Goldminer	1.0 $\pm$ 0.4	4.8 $\pm$ 1.4	8.9 $\pm$ 2.0
Grid	0.9 $\pm$ 0.1	4.4 $\pm$ 0.3	8.0 $\pm$ 0.2

Appendix

Table 19: Untrained baseline (random weights, Llama-3.3-70B).

	Plan-Variant		Extrapolation
Domain	Mean Hit@5	Exact Hit@5	Mean Hit@5	Exact Hit@5
Blocksworld	$38.7\pm 0.8$	$1.6\pm 0.2$	$3.7\pm 1.3$	$0.0\pm 0.0$
Depot	$30.1\pm 2.6$	$11.2\pm 1.5$	$2.4\pm 1.7$	$1.0\pm 1.0$
Ferry	$71.1\pm 1.1$	$39.1\pm 1.6$	$2.0\pm 0.7$	$0.5\pm 0.2$
Floortile	$74.1\pm 2.7$	$33.6\pm 3.3$	$11.6\pm 4.4$	$2.8\pm 1.2$
Goldminer	$55.4\pm 2.2$	$23.8\pm 1.6$	$18.5\pm 6.2$	$5.5\pm 1.8$

Appendix

Table 20: Plan-level evaluation (Llama-3.3-70B). Trajectory metrics under the Plan-Variant and Extrapolation splits.

	BGE-M3		MPNet
Plan-Variant	Mean Hit@5	Exact Hit@5	Mean Hit@5	Exact Hit@5
Blocksworld	$23.7\pm 1.5$	$0.2\pm 0.1$	$16.7\pm 0.3$	$0.3\pm 0.1$
Depot	$45.5\pm 1.0$	$20.9\pm 0.1$	$31.1\pm 1.6$	$10.8\pm 1.1$
Ferry	$63.3\pm 1.4$	$27.8\pm 2.1$	$60.9\pm 2.0$	$29.1\pm 2.3$
Floortile	$49.2\pm 1.0$	$13.1\pm 1.1$	$9.2\pm 0.5$	$3.0\pm 0.4$
Goldminer	$67.9\pm 2.6$	$35.0\pm 3.2$	$26.8\pm 0.8$	$10.5\pm 0.5$

Appendix

Table 21: Plan-level evaluation across encoders. Top : Plan-Variant trajectory metrics for BGE-M3 and MPNet (Llama-3.3-70B in Table 20 ). Bottom : Extrapolation, mean

\pm

SE over the six domains with an independent run for every encoder (Blocksworld, Depot, Ferry, Floortile, Goldminer, Grid).

Depth	$\bm{n}$	True state at each step	Snapped	Not snapped
1	300	$94.3\pm 0.7$	$94.3\pm 0.7$	$94.3\pm 0.7$
2	300	$96.3\pm 0.5$	$94.4\pm 0.7$	$52.4\pm 3.4$
3	57	$94.2\pm 1.9$	$92.4\pm 0.5$	$20.9\pm 2.4$
4	7	$96.7\pm 4.7$	$93.3\pm 9.4$	$10.3\pm 7.4$

Appendix

Table 22: Closed-loop rollout by depth on Ferry. Step-wise Hit@1 in % (mean

\pm

SE), where

n

is the number of test trajectories reaching that depth. Same models, trajectories, and pool as Table 4 , and the depth rows weighted by

n

reproduce the aggregate to rounding.

Statistic	Value
Interpolation (mean $\pm$ SE)	99.5% $\pm$ 0.2%
Extrapolation (mean $\pm$ SE)	47.7% $\pm$ 4.9%
Gap	51.8 pp
Paired $t$ -test	$t(8)=10.58$
$p$ -value	$5.57\times 10^{-6}$
95% CI	$[40.7,62.9]$ pp

Appendix

Table 23: Statistical analysis of the generalization gap (Qwen2.5-7B, paired across the nine domains).

Comparison	Encoder	$\Delta$	Cohen’s $d$	$p$
Interpolation vs. Extrapolation	Qwen2.5-7B	$-$ 51.8 pp	5.25	$<10^{-5}$
Extrapolation vs. Cross-Domain	Llama-3.3-70B	$-$ 48.0 pp	4.32	$<10^{-5}$
Llama vs. MPNet (Extrap.)	both	$+$ 27.8 pp	2.53	$<0.001$
Single vs. Multi (Extrap.)	Llama-3.3-70B	$+$ 17.4 pp	1.42	0.028

Appendix

Table 24: Effect sizes for major findings. The encoder column is given explicitly because the comparisons are drawn from different encoder runs.

Domain	Gap (pp)	$t$ -statistic	$p$ -value
Depot	73.4	12.87	$1.17\times 10^{-4}$
Ferry	63.3	65.49	$3.24\times 10^{-6}$
Satellite	59.9	153.39	$4.11\times 10^{-8}$
Blocksworld	58.4	5.95	$2.70\times 10^{-3}$
Logistics	55.0	4.28	$8.28\times 10^{-3}$
Rovers	50.5	20.22	$8.90\times 10^{-6}$

Appendix

Table 25: Independent

t

-tests comparing interpolation and extrapolation per domain (Qwen2.5-7B).

Panel of Figure 9 — Figure 9: Embedding space fragmentation across scales. PCA of frozen state embeddings, colored by domain. (A) MPNet places each domain in its own tight, isolated cluster. (B) Llama-3.3-70B, despite being roughly 640 $\times$ larger, shows the same fragmentation. Pre-trained embeddings thus group states by domain-specific surface features rather than by abstract planning roles, regardless of model scale.

Domain	Actions	Predicates	Gap
Depot	5	8	73.4
Logistics	6	6	55.0
Satellite	4	8	59.9
Blocksworld	4	5	58.4
Ferry	3	5	63.3
Rovers	9	26	50.5

Appendix

Table 26: Domain complexity metrics and generalization gaps (Qwen2.5-7B). Action counts are action schemas as defined in the PDDL domain file, matching Table 12 .

Encoder	Parameters	MLP	Hypernetwork
Llama-3.3-70B	70B	54.6 $\pm$ 5.5	53.8 $\pm$ 5.5
Qwen2.5-7B	7B	47.7 $\pm$ 4.9	46.2 $\pm$ 5.0

Appendix

Table 27: Architecture comparison for the two largest encoders (Extrapolation split, Hit@5 %).

	Hit@5 (%)		Action Acc@5 (%)
Domain	$\lambda{=}0$	$\lambda{=}2$	$\lambda{=}0$	$\lambda{=}2$
Blocksworld	30.2 $\pm$ 8	49.1 $\pm$ 10	1.8 $\pm$ 0.4	7.9 $\pm$ 2
Depot	13.4 $\pm$ 4	25.9 $\pm$ 6	1.6 $\pm$ 0.3	7.8 $\pm$ 3
Ferry	24.8 $\pm$ 3	40.6 $\pm$ 3	2.5 $\pm$ 0.5	10.3 $\pm$ 2
Floortile	45.3 $\pm$ 14	68.8 $\pm$ 16	8.2 $\pm$ 3	28.3 $\pm$ 15
Goldminer	55.7 $\pm$ 6	76.2 $\pm$ 5	4.9 $\pm$ 1.2	16.5 $\pm$ 2

Appendix

Table 28: Ablation of the action disambiguation loss. Comparison of models trained without (

\lambda{=}0

) and with (

\lambda{=}2

) action disambiguation under Extrapolation evaluation (Llama-3.3-70B). The action loss yields gains in both state prediction (

+

19.3 pp) and action accuracy (

3.4\times

Domain	Ground truth $s^{\prime}$	Retrieved $\hat{s}^{\prime}$
Blocksworld	Block A is clear, arm holds B , C is on table	Block A is clear, arm holds C , B is on table
Ferry	Car c1 at l0, ferry empty, c2 at l1	Car c1 at l0, ferry empty, c2 at l0
Logistics	Package p1 in truck t0 , t0 at l1-0	Package p1 at l1-0 , t0 at l1-0

Appendix

Table 29: Representative Hit@5 errors. Retrieved states differ from the ground truth by one or two predicates (in italics), typically involving object locations or holdings within the same problem instance.

Domain	Split	Hit@1	Hit@5	Hit@10
Blocksworld	Interpolation	94.3 $\pm$ 0.2	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0
Blocksworld	Extrapolation	14.2 $\pm$ 4.0	41.6 $\pm$ 8.8	56.6 $\pm$ 8.3
Depot	Interpolation	76.7 $\pm$ 1.0	98.2 $\pm$ 0.1	99.0 $\pm$ 0.0
Depot	Extrapolation	4.9 $\pm$ 1.2	24.8 $\pm$ 4.9	42.2 $\pm$ 7.8
Ferry	Interpolation	98.7 $\pm$ 0.1	99.9 $\pm$ 0.0	100.0 $\pm$ 0.0
Ferry	Extrapolation	10.7 $\pm$ 0.8	36.7 $\pm$ 0.7	52.8 $\pm$ 1.1

Appendix

Table 30: Complete metrics for Qwen2.5-7B across all domains and splits.

Domain	Split	Hit@1	Hit@5	Hit@10
Blocksworld	Interpolation	96.2 $\pm$ 0.2	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0
Blocksworld	Extrapolation	17.6 $\pm$ 5.2	49.1 $\pm$ 9.8	64.6 $\pm$ 7.5
Depot	Interpolation	79.5 $\pm$ 0.7	98.8 $\pm$ 0.1	99.2 $\pm$ 0.1
Depot	Extrapolation	4.7 $\pm$ 1.2	25.9 $\pm$ 6.4	41.2 $\pm$ 8.1
Ferry	Interpolation	99.1 $\pm$ 0.1	100.0 $\pm$ 0.0	100.0 $\pm$ 0.0
Ferry	Extrapolation	12.0 $\pm$ 1.7	40.6 $\pm$ 3.5	58.1 $\pm$ 4.0

Appendix

Table 31: Complete metrics for Llama-3.3-70B across all domains and splits.

Method	Ferry	Logistics	Goldminer	Mean
Lifted STRIPS induction (oracle)	$100.0\pm 0.0$	$100.0\pm 0.0$	$99.3\pm 0.3$	$99.8$
Char 3–5 grams	$71.1\pm 1.7$	$69.8\pm 7.4$	$97.3\pm 0.7$	$79.4$
Bag-of-words	$70.8\pm 2.7$	$67.3\pm 4.8$	$99.3\pm 0.2$	$79.1$
Bag-of-literals	$51.9\pm 5.7$	$81.4\pm 0.7$	$95.7\pm 2.0$	$76.3$
TF-IDF bigrams	$59.5\pm 2.1$	$50.0\pm 4.3$	$97.2\pm 1.2$	$68.9$
EmbedPlan (Llama-3.3-70B)	$40.6\pm 3.5$	$53.7\pm 9.8$	$76.2\pm 5.2$	$56.8$

Appendix

Table 32: Extrapolation, Hit@5 (%), mean

\pm

SE over 3 seeds. The Llama-3.3-70B row is reproduced from Table 13 , and all other rows were computed for this comparison under the identical protocol.

Method	Ferry	Logistics	Goldminer	Mean
Lifted STRIPS induction (oracle)	$100.0\pm 0.0$	$100.0\pm 0.0$	$99.3\pm 0.3$	$99.8$
Bag-of-words	$31.5\pm 3.3$	$25.4\pm 4.2$	$78.0\pm 1.9$	$45.0$
Char 3–5 grams	$31.0\pm 2.4$	$24.2\pm 2.4$	$70.6\pm 3.5$	$41.9$
TF-IDF bigrams	$24.7\pm 1.5$	$14.2\pm 2.9$	$72.8\pm 2.5$	$37.2$
Bag-of-literals	$14.2\pm 0.9$	$30.7\pm 0.8$	$66.5\pm 8.7$	$37.1$
EmbedPlan (Llama-3.3-70B)	$12.0\pm 1.7$	$16.5\pm 3.5$	$35.8\pm 6.4$	$21.4$

Appendix

Table 33: Extrapolation, Hit@1 (%), mean

\pm

SE over 3 seeds. The separation between EmbedPlan and the non-learned floors is wider here than at Hit@5.

Hit@5
Method	Ferry	Logistics	Goldminer	Mean
Bag-of-literals	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.1$	$100.0$
Bag-of-words	$99.9\pm 0.1$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0$
EmbedPlan (Llama-3.3-70B)	$100.0\pm 0.0$	$99.9\pm 0.0$	$100.0\pm 0.0$	$100.0$
TF-IDF bigrams	$99.9\pm 0.0$	$100.0\pm 0.0$	$100.0\pm 0.0$	$100.0$
Char 3–5 grams	$99.9\pm 0.0$	$99.9\pm 0.1$	$100.0\pm 0.0$	$100.0$

Appendix

Table 34: Interpolation, Hit@5 and Hit@1 (%), mean

\pm

SE over 3 seeds. Six methods lie within

0.2

points of one another on Hit@5.

Encoder treatment	Extrapolation	Interpolation
Frozen	$3.8$	$25.6$
LoRA, cold start	$6.2$	$39.6$
LoRA, warm start	$\mathbf{9.3}$	$\mathbf{41.7}$

Appendix

Table 35: Full-pool Hit@1 (%), mean over three domains and three seeds. The full pool contains every state in the domain (

12{,}237

–

46{,}205

) rather than 128 candidates.

Domain	Extrapolation	Interpolation
Goldminer	$+13.3$	$+10.5$
Ferry	$-0.1$	$+1.6$
Logistics	$-3.9$	$-5.9$

Appendix

Table 36: Effect of the warm start, in points of full-pool Hit@1 relative to the cold start. The sign is consistent per domain across two independent splits.

Domain	$\cos(s,s^{\prime})$	$\cos(s,\sigma(s))$	$d_{\text{order}}/d_{\text{action}}$
Ferry	$0.99672$	$0.99654$	$1.06$
Goldminer	$0.99881$	$0.99862$	$1.16$
Logistics	$0.99928$	$0.99915$	$1.18$
Mean			$\mathbf{1.13}$

Appendix

Table 37: Embedding displacement from a semantically null reordering (

d_{\text{order}}=1-\cos(s,\sigma(s))

, with

\sigma

a random clause order) against that from a real transition (

d_{\text{action}}=1-\cos(s,s^{\prime})

), BGE-M3.

Explore similar work

Jan 29, 2026cs.AI

Latent Chain-of-Thought as Planning: Decoupling Reasoning from Verbalization

Chain-of-Thought (CoT) empowers Large Language Models (LLMs) to tackle complex problems, but remains constrained by the computational cost and early token commitments in discrete reasoning traces. Recent latent reasoning approaches attempt to optimize efficiency by performing reasoning within continuous hidden states. However, many such methods optimize latent states end to end without a trained interface for intermediate textual readout, and several representative configurations use a pre-defined number of latent steps during inference. In this work, we introduce \textbf{PLaT} (\textbf{P}lanning with \textbf{La}tent \textbf{T}houghts), a framework that decouples latent planning from verbalization. The Planner deterministically evolves latent planning states, while an independent Decoder provides textual readouts when needed. Answer-aware textual stopping allows the latent rollout to use a problem-dependent number of groups rather than a pre-specified chain length. PLaT achieves competitive coverage at larger

k

in several mathematical settings, with lower Pass@1: on Llama-1B GSM8K, it reaches 80.59% Pass@128 versus CODI's 72.37%. These results support PLaT as a candidate-generation interface supplying multiple textual readouts for downstream verification or reranking.

Jiecong Wang, Hao Peng, Zhanyi Wang +2

Beihang University · Didi Chuxing

Efficient Latent Reasoning Large Language Model Planning Chain-of-Thought Reasoning Verbalization Planning

Ask PDF

May 26, 2026cs.AI

Latent Goal Prediction from Language for Model-Based Planning

Planning with world models is bottlenecked by compounding prediction errors and the difficulty of defining optimizable goals. Visual targets provide precise local gradients but poor distant guidance, while language is flexible yet limited by noisy cross-modal alignment or dependence on large generative models unsuited for the high-sampling nature of model-based planning. To address these challenges, we introduce Latent Goal Prediction from Language (LAGO), a framework that predicts both sequences of intermediate goal states from language instructions and action-conditioned rollouts, all within the same latent space. Rather than optimizing toward a single global objective, LAGO dynamically decomposes instructions into explicitly predicted, locally tractable latent subgoals. By updating these subgoals online and using a soft minimum trajectory cost during planning, LAGO enables an agent to follow coherent latent trajectories over long horizons. Evaluation across multiple environments planning horizons shows that LAGO avoids the sharp degradation of prior methods. By achieving robust and precise long-horizon planning purely from language, LAGO bridges the precision of visual goals with the flexibility of text-guided control.

Samuel Barbeau, Simon Roy, Giovanni Beltrame +2

1École de Technologie Supérieure, Montréal · 2International Laboratory on Learning Systems (ILLS) · 3Polytechnique Montréal +4

Long-Horizon Planning Latent World Models Planning Latent Prediction

Ask PDF

May 8, 2026cs.LG

Where's the Plan? Locating Latent Planning in Language Models with Lightweight Mechanistic Interventions

We study planning site formation in language models -- where internal representations of structurally-constrained future tokens form during the forward pass, and whether they causally drive generation. Using rhyming-couplet completion as a clean test of forward-looking constraint, we apply two lightweight methods (linear probing and activation patching) across Qwen3, Gemma-3, and Llama-3 at more than ten scales. Probing shows that future-rhyme information is linearly decodable at the line boundary, with signal that strengthens with scale in all three families. Activation patching reveals that only Gemma-3-27B causally relies on this encoding, exhibiting a handoff in which the causal driver migrates from the rhyme word to the line boundary around layer 30. Every other model we test conditions on the rhyme word throughout generation, with near-zero causal effect at the line boundary despite strong probe signal. We localize the Gemma-3-27B handoff to five attention heads through two-stage path patching that recover ~90% of the rhyme-routing capacity at the newline.

Nicole Ma, Nick Rui

Stanford University.

Large Language Model Planning

Ask PDF