Scope-WM: Scoped Computation for Efficient Visual World Models

Chunzheng Li; Zesheng Jia; Hongda Zhang; Jiaying Tang; Yuntian Wang; Siao Liu; Jin Wang

cs.CVSep 27, 2026

Scope-WM: Scoped Computation for Efficient Visual World Models

Authors: Chunzheng Li, Zesheng Jia, Hongda Zhang, Jiaying Tang, Yuntian Wang, Siao Liu, Jin Wang

Organizations: Soochow University · Fudan University

Abstract

Visual world models enable robotic planning by predicting future observations, but dense latent-state propagation and sample-intensive trajectory optimization incur high inference latency and peak memory usage, limiting real-time deployment on resource-constrained platforms. Existing sparse world-model acceleration methods either rely on unguided token sparsification, which may discard planning-relevant information and restrict achievable sparsity, or introduce heavy auxiliary modules and cumbersome multi-stage training pipelines. In this work, we present Scope-WM, an efficient visual world model that scopes computation to prediction-relevant latent regions and promising action sequences. Scope-WM distills prediction relevance into a lightweight action-conditioned selector and applies full dynamics prediction only to a compact subset of selected tokens. It updates the remaining tokens using a compact summary of foreground states and their changes, allowing the background to perceive foreground dynamics without costly token-to-token interactions. During planning, Scope-WM preserves and reuses high-quality action sequences discovered during the initial MPC search, focusing subsequent search under reduced rollout budgets. The resulting pipeline requires only a one-off selector distillation followed by a single joint training stage for the sparse world model. On the challenging Push-T task, Scope-WM reduces peak GPU memory usage and planning time to $18.1\%$ and $14.3\%$ of those of dense DINO-WM, respectively, corresponding to a $6.97\times$ planning speedup, while maintaining competitive task performance. Further evaluations across five diverse visual planning tasks demonstrate the general applicability of Scope-WM. Code is available at https://github.com/ChunZheng2022/Scope-WM.

Figures & tables

Figure 1: Performance–efficiency trade-off of Scope-WM on Push-T. (a) Different token budgets yield different performance–efficiency operating points under varying sparsity. (b) Scope-WM substantially reduces planning time and peak GPU memory over dense DINO-WM.

Figure 2: Overview of Scope-WM. Scope-WM scopes computation at the token, dynamics, and planning levels. At the token level, the action-conditioned DRS selector identifies a compact set of prediction-relevant foreground tokens from the encoded visual state. At the dynamics level, the sparse world-model predictor applies full dynamics prediction to the selected tokens, while FDBU uses a summarized foreground-dynamics context to update the remaining background tokens through a lightweight gated residual update, producing the complete latent rollout. At the planning level, EB-CEM performs test-time action optimization by retaining and reusing high-quality elite action sequences across MPC steps. Candidate action sequences are evaluated by a planning objective that compares the predicted latent rollout with the goal latent encoded by the frozen visual encoder.

Method	Full-dyn. tokens	SR ( $\uparrow$ )	FLOPs (G, $\downarrow$ )	Plan time (s, $\downarrow$ )	Speedup ( $\uparrow$ )	Peak Mem. (MB, $\downarrow$ )	Mem. Ratio ( $\downarrow$ )
DINO-WM (Dense)	196	90%	31.89	240.96	$1.00\times$	18436	100.0%
Scope-WM (Fast)	98	94%	15.04	93.06	$2.59\times$	7088	38.4%
Scope-WM (Faster)	32	88%	5.27	34.56	$\mathbf{6.97}\times$	3334	18.1%

Table 1: End-to-end performance–efficiency trade-off on Push-T. Fast and Faster apply full dynamics prediction to

98

and

32

196

visual tokens. FLOPs are reported per prediction sample, with memory ratios normalized to DINO-WM.

Method	PointMaze (SR $\uparrow$ )	Push-T (SR $\uparrow$ )	Wall (SR $\uparrow$ )	Rope (CD $\downarrow$ )	Granular (CD $\downarrow$ )
IRIS	74%	32%	4%	1.11	0.37
DreamerV3	100%	30%	100%	2.49	1.05
Sparse Imagination	100%	78.3%	95%	–	–
DINO-WM	98%	90%	96%	0.41	0.26
Scope-WM (Fast, $K=98$ )	100%	94%	92%	0.87	0.22
Scope-WM (Faster, $K=32$ )	98%	88%	92%	0.71	0.25

Table 2: Cross-task planning performance. SR (

\uparrow

) is reported for PointMaze, Push-T, and Wall, and CD (

\downarrow

) for Rope and Granular. Fast and Faster use

98

and

32

full-dynamics tokens.

Variant	Selection	BG update	Planner	SR ( $\uparrow$ )
w/o DRS	Random	FDBU	EB-CEM	76%
w/o FDBU	DRS	Copy	EB-CEM	56%
w/o EB-CEM	DRS	FDBU	CEM	82%
Scope-WM	DRS	FDBU	EB-CEM	88%

Table 3: Component ablation on Push-T under the

32

-token full-dynamics setting used by Scope-WM (Faster).

Figure 3: Effect of the full-dynamics token budget on Push-T. Bars show planning time and peak GPU memory normalized to dense DINO-WM, with absolute values annotated above the bars. The line shows success rate as the token budget varies across configurations.

Figure 4: Qualitative analysis of Scope-WM on Push-T. (a) DRS focuses computation on prediction-relevant regions. (b) FDBU captures background evolution without dense token-to-token interactions. (c) EB-CEM reuses promising early trajectories for subsequent replanning.

Selection	Background update
	Copy	FDBU
Random	72%	76%
DRS	56%	88%

Table 4: Interaction between DRS and FDBU on Push-T under the same 32-token budget and EB-CEM planner. Entries report success rate (SR,

\uparrow

Model	Planner	Samples	SR ( $\uparrow$ )	Plan time (s, $\downarrow$ )
DINO-WM	CEM	100/100	90%	240.96
DINO-WM	EB-CEM	300/100	84%	267.36
Scope-WM	CEM	100/100	82%	30.67
Scope-WM	CEM	200/200	88%	60.96
Scope-WM	EB-CEM	300/100	88%	34.56

Table 5: Interaction between sparse prediction and planning strategy on Push-T. Scope-WM uses the 32-token setting; sample budgets denote initial/subsequent MPC steps.

	CEM			EB-CEM
Early samples	100	100	300	300
Later samples	100	300	100	100
SR ( $\uparrow$ )	82%	82%	86%	88%

Table 6: Effect of sample allocation and elite reuse on Push-T. Early and later denote initial and subsequent MPC steps.

Appendix figures & tables4 assets

Supplementary material from the paper’s appendix.

Appendix

Component	Hyperparameter	Setting
Visual representation	Encoder	Frozen DINOv2 ViT-S/14
	Input resolution	$224\times 224$
	Visual tokens / token dimension	$196/384$
Dynamics predictor	Architecture	6-layer causal Transformer
	Attention heads / head dimension	$16/64$
	Feed-forward hidden dimension	$2048$

Appendix

Table S1: Shared model and optimization settings. LN denotes layer normalization.

Task	History	Frameskip	Global batch	Training GPUs	Epoch budget	Train $K$	Eval. instances	Max MPC steps
PointMaze	3	5	512	8 V100	$\leq 100$	98 / 32	50	15
Wall	1	5	512	8 V100	$\leq 100$	98 / 32	50	15
Push-T	3	5	512	8 V100	$\leq 100$	98 / 32	50	15
Rope	1	1	64	1 RTX 2080 Ti	100	98 / 32	10	7
Granular	1	1	64	1 RTX 2080 Ti	100	98 / 32	10	7

Appendix

Table S3: Task-specific training and evaluation settings. Fast and Faster are independently trained with

K=98

and

K=32

, respectively.

Task	Method	$K$	Plan time (s, $\downarrow$ )	Speedup ( $\uparrow$ )	Peak Mem. (MB, $\downarrow$ )	Mem. Ratio ( $\downarrow$ )	SR ( $\uparrow$ )
PointMaze	DINO-WM	196	204.52	$1.00\times$	6309	100.0%	98%
PointMaze	Scope-WM (Faster)	32	35.01	$\mathbf{5.84}\times$	3334	52.8%	98%
Wall	DINO-WM	196	59.44	$1.00\times$	1542	100.0%	96%
Wall	Scope-WM (Faster)	32	19.60	$\mathbf{3.03}\times$	1939	125.8%	92%
Push-T	DINO-WM	196	240.96	$1.00\times$	18436	100.0%	90%
Push-T	Scope-WM (Faster)	32	34.56	$\mathbf{6.97}\times$	3334	18.1%	88%

Appendix

Table S5: End-to-end efficiency on three rigid-body planning tasks. Time and peak memory use one fixed full-length run; SR uses the complete

50

-instance set. Memory ratios are normalized per task, and the peak includes the larger initial EB-CEM population.

Figure S1: Environment and decoded latent rollouts on Push-T. The upper row executes the planned actions in the environment; the lower row decodes the corresponding Scope-WM latent rollout with the dense DINO-WM decoder. Steps proceed left to right, and the highlighted final column is the goal. The decoder is used only for visualization.

World Models for Robotics

World models for robotics predict how scenes and robot states change under actions to support planning, control, and learning.

See all 369 papers

Fast LeWorldModel
Jun 24, 2026Yuntian Gao, Xiangyu XuLatent Dynamics Modeling World Model-Based Planning
Xi’an Jiaotong University
Pretrained Features as Teachers, Not States: Compact Latent World Models for Visual Planning
Oct 14, 2026Yanbo Wang, Nan Lin, Wendelin Böhmer +1World Model-Based Planning Latent World Models
Delft University of Technology (TU Delft)
SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation
Oct 13, 2026Yifei Yang, Zehua Fan, Zike Yan +2Visual Navigation World Model-Based Planning
Institute for AI Industry Research (AIR), Tsinghua University · Hong Kong Embodied AI Lab · The Chinese University of Hong Kong +1

Jun 24, 2026J/K move · Enter open · S save

Fast LeWorldModel

Yuntian Gao, Xiangyu Xu

Xi’an Jiaotong University

Latent Dynamics Modeling World Model-Based Planning Joint-Embedding Predictive Architecture Action-Conditioned World Models Efficient Inference for World Action Models Model-Based Planning+1

Open paper AskPDF arXiv

Joint-Embedding Predictive Architectures (JEPAs), including recent LeWorldModel (LeWM), have become a promising foundation for reconstruction-free visual world models. For visual planning, however, LeWM evaluates candidate action sequences by repeatedly applying a local one-step latent transition model. This autoregressive rollout makes planning computationally expensive and exposes the predicted trajectory to accumulated latent errors as the horizon grows. We propose Fast LeWorldModel (Fast-LeWM), a fast latent world model that replaces repeated local rollout with action-prefix prediction. Given the current latent and a candidate action sequence, Fast-LeWM encodes its prefixes and predicts the future latents reached after executing those prefixes in parallel. By making action prefixes the basic prediction unit, Fast-LeWM directly models action effects accumulated to different extents over multiple horizons. This prefix-level supervision forces the model to learn how states continuously evolve under different action prefixes, rather than only fitting one-step state transitions. During planning, the predictor can use the prefix token from the encoded action sequence to evaluate the corresponding future latent without explicitly rolling through each intermediate imagined state. Across multiple tasks, Fast-LeWM improves average success over LeWM while substantially reducing planning time, achieving lower open-loop latent loss whose growth becomes significantly slower as the rollout horizon increases.

Abstract

World Models for Robotics

Explore similar work

Fast LeWorldModel

Pretrained Features as Teachers, Not States: Compact Latent World Models for Visual Planning

SkipWorld: Latent Skip Planning with Distilled World Models for Efficient Visual Navigation