LOCI: Spatial Linear Memory for Streaming World Models

Ji Xia; Tingting Liao; Xuezhi Liang; Hao Li; Guangyi Liu

cs.CVSep 30, 2026

LOCI: Spatial Linear Memory for Streaming World Models

Authors: Ji Xia, Tingting Liao, Xuezhi Liang, Hao Li, Guangyi Liu

Organizations: Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence · Mohamed bin Zayed University of Artificial Intelligence · Pinscreen

Abstract

When a camera revisits a previously observed region, a video world model should reproduce what was there before. This requires both remembering past observations and retrieving the right one for the current viewpoint. Key-value caches preserve visual detail but grow with video length; recurrent memory is compact but compresses history into a fixed-size state, so individual past observations are no longer directly accessible. We introduce LOCI, a hybrid spatial-memory architecture that keeps both representations. In half of the transformer blocks, main attention keeps a key-value cache of past observations; in the other half, it is restricted to the current chunk and complemented by a recurrent linear-attention memory whose reads and writes are conditioned on projective camera geometry, so viewpoint enters both memory addressing and stored content. Recurrent readouts flow into subsequent cache-backed blocks and supply their queries with accumulated scene context. On the public MIND memory benchmark and on held-out recorded trajectories, LOCI reproduces revisited content more faithfully than representative world models and a same-recipe full-softmax model; with full history, it lowers peak memory at equal length by about 30% relative to full softmax. With a bounded bank of retained observations, it streams long videos at constant memory and remains more faithful than full softmax under the same budget.

Figures & tables

Figure 1: Returning to a previously seen place. Each model continues the recorded video (top: full history; bottom: held-out Tokyo map, same bounded KV budget). Full softmax, trained with the same recipe, replaces the tree with a facade and loses the building; Loci restores both (PSNR against ground truth; insets magnify the boxed region). Band: a fixed-size recurrent state shapes the queries into the retained KV (thumbnails: frames the Tokyo run retains).

Figure 2: Key idea of Loci . Projectively conditioned recurrent memory integrates historical context, while retained KV preserves observation-level detail. Recurrent readouts update the features used by subsequent historical attention, connecting the two representations through the feature stream.

Figure 3: Conceptual illustration of Loci . A camera-conditioned recurrent state accumulates context from past observations; at a revisit, its readout shapes the queries with which later historical-attention blocks attend to co-visible retained observations. The layer layout is given in Appendix A.1 .

Method	Params	Trained on MIND	$n$	MSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
Reference (reported in the GIM-World paper)
SSM ( Po et al., 2025 )	–	✓	–	0.0796	11.96	0.395	0.744
FramePack ( Zhang et al., 2025 )	–	✓	–	0.0764	12.04	0.376	0.706
Context-as-Memory ( Yu et al., 2025 )	–	✓	–	0.0706	12.50	0.392	0.695
GIM-World ( Wei et al., 2026 )	1.3B	✓	–	0.0614	13.40	0.414	0.630
Run by us (entire prediction segment)

Table 1: MIND memory test (first-person). Reference : numbers reported in the GIM-World paper, also evaluated on the entire prediction segment (not re-run by us). Run by us : all models evaluated identically on the entire prediction segment with the official evaluation code; external models use official weights and default sampling, with the memory segment provided as context.

n

: number of scored segments (Matrix-Game 3.0: one segment failed in the official scorer; LingBot-World: 28B in total, 14B active). ‡ CaR: the 36 shortest segments; generation of the 14 longest did not finish (on these 36 segments Loci obtains 0.0460 / 14.39 / 0.478 / 0.630). Bold: best among models run by us on all segments.

Figure 4: Qualitative revisit comparison on held-out recorded trajectories (first frame and camera trajectory as input for all methods). Each row is one model; columns show the first visit, a view after the camera has turned away, and the return to the first viewpoint, with the PSNR of the return frame against ground truth. Loci restores the previously observed scene on return, whereas the other models replace or distort it. For each clip we show the return at which Loci leads by the largest margin; clip-level averages are in Table 6 . Temple Plaza is a training map traversed along a new trajectory; the Tokyo map is not used in training.

Data mixture	History	$n$	Softmax	Loci	$\Delta$ [95% CI]	Softmax	Loci	$\Delta$ [95% CI]
			PSNR $\uparrow$			LPIPS $\downarrow$
Uniform	bounded	50	12.17	12.85	$+0.67$ [ $+0.42$ , $+0.93$ ]	0.700	0.680	$-0.020$ [ $-0.028$ , $-0.011$ ]
UE-weighted	bounded	50	12.13	13.02	$+0.89$ [ $+0.64$ , $+1.15$ ]	0.704	0.679	$-0.024$ [ $-0.034$ , $-0.015$ ]
UE-weighted	full	32	13.90	14.25	$+0.35$ [ $+0.08$ , $+0.62$ ]	0.669	0.641	$-0.029$ [ $-0.040$ , $-0.018$ ]

Table 2: Hybrid vs. full softmax with the same backbone, data, recipe and 5,000 updates on the MIND memory test (entire prediction segment;

\Delta

is the paired mean over segments, Loci

-

full softmax, with 95% bootstrap intervals). Bounded: both models access the same fixed set of retained frames (first frame, bank of 20, 8 most recent). Full history: the 32 segments completed by both models (full softmax, run on the 39 shorter segments, exceeds GPU memory on 7 of them).

Figure 5: Long-range recall under the same bounded KV budget. Both models receive the recorded video up to the stated time as history and generate the rest; shown are returns to places first seen more than 60 s earlier, with the PSNR of the generated return frame against the corresponding ground truth. Full softmax, which can only attend to the retained frames, replaces the scene (new buildings, a tree), whereas Loci restores it. We show three of the four trajectories, each at the return where Loci leads by the largest margin.

Figure 6: Linear readout of camera yaw. Left three panels: true yaw and readouts from the KDA state, the mean of past tokens, and the current chunk’s input features (before camera conditioning) on example trajectories. Right: held-out

R^{2}

(15-layer mean) for the main model (left group) and, on the uniform-mixture branch with an identical recipe, the recurrent memory with PRoPE (middle) and without camera encoding (right); dotted lines show the maximum over 20 label permutations.

Appendix figures & tables14 assets

Supplementary material from the paper’s appendix.

Appendix

Mode	Recurrent state	Main KV	Camera KV
Full softmax, dense	None	$30\times T$	$30\times T$
Loci , dense	15 fixed layer states	$15\times T$	$30\times T$
Full softmax, sparse	None	$30\times\min(T,34)$	$30\times\min(T,34)$
Loci , sparse	15 fixed layer states	$15\times\min(T,34)$	$30\times\min(T,34)$

Appendix

Table 3: Logical history storage. KV entries denote layers

\times

retained latent-frame support, not bytes; sparse entries are upper bounds. Each recurrent layer contains a fixed set of per-head matrices. These are architecture counts, not measured device-memory ratios.

Method	Rot. ( ∘ ) $\downarrow$	Trans. ( ∘ ) $\downarrow$	Mag. $\to$ 1
CaR ( Peng et al., 2026 )	2.14	8.0	0.74
HY-WorldPlay ( Sun et al., 2026 )	2.75	9.6	1.38
Matrix-Game 3.0 ( Wang et al., 2026 )	1.33	6.8	2.09
LingBot-World ( Robbyant Team et al., 2026 )	0.72	4.9	1.06
AlayaWorld ( AlayaWorld Team et al., 2026 )	7.49	—	—
SANA-WM ( Zhu et al., 2026 )	1.71	4.5	3.02

Appendix

Table 4: Camera following on the designed paths of set C (medians). Mag.: generated / commanded motion magnitude. —: the translation scale of AlayaWorld’s outputs could not be calibrated.

Comparison	Quantity	Value [95% CI]	Count
Loci vs. full softmax, full history	co-visible attention share	11.1% vs. 10.45%
	$\Delta$ enrichment	$+6.64$ [ $+3.53$ , $+8.45$ ]	8 / 8 clips
Loci vs. full softmax, bounded	co-visible attention share	9.66% vs. 8.91%
	$\Delta$ enrichment	$+6.82$ [ $+4.67$ , $+7.72$ ]	8 / 8 clips
Readout on $-$ off ( Loci )	$\Delta$ enrichment, downstream softmax layers	$+0.56$ [ $+0.21$ , $+0.77$ ]
	$\Delta$ enrichment, right after a hybrid block	$+0.86$ [ $+0.45$ , $+1.04$ ]

Appendix

Table 5: Co-visible attention and reconstruction (8 recorded clips, 154 revisit instants; 95% bootstrap intervals over clips). Enrichment: co-visible share of history attention divided by co-visible share of historical tokens.

Method	Params	Steps	Revisit (A+B)		Whole clip (A)			Identity	Cost
Method	Params	Steps	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$	DINO $\uparrow$	s/v-s $\downarrow$	GiB $\downarrow$
CaR ( Peng et al., 2026 )	5B	50	9.73	0.627	8.89	0.360	0.647	0.679	31.4	17.7
HY-WorldPlay ( Sun et al., 2026 )	8B	4	10.01	0.640	9.70	0.408	0.625	0.519	14.7	71.1
Matrix-Game 3.0 ( Wang et al., 2026 )	5B	3	8.40	0.687	7.85	0.313	0.678	0.626	6.2	35.0
LingBot-World ( Robbyant Team et al., 2026 )	28B	4	8.70	0.605	9.67	0.442	0.564	0.517	20.5	88.3
AlayaWorld ( AlayaWorld Team et al., 2026 )	15B	4	8.52	0.646	8.34	0.386	0.640	–	24.9	137.6

Appendix

Table 6: Comparison with camera-controlled world models. Revisit: generated vs. ground-truth frames at the revisit instants of 20 clips (16 from set A, 4 from set B). Whole clip: all frames of set A (18 clips). Identity: DINOv2 cosine between revisit and earlier views (set B). Camera following is checked in Appendix B . Ours (full history and bounded sparse) and full softmax average three seeds on set A. Cost: s/v-s is seconds per video second; for our models and full softmax, measured over the first 40 s of a 300 s trajectory on one H200 with the output-preserving implementation of Appendix F . Best in bold .

Versus	$\Delta$ PSNR $\uparrow$		$\Delta$ LPIPS $\uparrow$		Ours lower
Versus	Mean	95% CI	Mean	95% CI	Ours lower
Full softmax (same recipe)	+0.62	[+0.13, +1.12]	+0.018	[ $-$ 0.006, +0.042]	14 / 20
HY-WorldPlay ( Sun et al., 2026 )	+0.63	[ $-$ 0.49, +1.71]	+0.079	[+0.037, +0.122]	14 / 20
CaR ( Peng et al., 2026 )	+0.91	[ $-$ 0.10, +2.16]	+0.066	[+0.007, +0.128]	13 / 20
SANA-WM ( Zhu et al., 2026 )	+1.02	[+0.03, +2.04]	+0.210	[+0.146, +0.276]	19 / 20
LingBot-World ( Robbyant Team et al., 2026 )	+1.95	[+0.96, +2.99]	+0.044	[ $-$ 0.027, +0.112]	12 / 20

Appendix

Table 7: Paired revisit comparison. Loci minus the other method on the same 20 clips, with 95% bootstrap intervals;

\Delta

LPIPS is other

-

ours, so positive favors Loci . Ours lower counts clips with lower LPIPS for Loci . Bold: interval excludes zero.

Method	$n$	MSE $\downarrow$	PSNR $\uparrow$	SSIM $\uparrow$	LPIPS $\downarrow$
HY-WorldPlay ( Sun et al., 2026 )	50	0.0672	12.84	0.411	0.695
Matrix-Game 3.0 ( Wang et al., 2026 )	49	0.0737	12.51	0.387	0.635
AlayaWorld ( AlayaWorld Team et al., 2026 )	50	0.0815	11.97	0.379	0.659
Alaya-EVOKE ( Yin et al., 2026 )	50	0.0814	11.65	0.369	0.687
LingBot-World ( Robbyant Team et al., 2026 )	50	0.0855	11.96	0.363	0.669
Loci (ours)	50	0.0478	14.12	0.454	0.635

Appendix

Table 8: MIND memory test, first 20 s of the prediction segment. Same models, outputs and scorer as Table 1 .

Method	Memory-segment feed	Control signal	Res. / fps	Steps / CFG
HY-WorldPlay ( Sun et al., 2026 )	Retrieval-augmented KV-cache: memory GT frames VAE-encoded to clean latents and written into the latent tensor in place of the noise-initialized first chunk; later chunks retrieve by camera-FOV overlap over the full latent history (avg. 1141 frames, range 255–2406)	Continuous GT pose per predicted frame (no WASD discretization)	832 $\times$ 480, 24 fps	4 / none
Matrix-Game 3.0 ( Wang et al., 2026 )	Long-range FOV-retrieval bank ( x_memory , selected from the official AR history tensors) plus a 16-frame local re-diffusion window; memory GT frames VAE-encoded to clean latents and prefilled into the same history tensors (avg. 1144 frames, range 255–2406)	Discrete WSAD + mouse actions, quantized from MIND’s continuous GT poses (12.35 cm/frame translation; yaw $\pm 1.5^{\circ}$ /frame)	1280 $\times$ 704 native (17 fps) $\to$ 1280 $\times$ 720 at 24 fps for scoring	3 / none
LingBot-World ( Robbyant Team et al., 2026 )	Rolling self-attention KV-cache (sink 6 + recent 12 latent frames); the full memory segment is causally VAE-encoded chunk by chunk and written with the same clean-latent ( $t{=}0$ ) forward pass the model uses for its own history, but only the sink ( $\approx$ 1.3 s) and tail ( $\approx$ 3 s) remain in the window	Continuous GT pose via Plücker camera embedding	832 $\times$ 464, 16 fps	4 / none
AlayaWorld ( AlayaWorld Team et al., 2026 )	Sink (1 frame) + 4-latent history window + 9-frame RGB motion neighbors from the last 25 memory frames, plus the entire memory segment written into a ViGeo spatial point-map bank (top-10 frames retrieved per step by target-pose coverage)	Continuous GT camera pose (OpenCV c2w)	544 $\times$ 960, 24 fps	4 / 3.0
Alaya-EVOKE ( Yin et al., 2026 )	World-state-library prefill (v2v mode): the entire memory segment and its GT poses are fed to a persistent point-cloud world-state library (per-chunk monocular-depth back-projection); read = pose-indexed retrieval, top-8 covisible sources, z-buffer warp	Continuous GT camera pose	384 $\times$ 640, 24 fps	3 / none (guidance 1.0)
CaR ( Peng et al., 2026 )	Context frames: the entire memory segment (arbitrary length) is VAE-encoded and temporally resampled onto the memory-token grid by the official memory encoder	Continuous GT camera pose	480 $\times$ 832, 24 fps	50 / 3.0

Appendix

Table 9: How each external baseline receives the MIND memory segment. Res. / fps: resolution and frame rate of the official pipeline’s own generation. Steps / CFG: sampling steps and classifier-free guidance scale used (distilled checkpoints run CFG-free). No official inference code was modified: every baseline runs through an adapter that subclasses or wraps it unmodified.

	Task / conditioning	Softmax access	Recurrent path	Training	Memory evaluation
Video SSM ( Po et al., 2025 )	actions (Memory Maze, Minecraft)	dense local frame attention	block-wise state-space scan (per the authors, trading spatial consistency for memory)	trained on game data	spatial retrieval on synthetic scenes
Hybrid Forcing ( Li et al., 2026b )	text-to-video	local window	additive linear state of evicted KV	dense-to-hybrid distillation	–
ARL 2 ( Li et al., 2026a )	text-to-video, no camera	current frame (hybrid layers); causal softmax (others)	gated delta rule with per-token gates	conversion by layer-output and velocity distillation	VBench
SANA-WM ( Zhu et al., 2026 )	single image + camera path	attention sink + local window	Gated DeltaNet, one recurrent step per latent frame; UCPE camera encoding	multi-stage training	revisit vs. another generated frame
Loci (ours)	memory segment or first frame + camera path	all retained observations (full, or first + bank of 20 + 8 most recent)	KDA, token-wise writes, retention once per chunk, reads from the preceding state; PRoPE on $q,k,v$	direct diffusion-forcing training from Wan2.2	revisit vs. recorded ground truth

Appendix

Table 10: Hybrid recurrent video models. How each model is conditioned, which history its softmax attention reaches, what its recurrent path is, and how memory is trained and evaluated.

#	Model	Score	#	Model	Score
1	HY-World 1.5 ( Sun et al., 2026 )	84.92	12	Wan 2.7	71.00
2	Loci (ours)	82.00	13	LTX 2.3	70.17
3	Matrix-Game 3.0 ( Wang et al., 2026 )	80.37	14	LingBot-World ( Robbyant Team et al., 2026 )	67.14
4	Genie 3	78.39	15	InSpatio-World	66.47
5	Happy Oyster	75.83	16	LongCat-Video	66.23
6	Kling 3.0	75.14	17	Matrix-Game 2.0	64.51

Appendix

Table 11: WBench-Navi gated camera-return consistency (camera-controlled navigation cases whose trajectory returns to an earlier view; higher is better). External models: the models evaluated in the WBench paper, with values from its official leaderboard. Loci : official evaluation code on the same cases.

Model	History	Reached	60 s	120 s	180 s	Max
Full softmax	full	OOM at 157 s	63.6	102.8	—	128.8
Loci	full	OOM at 225 s	45.5	69.0	100.2	114.9
Full softmax	sparse	300 s	27.4	27.5	27.5	27.6
Loci	sparse	300 s	23.4	23.4	23.5	23.6

Appendix

Table 12: Peak GPU memory versus generated length (300 s trajectory, one H200; peak allocated, GiB).

	Revisit		Whole clip		Peak memory
Model, history access	PSNR $\uparrow$	LPIPS $\downarrow$	PSNR $\uparrow$	LPIPS $\downarrow$	(GiB, grows / constant)
Full softmax, full history	10.16	0.576	10.18	0.587	grows (OOM at 157 s)
Loci , full history	10.67	0.544	10.36	0.577	grows (OOM at 225 s)
Full softmax, bounded sparse	10.15	0.570	10.07	0.578	27.6, constant
Loci , bounded sparse	11.14	0.524	10.49	0.563	23.6, constant

Appendix

Table 13: Bounded sparse access vs. full history on the recorded set A (16 clips with revisit instants; generated vs. ground truth; seed 42). Sparse: first frame, a bank of 20 retained latents and the 8 most recent latents (identical for both models), with the recurrent state of Loci carried over the full history.

Figure 7: Localized consistency at short-horizon revisits. Earlier ground-truth view, ground-truth revisit, and the revisit generated by Loci and by full softmax (same recipe); outlines mark each model’s worst 5% of patches by DINOv2 distance to the ground truth, and the right column zooms into the region where the two models differ most. Instants are chosen by a fixed rule (bright scenes, ranks 1, 3, 5 of the paired difference).

Revisit gap	Segments	$\Delta$ local error	Chunk lower
8–20 s	26	$+0.0045$ [ $-0.0013$ , $+0.0103$ ]	17 / 26
20–60 s	44	$\mathbf{+0.0101}$ [ $+0.0039$ , $+0.0169$ ]	28 / 44
$>$ 60 s	35	$\mathbf{+0.0126}$ [ $+0.0038$ , $+0.0210$ ]	27 / 35
All	50	$\mathbf{+0.0084}$ [ $+0.0035$ , $+0.0133$ ]	35 / 50

Appendix

Table 14: Per-token minus per-chunk retention : local DINOv2 error at MIND revisit instants over the entire prediction segment, by revisit gap (positive: chunk-level lower; paired over segments, 95% bootstrap intervals).

Figure 8: Cost versus generated length on a 300 s recorded trajectory (single H200, 141 GiB). Peak GPU memory. With full history both models grow until they exhaust the GPU; bounded sparse access keeps memory and per-chunk time flat to 300 s.

Explore similar work

Addressable Memory for Video World Models
Aug 7, 2026Xindi Wu, Sven Elflein, James Lucas +5Visual Memory Video World Models
1NVIDIA · 2Princeton University · University of Toronto +1
Memorizon: Training World Models Beyond Their Context Window
Sep 30, 2026Tingting Liao, Xuezhi Liang, Hao Li +1Memory-Augmented Framework Long-Term Memory
IFM, MBZUAI · MBZUAI
Compression and Retrieval: Implicit Memory Retrieval for Video World Models
Jun 22, 2026Zhan Peng, Jie Ma, Huiqiang Sun +6Video World Models Long-Term Memory
Huazhong University of Science and Technology, China · HUJING Digital Media & Entertainment Group, China · Sun Yat-sen University, China

Aug 7, 2026, cs.CVJ/K move · Enter open · S save

Addressable Memory for Video World Models

Xindi Wu, Sven Elflein, James Lucas, Olga Russakovsky+4

1NVIDIA · 2Princeton University · University of Toronto+1

Visual Memory Video World Models Peak Memory World Models Rotary Position Embedding

Open paper AskPDF arXiv

We study visual persistence in interactive video world models. These models rely on a Key-Value (KV) cache as a growing visual memory to carry forward previously generated frames. However, we find that models can no longer reliably address stored content once rollouts extend beyond the training horizon, because temporal Rotary Positional Embeddings (RoPE) offsets then fall outside the range seen during training and the model struggles to retrieve the relevant visual information through attention. Moreover, naively compressing the cache in the RoPE-rotated space corrupts memory by averaging together incompatible positional phases. To address this, we propose WorldTrace, a training-free memory framework for long-horizon visual persistence. WorldTrace keeps compressed memory addressable by assigning each summary slot a distinct, in-distribution virtual position. Within this addressable cache, we study two memory compression approaches: WorldTrace-Field compresses history for temporal coherence, while WorldTrace-Landmark stores verbatim scene traces at detected transitions for episodic recall. We further introduce LoopBench, a benchmark evaluating whether a compressed cache can reconstruct a previously visited scene after a long detour. WorldTrace-Field improves temporal consistency by +15.5%, and WorldTrace-Landmark improves episodic recall by +19.5% on LoopBench, extending visually persistent generation without retraining.

Abstract

Explore similar work

Addressable Memory for Video World Models

Memorizon: Training World Models Beyond Their Context Window

Compression and Retrieval: Implicit Memory Retrieval for Video World Models