GLIMPSE: Global-Local Indexing with Multi-Precision Spectral Encoding for Explorable Video World Models

Zhufeng Xu; Henglei Lv; Bailin Deng; Yikang Ding; Xiaoqiang Liu; Guoxin Zhang; Hongbo Fu; Lin Gao

cs.CVOct 9, 2026

GLIMPSE: Global-Local Indexing with Multi-Precision Spectral Encoding for Explorable Video World Models

Authors: Zhufeng Xu, Henglei Lv, Bailin Deng, Yikang Ding, Xiaoqiang Liu, Guoxin Zhang, Hongbo Fu, Lin Gao

Organizations: Institute of Computing Technology, Chinese Academy of Sciences · University of Chinese Academy of Sciences · Cardiff University · Kling AI ·

Abstract

A camera-controlled video world model must keep a scene consistent when the camera returns to it. Memories that render stored features into the target frame can turn depth and pose errors into ghosting or duplicated structure. Positional transport avoids that resampling by relocating a historical patch's address rather than its content, but assigns full positional precision even to inaccurate projections. We present GLIMPSE, a global-local memory interface with band-wise multi-precision coordinates and geometry-conditioned spectral attenuation to reduce sensitivity to inaccurate projected addresses. Local patch memory supplies view-specific appearance, while recurrent global memory provides scene context. Footprint-guided retrieval selects historical patches under a fixed local read budget. Under matched retrieval and read budgets, GLIMPSE improves geometric correspondence and photometric consistency over deterministic positional transport.

Figures & tables

Figure 1: Camera-controlled scene exploration with GLIMPSE. From top to bottom: forward motion, left/right observation followed by backward motion, and free exploration over 81, 241, and 721 frames. Each row shows the requested camera trajectory on the left and five generated frames in temporal order. Insets highlight the corresponding camera pose, and colors indicate progression along the trajectory.

Figure 2: Motivation for robust positional addressing. Left: the coordinate supplied to RoPE against the continuous projected coordinate, where the high and mid bands hold a cell and a half-cell value, so a sub-cell error inside one bin changes nothing, while the low bands keep the continuous offset. Middle: the spectral multiplier $\gamma_{k}$ against frequency, where PE warp keeps every band at full amplitude, whereas Spatio-Temporal Aligned Positional Encoding (STaPE) attenuates the highest frequencies first, more strongly as $\sigma$ grows. Right , in reading order: the reference view, latent value warping, PE warping, and STaPE under the same four-patch address perturbation. Value warping drops the doorway and PE warping duplicates wall art, whereas STaPE preserves both.

Figure 3: Overview of GLIMPSE. Geometry-guided greedy retrieval selects local context from the source history. STaPE aligns the positions of retrieved latent tokens with the requested target view, while recurrent global context provides complementary scene memory to the video generation model.

Table 4

Figure 4: Camera-trajectory replay on two scenes. The enlarged source image is followed by GT, GLIMPSE, UCM, ReWorld, SPMem, and GEN3C (left to right), with two selected time steps per scene. Insets show the requested camera trajectory and current pose.

Table 6

Figure 5: Y-traj. comparisons on house and street scenes. References and trajectories appear on the left, with left-yaw, right-yaw, and backward views shown top to bottom for each method. Red boxes highlight inconsistencies.

Configuration	Multi- precision	Spectral attenuation	Mat. Pix.@5 (K) $\uparrow$	Inliers@5 (K) $\uparrow$	Inlier rate@5 (%) $\uparrow$	RGB MAE@5 $\downarrow$	Mean rot. ( ∘ ) $\downarrow$	Trans. NRMSE $\downarrow$	CLIP-T $\uparrow$	CLIP-F $\uparrow$
Latent/value warping	N/A	N/A	6.410	5.530	86.271	0.121	1.27	0.270	27.08	96.11
PE warp	Off	Off	6.407	5.690	88.002	0.135	1.30	0.213	29.314	97.43
Only multi-precision	On	Off	6.567	5.707	87.656	0.122	1.19	0.210	29.618	97.71
Only attenuation	Off	On	6.917	6.097	88.145	0.120	1.25	0.212	29.780	98.57
Full STaPE	On	On	7.153	6.480	90.587	0.109	1.16	0.204	30.089	99.13

Table 5: STaPE component ablation. Multi-precision coordinates and spectral attenuation are varied independently while footprint retrieval, local token budget, and GRM remain fixed. The latent/value-warping control resamples historical features onto the target grid and uses neither component. Both ablation tables use the same separate evaluation set with several trajectory types, including Y-traj. Rotation error is in degrees.

Configuration	GRM	LPM	Mat. Pix.@5 (K) $\uparrow$	Inliers@5 (K) $\uparrow$	Inlier rate@5 (%) $\uparrow$	RGB MAE@5 $\downarrow$	Mean rot. ( ∘ ) $\downarrow$	Trans. NRMSE $\downarrow$	CLIP-T $\uparrow$	CLIP-F $\uparrow$
No memory	Off	Off	5.393	4.693	87.021	0.183	1.27	0.316	29.771	97.50
Global only	On	Off	5.743	5.200	90.540	0.152	1.20	0.304	29.939	98.49
Local only	Off	On	6.637	6.007	90.507	0.133	1.26	0.298	30.012	97.99
Global + local	On	On	7.153	6.480	90.587	0.109	1.16	0.204	30.089	99.13

Table 6: Global recurrent (GRM) and local pointer (LPM) memory ablation. Source, text, and camera conditions are retained. LPM uses full STaPE at a fixed read budget, with the protocol of Table 5 .

Appendix figures & tables7 assets

Supplementary material from the paper’s appendix.

Appendix

Configuration	Future-as- memory	Synthetic revisit	Fast motion	Mat. Pix.@5 (K) $\uparrow$	Inliers@5 (K) $\uparrow$	RGB MAE@5 $\downarrow$	Mean rot. ( ∘ ) $\downarrow$	Trans. NRMSE $\downarrow$	CLIP-T $\uparrow$	CLIP-F $\uparrow$
All augmentations	On	On	On	7.153	6.480	0.109	1.16	0.204	30.089	99.13
No future-as-memory (causal only)	Off	On	On	6.851	5.918	0.134	1.53	0.288	28.172	97.12
No synthetic revisit	On	Off	On	6.929	6.020	0.147	1.33	0.273	28.090	97.03
No fast motion	On	On	Off	7.011	6.220	0.119	1.78	0.312	29.100	96.90
No augmentation	Off	Off	Off	6.629	5.644	0.162	1.90	0.338	27.936	96.02

Appendix

Table 7: Training-augmentation ablation. Each row retrains the complete system with the indicated augmentations and is evaluated on the same 100-case set under the same causal inference protocol. The causal-only variant isolates the contribution of training-time future-as-memory. Column definitions follow Table 5 .

Table 12

$s$	Addressing	Mat. Pix.@5 (K) $\uparrow$	Inliers@5 (K) $\uparrow$	RGB MAE@5 $\downarrow$
0	PE warp	6.407	5.690	0.135
	STaPE	7.153	6.480	0.109
0.25	PE warp	6.043	5.205	0.159
	STaPE	7.100	6.311	0.120
0.50	PE warp	6.019	4.897	0.183
	STaPE	6.894	6.242	0.139

Appendix

Table 10: Address perturbations with standard deviation

s

in patch-grid units.

Table 14

Table 15

Figure 7: Additional Y-traj. comparisons in a living room, mountain village, and urban plaza, following the layout of Figure 5 . Each scene shows left-yaw, right-yaw, and backward views across six methods.

Figure 8: DA3 reconstructions of generated videos, colored by generated RGB. Methods share reconstruction settings, source-depth alignment, and a common rendering viewpoint and display scale per scene. We sample frames 8–80 at stride 8 for the hall and pool, and 16–224 at stride 16 for the house and street. The shared source anchors reconstruction but is not displayed.

Video World Models

Video world models generate and predict action-conditioned visual futures to simulate changing scenes for embodied agents.

See all 256 papers

Geometry-Aware Implicit Memory for Video World Models
Jun 1, 2026Zhengxuan Wei, Xu Guo, Xinghui Li +8World Model Learning Memory-Augmented Video Generation
School of Intelligence Science and Technology, Nanjing University · 2Kling Team, Kuaishou Technology · 3Tsinghua University
MosaicMem: Hybrid Spatial Memory for Controllable Video World Models
Mar 17, 2026Wei Yu, Runjia Qian, Yumeng Li +8Memory-Augmented Video Generation Video World Models
University of Toronto · Vector Institute · The University of Osaka +4
LOCI: Spatial Linear Memory for Streaming World Models
Sep 30, 2026Ji Xia, Tingting Liao, Xuezhi Liang +2World Models Memory-Augmented Video Generation
Institute of Foundation Models, Mohamed bin Zayed University of Artificial Intelligence · Mohamed bin Zayed University of Artificial Intelligence · Pinscreen

Jun 1, 2026J/K move · Enter open · S save

Geometry-Aware Implicit Memory for Video World Models

Zhengxuan Wei, Xu Guo, Xinghui Li, Xunzhi Xiang+7

School of Intelligence Science and Technology, Nanjing University · 2Kling Team, Kuaishou Technology · 3Tsinghua University

World Model Learning Memory-Augmented Video Generation Video World Models

Open paper AskPDF arXiv

Video world models aim to simulate controllable visual environments, but long-horizon rollouts depend on what the model remembers after observations leave its native context window. Explicit memories retain frames or online 3D reconstructions, which can suffer from heuristic retrieval errors, redundant appearance storage, or reconstruction artifacts. Implicit memories compress history into a compact state, but existing designs are not explicitly constrained to encode cross-view scene geometry. We propose GIM-World, a geometry-aware implicit memory framework for video world models. A lightweight transformer encoder compresses variable-length history into fixed-size memory tokens, a camera-queryable geometry head distills 3D scene structure from a frozen foundation model into the memory during training, and an information-guided pruning rule keeps encoding cost bounded as history grows. The geometry teacher is discarded at inference, leaving a lightweight memory module. Experiments on MIND show that GIM-World better preserves long-horizon geometric and visual consistency than both explicit- and implicit-memory baselines.

Abstract

Video World Models

Explore similar work

Geometry-Aware Implicit Memory for Video World Models

MosaicMem: Hybrid Spatial Memory for Controllable Video World Models

LOCI: Spatial Linear Memory for Streaming World Models