Authors: Jensen, Zhou, Samuel Rota Bulo, Denis Rozumny, Duncan Zauss, Peter Hedman, Nikhil Keetha, Philip Torr, Christian Rupprecht, Andrea Vedaldi, Peter Kontschieder
Organizations: Jinghao · Meta · University of Oxford
We present the Autoregressive View Model (AVM), which generates a scene as a stream of views without constructing an explicit 3D representation. Through a unified in-context interface, AVM conditions on images or videos to generate short clips along a requested camera path. Visual context grounds reconstruction and interpolation, while a learned prior enables extrapolation into unseen regions. AVM retrieves relevant views from inputs and generated history to form a bounded active context. We fine-tune a pretrained bidirectional video diffusion transformer with flow matching in an N-in-1-out formulation. We then tune a causal model on its own rollouts to improve photorealism, reduce denoising steps, and limit long-horizon drift. We scale training to approximately 8M scenes and video clips. We evaluate generation capacity and scene persistence across benchmarks for camera control, text control, visual quality, dynamics, sparse and 4D reconstruction, and long-horizon consistency. The results demonstrate that AVM adapts to diverse input and output settings through one in-context interface.
Figures & tables
Figure 1: A scene as a stream of views conditioned on context. AVM generates short video clips along a requested camera path with observed views as context. These views come from prefill images or videos and from earlier clips, whose generated views become history that can condition later clips. The same interface handles reconstruction, exploration, and revisits without fitting any 3D representation.
Figure 2: AVM capabilities. The same model handles different input and output settings through a unified in-context interface. It uses context ranging from a single image to dense image sets and synchronized video streams, supporting camera control, event prompting, and long-horizon revisits in static and dynamic scenes. Each row shows context views (left), their camera poses and the requested trajectory (center), and target views (right).
Table 1: Comparison of related work. Context denotes images (single image , image set ) or videos (single stream , synchronized multi-camera streams ). Space denotes conditioning on action commands ( ), camera translation ( ), camera rotation ( ), or point-cloud renders ( ); parentheses group jointly used signals, and a slash separates alternative variants. Time denotes static ( ) or dynamic ( ) scenes; Text indicates text-prompt conditioning; Causal indicates an available causal variant.
Figure 3: In-context learning. AVM uses an N -in-1-out formulation, conditioning one target video clip on N context slots. Consolidated context (e.g., image #1, image #2, video clip #1) uses views from one scene without timestamps; synced context (e.g., synced clip #1) provides simultaneous views from another camera at target timestamps. The transformer processes context and target slots in one sequence.
Figure 4: Context management and retrieval. AVM conditions target views on sink , retrieved , and motion context. Input views populate prefill; generated views join history for later retrieval.
Method
Dimension
Controllability
Visual Quality
Dynamics
Metric
Camera ↑
Rot. ↓
Trans. ↓
Obj. ↑
Text ↑
Aesth. ↑
Image ↑
FID ↓
VMAF ↑
DSim ↓
Acc. ↑
Deg. ↑
Smooth. ↑
Split
D
Ws
D
Ws
D
Ws
Ws
Ws
D
Ws
D
Ws
D
D
D
Wd
Wd
Wd
View-synthesis models
GEN3C [ 62 ]
68.92
83.74
8.22
1.55
0.091
0.087
40.62
33.09
0.463
0.496
44.69
51.95
57.26
13.42
0.328
n.a.
n.a.
n.a.
Gen3R [ 29 ]
82.55
n.a.
3.25
n.a.
0.059
n.a.
n.a.
n.a.
0.540
n.a.
62.84
n.a.
26.33
14.83
0.139
n.a.
n.a.
n.a.
SEVA [ 106 ]
97.07
68.94
0.56
1.66
0.011
0.177
n.a.
n.a.
0.531
0.542
63.12
59.82
25.04
19.08
0.124
n.a.
n.a.
n.a.
Table 2: Generation capacity. We report controllability, visual quality, and dynamics scores on DL3DV-Benchmark [ 46 ] and the static and dynamic splits of WorldScore [ 20 ] . Each model receives one starting image as visual context. “n.a.” denotes unavailable results.
Figure 5: Qualitative comparison on generation capacity. We compare AVM with LingBot-World [ 63 ] , HY-WorldPlay 1.5 [ 75 ] , and DreamX-World [ 19 ] . The rows show camera control (top), text control with two prompt changes (middle), and dynamics (bottom).
Figure 6: Qualitative comparison on scene persistence. We compare AVM with SEVA [ 106 ] for sparse reconstruction with multiple images as context (top), ReCamMaster [ 3 ] for 4D reconstruction from video context (middle), and LingBot-World [ 63 ] for revisits with long-horizon history context (bottom).
Method
PSNR ↑
SSIM ↑
LPIPS ↓
VPR ↑
SP ↑
3D Cons. ↑
GEN3C [ 62 ]
14.98
0.413
0.441
0.76
0.44
66.77
Gen3R [ 29 ]
14.30
0.387
0.423
0.90
0.63
65.01
SEVA [ 106 ]
23.91
0.754
0.106
0.95
0.75
72.01
AVM
24.14
0.769
0.100
0.95
0.83
73.01
Table 3: Scene persistence against image context. Sparse reconstruction from seven posed images on DL3DV-Benchmark [ 46 ] .
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Mat. Pix. ↑
VPR ↑
SP ↑
3D Cons. ↑
ReCamMaster [ 3 ]
13.53
0.494
0.457
377.79
0.77
0.47
57.547
AVM
21.16
0.747
0.171
450.19
0.89
0.63
64.034
Table 4: Scene persistence against video context. 4D reconstruction on MultiCam [ 3 ] from an input video synchronized with the target views. Mat. Pix. counts are in thousands.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
VPR ↑
SP ↑
3D Cons. ↑
LingBot-World [ 63 ]
11.06
0.412
0.697
0.49
0.12
57.00
HY-WorldPlay 1.5 [ 75 ]
10.70
0.411
0.891
0.13
n.a.
15.76
DreamX-World [ 19 ]
12.29
0.409
0.727
0.40
0.07
47.86
AVM
28.04
0.836
0.136
0.92
0.77
53.13
Table 5: Scene persistence over history context. Revisit consistency on the Ws split of WorldScore [ 20 ] . “n.a.” denotes unavailable geometric matching scores.
CC ↑
VQ ↓
DY ↑
SR ↑
LC ↑
AVM
96.48
18.69
66.36
24.14
28.04
Model variant
AVM (base model)
97.12
19.86
66.98
24.32
28.53
AVM (causal init model)
88.90
34.86
56.41
18.97
10.12
AVM (w/o per-sample CFG dist.)
93.12
18.34
67.21
22.30
28.12
Inference strategy
Table 6: Ablation studies. Effects of model tuning, per-sample CFG distillation, retrieval, and active-window length.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Camera primitives. Reflection repeats a continuous out-and-back path. The translation & rotation path returns to the start before rotating in place. Overlapping passes and turn poses are offset for clarity. See Table 7 for the augmentation settings.
Figure 8: Sparse attention patterns. In each mask, rows index query views and columns index key views. The first nine slots show prefill context. The insets show diffusion forcing with prefix context in the boxed regions.
Figure 9: Standard distillation vs. context-augmented distillation. Standard distillation (left) conditions both score models on one overlap frame. Context-augmented distillation (right) uses sink , retrieved , and motion context. Both models score the same target block, and the difference between their denoised estimates drives the generator update.
Path or family
Prob. (%)
Motion variants
Reflected
80.0
Reflect the original ground-truth trajectory to the requested length.
Synthetic
20.0
Synthesize a trajectory of the requested length using the primitives below.
Static
1.7
Fixed camera.
Translation
56.5
Dolly in & back, truck left & right, pedestal up.
Rotation
13.0
Pan left & right, tilt up & down, roll in either direction.
Zoom
2.6
Zoom in & out.
Appendix
Table 7: Camera trajectory augmentation during on-policy tuning. Indented probabilities are conditional on selecting a synthetic path and are rounded. Variants are sampled uniformly within each family.
Figure 10: Comparison of camera-based retrieval policies. We show the fourth chunk from smerf/alameda [ 21 ] (top) and mipnerf360/room [ 4 ] (bottom). All panels in a row share an orthographic view and scale. Lrtr is the retrieved context length (see Section 4 ).
Retrieval policy
Mean Lrtr↑
Overlap (%) ↑
Empty (%) ↓
Greedy nearest
5.3
73.0
0.0
Spatial
3.5
65.1
22.0
Spatial + angular
6.5
84.4
0.0
Appendix
Table 8: Coverage and context length by retrieval policy. We average over chunks within each scene, then weight scenes equally. Lrtr counts retrieved slots per chunk. Empty is the percentage of chunks with no retrieved views. Slot counts and overlap exclude motion context. Bold marks the default policy.
Ws split (26 scenes) for the translation & rotation and loop closure trajectories
Table 9: Selected evaluation scenes. WorldScore and MultiCam scene identifiers for the settings in Section C .
Figure 11: Camera control. AVM uses image context to generate target views along requested camera paths in photographic and stylized scenes.
Figure 12: Image context. One or more posed images provide context for target views at the requested camera poses.
Figure 13: Video stream context. A single input video provides context for target views at requested camera poses and timestamps. Each mosaic shows sampled input frames.
Figure 14: Video rig context. Synchronized streams from four camera rigs provide synced context at the target timestamps.
Figure 15: Dense view prefilling. A dense set of posed images provides prefill context for target views along the requested trajectory. The mosaic shows a subset of the inputs.
Figure 16: Dynamics. Target views show scene motion as timestamps advance and the camera follows the requested trajectory.
Figure 17: Event prompting. Instruction labels specify the requested content changes as AVM generates target views along the camera path.
Figure 18: Long-horizon consistency. Target views at revisited locations illustrate scene persistence after long camera trajectories in static and dynamic scenes.
Figure 19: Perpetual view streaming. AVM generates successive blocks along an extended camera path. New target views extend the stream and become available as history context.
We address the task of long-video, camera-controlled generation from a single image. The dominant autoregressive paradigm suffers from two key limitations: (1) error accumulation over time, as distant frames are conditioned on previously generated outputs rather than ground truth, leading to drift; and (2) ambiguity in how to incorporate camera control. Standard solutions to both--such as self-forcing and conditioning on explicit 6-DOF poses--either incur significant computational overhead or produce drifting scene geometries over time. We propose a simple and lightweight framework that combines autoregressive generation (WanAR) with hierarchical long-context denoising (FixAR). To enforce scene geometry remains consistent with input cameras, we draw from feed-forward novel-view synthesis and reformulate video generation as a warping-and-inpainting problem. To mitigate drift, we introduce a hierarchical denoising strategy in which different stages condition on different temporal histories. Specifically, we first generate frames conditioned on the immediate past, and then refine them using a second denoiser conditioned on a temporally subsampled long-range history to enforce global consistency. This fine-to-coarse scheme enables each generated frame to effectively condition on the full temporal context without incurring a prohibitive cost. Experiments on RealEstate10k and DL3DV demonstrate that WanAR+FixAR outperforms prior methods for camera-controlled long-video generation, producing results that are both temporally stable and consistent with input camera trajectories.
Kaihua Chen, Yiming Gong, Deva Ramanan +1
Carnegie Mellon University, Pittsburgh, PA, USA · University of Oxford, Oxford, UK
Scene-consistent video generation aims to create videos that explore 3D scenes based on a camera trajectory. Previous methods rely on video generation models with external memory for consistency, or iterative 3D reconstruction and inpainting, which accumulate errors during inference due to incorrect intermediary outputs, non-differentiable processes, and separate models. To overcome these limitations, we introduce ``geometry-as-context". It iteratively completes the following steps using an autoregressive camera-controlled video generation model: (1) estimates the geometry of the current view necessary for 3D reconstruction, and (2) simulates and restores novel view images rendered by the 3D scene. Under this multi-task framework, we develop the camera gated attention module to enhance the model's capability to effectively leverage camera poses. During the training phase, text contexts are utilized to ascertain whether geometric or RGB images should be generated. To ensure that the model can generate RGB-only outputs during inference, the geometry context is randomly dropped from the interleaved text-image-geometry training sequence. The method has been tested on scene video generation with one-direction and forth-and-back trajectories. The results show its superiority over previous approaches in maintaining scene consistency and camera control.
JiaKui Hu, Jialun Liu, Liying Yang +7
Institute of Medical Technology, Peking University · TeleAI · MUST
Autoregressive video world models enable interactive, long-horizon exploration, but flexible control remains challenging. Exploring a source video from new viewpoints requires the generated rollout to remain synchronised with the recorded event, place observed content in the requested view, plausibly complete newly exposed regions, and recover previously generated appearance on revisits. Existing methods typically address these requirements through task-specific modules or additional training. We present World in World, a training-free inference-time interface that converts heterogeneous control evidence into camera- and time-labelled clean visual states, which are read through the native self attention of a frozen causal video model. The evidence comprises source-video observations, target-view scene projections, geometry renderings that guide completion of newly exposed subject regions, and retrieved generated states beyond the rolling cache. Each evidence source carries token-level support and its own availability schedule. A correspondence router combines persistent point identities with geometry to establish token correspondences, guiding supported queries towards matching source-video tokens. Evidence-wise attention CFG (EWA) then independently regulates each auxiliary channel's additional contribution using attention responses from the same denoising forward pass. The shared interface supports camera-controlled rerendering, long-horizon revisiting, and human-motion transfer with the same frozen backbone. We evaluate World in World on camera-controlled video rerendering under diverse viewpoint changes, assessing perceptual quality, temporal consistency, and camera-following accuracy.