Authors: Jensen, Zhou, Samuel Rota Bulo, Denis Rozumny, Duncan Zauss, Peter Hedman, Nikhil Keetha, Philip Torr, Christian Rupprecht, Andrea Vedaldi, Peter Kontschieder
Organizations: Jinghao · Meta · University of Oxford
We present the Autoregressive View Model (AVM), which generates a scene as a stream of views without constructing an explicit 3D representation. Through a unified in-context interface, AVM conditions on images or videos to generate short clips along a requested camera path. Visual context grounds reconstruction and interpolation, while a learned prior enables extrapolation into unseen regions. AVM retrieves relevant views from inputs and generated history to form a bounded active context. We fine-tune a pretrained bidirectional video diffusion transformer with flow matching in an N-in-1-out formulation. We then tune a causal model on its own rollouts to improve photorealism, reduce denoising steps, and limit long-horizon drift. We scale training to approximately 8M scenes and video clips. We evaluate generation capacity and scene persistence across benchmarks for camera control, text control, visual quality, dynamics, sparse and 4D reconstruction, and long-horizon consistency. The results demonstrate that AVM adapts to diverse input and output settings through one in-context interface.
Figures & tables
Figure 1: A scene as a stream of views conditioned on context. AVM generates short video clips along a requested camera path with observed views as context. These views come from prefill images or videos and from earlier clips, whose generated views become history that can condition later clips. The same interface handles reconstruction, exploration, and revisits without fitting any 3D representation.
Figure 2: AVM capabilities. The same model handles different input and output settings through a unified in-context interface. It uses context ranging from a single image to dense image sets and synchronized video streams, supporting camera control, event prompting, and long-horizon revisits in static and dynamic scenes. Each row shows context views (left), their camera poses and the requested trajectory (center), and target views (right).
Table 1: Comparison of related work. Context denotes images (single image , image set ) or videos (single stream , synchronized multi-camera streams ). Space denotes conditioning on action commands ( ), camera translation ( ), camera rotation ( ), or point-cloud renders ( ); parentheses group jointly used signals, and a slash separates alternative variants. Time denotes static ( ) or dynamic ( ) scenes; Text indicates text-prompt conditioning; Causal indicates an available causal variant.
Figure 3: In-context learning. AVM uses an N -in-1-out formulation, conditioning one target video clip on N context slots. Consolidated context (e.g., image #1, image #2, video clip #1) uses views from one scene without timestamps; synced context (e.g., synced clip #1) provides simultaneous views from another camera at target timestamps. The transformer processes context and target slots in one sequence.
Figure 4: Context management and retrieval. AVM conditions target views on sink , retrieved , and motion context. Input views populate prefill; generated views join history for later retrieval.
Method
Dimension
Controllability
Visual Quality
Dynamics
Metric
Camera ↑
Rot. ↓
Trans. ↓
Obj. ↑
Text ↑
Aesth. ↑
Image ↑
FID ↓
VMAF ↑
DSim ↓
Acc. ↑
Deg. ↑
Smooth. ↑
Split
D
Ws
D
Ws
D
Ws
Ws
Ws
D
Ws
D
Ws
D
D
D
Wd
Wd
Wd
View-synthesis models
GEN3C [ 62 ]
68.92
83.74
8.22
1.55
0.091
0.087
40.62
33.09
0.463
0.496
44.69
51.95
57.26
13.42
0.328
n.a.
n.a.
n.a.
Gen3R [ 29 ]
82.55
n.a.
3.25
n.a.
0.059
n.a.
n.a.
n.a.
0.540
n.a.
62.84
n.a.
26.33
14.83
0.139
n.a.
n.a.
n.a.
SEVA [ 106 ]
97.07
68.94
0.56
1.66
0.011
0.177
n.a.
n.a.
0.531
0.542
63.12
59.82
25.04
19.08
0.124
n.a.
n.a.
n.a.
Table 2: Generation capacity. We report controllability, visual quality, and dynamics scores on DL3DV-Benchmark [ 46 ] and the static and dynamic splits of WorldScore [ 20 ] . Each model receives one starting image as visual context. “n.a.” denotes unavailable results.
Figure 5: Qualitative comparison on generation capacity. We compare AVM with LingBot-World [ 63 ] , HY-WorldPlay 1.5 [ 75 ] , and DreamX-World [ 19 ] . The rows show camera control (top), text control with two prompt changes (middle), and dynamics (bottom).
Figure 6: Qualitative comparison on scene persistence. We compare AVM with SEVA [ 106 ] for sparse reconstruction with multiple images as context (top), ReCamMaster [ 3 ] for 4D reconstruction from video context (middle), and LingBot-World [ 63 ] for revisits with long-horizon history context (bottom).
Method
PSNR ↑
SSIM ↑
LPIPS ↓
VPR ↑
SP ↑
3D Cons. ↑
GEN3C [ 62 ]
14.98
0.413
0.441
0.76
0.44
66.77
Gen3R [ 29 ]
14.30
0.387
0.423
0.90
0.63
65.01
SEVA [ 106 ]
23.91
0.754
0.106
0.95
0.75
72.01
AVM
24.14
0.769
0.100
0.95
0.83
73.01
Table 3: Scene persistence against image context. Sparse reconstruction from seven posed images on DL3DV-Benchmark [ 46 ] .
Method
PSNR ↑
SSIM ↑
LPIPS ↓
Mat. Pix. ↑
VPR ↑
SP ↑
3D Cons. ↑
ReCamMaster [ 3 ]
13.53
0.494
0.457
377.79
0.77
0.47
57.547
AVM
21.16
0.747
0.171
450.19
0.89
0.63
64.034
Table 4: Scene persistence against video context. 4D reconstruction on MultiCam [ 3 ] from an input video synchronized with the target views. Mat. Pix. counts are in thousands.
Method
PSNR ↑
SSIM ↑
LPIPS ↓
VPR ↑
SP ↑
3D Cons. ↑
LingBot-World [ 63 ]
11.06
0.412
0.697
0.49
0.12
57.00
HY-WorldPlay 1.5 [ 75 ]
10.70
0.411
0.891
0.13
n.a.
15.76
DreamX-World [ 19 ]
12.29
0.409
0.727
0.40
0.07
47.86
AVM
28.04
0.836
0.136
0.92
0.77
53.13
Table 5: Scene persistence over history context. Revisit consistency on the Ws split of WorldScore [ 20 ] . “n.a.” denotes unavailable geometric matching scores.
CC ↑
VQ ↓
DY ↑
SR ↑
LC ↑
AVM
96.48
18.69
66.36
24.14
28.04
Model variant
AVM (base model)
97.12
19.86
66.98
24.32
28.53
AVM (causal init model)
88.90
34.86
56.41
18.97
10.12
AVM (w/o per-sample CFG dist.)
93.12
18.34
67.21
22.30
28.12
Inference strategy
Table 6: Ablation studies. Effects of model tuning, per-sample CFG distillation, retrieval, and active-window length.
Appendix figures & tables16 assets
Supplementary material from the paper’s appendix.
Appendix
Figure 7: Camera primitives. Reflection repeats a continuous out-and-back path. The translation & rotation path returns to the start before rotating in place. Overlapping passes and turn poses are offset for clarity. See Table 7 for the augmentation settings.
Figure 8: Sparse attention patterns. In each mask, rows index query views and columns index key views. The first nine slots show prefill context. The insets show diffusion forcing with prefix context in the boxed regions.
Figure 9: Standard distillation vs. context-augmented distillation. Standard distillation (left) conditions both score models on one overlap frame. Context-augmented distillation (right) uses sink , retrieved , and motion context. Both models score the same target block, and the difference between their denoised estimates drives the generator update.
Path or family
Prob. (%)
Motion variants
Reflected
80.0
Reflect the original ground-truth trajectory to the requested length.
Synthetic
20.0
Synthesize a trajectory of the requested length using the primitives below.
Static
1.7
Fixed camera.
Translation
56.5
Dolly in & back, truck left & right, pedestal up.
Rotation
13.0
Pan left & right, tilt up & down, roll in either direction.
Zoom
2.6
Zoom in & out.
Appendix
Table 7: Camera trajectory augmentation during on-policy tuning. Indented probabilities are conditional on selecting a synthetic path and are rounded. Variants are sampled uniformly within each family.
Figure 10: Comparison of camera-based retrieval policies. We show the fourth chunk from smerf/alameda [ 21 ] (top) and mipnerf360/room [ 4 ] (bottom). All panels in a row share an orthographic view and scale. Lrtr is the retrieved context length (see Section 4 ).
Retrieval policy
Mean Lrtr↑
Overlap (%) ↑
Empty (%) ↓
Greedy nearest
5.3
73.0
0.0
Spatial
3.5
65.1
22.0
Spatial + angular
6.5
84.4
0.0
Appendix
Table 8: Coverage and context length by retrieval policy. We average over chunks within each scene, then weight scenes equally. Lrtr counts retrieved slots per chunk. Empty is the percentage of chunks with no retrieved views. Slot counts and overlap exclude motion context. Bold marks the default policy.
Ws split (26 scenes) for the translation & rotation and loop closure trajectories
Table 9: Selected evaluation scenes. WorldScore and MultiCam scene identifiers for the settings in Section C .
Figure 11: Camera control. AVM uses image context to generate target views along requested camera paths in photographic and stylized scenes.
Figure 12: Image context. One or more posed images provide context for target views at the requested camera poses.
Figure 13: Video stream context. A single input video provides context for target views at requested camera poses and timestamps. Each mosaic shows sampled input frames.
Figure 14: Video rig context. Synchronized streams from four camera rigs provide synced context at the target timestamps.
Figure 15: Dense view prefilling. A dense set of posed images provides prefill context for target views along the requested trajectory. The mosaic shows a subset of the inputs.
Figure 16: Dynamics. Target views show scene motion as timestamps advance and the camera follows the requested trajectory.
Figure 17: Event prompting. Instruction labels specify the requested content changes as AVM generates target views along the camera path.
Figure 18: Long-horizon consistency. Target views at revisited locations illustrate scene persistence after long camera trajectories in static and dynamic scenes.
Figure 19: Perpetual view streaming. AVM generates successive blocks along an extended camera path. New target views extend the stream and become available as history context.