Solv3R: 3D Looped Transformer as a Multi-view Solver
Authors: Jeong-gi Kwak, Andrea Tagliasacchi, Sho Kagami, Yuki Ono, Kwang Moo Yi
Organizations: University of British Columbia · Wayve Labs · Simon Fraser University · University of Toronto · Sony Semiconductor Solutions Corporation · Sony Corporation
Classical structure-from-motion reconstruction proceeds iteratively, refining camera poses and scene geometry from a current estimate. This allows maps to easily expand without recomputing already computed views, or geometry estimates to improve as more observations are provided. Most geometric foundation models, however, predict entire geometry in a single forward pass or update a recurrent state as frames arrive sequentially, making this expansion and refinement non-trivial. We introduce Solv3R, a looped transformer that learns to continue solving multi-view geometry from its current state. Importantly, its update operator is independent of the iteration count, allowing previously refined and newly introduced views to be jointly updated regardless of how ``solved'' they are. We train the operator to preserve accurate reconstructions under continued iteration and correct inaccurate states, enabling refinement of fixed image collections and integration of new observations. Representations of solved views can also be cached once and reused as geometric context across iterations, enabling compact and efficient key-value caching. We analyze the solver dynamics and compare it with the state-of-the-art, demonstrating that a single learned solver supports offline reconstruction, incremental reconstruction, relocalization, and streaming without task-specific training or modules. The code and models will be made publicly available at https://ubc-vision.github.io/solv3r/.
Figures & tables
Figure 2: Comparison of geometric update mechanisms. Conventional GFMs [ 88 , 51 , 95 ] use successive layer-specific transformations, tying their intermediate representations to layer depth. Recurrent-memory models [ 90 , 21 ] process incoming frames while updating a persistent scene memory across observations. Although DéjàView [ 12 ] shares transformer weights, its updates remain conditioned on the refinement step. Solv3R applies the same stationary update operator regardless of the iteration count, allowing previously refined and newly introduced views to be jointly updated.
Figure 3: Continuing geometric solving with new observations. When new views are introduced, Solv3R continues solving from its existing state, jointly refining both previously reconstructed and newly added views. DéjàView struggles to maintain consistent reconstruction. Numbers indicate camera rotation errors in degrees.
Figure 4: Overview. Left: A stationary transformer block repeatedly updates the multi-view state through frame-wise and global attention, initial-state injection, and state normalization. Right: The same operator continues refining the reconstruction across solver calls, allowing new views to be introduced at any stage and jointly updated with existing views. Camera poses, depth maps, and ray maps can be decoded from the current state after any solver call.
Figure 5: Stable and corrective solver dynamics. Left : pose accuracy and relative state update norm over solver calls. DéjàView is trained with up to 16 steps (shaded). Middle : camera rotation error and relative state update aligned to the solver call at which each window meets the stopping criterion of 5 , showing median and interquartile range. Right : rotation error at call 16 after perturbations at call 8 using relative (rel.) or fixed-scale (fixed) Gaussian noise, or state resets (reset).
Figure 6: Warm-start convergence. Left : depth AbsRel along a KITTI [ 33 ] sequence, warm-starting each new frame from the previous solution with four solver calls. Right : percentage of new frames with rotation error below 2∘ versus solver calls in a 24-view window, comparing warm-starting with restarting from scratch. Solv3R remains stable under continued warm-starting and achieves accurate camera estimates with fewer calls, whereas DéjàView diverges.
Figure 7: Mixed-progress solving. Left : pose accuracy when each view is initialized from a randomly sampled reconstruction stage. Right : rotation error as each view progresses through a pipelined reconstruction window, where views enter and leave after each solver call. Solv3R maintains accurate reconstruction across views with varying reconstruction stages, whereas DéjàView is sensitive to their computational progress.
Efficiency
Camera Pose
Geometry
DTU
ETH3D
7-Scenes
ScanNet++
nuScenes
DTU
ETH3D
7-Scenes
ScanNet++
nuScenes
Method
Params
TFLOPs
@3
@30
@3
@30
@3
@30
@3
@30
@3
@30
L2
IR
L2
IR
L2
IR
L2
IR
L2
IR
MASt3R [ 47 ] (ECCV’24)
689M (5.9 × )
500.0
21.6
81.7
35.2
57.6
9.3
70.6
15.5
43.9
9.4
62.7
.011
94.9
.340
29.1
.076
41.7
.251
14.5
.360
11.4
MASt3R-SfM [ 27 ] (3DV’25)
690M (5.9 × )
1150.1
40.2
91.4
52.8
85.0
16.4
80.3
38.7
85.0
9.1
70.9
.009
96.9
.095
54.1
.051
64.7
.042
69.7
.311
18.4
VGGT [ 88 ] (CVPR’25)
1.26B (10.8 × )
190.0
96.5
99.8
35.4
82.8
11.1
79.1
15.1
74.7
39.2
83.8
.010
95.8
.053
52.6
.042
70.8
.034
68.4
.081
42.3
VGGT- Ω [ 89 ] (CVPR’26)
1.14B (9.7 × )
99.8
77.4
98.1
64.1
95.4
21.3
87.0
29.9
87.3
42.4
85.5
.009
97.2
.024
78.6
.039
65.3
.032
70.6
.055
62.3
Table 1: Standard (offline) multi-view reconstruction benchmarks. Quantitative comparison with state-of-the-art methods [ 47 , 27 , 88 , 89 , 95 , 42 , 51 , 12 ] across five benchmark datasets [ 40 , 70 , 71 , 107 , 16 ] . We report camera pose AUC@3/@30 ( ↑ ), geometry relative L2 ( ↓ ), and inlier ratio (IR, ↑ ). TFLOPs are measured for 24 views. Solv3R (fixed) uses 16 solver calls, while Solv3R (adaptive) stops based on the relative state update criterion in Eq. 5 with γ=0.02 .
Figure 8: Revising an existing reconstruction with new views. (a) Four views are solved for eight calls before twelve new views are introduced; camera colors indicate rotation error. (b,c) Median rotation error of the original and new views during continued refinement over 96 windows. Dashed curves in (b) indicate the no-join controls, where the original four views continue refinement for the same number of solver calls without introducing new views. Solv3R incorporates new views while improving existing camera estimates, whereas DéjàView fails to maintain a consistent reconstruction.
Figure 9: Revisability and reuse. (a) Original-view rotation error when new views join at different refinement stages, compared with the no-join control; percentages indicate the fraction of improved windows. (b) Final pose accuracy versus total reconstruction time as the view set expands from 8 to 64. Solv3R improves existing estimates across different join times and achieves comparable accuracy to re-solving with less computation.
Figure 10: Relocalization against a frozen map. Green dots are query estimates within 5 cm and 5 ∘ of the ground truth (gray rings), and red crosses are the others, linked to their ground truth. Above each panel are the cache size per map view and the fraction of queries within 5 cm and 5 ∘ . With the smallest cache, Solv3R localizes as many queries as the most accurate baseline.
Cache (MB/view) ↓
Time (ms) ↓
7-Scenes
Cambridge
Method
Med. ↓
5cm/5 ∘↑
Med. ↓
25cm/2 ∘↑
VGGT
97.6
186
6.35 / 1.35
45.5
183.2 / 1.77
16.4
VGGT- Ω
78.5
158
8.87 / 1.68
32.8
187.7 / 1.18
14.9
π3
89.7
161
4.11 / 0.99
66.9
85.9 / 0.95
14.7
DA3-L
30.4
62
6.90 / 1.16
36.7
147.9 / 1.86
12.7
DéjàView (all step)
45.8
88
11.30 / 1.31
28.3
131.0 / 1.07
0.7
Table 2: Map-based relocalization. Evaluation on indoor 7-Scenes [ 71 ] and outdoor Cambridge Landmarks [ 43 ] . Median translation/rotation errors (cm/ ∘ ), localization recall (%), cache size per view (MB), and query latency (ms).
Figure 11: Streaming reconstruction. Qualitative comparison with window-based reconstruction [ 26 ] , recurrent-state methods [ 90 , 21 ] , and geometric-context approaches [ 18 ] on long sequences, with camera trajectories colored by position error.
Method
Efficiency
Camera Pose
Video Depth
3D Reconstruction
Time
Mem.
Sintel
TUM-dyn.
ScanNet
Sintel
Bonn
KITTI
7-Scenes
NRGBD
(ms) ↓
(GB) ↓
ATE ↓
ATE ↓
ATE ↓
AbsRel ↓
δ↑
AbsRel ↓
δ↑
AbsRel ↓
δ↑
Cham. ↓
NC ↑
Cham. ↓
NC ↑
w/ Separate back-end
DROID-SLAM † [ 78 ] (NeurIPS’21)
36
5.7
.164
.065
.137
–
–
–
–
–
–
–
–
–
–
MASt3R-SLAM [ 62 ] (CVPR’25)
73
7.2
.273
.102
.139
–
–
–
–
–
–
–
–
–
–
SLAM-Former [ 109 ] (ECCV’26)
72
10.4
.147
.040
.096
–
–
–
–
–
–
–
–
–
–
Table 3: Streaming reconstruction benchmarks. Comparison with state-of-the-art streaming reconstruction methods across camera pose, video depth, and 3D reconstruction. We report per-frame runtime (ms) and peak GPU memory (GB), together with camera trajectory ATE (m), depth AbsRel and δ1.25 , and reconstruction Chamfer distance (m) and normal consistency (NC).
Figure 12: Streaming accuracy vs. efficiency. Comparison of online reconstruction methods in terms of geometric-mean pose ATE and per-frame runtime; bubble size indicates peak GPU memory. For Solv3R, S/L denotes small/large memory configurations, and numbers indicate solver calls per frame (L8 is the default).
AUC@5 ↑
Stability ↓
Join ↓
Cache ↓
Method
16
128
1024
( ∘ )
( ∘ )
( ∘ )
DéjàView (released)
76.5
0.0
0.0
8.37
1.80
55.5
Fine-tuned for 3k steps
DéjàView w/ our training
73.5
19.3
0.0
3.94
1.31
1.05
Solv3R
72.5
70.8
65.4
1.26
0.62
0.76
w/ step-conditioned ( P=16 )
71.5
52.1
23.5
1.96
0.81
0.95
Table 4: Ablation study. All ablated variants are fine-tuned for 3k steps from the same pretrained initialization. The released DéjàView and the final Solv3R models are included as references. We report pose accuracy at different solver calls, along with rotation errors after perturbation (Stability), new-view insertion (Join), and inference with frozen KV context (Cache). For step-conditioned gates, P is the number of calls their schedule is planned for.
Appendix figures & tables19 assets
Supplementary material from the paper’s appendix.
Appendix
Type
Dataset
Weight (%)
Indoor
ScanNet++ [ 107 ]
10.8
ScanNet [ 23 ]
7.4
CA-1M [ 46 ]
3.3
Taskonomy [ 110 ]
3.3
Object
CO3D [ 67 ]
7.4
NAVI [ 37 ]
3.3
Appendix
Table 5: Training data. Sampling weights (%) in the second training stage. ∗ Supervised only by the distillation loss.
Figure 13: Stopping calls. Left: number of windows per stopping call, for the 144 of the 148 windows of the stability analysis that reach γ=0.02 , stacked by dataset. Right: rotation error of each window at its stopping call.
DTU [ 40 ]
ETH3D [ 70 ]
7-Scenes [ 71 ]
ScanNet++ [ 107 ]
nuScenes [ 16 ]
γ
AUC
Calls
AUC
Calls
AUC
Calls
AUC
Calls
AUC
Calls
0.01
85.9
36.1
69.8
54.9
14.2
35.8
81.5
39.8
41.8
31.1
0.02
85.5
19.7
70.4
20.5
14.2
22.6
81.2
17.3
41.5
18.2
0.05
81.5
13.0
62.0
13.5
13.9
12.9
77.0
11.8
40.9
11.9
Appendix
Table 6: Sensitivity to the stopping threshold γ . Pose AUC@3 and the mean number of solver calls per scene.
Error ( ∘ )
Stops
Start diff. ( ∘ )
DéjàView
Windows
#
Stopped
Calls
Stop
128
>5∘
Noise
Swap
at 128 ( ∘ )
Stability analysis
148
97%
17.5
0.44
0.42
1
0.09
0.11
96.0
Tanks and Temples
39
100%
17.7
0.29
0.27
0
0.07
0.09
86.6
7-Scenes
36
94%
16.2
1.56
1.57
0
0.06
0.06
82.9
TUM dynamics
32
100%
15.7
0.42
0.39
0
0.04
0.05
75.1
Appendix
Table 7: Stopping on other datasets. Eight-view windows solved for 128 calls. Stopped is the share of windows that reach γ=0.02 , and Calls is their mean stopping call. Error is the median rotation error of the stopped windows at the stopping call and at call 128. Stops >5∘ counts the stopped windows with an error above 5∘ at the stopping call. Start diff. is the median rotation difference after 128 calls between the start from z0 and the noisy or swapped start. The last column is the median rotation error of DéjàView at call 128.
Figure 14: Initialization of new views. Median rotation error of a newly arrived frame over solver calls, for 656 frames per dataset, 41 from each of 16 sequences. The new frame starts from its initial state z0 , from z0 plus the solved offset of the previous frame (Offset), or from the solved state of the previous frame (Previous state). The frames that remain in the window keep their solved states. “From scratch” solves the whole window from the initial states.
Figure 15: Initialization in streaming. ATE of on-arrival predictions with the L8 window for different numbers of calls per frame. Each new frame starts from its initial state z0 (cold start) or from z0 plus the solved offset of the previous frame (warm start).
ATE (m) ↓
Video depth AbsRel ↓
Chamfer (m) ↓
Time ↓
GPU mem. ↓
Config
a
r
M
m
Sintel
TUM
ScanNet
Sintel
Bonn
KITTI
7-Scenes
NRGBD
(ms)
(GB)
L8
8
6
64
8
.157
.015
.077
.322
.064
.078
.018
.018
95
5.0
L4
8
6
64
4
.163
.017
.091
.321
.064
.083
.037
.024
65
5.0
S4
4
4
16
4
.179
.017
.180
.328
.065
.088
.072
.045
51
2.1
L8, refined
8
6
64
8
.162
.014
.076
.302
.065
.072
.017
.017
95
5.0
Appendix
Table 8: Streaming configurations. a , r , M , and m are the numbers of anchors, recent frames, memory frames, and calls per frame. Every configuration takes a memory frame every 16 frames. Runtime is the mean time per frame on an H100 GPU, including the image encoder and the prediction heads. As in the main paper, we measure it on the TUM dynamics sequences sitting_halfsphere, sitting_static, and walking_rpy. The last row reads each frame of the L8 run when it leaves the recent window instead of at its arrival.
Window
Method
Sintel (46)
TUM (878)
ScanNet (602)
No reuse
DéjàView
.124
.218
.560
Solv3R
.118
.214
.574
L8
DéjàView
.591
div.
div.
Solv3R
.157
.015
.077
no refresh
.444
.160
1.05
no recent window
.265
.014
.324
Appendix
Table 9: Reuse in streaming. ATE (m) of on-arrival predictions. Without reuse, the solver solves each new frame with the seven previous frames from their initial states for 16 calls. We chain its pose through the first frame of the window. In the L8 window, DéjàView runs its step schedule for each frame and then repeats its final step. Its poses diverge (div.). Without the refresh, the frozen frames never rejoin the update. No recent window drops the six recent frames, so that each call updates only the new frame.
Figure 16: Memory on a long sequence. GPU memory over the 2000 frames of the long office sequence of TUM RGB-D, on a logarithmic axis. All methods are evaluated on a single H100 GPU. Crosses mark where a method runs out of the 80 GB of GPU memory (gray dotted line).
Figure 17: Attention across solver calls. We show the three global-attention heads, out of 12, that best find ground-truth correspondences at call 16. These are the same heads in Solv3R (left) and DéjàView (right). Top: fraction of query patches whose strongest attention in an adjacent view falls within one patch of the ground-truth correspondence. Bottom: similarity to the call-16 attention map, measured as the share of the call-16 attention mass on the 64 most attended keys of each call and normalized to 1 at call 16. The shaded range is the 16-step schedule of DéjàView. Averaged over 82 windows of ETH3D and ScanNet++.
DL3DV [ 52 ]
ScanNet++ [ 107 ]
Order
Continue
Re-solve
Continue
Re-solve
Capture
92.0
91.2
92.4
92.5
Reverse
89.9
91.6
92.4
92.4
Random 1
91.7
91.7
92.1
91.8
Random 2
92.0
92.1
92.3
92.3
Appendix
Table 10: Sensitivity to the insertion order. AUC@10 after inserting 64 views, 8 at a time. Continue runs 8 calls per insertion from the current state, and re-solve runs 16 calls from scratch.
Figure 18: Failure case. In this window, views 5–8 share little content with views 1–4. Solv3R places them at a wrong position and orientation within 16 calls and keeps them there up to call 64.
Rubble
Building
Method
@3
@30
@3
@30
VGGT [ 88 ]
5.3
82.4
2.5
70.6
VGGT- Ω [ 89 ]
34.6
91.8
14.6
86.4
π3 [ 95 ]
14.9
88.3
1.8
75.9
DA3-L [ 51 ]
3.8
74.0
1.1
55.8
DéjàView [ 12 ]
39.4
93.0
11.0
87.0
Appendix
Table 11: Large aerial scenes (Mill19). Camera pose AUC@3 and AUC@30 from 96 images per scene.
Figure 19: Reconstructions that grow as views arrive. Each row shows one scene at three stages of one arrival pattern. Offline solves all views jointly, incremental adds the capture in thirds, streaming adds video frames one at a time, and arbitrary order adds random groups of views. Views that join continue the running solve, and the views already in it keep their states. Above each panel are the number of views in the solve and the number K of solver calls run so far. Arrows give the views that joined, and orange marks their cameras.
Figure 20: One scene under the four arrival patterns. Shown as in Fig. 19 . All four patterns end in nearly the same reconstruction, although the views arrive in different orders and receive different numbers of calls.
Figure 21: Offline reconstruction comparison. All methods see the same 100 images at once. VGGT, DA3-L, and VGGT- Ω misplace the second room, while Solv3R reconstructs both rooms in one piece.
Figure 22: Offline reconstruction comparison. All methods see the same 44 images at once. π3 and DA3-L misplace parts of the scene, while Solv3R keeps the walls and floor in place.
Figure 23: Offline reconstruction comparison. All methods see the same 100 images at once, and the ceiling is removed for display. Insets show the predicted cameras in color and the reference cameras in gray, seen from above. Solv3R places the cameras closest to the reference, while VGGT and VGGT- Ω fold the room.
Figure 24: Streaming reconstruction comparison. The methods and settings are those of the streaming figure of the main paper. Camera trajectories are colored by position error.