Recovering dense 3D geometry from unposed images remains a foundational challenge in computer vision. Current state-of-the-art models are predominantly trained on perspective datasets, which implicitly constrains them to a standard pinhole camera geometry. As a result, these models suffer from significant geometric degradation when applied to wide-angle imagery captured via non-rectilinear optics, such as fisheye or panoramic sensors. To address this, we present CAM3R, a Camera-Agnostic, feed-forward Model for 3D Reconstruction capable of processing images from wide-angle camera models without prior calibration. Our framework consists of a two-view network which is bifurcated into a Ray Module (RM) to estimate per-pixel ray directions and a Cross-view Module (CVM) to infer radial distance with confidence maps, pointmaps, and relative poses. To unify these pairwise predictions into a consistent 3D scene, we introduce a Ray-Aware Global Alignment framework for pose refinement and scale optimization while strictly preserving the predicted local geometry. Extensive experiments on various camera model datasets, including panorama, fisheye and pinhole imagery, demonstrate that CAM3R establishes a new state-of-the-art in pose estimation and reconstruction.
Figures & tables
Figure 1: CAM3R provides a robust, feed-forward 3D reconstruction for two-view and multi-view scenarios across disparate optical manifolds, including pinhole, fisheye and panoramic cameras, where recent 3D foundation models fail. Above, we highlight CAM3R’s performance on unseen scenes from the 360Loc dataset, visualizing both raw two-view predictions and multi-view reconstructions.
Figure 2: CAM3R Overview. Given an input image pair (I1,I2) , the framework operates through two parallel streams. The Shared Ray Module recovers the internal camera geometry by regressing Spherical Harmonic coefficients to reconstruct continuous ray directional fields di . Simultaneously, the Cross-view Module extracts features and utilizes a dual-block transformer decoder to facilitate information exchange between the two views. Specialized DPT heads then regress radial distances ri with confidence maps σi , while a Relative Pose Network estimates the rigid transformation P2→1 . The local pointmaps Xi,i are generated by fusing rays di with radial distances ri . Finally, the second view is transformed into the reference coordinate frame of the first view via P2→1 to produce the globally aligned 3D reconstruction.
Figure 3: Qualitative Two-View Reconstructions. Visualization of 3D point clouds for image pairs across diverse optical manifolds (panorama, fisheye, pinhole). Despite extreme radial distortions and camera geometries, relative poses are accurately recovered and structural consistency is maintained. Note this is the raw output of the network.
Model
2D3DS
MegaDepth
CO3Dv2 (Zero-shot)
360Loc
ADT
RRA@15
RTA@15
RRA@15
RTA@15
RRA@15
RTA@15
RRA@15
RTA@15
RRA@15
RTA@15
DUSt3R [ 36 ]
10.6
6.0
95.6
80.8
94.7
43.1
0.0
0.0
91.0
63.6
MASt3R [ 16 ]
18.3
9.3
69.7
56.4
98.4
33.4
39.8
5.3
96.6
63.5
Pow3R [ 13 ]
7.5
6.0
96.2
74.2
95.8
38.3
0.0
0.0
96.6
79.2
VGGT [ 35 ]
11.8
11.0
98.0
88.2
90.9
29.4
37.8
11.1
92.7
82.9
π3 [ 37 ]
16.8
11.4
99.8
93.3
90.7
22.7
38.5
13.0
97.5
93.8
Table 1: Two-view relative pose estimation results. We report Accuracy at 15∘ (higher is better). Bold indicates best performance.
Figure 4: Qualitative Pruning Analysis. From left to right: (1) Successful rejection of a non-overlapping pair; (2) A valid pair with dense 3D correspondences; (3) Rejection of a doppelganger case where visually similar computer monitors yield inconsistent relative geometry.
Figure 5: Qualitative Multi-View Reconstructions. Global camera trajectories and dense point clouds recovered from unstructured image pools across diverse datasets. Despite high radial distortion and lack of scenegraph information, globally consistent poses and structural geometry are maintained through the Ray-Aware Global Alignment, effectively mitigating trajectory drift and scale ambiguity.
Model
2D3DS
MegaDepth
CO3Dv2(Zero-Shot)
360Loc
ADT
RRA
RTA
mAA
RRA
RTA
mAA
RRA
RTA
mAA
RRA
RTA
mAA
RRA
RTA
mAA
VGGT
31.8
34.4
7.6
100.0
97.4
68.8
70.5
75.3
19.6
47.9
50.8
19.5
100.0
95.6
60.3
π3
40.0
35.8
9.6
100.0
98.4
73.4
89.5
91.6
22.7
48.6
47.4
17.8
100.0
100.0
75.8
CAM3R + DUSt3R GA
72.3
55.1
38.8
86.5
72.3
68.5
69.7
56.3
46.1
66.7
59.2
55.5
78.3
71.1
70.8
CAM3R + Our GA
94.0
91.5
73.5
96.6
96.3
87.4
85.0
85.2
64.9
98.7
91.2
82.6
92.2
91.5
77.3
Table 2: Multi-view relative pose estimation results. We report Accuracy at 30∘ (RRA and RTA) and mAA@30 (higher is better). Bold indicates best performance.
Model
2D3DS
MegaDepth
CO3Dv2(Zero-shot)
360Loc
ADT
VGGT
3.8
0.7
1.3
6.3
0.5
π3
2.9
0.6
0.7
5.8
0.4
CAM3R + DUSt3R BA
2.4
1.2
1.6
4.5
0.6
CAM3R + Our BA
1.8
0.8
1.1
2.7
0.4
Table 3: Multi-view trajectory alignment error (ATE RMSE). Lower is better. Bold indicates best performance.
Figure S1 : Zero-shot generalization to unseen datasets. CAM3R successfully recovers dense 3D point clouds on Matterport [ 5 ] , FIORD [ 11 ] , and CO3Dv2 [ 25 ] despite no multi-view training on these domains. While the Ray Module leverages prior exposure to Matterport’s optical manifold via UniK3D [ 23 ] initialization, the Cross-View Module demonstrates true zero-shot structural generalization. Notably, in the FIORD dataset, CAM3R unwraps extreme 2D fisheye aberrations, strictly preserving rectilinear structures ( e.g . straight walls and sharp corners) in 3D space. To simulate unconstrained real-world environments, these scenes contain a heterogeneous mix of panoramic, fisheye, and pinhole captures. Multiple viewing angles are provided to highlight the structural integrity of the reconstructions.
Figure S2 : In-domain multi-view 3D reconstructions. Extended qualitative results from the 360Loc [ 12 ] , ADT [ 22 ] , and MegaDepth [ 17 ] test splits demonstrate CAM3R’s robustness. In 360Loc, the model reconstructs expansive concourse regions despite the prevalence of smooth, highly reflective, and textureless surfaces that typically break traditional matching heuristics. For ADT, which features enclosed room scenes with dense, short-baseline egocentric fisheye captures, our rigorous scene-graph pruning yield a coherent pointcloud. Multiple viewing angles are provided to illustrate the dense coverage and structural consistency of the fused multi-modal outputs.
Figure S3 : Raw two-view reconstructions and cross-modal correspondences. We visualize constituent image pairs from the multi-view scenes evaluated in Fig. S2 . CAM3R demonstrates robustness in recovering two-view relative geometry not only for homogeneous pairs (panorama-panorama, fisheye-fisheye, pinhole-pinhole) but crucially across highly heterogeneous projection models ( e.g . panorama-pinhole and panorama-fisheye in 360Loc). We overlay 3D-3D Mutual Nearest Neighbor (MNN) matches utilized during our graph pruning phase, color-coded by the network’s predicted distance (brighter yellow indicates stronger matches and closer points in 3D). Note: These visualizations represent the raw, feed-forward output of the two-view network prior to Ray-Aware Global Alignment.
Figure S4 : Architectural evolution toward CAM3R. Three stages explored during development: (1) direct fine-tuning of a DUSt3R-style model on distorted data, (2) decoupled radial prediction with ground-truth camera rays which is not always feasible during inference, and (3) the final CAM3R design with Ray Module and Cross-View Module.
Figure S5 : Qualitative progression of architectural design choices. From left to right: (1) Input image pairs (panoramas) with overlaid ground-truth 3D point clouds; (2) Reconstruction from vanilla DUSt3R [ 36 ] , which assumes pinhole projection; (3) Reconstruction from fine-tuned DUSt3R (Stage 1) , directly trained on distorted dataset mixture; and (4) Our final decoupled formulation, CAM3R (Stage 3) . While vanilla DUSt3R collapses under non-rectilinear distortions, naive fine-tuning fails to resolve the underlying geometric conflicts, forcing the network to generate distorted pointmaps. By explicitly decoupling internal ray geometry from relative depth, CAM3R recovers rigid, structurally accurate 3D geometry.
Model Iteration
2D3DS
MegaDepth
CO3Dv2 (Zero-shot)
360Loc
ADT
RRA@15
RTA@15
RRA@15
RTA@15
RRA@15
RTA@15
RRA@15
RTA@15
RRA@15
RTA@15
Vanilla DUSt3R [ 36 ]
10.6
6.0
95.6
80.8
94.7
43.1
0.0
0.0
91.0
63.6
Fine-tuned DUSt3R (Stage 1)
17.8
10.9
94.2
72.5
94.3
52.7
13.0
9.2
87.5
65.4
CAM3R (Final - Stage 3)
97.7
94.3
96.8
94.2
97.5
88.2
96.0
91.0
99.0
95.0
Table S1 : Quantitative two-view validation of architectural design choices. We report two-view relative pose estimation metrics at 15∘ ( ↑ ). Comparing Vanilla DUSt3R, a naively fine-tuned DUSt3R (Stage 1), and our final decoupled CAM3R (Stage 3) reveals that simply exposing standard foundation models to distorted data is mathematically insufficient. Our model yields massive performance gains, particularly on extreme optical manifolds ( e.g . 2D3DS [ 1 ] and 360Loc [ 12 ] ).
Hyperparameter
Value / Description
Batch Size (per GPU)
4
Gradient Accumulation
2 steps
Effective Batch Size
32 ( 4 GPUs ×4 batch ×2 accumulation)
Optimizer
AdamW [ 19 ] ( β1=0.9 , β2=0.95 )
Base Learning Rate ( blr )
1.5×10−4
Learning Rate Scaling
lr=blr×256effective batch size
Table S2: Optimization hyperparameters for end-to-end CAM3R training.
Stage
Configuration
Input Shape
Output Shape
Ray Module for Camera Geometry
Pixel Encoder
ViT-L/14 (24 layers), outputs {6,12,18,24}
B×3×H×W
{B×N14×1024}×4
Token Projection
Linear projection ( 1024→512 )
{B×N14×1024}×4
{B×N14×512}×4
Angular Regressor
2 Transformer blocks, 8 heads, D=512
B×18×512
B×10 (Intrinsics + SH coeffs.)
Ray Generation
FoV projection + SH basis (deg ≤3 )
Intrinsics + SH + pixel grid
B×H×W×3
Cross-View Module for Scene Geometry
Table S3 : Architecture overview of CAM3R. NP=HW/P2 denotes the token sequence length for patch size P .
Dataset
Training Split
Testing Split
2D3DS [ 1 ]
Areas 1, 2, 3, 4
Areas 5a, 5b, 6
ADT [ 22 ]
Apartment: ∼ multi-user
Apartment: multi-user, Lite
MegaDepth [ 17 ]
0000–4541 (excl. 0015, 0022)
0015, 0022, 5000–5018
360Loc [ 12 ]
Hall, Piatrium
Atrium, Concourse
Table S4 : Summary of dataset splits used for training and evaluation. We adhere to official protocols to ensure fair benchmarking across all projection domains.
Feed-forward foundation models for multi-view 3-dimensional (3D) reconstruction have been trained on large-scale datasets of perspective images; when tested on wide field-of-view images, e.g., from a fisheye camera, their performance degrades. This degradation arises from changes in spatial arrangements of pixels induced by the non-linear projection model that maps 3D points onto the 2D image plane. While one may surmise that training on fisheye images would resolve this problem, there are far fewer fisheye images with ground truth than perspective images, which limits generalization. To enable inference on imagery exhibiting high radial distortion, we propose Fisheye3R, a novel adaptation framework that extends these multi-view 3D reconstruction foundation models to natively accommodate fisheye inputs without performance regression on perspective images. To address the scarcity of fisheye images and ground truth, we introduce flexible learning schemes that support self-supervised adaptation using only unlabeled perspective images and supervised adaptation without any fisheye training data. Extensive experiments across three foundation models, including VGGT, π3, and MapAnything, demonstrate that our approach consistently improves camera pose, depth, point map, and field-of-view estimation on fisheye images. Code is available at https://github.com/android-xr/fisheye3r.
Panoramic imagery offers a full 360° field of view and is increasingly common in consumer devices. However, it introduces non-pinhole distortions that challenge joint pose estimation and 3D reconstruction. Existing feed-forward models, built for perspective cameras, generalize poorly to this setting. We propose PanoVGGT, a permutation-equivariant Transformer framework that jointly predicts camera poses, depth maps, and 3D point clouds from one or multiple panoramas in a single forward pass. The model incorporates spherical-aware positional embeddings and a panorama-specific three-axis SO(3) rotation augmentation, enabling effective geometric reasoning in the spherical domain. To resolve inherent global-frame ambiguity, we further introduce a stochastic anchoring strategy during training. In addition, we contribute PanoCity, a large-scale outdoor panoramic dataset with dense depth and 6-DoF pose annotations. Extensive experiments on PanoCity and standard benchmarks demonstrate that PanoVGGT achieves competitive accuracy, strong robustness, and improved cross-domain generalization. Code and dataset will be released.
While recent feed-forward 3D reconstruction models accelerate 3D reconstruction by jointly inferring dense geometry and camera poses in a single pass, their reliance on dense attention imposes a quadratic complexity, creating a prohibitive computational bottleneck that severely limits inference speed. To resolve this, we introduce Speed3R, an end-to-end trainable model inspired by the core principle of Structure-from-Motion: that a sparse set of keypoints is sufficient for robust pose estimation. Speed3R features a dual-branch attention mechanism where a compression branch creates a coarse contextual prior to guide a selection branch, which performs fine-grained attention only on the most informative image tokens. This strategy mimics the efficiency of traditional keypoint matching, achieving a remarkable 12.4x inference speedup on 1000-view sequences, while introducing a minimal, controlled trade-off in geometric accuracy. Validated on standard benchmarks with both VGGT and π3 backbones, our method delivers high-quality reconstructions at a fraction of computational cost, paving the way for efficient large-scale scene modeling.